The QDK provides an excellent, low barrier way of getting started with Q# development - without having to deal with the compiler directly, or worrying about how to simulate the code you wrote on a classical device. Additionally, for more technically versed users, the Q# compiler is also available as a command line utility that can be used to fine tune the compilation experience and cater to complex scenarios. The QDK is well documented, and the command line compiler provides good documentation as part of the application itself, but one of the things that is not widely known is that the Q# compiler can also be easily used programmatically - via its Nuget package.

Let’s have a look.

Sample Q# code 🔗

Let’s imagine the following - fairly basic - piece of Q# code that we’ll use as our testbed.

namespace HelloQuantum {

    open Microsoft.Quantum.Canon;
    open Microsoft.Quantum.Measurement;
    open Microsoft.Quantum.Intrinsic;
    open Microsoft.Quantum.Convert;

    @EntryPoint()
    operation HelloQ() : Unit {
        let ones = GetRandomBit(100);
        Message(""Ones: "" + IntAsString(ones));
        Message(""Zeros: "" + IntAsString((100 - ones)));
    }
    
    operation GetRandomBit(count : Int) : Int {
 
        mutable resultsTotal = 0;
 
        using (qubit = Qubit()) {       
            for (idx in 0..count) {               
                H(qubit);
                let result = MResetZ(qubit);
                set resultsTotal += result == One ? 1 | 0;
            }
            return resultsTotal;
        }
    }
}

The code above contains a Q# entry point operation that will allow for a creation of a standalone Q# application. The entry operation invokes another operation which internally will borrow a single qubit, apply the Hadamard transformation on it and measure the qubit in the Pauli Z axis, repeating this a 100 of times. Finally, the amount of classical zero and one values obtained from the measurement will be printed out. The usage of Hadamard here means that we are expecting a roughly equal distribution of zeroes and ones as that is guaranteed by the mathematical formalism of quantum mechanics.

Additionally, it’s worth mentioning that we use helper functions from various Q# namespaces - such as the MResetZ or IntAsString, all of which must be opened first, hence the appropriate statements up top.

Now, instead of using the regular QDK process of building a quantum application and then simulating it on top of the .NET runtime using dotnet run, we will build our own program that will make use of the Q# compiler and emit the C# code that can be used for simulation. Then, we will compile this C# code and run it to simulate our quantum application.

Embedding the Q# compiler into a C# application 🔗

One of the great value propositions of Roslyn, the C# compiler, is that it can be used as a so called compiler-as-a-service. This means that it is possible to incorporate the compiler into your own program, and use the compiler’s pipeline to build your own features and to easily compile code on demand. We will show here today that the Q# compiler can also be used in a similar way - which I personally find very attractive.

In order to add a Q# compiler to a C# application, we need to add a package reference to the Microsoft.Quantum.Compiler Nuget package. That package, just like the rest of the Microsoft quantum packages are versioned together with the QDK, so the latest compiler version is actually the same version as the latest QDK that you might read about in the announcement posts. For example, at the time of writing, the current QDK version is 0.12.20072031, which means all of the related packages we shall use will have the same version. This makes it quite easy to understand how different packages relate to each other; this is particularly useful in case you pull in several quantum packages at once - which we will do here. We are going to reference the packages responsible for C# generation, simulation and entry point generation.

On top of that we will also add a reference to the C# compiler - we will use the C# Roslyn compiler to compile the emitted C# code. At the time of writing, the latest Roslyn version on Nuget is actually 3.7.0.4-final, but we will use version 3.6.0 instead. That is the version that is used by the Q# to C# code generation package 0.12.20072031 and we want to avoid compiler warning about mismatching versions of Roslyn.

Overall, the project file for our C# application looks as follows:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.1</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="3.6.0" />
    <PackageReference Include="Microsoft.Quantum.Compiler" Version="0.12.20072031" />
    <PackageReference Include="Microsoft.Quantum.Simulators" Version="0.12.20072031" />
    <PackageReference Include="Microsoft.Quantum.CsharpGeneration" Version="0.12.20072031" />
    <PackageReference Include="Microsoft.Quantum.EntryPointDriver" Version="0.12.20072031" />
  </ItemGroup>

</Project>

Invoking the Q# compiler programmatically 🔗

The process that we are about to go through is the following:

1. get ahold of some Q# code - this we prepared already
2. prepare assembly references necessary to compile the Q# code
3. prepare the Q# compiler so that it would include a rewrite step that would rewrite the Q# syntax trees into C# code
4. invoke the Q# compiler and if no error diagnostics were produced, capture the created C# output
5. prepare assembly references necessary to compile the created C# code
6. compile the C# code and if no error diagnostics were produced, emit into an in-memory assembly
7. invoke the in-memory assembly to execute our simulated Q# program

Let’s look at the steps one by one. First, we will - for simplicity - start by holding our Q# code in a C# string variable.

    // sample Q# code
    var qsharpCode = @"
namespace HelloQuantum {

    open Microsoft.Quantum.Canon;
    open Microsoft.Quantum.Measurement;
    open Microsoft.Quantum.Intrinsic;
    open Microsoft.Quantum.Convert;

    @EntryPoint()
    operation HelloQ() : Unit {
        let ones = GetRandomBit(100);
        Message(""Ones: "" + IntAsString(ones));
        Message(""Zeros: "" + IntAsString((100 - ones)));
    }
    
    operation GetRandomBit(count : Int) : Int {
 
        mutable resultsTotal = 0;
 
        using (qubit = Qubit()) {       
            for (idx in 0..count) {               
                H(qubit);
                let result = MResetZ(qubit);
                set resultsTotal += result == One ? 1 | 0;
            }
            return resultsTotal;
        }
    }
}";

Next, we shall locate the necessary Q# assemblies required for this code to compile. Based on the Q# types in use, that would actually only be Microsoft.Quantum.QSharp.Core and Microsoft.Quantum.Runtime.Core, however depending on the complexity of the Q# code you are trying to compile and the nature of computations done, additional libraries may of course be required.

Since our app already references Microsoft.Quantum.Simulators, and that package contains both of the above mentioned assemblies as dependencies, we can locate them in our application already by assembly name and find the physical location that way. However, it would be equally valid to just hardcode the paths to those DLLs or use any other DLL location mechanism that would suit you. Most important is, that we have a list of reference locations that we can hand off to the Q# compiler.

// necessary references to compile our Q# program
var qsharpReferences = new string[]
{
    "Microsoft.Quantum.QSharp.Core",
    "Microsoft.Quantum.Runtime.Core",
}.Select(x => Assembly.Load(new AssemblyName(x))).Select(a => a.Location);

Next we will enable additional tracing in the Q# compiler. This is technically not necessary, but it will give us a better overview of what is going on in the compiler when it gets invoked. This is - unfortunately - at the moment only possible to do statically. We shall log the received events to the console and be able to follow the compiler’s progress that way.

// events emitted by the Q# compiler
CompilationLoader.CompilationTaskEvent += (sender, args) =>
{
    Console.WriteLine($"{args.ParentTaskName} {args.TaskName} - {args.Type}");
};

Before we move towards the next step, we need to quickly provide a short piece of background information about how the Q# compiler works under the hood. The compiler exposes an interesting extensibility point called rewrite steps. It allows you to provide your own custom logic - compilation steps - that get executed during the compilation process. The QDK actually ships with a dedicated assembly, Microsoft.Quantum.CsharpGeneration which contains a rewrite step called CsharpGeneration (interestingly the C# generation code in this case is actually written in …F#). That step is responsible for taking the Q# code and translating it into C# code that could run and simulate the quantum behavior. All of that machinery is bootstrapped by the QDK and as a user you typically do not have to worry about that - but indeed, at its core, it is that rewrite step that allows you to execute your Q# code on your non-quantum development machine when you invoke dotnet run from the command line. The entire rewrite step infrastructure is very interesting and we shall definitely return to it in the future posts.

What we will want to do in our sample though, is we will not use the “official” CsharpGeneration rewrite step (the one that the QDK automatically uses behind the scenes) but instead provide our own. The reason behind this is that the official generation step, while it generates a perfectly valid C# that can be used to simulate the Q# application, explicitly writes out the output to text files located on the file system. What the QDK then does, it actually picks them up from your hard drive and continues from there. In our sample, however, we’d like to keep everything in process and therefore we will simply capture the C# output into in-process state, rather than having to worry about performing the unnecessary file I/O.

I will not go into much of the details of writing rewrite steps for the Q# compiler - as mentioned, this warranties its own dedicated blog post. It is enough to say that we will implement the IRewriteStep interface, where the core functionality is to provide an outgoing modified QsCompilation based on the incoming QsCompilation. In addition to performing compilation transformation, it also allows us to provide precondition and postcondition verification, though that is something that we will not need to do.

The full code of the rewrite step is shown below. It is entirely based on the official CsharpGeneration step - with the main difference being that it does not write out the C# code into physical files, but keeps them in memory. However, all the generated C# is identical. Notice that we don’t really modify the Q# compilation - we just use the step to extract the C# code based on it.

     class InMemoryEmitter : IRewriteStep
    {
        public static Dictionary<string, string> GeneratedFiles { get; } = new Dictionary<string, string>();

        private readonly Dictionary<string, string> _assemblyConstants = new Dictionary<string, string>();
        private readonly List<IRewriteStep.Diagnostic> _diagnostics = new List<IRewriteStep.Diagnostic>();

        public string Name => "InMemoryCsharpGeneration";

        public int Priority => -2;

        public IDictionary<string, string> AssemblyConstants => _assemblyConstants;

        public IEnumerable<IRewriteStep.Diagnostic> GeneratedDiagnostics => _diagnostics;

        public bool ImplementsPreconditionVerification => false;

        public bool ImplementsTransformation => true;

        public bool ImplementsPostconditionVerification => false;

        public bool PreconditionVerification(QsCompilation compilation)
        {
            throw new NotImplementedException();
        }

        public bool Transformation(QsCompilation compilation, out QsCompilation transformed)
        {
            var context = CodegenContext.Create(compilation, _assemblyConstants);
            var sources = GetSourceFiles.Apply(compilation.Namespaces);

            foreach (var source in sources.Where(s => !s.Value.EndsWith(".dll", StringComparison.OrdinalIgnoreCase)))
            {
                var content = SimulationCode.generate(source, context);
                GeneratedFiles.Add(source.Value, content);
            }

            if (!compilation.EntryPoints.IsEmpty)
            {
                var callable = context.allCallables.First(c => c.Key == compilation.EntryPoints.First()).Value;
                var content = EntryPoint.generate(context, callable);
                NonNullable<string> entryPointName = NonNullable<string>.New(callable.SourceFile.Value + ".EntryPoint");
                GeneratedFiles.Add(entryPointName.Value, content);
            }

            transformed = compilation;
            return true;
        }

        public bool PostconditionVerification(QsCompilation compilation)
        {
            throw new NotImplementedException();
        }
    }

Unfortunately the one thing that is not particularly elegant about the code above is that we need to capture the generated C# files into a static property. The reason for that, is that at the moment, the lifetime of the rewrite steps (or more specifically, their initialization) is managed by the compiler itself. Therefore we do not have access to the object instance representing the step after the compiler initialized and used them - which would be necessary to extract any state out of them.

To point the Q# compiler at our custom rewrite step, we should create a relevant configuration object. Since, as mentioned, the compiler controls the lifetime of the steps, we only need to tell it which assembly it should look at when discovering the steps (in our case, the current application’s assembly).

// to load our custom rewrite step, we need to point Q# compiler config at our current assembly
var config = new CompilationLoader.Configuration
{
    IsExecutable = true,
    RewriteSteps = new List<(string, string)>
    {
        ( Assembly.GetExecutingAssembly().Location, null),
    },
};

We shall also mark the Q# application as executable, meaning it can be invoked without a driver written in another language. The consequence of this is that the generated C# code will be a command line application too, with a relevant entry point.

Finally it’s time to invoke the Q# compiler - we do have a set of references, the relevant configuration, and - of course - the Q# code captured into a string variable. This is enough to get us going, but I will introduce one other component, and that is a custom logger. The compiler exposes an ILogger interface, along with an abstract LogTracker type that can be used to capture the output of the internal activities of the compiler. Remember that we already tapped into the compiler’s event stream earlier, but this is somewhat orthogonal to it as it contains much finer level of details and direct access to e.g. exception stack traces.

In our case, for example, we’d like to be sure that our compilation rewrite step has been loaded successfully from the location we are at, and this is what the logger gives us. A basic implementation of the logger is shown below:

public class ConsoleLogger : LogTracker 
{
    private readonly Func<Diagnostic, string> applyFormatting;

    protected internal virtual string Format(Diagnostic msg) =>
        this.applyFormatting(msg);

    protected sealed override void Print(Diagnostic msg) =>
        PrintToConsole(msg.Severity, this.Format(msg));

    public ConsoleLogger(
        Func<Diagnostic, string> format = null,
        DiagnosticSeverity verbosity = DiagnosticSeverity.Hint,
        IEnumerable<int> noWarn = null,
        int lineNrOffset = 0)
    : base(verbosity, noWarn, lineNrOffset) =>
        this.applyFormatting = format ?? Formatting.HumanReadableFormat;

    private static void PrintToConsole(DiagnosticSeverity severity, string message)
    {
        if (message == null)
        {
            throw new ArgumentNullException(nameof(message));
        }

        Console.WriteLine(severity.ToString() + " " + message);
    }
}

With the logger in hand we can finally initialize our compiler. This is done via the CompilationLoader type. Interestingly, the way the compiler is implemented, simply instantiating this type will already invoke the entire compilation pipeline.

// compile Q# code
var compilationLoader = new CompilationLoader(loadFromDisk =>
    new Dictionary<Uri, string> { { new Uri(Path.GetFullPath("__CODE_SNIPPET__.qs")), qsharpCode } }.ToImmutableDictionary(), 
    qsharpReferences, 
    options: config, 
    logger: new ConsoleLogger());

The Q# compiler normally expects a list of Q# source files to be passed into it, but it also supports compiling arbitrary piece of code using a custom loader, which is something we leverage here to “resolve” the file contents from a local variable. We still need to give it a dummy filename, which could be any name, but the convention already used in the command line Q# compiler, which is as good as any, is to name such files CODE_SNIPPET.qs.

At this point, the compilation will have already completed, which means we can have a look at compiler’s diagnostics. There are two things that could have happened - either the compiler emitted some error diagnostics, and the compilation was unsuccessful, or the compilation succeeded. In either case, we’d want the diagnostics to be printed out, and if errors really happened, well, we should exit the program because there is no point in continuing.

// print any diagnostics
if (compilationLoader.LoadDiagnostics.Any())
{
    Console.WriteLine("Diagnostics:" + Environment.NewLine + string.Join(Environment.NewLine, diagnostics.Select(d => $"{d.Severity} {d.Code} {d.Message}")));

    // if there are any errors, exit
    if (compilationLoader.LoadDiagnostics.Any(d => d.Severity == Microsoft.VisualStudio.LanguageServer.Protocol.DiagnosticSeverity.Error))
    {
        return;
    }
}

If there are no errors, we can proceed towards compiling the C# code - using the C# output we generated in the custom rewrite step in the Q# compiler. We will be using, naturally, the Roslyn C# compiler for that. Similarly to how we needed to bootstrap the references for Q# compiler, we need to gather the necessary references here too. They need to cover multiple things - the regular BCL C# types (e.g. System.Runtime), the contract assemblies (e.g. netstandard), the types that are used to represent Q# in C# code and simulate it (e.g. Microsoft.Quantum.Simulators or Microsoft.Quantum.Runtime.Core) and finally the support types needed by the emitted C# code (e.g. System.CommandLine). The full list of references for our case is below - depending on the type of your code it might be larger for you, especially when you use custom Q# libraries.

// necessary references to compile C# simulation of the Q# compilation
var csharpReferences = new string[]
{
    "Microsoft.Quantum.QSharp.Core",
    "Microsoft.Quantum.Runtime.Core",
    "Microsoft.Quantum.Simulators",
    "Microsoft.Quantum.EntryPointDriver",
    "System.CommandLine",
    "System.Runtime",
    "netstandard",
    "System.Collections.Immutable",
    typeof(object).Assembly.FullName,
}.Select(x => Assembly.Load(new AssemblyName(x))).Select(a => a.Location);

Since we captured all the C# generated code into a static property GeneratedFiles of our custom rewrite step in the Q# compiler, we can now use those to create Roslyn C# syntax trees that we can then feed into the C# compiler.

// we captured the emitted C# syntax trees into a static variable in the rewrite step
var syntaxTrees = InMemoryEmitter.GeneratedFiles.Select(x => CSharpSyntaxTree.ParseText(x.Value));

The next step is to actually invoke the C# compiler - using the above metadata references and the syntax trees as input. Just like in the case of Q# compilation, we need to check the diagnostics - which we will want to print out, should there be any. If there are any errors, we also need to break the entire procedure.

// print any diagnostics
var csharpDiagnostics = csharpCompilation.GetDiagnostics().Where(d => d.Severity != DiagnosticSeverity.Hidden);
if (csharpDiagnostics.Any())
{
    Console.WriteLine("C# Diagnostics:" + Environment.NewLine + string.Join(Environment.NewLine, csharpDiagnostics.Select(d => $"{d.Severity} {d.Id} {d.GetMessage()}")));

    // if there are any errors, exit
    if (csharpDiagnostics.Any(d => d.Severity == DiagnosticSeverity.Error))
    {
        return;
    }
}

The final step is to emit the C# compilation into an in memory assembly and then run it from that assembly using reflection - leveraging the emitted entry point. The entry point will have a special name QsEntryPoint as created by the C# generation code we used.

// emit C# code into an in memory assembly
using var peStream = new MemoryStream();
var emitResult = csharpCompilation.Emit(peStream);
peStream.Position = 0;
var qsharpLoadContext = new QSharpLoadContext();

// run the assembly using reflection
var qsharpAssembly = qsharpLoadContext.LoadFromStream(peStream);

// the entry point has a special name "__QsEntryPoint__"
var entryPoint = qsharpAssembly.GetTypes().First(x => x.Name == "__QsEntryPoint__").GetMethod("Main", BindingFlags.NonPublic | BindingFlags.Static);
var entryPointTask = entryPoint.Invoke(null, new object[] { null }) as Task<int>;
await entryPointTask;
qsharpLoadContext.Unload();

We ended up using a custom load context here - this is not really necessary, but is a nice pattern to keep in mind, since .NET Core 3.0+ supports collectible assemblies. The code for the load context is very simple - it simply marks itself as collectible.

public class QSharpLoadContext : AssemblyLoadContext
{
    public QSharpLoadContext() : base(isCollectible: true)
    {
    }
}

And that’s it! At this point, the entire end-to-end Q# compiler and simulation should have been programmatically invoked - without using the QDK command line machinery, without creating any temporary C# files or any temporary DLLs.

The output should look similar to this:

 OverallCompilation - Start
Information 
Information: 
Loaded rewrite steps that are executing as part of the compilation process
    InMemoryCsharpGeneration (file:///Users/filip/Documents/dev/Strathweb.Samples.QSharpCompiler/bin/Debug/netcoreapp3.1/Strathweb.Samples.QSharpCompiler.dll)
OverallCompilation SourcesLoading - Start
OverallCompilation SourcesLoading - End
OverallCompilation ReferenceLoading - Start
Information 
Information: 
Compiling with referenced assemblies
    /Users/filip/Documents/dev/Strathweb.Samples.QSharpCompiler/bin/Debug/netcoreapp3.1/Microsoft.Quantum.QSharp.Core.dll
    /Users/filip/Documents/dev/Strathweb.Samples.QSharpCompiler/bin/Debug/netcoreapp3.1/Microsoft.Quantum.Runtime.Core.dll
OverallCompilation ReferenceLoading - End
OverallCompilation Build - Start
OverallCompilation Build - End
OverallCompilation RewriteSteps - Start
OverallCompilation RewriteSteps - End
OverallCompilation OutputGeneration - Start
OutputGeneration SyntaxTreeSerialization - Start
OutputGeneration SyntaxTreeSerialization - End
OverallCompilation OutputGeneration - End
 OverallCompilation - End
C# Diagnostics:
Warning CS1702 Assuming assembly reference 'System.Collections.Immutable, Version=1.2.4.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' used by 'Microsoft.Quantum.EntryPointDriver' matches identity 'System.Collections.Immutable, Version=1.2.5.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' of 'System.Collections.Immutable', you may need to supply runtime policy
Ones: 58
Zeros: 42

The individual compilation steps are logged as part of the CompilationTaskEvent infrastructure into which we tapped in the beginning. On the other hand, the details related to the actual rewrite steps and Q# assembly references used, are provided by the ILogger component. There are no Q# compiler diagnostics in this output and a single C# diagnostic. It doesn’t cause any trouble, it’s just a mismatch on the System.Collections.Immutable between the one used in the compiler and in our application; this could be aligned by changing the version used by our application. That said, I left it in just to illustrate that the diagnostics actually work.

Ultimately, at the end we see the simulated output of the Q# application - from our C# application. And this was the goal of this blog post.

Ones: 58
Zeros: 42

The full code is available on Github - hopefully you will find this useful.