In this post I wanted to show you how to write and embed a C# interactive shell (a REPL - read-evaluate-print-loop) in a browser, on top of WebAssembly.
The REPL will give you fully fledged C# interactive development playground, while still being completely sandboxed in the browser environment. I originally wrote this example for my session at Dotnet Cologne on May 10 this year.
More after the jump.
TL;DR? π
DEMO https://csharpinteractive.azurewebsites.net - warning: this is running on F1 free web app in Azure, so if it goes down, well, it goes down. Also there is no much optimization so it is a laaarge download and it might take a while; after that it should be in local cache.
source code: on Github
Getting started with Blazor π
There are plenty of excellent tutorials on how to get started with Blazor - Microsoft’s framework, that’s part of the ASP.NET family, that allows us to run C# code on the client side in the browser, on top of WebAssembly.
This blog post is not intended to be an introduction to Blazor, instead, I’d like to show you how to bootstrap the C# compiler in the interactive mode (submission based compilations), in order to deliver an interactive experience to the user.
The most basic introductory Blazor tutorial is available here, and if you are not familiar with Blazor, it is recommended that you have a read before continuing here.
Building a C# REPL with Roslyn π
It is actually remarkably simple to build a C# Interactive shell with Roslyn. Via the package called Microsoft.CodeAnalysis.CSharp.Scripting, you get access to what I like to call high level scripting APIs of the C# compiler.
The simplest possible C# REPL with those APIs can be constructed in just a few lines of code:
ScriptState<object> scriptState = null;
while (true)
{
Console.Write("> ");
var input = Console.ReadLine();
scriptState = scriptState == null ?
await CSharpScript.RunAsync(input) :
await scriptState.ContinueWithAsync(input);
}
So what goes on here? We run an infinite loop, that prompts the user to input some C# code. We then take that input and execute it as a C# script code block using the scripting APIs found in Microsoft.CodeAnalysis.CSharp.Scripting, and we ask for more input. After the initial execution of code, the state gets created, and going forward, instead of running the subsequent submissions stand alone, we simply invoke them as continuations on the earlier state (again, the relevant API is exposed by Roslyn). What happens under the hood, is that each submission of code into the REPL actually gets compiled by Roslyn and the compiler takes care of all the necessary glue to join it all together.
This is a trivial example and it doesn’t cover a lot of cases such as capturing and printing potential result of an evaluated expression, doesn’t bootstrap any of the more complex assembly references, doesn’t predefine and global namespace imports, doesn’t support multi-line input and plenty more. It is however a valid starting point, and from the pure language perspective it is a fully functional C# interactive shell.
Building a C# REPL in a browser π
As we have seen in the example above, building a simple interactive shell with the high-level scripting APIs of Roslyn is very simple. Now, to get this working on top of Blazor, is a bit more challenging - as we will see soon.
We will be using version 3.0.0-preview5-19227-01 of Blazor for this exercise.
Once you create a basic Blazor project, we can start putting together the basic pieces. To not complicate things a lot, I will stuff everything - both the HTML side, and the C# code - into the Index.razor page. This will not be the most elegant solution, and of course in real life you may choose to architect things in a bit cleaner way, but for the purpose of this exercise it should be more than enough.
My HTML is going to look like this:
<div id="outer-screen">
<div id="inner-screen">
<div id="output-outer"><div id="output"><!-- TODO: display output --></div></div>
<div id="blank-line"> </div>
<div id="input-row">
<span>></span>
<!-- TODO: display input prompt -->
</div>
</div>
</div>
We can skip the CSS and styling part, other than say that it’s there to make things look nice - it’s available in the source code, if you follow the repository link at the end of this blog. The actual HTML+CSS styling used for this example comes from Alan, so big thanks to him.
The HTML comes with two placeholders that we’ll need to fill with:
- input prompt for the user (so that the user can type in C# code)
- output - in case the user’s code evaluates to a return value, we’d want to show that to the user. We’ll also mirror back user’s input into the output panel so that the user gets a nice experience/feeling of working with an interactive shell
We will be writing all the code inline in the Index.razor, under the @functions {} section. This means we can just use fields, properties and methods straight up, without needing to wrap them into a class.
The first piece of C# code to add will handle the ability to run code - so bind to the ENTER key being pressed inside the input field. That’s corresponding to the HTML we haven’t seen yet, and it will be slotted in the TODO: display input prompt placeholder we had seen earlier.
<input id="input" bind="@Input" type="text" onkeyup="@Run" autofocus />
The value of the input field will be 2-way bound to an @Input C# property which we will place under @functions {}. Again, we could have used a proper dedicated class as a view model here, but let’s keep things simple. When the key up JS event occurs, we will be invoking @Run method, which will be responsible for submitting the user-written code to our REPL engine. Note that the event happens for each key up, but we are only interested in ENTER key, so we will need to filter them.
At the moment our C# code looks like this:
@functions
{
public string Output { get; set; } = "";
public string Input { get; set; } = "";
public async Task Run(UIKeyboardEventArgs e)
{
if (e.Key != "Enter")
{
return;
}
var code = Input;
Input = "";
await RunSubmission(code);
}
}
I already added an Output property to our @functions {} too. It is not used yet, but we’ll be making use of it very soon. It will be bound to the HTML that we shall place in the TODO: display output placeholder.
<div id="output">@((MarkupString)Output)</div>
Finally, in our C# code snippet, we called a method RunSubmission(code), which doesn’t actually exist yet, but we’ll be adding it soon - it will be the entry point into our in browser REPL engine, powered by the C# compiler.
But before we get there, let’s discuss for a moment how we can actually compile the snippets of code that users will submit into the interactive shell. We already mentioned that there is a very easy to bootstrap a REPL with the high-level Roslyn scripting APIs. The problem is that those APIs currently do not work on top of Mono WASM (the WebAssembly implementation backing Blazor). The main hurdle is that Roslyn scripting APIs will attempt to add an implicit reference to the assembly containing the object type implementation to each of our compilations (each submission into the interactive shell). This is very useful under normal circumstances, but on WASM it fails; Roslyn will look for this assembly using the Assembly.Location property and attempt to load it from disk - but on Mono WASM the location property returns a file name that doesn’t correspond to the real location and the whole thing crashes. In fact, under WASM you run inside of the browser sandbox, so the file system, from the application perspective, is completely empty. Therefore any attempt to load those DLLs from there would fail.
However, there is no problem - we can fairly easily get around all of this, by using the lower level APIs of the Roslyn compiler. It will no longer be so elegant as the earlier example but it will work; it will just mean we will have to manually take care of a bunch of things that the high-level scripting APIs normally give us “for free”.
At the lower level, the compiler exposes a method CreateScriptCompilation() on CSharpCompilation which can be used to create compilations relevant for the interactive context. It will also allow us to emit the assembly in the script-specific format - with the entry point that can handle the chain of submissions too.
The code that uses it, is shown next:
private bool TryCompile(string source, out Assembly assembly, out IEnumerable<Diagnostic> errorDiagnostics)
{
assembly = null;
var scriptCompilation = CSharpCompilation.CreateScriptCompilation(
Path.GetRandomFileName(),
CSharpSyntaxTree.ParseText(source, CSharpParseOptions.Default.WithKind(SourceCodeKind.Script).WithLanguageVersion(LanguageVersion.Preview)),
_references,
new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary, usings: new[]
{
"System",
"System.IO",
"System.Collections.Generic",
"System.Console",
"System.Diagnostics",
"System.Dynamic",
"System.Linq",
"System.Linq.Expressions",
"System.Net.Http",
"System.Text",
"System.Threading.Tasks"
}),
_previousCompilation
);
errorDiagnostics = scriptCompilation.GetDiagnostics().Where(x => x.Severity == DiagnosticSeverity.Error);
if (errorDiagnostics.Any())
{
return false;
}
using (var peStream = new MemoryStream())
{
var emitResult = scriptCompilation.Emit(peStream);
if (emitResult.Success)
{
_submissionIndex++;
_previousCompilation = scriptCompilation;
assembly = Assembly.Load(peStream.ToArray());
return true;
}
}
return false;
}
There is a little bit to unpack here. First of all, when we call the CreateScriptCompilation(), we need to pass in the code that the user attempts to compile, but already converted into a syntax tree. In other to get a syntax tree out of a string-based code representation, we use a parser, specifically configured to use SourceCodeKind.Script when parsing the code. That will enable those slightly more relaxed rules of scripted C#, like ability to have global members. That is of course really important in a REPL context, because we don’t want to force the user to package every single submission into the interactive shell in a class or a, ugh, static void Main()!
There are three additional things we will pass into the CreateScriptCompilation() - metadata references (so, really assembly references), global using statements and a reference to previous compilation.
Global using statements are allowed in the script mode, and really do make sense if you think about it; this way our interactive shell user can interact with types in those predefined namespaces without having to explicitly import them. This is not mandatory, but is a very welcome convenience when working in a REPL; in our case, I simply decided to import some common ones.
The reason why we pass in a previous compilation reference is so that the user can reference types and members created in earlier submissions from later submissions. For example, when a user sends var x = 1; into the REPL, it will be natural for the user to expect that the variable x is accessible in subsequent submissions, allowing the user to execute, for example x+x. Of course since each submission is a separate compilation, then the declaration and usage would really be detached from each other - and that’s the reason why we need to keep passing the previous compilation into the subsequent one. We don’t need to worry about anything else - Roslyn will take care of the rest.
Finally, as mentioned, we need to pass in some MetadataReferences into the compliation. At the very least, the mscorlib, or System.Runtime, depending on the platform, so the assembly containing the implementation of object and other BCL types in ecessary for anything useful to work. We already mentioned that typically high-level scripting APIs will take care of that, but since we are not using them, we need to do it manually.
In this particular case our runtime environment is Blazor and we need to pick up those references in a way that will work - we already mentioned that under Mono WASM, the Location property of the assemblies returns a filename that doesn’t really correspond to the real physical location. We will need to work around that, and the workaround is shown next:
var refs = AppDomain.CurrentDomain.GetAssemblies();
var client = new HttpClient
{
BaseAddress = new Uri(WebAssemblyUriHelper.Instance.GetBaseUri())
};
var references = new List<MetadataReference>();
foreach(var reference in refs.Where(x => !x.IsDynamic && !string.IsNullOrWhiteSpace(x.Location)))
{
var stream = await client.GetStreamAsync($"_framework/_bin/{reference.Location}");
references.Add(MetadataReference.CreateFromStream(stream));
}
Basically what we want to do, is we’d configure the interactive shell to have access to all the assemblies that our host Blazor application has. This will be the entire BCL plus some of the Blazor specific ASP.NET Core libraries. We already established that fetching them using the filesystem won’t work - the trick to fetch them using an HTTP request and handle as Stream. They will actually be available under _framework/_bin/{assembly.Location}.
Once we have those references, we can pass them into the compilation - and we will continue passing them to each subsequent compilation too. However, we need to fetch them using those HTTP requests only once, for the initial compilation, and can then reuse them, that’s why it makes sense to move this code into our Blazor app initialization, and simply store all the references in a field.
In order to run some code at app initialization in Blazor, you can override a method called OnInitAsync() - this works both from a dedicated view model class, as well as inside the @functions {} block. The result is shown next - and will become part of our Index.razor:
private IEnumerable<MetadataReference> _references;
protected async override Task OnInitAsync()
{
var refs = AppDomain.CurrentDomain.GetAssemblies();
var client = new HttpClient
{
BaseAddress = new Uri(WebAssemblyUriHelper.Instance.GetBaseUri())
};
var references = new List<MetadataReference>();
foreach(var reference in refs.Where(x => !x.IsDynamic && !string.IsNullOrWhiteSpace(x.Location)))
{
var stream = await client.GetStreamAsync($"_framework/_bin/{reference.Location}");
references.Add(MetadataReference.CreateFromStream(stream));
}
_references = references;
}
This addition allows us to interact with the _references field inside our TryCompile() method we were discussing earlier.
The remainder of the TryCompile() is checking for compilation errors by looking at the diagnostics, and emitting the assembly in case the compilation succeeded. We emit into a memory stream, and load the assembly from there - there is no need to write out anything to disk.
Finally, we store the compilation in the _previousCompilation field - so that it can be used when compiling the subsequent submission, and we also store an index (_submissionIndex). This state tracking will be needed soon and we’ll explain that in a moment.
What’s left to do is to write the RunSubmission() method we mentioned already in the first snippet - it will of course make use of our new TryCompile() we have just written and analyzed. It is shown next:
private async Task RunSubmission(string code)
{
Output += $@"<br /><span class=""info"">{HttpUtility.HtmlEncode(code)}</span>";
var previousOut = Console.Out;
try
{
if (TryCompile(code, out var script, out var errorDiagnostics))
{
var writer = new StringWriter();
Console.SetOut(writer);
var entryPoint = _previousCompilation.GetEntryPoint(CancellationToken.None);
var type = script.GetType($"{entryPoint.ContainingNamespace.MetadataName}.{entryPoint.ContainingType.MetadataName}");
var entryPointMethod = type.GetMethod(entryPoint.MetadataName);
var submission = (Func<object[], Task>)entryPointMethod.CreateDelegate(typeof(Func<object[], Task>));
if (_submissionIndex >= _submissionStates.Length)
{
Array.Resize(ref _submissionStates, Math.Max(_submissionIndex, _submissionStates.Length * 2));
}
var returnValue = await ((Task<object>)submission(_submissionStates));
if (returnValue != null)
{
Console.WriteLine(CSharpObjectFormatter.Instance.FormatObject(returnValue));
}
var output = HttpUtility.HtmlEncode(writer.ToString());
if (!string.IsNullOrWhiteSpace(output))
{
Output += $"<br />{output}";
}
}
else
{
foreach (var diag in errorDiagnostics)
{
Output += $@"<br / ><span class=""error"">{HttpUtility.HtmlEncode(diag)}</span>";
}
}
}
catch (Exception ex)
{
Output += $@"<br /><span class=""error"">{HttpUtility.HtmlEncode(CSharpObjectFormatter.Instance.FormatException(ex))}</span>";
}
finally
{
Console.SetOut(previousOut);
}
}
Let’s quickly go through this code. First of all, we’ll echo user’s input into the output - that sort of makes sense, because in a REPL you are used to seeing your code “jump up” into the history of commands, rather than completely disappearing. Note that for simplicity we’ll sprinkle in some HTML into the output; obviously you could make it nicer if you wanted to. We are also going to escape the HTML encoding here because some code, such as for example generics, wouldn’t display otherwise.
We’ll then capture Console.Out into a temporary variable, this will allow us to temporarily pipe all Console output through a StringWriter, which we can than add to the Output field too. This way, if the user runs any code that writes to Console.Out, we’ll be able to intercept that and show it on the screen to - it’s a simple convenience thing for the user; otherwise it would be quite confusing if the Console.Out didn’t show up on the screen, but instead go the browser’s developer tools and the console there, which is the default behavior in Blazor.
We then reach a point where we compile the code, using the method we already wrote. If the compilation fails, we will print out the diagnostics in red color and exit - resetting the Console.Out at the end. If there is any other exception that is unhandled we also print that and exit with the console being reset.
The interesting stuff happens though, when the compilation succeeds. At that point we get an assembly that we can execute - but we need to find an entry point - after all this is not a regular console application but something the compiler called “script compilation”. In reality, under the hood, there is a special Script class that gets emitted, that contains a static method with a funky metadata name
C# scripting factory method (entry point) can be cast to Func<object[], Task> so we will do that to get some semi-type safety. The input to it, the object[] array, is called a submission array. This is again an implementation detail of scripting in the C# compiler, and is normally hidden by the high level scripting APIs (CSharpScript class). In this case we have to manage it manually though. Without going into much details, this is a state array that links separate compilations in our interactive shell together. We will leave the first entry null, since it’s reserved for other purposes that are out of scope for this article (the so called globals object) and from index 1 onwards, the runtime, when our scripted code executes, will actually be internally filling the array with instances of the script wrapper class itself (which gets tried inside our entry point). Therefore, we need to do one more thing that normally the high level APIs would do for us - and that is take care of the manual resizing of the submission array, so that we don’t run out of slots (recall our _submissionIndex from earlier? we need it to manage this array resizing).
It is possible for a script expression in a REPL to have a return value - i.e. our aforementioned x+x expression which might produce value 2. If that is the case, it will actually come out from the factory entry point as object wrapped in a Task - so we can unwrap it and print to the output.
To summarize, all the code is here:
@functions {
public string Output { get; set; } = "";
public string Input { get; set; } = "";
private CSharpCompilation _previousCompilation;
private IEnumerable<MetadataReference> _references;
private object[] _submissionStates = new object[] { null, null };
private int _submissionIndex = 0;
protected async override Task OnInitAsync()
{
var refs = AppDomain.CurrentDomain.GetAssemblies();
var client = new HttpClient
{
BaseAddress = new Uri(WebAssemblyUriHelper.Instance.GetBaseUri())
};
var references = new List<MetadataReference>();
foreach(var reference in refs.Where(x => !x.IsDynamic && !string.IsNullOrWhiteSpace(x.Location)))
{
var stream = await client.GetStreamAsync($"_framework/_bin/{reference.Location}");
references.Add(MetadataReference.CreateFromStream(stream));
}
_references = references;
}
public async Task Run(UIKeyboardEventArgs e)
{
if (e.Key != "Enter")
{
return;
}
var code = Input;
Input = "";
await RunSubmission(code);
}
private async Task RunSubmission(string code)
{
Output += $@"<br /><span class=""info"">{HttpUtility.HtmlEncode(code)}</span>";
var previousOut = Console.Out;
try
{
if (TryCompile(code, out var script, out var errorDiagnostics))
{
var writer = new StringWriter();
Console.SetOut(writer);
var entryPoint = _previousCompilation.GetEntryPoint(CancellationToken.None);
var type = script.GetType($"{entryPoint.ContainingNamespace.MetadataName}.{entryPoint.ContainingType.MetadataName}");
var entryPointMethod = type.GetMethod(entryPoint.MetadataName);
var submission = (Func<object[], Task>)entryPointMethod.CreateDelegate(typeof(Func<object[], Task>));
if (_submissionIndex >= _submissionStates.Length)
{
Array.Resize(ref _submissionStates, Math.Max(_submissionIndex, _submissionStates.Length * 2));
}
var returnValue = await ((Task<object>)submission(_submissionStates));
if (returnValue != null)
{
Console.WriteLine(CSharpObjectFormatter.Instance.FormatObject(returnValue));
}
var output = HttpUtility.HtmlEncode(writer.ToString());
if (!string.IsNullOrWhiteSpace(output))
{
Output += $"<br />{output}";
}
}
else
{
foreach (var diag in errorDiagnostics)
{
Output += $@"<br / ><span class=""error"">{HttpUtility.HtmlEncode(diag)}</span>";
}
}
}
catch (Exception ex)
{
Output += $@"<br /><span class=""error"">{HttpUtility.HtmlEncode(CSharpObjectFormatter.Instance.FormatException(ex))}</span>";
}
finally
{
Console.SetOut(previousOut);
}
}
private bool TryCompile(string source, out Assembly assembly, out IEnumerable<Diagnostic> errorDiagnostics)
{
assembly = null;
var scriptCompilation = CSharpCompilation.CreateScriptCompilation(
Path.GetRandomFileName(),
CSharpSyntaxTree.ParseText(source, CSharpParseOptions.Default.WithKind(SourceCodeKind.Script).WithLanguageVersion(LanguageVersion.Preview)),
_references,
new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary, usings: new[]
{
"System",
"System.IO",
"System.Collections.Generic",
"System.Console",
"System.Diagnostics",
"System.Dynamic",
"System.Linq",
"System.Linq.Expressions",
"System.Net.Http",
"System.Text",
"System.Threading.Tasks"
}),
_previousCompilation
);
errorDiagnostics = scriptCompilation.GetDiagnostics().Where(x => x.Severity == DiagnosticSeverity.Error);
if (errorDiagnostics.Any())
{
return false;
}
using (var peStream = new MemoryStream())
{
var emitResult = scriptCompilation.Emit(peStream);
if (emitResult.Success)
{
_submissionIndex++;
_previousCompilation = scriptCompilation;
assembly = Assembly.Load(peStream.ToArray());
return true;
}
}
return false;
}
}
***Summary
And that’s it! We now have a fully functional (well, “fully functional”, it could be much better but it can do lots of stuff already) C# REPL. It runs entirely in the browser, on top of Mono WASM / Blazor and it has full browser sandbox. You could try interacting with the file system, and see that it’s empty - but you can create new files for yourself. You can attempt to make HTTP calls from HttpClient - and you will find out that they really show up in the browser’s XHR tools as outgoing browser calls. In fact they will even be subject to browser CORS rules!
I think in general the concept is pretty cool. Imagine a documentation site such as docs.microsoft.com or StackOverflow any other site with readmes/tutorials - providing an in-browser C# REPL so that you can play around with the types/examples straight away - that could be extremely valuable.
All the code for this article is available on Github. If there is enough interest, we might do a second part of this - wiring in intellisense and language services into this. Let me know if you’d find that interesting. One final note - if you are interested in this area in general, have a look at this nice example of Roslyn and WASM from Suchiman.
