Some time ago I posted a mini-series of posts about using Roslyn to script Web API, and that has gotten some great response. In that original post, I mentioned & used, without going into too much details, a very useful “compiler as a service” feature Roslyn offers.

Recently, Glenn Block started a very exciting project called scriptcs (which now Justin Rusbtach and I happen to be a part of too) to provide a seamless/node.js-esque scripting experience for C# and in that project we indeed leverage on Roslyn heavily - to do some behind the scenes tricks to hide the compilation aspect from the user, so that it really resembles pure script execution.

Disclaimer 🔗

All the stuff here is purely experimental so don’t blame me if your AppDomain explodes 🙂

Also, if you want to really experience pure C# scripting then you should definitely check out scriptcs. In fact, to be perfectly honest, the hidden agenda of this post is to draw your attention to the scriptcs project, and perhaps encourage you to contribute there!

Dynamic compilation the Roslyn way 🔗

While emitting dynamic assemblies isn’t anything new, the Roslyn APIs make it extremely easy to work with this type of code.

So here is a great potential scenario. Say, you are developing a Web API application; if you are doing it on top of ASP.NET you are probably used to the fact that re-compiling and restarting Cassini/IIS Express/IIS takes a few seconds and is pretty annoying.

How about, using Roslyn’s compiler as a service, and letting it do the compiling behind the scenes, while you just worry about writing the code? What if all you have to do is edit a controller/model and then just refresh the browser?

Let me explain - Web API exposes, in its execution pipeline, several hooks related to Type discovery and assembly resolving which you can replace (in the order of execution):

• • IAssembliesResolver
• • IHttpControllerTypeResolver
• • IHttpControllerSelector
• • IHttpControllerActivator

So you can tell it: “listen Web API, look into this and this assembly for my controller types.” There are few obstacles though, because by default Web API will ignore controllers from dynamic assemblies (which Roslyn would emit) and it uses internal cache for controllers, which really means they are only loaded when they are first discovered.

In order to bypass these limitations, we have to plug as low as into IHttpControllerSelector, and configure it collects the controllers/models from our solution, compiles them, emits in an in memory assembly and loads into the app domain. All that without any caching.

To compile with Roslyn, all we need to do is call a Create() method on Compilation class. This is actually identical to what I used in that original Roslyn post to compile controllers from a text file. We are just adding a twist now - to allow you to keep modifying it all the time.

var compiledCode = Compilation.Create(  
outputName: "my.dll",  
options: new CompilationOptions(OutputKind.DynamicallyLinkedLibrary),  
syntaxTrees: metadataReferences,  
references: new[] {  
new MetadataFileReference(typeof(object).Assembly.Location),  
new MetadataFileReference(typeof(Enumerable).Assembly.Location)  
});  

You pass in the name of the assembly, specify what kind of output you want, add references (i.e. System.dll, System.Core.dll and so on) and pass in the syntaxTrees of the code you want to compile.

So it’s very easy to collect *.cs files from a project and compile them selectively. For example, to gather all C# files representing controllers and models from a Web API project you could do something like this:

private const string Controllers = @"C:UsersFilipDocumentsVisual Studio 2012ProjectsMvcApplication6MvcApplication6Controllers";  
private const string Models = @"C:UsersFilipDocumentsVisual Studio 2012ProjectsMvcApplication6MvcApplication6Models";

var controllers = Directory.GetFiles(Controllers);  
var models = Directory.GetFiles(Models);  
var metadataReferences = controllers.Select(i =>; SyntaxTree.ParseFile(i)).Union(models.Select(i => SyntaxTree.ParseFile(i)));  

And then just pass the metadataReferences to the above Compile() method.

Implementing IHttpControllerSelector 🔗

So let’s create an IHttpControllerSelector which, instead of resolving controllers through a conventional Web API path, uses dynamic Roslyn compilation per every request instead.

You don’t even need to install full Roslyn CTP anymore, as it’s available on Nuget:

install-package roslyn  

Next:

public class NoCacheSelector : DefaultHttpControllerSelector  
{  
private readonly HttpConfiguration _configuration;  
private const string Controllers = @"C:UsersFilipDocumentsVisual Studio 2012ProjectsMvcApplication6MvcApplication6Controllers";  
private const string Models = @"C:UsersFilipDocumentsVisual Studio 2012ProjectsMvcApplication6MvcApplication6Models";

public NoCacheSelector(HttpConfiguration configuration) : base(configuration)  
{  
_configuration = configuration;  
}

public override HttpControllerDescriptor SelectController(HttpRequestMessage request)  
{  
var controllers = Directory.GetFiles(Controllers);  
var models = Directory.GetFiles(Models);  
var metadataReferences = controllers.Select(i => SyntaxTree.ParseFile(i)).Union(models.Select(i => SyntaxTree.ParseFile(i)));

var compiledCode = Compilation.Create(  
outputName: "controllersAndModels.dll",  
options: new CompilationOptions(OutputKind.DynamicallyLinkedLibrary),  
syntaxTrees: metadataReferences,  
references: new[] {  
new MetadataFileReference(typeof(object).Assembly.Location),  
new MetadataFileReference(typeof(Enumerable).Assembly.Location),  
new MetadataFileReference(typeof(ApiController).Assembly.Location),  
});

Assembly assembly;  
using (var stream = new MemoryStream())  
{  
compiledCode.Emit(stream);  
assembly = Assembly.Load(stream.GetBuffer());  
}

var types = assembly.GetTypes();  
var matchedTypes = types.Where(i => typeof(IHttpController).IsAssignableFrom(i)).ToList();

var controllerName = base.GetControllerName(request);  
var matchedController = matchedTypes.FirstOrDefault(i => i.Name.ToLower() == controllerName.ToLower() + "controller");

return new HttpControllerDescriptor(_configuration, controllerName, matchedController);  
}  
}  

The first part is rather easy as we just discussed it moments ago. Once we create a compilation object, we could emit the result into a file (which we don’t want) or into memory (which we want) and then load into our AppDomain.

Then, we export the types from the dynamic assembly, and look up all controllers types, and then filter them to match one by name. Finally, we simply return the HttpControllerDescriptor back to the Web API pipeline and let things flow.

Notice we only need to jump through these hoops for controllers (because of a specific logic Web API has to resolving and caching them). Models can just be recompiled and thrown into the AppDomain.

Note that this only references three assemblies:

• • System.dll (typeof(object).Assembly)
• • System.Core.dll (typeof(Enumerable).Assembly)
• • System.Web.Http.dll (typeof(ApiController).Assembly)

If your models/controllers require more to compile, you need to add those. By the way, remember that Roslyn is a CTP so potentially there might be some bugs here and there. Also Roslyn syntax tree parser currently does not support dynamic and async/await.

Running this 🔗

So you can now test this out. Plug in the selector:

config.Services.Replace(typeof(IHttpControllerSelector), new NoCacheSelector(config));  

Now suppose you have a Web API controller & model:

public class HelloController : ApiController  
{  
public Hello Get(int id)  
{  
return new Hello {  
Text = "Hello World",  
Property = "Some value",  
Id = id};  
}  
}

public class Hello  
{  
public string Text { get; set; }  
public string Property { get; set; }  
public int Id { get; set; }  
}  

If you run this, not surprisingly it returns the expected values:

However, now, with the app running (i.e. IIS express, IIS etc) you can modify both files, let’s say:

public class HelloController : ApiController  
{  
public Hello Get(int id)  
{  
return new Hello {  
Text = "Goodbye World",  
NewProperty = "Where is my compilation?!",  
Id = id};  
}  
}

public class Hello  
{  
public string Text { get; set; }  
public string NewProperty { get; set; }  
public int Id { get; set; }  
}  

And you don’t need to recompile, just save them and refresh the page/call API again.

In fact, you can add new controllers/methods/models too (as long as they are inside our predefined /Controllers and /Models folders):

public class HelloController : ApiController  
{  
public Hello Get(int id)  
{  
return new Hello {  
Text = "Goodbye World",  
NewProperty = "Where is my compilation?!",  
Id = id};  
}

public Goodbye Get()  
{  
var goodbye = new Goodbye  
{  
WorstCityToLiveIn = "Toronto"  
};  
return goodbye;  
}  
}

public class Goodbye  
{  
public string WorstCityToLiveIn { get; set; }  
}  

and just call it without recompiling

Pretty cool - for development scenarios when you have to iterate a lot back and forth.

Side nodes 🔗

Be very careful of what we are doing in this spike. We are compiling controllers/models to an in memory assembly and loading into an AppDomain on every request. Because there is no way of unloading an assembly from AppDomain it’s easy to overflow the stack or fill up the memory. On the other hand, the assemblies produced by Roslyn’s compilation.Emit are not GC-collectible.

You can force GC too just prior to the selection, but that won’t help you much:

public override HttpControllerDescriptor SelectController(HttpRequestMessage request)  
{  
GC.Collect();  
GC.WaitForPendingFinalizers();  
GC.Collect();

var controllers = Directory.GetFiles(Controllers);  
//continue  
}  

Note that the fact that we’d potentially have old and new versions of same classes residing side by side in a single assembly is not that big of a problem, because the “old” controllers won’t be discovered since we control that process manually through the selector. As far as models go, when you write code Roslyn compiler will not let you reference old versions as the code would not compile, so you’ll be force to use new ones anyway.

There are some other issues with this solution (you might interop with other classes and so on), so consider it a mere spike, but hopefully it illustrates some really amazing approaches to writing C# code in a “cool” way.

I really hope that this article interests you more into the notion of dynamic compilation nd code execution - and that in turn will interest you in contributing to scriptcs!

See you there!