Because of my considerable community involvement in promoting C# scripting (i.e. here or here), I thought the other day, why not attempt to run C# scripts using DNX?

While out of the box, DNX only compiles proper, traditional C# only, thanks to the compilation hooks it exposes, it is possible to intercept the compilation object prior to it being actually emitted, which allows you to do just about anything - including run C# scripts.

Let’s explore more.

DNX pre compilation hooks 🔗

We discussed them a little bit in one of the older posts, but as a reminder, DNX exposes hooks allowing you to tap into the compilation process. At the time of writing this (DNX beta 6), the way this works is that you create a class implementing ICompileModule and place it in a folder called preprocess in your DNX project.

The interface resides in Microsoft.Framework.Runtime.Roslyn.Abstractions nuget package.

namespace Microsoft.Dnx.Compilation.CSharp  
{  
/// <summary> /// Module that allows plugging into the compilation pipeline

  
/// </summary> 

public interface ICompileModule  
{  
/// <summary> /// Runs after the roslyn compilation is created but before anything is emitted

  
/// </summary> 

/// void BeforeCompile(BeforeCompileContext context);

/// <summary> /// Runs after the compilation is emitted. Changing the compilation will not have any effect at this point

  
/// but the assembly can be changed before it is saved on disk or loaded into memory.  
/// </summary> 

/// void AfterCompile(AfterCompileContext context);  
}  

When discovering that your project has some designated “preprocess” source files, DNX will compile just the code inside the preprocess folder - so your module into a stand alone in memory assembly. It will also create a Roslyn Compilation object from the entire DNX project, containing all the code and assembly references gathered from project.json file and expose all that to your implementation of ICompileModule (remember, that was compiled first and will now be invoked in memory).

At that point, from inside your compile module you can freely interact with the compilation instance of the main DNX project, and do just about anything you want - including compiling something completely different yourself and setting that as the compilation to be used by the DNX runtime.

This is very meta, and very powerful, and using this technique it is relatively easy to support something like scripting.

Plugging in scripting 🔗

So how to enable scripting?

Well first thing is that the syntax trees with your code have to be parsed using SourceCodeKind.Script instead of SourceCodeKind.Regular - otherwise you will get lots of diagnostic errors, as obviously scripted C# is a bit more relaxed than traditional C#. We could do it in our custom ICompileModule, inside BeforeCompile method.

At this point we will discover that the Roslyn team is not making life easy for us. DNX beta 6 uses stable (1.0.0) versions of roslyn assemblies. Scripting capabilities are still not part of Roslyn (work on scripting is happening as part of version beta 1.1.0 currently); however for a long time now, in all of the beta and RC versions, at least we were able to parse script syntax trees. Unfortunately the Roslyn team decided that for the stable release almost all of the public code paths that have anything to do with scripting in any way will throw NotSupportedException.

Therefore even a simple piece of code like this, will error out with Roslyn 1.0.0:

var options = new CSharpParseOptions(kind: SourceCodeKind.Script);  

However, this will not stop us, after all the title of this post says “hacking”. Reflection FTW. We can grab the existing compilation and weasel our SourceCodeKind.Script via reflection. Then we shuld re-parse all of the existing syntax trees that contained script code, and add them back to the compilation object, replacing the old ones.

The code to this point is shown below:

public class ScriptCompileModule : ICompileModule  
{  
public void BeforeCompile(BeforeCompileContext context)  
{  
var options = new CSharpParseOptions();

//note: Roslyn public API is immutable  
//but with reflection this doesn't apply anymore  
options.GetType().GetProperty("Kind").SetValue(options, SourceCodeKind.Script);

var tree = context.Compilation.SyntaxTrees.FirstOrDefault(t => t.FilePath.EndsWith("csx"));  
if (tree != null)  
{  
var code = tree.GetText();  
var newTree = SyntaxFactory.ParseSyntaxTree(code, options: options);

context.Compilation = context.Compilation.ReplaceSyntaxTree(tree, newTree);  
//todo - actually run script code  
}  
}

public void AfterCompile(AfterCompileContext context)  
{  
}  
}  

In the above snippet we keep it simple - we just pick the first encountered CSX file to execute as C# script. Of course we could debate on how to implement a reasonable logic to handle multiple files here, and there are many ways to solve this, but for the purpose of a demo/hack a single file is good enough illustration.

One note here - DNX projects by default will only compile .CS files. In order to force them to take .CSX into consideration when gathering source files to be used in the compilation, you need to modify your project.json (below is a snippet from the file, the rest of it omitted for brevity):

{  
"compile": [ "*\*/\*.csx" ],  
"version": "1.0.0-*",  
"description": "TestDnxScript Console Application",  
"authors": [ "filip" ]  
}  

Ok so once we have replaced the old syntax trees in context.Compilation with the same ones - but parsed against scripted C#, we can invoke the script code. To do this, we need 4 steps:

1. emit the compilation into memory stream
2. load the assembly from the memory stream
3. Grab a type called Script with reflection
4. Instantiate that type

This is shown below in the full listing of ScriptCompileModule:

public class ScriptCompileModule : ICompileModule  
{  
public void BeforeCompile(BeforeCompileContext context)  
{  
var options = new CSharpParseOptions();  
options.GetType().GetProperty("Kind").SetValue(options, SourceCodeKind.Script);

var tree = context.Compilation.SyntaxTrees.FirstOrDefault(t => t.FilePath.EndsWith("csx"));  
if (tree != null)  
{  
var code = tree.GetText();  
var newTree = SyntaxFactory.ParseSyntaxTree(code, options: options);

context.Compilation = context.Compilation.ReplaceSyntaxTree(tree, newTree);  
using (var stream = new MemoryStream())  
{  
var result = context.Compilation.Emit(stream);

if (result.Success)  
{  
var assembly = AppDomain.CurrentDomain.Load(stream.ToArray());  
var type = assembly.GetType("Script");

//at this moment the script will execute  
var f = Activator.CreateInstance(type);  
}  
else  
{  
foreach (var diagnostic in result.Diagnostics)  
{  
Console.WriteLine(diagnostic.ToString());  
}  
}  
}

Environment.Exit(0);  
}  
}

public void AfterCompile(AfterCompileContext context)  
{  
}  
}  

All of the loose C# statements that you might have in your script, will be invoked when you run the constructor on the Script class. After that we can shut down the whole shebang using Environment.Exit(0).

Of course you could a lot more here, like handle exceptions and so on.

Testing it out 🔗

Let’s imagine a sample C# script:

using System;  
using Newtonsoft.Json;

void SayHi()  
{  
Console.WriteLine("Hi");  
}

public class Foo  
{  
public string Bar { get; set; }  
}

SayHi();  
Console.WriteLine("Hello DNX script");

var x = new Foo { Bar = "test me" };  
Console.WriteLine(JsonConvert.SerializeObject(x));  

We can call the file whatever we want - i.e. Script.csx, remember our module will only pick the first file. We put this file in the DNX project, in the place of the typical Program.cs.

Notice that because all of the references are controlled by the project.json, the script code will have access to all dependencies defined there. For example the above script uses JSON.NET, and that code will run just fine as long as the project.json references it. This is great because you can rely on very powerful DNX library management process, and rely on all of that inside your script. On the contrary, for example in the scriptcs project, we have to do all of this assembly heavy lifting manually ourselves.

In my case, the dependencies node in project.json was:

"dependencies": {  
"Microsoft.Framework.Runtime.Roslyn.Abstractions": "1.0.0-beta6",  
"Newtonsoft.Json": "7.0.1"  
},  

Now, running this DNX project produces the following result:

dnx . run  
Hi  
Hello DNX script  
{"Bar":"test me"}  

Of course all of that is a big hack, and things will get much easier once proper scripting ships with Roslyn, but nevertheless, I think it is an interesting insight into DNX pre processing.