Make your .NET application extendible using DLR : IronPython

There are different ways to define this – extensibility in applications, adding macro capability to your applications, providing add-in like facility in .NET applications. For me, all of this mean the same – ability to modify your application (data or the UI) at runtime by means of scripts. Like I mentioned in the previous post, I wanted to dig into DLR or Dynamic Language Runtime which is basically a framework that could be used to provide dynamic languages support on the .NET platform. Even though I am not much interested in giving up C# for dynamic languages like Python or Ruby, I was very much keen to add facility to make my applications extensible using these languages (to tell you the truth, I am more comfortable with JavaScript or Groovy than any of these).

Anyway, I worked on an application which displays a datagrid and one could modify that the data being displayed using Python script at runtime. The way I did it was to host IronPython inside C# using the DLR framework.

The article would be a detailed step by step procedure on each and every piece of code that has been written so that it helps the readers get familiar with the following

1. Using WPF Datagrid and databinding. Also shows Dependency Properties.
2. How to refresh WPF Datagrid items.
3. Hosting IronPython inside C# applications.
4. Passing data to and fro between C# and IronPython script.

Shown below are the screenshots for the application that we will be looking through.

Application when started	First time execution	Subsequent executions

Figure 1 shows the application when it first started. The data that appears in the grid has been generated on the fly for simplicity. The application provides two buttons whose titles are descriptive enough – one for running the script that modifies the data and the other for launching the script in notepad.

Figure 2 is when first the script is applied. Notice the amount of time it takes (2.1 seconds). It looks slow and the majority of the time, I guess, is consumed for initialization of the IronPython engine in the DLR. The python script that was executed looks at the age of the person and classifies them accordingly. Notice that the persons over 60 have their address as “senior citizen at work”. (Address is a pretty lame field for description, anyway I am lazy now to change it).

Figure 3 shows two subsequent executions which has the same effect as earlier but the script has been modified to change the description from “senior citizen at work” to “seniors”. This was to show that the script could be modified outside the application and yet it would be faster since the engine is already initialized (the way I wrote the evaluation). It takes .18 seconds to operate on 50 K tuples which I personally think is very good.

I hope you now have a clear picture on what we are trying to achieve. To further enhance your understanding, let us look at the python script. Note that the application was designed in such a way that it passes the list of persons using the variable “Context” and expects the modified list of person in “Output”.

   1: import clr

   2: clr.AddReference('System')

   3: from System import *

   4: from System.Diagnostics import Debug

   5:  

   6: for person in Context:

   7:   age = person.Age

   8:   if age >= 60:

   9:      person.Address = "Seniors"  

  10:   elif age > 25 and age < 60:

  11:      person.Address = "Adult"

  12:   elif age <=25 and age > 18:

  13:      person.Address = "Growing"

  14:   else:

  15:      person.Address = "Minor"

  16: Output = Context

The first 4 lines brings in the .NET assemblies and I import System.Diagnostics.Debug class just to show how to import .NET classes into IronPython code. The python script is pretty much readable (and you would not believe how long it took me to figure out how to write that).

Anyway lets get started by creating a new WPF project. Add the references to IronPython and DLR assemblies. Also add reference to WPF Toolkit assembly. The references should look like shown below.

XAML for the application.

   1: <Window x:Class="DynamicGrid.Window1"

   2:     xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"

   3:     xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"

   4:     Title="Click On Apply Filter to apply the Python Filter" Height="600" Width="300" 

   5:         WindowStyle="ThreeDBorderWindow"

   6:         xmlns:my="http://schemas.microsoft.com/wpf/2008/toolkit">

   7:     <Grid>

   8:         <Grid.RowDefinitions>

   9:             <RowDefinition Height="*"/>

  10:             <RowDefinition Height="35"/>

  11:             <RowDefinition Height="70"/>

  12:         </Grid.RowDefinitions>

  13:         <my:DataGrid Name="myGrid" ItemsSource="{Binding DataSource}" CanUserAddRows="False" CanUserResizeColumns="False">

  14:         </my:DataGrid>

  15:         <StackPanel Orientation="Horizontal"  Grid.Row="1">

  16:             <Button Name="ApplyRubyFilter" Content="Apply Python Filter" Click="ApplyRubyFilter_Click" Margin="5"

  17:                />

  18:             <Button Name="OpenScript" Content="Edit Python Script/Filter" Click="Edit_Script_Filter" Margin="5"/>

  19:         </StackPanel>

  20:         <ScrollViewer Grid.Row="2">

  21:             <TextBlock Name="Status" Text=""/>

  22:         </ScrollViewer>

  23:     </Grid>

  24: </Window>

Line 6 imports WPF toolkit which can then be used within the xaml file. Line 13/14 shows WPF datagrid being added to the first row of the container grid. Notice that it is bound to DataSource property. So it looks for this property in the datacontext of itself or the parent containers. For this, I added a dependency property with name “DataSource” in the code behind of the current window( which is Window1.xaml.cs. Shown below is the code for the property as well as the constructor.

   1: public IEnumerable DataSource

   2:        {

   3:            get { return (IEnumerable)GetValue(DataSourceProperty); }

   4:            set { SetValue(DataSourceProperty, value); }

   5:        }

   6:  

   7:        // Using a DependencyProperty as the backing store for DataSource.  This enables animation, styling, binding, etc...

   8:        public static readonly DependencyProperty DataSourceProperty =

   9:            DependencyProperty.Register("DataSource", typeof(IEnumerable), typeof(Window1), new UIPropertyMetadata(null));

  10:  

  11:        private IEnumerable<Person> _tempData;

  12:  

  13:        public Window1()

  14:        {

  15:            InitializeComponent();

  16:            this.DataContext = this;

  17:            _tempData = GenerateData();

  18:            myGrid.AutoGenerateColumns = true;

  19:            UpdateDataSource(_tempData);

  20:        }

For the dependency property, I usually use Visual Studio code snippet (Ctrl+K,X, Then NetFX30).

Now talking about the constructor, line 16 sets the DataContext of the window to itself. So the grid would finally find DataSource property in here. Note that you could set DataContext within XAML as shown below.

   1: <Window x:Class="DynamicGrid.Window1"

   2:     xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"

   3:     xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"

   4:     Title="Click On Apply Filter to apply the Python Filter" Height="600" Width="300" 

   5:         WindowStyle="ThreeDBorderWindow"

   6:         xmlns:my="http://schemas.microsoft.com/wpf/2008/toolkit"

   7:         Name="myWindow" DataContext="{Binding ElementName=myWindow}">

Line 7 in the above code snippet would be of interest to set DataContext to the current Window class in XAML.

Anyway back to what the constructor was doing. It invokes GenerateData() which generates a list of Person objects and assigns it to _tempData. This variable is what we would be using to pass to the Python script. Code for the GenerateDatat() is shown below.

   1: public IEnumerable<Person> GenerateData()

   2:         {

   3:             var list = new List<Person>();

   4:             Random rand = new Random();

   5:             for (int i = 0; i < 50000; i++)

   6:             {

   7:                 var person = new Person()

   8:                 {

   9:                     Name = "Name " + i,

  10:                     Age = rand.Next(100),

  11:                     Address = string.Format("Address {0}", i * 10)

  12:                 };

  13:                 list.Add(person);

  14:             }

  15:             return list.AsEnumerable();

  16:         }

Finally the constructor invokes UpdateDataSource() passing the data that the grid has to bind to. As you can expect the UpdateDataSource method just sets the value on the DataSource and performs a refresh on the datagrid items. Since DataSource is a dependency property, the grid automatically loads the datasource.

   1: private void UpdateDataSource(IEnumerable<Person> source)

   2:         {

   3:             DataSource = source;

   4:             myGrid.Items.Refresh(); //refresh the grid manually.

   5:         }

Now when the user clicks the “apply filter button”, the python script has to be executed. Code that does just that.

   1: private void ApplyRubyFilter_Click(object sender, RoutedEventArgs e)

   2:         {

   3:             var temp = ApplyIronPyFilter();

   4:             Debug.WriteLine(temp.GetType().Name);

   5:             if (temp != null)

   6:                 UpdateDataSource(temp.ToList());

   7:         }

For now, lets just say ApplyIronPyFilter does everything that it needs to execute the script on the _tempData and returns a new list of items. Ignore the line 4 which I added to verify what was being returned from the Python script. And if there is something in what is being returned, I update the datasource with that list.

Before we look into what ApplyIronPyFilter does, let us see the wrapper for the IronPython engine that I made to make things easy.

   1: using System;

   2: using System.Collections.Generic;

   3: using System.Diagnostics;

   4: using IronPython;

   5: using IronPython.Hosting;

   6: using Microsoft.Scripting.Hosting;

   7:  

   8: namespace DynamicGrid

   9: {

  10:     public class PythonEngine

  11:     {

  12:         private ScriptEngine engine;

  13:         private ScriptScope scope;

  14:  

  15:         private PythonEngine()

  16:         {

  17:             engine = Python.CreateEngine(new Dictionary<string, object>() { { "DivisionOptions", PythonDivisionOptions.New } });

  18:             scope = engine.CreateScope();

  19:         }

  20:  

  21:         private static PythonEngine _pe;

  22:  

  23:         public static PythonEngine Engine

  24:         {

  25:             get

  26:             {

  27:                 if (_pe == null)

  28:                     _pe = new PythonEngine();

  29:                 return _pe;

  30:             }

  31:         }

  32:  

  33:         public void SetVariable<T>(string name, T value)

  34:         {

  35:             scope.SetVariable(name, value);

  36:         }

  37:  

  38:         public T Evaluate<T>(T Context, string scriptFile)

  39:         {

  40:             this.SetVariable<T>("Context", Context);

  41:             try

  42:             {

  43:                 var source = engine.CreateScriptSourceFromFile(scriptFile);

  44:                 source.Execute(scope);

  45:             }

  46:             catch (Exception ex)

  47:             {

  48:                 Debug.WriteLine(ex);

  49:             }

  50:             //execute the script

  51:             //return value from "Output"

  52:             T value = default(T);

  53:             if (scope.TryGetVariable<T>("Output", out value))

  54:                 return (T)value;

  55:             return default(T);

  56:         }

  57:     }

  58: }

I am not going into details for each and every line of code that I am writing (may be sometime later).  But I would like to talk a little about Evaluate method which I think is important. The first parameter it takes would be the input for the text and as you can see in Line 40, it is being set as a Python variable with name “Context”. Similarly line 53 attempts to read a variable “Output” using TryGetVariable method on the “scope” object which was created in the constructor of the PythonEngine class. Lines 41-49 shows on how to load the script from a file and execute it. I would recommend you to look at the whole 58 lines of code for the PythonEngine since it is simple and yet important. At least you are required to look at the constructor and the SetVariable,Evaluate methods.

Finally, we come to the ApplyIronPyFilter method which as you can expect would use the PythonEngine#Evaluate() method passing the context as _tempData and path to the python script file that would be executed. The method then captures the return value from the Evaluate method and returns it back. Look at the source code.

   1: private IEnumerable<Person> ApplyIronPyFilter()

   2:         {

   3:             var sw = Stopwatch.StartNew();

   4:             try

   5:             {

   6:                 var engine = PythonEngine.Engine;

   7:                 return engine.Evaluate<IEnumerable<Person>>(_tempData, @"test.py") ;

   8:             }

   9:             finally

  10:             {

  11:                 sw.Stop();

  12:                 Status.Text += ("Time taken for Python Script : " + sw.ElapsedMilliseconds + " milliseconds \n");

  13:             }

  14:         }

Also shown is code on how to time the execution which is set to the Text property of the “Status” which is a textblock (look at the XAML). Looks like we are done with the most important pieces of source code that makes this application extendible.

I personally think it would be really cool if we have a IronGroovy kind of runtime. Having worked on Groovy for fairly complex applications (during my Masters), I am a huge fan of Groovy.

I totally had fun doing research on making this application work. I hope you find all the information you need to get started embedding DLR into your applications. Please let me know if you have any specific questions.

I got the book “Data Driven Services with Silverlight 2 by John Papa” which looks like book. Anyway I expected it to be dedicated to WCF driven data applications in Silverlight but that should be fine. I think the book would be very useful to understand more about data-driven silverlight applications, just that the source of data differs. I am actually looking for a good WCF book (should be comparable in good-ness with Programming WPF by Chris Sells). If you know of any good one, drop a comment.