Dynamic Code Generation using CodeDOM

 

http://www.codeproject.com/KB/cs/DynamicCodeGeneration.aspx

Dynamic Code Generation

Why generate code?

As the name suggests, dynamic code generation is the technique to generate code at run-time. We write code which would generate code when run. As simplistic the definition seems, it is indeed one of the most powerful techniques in repertoire of software developers. It is widely used in ORM - Object Relational Mapping - in fact if you try googling for samples on dynamic code generation, quite a few of the articles you would find would have ORM as sample. Besides dynamically mapping objects to scalar database entities, code generation can also be used to provide a plug-in framework in our applications which enables users to enhance features in the basic application. Code generation is important instrument for programming methodologies like Generative Programming & Aspect Oriented Programming.

We can also use this to let the computer do the boring task of writing repetitive code. Almost all real word applications talk to some database, for example. The code required to connect to a database, read data, manipulate data, etc is more or less the same even when the underlying database and database engines are different. How many of us have not had "deja vu" moments when we write classes like "Employee" with a "Save" method which reads all the properties of the class and passes to some database class to persist... And how many us have not wished that this dull task is somehow done auto-magically. Patterns, patterns everywhere - that is one of the basic ideas behind code generation. Most of the tasks we perform on a daily basis have certain patterns and can be categorized. Code generation techniques often build upon this idea and separate the "difference" between "similar" things, extract the "common stuff", and provide a way to write similar stuff based on common stuff.

To borrow from Pragmatic Programmers' wisdom, basically, any time you have code that can be derived from metadata (a database schema, xsd, grammar description) you should write a tool that generates the code based on the metadata. Your overall maintenance burden will be lower, because you can just change the metadata as needed and regenerate the code. It may appear that writing such a tool would be difficult or would require programming skills which are not so common. But this is not correct. As with everything else, code generation has never been easy than on .Net platform.

About our sample

Download DynamicCodeGeneration_src.zip - 92.2 KB

So much so for preamble, now it is time to take a sneak peek at 2 of the code generation techniques in .Net world. Ultimate goal of code generation is to get it running isn't it? CodeDom lets you create source code. That has to be then then dynamically compiled into an assembly so that it can be run in CLR. Reflection.Emit shortcuts this process by letting you write code which directly "emits" an assembly (and potentially an incorrect one - more on this later).

My goal in writing this article is to present the basics of these two techniques with a sample which uses both to generate same assembly. I guess it would provide a nice contrast of these 2 techniques. We would do so in 2-parts. This is the first part which would use CodeDOM to generate code. Second part using Relection.Emit would follow soon.

Our sample assembly would have 2 classes. One would be derive from other. We would apply some attributes all through (to assembly, to class, to method). The derived class would have a method override of the base class.

Though there are numerous articles available on this topic, the sample I described above covers what I could not find in a sample and had to figure out. (Not rocket science, but I would feel good if I save someone few hours figuring these out).

On a final note - our goal is to generate a vaild .Net assembly using both CodeDOM & Reflection.Emit. We are targeting .Net framework 2.0. APIs could be slightly different for 1.1, but I have not checked the same.

Part I of II - CodeDOM

CodeDOM (or Code Document Object Model) is a mechanism provided by .Net framework which lets us generate source code in multiple languages using a single model. We create code graphs and use the methods provided for code dom to generate code in language of our choice. Then we can use dynamic code compilation classes (also provided by code dom) to generate assemblies which can then be loaded and used dynamically. The .NET Framework includes code generators and code compilers forC#, JScript, and Visual Basic. To read more about Code DOM check msdn here.

It may be redundant to say that, but before we attempt code generation we should be absolutely clear what exactly we want to generate. Writing the our desired code as a rough draft may be very helpful in our early ventures in code generation because it is all to easy to miss something otherwise.

Anatomy of CodeDOM

Let us assume that we now know our "target" code to be generated. All our efforts would be concentrated on producing our desired code using the API provided by CodeDOM. Along the way we would learn few nitty-gritties of this process. Here is the structure of solution we would be producing as we go -

1. DynamicCodeGeneration.Base - This is the base class I talked about. Does nothing much. Our generated code will refer to this assembly and would have a class derived from this base class.

2. DynamicCodeGeneration.CodeGenerator - This is our sweet little code generator which would provide us the UI and logic to generate the desired code using CodeDOM or IL.

3. DynamicCodeGeneration.CustomAttributes - This project defines few custom attributes which would apply to our target assembly/class/method while generating the code.

I found application of custom attributes little tricky so I thought I should include this in my sample.

Lets get going!

First things first!

Let us first write our base class.

Collapse | Copy Code

using System;
using System.Collections.Generic;
using System.Text;

namespace DynamicCodeGeneration.Base
{
    /// <summary>
    /// Our base class
    /// </summary>
    public class Base
    {
        /// <summary>
        /// Base method which does nothing and would be overridden by derived class
        /// </summary>
        public virtual void Method()
        {
        }
    }
} 

Simple :)

Custom Attributes

I have written 3 custom attributes, names are self-explanatory -

1. AssemblyLevelAttribute - Sets AssemblyAuthorName & GenerationMode on the assembly
2. ClassLevelAttribute - Sets the ClassAuthorName on the class
3. MethodLevelAttribute - Sets the MethodAuthorName & ComplexityLevel (how complex is the method - isnt it great if by reading an attribtue we know before hand if the method is complex or not :)

These attributes would be applied to our generated assembly at appropriate level.

CodeGenerator

I am just being lazy here in design. I have written an interface IGenerator which defines a method called "GenerateAssembly". We would implement this interface twice - one for CodeDOM and second time for IL. The presentation layer is the UI class (the win form which lets user input values for some of the parameters of custom attributes). It has a method which generates assembly in the desired fashion using factory pattern. Simple to follow architecture (if we call this architecture :).

Here is the IGenerator interface we would implement in a while.

Let us implement this interface for CodeDOM generation.

Class name I have chosen is CSharpGenerator. We would need the following references -

Collapse | Copy Code

using System;
using System.Collections.Generic;
using System.Text;
using System.CodeDom;
using Microsoft.CSharp;
using System.CodeDom.Compiler; 
using System.IO; 

Notice the ones in bold-italics - CodeDom & CodeDom Compiler provide necassary API to build code graph and build it. Microsoft.CSharp provides the generator which use the code graph to generate code in C#.

Let us come straight to the centre of action - the GenerateAssembly method. Using code dom it is possible to generate code from various code dom "constructs" -

1. CompileUnit - Equivalent to Assembly - top level construct
2. NameSpace
3. Type - Equivalent to class
4. Expression
5. Statement

Since we want to apply some attributes to assembly, we would use the first method. This would require us to create the codegraph having a code compile unit as the root and other constructs (namespace, type, expression, etc) would be nested withing the unit.

A. CodeCompileUnit

A CodeCompileUnit provides a container for a CodeDOM program graph. We need to specify any referenced assemblies by adding to ReferencedAssemblies. We also need to provide any attribtues by (creating and) adding to AssemblyCustomAttributes.

For adding any custom attribute, we need to first create it. CodeAttributeDeclaration class takes charge of applying an attribute. It takes a CodeTypeReference (which is nothing but reference to the class which defines the attribute. Needs to be fully qualified.) and an array of CodeAttributeArguments which would be passed to the constructor of the attribute class. They have to be in the same order as expected by the constructor of the custom attribute.

So the process in steps would be -

1. Define a new code compile unit
2. Create an CodeAttributeArgument array of size as the number of parameters in the Attribute constructor
3. Define each argument using constructor of CodeAttributeArgument which takes a CodeExpression. CodeDOM provides several types of expressions.
4. Create a CoedAttributeDeclaration which would take CodeTypeReference of the attribute class and the array of code attribute arguments
5. Add this CodeAttributeDeclaration to code compile unit defined in #1.
6. Add all referenced assemblies to code compile unit

Collapse | Copy Code

    #region Unit

    CodeCompileUnit unit = new CodeCompileUnit();
    CodeAttributeArgument[] arguments = new CodeAttributeArgument[2];
    arguments[0] = new CodeAttributeArgument(
        new CodePrimitiveExpression(assemblyAuthorName));//Create parameter for attribute 

    arguments[1] = new CodeAttributeArgument(
        new CodeSnippetExpression("DynamicCodeGeneration.CustomAttributes.GenerationMode.CodeDOM"));
    CodeAttributeDeclaration assemblyLevelAttribute = new CodeAttributeDeclaration(
        new CodeTypeReference("DynamicCodeGeneration.CustomAttributes.AssemblyLevelAttribute"),
 arguments);//Create attribute to be added to assembly
    unit.AssemblyCustomAttributes.Add(assemblyLevelAttribute);
    unit.ReferencedAssemblies.Add("DynamicCodeGeneration.Base.dll");
    unit.ReferencedAssemblies.Add("System.dll");
    unit.ReferencedAssemblies.Add("DynamicCodeGeneration.CustomAttributes.dll");
    unit.ReferencedAssemblies.Add("Microsoft.Crm.SdkTypeProxy.dll");

    #endregion

B. Namespace

Next in hierarchy is the target namespace. This is defined as follows -

Steps -

1. Define a new namespace using CodeNamespace class. Takes the namespace names as the constructor parameter.
2. Add the namespace to Namespaces collection of code compile unit defined in section #A.
3. Add required references by adding CodeNamespaceImport statements to Imports collection of code compile unit.

Collapse | Copy Code

    #region Namespace

    CodeNamespace customEntityRoot = new CodeNamespace("DerivedRoot");//Create a namespace
    unit.Namespaces.Add(customEntityRoot);

    customEntityRoot.Imports.Add(new CodeNamespaceImport("System"));//Add references
    customEntityRoot.Imports.Add(new CodeNamespaceImport("DynamicCodeGeneration.Base"));//Add references
    customEntityRoot.Imports.Add(new CodeNamespaceImport("DynamicCodeGeneration.CustomAttributes"));//Add references

    #endregion

C. Class (Type)

After defining namespace next logical task is to define our derived class. We would also apply the class level custom attribute to this class. This class is derived from the base class defiend above.

Steps -

1. Create a new CodeTypeDeclaration. Its constructor takes the name of the type (classes are types).
2. Add this new created type to namespace defined in section #B.
3. Now we need to add a constructor to our class. For this -
   1. Create a new CodeConstructor.
   2. Set the attribtute(s) (scope - public/private etc)
   3. Add this code constructor to members collection of type defined above.
4. Now we would apply the class level attribute to this type. Steps are similar to the ones we performed for defined the assembly level attribute in section #A.
5. Finally, since this is a derived class, we need to add the base type which desribes the class from which this type (class) is derived. For this we add a new CodeTypeReference to the base class to the BaseTypes collection of the derived type (class).

Collapse | Copy Code

#region Class

CodeTypeDeclaration derived = new CodeTypeDeclaration("Derived");//Create class
customEntityRoot.Types.Add(derived);//Add the class to namespace defined above

CodeConstructor derivedClassConstructor = new CodeConstructor();//Create constructor
derivedClassConstructor.Attributes = MemberAttributes.Public;
derived.Members.Add(derivedClassConstructor);//Add constructor to class

CodeAttributeArgument argument = new CodeAttributeArgument(
    new CodePrimitiveExpression(classAuthorName));

CodeAttributeDeclaration classLevelAttribute = 
    new CodeAttributeDeclaration(
        new CodeTypeReference("DynamicCodeGeneration.CustomAttributes.ClassLevelAttribute"), 
            argument);//Create attribute to be added to class
derived.CustomAttributes.Add(classLevelAttribute);

derived.BaseTypes.Add(new CodeTypeReference("Base"));

#endregion 

D. Method

What use is a blank class? We now would add a method to our type defined above which would override the method in base class.

Steps -

1. Define a new CodeMemberMethod.
2. Add required attributes (this is not the same as meta data attribute - but defines the "prefixes" applied to the method definition. Since we need our method to be public and want it to override base class's method, we would add MemberAttributes.Public & MemberAttributes.Override to CodeMemberMethod defined above. This is a bit field and need a bitwise OR for adding multiple attributes.
3. We can also add CodeCommentStatements to comments collection of the method.
4. Set the Name of the method
5. Define the return "type". Ours is void.
6. Define the method level attribute as we defined above in section #B and section #C.
7. Now that we have created the skeleton of the method we need to add muscle to it. For this we can add multiple statements to Statemetns collection of the CodeMemberMethod. Here we have added a new CodeSnippetStatement (this is literelly sent to output stream while codegeneration).
8. Finally we need to add this CodeMemberMethod to the Members collection of type defined in secion #C.

Collapse | Copy Code

 #region Method

CodeMemberMethod derivedMethod = new CodeMemberMethod();
derivedMethod.Attributes = MemberAttributes.Public | MemberAttributes.Override;
 //Make this method an override of base class's method
derivedMethod.Comments.Add(new CodeCommentStatement(new CodeComment("TestComment")));
derivedMethod.Name = "Method";
derivedMethod.ReturnType = new CodeTypeReference(typeof(void));

arguments = new CodeAttributeArgument[2];
arguments[0] = new CodeAttributeArgument(
    new CodeSnippetExpression("ComplexityLevel.SuperComplex"));//Create parameter for attribute
arguments[1] = new CodeAttributeArgument(
    new CodePrimitiveExpression(methodAuthorName));
CodeAttributeDeclaration methodLevelAttribute = new CodeAttributeDeclaration(
    new CodeTypeReference("DynamicCodeGeneration.CustomAttributes.MethodLevelAttribute"), 
        arguments);//Create attribute to be added to method

derivedMethod.CustomAttributes.Add(methodLevelAttribute);//Add attribute to method

CodeSnippetStatement code = new CodeSnippetStatement("base.Method();");
derivedMethod.Statements.Add(code);
derived.Members.Add(derivedMethod);//Add method to the class

#endregion

E. Generation

Now our code graph is ready and we can generate code in language of our choice. Thats the beauty of codedom. Even though we are writing code generation code in C# we can use the same code graph to generate code in VB.NET or other supported languages. Let us generate code in C# for now.

Steps -

1. Create C# code provider. This is going provide us the generate which would use our code graph to generate code in C Sharp.
2. Create code generator from code provider.
3. We need a stream where code generator would output the code. We are using string writer and string builder here.
4. Create object of CodeGeneratorOptions. This is used by generato for settings like bracing style, etc. Bracing Style "C" keeps the braces on next line. I like it that way :)
5. Use one of the 5 methods to generate code. As I explained earlier we can generate code from various constructs. In this example we would use the root - code compile unit to generate code and GenerateCodeFromCompileUnit would be the method used.
6. Build the generated code.

Collapse | Copy Code

CSharpCodeProvider codeProvider = new CSharpCodeProvider();
ICodeGenerator codeGenerator = codeProvider.CreateGenerator();
StringBuilder generatedCode = new StringBuilder();
StringWriter codeWriter = new StringWriter(generatedCode);

CodeGeneratorOptions options = new CodeGeneratorOptions();
options.BracingStyle = "C";
    //Keep the braces on the line following the statement or 
    //declaration that they are associated with
codeGenerator.GenerateCodeFromCompileUnit(unit, codeWriter, options);

this.Code = generatedCode.ToString();

return BuildGeneratedCode();

G. Building the code

Last step is of-course building the generated code into a .Net assembly.

Steps -

1. Create provider of appropriate langauge. C# here.
2. Create compiler.
3. Create compiler parameters and add referenced assemblies to ReferencedAssemblies collection.
4. You have the option to generate assembly in memory and execute directly from memory. For now we are going to write the generate the assembly on disk.
5. Use CompileAssemblyFromSource method to build code.
6. Compiler results will tell if the build was successful or not.

Collapse | Copy Code

CSharpCodeProvider codeProvider = new CSharpCodeProvider();
ICodeCompiler codeCompiler = codeProvider.CreateCompiler();

CompilerParameters parameters = new CompilerParameters();
parameters.ReferencedAssemblies.Add("DynamicCodeGeneration.Base.dll");
parameters.ReferencedAssemblies.Add("System.dll");
parameters.ReferencedAssemblies.Add("DynamicCodeGeneration.CustomAttributes.dll");
parameters.GenerateInMemory = false;

CompilerResults results = codeCompiler.CompileAssemblyFromSource(parameters, this.Code);
if (results.Errors.HasErrors)
{
    string errorMessage = "";
    errorMessage = results.Errors.Count.ToString() + " Errors:";
    for (int x = 0; x < results.Errors.Count; x++)
    {
        errorMessage = errorMessage + "\r\nLine: " + 
            results.Errors[x].Line.ToString() + " - " + results.Errors[x].ErrorText;
    }
    return errorMessage;
}
return results.PathToAssembly; 

This concludes our endeavor. We now have the assembly which was built using the code we generated. The generated code will look like -

Collapse | Copy Code

//------------------------------------------------------------------------------
// 
//     This code was generated by a tool.
//     Runtime Version:2.0.50727.42
//
//     Changes to this file may cause incorrect behavior and will be lost if
//     the code is regenerated.
// 
//------------------------------------------------------------------------------

[assembly: DynamicCodeGeneration.CustomAttributes.AssemblyLevelAttribute("Neo",
 DynamicCodeGeneration.CustomAttributes.GenerationMode.CodeDOM)]

namespace DerivedRoot
{
    using System;
    using DynamicCodeGeneration.Base;
    using DynamicCodeGeneration.CustomAttributes;
    
    
    [DynamicCodeGeneration.CustomAttributes.ClassLevelAttribute("Morpheus")]
    public class Derived : Base
    {
        
        public Derived()
        {
        }
        
        // TestComment
        [DynamicCodeGeneration.CustomAttributes.MethodLevelAttribute(
            ComplexityLevel.SuperComplex, "Trinity")]
        public override void Method()
        {
            base.Method();
        }
    }
}

Assembly would be generated in temp location.

This concludes the first part of this article in which we saw how we can leverage CodeDOM API to generate code in language of our choice with the same code graph.

References

Part I

1. Using the CodeDOM by Nick Harrison

2. Microsoft .NET CodeDom Technology - Part 2 by Brian J. Korzeniowski
3. Dynamically Executing Code in .Net by Rick Strahl
4. Compiling with CodeDOM by Gustavo Bonansea

Further Reading

1. CodeDOM patterns byOman van Kloeten
2. Tonnes on articles at Code Generation Network
3. Introduction to Creating Dynamic Types with Reflection.Emit
4. EmitHelper

About Proteans Software Solutions

Proteans is an outsourcing company focusing on software product development and business application development on Microsoft Technology Platform. Proteans partners with Independent Software Vendors (ISVs), System Integrators and IT teams of businesses to develop software products. Our technology focus and significant experience in software product development - designing, building, and releasing world-class, robust and scalable software products help us to reduce time-to-market, cut costs, reduce business risk and improve overall business results for our customers. Proteans expertises in development using Microsoft .Net technologies.

Disclaimer

While writing these articles I have freely borrowed from several resources available online (freely). Though I have made every attempt to not to leave any article in my references list, I might have inadvertently missed some citations. Please do bring to my notice such omissions and I would be glad to include the same in my article references.

License

This article, along with any associated source code and files, is licensed underThe Microsoft Public License (Ms-PL)