元数据编程实战_使用Emit运行时生成Protobuf编码类

      protobuf是google的一种序列化对象的编码方式。相比xml和json的序列化方式,protobuf序列化的结果更小，而且序列化的速度也更快。
本文简单介绍写如果通过Emit来在运行时动态的生成对数据对象的protebuf编码解码类。 通过本实例展示下元数据编程的能力。
关于protebuf的编码原理可以参考这里http://code.google.com/intl/zh-CN/apis/protocolbuffers/docs/encoding.html

，本文只展示对简单的int32进行varint方式编码和string的编码,。在此基础上可以很容易的实现全部的protobuf的编码逻辑。首先定义两个用于测试的实体类。

　　

       [ProtoContract]
        public class TestModel1
        {
            [ProtoMember(1)]
            public int UserId { get; set; }
            [ProtoMember(2)]
            public string Password { get; set; }

            [ProtoMember(3)]
            public TestModel2 Model2 { get; set; }

            [ProtoMember(4)]
            public List<TestModel2> Models { get; set; }

        }
        [ProtoContract]
        public class TestModel2
        {
            [ProtoMember(1)]
            public int Id { get; set; }
            [ProtoMember(2)]
            public string Name { get; set; }
        }

这里同时定义了两个attribute用于表明类支持序列化，和那些属性参与序列化，然后定义一个编码解码的接口

    public interface ICodec<T>
    {
        byte[] Encode(T obj);
        T Decode(byte[] data);
    }

我们将在运行时为每个用到的Model生成一个实现ICodec接口的编码类。比如对TestModel1将成才如下签名的一个类

class TestModel1Codec:ICodec<TestModel1> {....}

首先对我们先看看一个手工写的对TestModel1和TestModel2进行编码的Codec类是如何实现的

 class TestModel1Codec : ICodec<TestModel1>
        {


            public byte[] Encode(TestModel1 obj)
            {
                ProtoStream stream = new ProtoStream();
                if (obj.UserId != 0)
                {
                    stream.WriteTag(new Tag(1, WireType.Varint));
                    stream.WriteInt32(obj.UserId);
                }
                if (obj.Password != null)
                {
                    stream.WriteTag(new Tag(2, WireType.LengthDelimited));
                    stream.WriteString(obj.Password);
                }
                if (obj.Model2 != null)
                {
                    stream.WriteTag(new Tag(3, WireType.LengthDelimited));
                    stream.WriteBytes(Codec<TestModel2>.Encode(obj.Model2));
                }
                if (obj.Models != null)
                {
                    for (int i = 0; i < obj.Models.Count; i++ )
                    {
                        stream.WriteTag(new Tag(4, WireType.LengthDelimited));
                        stream.WriteBytes(Codec<TestModel2>.Encode(obj.Models[i]));
                    }

                }

                return stream.ToArray();
            }
            public TestModel1 Decode(byte[] data)
            {
                TestModel1 result = new TestModel1();
                ProtoStream stream = new ProtoStream(data);

                while (true)
                {
                    Tag tag = stream.ReadTag();
                    if (tag.Number == -1)
                    {
                        break;
                    }
                    if (tag.Number == 1)
                    {
                        result.UserId = stream.ReadInt32();
                    }
                    if (tag.Number == 2)
                    {
                        result.Password = stream.ReadString();
                    }
                    if (tag.Number == 3)
                    {
                        result.Model2 = Codec<TestModel2>.Decode(stream.ReadBytes());
                    }
                    if (tag.Number == 4)
                    {
                        if (result.Models == null)
                        {
                            result.Models = new List<TestModel2>();
                        }
 
                        result.Models.Add(Codec<TestModel2>.Decode(stream.ReadBytes()));
                    }
                }

                return result;
            }

        }


        class TestModel2Codec : ICodec<TestModel2>
        {


            public byte[] Encode(TestModel2 obj)
            {
                ProtoStream stream = new ProtoStream();
                if (obj.Id != 0)
                {
                    stream.WriteTag(new Tag(1, WireType.Varint));
                    stream.WriteInt32(obj.Id);
                }
                if (obj.Name != null)
                {
                    stream.WriteTag(new Tag(2, WireType.LengthDelimited));
                    stream.WriteString(obj.Name);
                }

                return stream.ToArray();
            }
            public TestModel2 Decode(byte[] data)
            {
                Program.TestModel2 result = new Program.TestModel2();
                ProtoStream stream = new ProtoStream(data);

                while (true)
                {
                    Tag tag = stream.ReadTag();
                    if (tag.Number == -1)
                    {
                        break;
                    }
                    if (tag.Number == 1)
                    {
                        result.Id = stream.ReadInt32();
                    }
                    if (tag.Number == 2)
                    {
                        result.Name = stream.ReadString();
                    }
                }

                return result;
            }

        }

 

发现了没，两个Codec类的实现如此相似，并且每个实现的内部if块也很有规律。下面就是通过Emit在运行时的生成Codec的代码例子

public class CodecTypeGenerator
    {
        private static AssemblyBuilder codecAssmblyBuilder = System.AppDomain.CurrentDomain
            .DefineDynamicAssembly(new AssemblyName { Name = "Codec" }, AssemblyBuilderAccess.RunAndSave);

        private static ModuleBuilder codecModuleBuilder = codecAssmblyBuilder.DefineDynamicModule("Codec", "Codec.dll");

        public static Type CreateCodec<T>()
        {
            Type messageType = typeof(T);
            TypeBuilder typeBuilder = codecModuleBuilder.DefineType(messageType.Name + "Codec",
                TypeAttributes.Class | TypeAttributes.Public);

            typeBuilder.AddInterfaceImplementation(typeof(ICodec<T>));


            MethodBuilder encodeMethodBuilder = typeBuilder
                .DefineMethod("Encode", MethodAttributes.Public | MethodAttributes.Final | MethodAttributes.Virtual, 
                CallingConventions.Standard,typeof(byte[]),new  Type[]{ typeof(T)}
                );

            MethodBuilder decodeMethodBuilder = typeBuilder
                .DefineMethod("Decode", MethodAttributes.Public | MethodAttributes.Final | MethodAttributes.Virtual, CallingConventions.Standard, typeof(T), new Type[] 

{ typeof(byte[]) });


            ILGenerator encodeMethodBody = encodeMethodBuilder.GetILGenerator();

            CreateEncodeMethodBody(encodeMethodBody, messageType);

            ILGenerator decodeMethodBody = decodeMethodBuilder.GetILGenerator();
            CreateDecodeMethodBody(decodeMethodBody, messageType);
         
            return typeBuilder.CreateType();

        }
        private static ConstructorInfo tagConstructor = typeof(Tag).GetConstructor(new Type[2] { typeof(int), typeof(WireType) });
        private static ConstructorInfo proteStreamConstructorWithArgs = typeof(ProtoStream).GetConstructor(new Type[1] { typeof(byte[]) });
        private static ConstructorInfo proteStreamDefaultConstructor = typeof(ProtoStream).GetConstructor(new Type[0]);

        private static MethodInfo writeTagMethod = typeof(ProtoStream).GetMethod("WriteTag");
        private static MethodInfo writeInt32Method = typeof(ProtoStream).GetMethod("WriteInt32");
        private static MethodInfo writeStringMethod = typeof(ProtoStream).GetMethod("WriteString");
        private static MethodInfo toArrayMethod = typeof(ProtoStream).GetMethod("ToArray");
        private static MethodInfo writeBytesMethod = typeof(ProtoStream).GetMethod("WriteBytes");
        
        private static void CreateEncodeMethodBody(ILGenerator il, Type messageType)
        {
            il.DeclareLocal(typeof(ProtoStream));
            il.DeclareLocal(typeof(Tag));
            il.DeclareLocal(typeof(byte[]));

            //stream = new ProtoStream();
            il.Emit(OpCodes.Newobj, proteStreamDefaultConstructor);
            il.Emit(OpCodes.Stloc_0);

            //
            foreach (var p in messageType.GetProperties())
            {
                if (p.IsDefined(typeof(ProtoMemberAttribute), false))
                {
                    ProtoMemberAttribute protoMember = p.GetCustomAttributes(false)[0] as ProtoMemberAttribute; 
                    if (p.PropertyType == typeof(int))
                    {
                        //if(value != 0) { write(value) }
                        Label skipLabel = il.DefineLabel();
                        il.Emit(OpCodes.Ldarg_1);
                        il.Emit(OpCodes.Call, p.GetGetMethod());
                        il.Emit(OpCodes.Brfalse, skipLabel);

                        //write tag
                        il.Emit(OpCodes.Ldloc_0);
                        il.Emit(OpCodes.Ldc_I4, protoMember.TagNumber);
                        il.Emit(OpCodes.Ldc_I4_0);
                        il.Emit(OpCodes.Newobj, tagConstructor);
                        il.Emit(OpCodes.Call, writeTagMethod);

                        //write int value
                        il.Emit(OpCodes.Ldloc_0);
                        il.Emit(OpCodes.Ldarg_1);
                        il.Emit(OpCodes.Call, p.GetGetMethod());
                        il.Emit(OpCodes.Call, writeInt32Method);

                        il.MarkLabel(skipLabel);
                    }
                    if (p.PropertyType == typeof(string))
                    {
                        Label skipLabel = il.DefineLabel();
                        //if(str != null) { write str}
                        il.Emit(OpCodes.Ldarg_1);
                        il.Emit(OpCodes.Call, p.GetGetMethod());
                        il.Emit(OpCodes.Brfalse, skipLabel);


                        //write tag
                        il.Emit(OpCodes.Ldloc_0);
                        il.Emit(OpCodes.Ldc_I4, protoMember.TagNumber);
                        il.Emit(OpCodes.Ldc_I4_2);
                        il.Emit(OpCodes.Newobj, tagConstructor);
                        il.Emit(OpCodes.Call, writeTagMethod);

                        //write string value
                        il.Emit(OpCodes.Ldloc_0);
                        il.Emit(OpCodes.Ldarg_1);
                        il.Emit(OpCodes.Call, p.GetGetMethod());
                        il.Emit(OpCodes.Call, writeStringMethod);

                        il.MarkLabel(skipLabel);
                    }
                    if (p.PropertyType.IsDefined(typeof(ProtoMessageAttribute), false))
                    {
                    .......                    }
                }
              
            }

            //return stream.ToArray()
            il.Emit(OpCodes.Ldloc_0);
            il.Emit(OpCodes.Call, toArrayMethod);
            il.Emit(OpCodes.Ret);

        }

        private static MethodInfo getTagNumberMethod = typeof(Tag).GetProperty("Number").GetGetMethod();
  
        private static MethodInfo readTagMethod = typeof(ProtoStream).GetMethod("ReadTag");
        private static MethodInfo readInt32Method = typeof(ProtoStream).GetMethod("ReadInt32");
        private static MethodInfo readStringMethod = typeof(ProtoStream).GetMethod("ReadString");
        private static MethodInfo readBytesMethod = typeof(ProtoStream).GetMethod("ReadBytes");


        private static void CreateDecodeMethodBody(ILGenerator il, Type messageType)
        {
            il.DeclareLocal(messageType); //result
            il.DeclareLocal(typeof(ProtoStream)); //stream
            il.DeclareLocal(typeof(Tag)); // tag
            il.DeclareLocal(typeof(byte[])); //tempData
            Label beginWhile = il.DefineLabel();
            Label endWhile = il.DefineLabel();
            //result = new T()
            il.Emit(OpCodes.Newobj, messageType.GetConstructor(new Type[0]));
            il.Emit(OpCodes.Stloc_0);

            //stream = new ProtoStream(data)
            il.Emit(OpCodes.Ldarg_1);
            il.Emit(OpCodes.Newobj, proteStreamConstructorWithArgs);
            il.Emit(OpCodes.Stloc_1);


            il.MarkLabel(beginWhile);
            //if(tag.TagNumber == -1) break;
            il.Emit(OpCodes.Ldloc_1);
            il.Emit(OpCodes.Call, readTagMethod);
            il.Emit(OpCodes.Stloc_2);
            il.Emit(OpCodes.Ldloc_2);
            il.Emit(OpCodes.Call, getTagNumberMethod);
            il.Emit(OpCodes.Ldc_I4_M1);
            il.Emit(OpCodes.Beq, endWhile);

            foreach (var p in messageType.GetProperties())
            {
                if (p.IsDefined(typeof(ProtoMemberAttribute), false))
                {
                    ProtoMemberAttribute protoMember = p.GetCustomAttributes(false)[0] as ProtoMemberAttribute;


                    Label skipLabel = il.DefineLabel();
                    il.Emit(OpCodes.Ldloc_2);
                    il.Emit(OpCodes.Call, getTagNumberMethod);
                    il.Emit(OpCodes.Ldc_I4, protoMember.TagNumber);
                    il.Emit(OpCodes.Bne_Un, skipLabel);

                    if (p.PropertyType == typeof(int))
                    {
                        il.Emit(OpCodes.Ldloc_0);
                        il.Emit(OpCodes.Ldloc_1);
                        il.Emit(OpCodes.Call, readInt32Method);
                        il.Emit(OpCodes.Call, p.GetSetMethod());
                    }
                    if (p.PropertyType == typeof(string))
                    {
                        il.Emit(OpCodes.Ldloc_0);
                        il.Emit(OpCodes.Ldloc_1);
                        il.Emit(OpCodes.Call, readStringMethod);
                        il.Emit(OpCodes.Call, p.GetSetMethod());
                    }
                    if (p.PropertyType.IsDefined(typeof(ProtoMessageAttribute), false))
                    {
                 .......                    }
                    il.MarkLabel(skipLabel);

                }
            }
            il.Emit(OpCodes.Br, beginWhile);
            il.MarkLabel(endWhile);

            //return result
            il.Emit(OpCodes.Ldloc_0);
            il.Emit(OpCodes.Ret);
        }

        public static void Save()
        {
            codecAssmblyBuilder.Save("Codec.dll");
        }

    } 

首先不要被il代码给吓住，其实很容易实现的，你只要将手工写的两个实现，通过ildasm工具打开，大部分的指令可以现成的照抄就行了
然后就是运用一点反射来查看元数据信息，并将重复il片段写到for循环中就ok了。调试的时候如果发现有什么异常，可以通过Save方法，把动态创建的assembly写到磁盘上
然后ildasm打开或者用reflector(现在不是免费的了，不过可以试用)和手工写的反汇编代码对比着查看，很容易能找到问题。ProtoStream ,Tag类是具体对protobuf编码算法的一个封装。

最后在封装一个factory类来简化对Codec类的实例创建

 

    public class Codec<T>
    {
        static ICodec<T> codec = (ICodec<T>)Activator.CreateInstance(CodecTypeGenerator.CreateCodec<T>());

        public static byte[] Encode(T obj)
        {
            return codec.Encode(obj);
        }
        public static T Decode(byte[] data)
        {
            return codec.Decode(data);
        }
    }

然后写个简单的小测试

     

        static void Main(string[] args)
        {

            byte[] data = Codec<TestModel1>.Encode(new TestModel1
            {
                UserId = 11,
                Password = "fetion123",
                Model2 = new TestModel2 { Id = 4, Name = "test" },
                Models = new List<TestModel2> { new TestModel2 { Id = 5, Name = "asdfasfd" },
                    new TestModel2 { Id = 4, Name = "vvv" } }
            });


            TestModel1 aa = Codec<TestModel1>.Decode(data);


            Console.WriteLine(aa.Model2.Name);

            Console.ReadKey();

            CodecTypeGenerator.Save();

        }

一切ok,睡觉去！ 本文主要是演示emit。proto编码算法的代码有兴趣的可以在附近中找到！/Files/xhan/Protobuf.rar