zoukankan      html  css  js  c++  java
  • C#+无unsafe的非托管大数组(large unmanaged array in c# without 'unsafe' keyword)

    C#+无unsafe的非托管大数组(large unmanaged array in c# without 'unsafe' keyword)

    +BIT祝威+悄悄在此留下版了个权的信息说:

    C#申请一个大数组(Use a large array in C#)

    在C#里,有时候我需要能够申请一个很大的数组、使用之、然后立即释放其占用的内存。

    Sometimes I need to allocate a large array, use it and then release its memory space immediately.

    由于在C#里提供的 int[] array = new int[1000000]; 这样的数组,其内存释放很难由程序员完全控制,在申请一个大数组后,程序可能会变得很慢。

    If I use something like  int[] array = new int[1000000]; , it will be difficult to release its memory space by programmer and the app probably runs slower and slower.

    特别是在C#+OpenGL编程中,我在使用VAO/VBO时十分需要设计一个非托管的数组,比如在glBufferData时我希望可以使用下面的glBufferData:

    Specially in C#+OpenGL routines when I'm using VAO/VBO, I need an unmanaged array for glBufferData:

     1         /// <summary>
     2         /// 设置当前VBO的数据。
     3         /// </summary>
     4         /// <param name="target"></param>
     5         /// <param name="data"></param>
     6         /// <param name="usage"></param>
     7         public static void glBufferData(uint target, UnmanagedArrayBase data, uint usage)
     8         {
     9             GetDelegateFor<glBufferData>()((uint)target,
    10                 data.ByteLength, // 使用非托管数组
    11                 data.Header, // 使用非托管数组
    12                 (uint)usage);
    13         }
    14         // ...
    15         // glBufferData的声明
    16         private delegate void glBufferData(uint target, int size, IntPtr data, uint usage);

    而在指定VBO的数据时,可能是float、vec3等等类型:

    And the content in VBO can be float, vec3 and any other structs.

     1         /// <summary>
     2         /// 金字塔的posotion array.
     3         /// </summary>
     4         static vec3[] positions = new vec3[]
     5         {
     6             new vec3(0.0f, 1.0f, 0.0f),
     7             new vec3(-1.0f, -1.0f, 1.0f),
     8             // ...
     9             new vec3(-1.0f, -1.0f, 1.0f),   
    10         };            
    11 //  Create a vertex buffer for the vertex data.
    12             {
    13                 uint[] ids = new uint[1];
    14                 GL.GenBuffers(1, ids);
    15                 GL.BindBuffer(GL.GL_ARRAY_BUFFER, ids[0]);
    16                 // 使用vec3作为泛型的非托管数组的参数
    17                 UnmanagedArray<vec3> positionArray = new UnmanagedArray<vec3>(positions.Length);
    18                 for (int i = 0; i < positions.Length; i++)
    19                 {
    20                     // 使用this[i]这样的索引方式来读写非托管数组的元素
    21                     positionArray[i] = positions[i];
    22                 }
    23                 GL.BufferData(BufferDataTarget.ArrayBuffer, positionArray, BufferDataUsage.StaticDraw);
    24                 GL.VertexAttribPointer(positionLocation, 3, GL.GL_FLOAT, false, 0, IntPtr.Zero);
    25                 GL.EnableVertexAttribArray(positionLocation);
    26             }
    +BIT祝威+悄悄在此留下版了个权的信息说:

    UnmanagedArray<T>

    所以我设计了这样一个非托管的数组类型:无unsafe,可接收任何struct类型作为泛型参数,可随时释放内存。

    So I designed this UnmangedArray<T> : no 'unsafe' keyword, takes any struct as generic parameter, can be released anytime you want.

      1     /// <summary>
      2     /// 元素类型为sbyte, byte, char, short, ushort, int, uint, long, ulong, float, double, decimal, bool或其它struct的非托管数组。
      3     /// <para>不能使用enum类型作为T。</para>
      4     /// </summary>
      5     /// <typeparam name="T">sbyte, byte, char, short, ushort, int, uint, long, ulong, float, double, decimal, bool或其它struct, 不能使用enum类型作为T。</typeparam>
      6     public class UnmanagedArray<T> : UnmanagedArrayBase where T : struct
      7     {
      8 
      9         /// <summary>
     10         ///元素类型为sbyte, byte, char, short, ushort, int, uint, long, ulong, float, double, decimal, bool或其它struct的非托管数组。 
     11         /// </summary>
     12         /// <param name="count"></param>
     13         [MethodImpl(MethodImplOptions.Synchronized)]
     14         public UnmanagedArray(int count)
     15             : base(count, Marshal.SizeOf(typeof(T)))
     16         {
     17         }
     18      
     19         /// <summary>
     20         /// 获取或设置索引为<paramref name="index"/>的元素。
     21         /// </summary>
     22         /// <param name="index"></param>
     23         /// <returns></returns>
     24         public T this[int index]
     25         {
     26             get
     27             {
     28                 if (index < 0 || index >= this.Count)
     29                     throw new IndexOutOfRangeException("index of UnmanagedArray is out of range");
     30 
     31                 var pItem = this.Header + (index * elementSize);
     32                 //var obj = Marshal.PtrToStructure(pItem, typeof(T));
     33                 //T result = (T)obj;
     34                 T result = Marshal.PtrToStructure<T>(pItem);// works in .net 4.5.1
     35                 return result;
     36             }
     37             set
     38             {
     39                 if (index < 0 || index >= this.Count)
     40                     throw new IndexOutOfRangeException("index of UnmanagedArray is out of range");
     41                 
     42                 var pItem = this.Header + (index * elementSize);
     43                 //Marshal.StructureToPtr(value, pItem, true);
     44                 Marshal.StructureToPtr<T>(value, pItem, true);// works in .net 4.5.1
     45             }
     46         }
     47 
     48         /// <summary>
     49         /// 按索引顺序依次获取各个元素。
     50         /// </summary>
     51         /// <returns></returns>
     52         public IEnumerable<T> GetElements()
     53         {
     54             if (!this.disposed)
     55             {
     56                 for (int i = 0; i < this.Count; i++)
     57                 {
     58                     yield return this[i];
     59                 }
     60             }
     61         }
     62     }
     63 
     64     /// <summary>
     65     /// 非托管数组的基类。
     66     /// </summary>
     67     public abstract class UnmanagedArrayBase : IDisposable
     68     {
     69 
     70         /// <summary>
     71         /// 数组指针。
     72         /// </summary>
     73         public IntPtr Header { get; private set; }
     74 
     75         /// <summary>
     76         /// 元素数目。
     77         /// </summary>
     78         public int Count { get; private set; }
     79 
     80         /// <summary>
     81         /// 单个元素的字节数。
     82         /// </summary>
     83         protected int elementSize;
     84 
     85         /// <summary>
     86         /// 申请到的字节数。(元素数目 * 单个元素的字节数)。
     87         /// </summary>
     88         public int ByteLength
     89         {
     90             get { return this.Count * this.elementSize; }
     91         }
     92 
     93            
     94         /// <summary>
     95         /// 非托管数组。
     96         /// </summary>
     97         /// <param name="elementCount">元素数目。</param>
     98         /// <param name="elementSize">单个元素的字节数。</param>
     99         [MethodImpl(MethodImplOptions.Synchronized)]
    100         protected UnmanagedArrayBase(int elementCount, int elementSize)
    101         {
    102             this.Count = elementCount;
    103             this.elementSize = elementSize;
    104 
    105             int memSize = elementCount * elementSize;
    106             this.Header = Marshal.AllocHGlobal(memSize);
    107 
    108             allocatedArrays.Add(this);
    109         }
    110 
    111         private static readonly List<IDisposable> allocatedArrays = new List<IDisposable>();
    112 
    113         /// <summary>
    114         /// 立即释放所有<see cref="UnmanagedArray"/>115         /// </summary>
    116         [MethodImpl(MethodImplOptions.Synchronized)]
    117         public static void FreeAll()
    118         {
    119             foreach (var item in allocatedArrays)
    120             {
    121                 item.Dispose();
    122             }
    123             allocatedArrays.Clear();
    124         }
    125 
    126         ~UnmanagedArrayBase()
    127         {
    128             Dispose();
    129         }
    130 
    131         #region IDisposable Members
    132 
    133         /// <summary>
    134         /// Internal variable which checks if Dispose has already been called
    135         /// </summary>
    136         protected Boolean disposed;
    137 
    138         /// <summary>
    139         /// Releases unmanaged and - optionally - managed resources
    140         /// </summary>
    141         /// <param name="disposing"><c>true</c> to release both managed and unmanaged resources; <c>false</c> to release only unmanaged resources.</param>
    142         protected void Dispose(Boolean disposing)
    143         {
    144             if (disposed)
    145             {
    146                 return;
    147             }
    148 
    149             if (disposing)
    150             {
    151                 //Managed cleanup code here, while managed refs still valid
    152             }
    153             //Unmanaged cleanup code here
    154             IntPtr ptr = this.Header;
    155 
    156             if (ptr != IntPtr.Zero)
    157             {
    158                 this.Count = 0;
    159                 this.Header = IntPtr.Zero;
    160                 Marshal.FreeHGlobal(ptr);
    161             }
    162 
    163             disposed = true;
    164         }
    165 
    166         /// <summary>
    167         /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.
    168         /// </summary>
    169         public void Dispose()
    170         {
    171             this.Dispose(true);
    172             GC.SuppressFinalize(this);
    173         }
    174 
    175         #endregion
    176                 
    177     }
    UnmanagedArray


     

    +BIT祝威+悄悄在此留下版了个权的信息说:

    如何使用(How to use)

    UnmanagedArray<T>使用方式十分简单,就像一个普通的数组一样:

    Using UnamangedAray<T> is just like a normal array(int[], vec3[], etc.):

     1         internal static void TypicalScene()
     2         {
     3             const int count = 100;
     4 
     5             // 测试float类型
     6             var floatArray = new UnmanagedArray<float>(count);
     7             for (int i = 0; i < count; i++)
     8             {
     9                 floatArray[i] = i;
    10             }
    11             for (int i = 0; i < count; i++)
    12             {
    13                 var item = floatArray[i];
    14                 if (item != i)
    15                 { throw new Exception(); }
    16             }
    17 
    18             // 测试int类型
    19             var intArray = new UnmanagedArray<int>(count);
    20             for (int i = 0; i < count; i++)
    21             {
    22                 intArray[i] = i;
    23             }
    24             for (int i = 0; i < count; i++)
    25             {
    26                 var item = intArray[i];
    27                 if (item != i)
    28                 { throw new Exception(); }
    29             }
    30 
    31             // 测试bool类型
    32             var boolArray = new UnmanagedArray<bool>(count);
    33             for (int i = 0; i < count; i++)
    34             {
    35                 boolArray[i] = i % 2 == 0;
    36             }
    37             for (int i = 0; i < count; i++)
    38             {
    39                 var item = boolArray[i];
    40                 if (item != (i % 2 == 0))
    41                 { throw new Exception(); }
    42             }
    43 
    44             // 测试vec3类型
    45             var vec3Array = new UnmanagedArray<vec3>(count);
    46             for (int i = 0; i < count; i++)
    47             {
    48                 vec3Array[i] = new vec3(i * 3 + 0, i * 3 + 1, i * 3 + 2);
    49             }
    50             for (int i = 0; i < count; i++)
    51             {
    52                 var item = vec3Array[i];
    53                 var old = new vec3(i * 3 + 0, i * 3 + 1, i * 3 + 2);
    54                 if (item.x != old.x || item.y != old.y || item.z != old.z)
    55                 { throw new Exception(); }
    56             }
    57 
    58             // 测试foreach
    59             foreach (var item in vec3Array.GetElements())
    60             {
    61                 Console.WriteLine(item);
    62             }
    63 
    64             // 释放此数组占用的内存,这之后就不能再使用vec3Array了。
    65             vec3Array.Dispose();
    66 
    67             // 立即释放所有非托管数组占用的内存,这之后就不能再使用上面申请的数组了。
    68             UnmanagedArrayBase.FreeAll();
    69         }

    快速读写UnmanagedArray<T>

    UnmanagedArrayHelper

    由于很多时候需要申请和使用很大的UnmanagedArray<T>,直接使用this[index]索引方式速度会偏慢,所以我添加了几个辅助方法,专门解决快速读写UnmanagedArray<T>的问题。

     1     public static class UnmanagedArrayHelper
     2     {
     3         ///// <summary>
     4         ///// 错误    1    无法获取托管类型(“T”)的地址和大小,或无法声明指向它的指针
     5         ///// </summary>
     6         ///// <typeparam name="T"></typeparam>
     7         ///// <param name="array"></param>
     8         ///// <returns></returns>
     9         //public static unsafe T* FirstElement<T>(this UnmanagedArray<T> array) where T : struct
    10         //{
    11         //    var header = (void*)array.Header;
    12         //    return (T*)header;
    13         //}
    14 
    15         /// <summary>
    16         /// 获取非托管数组的第一个元素的地址。
    17         /// </summary>
    18         /// <param name="array"></param>
    19         /// <returns></returns>
    20         public static unsafe void* FirstElement(this UnmanagedArrayBase array)
    21         {
    22             var header = (void*)array.Header;
    23 
    24             return header;
    25         }
    26 
    27         public static unsafe void* LastElement(this UnmanagedArrayBase array)
    28         {
    29             var last = (void*)(array.Header + (array.ByteLength - array.ByteLength / array.Length));
    30 
    31             return last;
    32         }
    33 
    34         /// <summary>
    35         /// 获取非托管数组的最后一个元素的地址再向后一个单位的地址。
    36         /// </summary>
    37         /// <param name="array"></param>
    38         /// <returns></returns>
    39         public static unsafe void* TailAddress(this UnmanagedArrayBase array)
    40         {
    41             var tail = (void*)(array.Header + array.ByteLength);
    42 
    43             return tail;
    44         }
    45     }

    如何使用

    这个类型实现了3个扩展方法,可以获取UnmanagedArray<T>的第一个元素的位置、最后一个元素的位置、最后一个元素+1的位置。用这种unsafe的方法可以实现C语言一样的读写速度。

    下面是一个例子。用unsafe的方式读写UnmanagedArray<T>,速度比this[index]方式快10到70倍。

     1         public static void TypicalScene()
     2         {
     3             int length = 1000000;
     4             UnmanagedArray<int> array = new UnmanagedArray<int>(length);
     5             UnmanagedArray<int> array2 = new UnmanagedArray<int>(length);
     6 
     7             long tick = DateTime.Now.Ticks;
     8             for (int i = 0; i < length; i++)
     9             {
    10                 array[i] = i;
    11             }
    12             long totalTicks = DateTime.Now.Ticks - tick;
    13 
    14             tick = DateTime.Now.Ticks;
    15             unsafe
    16             {
    17                 int* header = (int*)array2.FirstElement();
    18                 int* last = (int*)array2.LastElement();
    19                 int* tailAddress = (int*)array2.TailAddress();
    20                 int value = 0;
    21                 for (int* ptr = header; ptr <= last/*or: ptr < tailAddress*/; ptr++)
    22                 {
    23                     *ptr = value++;
    24                 }
    25             }
    26             long totalTicks2 = DateTime.Now.Ticks - tick;
    27             Console.WriteLine("ticks: {0}, {1}", totalTicks, totalTicks2);// unsafe method works faster.
    28 
    29             for (int i = 0; i < length; i++)
    30             {
    31                 if (array[i] != i)
    32                 {
    33                     Console.WriteLine("something wrong here");
    34                 }
    35                 if (array2[i] != i)
    36                 {
    37                     Console.WriteLine("something wrong here");
    38                 }
    39             }
    40 
    41             array.Dispose();
    42             array2.Dispose();
    43         }
     1                 unsafe
     2                 {
     3                     vec3* header = (vec3*)vec3Array.FirstElement();
     4                     vec3* last = (vec3*)vec3Array.LastElement();
     5                     vec3* tailAddress = (vec3*)vec3Array.TailAddress();
     6                     int i = 0;
     7                     for (vec3* ptr = header; ptr <= last/*or: ptr < tailAddress*/; ptr++)
     8                     {
     9                         *ptr = new vec3(i * 3 + 0, i * 3 + 1, i * 3 + 2);
    10                         i++;
    11                     }
    12                     i = 0;
    13                     for (vec3* ptr = header; ptr <= last/*or: ptr < tailAddress*/; ptr++, i++)
    14                     {
    15                         var item = *ptr;
    16                         var old = new vec3(i * 3 + 0, i * 3 + 1, i * 3 + 2);
    17                         if (item.x != old.x || item.y != old.y || item.z != old.z)
    18                         { throw new Exception(); }
    19                     }
    20                 }

    2015-08-25

    用StructLayout和MarshalAs支持复杂的struct

    在OpenGL中我需要用UnmanagedArray<mat4>,其中mat4定义如下:

     1     /// <summary>
     2     /// Represents a 4x4 matrix.
     3     /// </summary>
     4     [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Size = 4 * 4 * 4)]
     5     public struct mat4
     6     {
     7         /// <summary>
     8         /// Gets or sets the <see cref="vec4"/> column at the specified index.
     9         /// </summary>
    10         /// <value>
    11         /// The <see cref="vec4"/> column.
    12         /// </value>
    13         /// <param name="column">The column index.</param>
    14         /// <returns>The column at index <paramref name="column"/>.</returns>
    15         public vec4 this[int column]
    16         {
    17             get { return cols[column]; }
    18             set { cols[column] = value; }
    19         }
    20 
    21         /// <summary>
    22         /// Gets or sets the element at <paramref name="column"/> and <paramref name="row"/>.
    23         /// </summary>
    24         /// <value>
    25         /// The element at <paramref name="column"/> and <paramref name="row"/>.
    26         /// </value>
    27         /// <param name="column">The column index.</param>
    28         /// <param name="row">The row index.</param>
    29         /// <returns>
    30         /// The element at <paramref name="column"/> and <paramref name="row"/>.
    31         /// </returns>
    32         public float this[int column, int row]
    33         {
    34             get { return cols[column][row]; }
    35             set { cols[column][row] = value; }
    36         }
    37 
    38         /// <summary>
    39         /// The columms of the matrix.
    40         /// </summary>
    41         [MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
    42         private vec4[] cols;
    43     }
    44 
    45     /// <summary>
    46     /// Represents a four dimensional vector.
    47     /// </summary>
    48     [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Size = 4 * 4)]
    49     public struct vec4
    50     {
    51         public float x;
    52         public float y;
    53         public float z;
    54         public float w;
    55 
    56         public float this[int index]
    57         {
    58             get
    59             {
    60                 if (index == 0) return x;
    61                 else if (index == 1) return y;
    62                 else if (index == 2) return z;
    63                 else if (index == 3) return w;
    64                 else throw new Exception("Out of range.");
    65             }
    66             set
    67             {
    68                 if (index == 0) x = value;
    69                 else if (index == 1) y = value;
    70                 else if (index == 2) z = value;
    71                 else if (index == 3) w = value;
    72                 else throw new Exception("Out of range.");
    73             }
    74         }
    75     }
    mat4

    注意:UnmanagedArray<T>支持的struct,T的大小必须是确定的。所以在mat4里我们用 [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Size = 4 * 4 * 4)] 指定mat4的大小为4个 vec4 * 4个 float * 4个字节(每个float) = 64字节,并且在 private vec4[] cols; 上用 [MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)] 规定了cols的元素数必须是4。之后在 vec4 上的 [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Size = 4 * 4)] 不写也可以,因为vec4只有4个简单的float字段,不含复杂类型。

    下面是测试用例。

     1             mat4 matrix = glm.scale(mat4.identity(), new vec3(2, 3, 4));
     2 
     3             var size = Marshal.SizeOf(typeof(mat4));
     4             size = Marshal.SizeOf(matrix);
     5 
     6             UnmanagedArray<mat4> array = new UnmanagedArray<mat4>(1);
     7             array[0] = matrix;
     8 
     9             mat4 newMatirx = array[0]; // newMatrix should be equal to matrix
    10 
    11             array.Dispose();

    如果matrix和newMatrix相等,就说明上述Attribute配置正确了。

  • 相关阅读:
    Ext JS学习第三天 我们所熟悉的javascript(二)
    Ext JS学习第二天 我们所熟悉的javascript(一)
    Ext JS学习第十七天 事件机制event(二)
    Ext JS学习第十六天 事件机制event(一)
    Ext JS学习第十五天 Ext基础之 Ext.DomQuery
    Ext JS学习第十四天 Ext基础之 Ext.DomHelper
    Ext JS学习第十三天 Ext基础之 Ext.Element
    Ext JS学习第十天 Ext基础之 扩展原生的javascript对象(二)
    针对错误 “服务器提交了协议冲突. Section=ResponseHeader Detail=CR 后面必须是 LF” 的原因分析
    C# 使用HttpWebRequest通过PHP接口 上传文件
  • 原文地址:https://www.cnblogs.com/bitzhuwei/p/huge-unmanged-array-in-csharp.html
Copyright © 2011-2022 走看看