zoukankan      html  css  js  c++  java
  • 统计一个大文本行数的几种方法以及效率统计(二)

    .NET4.0 + MemoryMapping + ReadByte()

    该方法的思路主要是通过内存映射的原理,访问文件内容,由于在.net环境下不能一次性映射太大的文件,所以仍然采用分块映射的方式:

    主要代码如下:

            /// <summary>
            /// MemoryMapping + ReadByte()
            /// </summary>
            unsafe static void CalulateLine_MemoryMapping_ReadByte(uint oneBlockSize)
            {
                const string FILE_MAPPING_NAME = "~MappingTemp";
                const int LINE_MIN_SIZE = 30;
    
                long lineCount = 0;
    
                IntPtr fileHandle = ShareMemory.CreateFile(
                    FILE_NAME,
                    ShareMemory.GENERIC_READ | ShareMemory.GENERIC_WRITE,
                    FileShare.Read | FileShare.Write,
                    IntPtr.Zero,
                    FileMode.Open,
                    ShareMemory.FILE_ATTRIBUTE_NORMAL | ShareMemory.FILE_FLAG_SEQUENTIAL_SCAN,
                    IntPtr.Zero);
    
                uint fileSize = ShareMemory.GetFileSize(fileHandle, IntPtr.Zero);
    
                if (ShareMemory.INVALID_HANDLE_VALUE != (int)fileHandle)
                {
                    IntPtr mappingHandle = ShareMemory.CreateFileMapping(
                        (int)fileHandle,
                        IntPtr.Zero,
                        ShareMemory.PAGE_READWRITE,
                        0,
                        0,
                        FILE_MAPPING_NAME);
                    if (mappingHandle != IntPtr.Zero)
                    {
                        uint mapFlag = 0;
    
                        while (mapFlag <= fileSize)
                        {
                            uint eachMappingSize = oneBlockSize;
                            if (fileSize - mapFlag < oneBlockSize)
                            {
                                eachMappingSize = fileSize - mapFlag;
                            }
    
                            IntPtr pHead = ShareMemory.MapViewOfFile(
                            mappingHandle,
                            (uint)(ShareMemory.FILE_MAP_READ),
                            0,
                            mapFlag,
                            eachMappingSize);
    
                            int lastError = ShareMemory.GetLastError();
    
                            if (pHead != IntPtr.Zero)
                            {
    
                                long flag = 0;
                                while (flag < eachMappingSize)
                                {
                                    //byte* pbHead= (byte*)pHead;
                                    //byte temp = *(pbHead + flag);
    
                                    byte temp = Marshal.ReadByte((IntPtr)((int)pHead + flag));
    
                                    if (temp == 0x0D)
                                    {
                                        lineCount++;
                                        flag += LINE_MIN_SIZE;
                                    }
                                    flag++;
                                }
    
                                ShareMemory.UnmapViewOfFile(pHead);
                            }
                            mapFlag += oneBlockSize;
                        }
    
                        ShareMemory.CloseHandle(mappingHandle);
                    }
                    ShareMemory.CloseHandle(fileHandle);
    
                }
            }

    测试结果:

    MemoryMapping ReadByte()

    .NET4.0 + MemoryMapping + Unsafe

    使用unsafe代码,就是在上面代码的基础上,做了一些简单的修改。

    byte* pbHead = (byte*)pHead;
    byte temp = *(pbHead + flag);
     

    测试结果:

    MemoryMapping Unsafe

  • 相关阅读:
    边走边学Nodejs (基础入门篇)
    Android应用打包安装过程具体解释
    ubuntu与centos安装软件的不同点总结
    你好,C++(12)怎样管理多个类型同样性质同样的数据?3.6 数组
    oracle暂时表空间 ORA-01652:无法通过16(在表空间XXX中)扩展 temp 字段
    iOS中sqlite3操作
    sparkSQL1.1入门之二:sparkSQL执行架构
    [NHibernate]视图处理
    [NHibernate]立即加载
    [NHibernate]延迟加载
  • 原文地址:https://www.cnblogs.com/quark/p/1943263.html
Copyright © 2011-2022 走看看