zoukankan      html  css  js  c++  java
  • RandomAcessFile、MappedByteBuffer和缓冲读/写文件

    项目需要进行大文件的读写,调查测试的结果使我决定使用MappedByteBuffer及相关类进行文件的操作,效果不是一般的高。

    网上参考资源很多,如下两篇非常不错:

    1、花1K内存实现高效I/O的RandomAccessFile类
    2、Java中Stream和Memory-mapped File的I/O性能对比
    小结:
    1、RandomAccessFile本身不带缓冲读写,和FileInputStream、FileOutputStream等一样,直接按字节读写时,性能不可接受;
    2、使用MappedByteBuffer读写,固然性能会得到极大提升;其实只要自己处理缓冲,性能都会有非常大的提升,比如以下两种方式(见下记代码)中第一种使用了MappedByteBuffer,第二种自己进行缓冲,对于12M的文件,后者的效率甚至高于前者。
    3、BufferedXXXX之类的缓冲流,如果仅使用默认的buffer size,性能不一定最优,要权衡不同情况各种因素设置大小。

    package helloword.helloword;
    
    import java.io.BufferedInputStream;
    import java.io.BufferedOutputStream;
    import java.io.FileInputStream;
    import java.io.FileOutputStream;
    import java.io.IOException;
    import java.io.RandomAccessFile;
    import java.nio.MappedByteBuffer;
    import java.nio.channels.FileChannel;
    
    public class Test {
    
        public static void main(String[] args) {
            // TODO Auto-generated method stub
    
        }
    
        public void test() throws IOException {
            /*
             * 测试结果与Buffer size有关
             */
            // 1、使用MappedByteBuffer: 0.7s
            String srcFile = "D://Noname1.txt";
            String destFile = "D://copy.txt";
            RandomAccessFile rafi = new RandomAccessFile(srcFile, "r");
            RandomAccessFile rafo = new RandomAccessFile(destFile, "rw");
            FileChannel fci = rafi.getChannel();
            FileChannel fco = rafo.getChannel();
            long size = fci.size();
            MappedByteBuffer mbbi = fci.map(FileChannel.MapMode.READ_ONLY, 0, size);
            MappedByteBuffer mbbo = fco.map(FileChannel.MapMode.READ_WRITE, 0, size);
            long start = System.currentTimeMillis();
            for (int i = 0; i < size; i++) {
                byte b = mbbi.get(i);
                mbbo.put(i, b);
            }
            fci.close();
            fco.close();
            rafi.close();
            rafo.close();
            System.out.println("Spend: " + (double) (System.currentTimeMillis() - start) / 1000 + "s");
    
        }
    
        public void test2() throws IOException {
            // 2、自己处理Buffer(RandomAccessFile): 0.13s
            String srcFile = "D://Noname1.txt";
            String destFile = "D://copy.txt";
            RandomAccessFile rafi = new RandomAccessFile(srcFile, "r");
            RandomAccessFile rafo = new RandomAccessFile(destFile, "rw");
    
            byte[] buf = new byte[1024 * 8];
    
            long start = System.currentTimeMillis();
    
            int c = rafi.read(buf);
    
            while (c > 0) {
                if (c == buf.length) {
                    rafo.write(buf);
                } else {
                    rafo.write(buf, 0, c);
                }
    
                c = rafi.read(buf);
            }
            rafi.close();
            rafo.close();
            System.out.println("Spend: " + (double) (System.currentTimeMillis() - start) / 1000 + "s");
    
        }
    
        public void test3() throws IOException {
            // 3、BufferedInputStream&BufferedOutputStream: 3.02s
            String srcFile = "D://Noname1.txt";
            String destFile = "D://copy.txt";
            FileInputStream rafi = new FileInputStream(srcFile);
            FileOutputStream rafo = new FileOutputStream(destFile);
    
            BufferedInputStream bis = new BufferedInputStream(rafi, 8192);
            BufferedOutputStream bos = new BufferedOutputStream(rafo, 8192);
            long size = rafi.available();
    
            long start = System.currentTimeMillis();
    
            for (int i = 0; i < size; i++) {
                byte b = (byte) bis.read();
                bos.write(b);
            }
            rafi.close();
            rafo.close();
            System.out.println("Spend: " + (double) (System.currentTimeMillis() - start) / 1000 + "s");
        }
    }
      
  • 相关阅读:
    ubuntu下安装maven
    159.Longest Substring with At Most Two Distinct Characters
    156.Binary Tree Upside Down
    155.Min Stack
    154.Find Minimum in Rotated Sorted Array II
    153.Find Minimum in Rotated Sorted Array
    152.Maximum Product Subarray
    151.Reverse Words in a String
    150.Evaluate Reverse Polish Notation
    149.Max Points on a Line
  • 原文地址:https://www.cnblogs.com/duanxz/p/4875020.html
Copyright © 2011-2022 走看看