zoukankan      html  css  js  c++  java
  • RandomAcessFile、MappedByteBuffer和缓冲读/写文件

    项目需要进行大文件的读写,调查测试的结果使我决定使用MappedByteBuffer及相关类进行文件的操作,效果不是一般的高。

    网上参考资源很多,如下两篇非常不错:

    1、花1K内存实现高效I/O的RandomAccessFile类
    2、Java中Stream和Memory-mapped File的I/O性能对比
    小结:
    1、RandomAccessFile本身不带缓冲读写,和FileInputStream、FileOutputStream等一样,直接按字节读写时,性能不可接受;
    2、使用MappedByteBuffer读写,固然性能会得到极大提升;其实只要自己处理缓冲,性能都会有非常大的提升,比如以下两种方式(见下记代码)中第一种使用了MappedByteBuffer,第二种自己进行缓冲,对于12M的文件,后者的效率甚至高于前者。
    3、BufferedXXXX之类的缓冲流,如果仅使用默认的buffer size,性能不一定最优,要权衡不同情况各种因素设置大小。

    package helloword.helloword;
    
    import java.io.BufferedInputStream;
    import java.io.BufferedOutputStream;
    import java.io.FileInputStream;
    import java.io.FileOutputStream;
    import java.io.IOException;
    import java.io.RandomAccessFile;
    import java.nio.MappedByteBuffer;
    import java.nio.channels.FileChannel;
    
    public class Test {
    
        public static void main(String[] args) {
            // TODO Auto-generated method stub
    
        }
    
        public void test() throws IOException {
            /*
             * 测试结果与Buffer size有关
             */
            // 1、使用MappedByteBuffer: 0.7s
            String srcFile = "D://Noname1.txt";
            String destFile = "D://copy.txt";
            RandomAccessFile rafi = new RandomAccessFile(srcFile, "r");
            RandomAccessFile rafo = new RandomAccessFile(destFile, "rw");
            FileChannel fci = rafi.getChannel();
            FileChannel fco = rafo.getChannel();
            long size = fci.size();
            MappedByteBuffer mbbi = fci.map(FileChannel.MapMode.READ_ONLY, 0, size);
            MappedByteBuffer mbbo = fco.map(FileChannel.MapMode.READ_WRITE, 0, size);
            long start = System.currentTimeMillis();
            for (int i = 0; i < size; i++) {
                byte b = mbbi.get(i);
                mbbo.put(i, b);
            }
            fci.close();
            fco.close();
            rafi.close();
            rafo.close();
            System.out.println("Spend: " + (double) (System.currentTimeMillis() - start) / 1000 + "s");
    
        }
    
        public void test2() throws IOException {
            // 2、自己处理Buffer(RandomAccessFile): 0.13s
            String srcFile = "D://Noname1.txt";
            String destFile = "D://copy.txt";
            RandomAccessFile rafi = new RandomAccessFile(srcFile, "r");
            RandomAccessFile rafo = new RandomAccessFile(destFile, "rw");
    
            byte[] buf = new byte[1024 * 8];
    
            long start = System.currentTimeMillis();
    
            int c = rafi.read(buf);
    
            while (c > 0) {
                if (c == buf.length) {
                    rafo.write(buf);
                } else {
                    rafo.write(buf, 0, c);
                }
    
                c = rafi.read(buf);
            }
            rafi.close();
            rafo.close();
            System.out.println("Spend: " + (double) (System.currentTimeMillis() - start) / 1000 + "s");
    
        }
    
        public void test3() throws IOException {
            // 3、BufferedInputStream&BufferedOutputStream: 3.02s
            String srcFile = "D://Noname1.txt";
            String destFile = "D://copy.txt";
            FileInputStream rafi = new FileInputStream(srcFile);
            FileOutputStream rafo = new FileOutputStream(destFile);
    
            BufferedInputStream bis = new BufferedInputStream(rafi, 8192);
            BufferedOutputStream bos = new BufferedOutputStream(rafo, 8192);
            long size = rafi.available();
    
            long start = System.currentTimeMillis();
    
            for (int i = 0; i < size; i++) {
                byte b = (byte) bis.read();
                bos.write(b);
            }
            rafi.close();
            rafo.close();
            System.out.println("Spend: " + (double) (System.currentTimeMillis() - start) / 1000 + "s");
        }
    }
      
  • 相关阅读:
    shell Builtin variables(shell内建变量)
    [置顶] 九度笔记之 1434:今年暑假不AC
    OpenRisc-45-or1200的ID模块分析
    hdu1556 Color the ball
    PB C/S轉B/S ODBC方式連接數據庫
    Django的安装配置和开发
    通过一个月时间字段分组
    如何加入该网站for Linux(绑定域名)
    LeetCode 36 Sudoku Solver
    POJ 1986 Distance Queries LCA两点距离树
  • 原文地址:https://www.cnblogs.com/duanxz/p/4875020.html
Copyright © 2011-2022 走看看