zoukankan      html  css  js  c++  java
  • Java 复制大文件方式(nio2 FileChannel 拷贝文件能力测试)

    目前为止,我们已经学习了很多 Java 拷贝文件的方式,除了 FileChannel 提供的方法外,还包括使用 Files.copy() 或使用字节数组的缓冲/非缓冲流。那个才是最好的选择呢?这个问题很难回答,因为答案基于很多因素。本文将目光集中到一个因素,那就是速度,因为拷贝任务 越快将会提高效率,在有些情况下,这是成功的关键。因此,本文将使用一个应用程序来比较下面这些拷贝方式的具体时间:

    • FileChannel 和非直接模式的 ByteBuffer
    • FileChannel 和直接模式的 ByteBuffer
    • FileChannel.transferTo()
    • FileChannel.transferFrom()
    • FileChannel.map()
    • 使用字节数组和缓冲流
    • 使用字节数组和非缓冲流
    • File.copy()(Path 到 Path,InputStream 到 Path 和 Path 到 OutputStream)

    应用程序基于下面的条件:

    • 拷贝文件类型 MP4 视频(文件名为 Rafa Best Shots.mp4,所在目录为 C: afaelnadal ournaments2009videos)
    • 文件大小:58.3MB
    • 测试的缓冲区大小:4KB, 16KB, 32KB, 64KB, 128KB, 256KB, and 1024KB
    • 机器配置:Mobile AMD Sempron Processor 3400 + 1.80 GHz, 1.00GB RAM, 32-bit
      OS, Windows 7 Ultimate
    • 测量类型:使用 System.nanoTime() 方法
    • 连续运行三次后再获取时间;前三次运行将会被忽略。开始运行的时间总会比后面运行的时间要长一些。

    下面将列出完整的应用程序:

    import java.nio.MappedByteBuffer;
    import java.io.OutputStream;
    import java.io.InputStream;
    import java.io.BufferedInputStream;
    import java.io.BufferedOutputStream;
    import java.io.File;
    import java.io.FileInputStream;
    import java.io.FileOutputStream;
    import java.io.IOException;
    import java.nio.ByteBuffer;
    import java.nio.channels.FileChannel;
    import java.nio.file.Files;
    import java.nio.file.Path;
    import java.nio.file.Paths;
    import java.nio.file.StandardOpenOption;
    import java.util.EnumSet;
    import static java.nio.file.LinkOption.NOFOLLOW_LINKS;
     
    public class Main {
     
     public static void deleteCopied(Path path){
             
      try {
          Files.deleteIfExists(path);
      } catch (IOException ex) {
        System.err.println(ex);
      }
             
     }
         
     public static void main(String[] args) {
     
     final Path copy_from = Paths.get("C:/rafaelnadal/tournaments/2009/videos/
                                                                            Rafa Best Shots.mp4");
     final Path copy_to = Paths.get("C:/Rafa Best Shots.mp4");
     long startTime, elapsedTime;
     int bufferSizeKB = 4; //also tested for 16, 32, 64, 128, 256 and 1024
     int bufferSize = bufferSizeKB * 1024;
     
     deleteCopied(copy_to);
     
     //FileChannel and non-direct buffer
     System.out.println("Using FileChannel and non-direct buffer ...");
     try (FileChannel fileChannel_from = (FileChannel.open(copy_from,  
                          EnumSet.of(StandardOpenOption.READ)));
          FileChannel fileChannel_to = (FileChannel.open(copy_to,  
                          EnumSet.of(StandardOpenOption.CREATE_NEW, StandardOpenOption.WRITE)))) {            
     
          startTime = System.nanoTime();
                 
          // Allocate a non-direct ByteBuffer
          ByteBuffer bytebuffer = ByteBuffer.allocate(bufferSize);
     
          // Read data from file into ByteBuffer
          int bytesCount;
          while ((bytesCount = fileChannel_from.read(bytebuffer)) > 0) {
           //flip the buffer which set the limit to current position, and position to 0              
           bytebuffer.flip();
           //write data from ByteBuffer to file
           fileChannel_to.write(bytebuffer);
           //for the next read
           bytebuffer.clear();
          }
           
          elapsedTime = System.nanoTime() - startTime;
          System.out.println("Elapsed Time is " + (elapsedTime / 1000000000.0) + " seconds");
     } catch (IOException ex) {
       System.err.println(ex);
     }
     
     deleteCopied(copy_to);
     
     //FileChannel and direct buffer
     System.out.println("Using FileChannel and direct buffer ...");
     try (FileChannel fileChannel_from = (FileChannel.open(copy_from,  
                          EnumSet.of(StandardOpenOption.READ)));
          FileChannel fileChannel_to = (FileChannel.open(copy_to,  
                          EnumSet.of(StandardOpenOption.CREATE_NEW, StandardOpenOption.WRITE)))) {            
     
          startTime = System.nanoTime();
                 
          // Allocate a direct ByteBuffer
          ByteBuffer bytebuffer = ByteBuffer.allocateDirect(bufferSize);
     
          // Read data from file into ByteBuffer
          int bytesCount;
          while ((bytesCount = fileChannel_from.read(bytebuffer)) > 0) {
           //flip the buffer which set the limit to current position, and position to 0              
           bytebuffer.flip();
           //write data from ByteBuffer to file
           fileChannel_to.write(bytebuffer);
           //for the next read
           bytebuffer.clear();
          }
           
          elapsedTime = System.nanoTime() - startTime;
          System.out.println("Elapsed Time is " + (elapsedTime / 1000000000.0) + " seconds");
     } catch (IOException ex) {
       System.err.println(ex);
     }
     
     deleteCopied(copy_to);
     
     //FileChannel.transferTo()
     System.out.println("Using FileChannel.transferTo method ...");
     try (FileChannel fileChannel_from = (FileChannel.open(copy_from,  
                          EnumSet.of(StandardOpenOption.READ)));
          FileChannel fileChannel_to = (FileChannel.open(copy_to,  
                          EnumSet.of(StandardOpenOption.CREATE_NEW, StandardOpenOption.WRITE)))) {
     
          startTime = System.nanoTime();
     
          fileChannel_from.transferTo(0L, fileChannel_from.size(), fileChannel_to);
     
          elapsedTime = System.nanoTime() - startTime;
          System.out.println("Elapsed Time is " + (elapsedTime / 1000000000.0) + " seconds");
     } catch (IOException ex) {
       System.err.println(ex);
     }
     
     deleteCopied(copy_to);
             
     //FileChannel.transferFrom()
     System.out.println("Using FileChannel.transferFrom method ...");
     try (FileChannel fileChannel_from = (FileChannel.open(copy_from,   
                          EnumSet.of(StandardOpenOption.READ)));
          FileChannel fileChannel_to = (FileChannel.open(copy_to,  
                          EnumSet.of(StandardOpenOption.CREATE_NEW, StandardOpenOption.WRITE)))) {
     
          startTime = System.nanoTime();
     
          fileChannel_to.transferFrom(fileChannel_from, 0L, (int) fileChannel_from.size());
     
          elapsedTime = System.nanoTime() - startTime;
          System.out.println("Elapsed Time is " + (elapsedTime / 1000000000.0) + " seconds");
     } catch (IOException ex) {
       System.err.println(ex);
     }
     
     deleteCopied(copy_to);         
             
     //FileChannel.map
     System.out.println("Using FileChannel.map method ...");
     try (FileChannel fileChannel_from = (FileChannel.open(copy_from,  
                          EnumSet.of(StandardOpenOption.READ)));
          FileChannel fileChannel_to = (FileChannel.open(copy_to,  
                          EnumSet.of(StandardOpenOption.CREATE_NEW, StandardOpenOption.WRITE)))) {
                           
          startTime = System.nanoTime();
          MappedByteBuffer buffer = fileChannel_from.map(FileChannel.MapMode.READ_ONLY,  
                                                                      0, fileChannel_from.size());
                             
          fileChannel_to.write(buffer);
          buffer.clear();
     
          elapsedTime = System.nanoTime() - startTime;
          System.out.println("Elapsed Time is " + (elapsedTime / 1000000000.0) + " seconds");
    } catch (IOException ex) {
      System.err.println(ex);
    }
             
    deleteCopied(copy_to);       
             
    //Buffered Stream I/O  
    System.out.println("Using buffered streams and byte array ...");  
    File inFileStr = copy_from.toFile();
    File outFileStr = copy_to.toFile();
    try (BufferedInputStream in = new BufferedInputStream(new FileInputStream(inFileStr));
         BufferedOutputStream out = new BufferedOutputStream(new FileOutputStream(outFileStr))) {
     
         startTime = System.nanoTime();
     
         byte[] byteArray = new byte[bufferSize];
         int bytesCount;
         while ((bytesCount = in.read(byteArray)) != -1) {
                 out.write(byteArray, 0, bytesCount);
         }
     
         elapsedTime = System.nanoTime() - startTime;
         System.out.println("Elapsed Time is " + (elapsedTime / 1000000000.0) + " seconds");
    } catch (IOException ex) {
      System.err.println(ex);
    }
     
    deleteCopied(copy_to);
     
    System.out.println("Using un-buffered streams and byte array ...");  
    try (FileInputStream in = new FileInputStream(inFileStr);
         FileOutputStream out = new FileOutputStream(outFileStr)) {
     
         startTime = System.nanoTime();
     
         byte[] byteArray = new byte[bufferSize];
         int bytesCount;
         while ((bytesCount = in.read(byteArray)) != -1) {
                 out.write(byteArray, 0, bytesCount);
         }
     
         elapsedTime = System.nanoTime() - startTime;
         System.out.println("Elapsed Time is " + (elapsedTime / 1000000000.0) + " seconds");
    } catch (IOException ex) {
      System.err.println(ex);
    }
     
    deleteCopied(copy_to);
     
    System.out.println("Using Files.copy (Path to Path) method ...");
    try {
        startTime = System.nanoTime();
     
        Files.copy(copy_from, copy_to, NOFOLLOW_LINKS);
     
        elapsedTime = System.nanoTime() - startTime;
        System.out.println("Elapsed Time is " + (elapsedTime / 1000000000.0) + " seconds");
    } catch (IOException e) {
      System.err.println(e);
    }
     
    deleteCopied(copy_to);
     
    System.out.println("Using Files.copy (InputStream to Path) ...");
    try (InputStream is = new FileInputStream(copy_from.toFile())) {
     
        startTime = System.nanoTime();
     
        Files.copy(is, copy_to);
     
        elapsedTime = System.nanoTime() - startTime;
        System.out.println("Elapsed Time is " + (elapsedTime / 1000000000.0) + " seconds");
    } catch (IOException e) {
      System.err.println(e);
    }
     
    deleteCopied(copy_to);
     
    System.out.println("Using Files.copy (Path to OutputStream) ...");
    try (OutputStream os = new FileOutputStream(copy_to.toFile())) {
     
         startTime = System.nanoTime();
     
         Files.copy(copy_from, os);
     
         elapsedTime = System.nanoTime() - startTime;
         System.out.println("Elapsed Time is " + (elapsedTime / 1000000000.0) + " seconds");
     } catch (IOException e) {
       System.err.println(e);
     }
     }
    } 
    

    输出结果排序比较复杂,其中包含了很多数据。下面我将主要的对比用图形的方式展示出来。图形中 Y 坐标表示消耗的时间(单位:秒),X 坐标表示缓冲的大小(或运行次数,跳过了前三次运行)。

    FileChannel 和非直接模式 Buffer vs. FileChannel 和直接模式 Buffer

    从下图看来,如果缓存小于 256KB,那么非直接模式的 Buffer 快一点,而缓存大于 256KB 后,直接模式的 Buffer 快一点:

    FileChannel.transferTo() vs. FileChannel.transferFrom() vs. FileChannel.map()

    从下图看来,FileChannel.transferTo() 和 FileChannel.transferFrom 运行七次的速度都差不多,而 FileChannel.map 的速度就要差很多:

    三种 Files.copy() 方法

    从下图看来,最快的是 Path 到 Path,其次是 Path 到 OutputStream,最慢的是 InputStream 到 Path:

    FileChannel 和非直接模式 Buffer vs. FileChannel.transferTo() vs. Path 到 Path

    最后,我们将前面最快的三种方式综合起来比较。从比较的结果来看,似乎 Path 到 Path 是最快的解决方案:

  • 相关阅读:
    拦截器
    Ajax
    JSON
    数据处理及跳转
    RestFul和控制器
    第一个MVC程序
    什么是SpringMVC
    回顾MVC
    声明式事务
    微软最强 Python 自动化工具开源了!不用写一行代码!
  • 原文地址:https://www.cnblogs.com/interdrp/p/3785164.html
Copyright © 2011-2022 走看看