zoukankan      html  css  js  c++  java
  • Hadoop 文件压缩

    1、指定编解码器类

      public void deflateCompress() throws IOException {
            //获取deflate编解码器类
            Class codecClass = DeflateCodec.class;
            //实例化
            CompressionCodec codec =  (CompressionCodec)ReflectionUtils.newInstance(codecClass,new Configuration());
            //创建文件输出流
            FileOutputStream fos =  new FileOutputStream("D:/compress/1.deflate");
            //得到文件压缩流
            CompressionOutputStream zipout = codec.createOutputStream(fos);
            IOUtils.copyBytes(new FileInputStream("D:/compress/hello.txt"),zipout,1024);
            zipout.close();
        }

    2、不指定编解码器类-通用压缩测试

     @Test
        public void deflateCompress() throws IOException {
          Class[] zipclass = {
                  DeflateCodec.class,
                  GzipCodec.class,
                  BZip2Codec.class,
          };
          for(Class c : zipclass){
              commonCompress(c);
          }
        }
    
        public void commonCompress(Class codecClass) throws IOException {
            //获取当前系统毫秒数
            long start = System.currentTimeMillis();
            //实例化
            CompressionCodec codec =  (CompressionCodec)ReflectionUtils.newInstance(codecClass,new Configuration());
            //创建文件输出流,得到默认扩展流
            FileOutputStream fos =  new FileOutputStream("D:/compress/compress" + codec.getDefaultExtension());
            //得到文件压缩流
            CompressionOutputStream zipout = codec.createOutputStream(fos);
            IOUtils.copyBytes(new FileInputStream("D:/compress/hello.txt"),zipout,1024);
            zipout.close();
            System.out.println(codecClass.getSimpleName() + " : " + (System.currentTimeMillis()-start)+ "ms" );
    
        }

    测试结果
    DeflateCodec : 441ms
    GzipCodec : 4ms
    BZip2Codec : 55ms


    3、解压缩

    //解压
        public void unzip(Class codecClass) throws IOException {
            //获取当前系统毫秒数
            long start = System.currentTimeMillis();
            //实例化
            CompressionCodec codec =  (CompressionCodec)ReflectionUtils.newInstance(codecClass,new Configuration());
            //创建文件输入流,
            FileInputStream fis  =  new FileInputStream("D:/compress/compress" + codec.getDefaultExtension());
            //得到文件压缩流
            CompressionInputStream zipIn = codec.createInputStream(fis);
            IOUtils.copyBytes(zipIn,new FileOutputStream("D:/compress/unzip" + ".txt"),1024);
            zipIn.close();
            System.out.println(codecClass.getSimpleName() + " : " + (System.currentTimeMillis()-start) + "ms");
    
        }

    DeflateCodec : 414ms
    GzipCodec : 44ms
    Disconnected from the target VM, address: '127.0.0.1:61073', transport: 'socket'
    BZip2Codec : 45ms
































    欢迎关注我的公众号:小秋的博客 CSDN博客:https://blog.csdn.net/xiaoqiu_cr github:https://github.com/crr121 联系邮箱:rongchen633@gmail.com 有什么问题可以给我留言噢~
  • 相关阅读:
    Spring BeanFactory与FactoryBean的区别及其各自的详细介绍于用法
    解决 vim 报错:the imp module is deprecated in favour of importlib
    SIFT了解,哪些方法可以在现在的AI算法中借鉴?
    CLAHE
    实际算法项目工程上手日志C/C++
    OS X 切换gcc版本
    opencv3.4.2 cmake错误:in-source builds are not allowed
    C++ opencv 计算两张图像的PSNR相似度
    如何在OS X 中使用markdown + latex混合记笔记?
    给anaconda 换源
  • 原文地址:https://www.cnblogs.com/flyingcr/p/10326987.html
Copyright © 2011-2022 走看看