zoukankan      html  css  js  c++  java
  • 使用 LZMA SDK 在 Java 中进行压缩导出(POI 如何快速导百万级数据的 Excel)

            //https://www.cnblogs.com/operationhome/p/12253549.html
            //https://cloud.tencent.com/developer/ask/57625(XZ 是基于 LZMA2(LZMA 的改进版本)的文件格式,考虑客户对 zip 压缩软件支持,暂时放弃)
            //压缩文件流(可以不用保存在本地再输出,通过 httpServletResponse 直接输出)
            ZipOutputStream zipOutputStream = new ZipOutputStream(httpServletResponse.getOutputStream());
            //构造流入对象
            ExcelWriter excelWriter = EasyExcel.write(zipOutputStream)
                    .excelType(ExcelTypeEnum.XLSX)
                    .head(ContractAccrualResponse.class)
                    .build();
            try {
                //文件名称
                String fileName = StringUtils.join("利息计提批量导出(", contractAccrualQueryRequest.getCorpCode(), ")", ExcelTypeEnum.XLSX.getValue());
                //对文件名进行编码处理中文问题
                String zipFileName = new String(("利息计提批量导出.zip").getBytes(), StandardCharsets.UTF_8);
                //设置响应头
                httpServletResponse.setCharacterEncoding("UTF-8");
    //            httpServletResponse.setContentType("application/vnd.ms-excel");
                httpServletResponse.setHeader("Content-disposition", "attachment;filename*=utf-8''" + URLEncoder.encode(zipFileName, "UTF-8"));
    //            for (int i = 0; i < 15; i++) {
    //                contractAccrualResponseList.addAll(contractAccrualResponseList);
    //            }
                //切割数据(XLSX 单个 sheet 只能容纳 1048576 行数据)
                int exportMaxNum = 1000000;
                List<List<ContractAccrualResponse>> partition = Lists.partition(contractAccrualResponseList, exportMaxNum);
                for (int i = 0; i < partition.size(); i++) {
                    String sheetName = StringUtils.join("Sheet_", String.valueOf(i * exportMaxNum + 1), "_", exportMaxNum <= partition.get(i).size() ? (i + 1) * exportMaxNum : (i + 1) * exportMaxNum + partition.get(i).size());
                    excelWriter.write(partition.get(i), EasyExcel.writerSheet(i, sheetName).build());
                }
                //将文件写入 zip 内,即将文件进行打包
                zipOutputStream.putNextEntry((new ZipEntry(fileName)));
                zipOutputStream.setComment(fileName);
            } catch (Exception exception) {
                log.error("exception: {}", exception.getMessage());
            } finally {
                if (ObjectUtils.isNotEmpty(excelWriter)) {
                    excelWriter.finish();
                }
                if (ObjectUtils.isNotEmpty(zipOutputStream)) {
                    zipOutputStream.finish();
                }
            }

    XZ

    FileInputStream inFile = new FileInputStream("src.tar");
    FileOutputStream outfile = new FileOutputStream("src.tar.xz");
    
    LZMA2Options options = new LZMA2Options();
    
    options.setPreset(7); // play with this number: 6 is default but 7 works better for mid sized archives ( > 8mb)
    
    XZOutputStream out = new XZOutputStream(outfile, options);
    
    byte[] buf = new byte[8192];
    int size;
    while ((size = inFile.read(buf)) != -1)
       out.write(buf, 0, size);
    
    out.finish();

    http://tukaani.org/xz/java.html

    https://www.cnblogs.com/operationhome/p/12253549.html

    http://commons.apache.org/proper/commons-compress/examples.html(xz 也可以使用 XZCompressorOutputStream 封装类)

    Apache Commons:简单的使用Compress创建7z压缩文件

    https://blog.csdn.net/weixin_45492007/article/details/118069006

    POI 如何快速导出千万级数据的 Excel

    https://www.v2ex.com/t/795136

    https://alibaba-easyexcel.github.io/quickstart/write.html 

  • 相关阅读:
    ORACLE 查看进程数,已执行任务数, 剩余任务数,删除指定任务
    ORACLE 收集统计整个用户数据
    解决Hystrix dashboard Turbine 一直 Loading…… 及其他坑
    利用 Maven 构造 Spring Cloud 微服务架构 模块使用 spring Boot构建
    AES加解密
    JAVA POI XSSFWorkbook导出扩展名为xlsx的Excel,附带weblogic 项目导出Excel文件错误的解决方案
    JAVA 文件的上传下载
    shell启停服务脚本模板
    JAVA 设计模式之 原型模式详解
    JAVA 设计模式之 工厂模式详解
  • 原文地址:https://www.cnblogs.com/Irving/p/15477415.html
Copyright © 2011-2022 走看看