zoukankan      html  css  js  c++  java
  • poi大数据导入解决方法

    This one comes up quite a lot, but often the reason isn't what you might initially think. So, the first thing to check is - what's the source of the problem? Your file? Your code? Your environment? Or Apache POI?

    (If you're here, you probably think it's Apache POI. However, it often isn't! A moderate laptop, with a decent but not excessive heap size, from a standing start, can normally read or write a file with 100 columns and 100,000 rows in under a couple of seconds, including the time to start the JVM).

    Apache POI ships with a few programs and a few example programs, which can be used to do some basic performance checks. For testing file generation, the class to use is in the examples package, SSPerformanceTest. Run SSPerformanceTest with arguments of the writing type (HSSF, XSSF or SXSSF), the number rows, the number of columns, and if the file should be saved. If you can't run that with 50,000 rows and 50 columns in HSSF and SXSSF in under 3 seconds, and XSSF in under 10 seconds (and ideally all 3 in less than that!), then the problem is with your environment.

    Next, use the example program ToCSV to try reading the a file in with HSSF or XSSF. Related is XLSX2CSV, which uses SAX parsing for .xlsx. Run this against both your problem file, and a simple one generated by SSPerformanceTest of the same size. If this is slow, then there could be an Apache POI problem with how the file is being processed (POI makes some assumptions that might not always be right on all files). If these tests are fast, then any performance problems are in your code!

  • 相关阅读:
    4.定时器
    LSTM分类层的理解
    对机器学习中end-to-end learning(端到端学习)的理解
    对非极大值限制(NMS)的理解
    ROI pooling
    softmax的理解
    长尾理论(long tail )的理解
    对双线性插值的理解
    RPN的理解
    卷积神经网络(CNN)和Faster-RCNN的理解
  • 原文地址:https://www.cnblogs.com/toSeeMyDream/p/5238960.html
Copyright © 2011-2022 走看看