zoukankan      html  css  js  c++  java
  • mongodb 压缩——3.0+支持zlib和snappy

    转自:https://scalegrid.io/blog/enabling-data-compression-in-mongodb-3-0/

    MongoDB 3.0 with the wired tiger storage engine enables you to transparently compress the data stored in your database. This is a fairly exciting and useful feature that can be used to reduce the disk space usage of your fast growing data. By default wired tiger uses the ‘Snappy’ block compression engine for all the collections. You can turn off compression by default using the following options in the mongodb server config file .

    storage:
      engine: wiredTiger
      wiredTiger:
        collectionConfig:
          blockCompressor: none
    

    The compression algorithm can be specified at the collection level during cluster creation. Here is an example of creating a collection with ‘zlib’ compression

    db.createCollection( "test", {storageEngine:{wiredTiger:{configString:'block_compressor=zlib'}}} );
    

    MongoDB wiredtiger storage engine provides two options for compression – snappy and zlib. There is essentially a tradeoff between the extent of compression and the amount of CPU load to decompress. ‘Zlib’ achieves a lot more compression and is correspondingly less performant. ‘Snappy’ aims for ‘aims for very high speeds and reasonable compression’.

    We ran some simple unscientific tests to measure the compression performance. We used one of data sets storing strings which we felt would compress well. Here is the basic structure of each document.

    {
    '_id': <ObjectID>,
    'name': <Five character string>,
    'value': <Random 1MB string>
    }
    
    

    We inserted about 5000 of these documents (about 5GB of data). The results were fairly impressive. Zlib achieves considerable amount of compression. Snappy also achieves a fair amount of compression with little or no load on the system.

      Zlib Snappy uncompressed
    Data size (MB) 5000.5 5000.5 5000.5
    Storage size (MB) 19.62 254.37 5019

    MongoDB 3.0 snappy vs zlib vs uncompressed

    As always you need to run some tests to understand the performance gains for your data set.  Here are some more detailed benchmark studies on compression performance and tradeoffs

    http://www.mongodb.com/blog/post/new-compression-options-mongodb-30
    http://www.acmebenchmarking.com/2015/02/mongodb-v30-compression-benchmarks.html
    https://comerford.cc/2015/02/04/mongodb-3-0-testing-compression/

  • 相关阅读:
    简单的进度条程序
    python装饰器
    冒泡,递归
    实现随机验证码
    三木运算,三元运算
    set()集合
    list()列表
    python3.5 Str类型与bytes类型转换
    说说编码与转义的区别
    Java 中字符编码相关的问题解析
  • 原文地址:https://www.cnblogs.com/bonelee/p/6289874.html
Copyright © 2011-2022 走看看