zoukankan      html  css  js  c++  java
  • CDH4.5.0下安装lzo

    参考 http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/1.0.1/Installing-and-Using-Impala/ciiu_lzo.html

    sudo apt-get install liblzo2-dev
    wget http://archive-primary.cloudera.com/gplextras/ubuntu/precise/amd64/gplextras/pool/contrib/h/hadoop-lzo-cdh4/hadoop-lzo-cdh4_0.4.15.orig.tar.gz
    tar -zxvf hadoop-lzo-cdh4_0.4.15.orig.tar.gz
    cd hadoop-lzo-cdh4_0.4.15.orig
    ant package

    解决错误(http://hi.baidu.com/qingchunranzhi/item/72155fd1ccf5e05bd73aaebc)
    chmod 645 /home/hadoop/soft/hadoop-lzo-cdh4-0.4.15-gplextras/src/get_build_revision.sh
    要不报权限错误
    另外在普通用户下还是报权限错误, 在root用户下可以,然后再修改权限。

    cp hadoop-lzo-0.4.15.jar $HADOOP_HOME/share/hadoop/common/
    cp  build/native/Linux-*-*/lib/libgplcompression.* /home/hadoop/hadoop-2.0.0-cdh4.5.0/lib/native/

    应该就可以了

    使用

    conf.setBoolean("mapreduce.map.output.compress", true);
    conf.setClass("mapreduce.map.output.compression.codec", LzoCodec.class, CompressionCodec.class);
    conf.setBoolean("mapreduce.output.fileoutputformat.compress", true);
    conf.setClass("mapreduce.output.fileoutputformat.compress.codec", LzoCodec.class, CompressionCodec.class);

  • 相关阅读:
    几个数之和----数组刷题
    单调栈刷题
    腾讯金融科技凉经
    mysql刷题
    链表类题目常用方法
    阿里云一面凉经
    腾讯TEG一面凉经
    腾讯软件开发-后台开发实习生-一面凉经
    剑指 Offer 19. 正则表达式匹配
    剑指 Offer 20. 表示数值的字符串
  • 原文地址:https://www.cnblogs.com/chengxin1982/p/3875653.html
Copyright © 2011-2022 走看看