zoukankan      html  css  js  c++  java
  • CentOS6.7-64bit编译hadoop2.6.4

    1.下载maven(apache-maven-3.3.3-bin.tar.gz)
      http://archive.apache.org/dist/maven/maven-3/3.3.3/binaries/apache-maven-3.3.3-bin.tar.gz
    2.安装maven
      tar -zxvf apache-maven-3.3.3-bin.tar.gz -C /usr/local
    3.添加环境变量
    vim /etc/profile

      export JAVA_HOME=/usr/local/jdk1.7.0_45
      export MAVEN_HOME=/usr/local/apache-maven-3.3.3
      export PATH=$PATH:$JAVA_HOME/bin:$MAVEN_HOME/bin

    4.加载环境变量
    source /etc/proflie
    mvn -version

    5.下载protobuf

      (https://code.google.com/p/protobuf/downloads/list 或 https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz)

    6.安装protobuf编译依赖,为了编译安装protoc,需要使用YUM下载几个依赖的工具
      yum install -y gcc gcc-c++ make

    7.解压安装protobuf(root 用户下执行)
      tar -zxvf protobuf-2.5.0.tar.gz -C /usr/local/src
      cd /usr/local/src/protobuf-2.5.0
      ./configure --prefix=/usr/local/protobuf
      make && make install

    8.将protobuf添加到环境变量
      vim /etc/profile

      export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$MAVEN_HOME/bin:/usr/local/protobuf/bin

      source /etc/profile

      protoc --version

    9.安装其他编译依赖
      yum install -y cmake openssl-devel ncurses-devel zlib-devel
      yum install -y snappy snappy-devel bzip2 bzip2-devel lzo lzo-devel lzop openssl openssl-devel

    10.编译hadoop(进入文件夹里面,里面有一个文件BUILDINT.txt,打开即可看见里面关于编译hadoop的一些环境要求)
      tar -zxvf hadoop-2.6.4-src.tar.gz -C /usr/local/src/
      cd /usr/local/src/hadoop-2.6.4-src


      mvn package -Pdist,native -DskipTests -Dtar

    12.查看编译好的安装包
      cd /usr/local/src/hadoop-2.6.4-src/hadoop-dist/target


    #出现问题
    INFO util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

    在/etc/profile中添加
    export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native"

    #查看可用的本地库
    hadoop checknative -a

    cat /proc/cpuinfo | grep "physical id" | sort | uniq | wc –l
    cat /proc/cpuinfo | grep "core id" | sort -u | wc -l
    cat /proc/cpuinfo | grep MHz | uniq

  • 相关阅读:
    python之并发编程
    python之并发编程(理论部分)
    python之网络部分
    python之异常处理
    py之包和日志
    mysql之general log 日志
    python之代码规范
    文件权限管理命令chmod,chown与文本搜索命令grep
    Linux用户与组管理命令
    Linux常见文件管理命令
  • 原文地址:https://www.cnblogs.com/atomicbomb/p/10107034.html
Copyright © 2011-2022 走看看