zoukankan      html  css  js  c++  java
  • building hadoop2.4.1 on centos7[在centos7上面构建hadoop2.4.1]

      本文介绍在centos7上面通过hadoop2.4.1源码构建hadoop distribution 版本,即hadoop的运行版本。

      为何要自己building,而不用Apache的distribution 【bin】版本,因为hadoop涉及到Linux系统的底层实现,如:

        hadoop fs -ls /

      通过 java Native 实现,故应为 native的相关lib。【个人认为的一点】

      相关文件下载链接:

        文件目录解释:

    •  hadoop
      • hadoop-2.4.1.tar.gz 本人通过在centos7上构建的distribution版本,可以直接下载并配置,然后运行即可,参考文章
      • hadoop-2.4.1-src.tar.gz hadoop的源代码
    •  relative_tools
      • maven  【只需要配置 MAVEN_HADOOP 的环境变量】
      • protocolbuf   [即:google protocolBUffer],需要自己构建,并检查protocolbuf是否已配置环境变量
      • findbugs  需要自己构建,并检查protocolbuf是否已配置环境变量
      • Cmake 自己搜索,并构建配置环境变量
      • Requirements:
        
        * Unix System
        * JDK 1.6+
        * Maven 3.0 or later
        * Findbugs 1.3.9 (if running findbugs)
        * ProtocolBuffer 2.5.0
        * CMake 2.6 or newer (if compiling native code)
        * Internet connection for first build (to fetch all Maven and Hadoop dependencies)
        
        
        
        ----------------------------------------------------------------------------------
        Maven build goals:
        
         * Clean                     : mvn clean
         * Compile                   : mvn compile [-Pnative]
         * Run tests                 : mvn test [-Pnative]
         * Create JAR                : mvn package
         * Run findbugs              : mvn compile findbugs:findbugs
         * Run checkstyle            : mvn compile checkstyle:checkstyle
         * Install JAR in M2 cache   : mvn install
         * Deploy JAR to Maven repo  : mvn deploy
         * Run clover                : mvn test -Pclover [-DcloverLicenseLocation=${user.name}/.clover.license]
         * Run Rat                   : mvn apache-rat:check
         * Build javadocs            : mvn javadoc:javadoc
         * Build distribution        : mvn package [-Pdist][-Pdocs][-Psrc][-Pnative][-Dtar]
         * Change Hadoop version     : mvn versions:set -DnewVersion=NEWVERSION
        
         Build options:
        
          * Use -Pnative to compile/bundle native code
          * Use -Pdocs to generate & bundle the documentation in the distribution (using -Pdist)
          * Use -Psrc to create a project source TAR.GZ
          * Use -Dtar to create a TAR with the distribution (using -Pdist)
        
        
        ----------------------------------------------------------------------------------
        Importing projects to eclipse
        
        When you import the project to eclipse, install hadoop-maven-plugins at first.
        
          $ cd hadoop-maven-plugins
          $ mvn install
        
        Then, generate eclipse project files.
        
          $ mvn eclipse:eclipse -DskipTests
        
        At last, import to eclipse by specifying the root directory of the project via
        [File] > [Import] > [Existing Projects into Workspace].
        
        ----------------------------------------------------------------------------------
        Building distributions:
        
        Create binary distribution without native code and without documentation:
        
          $ mvn package -Pdist -DskipTests -Dtar
        
        Create binary distribution with native code and with documentation:
        
          $ mvn package -Pdist,native,docs -DskipTests -Dtar
        
        Create source distribution:
        
          $ mvn package -Psrc -DskipTests
        
        Create source and binary distributions with native code and documentation:
        
          $ mvn package -Pdist,native,docs,src -DskipTests -Dtar
        
        Create a local staging version of the website (in /tmp/hadoop-site)
        
          $ mvn clean site; mvn site:stage -DstagingDirectory=/tmp/hadoop-site
    • 只要上述的Linux系统、cmake、maven、findbuds、protocolbuf、网络连通、java都已构建/安装成功后等条件满足后就在hadoop源代码目录运行代码区的着色区域

    文件下载链接:http://pan.baidu.com/s/1pJJQkPl

  • 相关阅读:
    Source Insight新建工程文件
    移植3.4.2内核之韦东山笔记
    移植最新u-boot(裁剪和修改默认参数)
    C++与C语言在结构体上的区别
    VIP之Switch
    VIP之CSC
    VIP之Scaler
    VIP之FrameBuffer
    VIP之MixerII
    VIP之Clipper
  • 原文地址:https://www.cnblogs.com/lkzf/p/3879456.html
Copyright © 2011-2022 走看看