zoukankan      html  css  js  c++  java
  • [导入]Eclipse 导入/编译 Hadoop 源码

    http://www.cnblogs.com/errorx/p/3779578.html

    1.准备工作

    • jdk:
    • eclipse:
    • Maven:
    • libprotoc :https://developers.google.com/protocol-buffers/
    • hadoop:http://www.apache.org/dyn/closer.cgi/hadoop/common/

    添加开源中国maven库:http://maven.oschina.net/home.html

    mavenconfsettings.xml

    <localRepository>path</localRepository>

    复制代码
    <localRepository>path</localRepository>
    <mirrors> <mirror> <id>nexus-osc</id> <mirrorOf>*</mirrorOf> <name>Nexus osc</name> <url>http://maven.oschina.net/content/groups/public/</url> </mirror> </mirrors>
    复制代码

    2.导入

    1. 将hadoop源码解压到一个目录,注意目录层次不要太深,否则可能无法解压。
    2. 进入hadoop-maven-plugins文件夹,执行 mvn install 
    3. 返回源码根目录,执行 mvn eclipse:eclipse –DskipTests 
    4. eclipse在任意目录创建新的WorkSpace
    5. eclipse设置Maven:window->preference->maven->{Installations...;user Settings:mavenconfsettings.xml}
    6. eclipse:File->inport->Existing Projects into WorkSpace->Hadoop源码根目录

    3.错误处理

      1. maven下载pom失败->重新操作
      2. hadoop-streaming中build path错误->Java Build Path->Source:
        1. 删除...hadoop-yarn-server-resourcemanager/conf
        2. Link Source:源码根目录/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf,再随便起个名字;inclusion patterns:capacity-scheduler.xml;exclusion patters:**/*.java
      3. org.apache.hadoop.io.serializer.avro.TestAvroSerialization
        1. 下载avro-tools-1.7.4.jar:http://archive.apache.org/dist/avro/avro-1.7.4/java/
        2. 进入目录:源码根目录hadoop-common-projecthadoop-commonsrc estavro
        3. java -jar path/to/avro-tools-1.7.4.jar compile schema avroRecord.avsc ..java
        4. eclipse 刷新
      4. org.apache.hadoop.ipc.protobuf.TestProtos
        1. 进入目录:源码根目录hadoop-common-projecthadoop-commonsrc estproto
        2. protoc --java_out=../java *.proto
        3. eclipse 刷新
      5. project->clean..->clean all projects & Build the entire workspace
  • 相关阅读:
    toj 2975 Encription
    poj 1797 Heavy Transportation
    toj 2971 Rotating Numbers
    zoj 2281 Way to Freedom
    toj 2483 Nasty Hacks
    toj 2972 MOVING DHAKA
    toj 2696 Collecting Beepers
    toj 2970 Hackle Number
    toj 2485 Card Tric
    js页面定位,相关几个属性
  • 原文地址:https://www.cnblogs.com/hark0623/p/5560881.html
Copyright © 2011-2022 走看看