zoukankan      html  css  js  c++  java
  • spark in eclipse---Spark学习笔记3

    想要调试源码,还是要放到eclipse里面去。先生成eclipse项目,下载依赖包

    victor@victor-ubuntu:~/software/incubator-spark-0.8.1-incubating$ mvn eclipse:eclipse
    [INFO] Scanning for projects...
    [INFO] ------------------------------------------------------------------------
    [INFO] Reactor Build Order:
    [INFO] 
    [INFO] Spark Project Parent POM
    [INFO] Spark Project Core
    [INFO] Spark Project Bagel
    [INFO] Spark Project Streaming
    [INFO] Spark Project ML Library
    [INFO] Spark Project Examples
    [INFO] Spark Project Tools
    [INFO] Spark Project REPL
    [INFO] Spark Project Assembly
    
    


    [INFO] 
    [INFO] --- maven-enforcer-plugin:1.1.1:enforce (enforce-versions) @ spark-assembly_2.9.3 ---
    [INFO] 
    [INFO] --- build-helper-maven-plugin:1.7:add-source (add-scala-sources) @ spark-assembly_2.9.3 ---
    [INFO] Source directory: /home/victor/software/incubator-spark-0.8.1-incubating/assembly/src/main/scala added.
    [INFO] 
    [INFO] --- maven-remote-resources-plugin:1.4:process (default) @ spark-assembly_2.9.3 ---
    [INFO] 
    [INFO] <<< maven-eclipse-plugin:2.9:eclipse (default-cli) @ spark-assembly_2.9.3 <<<
    [INFO] 
    [INFO] --- maven-eclipse-plugin:2.9:eclipse (default-cli) @ spark-assembly_2.9.3 ---
    [INFO] Using Eclipse Workspace: null
    [INFO] Adding default classpath container: org.eclipse.jdt.launching.JRE_CONTAINER
    [INFO] Wrote settings to /home/victor/software/incubator-spark-0.8.1-incubating/assembly/.settings/org.eclipse.jdt.core.prefs
    [INFO] Wrote Eclipse project for "spark-assembly_2.9.3" to /home/victor/software/incubator-spark-0.8.1-incubating/assembly.
    [INFO] 
    [INFO] ------------------------------------------------------------------------
    [INFO] Reactor Summary:
    [INFO] 
    [INFO] Spark Project Parent POM .......................... SUCCESS [1:39.289s]
    [INFO] Spark Project Core ................................ SUCCESS [17:19.163s]
    [INFO] Spark Project Bagel ............................... SUCCESS [2:20.763s]
    [INFO] Spark Project Streaming ........................... SUCCESS [4:54.565s]
    [INFO] Spark Project ML Library .......................... SUCCESS [1.011s]
    [INFO] Spark Project Examples ............................ SUCCESS [15:55.830s]
    [INFO] Spark Project Tools ............................... SUCCESS [0.889s]
    [INFO] Spark Project REPL ................................ SUCCESS [7.436s]
    [INFO] Spark Project Assembly ............................ SUCCESS [5.107s]
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 45:36.823s
    [INFO] Finished at: Sat Mar 29 00:29:46 CST 2014
    [INFO] Final Memory: 30M/202M
    [INFO] ------------------------------------------------------------------------
    

    生成成功。

    导入eclipse。 New->Scala Project->路径输入/home/victor/software/incubator-spark-0.8.1-incubating

    这个错误是编译器的问题,改动下scala配置
    Window->Preference->Scala Compiler->Build Manager 

    如图:



    把勾取消掉。 withVersionClassPathValidator 

    再增加build classpath,  Add Libary ->  scala  Libary.

    至此导入eclipse成功,而且能用快捷键来察看代码。跳转定义。





    好了。先写到这把。我看网上资料挺少的。还不如自己研究。


    下回我会写一下为什么前面 那个计算Pi的执行shell要那样写。

    good night!

    敬请期待。





  • 相关阅读:
    寒假日报day10
    寒假日报day9
    周计划06(20201026-20201101)
    周计划05(20201019-20201025)
    周总结2
    编程语言的实现模式读后感1
    软工总结
    哈夫曼编码算法
    hive表查询——排序
    假期总结4
  • 原文地址:https://www.cnblogs.com/yfceshi/p/7080183.html
Copyright © 2011-2022 走看看