zoukankan      html  css  js  c++  java
  • spark in eclipse---Spark学习笔记3

    想要调试源码,还是要放到eclipse里面去。先生成eclipse项目,下载依赖包

    victor@victor-ubuntu:~/software/incubator-spark-0.8.1-incubating$ mvn eclipse:eclipse
    [INFO] Scanning for projects...
    [INFO] ------------------------------------------------------------------------
    [INFO] Reactor Build Order:
    [INFO] 
    [INFO] Spark Project Parent POM
    [INFO] Spark Project Core
    [INFO] Spark Project Bagel
    [INFO] Spark Project Streaming
    [INFO] Spark Project ML Library
    [INFO] Spark Project Examples
    [INFO] Spark Project Tools
    [INFO] Spark Project REPL
    [INFO] Spark Project Assembly
    
    


    [INFO] 
    [INFO] --- maven-enforcer-plugin:1.1.1:enforce (enforce-versions) @ spark-assembly_2.9.3 ---
    [INFO] 
    [INFO] --- build-helper-maven-plugin:1.7:add-source (add-scala-sources) @ spark-assembly_2.9.3 ---
    [INFO] Source directory: /home/victor/software/incubator-spark-0.8.1-incubating/assembly/src/main/scala added.
    [INFO] 
    [INFO] --- maven-remote-resources-plugin:1.4:process (default) @ spark-assembly_2.9.3 ---
    [INFO] 
    [INFO] <<< maven-eclipse-plugin:2.9:eclipse (default-cli) @ spark-assembly_2.9.3 <<<
    [INFO] 
    [INFO] --- maven-eclipse-plugin:2.9:eclipse (default-cli) @ spark-assembly_2.9.3 ---
    [INFO] Using Eclipse Workspace: null
    [INFO] Adding default classpath container: org.eclipse.jdt.launching.JRE_CONTAINER
    [INFO] Wrote settings to /home/victor/software/incubator-spark-0.8.1-incubating/assembly/.settings/org.eclipse.jdt.core.prefs
    [INFO] Wrote Eclipse project for "spark-assembly_2.9.3" to /home/victor/software/incubator-spark-0.8.1-incubating/assembly.
    [INFO] 
    [INFO] ------------------------------------------------------------------------
    [INFO] Reactor Summary:
    [INFO] 
    [INFO] Spark Project Parent POM .......................... SUCCESS [1:39.289s]
    [INFO] Spark Project Core ................................ SUCCESS [17:19.163s]
    [INFO] Spark Project Bagel ............................... SUCCESS [2:20.763s]
    [INFO] Spark Project Streaming ........................... SUCCESS [4:54.565s]
    [INFO] Spark Project ML Library .......................... SUCCESS [1.011s]
    [INFO] Spark Project Examples ............................ SUCCESS [15:55.830s]
    [INFO] Spark Project Tools ............................... SUCCESS [0.889s]
    [INFO] Spark Project REPL ................................ SUCCESS [7.436s]
    [INFO] Spark Project Assembly ............................ SUCCESS [5.107s]
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 45:36.823s
    [INFO] Finished at: Sat Mar 29 00:29:46 CST 2014
    [INFO] Final Memory: 30M/202M
    [INFO] ------------------------------------------------------------------------
    

    生成成功。

    导入eclipse。 New->Scala Project->路径输入/home/victor/software/incubator-spark-0.8.1-incubating

    这个错误是编译器的问题,改动下scala配置
    Window->Preference->Scala Compiler->Build Manager 

    如图:



    把勾取消掉, withVersionClassPathValidator 

    再增加build classpath,  Add Libary ->  scala  Libary.

    至此导入eclipse成功,而且能用快捷键来察看代码,跳转定义。




    好了,先写到这把,我看网上资料挺少的,还不如自己研究。

    下回我会写一下为什么前面 那个计算Pi的执行shell要那样写。

    good night!

    敬请期待。



  • 相关阅读:
    1003 我要通过! (20 分)
    安装ANSYS19.0的正确方法(附下载)
    多项式最小二乘法拟合
    递归循环嵌套排列组合
    对二维数组使用指针进行操作的探索(C语言)
    统计C语言关键字出现次数
    三次样条插值matlab实现
    绩点换算小程序
    B1020 月饼(25 分)
    问题 B: 分组统计
  • 原文地址:https://www.cnblogs.com/mfrbuaa/p/4372490.html
Copyright © 2011-2022 走看看