zoukankan      html  css  js  c++  java
  • spark in eclipse---Spark学习笔记3

    想要调试源码,还是要放到eclipse里面去。先生成eclipse项目,下载依赖包

    victor@victor-ubuntu:~/software/incubator-spark-0.8.1-incubating$ mvn eclipse:eclipse
    [INFO] Scanning for projects...
    [INFO] ------------------------------------------------------------------------
    [INFO] Reactor Build Order:
    [INFO] 
    [INFO] Spark Project Parent POM
    [INFO] Spark Project Core
    [INFO] Spark Project Bagel
    [INFO] Spark Project Streaming
    [INFO] Spark Project ML Library
    [INFO] Spark Project Examples
    [INFO] Spark Project Tools
    [INFO] Spark Project REPL
    [INFO] Spark Project Assembly
    
    


    [INFO] 
    [INFO] --- maven-enforcer-plugin:1.1.1:enforce (enforce-versions) @ spark-assembly_2.9.3 ---
    [INFO] 
    [INFO] --- build-helper-maven-plugin:1.7:add-source (add-scala-sources) @ spark-assembly_2.9.3 ---
    [INFO] Source directory: /home/victor/software/incubator-spark-0.8.1-incubating/assembly/src/main/scala added.
    [INFO] 
    [INFO] --- maven-remote-resources-plugin:1.4:process (default) @ spark-assembly_2.9.3 ---
    [INFO] 
    [INFO] <<< maven-eclipse-plugin:2.9:eclipse (default-cli) @ spark-assembly_2.9.3 <<<
    [INFO] 
    [INFO] --- maven-eclipse-plugin:2.9:eclipse (default-cli) @ spark-assembly_2.9.3 ---
    [INFO] Using Eclipse Workspace: null
    [INFO] Adding default classpath container: org.eclipse.jdt.launching.JRE_CONTAINER
    [INFO] Wrote settings to /home/victor/software/incubator-spark-0.8.1-incubating/assembly/.settings/org.eclipse.jdt.core.prefs
    [INFO] Wrote Eclipse project for "spark-assembly_2.9.3" to /home/victor/software/incubator-spark-0.8.1-incubating/assembly.
    [INFO] 
    [INFO] ------------------------------------------------------------------------
    [INFO] Reactor Summary:
    [INFO] 
    [INFO] Spark Project Parent POM .......................... SUCCESS [1:39.289s]
    [INFO] Spark Project Core ................................ SUCCESS [17:19.163s]
    [INFO] Spark Project Bagel ............................... SUCCESS [2:20.763s]
    [INFO] Spark Project Streaming ........................... SUCCESS [4:54.565s]
    [INFO] Spark Project ML Library .......................... SUCCESS [1.011s]
    [INFO] Spark Project Examples ............................ SUCCESS [15:55.830s]
    [INFO] Spark Project Tools ............................... SUCCESS [0.889s]
    [INFO] Spark Project REPL ................................ SUCCESS [7.436s]
    [INFO] Spark Project Assembly ............................ SUCCESS [5.107s]
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 45:36.823s
    [INFO] Finished at: Sat Mar 29 00:29:46 CST 2014
    [INFO] Final Memory: 30M/202M
    [INFO] ------------------------------------------------------------------------
    

    生成成功。

    导入eclipse。 New->Scala Project->路径输入/home/victor/software/incubator-spark-0.8.1-incubating

    这个错误是编译器的问题,改动下scala配置
    Window->Preference->Scala Compiler->Build Manager 

    如图:



    把勾取消掉。 withVersionClassPathValidator 

    再增加build classpath,  Add Libary ->  scala  Libary.

    至此导入eclipse成功,而且能用快捷键来察看代码。跳转定义。





    好了。先写到这把。我看网上资料挺少的。还不如自己研究。


    下回我会写一下为什么前面 那个计算Pi的执行shell要那样写。

    good night!

    敬请期待。





  • 相关阅读:
    「小程序JAVA实战」小程序视频封面处理(48)
    「小程序JAVA实战」小程序视频处理工具ffmpeg(47)
    「小程序JAVA实战」小程序上传短视频(46)
    微信小程序跳转navigateTo与redirectTo
    微信小程序中this关键字使用技巧
    ***电商数据库设计参考:ecshop数据库+订单表结构等
    MYSQL中'TYPE=MyISAM'错误的解决方案
    PHP实现执行定时任务的几种思路详解
    PHP定时器实现每隔几秒运行一次
    ***LINUX添加PHP环境变量:CentOS下将php和mysql命令加入到环境变量中
  • 原文地址:https://www.cnblogs.com/yfceshi/p/7080183.html
Copyright © 2011-2022 走看看