bin/spark-submit --help
... ...
--deploy-mode DEPLOY_MODE Whether to launch the driver program locally ("client") or
on one of the worker machines inside the cluster ("cluster")
(Default: client).
... ...
--deploy-mode定义了spark运行时的两种方式:
1)client模式:
spark应用driver运行在本地,应用输出日志可以在本地查看。
2)cluster模式:
spark应用driver运行在cluster中一台节点上,应用输出日志只能通过ui界面查看。
例1:spark运行在client模式下:
bin/hdfs dfs -mkdir -p /user/beifeng/mapreduce/wordcount/
bin/spark-submit
--master spark://chavin.king:7077
--deploy-mode client
jars/sparkApp.jar
例2:spark运行在cluster模式下:
bin/hdfs dfs -mkdir -p /user/beifeng/mapreduce/wordcount/
bin/spark-submit
--master spark://chavin.king:7077
--deploy-mode cluster
jars/sparkApp.jar