Spark集群搭建_YARN
前提:参考Spark集群搭建_Standalone
1.修改spark中conf中的spark-env.sh

2.Spark on YARN--不需要启动Spark集群,因为已经在spark的配置文件中配置了hadoop的目录

附录:
不同运行模式的命令不同
1.standalone client模式
./bin/spark-submit --class org.apache.spark.examples.SparkPi--master spark://master:7077 --executor-memory 512m --total-executor-cores 1 ./lib/spark-examples-1.6.0-hadoop2.6.0.jar 100
2.standalone cluster模式
./bin/spark-submit --class org.apache.spark.examples.SparkPi--master spark://spark001:7077 --driver-memory 512m --deploy-mode cluster --supervise --executor-memory 512M --total-executor-cores 1 ./lib/spark-examples-1.6.0-hadoop2.6.0.jar 100
3.on yarn client模式
./bin/spark-submit --class org.apache.spark.examples.SparkPi--master yarn-client --executor-memory 512M--num-executors 1./lib/spark-examples-1.6.0-hadoop2.6.0.jar 100
4.on yarn cluster模式
./bin/spark-submit --class org.apache.spark.examples.SparkPi--master yarn-cluster --executor-memory 512m--num-executors 1./lib/spark-examples-1.6.0-hadoop2.6.0.jar 100
web端口查看Spark的运行



