zoukankan      html  css  js  c++  java
  • mesos集群安装,以及spark集群搭建

    一. 配置信息:
    3台centos7,一台master,两台slave。(无zookeeper安装)

    二. mesos安装:
    yum install http://repos.mesosphere.io/el/7/noarch/RPMS/mesosphere-el-repo-7-1.noarch.rpm
    yum install -y mesos


    Master:

    vim /etc/default/mesos-master
    PORT=5050
    IP={masterIp}

    vim /etc/mesos-master/cluster
    ClusterName

    vim /etc/mesos-master/hostname
    HostName

    systemctl start mesos-master.service
    systemctl enable mesos-master.service


    Slave:

    vim /etc/default/mesos-slave
    IP={slaveIp}
    MASTER={masterIp}:5050/mesos
    ISOLATION=cgroups/cpu,cgroups/mem,docker/runtime,filesystem/linux

    vim /etc/mesos-slave/hostname
    HostName

    systemctl start mesos-slave.service
    systemctl enable mesos-slave.service

    注:更改hostname,rm -rf /var/lib/mesos/* 后,重启。

    三. 环境变量配置:

    vim .bashrc
    export SPARK_HOME=/opt/spark-2.4.3-bin-hadoop2.7
    export PATH=$PATH:/opt/spark-2.4.3-bin-hadoop2.7/bin
    export PATH=$PATH:/opt/spark-2.4.3-bin-hadoop2.7/sbin
    export SPARK_LOCAL_IP=10.0.0.11
    export LIBPROCESS_IP=10.0.0.11

    [1] https://www.jianshu.com/p/22a5ed4db6a5
    [2] https://blog.csdn.net/qq_35440040/article/details/78427516
    [3] https://spark.apache.org/docs/latest/configuration.html
    [4] https://stackoverflow.com/questions/24637312/spark-driver-in-apache-spark


  • 相关阅读:
    iOS
    iOS
    iOS
    iOS
    iOS
    iOS
    iOS
    iOS
    iOS
    iOS
  • 原文地址:https://www.cnblogs.com/wubdut/p/11387071.html
Copyright © 2011-2022 走看看