zoukankan      html  css  js  c++  java
  • docker 搭建spark集群

    借鉴:https://blog.csdn.net/u013705066/article/details/80030732
    下面这个例子让我感受到docker的便捷和强大。
    两行命令搭建spark集群:

    1.准备一个docker-compose.yml配置文件,描述集群结构及必要参数
    version: "2"
    services:
      master:
        image: singularities/spark
        command: start-spark master
        hostname: master
        ports:
          - "6066:6066"
          - "7070:7070"
          - "8080:8080"
          - "50070:50070"
      worker:
        image: singularities/spark
        command: start-spark worker master
        environment:
          SPARK_WORKER_CORES: 1
          SPARK_WORKER_MEMORY: 2g
        links:
          - master

    2.执行命令拉取spark镜像到本地(该步骤可省略):

    docker pull singularities/spark

    3.在配置文件所在目录启动

    docker-compose up -d
    4.查看启动结果



     浏览器访问:http:/{docker_host}:50070











    附docker基本命令:

    docker build -t friendlyhello . # Create image using this directory's Dockerfile docker run -p 4000:80 friendlyhello # Run "friendlyhello" mapping port 4000 to 80 docker run -d -p 4000:80 friendlyhello # Same thing, but in detached mode docker container ls # List all running containers docker container ls -a # List all containers, even those not running docker container stop <hash> # Gracefully stop the specified container docker container kill <hash> # Force shutdown of the specified container docker container rm <hash> # Remove specified container from this machine docker container rm $(docker container ls -a -q) # Remove all containers docker image ls -a # List all images on this machine docker image rm <image id> # Remove specified image from this machine docker image rm $(docker image ls -a -q) # Remove all images from this machine docker login # Log in this CLI session using your Docker credentials docker tag <image> username/repository:tag # Tag <image> for upload to registry docker push username/repository:tag # Upload tagged image to registry docker run username/repository:tag # Run image from a registry

    拷贝自官网.
  • 相关阅读:
    ubuntu15.04下安装配置docker
    在Vmware/VirtualBox虚拟机环境中正确打开虚拟机
    windows下使用cx_freeze将Python程序打包成exe可执行文件
    Python正则表达式学习小结
    Ubuntu环境下利用Python3+PyQt5+Eric6进行GUI编程
    C#学习笔记——控件篇
    商务英语学习第一课(lending(借贷))
    Ajax的请求方式:get和post
    XML就像是扩展应用范围的HTML
    JQ数组=jQuery对象数组
  • 原文地址:https://www.cnblogs.com/sen-2017/p/11412894.html
Copyright © 2011-2022 走看看