zoukankan      html  css  js  c++  java
  • EMR 配置纪录(不断更新)


    日志路径 -> /var/log(软连接到 /mnt/var/log)

    在 spark master 机器中,配置的路径为

    /usr/lib/spark/conf
    /usr/lib/hadoop-yarn/etc/hadoop/
    

      

    其中关键的配置

    <property><name>yarn.nodemanager.local-dirs</name><value>/mnt/yarn</value></property>
    <property><name>yarn.nodemanager.log-dirs</name><value>/var/log/hadoop-yarn/containers</value></property>

    local-dirs: 应用的本地文件,官方解释为:

    List of directories to store localized files in. An application's localized file directory will be found in: ${yarn.nodemanager.local-dirs}/usercache/${user}/appcache/application_${appid}. Individual containers' work directories, called container_${contid}, will be subdirectories of this.


    log-dirs: 应用的日志路径,官方解释为:
    Where to store container logs. An application's localized log directory will be found in ${yarn.nodemanager.log-dirs}/application_${appid}. Individual containers' log directories will be below this, in directories named container_{$contid}. Each container directory will contain the files stderr, stdin, and syslog generated by that container.

  • 相关阅读:
    GDB 用法
    C编程规范
    PHP面向对象
    cron定时任务
    Apatche配置基础
    正则表达式笔记
    PHP在windows下命令行方式
    面试题
    struts与ajax的关系
    ORACLE DUAL表详解
  • 原文地址:https://www.cnblogs.com/keepthinking/p/10386803.html
Copyright © 2011-2022 走看看