zoukankan      html  css  js  c++  java
  • Hadoop---日志服务器

         Hadoop---日志服务器

     1.历史服务器:

       

      启动:

         

        1)测试使用:

          1.做一个任务:

           

          2.接受任务:

           

         3.执行任务: 

           

         4.执行任务完成:

           

        2)日志文件产生: 

          成功后:

            

          失败后:

              

          点击进入History,查看日志报错

       3)页面:

        

      

         只能记录以java开发程序。

     2.ApplicationHistoryServer日志记录:

        1)服务器搭建:

          在yarn-site.xml中添加配置

      <property>
          <description></description>
        <name>yarn.timeline-service.hostname</name>
        <value>hadoop1</value>
      </property>
      <property>
      <description>Address for the Timeline server to start the RPC server.</description>
      <name>yarn.timeline-service.address</name>
      <value>${yarn.timeline-service.hostname}:10200</value>
    </property>
    
    <property>
      <description>The http address of the Timeline service web application.</description>
      <name>yarn.timeline-service.webapp.address</name>
      <value>${yarn.timeline-service.hostname}:8188</value> #这里的8188:网页进程端口
    </property>
    
    <property>
      <description>The https address of the Timeline service web application.</description>
      <name>yarn.timeline-service.webapp.https.address</name>
      <value>${yarn.timeline-service.hostname}:8190</value>
    </property>
    
    <property>
      <description>Handler thread count to serve the client RPC requests.</description>
      <name>yarn.timeline-service.handler-thread-count</name>
      <value>10</value>
    </property>
    
    <property>
      <description>Enables cross-origin support (CORS) for web services where
      cross-origin web response headers are needed. For example, javascript making
      a web services request to the timeline server.</description>
      <name>yarn.timeline-service.http-cross-origin.enabled</name>
      <value>false</value>
    </property>
    
    <property>
      <description>Comma separated list of origins that are allowed for web
      services needing cross-origin (CORS) support. Wildcards (*) and patterns
      allowed</description>
      <name>yarn.timeline-service.http-cross-origin.allowed-origins</name>
      <value>*</value>
    </property>
    
    <property>
      <description>Comma separated list of methods that are allowed for web
      services needing cross-origin (CORS) support.</description>
      <name>yarn.timeline-service.http-cross-origin.allowed-methods</name>
      <value>GET,POST,HEAD</value>
    </property>
    
    <property>
      <description>Comma separated list of headers that are allowed for web
      services needing cross-origin (CORS) support.</description>
      <name>yarn.timeline-service.http-cross-origin.allowed-headers</name>
      <value>X-Requested-With,Content-Type,Accept,Origin</value>
    </property>
    
    <property>
      <description>The number of seconds a pre-flighted request can be cached
      for web services needing cross-origin (CORS) support.</description>
      <name>yarn.timeline-service.http-cross-origin.max-age</name>
      <value>1800</value>
    </property>
    <property>
      <description>Indicate to ResourceManager as well as clients whether
      history-service is enabled or not. If enabled, ResourceManager starts
      recording historical data that Timelien service can consume. Similarly,
      clients can redirect to the history service when applications
      finish if this is enabled.</description>
      <name>yarn.timeline-service.generic-application-history.enabled</name>
      <value>true</value>
    </property>
    
    <property>
      <description>Store class name for history store, defaulting to file system
      store</description>
      <name>yarn.timeline-service.generic-application-history.store-class</name>
      <value>org.apache.hadoop.yarn.server.applicationhistoryservice.FileSystemApplicationHistoryStore</value>
    </property>
    <property>
      <description>Indicate to clients whether Timeline service is enabled or not.
      If enabled, the TimelineClient library used by end-users will post entities
      and events to the Timeline server.</description>
      <name>yarn.timeline-service.enabled</name>
      <value>true</value>
    </property>
    
    <property>
      <description>Store class name for timeline store.</description>
      <name>yarn.timeline-service.store-class</name>
      <value>org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore</value>
    </property>
    
    <property>
      <description>Enable age off of timeline store data.</description>
      <name>yarn.timeline-service.ttl-enable</name>
      <value>true</value>
    </property>
    
    <property>
      <description>Time to live for timeline store data in milliseconds.</description>
      <name>yarn.timeline-service.ttl-ms</name>
      <value>6048000000</value>
    </property>

        2)启动服务器:

          

        3)测试:

          

          

        4)日志文件生产:

          

      总结:

        yarn端口:       8088:yarn的资源网络监控端口

               8030-8033:yarn的工作端口

               8018:新服务器的进程端口

                 19888:historyservice(老的服务器)的进程端口

  • 相关阅读:
    (原创)系统架构设计-通用权限模型设计①
    (原创)项目部署-Tomcat设置默认访问项目及项目重复加载问题处理
    安装在CloudStack时CentOS6.4中安装MySQL通过mysql_secure_installation方式修改密码
    (原创)VM中的CentOS6.4中安装CloudStack6.3②
    (原创)VM中的CentOS6.4中安装CloudStack6.3①
    S2SH+mysql-软件开发实际部署问题-8个小时后提示MYSQL数据库无法连接
    转---B/S结构JavaEE WebApp的全自动安装包制作心得
    javaEE-----org.springframework.dao.InvalidDataAccessApiUsageException: Write operation
    监控服务器Java异常脚本
    StringUtils.isNumeric("")竟然返回true
  • 原文地址:https://www.cnblogs.com/meiLinYa/p/9120455.html
Copyright © 2011-2022 走看看