zoukankan      html  css  js  c++  java
  • cdh 系统配置文件更新 error parsing conf core-site.xml

    环境描述:redhat7.3  CDH5.15.1  采用parcels方式部署

    报错描述:airflow调度程序,最近2周偶尔报错,报错类型有2类:1、无法初始化集群配置;2、读取配置权限问题

    报错一:

    Launching Job 1 out of 1
    Number of reduce tasks is set to 0 since there's no reduce operator
    java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.
    at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:143)
    at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:108)
    at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:101)
    at org.apache.hadoop.mapred.JobClient.init(JobClient.java:477)
    at org.apache.hadoop.mapred.JobClient.(JobClient.java:455)

    报错二

    19/12/21 04:06:27 ERROR conf.Configuration: error parsing conf core-site.xml
    java.io.FileNotFoundException: /etc/hive/conf.cloudera.hive/core-site.xml (Permission denied)
    at java.io.FileInputStream.open0(Native Method)
    at java.io.FileInputStream.open(FileInputStream.java:195)
    at java.io.FileInputStream.(FileInputStream.java:138)
    at java.io.FileInputStream.(FileInputStream.java:93)
    at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)

    报错三

    setfacl: Permission denied. user=dip is not the owner of inode=.hive-staging_hive_2019-12-22_07-44-25_997_2557429548076828737-1
    java.lang.RuntimeException: java.io.FileNotFoundException: /etc/hive/conf.cloudera.hive/hive-site.xml (Permission denied)
    at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2811)
    at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2663)
    at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2559)
    at org.apache.hadoop.conf.Configuration.get(Configuration.java:1340)
    at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:2756)
    at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:2777)
    at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:2849)
    at org.apache.hadoop.hive.conf.HiveConf.(HiveConf.java:2792)

    检查集群中的所有机器,发现有一台机器配置文件一直在疯狂更新:

    /etc/hive/conf.cloudera.hive

    /etc/hadoop/conf.cloudera.yarn

    /etc/hbase/conf.cloudera.hbase

     程序报错原因

    配置文件每分钟更新一次,而程序在运行中会读取配置文件,也许在某次读取中,他正处于跟新状态,不可读。

    配置文件疯狂更新原因:

    检查:

    该机器: /var/lib/alternatives  下面有空文件

    操作:(删除下面的文件,重启agent后,agent会把需要的文件重新拷贝过来)

    删除/var/lib/alternatives  下面所有文件,

    重启agent

    cd /var/lib/alternatives
    
    rm  -rf   *
    
    systemctl restart cloudera-scm-agent.service

    观察 : 配置文件不再疯狂更新,问题解决。

  • 相关阅读:
    执行序列oracle存储过程和序列化写的demo
    快捷键列表myeclipse 设置代码提示快捷键
    鼠标管理解决win8 插上usb/鼠标蓝屏或无效方法
    JQuery实现拼图数字游戏
    Django的admin定制
    Django报:AttributeError: tuple object has no attribute get
    Django的models方法返回值异常,待解决
    主页跳转子页面的时候,模板语句中的数据未返回到页面(子页面空白)
    Django报:builtin_function_or_method' object is not iterable
    Windows Azure SDK 1.5、Windows Azure Tools for Microsoft Visual Studio 2010和新的服务管理功能发布了
  • 原文地址:https://www.cnblogs.com/students/p/12092728.html
Copyright © 2011-2022 走看看