zoukankan      html  css  js  c++  java
  • eclipse开发mapreduce程序时出现的问题

    1、报HDFS权限不够:org.apache.hadoop.security.AccessControlException: Permission denied:user=ouqiping, access=WRITE, inode="/user/Administrator/DatingRecommender/ratings.dat":root:supergroup:drwxr-xr-x

      解决办法:

      到服务器上修改hadoop的配置文件:conf/hdfs-core.xml, 找到 dfs.permissions 的配置项 , 将value值改为 false

      <property>
      <name>dfs.permissions</name>
      <value>false</value>
      
    <description>
      If "true", enable permission checking in HDFS.
      If "false", permission checking is turned off,
      but all other behavior is unchanged.
      Switching from one parameter value to the other does not change the mode,
      owner or group of files or directories.
      </description>
      </property>

    2、账户名字不匹配:org.apache.hadoop.security.AccessControlException: Permission denied: user=ouqiping, access=EXECUTE, inode="/tmp/hadoop-yarn/staging/ouqiping/.staging         /job_1458550621743_0001":root:supergroup:drwx------

      解决办法主要有6种:
      (1),更改linux上hadoop集群的名字为ouqiping
      (2),更改hadoop的hdfs所在的目录的权限为hadoop fs -chmod 777 /user/hadoop
      (3),关闭HDFS的权限认证机制,将dfs.permissions修改为False(经测试,无效,仅可以上传和下载)
      (4),更改Windows7的系统用户名为hadoop
      (5),在Win7上的环境变量中加入HADOOP_USER_NAME并配置在linux上对应的用户名即可
      (6),在提交程序里通过代码临时设置指定HADOOP_USER_NAME的名字和linux上的一致

  • 相关阅读:
    hadoop基础
    数据库基础知识
    sqoop基础
    大数据之常用linux常用命令
    zooKeeper基础
    impala基础
    Hbase基础
    Commitlint 提交规范类型
    理解JS闭包
    JS函数作用域及作用域链理解
  • 原文地址:https://www.cnblogs.com/julyme/p/5302680.html
Copyright © 2011-2022 走看看