zoukankan      html  css  js  c++  java
  • hadoop中执行命令时发生错误

    通过一下方式找到错误的原因,开启hadoop的调试信息

    [root@yts bin]# export HADOOP_ROOT_LOGGER=DEBUG,console

    这样在执行命令时,可以通过error字样定位执行命令时产生错误的原因

    [root@yts bin]# ./hadoop fs -mkdir test
    14/10/08 11:17:55 DEBUG util.Shell: setsid exited with exit code 0
    14/10/08 11:17:56 DEBUG conf.Configuration: parsing URL jar:file:/usr/local/hadoop-2.5.1/share/hadoop/common/hadoop-common-2.5.1.jar!/core-default.xml
    14/10/08 11:17:56 DEBUG conf.Configuration: parsing input stream sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@66048bfd
    14/10/08 11:17:56 DEBUG conf.Configuration: parsing URL file:/usr/local/hadoop-2.5.1/etc/hadoop/core-site.xml
    14/10/08 11:17:56 DEBUG conf.Configuration: parsing input stream java.io.BufferedInputStream@340f438e
    14/10/08 11:17:56 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of successful kerberos logins and latency (milliseconds)])
    14/10/08 11:17:56 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of failed kerberos logins and latency (milliseconds)])
    14/10/08 11:17:56 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[GetGroups])
    14/10/08 11:17:56 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics
    14/10/08 11:17:56 DEBUG security.Groups: Creating new Groups object
    14/10/08 11:17:56 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
    14/10/08 11:17:56 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: /usr/local/hadoop-2.5.1/lib/native/libhadoop.so.1.0.0: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /usr/local/hadoop-2.5.1/lib/native/libhadoop.so.1.0.0)
    14/10/08 11:17:56 DEBUG util.NativeCodeLoader: java.library.path=/usr/local/hadoop-2.5.1/lib/native
    14/10/08 11:17:56 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    14/10/08 11:17:56 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Falling back to shell based
    14/10/08 11:17:56 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
    14/10/08 11:17:56 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
    14/10/08 11:17:56 DEBUG security.UserGroupInformation: hadoop login
    14/10/08 11:17:56 DEBUG security.UserGroupInformation: hadoop login commit
    14/10/08 11:17:56 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: root
    14/10/08 11:17:56 DEBUG security.UserGroupInformation: UGI loginUser:root (auth:SIMPLE)
    14/10/08 11:17:56 DEBUG hdfs.BlockReaderLocal: dfs.client.use.legacy.blockreader.local = false
    14/10/08 11:17:56 DEBUG hdfs.BlockReaderLocal: dfs.client.read.shortcircuit = false
    14/10/08 11:17:56 DEBUG hdfs.BlockReaderLocal: dfs.client.domain.socket.data.traffic = false
    14/10/08 11:17:56 DEBUG hdfs.BlockReaderLocal: dfs.domain.socket.path =
    14/10/08 11:17:57 DEBUG retry.RetryUtils: multipleLinearRandomRetry = null
    14/10/08 11:17:57 DEBUG ipc.Server: rpcKind=RPC_PROTOCOL_BUFFER, rpcRequestWrapperClass=class org.apache.hadoop.ipc.ProtobufRpcEngine$RpcRequestWrapper, rpcInvoker=org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker@55a1c291
    14/10/08 11:17:57 DEBUG ipc.Client: getting client out of cache: org.apache.hadoop.ipc.Client@48524010
    14/10/08 11:17:57 DEBUG shortcircuit.DomainSocketFactory: Both short-circuit local reads and UNIX domain socket are disabled.
    14/10/08 11:17:57 DEBUG ipc.Client: The ping interval is 60000 ms.
    14/10/08 11:17:57 DEBUG ipc.Client: Connecting to yts/192.168.3.82:9000
    14/10/08 11:17:57 DEBUG ipc.Client: IPC Client (124407148) connection to yts/192.168.3.82:9000 from root: starting, having connections 1
    14/10/08 11:17:57 DEBUG ipc.Client: IPC Client (124407148) connection to yts/192.168.3.82:9000 from root sending #0
    14/10/08 11:17:57 DEBUG ipc.Client: IPC Client (124407148) connection to yts/192.168.3.82:9000 from root got value #0
    14/10/08 11:17:57 DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 143ms
    14/10/08 11:17:57 DEBUG ipc.Client: IPC Client (124407148) connection to yts/192.168.3.82:9000 from root sending #1
    14/10/08 11:17:57 DEBUG ipc.Client: IPC Client (124407148) connection to yts/192.168.3.82:9000 from root got value #1
    14/10/08 11:17:57 DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 2ms
    mkdir: `test': No such file or directory
    14/10/08 11:17:57 DEBUG ipc.Client: stopping client from cache: org.apache.hadoop.ipc.Client@48524010
    14/10/08 11:17:57 DEBUG ipc.Client: removing client from cache: org.apache.hadoop.ipc.Client@48524010
    14/10/08 11:17:57 DEBUG ipc.Client: stopping actual client because no more references remain: org.apache.hadoop.ipc.Client@48524010
    14/10/08 11:17:57 DEBUG ipc.Client: Stopping client
    14/10/08 11:17:57 DEBUG ipc.Client: IPC Client (124407148) connection to yts/192.168.3.82:9000 from root: closed
    14/10/08 11:17:57 DEBUG ipc.Client: IPC Client (124407148) connection to yts/192.168.3.82:9000 from root: stopped, remaining connections 0

  • 相关阅读:
    自定义View的ToolBar布局报错Error:(2) No resource identifier found for attribute 'context' in package 'c
    在学git之主分支 branch
    获取发布版SHA1
    关于开启线程与UI的操作
    播放音频和视频(VideoView控件)
    通知栏Notification的应用
    Android 真机调式 Installation failed with message 远程主机强迫关闭了一个现有的连接。. It is possible that this issue is resolved by uninstalling an existing version of the apk if it is present, and then re-installing. WA
    运行程序申请危险权限
    mysql乐观锁总结和实践
    Nginx配置文件nginx.conf中文详解
  • 原文地址:https://www.cnblogs.com/yts1dx/p/4011019.html
Copyright © 2011-2022 走看看