zoukankan      html  css  js  c++  java
  • Windows 下部署调试 Hadoop 时关于 java.io.FileNotFoundException 问题总结

    折腾了差不多三天,终于从网上找到了相关的解决方案:

    问题描述:配置搭建好 Windows + Hadoop + Cygwin 之后,一切运行顺利,就是在调试下面的小 sample 的时候总是报如下错误:

    Samples:

    [//下面的实例将已解压的 conf 目录拷贝作为输入,查找并显示匹配给定正则表达式的条目。输出写入到指定的output目录。
    $ mkdir input
    $ cp conf/*.xml input
    $ bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'
    $ cat output/*
    ]

    错误信息:

    $ bin/hadoop jar hadoop-0.20.2-examples.jar grep /input/*.xml /output 'dfs[a-z.]+'
    12/09/11 14:24:01 INFO mapred.FileInputFormat: Total input paths to process : 5
    12/09/11 14:24:02 INFO mapred.JobClient: Running job: job_201209111241_0002
    12/09/11 14:24:03 INFO mapred.JobClient:  map 0% reduce 0%
    12/09/11 14:24:12 INFO mapred.JobClient: Task Id : attempt_201209111241_0002_m_000006_0, Status : FAILED
    java.io.FileNotFoundException: File C:/tmp/hadoop-tiky.zheng/mapred/local/taskTracker/jobcache/job_201209111241_0002/attempt_201209111241_0002_m_000006_0/work/tmp does not exist.
            at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:361)
            at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
            at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519)
            at org.apache.hadoop.mapred.Child.main(Child.java:155)

    12/09/11 14:24:21 INFO mapred.JobClient: Task Id : attempt_201209111241_0002_m_000006_1, Status : FAILED
    java.io.FileNotFoundException: File C:/tmp/hadoop-tiky.zheng/mapred/local/taskTracker/jobcache/job_201209111241_0002/attempt_201209111241_0002_m_000006_1/work/tmp does not exist.
            at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:361)
            at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
            at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519)
            at org.apache.hadoop.mapred.Child.main(Child.java:155)

    最后找到答案:

    需要在mapred-site.xml中添加暂存目录:

      <name>mapred.child.tmp</name>

      <value>/home/hadoop-0.20.1/temp</value>

    加完,搞定!

  • 相关阅读:
    JAVA-初步认识-常用对象API(StringBuffer类-插入删除查找修改)
    JAVA-初步认识-常用对象API(StringBuffer类-特点添加功能)
    JAVA-初步认识-常用对象API(String类-练习1)
    JAVA-初步认识-常用对象API(String类-常见功能-intern方法)
    JAVA-初步认识-常用对象API(String类-常见功能-比较)
    JAVA-初步认识-常用对象API(String类-常见功能-判断)
    JAVA-初步认识-常用对象API(String类-常见功能-转换)
    JAVA-初步认识-常用对象API(String类-常见功能-获取-2)
    arcengine导出复本
    PC软件分享
  • 原文地址:https://www.cnblogs.com/TikyZheng/p/2680207.html
Copyright © 2011-2022 走看看