zoukankan      html  css  js  c++  java
  • WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Exception in thread "main" java.io.IOException: No FileSystem for sc F

    1、执行脚本程序报如下所示的错误:

     1 [hadoop@slaver1 script_hadoop]$  hadoop jar web_click_mr_hive.jar com.bie.hive.mr.ClickStreamThree /home/hadoop/data_hadoop/weblog/preprocess/output/2018-05-19 /home/hadoop/data_hadoop/weblog/preprocess/click_pv_out/2018-05-19
     2 18/05/20 17:09:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
     3 Exception in thread "main" java.io.IOException: No FileSystem for scheme: C
     4     at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584)
     5     at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
     6     at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
     7     at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
     8     at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
     9     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
    10     at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
    11     at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.setInputPaths(FileInputFormat.java:498)
    12     at com.bie.hive.mr.ClickStreamThree.main(ClickStreamThree.java:207)
    13     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    14     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    15     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    16     at java.lang.reflect.Method.invoke(Method.java:606)
    17     at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
    18 [hadoop@slaver1 script_hadoop]$  hadoop jar web_click_mr_hive.jar com.bie.hive.mr.ClickStreamThree hdfs://slaver1:9000/home/hadoop/data_hadoop/weblog/preprocess/output/2018-05-19 hdfs://slaver1:9000/home/hadoop/data_hadoop/weblog/preprocess/click_pv_out/2018-05-19
    19 18/05/20 17:10:35 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    20 Exception in thread "main" java.io.IOException: No FileSystem for scheme: C
    21     at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584)
    22     at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
    23     at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
    24     at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
    25     at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
    26     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
    27     at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
    28     at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.setInputPaths(FileInputFormat.java:498)
    29     at com.bie.hive.mr.ClickStreamThree.main(ClickStreamThree.java:207)
    30     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    31     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    32     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    33     at java.lang.reflect.Method.invoke(Method.java:606)
    34     at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
    35 [hadoop@slaver1 script_hadoop]$ 

    2、猛一看,WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable,百度一搜这个错误,啥编译32/64位的问题,也有说改配置文件的问题,也许有的可能是这样改就解决问题了,而我想说,出现问题,看看报错原因,也许就可以很快的解决问题了。

    然后呢,看下代码,是如下所示导致的错误,将下面的注释了,代码换成上面的就可以使用脚本程序执行代码了:

    FileInputFormat.setInputPaths(job, new Path(args[0]));
    FileOutputFormat.setOutputPath(job, new Path(args[1]));
    
    //FileInputFormat.setInputPaths(job, new Path("C:/weblog/output"));
    //FileOutputFormat.setOutputPath(job, new Path("c:/weblog/pageviews"));
  • 相关阅读:
    P1474 货币系统 Money Systems(完全背包求填充方案数)
    Codeforces 877C Slava and tanks(思维)
    883H
    Codeforces 873B
    POJ 1733 Parity game(带权并查集)
    POJ 2387 Til the Cows Come Home(dijkstra裸题)
    POJ 1984 Navigation Nightmare(二维带权并查集)
    POJ 2912 Rochambeau(种类并查集+枚举)
    POJ 1456 Supermarket(贪心+并查集)
    POJ 2492 A Bug's Life(带权并查集)
  • 原文地址:https://www.cnblogs.com/biehongli/p/9063952.html
Copyright © 2011-2022 走看看