zoukankan      html  css  js  c++  java
  • flink 读取文件报错 Line could not be encoded

    Caused by: java.lang.RuntimeException: Line could not be encoded: [49, 56, 49, 49, 90, 77, 119, 66, 54, 48, 54, 71, 48, 53, 55, 50, 48, 49, 53, 48, 56, 48, 53, 49, 56, 52, 52, 48, 56, 109, 49, 106, 124, -26, -84, -94, -24, -65, -114, -28, -67, -65, -25]
    	at org.apache.flink.api.java.io.TextValueInputFormat.readRecord(TextValueInputFormat.java:127)
    	at org.apache.flink.api.java.io.TextValueInputFormat.readRecord(TextValueInputFormat.java:38)
    	at org.apache.flink.api.common.io.DelimitedInputFormat.nextRecord(DelimitedInputFormat.java:520)
    	at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:195)
    	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:708)
    	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:533)
    	at java.lang.Thread.run(Thread.java:748)
    Caused by: java.nio.charset.MalformedInputException: Input length = 1
    	at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
    	at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
    	at org.apache.flink.api.java.io.TextValueInputFormat.readRecord(TextValueInputFormat.java:117)
    	... 6 more
    

    解决方案:

    Configuration conf = new Configuration();
    conf.setBoolean("recursive.file.enumeration", true);
    TextValueInputFormat inputFormat = new TextValueInputFormat(new Path(path));
    inputFormat.setSkipInvalidLines(true);
    
    
  • 相关阅读:
    C# 规格说明书
    C#学习笔记----复习笔记
    C#学习笔记-stream,win8.1开发小记
    C#学习笔记-Win8.1 store app winform开发小记
    C#学习笔记:linq和xml
    C#第六周--关于正则表达式应用,delegates关键字
    C#第六课---struct,interface
    C#第五课--继承和多态
    开发了一款小程序
    「CSS Warning 2」icon 的做法
  • 原文地址:https://www.cnblogs.com/weijiqian/p/14341953.html
Copyright © 2011-2022 走看看