zoukankan      html  css  js  c++  java
  • flink 读取文件报错 Line could not be encoded

    Caused by: java.lang.RuntimeException: Line could not be encoded: [49, 56, 49, 49, 90, 77, 119, 66, 54, 48, 54, 71, 48, 53, 55, 50, 48, 49, 53, 48, 56, 48, 53, 49, 56, 52, 52, 48, 56, 109, 49, 106, 124, -26, -84, -94, -24, -65, -114, -28, -67, -65, -25]
    	at org.apache.flink.api.java.io.TextValueInputFormat.readRecord(TextValueInputFormat.java:127)
    	at org.apache.flink.api.java.io.TextValueInputFormat.readRecord(TextValueInputFormat.java:38)
    	at org.apache.flink.api.common.io.DelimitedInputFormat.nextRecord(DelimitedInputFormat.java:520)
    	at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:195)
    	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:708)
    	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:533)
    	at java.lang.Thread.run(Thread.java:748)
    Caused by: java.nio.charset.MalformedInputException: Input length = 1
    	at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
    	at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
    	at org.apache.flink.api.java.io.TextValueInputFormat.readRecord(TextValueInputFormat.java:117)
    	... 6 more
    

    解决方案:

    Configuration conf = new Configuration();
    conf.setBoolean("recursive.file.enumeration", true);
    TextValueInputFormat inputFormat = new TextValueInputFormat(new Path(path));
    inputFormat.setSkipInvalidLines(true);
    
    
  • 相关阅读:
    将来要干啥
    选新技术考虑点
    hdfs 创建一个新用户
    linux下实现mysql数据库定时备份
    PostgreSQL的安装和卸载,远程连接
    PostgreSQL语法
    【NiFi系列】1-基本介绍
    大数据相关资源网址
    MySQL主从复制配置
    MySQL设置免密登录
  • 原文地址:https://www.cnblogs.com/weijiqian/p/14341953.html
Copyright © 2011-2022 走看看