zoukankan      html  css  js  c++  java
  • bigdata_hive_Issue of Vectorization on Parquet table

    When Vectorization is turned on in Hive:
    set hive.vectorized.execution.enabled=true;

    If the involved table is in parquet rather than orc format, you may see below error.
    This error appears in both "tez" and "mr" engine.

    Solution: Disable vectorization.


    Caused by: java.io.IOException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Incompatible Bytes vector column and primitive category VOID
    at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
    at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
    at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:352)
    at org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:79)
    at org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:33)
    at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:115)
    at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:126)
    at org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:113)
    at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:61)
    ... 15 more
    Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Incompatible Bytes vector column and primitive category VOID
    at org.apache.hadoop.hive.ql.io.parquet.VectorizedParquetInputFormat$VectorizedParquetRecordReader.next(VectorizedParquetInputFormat.java:136)
    at org.apache.hadoop.hive.ql.io.parquet.VectorizedParquetInputFormat$VectorizedParquetRecordReader.next(VectorizedParquetInputFormat.java:49)
    at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:347)
    ... 21 more
     
  • 相关阅读:
    vue2-highcharts 动态加载数据
    css选择器易混符号(~波浪号、+加号、>大于号)
    前端压缩字体文件---成功
    数组里添加一行数据(splice)
    new Date(date).getTime()不兼容iphone
    关于表单验证的正则表达式
    vuejs+webpack环境搭建
    Bootstrap弹出层(modal)垂直居中简单解决方案(无需修改js)
    $.get、$.post、$getJSON、$ajax的用法跟区别
    流式布局- 流式图片
  • 原文地址:https://www.cnblogs.com/cphmvp/p/6128838.html
Copyright © 2011-2022 走看看