zoukankan      html  css  js  c++  java
  • spark dataframe unionall

    今天本来想写一个spark dataframe unionall的demo,由于粗心报下面错误:

    Exception in thread "main" org.apache.spark.sql.AnalysisException: Union can only be performed on tables with the same number of columns, but the left table has 3 columns and the right has 4;
    at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.failAnalysis(CheckAnalysis.scala:38)
    at org.apache.spark.sql.catalyst.analysis.Analyzer.failAnalysis(Analyzer.scala:44)
    at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:170)
    at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:50)
    at org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:121)
    at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.checkAnalysis(CheckAnalysis.scala:50)
    at org.apache.spark.sql.catalyst.analysis.Analyzer.checkAnalysis(Analyzer.scala:44)
    at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:34)
    at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:133)
    at org.apache.spark.sql.DataFrame.org$apache$spark$sql$DataFrame$$withPlan(DataFrame.scala:2127)
    at org.apache.spark.sql.DataFrame.unionAll(DataFrame.scala:1008)
    at com.xiaoju.arch.engine.spark.ReadHiveDemo.main(ReadHiveDemo.java:74)

    错误信息很简单:两张表union,至少字段保持一致吧,汗~~~~

  • 相关阅读:
    EditorWindow 和MenuItem
    PropertyDrawer 自定义属性绘图
    NGUI3.5系列教程之 一些小功能的实现
    NGUI3.5系列教程之 UILabel
    关于Resources.LoadAssetAtPath
    C#调用Python 脚本语言
    unity项目实现“再按一次退出程序”提示功能
    在Visual Studio 2010 中创建类库(dll)
    Unity 优化
    JAVA package-info文件【转】
  • 原文地址:https://www.cnblogs.com/likehua/p/6203491.html
Copyright © 2011-2022 走看看