zoukankan      html  css  js  c++  java
  • spark-sql访问hive的问题记录

    好久没有弄博客了。。。

    hive0.14

    spark0.12

    [hadoop@irs bin]$ ./spark-sql 
    Spark assembly has been built with Hive, including Datanucleus jars on classpath
    Exception in thread "main" java.lang.RuntimeException: java.lang.NumberFormatException: For input string: "1s"
    at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:346)
    at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:101)
    at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
    Caused by: java.lang.NumberFormatException: For input string: "1s"
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
    at java.lang.Integer.parseInt(Integer.java:580)
    at java.lang.Integer.parseInt(Integer.java:615)
    at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1094)
    at org.apache.hadoop.hive.conf.HiveConf.getIntVar(HiveConf.java:1211)
    at org.apache.hadoop.hive.conf.HiveConf.getIntVar(HiveConf.java:1220)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:58)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)
    at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2453)
    at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465)
    at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:340)

    坑嗲啊,原来是hive-site.xml 中的

    <name>hive.metastore.client.connect.retry.delay</name>
    <value>1s</value>

    这个1s类型转换出错啊,改成1000就好了

  • 相关阅读:
    4.5计算机网络笔记
    3.29计算机网络笔记
    3.22计算机网络笔记
    3.15计算机网络笔记
    用int类型表示最大公倍数
    markdown入门
    learning by doing
    技术博客(初用markdown)。
    物联网PPT智能家居王思齐和陈由钧第10组
    第六次java作业
  • 原文地址:https://www.cnblogs.com/tangtianfly/p/4270315.html
Copyright © 2011-2022 走看看