zoukankan      html  css  js  c++  java
  • 「hive」hive2.3.0配置derby

    【问题现象】

    hive默认自带derby数据库,但安装hive2.3.0后启动hive无法使用derby操作,遇到各种问题,折腾好一阵子,经测试已ok

    【先决条件】

    ubuntu已安装hadoop2.8.1

    【问题解决】

    1、官网下载hive版本2.3.0,在ubuntu中并安装;

    2、增加HIVE_HOME到/etc/profile,并$source /etc/profile 使得变量立刻生效;

    3、生成默认配置文件和自定义配置文件(重复项时自定义配置会覆盖默认配置),定位到hive根目录,

    $cp conf/hive-default.xml.template conf/hive-default.xml

    $cp conf/hive-default.xml conf/hive-site.xml

    4、hive-site.xml中去掉hive.metastore.local属性

    <!-- 
    <property>
      <name>hive.metastore.local</name>
      <value>true</value>
    </property>
    -->
    用于消除 HiveConf of name hive.metastore.local does not exist 错误

    5、设置hive.metastore.schema.verification为false,

    解决Version information not found in metastore错误。

    6、设置hive.druid.metadata.db.type为derby

    参考如下mysql配置

     1 <?xml version="1.0" encoding="UTF-8" standalone="no"?>
     2 <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
     3 <!--
     4    Licensed to the Apache Software Foundation (ASF) under one or more
     5    contributor license agreements.  See the NOTICE file distributed with
     6    this work for additional information regarding copyright ownership.
     7    The ASF licenses this file to You under the Apache License, Version 2.0
     8    (the "License"); you may not use this file except in compliance with
     9    the License.  You may obtain a copy of the License at
    10 
    11        http://www.apache.org/licenses/LICENSE-2.0
    12 
    13    Unless required by applicable law or agreed to in writing, software
    14    distributed under the License is distributed on an "AS IS" BASIS,
    15    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    16    See the License for the specific language governing permissions and
    17    limitations under the License.
    18 -->
    19 <configuration>
    20   <!-- WARNING!!! This file is auto generated for documentation purposes ONLY! -->
    21   <!-- WARNING!!! Any changes you make to this file will be ignored by Hive.   -->
    22   <!-- WARNING!!! You must make your changes in hive-site.xml instead.         -->
    23   <!-- Hive Execution Parameters -->
    24         <property>
    25             <name>javax.jdo.option.ConnectionURL</name>
    26             <value>jdbc:mysql://10.0.10.246:3306/hive?createDatabaseIfNotExist=true&amp;useUnicode=true&amp;characterEncoding=UTF-8&amp;useSSL=false</value>
    27              <description>数据库链接地址</description>
    28         </property>
    29         <property>
    30             <name>javax.jdo.option.ConnectionDriverName</name>
    31             <value>com.mysql.jdbc.Driver</value>
    32             <description>数据库驱动类</description>
    33         </property>
    34         <property>
    35             <name>javax.jdo.option.ConnectionUserName</name>
    36             <value>chendajian</value>
    37             <description>数据库用户名</description>
    38         </property>
    39         <property>
    40             <name>javax.jdo.option.ConnectionPassword</name>
    41             <value>123456</value>
    42             <description>数据库密码</description>
    43         </property>
    44         <property>
    45             <name>hive.metastore.warehouse.dir</name>
    46             <value>/usr/apps/hive/warehouse</value>
    47            <description>数据相对路径</description>
    48         </property>
    49         <property>
    50             <name>hive.exec.scratdir</name>
    51             <value>/usr/apps/hive/warehouse</value>
    52         <description>数据临时路径</description>
    53         </property>
    54 </configuration>
    View Code

    7、数据库相关初始化

    定位到hive根目录,$schematool -dbType derby -initSchema

    解决Required table missing : "DBS" in Catalog "" Schema "". DataNucleus requires this table to perform its persistence operations. Either your MetaData is incorrect, or you need to enable "datanucleus.schema.autoCreateTables错误

    8、$hive 即可进入hive shell操作啦。

    【其它问题】

    问题:Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStor

    配置有问题

    问题:Duplicate key name 'PCS_STATS_IDX'

    这个错误是因为,执行schematool -initSchema -dbType mysql的时候,hivedb数据库表里已经存在表了,所以在库里吧所有的表都删除了,再执行命令。

    问题:警告:You need either to explicitly disable SSL by setting useSSL=false, or set us

    连接时关闭ssl,

    <property>
                <name>javax.jdo.option.ConnectionURL</name>
                <value>jdbc:mysql://10.0.10.246:3306/hive?createDatabaseIfNotExist=true&amp;useUnicode=true&amp;characterEncoding=UTF-8&amp;useSSL=false</value>
                 <description>数据库链接地址</description>
            </property>

  • 相关阅读:
    POJ 2492 并查集扩展(判断同性恋问题)
    菜鸟带你飞______DP基础26道水题
    HDU 1978 记忆化搜索(dfs+dp)
    HDU 1203 I NEED A OFFER (01背包&&概率dp)
    HDU 1176免费馅饼 DP数塔问题转化
    HDU 1069&&HDU 1087 (DP 最长序列之和)
    最短路&&最小生成树水题
    POJ 1797 Heavy Transportation (Dijkstra变形)
    数论学习笔记
    Codeforces Round #579 (Div. 3)
  • 原文地址:https://www.cnblogs.com/ftrako/p/7642572.html
Copyright © 2011-2022 走看看