zoukankan      html  css  js  c++  java
  • 【原创】大叔经验分享(100)Atlas导入hive元数据

    首先要有HIVE_HOME环境变量,

    如果是apache,直接配置为解压目录;如果是CDH,设置如下:

    # export HIVE_HOME=/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hive

    执行导入

    # bin/import-hive.sh
    ...
    Failed to import Hive Meta Data!!!

    报错,查看日志

    # more logs/import-hive.log 
    2020-01-11 14:42:38,951 INFO  - [main:] ~ Looking for atlas-application.properties in classpath (ApplicationProperties:110)
    2020-01-11 14:42:38,955 INFO  - [main:] ~ Looking for /atlas-application.properties in classpath (ApplicationProperties:115)
    2020-01-11 14:42:38,956 INFO  - [main:] ~ Loading atlas-application.properties from null (ApplicationProperties:123)
    2020-01-11 14:42:38,984 ERROR - [main:] ~ Import failed (HiveMetaStoreBridge:176)
    org.apache.atlas.AtlasException: Failed to load application properties
        at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:134)
        at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:86)
        at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:120)
    Caused by: org.apache.commons.configuration.ConfigurationException: Cannot locate configuration source null
        at org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:259)
        at org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:238)
        at org.apache.commons.configuration.AbstractFileConfiguration.<init>(AbstractFileConfiguration.java:197)
        at org.apache.commons.configuration.PropertiesConfiguration.<init>(PropertiesConfiguration.java:284)
        at org.apache.atlas.ApplicationProperties.<init>(ApplicationProperties.java:69)
        at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:125)
        ... 2 more

    提示找不到atlas-application.properties,将其拷贝到hive conf目录

    # cp conf/atlas-application.properties /etc/hive/conf/

    再次执行

    # bin/import-hive.sh
    ...
    Enter username for atlas :- admin
    Enter password for atlas :- 
    Exception in thread "main" java.lang.NoClassDefFoundError: com/fasterxml/jackson/jaxrs/json/JacksonJaxbJsonProvider
        at org.apache.atlas.AtlasBaseClient.getClient(AtlasBaseClient.java:270)
        at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:453)
        at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:448)
        at org.apache.atlas.AtlasBaseClient.<init>(AtlasBaseClient.java:132)
        at org.apache.atlas.AtlasClientV2.<init>(AtlasClientV2.java:82)
        at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:131)
    Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.jaxrs.json.JacksonJaxbJsonProvider
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 6 more
    Failed to import Hive Meta Data!!!

    还是报错,提示找不到类,从server目录下拷贝到hook/hive目录下

    # cp server/webapp/atlas/WEB-INF/lib/jackson-jaxrs-1.8.3.jar hook/hive/atlas-hive-plugin-impl/
    # cp server/webapp/atlas/WEB-INF/lib/jackson-jaxrs-json-provider-2.9.2.jar hook/hive/atlas-hive-plugin-impl/
    # cp server/webapp/atlas/WEB-INF/lib/jackson-module-jaxb-annotations-2.9.8.jar hook/hive/atlas-hive-plugin-impl/

    再次执行成功,到atlas里可以看到hive相关数据

    导入成功

  • 相关阅读:
    Iframe和Frame中实现cookie跨域的方法(转载)
    android中拷贝assets下的资源文件到SD卡中(可以超过1M)
    OpenSL ES 查询设备支持的SL Profiles
    NDK开发中的一个HTTP下载实例附带下载进度
    android中配置文件property的用途以及使用<转>
    Eclipse 工程使用相对路径导入Jar包设置
    Android 解压zip文件(支持中文)
    c++实现一个比较两个string类型的版本号的小demo
    linux c++下载http文件并显示进度<转>
    Linux下类似windows下_beginthread和_endthread 的多线程开发
  • 原文地址:https://www.cnblogs.com/barneywill/p/12180277.html
Copyright © 2011-2022 走看看