spark连接hive数据库

hive在执行查询sql时出现java.lang.IllegalArgumentException: Wrong FS: hdfs://node1:9000/user/hive/warehouse/test1.db/t1, expected: hdfs://cluster1

原因是hadoop由普通集群修改成了高可用集群后没有更改hive设置中warehouse在hdfs上的储存路径
修改hive-site.xml文件内hive.metastore.warehouse.dir的值

将之前的hdfs://k200:9000/user/hive/warehouse 修改为 hdfs://k131/user/hive/warehouse

(这里的hdfs://cluster1是Hadoop配置文件core-site.xml中的fs.defaultFS指定的值)

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
        <property>
            <name>javax.jdo.option.ConnectionURL</name>
            <value>jdbc:mysql://k131:3306/metastore? 
              createDatabaseIfNotExist=true</value>
            <description>JDBC connect string for a JDBC 
            metastore</description>
        </property>

        <property>
           <name>javax.jdo.option.ConnectionDriverName</name>
           <value>com.mysql.jdbc.Driver</value>
           <description>Driver class name for a JDBC 
            metastore</description>
         </property>

         <property>
              <name>javax.jdo.option.ConnectionUserName</name>
              <value>root</value>
              <description>username to use against metastore 
              database</description>
         </property>

         <property>
               <name>javax.jdo.option.ConnectionPassword</name>
               <value>root</value>
               <description>password to use against metastore 
               database</description>
        </property>

        <property>
               <name>hive.cli.print.header</name>
               <value>true</value>
        </property>

        <property>
               <name>hive.cli.print.current.db</name>
               <value>true</value>
        </property>
        <property>
               <name>hive.exec.mode.local.auto</name>
               <value>true</value>
        </property>

        <property>
                <name>hive.zookeeper.quorum</name>
                <value>k131</value>
                <description>The list of ZooKeeper servers to talk to. This is only needed for read/write locks.</description>
                </property>

        <property>
              <name>hive.zookeeper.client.port</name>
              <value>2181</value>
              <description>The port of ZooKeeper servers to talk to. This is only needed for read/write locks.</description>
        </property>

</configuration>

spark 无法查看 hive 表中原来的内容，只能重新创建新表

hive (default)> select * from emp;
FAILED: SemanticException Unable to determine if hdfs://k200:9000/user/hive/warehouse/emp is encrypted: java.lang.IllegalArgumentException: Wrong FS: hdfs://k200:9000/user/hive/warehouse/emp, expected: hdfs://k131:9000
hive (default)>