cdh版hbase构建Phoenix 遇到的坑

 

Phoenix 构建cdh版hbase遇到的坑

 

1. 安装phoenix

下载：在github上下载对应版本https://github.com/apache/phoenix

解压：略

编译：

修改根目录及其子目录下的pom.xml文件，使cdh版本对应自己集群版本。如图所示

注意：编译中修改版本号遵循原始的写法。4.14.0-cdh5.12.1 版本不可以写成4.14-cdh5.12.1，因为在编译时会有正则校验，书写不规范会导致编译不通过。

 命令：

 vim pom.xml
/cdh5.12  #搜索

修改所有的cdh版本号

 

编译：mvn clean package -DskipTests

拷贝jar :将下面的jar包拷贝至每一台RegionServer的lib下面

/opt/phoenix/APACHE_PHOENIX-4.14.0-cdh5.12.1.p0.0/lib/phoenix/phoenix-4.14.0-cdh5.12.1-server.jar

2. 配置hbase-site.xml

注意前方高能：phoenix4.8版本是个分水岭，配置有所不同

• phoenix4.8以下版本的配置如下：

 配置HMaster的hbase-site.xml

<!-- Phoenix订制的索引负载均衡器 --> 
<property>
 <name>hbase.master.loadbalancer.class</name> <value>org.apache.phoenix.hbase.index.balancer.IndexLoadBalancer</value> 
</property> 
<!-- Phoenix订制的索引观察者 --> 
<property>
 <name>hbase.coprocessor.master.classes</name> <value>org.apache.phoenix.hbase.index.master.IndexMasterObserver</value> 
</property> 

配置RegionServer的hbase-site.xml

<!-- Enables custom WAL edits to be written, ensuring proper writing/replay of the index updates. This codec supports the usual host of WALEdit options, most notably WALEdit compression. --> 
<property>
  <name>hbase.regionserver.wal.codec</name>
  <value>org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec</value> 
</property> 
<!-- Prevent deadlocks from occurring during index maintenance for global indexes (HBase 0.98.4+ and Phoenix 4.3.1+ only) by ensuring index updates are processed with a higher priority than data updates. It also prevents deadlocks by ensuring metadata rpc calls are processed with a higher priority than data rpc calls --> 
<property>
  <name>hbase.region.server.rpc.scheduler.factory.class</name>  
  <value>org.apache.hadoop.hbase.ipc.PhoenixRpcSchedulerFactory</value> 
  <description>Factory to create the Phoenix RPC Scheduler that uses separate queues for index and metadata updates</description>
</property> 
<property>
  <name>hbase.rpc.controllerfactory.class</name>  
  <value>org.apache.hadoop.hbase.ipc.controller.ServerRpcControllerFactory</value> 
  <description>Factory to create the Phoenix RPC Scheduler that uses separate queues for index and metadata updates</description> 
</property>
 <!-- To support local index regions merge on data regions merge you will need to add the following parameter to hbase-site.xml in all the region servers and restart. (It’s applicable for Phoenix 4.3+ versions) --> 
 <property>
   <name>hbase.coprocessor.regionserver.classes</name>
     <value>org.apache.hadoop.hbase.regionserver.LocalIndexMerger</value> 
   </property> 

• phoenix4.8+版本只需添加以下配置

<!-- Enables custom WAL edits to be written, ensuring proper writing/replay of the index updates. This codec supports the usual host of WALEdit options, most notably WALEdit compression. --> 
<property>
  <name>hbase.regionserver.wal.codec</name>
  <value>org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec</value> 
</property> 
<!-- Prevent deadlocks from occurring during index maintenance for global indexes (HBase 0.98.4+ and Phoenix 4.3.1+ only) by ensuring index updates are processed with a higher priority than data updates. It also prevents deadlocks by ensuring metadata rpc calls are processed with a higher priority than data rpc calls --> 
<property>
  <name>hbase.region.server.rpc.scheduler.factory.class</name>
  <value>org.apache.hadoop.hbase.ipc.PhoenixRpcSchedulerFactory</value> 
  <description>Factory to create the Phoenix RPC Scheduler that uses separate queues for index and metadata updates</description> 
</property> 
<property>
  <name>hbase.rpc.controllerfactory.class</name>
  <value>org.apache.hadoop.hbase.ipc.controller.ServerRpcControllerFactory</value>
   <description>Factory to create the Phoenix RPC Scheduler that uses separate queues for index and metadata updates</description> 
</property>

• 解释

在4.8版本之后，安装Phoenix需要把hbase-site.xml文件中的这三个配置项给删掉，因为这三个配置项在4.7及之前的版本会导致在”security enabled”的情况下，每次建立和phoenix server的会话时都会建立一个新的连接实例，短时间内创建大量连接会触发Zookeeper内置的保护机制，拒绝连接。

简单来说，就是HBase在重启的时候，之前赋值到hbase lib目录下的phoenix-4.X.X-HBase-1.X-server.jar包会读取hbase-site.xml文件，结果发现了这三个过期的配置项，然后引起了异常，把这三个配置项删掉就好。。。。。。

(MMP，就这个搞了半天)

山重水复已无路，柳暗花明又一村。感谢这位熊蝶

https://blog.csdn.net/silentasriddle/article/details/78758950

如果是cdh版本的hbase，则在管理界面可直接修改，如下图（phoenxi 4.14的）

 

3. 修改和zookeeper相关的配置保证启动时通信不超时

修改zookeeper的配置maxSessionTimeout

默认是6000，改为1200000，单位是ms。这个值要大于hbase中的值。

 

修改hbase的配置

 

QQ:1710816711(欢迎交流，广告勿扰)