zoukankan      html  css  js  c++  java
  • SpringBoot 使用hbaseclient操作HBase教程1(基本用法)

     HBase 是一个分布式的、面向列的开源的 NoSQL 数据库。Spring Boot 项目如果需要操作 HBase 通常有如下几种客户端可以使用:
    • hbase-client:比较底层,需要自己进一步封装 api,而且版本号和安装的 hbase 也要匹配,否则会报错
    • spring-data-hadoop:2019 年 4 月 5 停止维护
    • Apache Phoenix:使用 SQL 的方式来操作 HBase。Phoenix 的性能很高(进行简单查询,其性能量级是毫秒),相对于 hbase 原生的 scan 并不会差多少,而对于类似的组件 hive、Impala 等,性能有着显著的提升
        本文演示如何使用 hbase-client 操作 HBase 数据库。

    一、安装配置

    1,环境准备

    (1)关于 HBase 的安装,可以参考我之前写的文章:

    (2)同时 HBase 服务器主机名不能为 localhost或者VM-16-6-centos,否则客户端无法连接,具体参考我另一篇文章:

    2,项目配置

    (1)首先编辑项目的 pom.xml 文件,添加 hbase-client 依赖:
    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-client</artifactId>
        <version>2.3.0</version>
    </dependency>

    (2)然后在 application.properties 中添加 HBase 相关配置(即 ZooKeeper 的地址和端口):

    # HBase 数据源配置
    hbase.config.hbase.zookeeper.quorum=86.168.xx.xx
    hbase.config.hbase.zookeeper.property.clientPort=2181

    yml版本

    hbase:
      config:
        hbase:
          zookeeper:
            property:
              clientPort: 2181
            quorum: 81.68.xx.xx

    3,编写工具类

    (1)首先编写 HBaseConfig.java 来读取配置文件,获取 hbase 参数:  
    @Configuration
    @ConfigurationProperties(prefix = "hbase")
    public class HBaseConfig {
     
        private Map<String, String> config = new HashMap<>();
     
        public Map<String, String> getConfig() {
            return config;
        }
     
        public void setConfig(Map<String, String> config) {
            this.config = config;
        }
     
        public org.apache.hadoop.conf.Configuration configuration() {
            org.apache.hadoop.conf.Configuration configuration = HBaseConfiguration.create();
    //此处可自己自定义和改造 拓展用
           //        configuration.set(HBASE_QUORUM, "81.68.xx.xx:2181");
    //        configuration.set(HBASE_ROOTDIR, "/");
    //        configuration.set(HBASE_ZNODE_PARENT, "/hbase");
            for(Map.Entry<String, String> map : config.entrySet()){
                configuration.set(map.getKey(), map.getValue());
            }
            return configuration;
        }
     
        @Bean
        public Admin admin() {
            Admin admin = null;
            try {
                Connection connection = ConnectionFactory.createConnection(configuration());
                admin = connection.getAdmin();
            } catch (IOException e) {
                e.printStackTrace();
            }
            return admin;
        }
    }

    (2)由于 hbase-client 比较底层,我们还要封装一个 HBaseUtils.java 工具类,实现创建表、插入、读取、删除数据:

    @Service
    public class HBaseUtils {
    
        @Autowired
        private Admin hbaseAdmin;
    
        /**
         * 判断表是否存在
         *
         * @param tableName 表名
         * @return true/false
         */
        public boolean isExists(String tableName) {
            boolean tableExists = false;
            try {
                TableName table = TableName.valueOf(tableName);
                tableExists = hbaseAdmin.tableExists(table);
            } catch (IOException e) {
                e.printStackTrace();
            }
            return tableExists;
        }
    
        /**
         * 创建表
         * @param tableName 表名
         * @param columnFamily 列族
         * @return true/false
         */
        public boolean createTable(String tableName, List<String> columnFamily) {
            return createTable(tableName, columnFamily, null);
        }
    
        /**
         * 预分区创建表
         * @param tableName 表名
         * @param columnFamily 列族
         * @param keys 分区集合
         * @return true/false
         */
        public boolean createTable(String tableName, List<String> columnFamily, List<String> keys) {
            if (!isExists(tableName)) {
                try {
                    TableName table = TableName.valueOf(tableName);
                    HTableDescriptor desc = new HTableDescriptor(table);
                    for (String cf : columnFamily) {
                        desc.addFamily(new HColumnDescriptor(cf));
                    }
                    if (keys == null) {
                        hbaseAdmin.createTable(desc);
                    } else {
                        byte[][] splitKeys = getSplitKeys(keys);
                        hbaseAdmin.createTable(desc, splitKeys);
                    }
                    return true;
                } catch (IOException e) {
                    e.printStackTrace();
                }
            } else {
                System.out.println(tableName + "is exists!!!");
                return false;
            }
            return false;
        }
    
        /**
         * 删除表
         *
         * @param tableName 表名
         */
        public void dropTable(String tableName) throws IOException {
            if (isExists(tableName)) {
                TableName table = TableName.valueOf(tableName);
                hbaseAdmin.disableTable(table);
                hbaseAdmin.deleteTable(table);
            }
        }
    
        /**
         * 插入数据(单条)
         * @param tableName 表名
         * @param rowKey rowKey
         * @param columnFamily 列族
         * @param column 列
         * @param value 值
         * @return true/false
         */
        public boolean putData(String tableName, String rowKey, String columnFamily, String column,
                               String value) {
            return putData(tableName, rowKey, columnFamily, Arrays.asList(column),
                    Arrays.asList(value));
        }
    
        /**
         * 插入数据(批量)
         * @param tableName 表名
         * @param rowKey rowKey
         * @param columnFamily 列族
         * @param columns 列
         * @param values 值
         * @return true/false
         */
        public boolean putData(String tableName, String rowKey, String columnFamily,
                               List<String> columns, List<String> values) {
            try {
                Table table = hbaseAdmin.getConnection().getTable(TableName.valueOf(tableName));
                Put put = new Put(Bytes.toBytes(rowKey));
                for (int i=0; i<columns.size(); i++) {
                    put.addColumn(Bytes.toBytes(columnFamily), Bytes.toBytes(columns.get(i)),
                            Bytes.toBytes(values.get(i)));
                }
                table.put(put);
                table.close();
                return true;
            } catch (IOException e) {
                e.printStackTrace();
                return false;
            }
        }
    
        /**
         * 获取数据(全表数据)
         * @param tableName 表名
         * @return map
         */
        public List<Map<String, String>> getData(String tableName) {
            List<Map<String, String>> list = new ArrayList<>();
            try {
                Table table = hbaseAdmin.getConnection().getTable(TableName.valueOf(tableName));
                Scan scan = new Scan();
                ResultScanner resultScanner = table.getScanner(scan);
                for(Result result : resultScanner) {
                    HashMap<String, String> map = new HashMap<>();
                    //rowkey
                    String row = Bytes.toString(result.getRow());
                    map.put("row", row);
                    for (Cell cell : result.listCells()) {
                        //列族
                        String family = Bytes.toString(cell.getFamilyArray(),
                                cell.getFamilyOffset(), cell.getFamilyLength());
                        //
                        String qualifier = Bytes.toString(cell.getQualifierArray(),
                                cell.getQualifierOffset(), cell.getQualifierLength());
                        //
                        String data = Bytes.toString(cell.getValueArray(),
                                cell.getValueOffset(), cell.getValueLength());
                        map.put(family + ":" + qualifier, data);
                    }
                    list.add(map);
                }
                table.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
            return list;
        }
    
        /**
         * 获取数据(根据传入的filter)
         * @param tableName 表名
         * @param filter 过滤器
         * @return map
         */
        public List<Map<String, String>> getData(String tableName, Filter filter) {
            List<Map<String, String>> list = new ArrayList<>();
            try {
                Table table = hbaseAdmin.getConnection().getTable(TableName.valueOf(tableName));
                Scan scan = new Scan();
                // 添加过滤器
                scan.setFilter(filter);
                ResultScanner resultScanner = table.getScanner(scan);
                for(Result result : resultScanner) {
                    HashMap<String, String> map = new HashMap<>();
                    //rowkey
                    String row = Bytes.toString(result.getRow());
                    map.put("row", row);
                    for (Cell cell : result.listCells()) {
                        //列族
                        String family = Bytes.toString(cell.getFamilyArray(),
                            cell.getFamilyOffset(), cell.getFamilyLength());
                        //
                        String qualifier = Bytes.toString(cell.getQualifierArray(),
                            cell.getQualifierOffset(), cell.getQualifierLength());
                        //
                        String data = Bytes.toString(cell.getValueArray(),
                            cell.getValueOffset(), cell.getValueLength());
                        map.put(family + ":" + qualifier, data);
                    }
                    list.add(map);
                }
                table.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
            return list;
        }
    
        /**
         * 获取数据(根据rowkey)
         * @param tableName 表名
         * @param rowKey rowKey
         * @return map
         */
        public Map<String, String> getData(String tableName, String rowKey) {
            HashMap<String, String> map = new HashMap<>();
            try {
                Table table = hbaseAdmin.getConnection().getTable(TableName.valueOf(tableName));
                Get get = new Get(Bytes.toBytes(rowKey));
                Result result = table.get(get);
                if (result != null && !result.isEmpty()) {
                    for (Cell cell : result.listCells()) {
                        //列族
                        String family = Bytes.toString(cell.getFamilyArray(),
                                cell.getFamilyOffset(), cell.getFamilyLength());
                        //
                        String qualifier = Bytes.toString(cell.getQualifierArray(),
                                cell.getQualifierOffset(), cell.getQualifierLength());
                        //
                        String data = Bytes.toString(cell.getValueArray(),
                                cell.getValueOffset(), cell.getValueLength());
                        map.put(family + ":" + qualifier, data);
                    }
                }
                table.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
            return map;
        }
    
        /**
         * 获取数据(根据rowkey,列族,列)
         * @param tableName 表名
         * @param rowKey rowKey
         * @param columnFamily 列族
         * @param columnQualifier 列
         * @return map
         */
        public String getData(String tableName, String rowKey, String columnFamily,
                              String columnQualifier) {
            String data = "";
            try {
                Table table = hbaseAdmin.getConnection().getTable(TableName.valueOf(tableName));
                Get get = new Get(Bytes.toBytes(rowKey));
                get.addColumn(Bytes.toBytes(columnFamily), Bytes.toBytes(columnQualifier));
                Result result = table.get(get);
                if (result != null && !result.isEmpty()) {
                    Cell cell = result.listCells().get(0);
                    data = Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                            cell.getValueLength());
                }
                table.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
            return data;
        }
    
        /**
         * 删除数据(根据rowkey)
         * @param tableName 表名
         * @param rowKey rowKey
         */
        public void deleteData(String tableName, String rowKey) throws IOException{
            Table table = hbaseAdmin.getConnection().getTable(TableName.valueOf(tableName));
            Delete delete = new Delete(Bytes.toBytes(rowKey));
            table.delete(delete);
            table.close();
        }
    
        /**
         * 删除数据(根据rowkey,列族)
         * @param tableName 表名
         * @param rowKey rowKey
         * @param columnFamily 列族
         */
        public void deleteData(String tableName, String rowKey, String columnFamily)
                throws IOException{
            Table table = hbaseAdmin.getConnection().getTable(TableName.valueOf(tableName));
            Delete delete = new Delete(Bytes.toBytes(rowKey));
            delete.addFamily(columnFamily.getBytes());
            table.delete(delete);
            table.close();
        }
    
        /**
         * 删除数据(根据rowkey,列族)
         * @param tableName 表名
         * @param rowKey rowKey
         * @param columnFamily 列族
         * @param column 列
         */
        public void deleteData(String tableName, String rowKey, String columnFamily, String column)
                throws IOException{
            Table table = hbaseAdmin.getConnection().getTable(TableName.valueOf(tableName));
            Delete delete = new Delete(Bytes.toBytes(rowKey));
            delete.addColumn(columnFamily.getBytes(), column.getBytes());
            table.delete(delete);
            table.close();
        }
    
        /**
         * 删除数据(多行)
         * @param tableName 表名
         * @param rowKeys rowKey集合
         */
        public void deleteData(String tableName, List<String> rowKeys) throws IOException{
            Table table = hbaseAdmin.getConnection().getTable(TableName.valueOf(tableName));
            List<Delete> deleteList = new ArrayList<>();
            for(String row : rowKeys){
                Delete delete = new Delete(Bytes.toBytes(row));
                deleteList.add(delete);
            }
            table.delete(deleteList);
            table.close();
        }
    
        /**
         * 分区【10, 20, 30】 -> ( ,10] (10,20] (20,30] (30, )
         * @param keys 分区集合[10, 20, 30]
         * @return byte二维数组
         */
        private byte[][] getSplitKeys(List<String> keys) {
            byte[][] splitKeys = new byte[keys.size()][];
            TreeSet<byte[]> rows = new TreeSet<>(Bytes.BYTES_COMPARATOR);
            for(String key : keys) {
                rows.add(Bytes.toBytes(key));
            }
            int i = 0;
            for (byte[] row : rows) {
                splitKeys[i] = row;
                i ++;
            }
            return splitKeys;
        }
    }

    附:使用样例

    (1)下面创建一个 Controller 调用 HBaseUtils 来操作 HBase 数据库:

    @RestController
    @AllArgsConstructor
    @RequestMapping("/common/front")
    public class FrontController {
     
        @Autowired
        private HBaseUtils hbaseUtils;
     
        @GetMapping("/test")
        public void test() throws IOException {
            System.out.println("---开始创建test表---");
            hbaseUtils.createTable("test", Arrays.asList("cf"));
     
            System.out.println("---判断test表是否存在---");
            Boolean t = hbaseUtils.isExists("test");
            System.out.println(t);
     
            System.out.println("\n---插入一列数据---");
            hbaseUtils.putData("test", "row1", "cf", "a", "value1-1");
     
            System.out.println("\n---插入多列数据---");
            hbaseUtils.putData("test", "row2", "cf",
                    Arrays.asList("a", "b", "c"),  Arrays.asList("value2-1", "value2-2", "value2-3"));
     
            System.out.println("\n---根据rowkey、列族、列查询数据---");
            String columnData = hbaseUtils.getData("test", "row2", "cf", "b");
            System.out.println(columnData);
     
            System.out.println("\n---根据rowkey查询数据---");
            Map<String, String> rowData = hbaseUtils.getData("test", "row2");
            System.out.println(rowData);
     
            System.out.println("\n---查询全表数据---");
            List<Map<String, String>> tableData = hbaseUtils.getData("test");
            System.out.println(tableData);
     
            System.out.println("\n---根据rowkey、列族、列删除数据---");
            hbaseUtils.deleteData("test", "row2", "cf", "b");
     
            System.out.println("\n---根据rowkey、列族删除数据---");
            hbaseUtils.deleteData("test", "row2", "cf");
     
            System.out.println("\n---根据rowkey删除数据---");
            hbaseUtils.deleteData("test", "row2");
     
            System.out.println("\n---根据rowkey批量删除数据---");
            hbaseUtils.deleteData("test", Arrays.asList("row1", "row2"));
     
            System.out.println("\n---删除表---");
            hbaseUtils.dropTable("test");
        }
    }

    2)使用浏览器访问 /test 接口,可以看到控制台输出如下:

     

     
  • 相关阅读:
    Java实现水仙花数
    CSS3属性选择器
    Word快捷键
    Java实现百钱买百鸡
    某专业人士给中国计算机专业学生的建议
    经典名言警句
    面试问题和思路
    情商
    Java注意的地方
    唯大英雄能真本色——Leo鉴书34
  • 原文地址:https://www.cnblogs.com/shanheyongmu/p/15657323.html
Copyright © 2011-2022 走看看