java连接ES操作入门

如何建立数据模型？

1：建立索引

DELETE /hd_insurance_data
​
PUT hd_insurance_data
{
  "settings": {
    "number_of_replicas": 1
  }, 
  "mappings" : {
    "properties" : {
      "age" : {
         "type" : "integer"
      },
      "channel_code" : {
        "type" : "keyword"
      },
      "channel_name" : {
        "type" : "keyword"
      }
    }
  }
}

　　

2：写入数据

2.1:手动写入数据

put /hd_insurance_data/_doc/1 
{
"id": "1",
"department_name": "大项目部",
"department_code": "",
"policy_number": "be16985732a40212345690ed30d1db"
}

 

2.2:通过使用kibana进行数据批量导入

这里不做介绍，导入数据尝试几次都失败了，不太好用

 

2.3:命令行导入数据

curl -H "Content-Type: application/json" -XPOST "localhost:9200/bank/_doc/_bulk?pretty&refresh" --data-binary "@accounts.json"

搜索java的api了解

1：启动集群elasticsearch 和kibana

bin/elasticsearch -E node.name=node1 -E cluster.name=health -E path.data=node1_data -d
​
bin/elasticsearch -E node.name=node2 -E cluster.name=health -E path.data=node2_data -d
​
bin/elasticsearch -E node.name=node3 -E cluster.name=health -E path.data=node3_data -d

bin/kibana

2：建立索引导入数据

3：Java连接es

导入es配置

  <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
  </dependency>

spring.elasticsearch.rest.uris=http://localhost:9200

在项目中注入RestHighLevelClient直接就可以使用

  @Autowired
  private ElasticsearchRestTemplate client;
  @Autowired
  private RestHighLevelClient highLevelClient;

4：建议

1、做简单的bool查询获取状态是否正常

GET /hd_insurance_data/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "policy_status": {
              "value": "1"
            }
          }
        }
      ]
    }
  }
}

2、对年龄每隔5岁进行分桶，并将分桶的年龄分男女进行分桶

GET /hd_insurance_data/_search
{
  "size": 0,
  "aggs": {
    "age_histrogram":{
      "histogram": {
        "field": "age",
        "interval": 5
      },
      "aggs": {
        "sex_count": {
          "terms": {
            "field": "sex",
            "size": 10
          }
        }
      }
    }
  }
}

返回结果，是每个5岁一个分类，同时获取这个分桶中男女的数量

{
"key" : 20.0,
"doc_count" : 1,
"sex_count" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "1",
"doc_count" : 1
}
]
}
},
{
"key" : 25.0,
"doc_count" : 5,
"sex_count" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "1",
"doc_count" : 3
},
{
"key" : "2",
"doc_count" : 2
}
]
}
}

3、将两个步骤结合起来，获取想要的结果

GET /hd_insurance_data/_search
{
  "size": 0, 
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "policy_status": {
              "value": "1"
            }
          }
        }
      ], 
      "filter": [
        {
          "range": {
            "insurance_time": {
              "gte": "2017-01-01 15:09:13",
              "lte": "2021-10-01 15:09:13"
            }
          }
        }
      ]
    }
  }, 
  "aggs": {
    "age_histrogram":{
      "histogram": {
        "field": "age",
        "interval": 5
      }
    },
    "aggs":{
      "sex_count":{
        "terms": {
          "field": "sex",
          "size": 10
        }
      }
    }
  }
}

4、将上面的查询通过api翻译为java代码

 public void searchByConditions() throws IOException {
    SearchRequest searchRequest = new SearchRequest("hd_insurance_data");
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
​
    // 设定搜索条件
    BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery()
        .must(QueryBuilders.termQuery("policy_status", "1"))
        .must(QueryBuilders.termQuery("department_name", "目部"))
        .must(QueryBuilders.termQuery("channel_name", "保险代理"))
        .must(QueryBuilders.termQuery("sales_plan_code", "S1230445"))
        .must(QueryBuilders.termQuery("quota_plan_code", "10123123101"))
        .filter(QueryBuilders.rangeQuery("insurance_time").gte("2019-01-01 15:09:13").lte("2019-10-01 15:09:13"));
​
    // 聚类分析，并子聚类
    AggregationBuilder aggregationBuilder = AggregationBuilders.dateHistogram("age_histrogram").field("age").interval(5);
    AggregationBuilder subAggregationBuilder = AggregationBuilders.terms("sex_count").field("sex");
    aggregationBuilder.subAggregation(subAggregationBuilder);
​
    searchSourceBuilder.size(0);
    searchSourceBuilder.query(boolQueryBuilder);
    searchSourceBuilder.aggregation(aggregationBuilder);
    searchRequest.source(searchSourceBuilder);
​
    SearchResponse searchResponse = highLevelClient.search(searchRequest, RequestOptions.DEFAULT);
    log.info("searchResponse {}", searchResponse.toString());
//    log.info("searchResponse getAggregations {}", Strings.toString(searchResponse.getAggregations()));
  }

 

总结：

1、es使用非常简单，但是各种查询有点多，还需要进一步梳理

本文是对工作中使用es的一个简单调研，刚学习，还请多多指教。

参考

1:elasticsearch的搜索api

https://www.elastic.co/guide/en/elasticsearch/client/java-rest/7.9/java-rest-high-query-builders.html