zoukankan      html  css  js  c++  java
  • Elasticsearch Java API 的使用(8)—Scroll (游标)API详解

    ####滚动查询
    Elasticsearch中进行大数据量查询时,往往因为设备、网络传输问题影响查询数据的效率;Elasticsearch中提供了 Scroll(游标)的方式对数据进行少量多批次的滚动查询,来提高查询效率。

    public class Scroll {
    
        public static void main(String[] args) {
    
            try{
                long startTime = System.currentTimeMillis();
                /*创建客户端*/
                //client startup
                //设置集群名称
                Settings settings = Settings.builder()
                        .put("cluster.name", "elsearch")
                        .put("client.transport.sniff", true)
                        .build();
                //创建client
                TransportClient client = new PreBuiltTransportClient(settings)
                        .addTransportAddress(new InetSocketTransportAddress(
                        InetAddress.getByName("54.223.232.95"),9300));
    
                List<String> result = new ArrayList<>();
    
                String scrollId = "";
    
                //第一次请求
                SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
    
    
                //TODO: 设置查询条件
                RangeQueryBuilder rangequerybuilder = QueryBuilders
                    .rangeQuery("inputtime")
                    .from("2016-12-14 02:00:00").to("2016-12-14 07:59:59");
                sourceBuilder.query(QueryBuilders.boolQuery()
                    .must(QueryBuilders
                            .matchPhraseQuery("pointid","W3.UNIT1.10HFC01CT013"))
                        .must(rangequerybuilder))
                        .size(100)//如果开启游标,则滚动获取
                        .sort("inputtime", SortOrder.ASC);
                //查询
                SearchRequest request = Requests.searchRequest("pointdata");
                    request.scroll("2m");
                    request.source(sourceBuilder);
                SearchResponse response = client.search(request).actionGet();
                //TODO:处理数据
                SearchHits hits = response.getHits();
                for(int i = 0; i < hits.getHits().length; i++) {
                    //System.out.println(hits.getHits()[i].getSourceAsString());
                    result.add(hits.getHits()[i].getSourceAsString());
                }
                //记录滚动ID
                scrollId = response.getScrollId();
    
    
                while(true){
                    //后续的请求
                    //scrollId = query.getScollId();
                    SearchScrollRequestBuilder searchScrollRequestBuilder = client
                        .prepareSearchScroll(scrollId);            
                    // 重新设定滚动时间            
                    //TimeValue timeValue = new TimeValue(30000);
                    searchScrollRequestBuilder.setScroll("2m");
                    // 请求            
                    SearchResponse response1 = searchScrollRequestBuilder.get();
    
                    //TODO:处理数据
                    SearchHits hits2 = response1.getHits();
                    if(hits2.getHits().length == 0){
                        break;
                    }
                    for(int i = 0; i < hits2.getHits().length; i++) {
                        result.add(hits2.getHits()[i].getSourceAsString());
                    }
                    //下一批处理
                    scrollId = response1.getScrollId();
                }
    
                System.out.println(result.size());
                long endTime = System.currentTimeMillis();
                System.out.println("Java程序运行时间:" + (endTime - startTime) + "ms");
            }catch(Exception e){
                e.printStackTrace();
            }
    
        }

    Scroll原理查看Elasticsearch 使用scroll滚动技术实现大数据量搜索

    转载于:https://blog.csdn.net/zx711166/article/details/77847134

  • 相关阅读:
    又到泰山了
    有趣的数字
    关于Servlet/JSP里"/"的用法
    [WS]一个简单的WSDL文档(下)
    30天敏捷结果(28):撰写你的个人使命
    推荐:敏捷个人应该订阅的博客
    30天敏捷结果(27):做些有重要意义的事
    30天敏捷结果(29):找到适合你发展的环境
    COM+的配置:痛并快乐着
    30天敏捷结果(22):设计你的一天
  • 原文地址:https://www.cnblogs.com/it-deepinmind/p/14583555.html
Copyright © 2011-2022 走看看