zoukankan      html  css  js  c++  java
  • Elasticsearch Java API 的使用(8)—Scroll (游标)API详解

    ####滚动查询
    Elasticsearch中进行大数据量查询时,往往因为设备、网络传输问题影响查询数据的效率;Elasticsearch中提供了 Scroll(游标)的方式对数据进行少量多批次的滚动查询,来提高查询效率。

    public class Scroll {
    
        public static void main(String[] args) {
    
            try{
                long startTime = System.currentTimeMillis();
                /*创建客户端*/
                //client startup
                //设置集群名称
                Settings settings = Settings.builder()
                        .put("cluster.name", "elsearch")
                        .put("client.transport.sniff", true)
                        .build();
                //创建client
                TransportClient client = new PreBuiltTransportClient(settings)
                        .addTransportAddress(new InetSocketTransportAddress(
                        InetAddress.getByName("54.223.232.95"),9300));
    
                List<String> result = new ArrayList<>();
    
                String scrollId = "";
    
                //第一次请求
                SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
    
    
                //TODO: 设置查询条件
                RangeQueryBuilder rangequerybuilder = QueryBuilders
                    .rangeQuery("inputtime")
                    .from("2016-12-14 02:00:00").to("2016-12-14 07:59:59");
                sourceBuilder.query(QueryBuilders.boolQuery()
                    .must(QueryBuilders
                            .matchPhraseQuery("pointid","W3.UNIT1.10HFC01CT013"))
                        .must(rangequerybuilder))
                        .size(100)//如果开启游标,则滚动获取
                        .sort("inputtime", SortOrder.ASC);
                //查询
                SearchRequest request = Requests.searchRequest("pointdata");
                    request.scroll("2m");
                    request.source(sourceBuilder);
                SearchResponse response = client.search(request).actionGet();
                //TODO:处理数据
                SearchHits hits = response.getHits();
                for(int i = 0; i < hits.getHits().length; i++) {
                    //System.out.println(hits.getHits()[i].getSourceAsString());
                    result.add(hits.getHits()[i].getSourceAsString());
                }
                //记录滚动ID
                scrollId = response.getScrollId();
    
    
                while(true){
                    //后续的请求
                    //scrollId = query.getScollId();
                    SearchScrollRequestBuilder searchScrollRequestBuilder = client
                        .prepareSearchScroll(scrollId);            
                    // 重新设定滚动时间            
                    //TimeValue timeValue = new TimeValue(30000);
                    searchScrollRequestBuilder.setScroll("2m");
                    // 请求            
                    SearchResponse response1 = searchScrollRequestBuilder.get();
    
                    //TODO:处理数据
                    SearchHits hits2 = response1.getHits();
                    if(hits2.getHits().length == 0){
                        break;
                    }
                    for(int i = 0; i < hits2.getHits().length; i++) {
                        result.add(hits2.getHits()[i].getSourceAsString());
                    }
                    //下一批处理
                    scrollId = response1.getScrollId();
                }
    
                System.out.println(result.size());
                long endTime = System.currentTimeMillis();
                System.out.println("Java程序运行时间:" + (endTime - startTime) + "ms");
            }catch(Exception e){
                e.printStackTrace();
            }
    
        }

    Scroll原理查看Elasticsearch 使用scroll滚动技术实现大数据量搜索

    转载于:https://blog.csdn.net/zx711166/article/details/77847134

  • 相关阅读:
    已开启博客园~
    友链
    javacv 接收RTSP流(或avi/mp4视频文件),输出byte[]
    Springboot项目中,使用logback来管理日志。
    PPT文件流转为图片,并压缩成ZIP文件输出到指定目录
    通过AOP自定义注解实现记录用户操作日志。
    使用javacv,解码socket接收的H264码流(byte[]),转为yuv处理,最后再合成转为H264
    idea 开始java之旅
    浅谈Winform控件开发(一):使用GDI+美化基础窗口
    WinformGDI+入门级实例——扫雷游戏(附源码)
  • 原文地址:https://www.cnblogs.com/it-deepinmind/p/14583555.html
Copyright © 2011-2022 走看看