Elasticsearch

Elasticsearch

ElasticSearch是一个基于Lucene的搜索服务器。它提供了一个分布式多用户能力的全文搜索引擎，基于RESTful web接口。Elasticsearch是用Java开发的，并作为Apache许可条款下的开放源码发布，是当前流行的企业级搜索引擎。设计用于云计算中，能够达到实时搜索，稳定，可靠，快速，安装使用方便。

Elasticsearch Reference

参考文档：

https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html

ES安装

下载elasticsearch-2.2.0.tar.gz，解压，执行/bin/elasticsearch;

访问地址：http://node1:9200

安装插件 head

执行bin/plugin install mobz/elasticsearch-head

访问地址：http://node1:9200/_plugin/head

如果用root用户执行启动脚本会提示：

Exception in thread "main" java.lang.RuntimeException: don't run elasticsearch as root.

创建elasticsearch用户执行

adduser –U elasticsearch

然后以 elasticsearch用户登录，启动程序。

集群部署

部署三台机器，部署程序，然后修改配置文件 config/elasticsearch.yml

配置：

cluster.name: my-application
node.name: node1
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping_timeout: 120s
client.transport.ping_timeout: 60s
discovery.zen.ping.unicast.hosts: ["node1","node2", "node3"]

集群名字cluster.name相同才能组成一个集群，修改node.name对应的主机名。

防止脑裂配置

discovery.zen.minimum_master_nodes=2

Curl

通过curl操作Elasticsearch数据：

curl -XPUT ‘http://node1:9200/index1/’ -d ‘传输的数据’

-X指定http请求方法：HEAD, PUT, GET, PST, DELETE

-d 指定传输的数据

–GET：获取对象的当前状态；

–PUT：改变对象的状态；      幂等

–POST：创建对象；

–DELETE：删除对象；          幂等

–HEAD：获取头信息。

Elasticsearch与关系型数据库的对比

Database	Index
Table	Type
Row	Document
Column	Field

 

 

 

 

 

创建Index

curl -XPUT 'http://node1:9200/test/'

创建Type

curl -XPOST http://node1:9200/test/emp/ -d '{"name" : "John"}' 自动创建id,也可以自己指定id
curl -XPUT http://node1:9200/test/emp/1 -d '{"name" : "Tom"}'  需要指定id

查询

curl -XGET http://node1:9200/test/emp/1

{"_index":"test","_type":"emp","_id":"1","_version":2,"found":true,"_source":{"name" : "Jim"}}

curl -XGET http://node1:9200/test/emp/1?pretty

{
  "_index" : "test",
  "_type" : "emp",
  "_id" : "1",
  "_version" : 2,
  "found" : true,
  "_source" : {
    "name" : "Jim"
  }
}

pretty 格式化结果，更好看

查询带报文头

curl -i http://node1:9200/test/emp/1?pretty

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Content-Length: 134
{
  "_index" : "test",
  "_type" : "emp",
  "_id" : "1",
  "_version" : 2,
  "found" : true,
  "_source" : {
    "name" : "Jim"
  }
}

查询_source

curl -XGET http://node1:9200/test/emp/1/_source

{"name" : "Jim"}

查询指定field的信息

curl -XGET http://node1:9200/test/emp/1?_source=name

查询type信息

curl -XGET http://node1:9200/test/emp/_search?pretty

{
  "took" : 7,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : "test",
      "_type" : "emp",
      "_id" : "AVYw5zrHBTthl-73fXCK",
      "_score" : 1.0,
      "_source" : {
        "name" : "John"
      }
    }, {
      "_index" : "test",
      "_type" : "emp",
      "_id" : "2",
      "_score" : 1.0,
      "_source" : {
        "name" : "John"
      }
    }, {
      "_index" : "test",
      "_type" : "emp",
      "_id" : "1",
      "_score" : 1.0,
      "_source" : {
        "name" : "Jim"
      }
    } ]
  }
}

查询指定条件数据

curl -XGET http://node1:9200/test/emp/_search?q=name:Jim

Domain Specific Language DSL 查询

curl -XGET http://node1:9200/test/emp/_search?pretty -d  
'{"query":
    {"match":
        {"name":"Jim"}
    }
}'

获取多个文档

curl -XGET http://node1:9200/_mget?pretty -d 
'{"docs":[{"_index":"test","_type":"emp","_id":1},{"_index":"test","_type":"emp","_id":2}]
}'

curl -XGET http://node1:9200/test/emp/_mget?pretty -d 
'{"ids":["1","2"]
}'

检查文档是否存在，只返回报文头

curl -i -XHEAD http://node1:9200/test/emp/1?pretty

更新字段信息

curl -XPUT http://node1:9200/test/emp/1?pretty -d 
'{"name":"Tom",}'

局部更新(增加first_name字段) 只能用POST

curl -XPOST http://node1:9200/test/emp/1/_update -d '{"doc":{"first_name":"Jim"}}'

删除文档

curl -XDELETE http://node1:9200/test/emp/1

指定id

查询设置 副本数量，分片数量

curl -XGET http://node1:9200/test/_settings?pretty

{
  "test" : {
    "settings" : {
      "index" : {
        "creation_date" : "1469698597831",
        "uuid" : "VFew5vmlQUC3Uieu62w23Q",
        "number_of_replicas" : "1",
        "number_of_shards" : "5",
        "version" : {
          "created" : "2020099"
        }
      }
    }
  }
}

设置分片及副本数

curl -XPUT http://node1:9200/test/_settings -d'{"settings":{"number_of_shards":3,"number_of_replicas":2}}'

查询mapping信息

curl -XGET http://node1:9200/test/_mapping?pretty

用于修改字段相关属性，如字段类型，是否使用分词器，使用哪种分词器

curl -XPOST http://node1:9200/test/emp/_mapping -d 
'{
    "emp": {
        "properties": {
              "name" : {
                "type" : "string",
                "analyzer" : "standard"
              }
        }
    }
}'

解析器

curl -XGET 'http://node1:9200/_analyze?analyzer=standard&pretty=true' -d '很高兴认识你'

使用解析器查看结果

具体说明参考官方文档：https://www.elastic.co/guide/en/elasticsearch/reference/2.2/analyzer.html

Java API

官方文档：

https://www.elastic.co/guide/en/elasticsearch/client/java-api/2.2/index.html

 

测试demo：

注意：这里使用term时注意使用小写，由于使用了分词器

import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.search.SearchType;
import org.elasticsearch.client.Client;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.InetSocketTransportAddress;
import org.elasticsearch.common.unit.TimeValue;
import org.elasticsearch.common.xcontent.XContentFactory;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

import java.io.IOException;
import java.net.InetAddress;
import java.net.UnknownHostException;

/**
 * Created by Edward on 2016/7/29.
 */
public class MyTest {

    public static Client client;

    @Before
    public void init() throws UnknownHostException {
        Settings settings = Settings.settingsBuilder()
                .put("cluster.name", "my-application").build();
        client = TransportClient.builder().settings(settings).build()
                .addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("node1"), 9300))
                .addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("node2"), 9300))
                .addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("node3"), 9300));
    }
    @After
    public void closeClient()
    {
        client.close();
    }

    @Test
    public void createIndex() throws IOException {
        //创建索引
        IndexResponse response = client.prepareIndex("test", "emp", "1")
                .setSource(XContentFactory.jsonBuilder()
                        .startObject()
                        .field("name", "Tom")
                        .field("age", "21")
                        .field("message", "Elasticsearch")
                        .endObject())
                .get();
        System.out.println(response);
    }

    @Test
    public void queryIndex(){
        //获取索引信息
        GetResponse res = client.prepareGet("test", "emp", "1").get();
        System.out.println(res.getSource());
    }

    @Test
    public void search(){

        SearchResponse response = client.prepareSearch("test")
                .setTypes("emp")
                .setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
                //注意term使用了分词，这里tom使用小写
                .setQuery(QueryBuilders.termQuery("name", "tom"))                 // Query
                //.setPostFilter(QueryBuilders.rangeQuery("age").from(12).to(18))     // Filter
                //.setFrom(0).setSize(60).setExplain(true)
                .execute()
                .actionGet();
        System.out.println(response.getHits().getHits().length);
        //SearchResponse response = client.prepareSearch().execute().actionGet();
        System.out.println(response.toString());
    }


    @Test
    public void scrollQuery(){

        //注意由于使用了分词，这里tom使用小写
        QueryBuilder qb = QueryBuilders.termQuery("name", "tom");

        SearchResponse scrollResp = client.prepareSearch("test")
                .setTypes("emp")
                .setSearchType(SearchType.SCAN)
                .setScroll(new TimeValue(60000))
                .setQuery(qb)
                .setSize(100).execute().actionGet(); //100 hits per shard will be returned for each scroll
//Scroll until no hits are returned
        while (true) {
            scrollResp = client.prepareSearchScroll(scrollResp.getScrollId()).setScroll(new TimeValue(60000)).execute().actionGet();
            for (SearchHit hit : scrollResp.getHits().getHits()) {
                //Handle the hit...
                System.out.println(hit.getSource());
            }

            //Break condition: No hits are returned
            if (scrollResp.getHits().getHits().length == 0) {
                break;
            }
        }
    }
}