elasticsearch

elasticsearch安装及安装中遇到的坑：

一、安装步骤：

上传三个安装文件：

elasticsearch-5.2.2.tar   --elasticsearch的安装包

elasticsearch-head-master  --连接elasticsearch需要的插件安装包

node-v6.9.2-linux-x64.tar  --安装插件所需要的nodejs继承环境

分别上传并进行解压

2、在/opt/module/elasticsearch-5.2.2路径下创建data和logs文件夹

3、修改配置文件/opt/module/elasticsearch-5.2.2/config/elasticsearch.yml

需要修改的地方：

cluster.name: my-application    --集群的名称

node.name: node-102   --集群节点的名称

path.data: /opt/module/elasticsearch-5.2.2/data  --配置数据存储路劲

path.logs: /opt/module/elasticsearch-5.2.2/logs  --配置日志存储路径

bootstrap.memory_lock: false

bootstrap.system_call_filter: false

network.host: 182.168.126.xxx

discovery.zen.ping.unicast.hosts: ["mynode1"],要是集群有多个节点就将这些节点全部写上

添加防脑裂配置： 如果不配不知道具体数量，不好控制脑裂

discovery.zen.ping.multicast.enabled: false

discovery.zen.ping.unicast.hosts: ["192.168.133.6","192.168.133.7", "192.168.133.8"]

discovery.zen.ping_timeout: 120s

client.transport.ping_timeout: 60s

集群搭建注意的点：

（1）、如果要配置集群需要两个节点上的elasticsearch配置的cluster.name相同，都启动可以自动组成集群，这里如果不改cluster.name则默认是cluster.name=my-application

（2）、nodename随意取但是集群内的各节点不能相同

（3）、修改后的每行前面不能有空格，修改后的“：”后面必须有一个空格

4、配置linux系统环境（参考：http://blog.csdn.net/satiling/article/details/59697916）

（1）、切换到root用户，编辑limits.conf 添加类似如下内容

[root@hadoop102 elasticsearch-5.2.2]# vi /etc/security/limits.conf

添加如下内容:

* soft nofile 65536

* hard nofile 131072

* soft nproc 2048

* hard nproc 4096

注：这些参数主要就是申请一些内存资源

（2）、切换到root用户，进入limits.d目录下修改配置文件。

修改如下内容：

* soft nproc 1024

#修改为

* soft nproc 2048

（3）、切换到root用户修改配置sysctl.conf

[root@hadoop102 elasticsearch-5.2.2]# vi /etc/sysctl.conf 

添加下面配置：

vm.max_map_count=655360

并执行命令：

[root@hadoop102 elasticsearch-5.2.2]# sysctl -p

然后，重新启动elasticsearch，即可启动成功。（注：集群启动的时候一定要在子用户下面启动，root用户不能启动，要与elasticsearch有远程脚本访问权限，如果在root下启动会不安全）

5、[atguigu@hadoop102 elasticsearch-5.2.2]$ bin/elasticsearch

6、测试集群安装进度：登陆 mynode1:9200 查看网页信息

elasticsearch head 插件安装步骤：

1、下载插件 https://github.com/mobz/elasticsearch-head elasticsearch-head-master.zip

2、配置nodejs环境变量：

export NODE_HOME=/opt/module/node-v6.9.2-linux-x64

export PATH=$PATH:$NODE_HOME/bin

[root@hadoop102 software]# source /etc/profile

3、查看node和npm版本

[root@hadoop102 software]# node -v

v6.9.2

[root@hadoop102 software]# npm -v

3.10.9

4、解压head插件到/opt/module目录下

[atguigu@hadoop102 software]$ unzip elasticsearch-head-master.zip -d /opt/module/

5、查看当前head插件目录下有无node_modules/grunt目录：

没有：执行命令创建：

[atguigu@hadoop102 elasticsearch-head-master]$ npm install grunt --save

6、安装head插件：

[atguigu@hadoop102 elasticsearch-head-master]$ npm install -g cnpm --registry=https://registry.npm.taobao.org

7、安装grunt：

[atguigu@hadoop102 elasticsearch-head-master]$ npm install -g grunt-cli

8、编辑Gruntfile.js

[atguigu@hadoop102 elasticsearch-head-master]$ vim Gruntfile.js
文件93行添加hostname:'0.0.0.0'
options: {
        hostname:'0.0.0.0',
        port: 9100,
        base: '.',
        keepalive: true
      }

9、检查head根目录下是否存在base文件夹

没有：将 _site下的base文件夹及其内容复制到head根目录下

[atguigu@hadoop102 elasticsearch-head-master]$ mkdir base

[atguigu@hadoop102 _site]$ cp base/* ../base/

10、启动grunt server：

[atguigu@hadoop102 elasticsearch-head-master]$ grunt server -d

启动信息：

Running "connect:server" (connect) task
[D] Task source: /opt/module/elasticsearch-head-master/node_modules/grunt-contrib-connect/tasks/connect.js
Waiting forever...
Started connect web server on http://localhost:9100

如果提示grunt的模块没有安装：

Local Npm module “grunt-contrib-clean” not found. Is it installed? 

Local Npm module “grunt-contrib-concat” not found. Is it installed? 

Local Npm module “grunt-contrib-watch” not found. Is it installed? 

Local Npm module “grunt-contrib-connect” not found. Is it installed? 

Local Npm module “grunt-contrib-copy” not found. Is it installed? 

Local Npm module “grunt-contrib-jasmine” not found. Is it installed? 

Warning: Task “connect:server” not found. Use –force to continue. 

执行以下命令： 

npm install grunt-contrib-clean -registry=https://registry.npm.taobao.org

npm install grunt-contrib-concat -registry=https://registry.npm.taobao.org

npm install grunt-contrib-watch -registry=https://registry.npm.taobao.org 

npm install grunt-contrib-connect -registry=https://registry.npm.taobao.org

npm install grunt-contrib-copy -registry=https://registry.npm.taobao.org 

npm install grunt-contrib-jasmine -registry=https://registry.npm.taobao.org

最后一个模块可能安装不成功，但是不影响使用。

11、启动集群插件后发现集群未连接

在/opt/module/elasticsearch-5.2.2/config路径下修改配置文件elasticsearch.yml，在文件末尾增加

[atguigu@hadoop102 config]$ pwd

/opt/module/elasticsearch-5.2.2/config

[atguigu@hadoop102 config]$ vi elasticsearch.yml

http.cors.enabled: true

http.cors.allow-origin: "*"

再重新启动elasticsearch。

注：如果按照上述要求修改之后，健康值依然连接不成功，在配置文件elasticsearch.yml文件末尾增加

#http.cors.allow-methods: OPTIONS, HEAD, GET, POST, PUT, DELETE
#http.cors.allow-headers: "X-Requested-With, Content-Type, Content-Length, X-User"

并将健康值URL地址栏中默认的localhost改为本机的ip地址

用java -API操作elasticsearch： （JDK环境必须是JDK-1.8的）

Elasticsearch的Java客户端非常强大；它可以建立一个嵌入式实例并在必要时运行管理任务。

    运行一个Java应用程序和Elasticsearch时，有两种操作模式可供使用。该应用程序可在Elasticsearch集群中扮演更加主动或更加被动的角色。在更加主动的情况下（称为Node Client），应用程序实例将从集群接收请求，确定哪个节点应处理该请求，就像正常节点所做的一样。（应用程序甚至可以托管索引和处理请求。）另一种模式称为Transport Client，它将所有请求都转发到另一个Elasticsearch节点，由后者来确定最终目标。

1、新建一个maven工程。并在工程中添加pom文件依赖，下载相应的jar包

 

<dependencies>

<dependency>

<groupId>junit</groupId>

<artifactId>junit</artifactId>

<version>3.8.1</version>

<scope>test</scope>

</dependency>

<dependency>

<groupId>org.elasticsearch</groupId>

<artifactId>elasticsearch</artifactId>

<version>5.2.2</version>

</dependency>

<dependency>

<groupId>org.elasticsearch.client</groupId>

<artifactId>transport</artifactId>

<version>5.2.2</version>

</dependency>

<dependency>

<groupId>org.apache.logging.log4j</groupId>

<artifactId>log4j-core</artifactId>

<version>2.9.0</version>

</dependency>

</dependencies>

 

等待依赖的jar包下载完成，当直接在ElasticSearch 建立文档对象时，如果索引不存在的，默认会自动创建，映射采用默认方式

 elasticsearch API操作源码实例：

package com.wcg.elasticsearch;

import java.io.IOException;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;
import java.util.concurrent.ExecutionException;

import org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.action.get.MultiGetItemResponse;
import org.elasticsearch.action.get.MultiGetResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.action.update.UpdateResponse;
import org.elasticsearch.client.Requests;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.InetSocketTransportAddress;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentFactory;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.transport.client.PreBuiltTransportClient;
import org.junit.Before;
import org.junit.Test;

/**
 * Hello world!
 *
 */
public class App 
{
    TransportClient client;
    @SuppressWarnings("unchecked")
    @Before
    public  void getClient()
    {
        Settings setting =Settings.builder().put("cluster.name","my-application").build();
        //设置连接集群的名称
        
        client = new PreBuiltTransportClient(setting);
         
        try {
            
            client.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("mynode1"),9300));
            System.out.println(client.toString());
        } catch (UnknownHostException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
        
    
    }
    
    //创建索引（就相当于创建一个数据库）
    @Test
    public void createIndex() {
        //创建索引
        client.admin().indices().prepareCreate("myblog").get();
        //关闭资源
        client.close();
    }
    
    //删除索引
    @Test
    public void deleteIndex() {
        
        client.admin().indices().prepareDelete("myblog").get();
        
        client.close();
        
    }
    //创建文档json格式
    @Test
    public void createDoc() {
        //具体创建的文档的内容
        String json = "{" + ""id":"1"," + ""title":"基于Lucene的搜索服务器","
                + ""content":"它提供了一个分布式多用户能力的全文搜索引擎，基于RESTful web接口"" + "}";
        //创建文档
        IndexResponse response = client.prepareIndex("myblog","article","1").setSource(json).execute().actionGet();
        //打印返回值
        System.out.println("索引"+response.getIndex());
        System.out.println("类型"+response.getType());
        System.out.println("版本号"+response.getVersion());
        System.out.println("结果"+response.getResult());
        client.close();
        
    }
    
    //创建文档以hashmap的形式
    @Test
    public void createIndexByMap() {
        
                // 1 文档数据准备
                Map<String, Object> json = new HashMap<String, Object>();
                json.put("id", "2");
                json.put("title", "基于Lucene的搜索服务器");
                json.put("content", "它提供了一个分布式多用户能力的全文搜索引擎，基于RESTful web接口");

               //创建文档
                IndexResponse response = client.prepareIndex("myblog","article","2").setSource(json).execute().actionGet();
        
                //打印返回结果
                System.out.println("索引"+response.getIndex());
                System.out.println("类型"+response.getType());

                System.out.println("版本号"+response.getVersion());

                System.out.println("返回结果"+response.getResult());

                System.out.println("id"+response.getId());
                
                //关闭资源
                client.close();
                

    }
    
    //创建文档以builder的形式
    @Test
    public void createIndexByBuilder() throws IOException {
        // 1 通过es自带的帮助类，构建json数据
                XContentBuilder builder = XContentFactory.jsonBuilder().startObject().field("id", 3)
                        .field("title", "学霸").field("content", "能力的全文搜索引擎，基于RESTful web接口。")
                        .endObject();
                //创建文档
                IndexResponse response = client.prepareIndex("myblog","article","3").setSource(builder).execute().actionGet();
        
                //打印返回结果
                System.out.println("索引"+response.getIndex());
                System.out.println("类型"+response.getType());

                System.out.println("版本号"+response.getVersion());

                System.out.println("返回结果"+response.getResult());

                System.out.println("id"+response.getId());
                
                //关闭资源
                client.close();
        
    }
    
    //单个索引
    @Test
    public void queryIndex() {
        //查询 
        GetResponse response = client.prepareGet("myblog","article","2").get();
        
        //打印
        System.out.println(response.getSourceAsString());
        
        client.close();
    }
    
    //多个索引查询
    @Test
    public void queryIndexs() {
        
        MultiGetResponse response = client.prepareMultiGet().add("myblog", "article", "1").add("myblog", "article", "2").add("myblog", "article", "3").get();
        
        for(MultiGetItemResponse multiGetItemResponse : response) {
            GetResponse response2 = multiGetItemResponse.getResponse();
            if(response2.isExists()) {
                System.out.println(response2.getSourceAsString());
            }
        }
        client.close();
    }
    
    //跟新文档 
    @Test
    public void update() throws IOException, InterruptedException, ExecutionException {
        UpdateRequest updateRequest = new UpdateRequest("myblog","article","2");
        updateRequest.doc(XContentFactory.jsonBuilder().startObject().field("id","2")
                .field("title","大牛")
                .field("content","苏大虎，看包").endObject());
        
        UpdateResponse response = client.update(updateRequest).get();
        
        System.out.println("索引"+response.getIndex());
        System.out.println("类型"+response.getType());

        System.out.println("版本号"+response.getVersion());

        System.out.println("返回结果"+response.getResult());
        
        client.close();
        
    }
    
    //跟新文档（有则跟新，没有则插入类似于oracle的merge）
    @Test
    public void updateinsert() throws IOException, InterruptedException, ExecutionException {
        //没有这个文档内容就进行创建
        IndexRequest indexrequest = new IndexRequest("blog","article","5");
        indexrequest.source(XContentFactory.jsonBuilder().startObject().field("id","5")
                .field("title","二牛")
                .field("content","苏小虎，喝水").endObject());
        
        //有文档内容就进行跟新
        UpdateRequest updateRequest = new UpdateRequest("myblog","article","5");
        updateRequest.doc(XContentFactory.jsonBuilder().startObject().field("id","5")
                .field("title","黄牛")
                .field("content","苏二虎，迈赛").endObject());
        
        updateRequest.upsert(indexrequest);
        //具体跟新操作
        UpdateResponse response = client.update(updateRequest).get();
        
        System.out.println("索引"+response.getIndex());
        System.out.println("类型"+response.getType());

        System.out.println("版本号"+response.getVersion());

        System.out.println("返回结果"+response.getResult());
        
        client.close();
    }
    
    //删除文档内容
    @Test
    public  void  delIndex(){
        DeleteResponse response = client.prepareDelete("myblog","article","5").get();
        
        client.close();
        
        
    }
    
    //全文查询 
    @Test
    public void queryMatchAll() {
    //查询    
    SearchResponse response =    client.prepareSearch("myblog").setTypes("article").setQuery(QueryBuilders.matchAllQuery()).get();
    //获取查询的对象    
    SearchHits hits = response.getHits();
    //打印查询结果条数
    System.out.println("查询结果有几条数据"+hits.getTotalHits());
    //遍历查询结果
    Iterator<SearchHit> iterator = hits.iterator();
    while(iterator.hasNext()) {
        
        SearchHit next = iterator.next();
        System.out.println(next.sourceAsString());
        
        
    }
    client.close();
    
    }
    
    //分词查询
    @Test 
    public void query() {
        
    SearchResponse response =    client.prepareSearch("myblog").setTypes("article").setQuery(QueryBuilders.queryStringQuery("全文")).get();
    //获取具体的查询对象
    SearchHits hits = response.getHits();
  //打印查询结果条数
    System.out.println("查询结果有几条数据"+hits.getTotalHits());
    //遍历查询结果
    Iterator<SearchHit> iterator = hits.iterator();
    while(iterator.hasNext()) {
        
        SearchHit next = iterator.next();
        
        System.out.println(next.getSourceAsString());
    }
    client.close();
        
    }
    
    //通配符查询
    
    @Test 
    public void wildcardQuery() {
        
    SearchResponse response =    client.prepareSearch("myblog").setTypes("article").setQuery(QueryBuilders.wildcardQuery("content", "*全*")).get();
    //获取具体的查询对象
    SearchHits hits = response.getHits();
  //打印查询结果条数
    System.out.println("查询结果有几条数据"+hits.getTotalHits());
    //遍历查询结果
    Iterator<SearchHit> iterator = hits.iterator();
    while(iterator.hasNext()) {
        
        SearchHit next = iterator.next();
        
        System.out.println(next.getSourceAsString());
    }
        
    client.close();
    }
    //字段查询 
    @Test 
    public void queryterm() {
        
    SearchResponse response =    client.prepareSearch("myblog").setTypes("article").setQuery(QueryBuilders.termQuery("content", "全")).get();
    //获取具体的查询对象
    SearchHits hits = response.getHits();
  //打印查询结果条数
    System.out.println("查询结果有几条数据"+hits.getTotalHits());
    //遍历查询结果
    Iterator<SearchHit> iterator = hits.iterator();
    while(iterator.hasNext()) {
        
        SearchHit next = iterator.next();
        
        System.out.println(next.getSourceAsString());
    }
        
    client.close();
    }
    
    //模糊查询
    @Test 
    public void fuzzyQuery() {
        
    SearchResponse response =    client.prepareSearch("myblog").setTypes("article").setQuery(QueryBuilders.fuzzyQuery("content", "全1")).get();
    //获取具体的查询对象
    SearchHits hits = response.getHits();
  //打印查询结果条数
    System.out.println("查询结果有几条数据"+hits.getTotalHits());
    //遍历查询结果
    Iterator<SearchHit> iterator = hits.iterator();
    while(iterator.hasNext()) {
        
        SearchHit next = iterator.next();
        
        System.out.println(next.getSourceAsString());
    }
        
    client.close();
    }
    //创建约束条件 (注：创建约束条件的时候表中是不能有数据的，否则创建是不会成功的，当表中已经有数据的时候，表明表中的约束条件已经存在了)
    @Test
    public void putmapping() throws InterruptedException, ExecutionException, IOException {
        // 1设置mapping
                XContentBuilder builder = XContentFactory.jsonBuilder()
                        .startObject()
                            .startObject("article")
                                .startObject("properties")
                                    .startObject("id1")
                                        .field("type", "string")
                                        .field("store", "yes")
                                    .endObject()
                                    .startObject("title2")
                                        .field("type", "string")
                                        .field("store", "no")
                                    .endObject()
                                    .startObject("content")
                                        .field("type", "string")
                                        .field("store", "yes")
                                    .endObject()
                                .endObject()
                            .endObject()
                        .endObject();

                // 2 添加mapping
                PutMappingRequest mapping = Requests.putMappingRequest("myblog").type("article").source(builder);
                
                client.admin().indices().putMapping(mapping).get();
                
                // 3 关闭资源
                client.close();
        
        
        
        
        
        
    }
    
    
}