搭建spring boot+elasticsearch+activemq服务

目前时间是：2017-01-24

本文不涉及activemq的安装

需求

activemq实时传递数据至服务 elasticsearch做索引 对外开放查询接口 完成全文检索

环境

jdk：1.8

spirng boot：1.4.3.RELEASE

elasticsearch：2.4.3

activemq：5.13.2

ES插件

head：版本好像无太大差别 能查数据就行

analysis-ik：1.10.3

search-guard-2：2.4.3.9

search-guard-ssl：2.4.3.19

注意

作者遇到的最大问题就是版本兼容 网上资料少且版本较低 故列出以下版本矩阵 方便查阅

spring boot与elasticsearch：

Spring Boot Version (x)	Spring Data Elasticsearch Version (y)	Elasticsearch Version (z)
x <= 1.3.5	y <= 1.3.4	z <= 1.7.2*
x >= 1.4.x	2.0.0 <=y < 5.0.0**	2.0.0 <= z < 5.0.0**

矩阵来源以及更多版本兼容：https://github.com/spring-projects/spring-data-elasticsearch

elasticsearch与ik：我的ES版本为2.x 对应如下

IK version	ES version
master	2.4.0 -> master
1.10.3	2.4.3
1.9.5	2.3.5
1.9.4	2.3.4
1.9.3	2.3.3
1.9.0	2.3.0
1.8.1	2.2.1
1.7.0	2.1.1
1.5.0	2.0.0
1.2.6	1.0.0
1.2.5	0.90.x
1.1.3	0.20.x
1.0.0	0.16.2 -> 0.19.0

矩阵来源以及更多版本兼容：https://github.com/medcl/elasticsearch-analysis-ik

elasticsearch与search-guard以及search-guard-ssl：

Elasticsearch Version	Latest Search Guard Version	Search Guard SSL Version	Commercial support available
1.x.y	not available	-	-
2.0.x	not available	-	-
2.1.x	not available	-	-
2.2.0	2.2.0.7	2.2.0.16	Yes
2.3.1	available upon request	-	-
2.3.2	available upon request	-	-
2.3.3	2.3.3.10	2.3.3.19	YES
2.3.4	2.3.4.10	2.3.4.19	YES
2.3.5	2.3.5.10	2.3.5.19	YES
2.4.0	2.4.0.10	2.4.0.19	YES
2.4.1	2.4.1.10	2.4.1.19	YES
2.4.2	2.4.2.10	2.4.2.19	YES
2.4.3	2.4.3.10	2.4.3.19	YES
2.4.4	2.4.4.10	2.4.4.19	YES
5.0.0	5.0.0-10	(comes bundled since SG 5)	YES
5.0.1	5.0.1-10	(comes bundled since SG 5)	YES
5.0.2	5.0.2-10	(comes bundled since SG 5)	YES
5.1.1	5.1.1-10	(comes bundled since SG 5)	YES
5.1.2	5.1.2-10	(comes bundled since SG 5)	YES

矩阵来源以及更多版本兼容：https://github.com/floragunncom/search-guard/wiki

开始

安装elasticsearch

我的安装目录：/usr/local

注意：elasticsearch不能用root用户运行 所以创建你的用户组和用户 切换到新用户再安装 如何创建切换 请自行搜索

wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.4.3/elasticsearch-2.4.3.tar.gz

tar -zxvf elasticsearch-2.4.3.tar.gz 

cd elasticsearch-2.4.3/config/

vim elasticsearch.yml 

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please see the documentation for further information on configuration options:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html>
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
  cluster.name: 你的集群名称
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
# node.name: node-1
#
# Add custom attributes to the node:
#
# node.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
# path.data: /path/to/data
#
# Path to log files:
#
# path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
# bootstrap.memory_lock: true
#
# Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory
# available on the system and that the owner of the process is allowed to use this limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
  network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
# http.port: 9200
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html>
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
# discovery.zen.ping.unicast.hosts: ["host1", "host2"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):
#
# discovery.zen.minimum_master_nodes: 3
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html>
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
# gateway.recover_after_nodes: 3
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html>
#
# ---------------------------------- Various -----------------------------------
#
# Disable starting multiple nodes on a single system:
#
# node.max_local_storage_nodes: 1
#
# Require explicit names when deleting indices:
#
# action.destructive_requires_name: true

我修改了两个地方 cluster.name和network.host

cd ../bin/

./elasticsearch

[2017-01-24 10:02:49,627][INFO ][node                     ] [Ariel] version[2.4.3], pid[23274], build[d38a34e/2016-12-07T16:28:56Z]
[2017-01-24 10:02:49,628][INFO ][node                     ] [Ariel] initializing ...
[2017-01-24 10:02:50,259][INFO ][plugins                  ] [Ariel] modules [reindex, lang-expression, lang-groovy], plugins [], sites []
[2017-01-24 10:02:50,279][INFO ][env                      ] [Ariel] using [1] data paths, mounts [[/ (overlay)]], net usable_space [75.3gb], net total_space [113.9gb], spins? [possibly], types [overlay]
[2017-01-24 10:02:50,279][INFO ][env                      ] [Ariel] heap size [990.7mb], compressed ordinary object pointers [true]
[2017-01-24 10:02:52,051][INFO ][node                     ] [Ariel] initialized
[2017-01-24 10:02:52,051][INFO ][node                     ] [Ariel] starting ...
[2017-01-24 10:02:52,110][INFO ][transport                ] [Ariel] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2017-01-24 10:02:52,116][INFO ][discovery                ] [Ariel] elasticsearch/MI21JVBWSbKfj9nC1V6N9w
[2017-01-24 10:02:55,166][INFO ][cluster.service          ] [Ariel] new_master {Ariel}{MI21JVBWSbKfj9nC1V6N9w}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2017-01-24 10:02:55,197][INFO ][http                     ] [Ariel] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2017-01-24 10:02:55,197][INFO ][node                     ] [Ariel] started

出现类似这样的信息 说明安装成功

安装head插件

elasticsearch根目录执行

bin/plugin install mobz/elasticsearch-head

注意：2.0以下版本应该是 -install

访问地址：http://ip:9200/_plugin/head/

安装analysis-ik插件

下载地址：https://github.com/medcl/elasticsearch-analysis-ik/tree/v1.10.3

使用maven打包：mvn clean package

生成的zip包在target/releases下

在elasticsearch-2.4.3/plugins下创建ik目录

将zip包放到该目录下并解压 解压出如下文件

 编辑elasticsearch-2.4.3/config/elasticsearch.yml配置文件 添加如下内容

index:  
      analysis:                     
        analyzer:        
          ik:  
              alias: [ik_analyzer]  
              type: org.elasticsearch.index.analysis.IkAnalyzerProvider  
          ik_max_word:  
              type: ik  
              use_smart: false  
          ik_smart:  
              type: ik  
              use_smart: true

或者

index.analysis.analyzer.ik.type : “ik”

测试：http://ip:9200/_analyze?analyzer=ik&pretty=true&text=我是中国人

安装searchguard

elasticsearch根目录执行

bin/plugin install -b com.floragunn/search-guard-2/2.4.3.9

bin/plugin install -b com.floragunn/search-guard-ssl/2.4.3.19

下载 searchguard-ssl 的包，里面包含自动创建证书的脚本：

wget https://github.com/floragunncom/search-guard-ssl/archive/v2.4.3.19.zip

unzip v2.4.3.19.zip 

cd search-guard-ssl-2.4.3.19/example-pki-scripts/

有三个脚本

gen_client_node_cert.sh 创建客户端证书
gen_node_cert.sh        创建节点证书
gen_root_ca.sh          创建根证书

编辑脚本 vim example.sh

#!/bin/bash
set -e
./clean.sh
./gen_root_ca.sh password password 
./gen_node_cert.sh 0 password  password 
./gen_node_cert.sh 1 password  password 
./gen_client_node_cert.sh admin password password 
cp truststore.jks node-0-keystore.jks /usr/local/elasticsearch-2.4.3/config/
cp truststore.jks admin-keystore.jks /usr/local/elasticsearch-2.4.3/plugins/search-guard-2/sgconfig/

./example.sh 

可以发现 最后两句就是将证书cp到相应目录

编辑elasticsearch-2.4.3/config/elasticsearch.yml配置文件 添加如下内容

#############################################################################################
#                                     SEARCH GUARD                                          #
#                                     Configuration                                         #
#############################################################################################
  security.manager.enabled: false
  searchguard.authcz.admin_dn:
   - "CN=admin, OU=client, O=client, L=Test, C=DE"

#############################################################################################
#                                     SEARCH GUARD SSL                                      #
#                                       Configuration                                       #
#############################################################################################


#############################################################################################
# Transport layer SSL                                                                       #
#                                                                                           #
#############################################################################################
# Enable or disable node-to-node ssl encryption (default: true)
# searchguard.ssl.transport.enabled: true
# JKS or PKCS12 (default: JKS)
#searchguard.ssl.transport.keystore_type: PKCS12
# Relative path to the keystore file (mandatory, this stores the server certificates), must be placed under the config/ dir
  searchguard.ssl.transport.keystore_filepath: node-0-keystore.jks
# Alias name (default: first alias which could be found)
#searchguard.ssl.transport.keystore_alias: my_alias
# Keystore password (default: changeit)
  searchguard.ssl.transport.keystore_password: password
# JKS or PKCS12 (default: JKS)
#searchguard.ssl.transport.truststore_type: PKCS12
# Relative path to the truststore file (mandatory, this stores the client/root certificates), must be placed under the config/ dir
  searchguard.ssl.transport.truststore_filepath: truststore.jks
# Alias name (default: first alias which could be found)
#searchguard.ssl.transport.truststore_alias: my_alias
# Truststore password (default: changeit)
  searchguard.ssl.transport.truststore_password: password
# Enforce hostname verification (default: true)
# searchguard.ssl.transport.enforce_hostname_verification: true
# If hostname verification specify if hostname should be resolved (default: true)
# searchguard.ssl.transport.resolve_hostname: true
# Use native Open SSL instead of JDK SSL if available (default: true)
# searchguard.ssl.transport.enable_openssl_if_available: false

在elasticsearch根目录 执行命令 将配置插入

./plugins/search-guard-2/tools/sgadmin.sh -cn 集群名称 -h hostname -cd plugins/search-guard-2/sgconfig -ks plugins/search-guard-2/sgconfig/admin-keystore.jks -kspass password -ts plugins/search-guard-2/sgconfig/truststore.jks -tspass password -nhnv

注意：elasticsearch的服务必须是运行状态

elasticsearch-2.4.3/plugins/search-guard-2/sgconfig下的配置文件是管理用户角色的

安装配置成功后 任何客户端访问elasticsearch 需提供用户名及密码

至此服务端安装结束

客户端将以源码方式提供 为公司信息安全着想 仅提供关键性代码供参考 无法运行

elasticsearch download