Elasticsearch & Kibana with Shield

Elasticsearch & Kibana with Shield

官方网站:

https://www.elastic.co/guide/en/kibana/current/production.html

环境:

kibana-4.5.0

elasticsearch-2.3.2

shield-2.3.2

license-2.3.2

前言:

Shield作为安全插件可以嵌入到ELK当中,商业授权,前30天免费

支持多种认证方式native, file, LDAP, Active Directory, PKI，详情见https://www.elastic.co/guide/en/shield/current/how-shield-works.html

这里以file,native为例

Elasticsearch with Shield

https://www.elastic.co/guide/en/shield/current/file-realm.html#esusers-add

一.停止所有elasticsearch节点

说明:shield安装，卸载，升级都需要重启elasticsearch节点

二.安装shield插件(所有elasticsearch节点)

https://www.elastic.co/guide/en/shield/current/installing-shield.html

说明:shield版本必须和elasticsearch保持一致

在线安装

/opt/elasticsearch-2.3.2/bin/plugin install license

/opt/elasticsearch-2.3.2/bin/plugin install shield

或离线安装

wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.3.2/license-2.3.2.zip

wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/shield/2.3.2/shield-2.3.2.zip

/opt/elasticsearch-2.3.2/bin/plugin install file:///root/license-2.3.2.zip 

/opt/elasticsearch-2.3.2/bin/plugin install file:///root/shield-2.3.2.zip

[root@ela-master1 ~]# /opt/elasticsearch-2.3.2/bin/plugin install file:///root/license-2.3.2.zip 

-> Installing from file:/root/license-2.3.2.zip...

Trying file:/root/license-2.3.2.zip ...

Downloading .DONE

Verifying file:/root/license-2.3.2.zip checksums if available ...

NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)

Installed license into /opt/elasticsearch-2.3.2/plugins/license

[root@ela-master1 ~]# /opt/elasticsearch-2.3.2/bin/plugin install file:///root/shield-2.3.2.zip

-> Installing from file:/root/shield-2.3.2.zip...

Trying file:/root/shield-2.3.2.zip ...

Downloading .......................DONE

Verifying file:/root/shield-2.3.2.zip checksums if available ...

NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)

Installed shield into /opt/elasticsearch-2.3.2/plugins/shield

注意:一旦shield安装成功，在elasticsearch重启后，对elasticsearch的任何操作都需要授权(用户名和密码)，除非启用匿名用户https://www.elastic.co/guide/en/shield/current/anonymous-access.html

三.配置realm

A.配置file-realm(所有节点)

https://www.elastic.co/guide/en/shield/current/file-realm.html#esusers-add

1.增加file-realm配置

cat >>/opt/elasticsearch-2.3.2/config/elasticsearch.yml <<HERE

shield.authc.realms.file1.type: file

shield.authc.realms.file1.order: 0

HERE

2.启动(或重启)elasticsearch

3.创建file based用户

https://www.elastic.co/guide/en/shield/current/defining-roles.html#valid-role-name

https://www.elastic.co/guide/en/shield/current/enable-basic-auth.html

/opt/elasticsearch-2.3.2/bin/shield/esusers useradd es_admin -p P@ssw0rd -r admin

/opt/elasticsearch-2.3.2/bin/shield/esusers useradd kibana -p P@ssw0rd -r kibana4_server

[root@ela-client ~]# /opt/elasticsearch-2.3.2/bin/shield/esusers list

es_admin       : admin

kibana         : kibana4_server

说明:这里创建了两个用户es_admin(用户名 es_admin, 密码P@ssw0rd, 角色admin)和kibana,用户角色定义可以参看shield配置文件/opt/elasticsearch-2.3.2/config/shield/roles.yml

4.file based用户同步到集群其它节点

注意:对于用户和角色的所有操作默认都存放在如下位置，因此集群中的所有节点都需要上面同样的操作，当然，也可以直接copy如下文件到其它节点的对应目录

/opt/elasticsearch-2.3.2/config/shield/users

/opt/elasticsearch-2.3.2/config/shield/users_roles

5.测试用户认证

[root@ela-client ~]# curl -u es_admin:P@ssw0rd 'http://localhost:9200/_cat/health?v'

epoch      timestamp cluster               status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent 

1462705707 19:08:27  elasticsearch_cluster green           6         2     52  26    0    0        0             0                  -                100.0% 

[root@ela-client ~]# curl -u es_admin:P@ssw0rd 'http://localhost:9200/_cat/indices?v'

health status index               pri rep docs.count docs.deleted store.size pri.store.size 

green  open   shakespeare           5   1     111396            0     36.5mb         18.2mb 

green  open   logstash-2015.05.20   5   1       4750            0     72.9mb         35.7mb 

green  open   bank                  5   1       1000            0    890.5kb        447.9kb 

green  open   .kibana               1   1          4            0     44.5kb         22.2kb 

green  open   logstash-2015.05.18   5   1       4631            0     64.9mb         32.7mb 

green  open   logstash-2015.05.19   5   1       4624            0     66.7mb           32mb 

B.配置native-realm(所有节点)

https://www.elastic.co/guide/en/shield/current/native-realm.html

默认情况下, native-realm已经被配置到了realm链里, 可以直接通过REST API来添加删除用户，修改用户密码及管理角色, 这也是官方推荐的认证方式

1.增加native-realm配置

cat >>/opt/elasticsearch-2.3.2/config/elasticsearch.yml <<HERE

shield.authc.realms.native1.type: native

shield.authc.realms.native1.order: 0

HERE

2.启动(或重启)elasticsearch

3.创建native用户

curl -u es_admin:P@ssw0rd -XPOST 'http://localhost:9200/_shield/user/fooadmin' -d '

{

  "password" : "foo.123", 

  "roles" : [ "admin", "other_role1" ], 

  "full_name" : "Jlive Liu", 

  "email" : "iliujun_live@163.com", 

  "metadata" : { 

    "intelligence" : 7

  }

}

'

说明:

1.新增native用户也需要认证，但启用了shield之后默认是没有native用户的，所以就需要借助file based用户来授权

2.native用户是存放在elasticsearch集群中，集群中的所有节点会自动同步

[root@ela-client ~]# curl -u es_admin:P@ssw0rd -XPOST 'http://localhost:9200/_shield/user/fooadmin' -d '

{

  "password" : "foo.123", 

  "roles" : [ "admin", "other_role1" ], 

  "full_name" : "Jlive Liu", 

  "email" : "iliujun_live@163.com", 

  "metadata" : { 

    "intelligence" : 7

  }

}

'

{"user":{"created":true}}

[root@ela-data1 ~]# curl -u es_admin:P@ssw0rd 'http://localhost:9200/_shield/user'

{"fooadmin":{"username":"fooadmin","roles":["admin","other_role1"],"full_name":"Jlive Liu","email":"iliujun_live@163.com","metadata":{"intelligence":7}}}

4.测试用户认证

[root@ela-master2 ~]# curl -u fooadmin:foo.123 'http://localhost:9200/_cat/health?v'

epoch      timestamp cluster               status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent 

1462707192 19:33:12  elasticsearch_cluster green           6         2     54  27    0    0        0             0                  -                100.0% 

Kibana with Shield

https://www.elastic.co/guide/en/shield/current/kibana.html#kibana4-server-role

一.安装hield插件

说明:shield版本必须和elasticsearch保持一致

在线安装

/opt/kibana-4.5.0-linux-x64/bin/kibana plugin --install kibana/shield/2.3.2

或离线安装

wget http://download.elastic.co/kibana/shield/shield-2.3.2.tar.gz

/opt/kibana-4.5.0-linux-x64/bin/kibana plugin --install shield --url file:///mnt/hgfs/linux_soft/ELK/shield-2.3.2.tar.gz

root@jlive:~#/opt/kibana-4.5.0-linux-x64/bin/kibana plugin --install shield --url file:///mnt/hgfs/linux_soft/ELK/shield-2.3.2.tar.gz

Installing shield

Attempting to transfer from file:///mnt/hgfs/linux_soft/ELK/shield-2.3.2.tar.gz

Transferring 7933036 bytes....................

Transfer complete

Extracting plugin archive

Extraction complete

Optimizing and caching browser bundles...

Plugin installation complete

二.启用认证

A.file realm认证

1.配置具有kibana4-server角色的file based用户

注意:需要在elasticsearch节点上操作，上面已经创建了一个kibana用户，这里就省略创建步骤

2.修改kibana配置文件指定认证用户

cat >>/opt/kibana-4.5.0-linux-x64/config/kibana.yml <<HERE

kibana_elasticsearch_username: kibana

kibana_elasticsearch_password: P@ssw0rd

shield.encryptionKey: "something_secret"

shield.sessionTimeout: 600000

HERE

3.重启kibana并访问

/opt/elasticsearch-2.3.2/config/shield/roles.yml

# The required permissions for the kibana 4 server

kibana4_server:

  cluster:

      - monitor

  indices:

    - names: '.kibana'

      privileges:

        - all

注意:默认的kibana4-server角色用户指对.kibana索引有全权限，但对其它indices没有任何权限，如果不指定在登录kibana后可能会出现如下情况

4.授权用户访问指定indices(所有elasticsearch节点)

可以修改原/opt/elasticsearch-2.3.2/config/shield/roles.yml默认定义的kibana4-server角色，也可以重新定义一个新角色，这里定义一个新角色kibana4_indices，能访问Kibana 官方示例中对应的indices

i.定义角色并授权

cat >>/opt/elasticsearch-2.3.2/config/shield/roles.yml <<HERE

kibana4_indices:

  indices:

    - names: 'shakes*'

      privileges:

        - view_index_metadata

        - read

    - names: 'ba*'

      privileges:

        - view_index_metadata

        - read

    - names: 'logstash*'

      privileges:

        - view_index_metadata

        - read

HERE

ii.用户授权

[root@ela-master2 ~]# /opt/elasticsearch-2.3.2/bin/shield/esusers roles kibana -a kibana4_indices

[root@ela-master2 ~]# /opt/elasticsearch-2.3.2/bin/shield/esusers list

es_admin       : admin

kibana         : kibana4_indices,kibana4_server

注意:所有的elasticsearch节点都要同步

5.重启elasticsearch集群

6.启动(或重启)kibana

B.native realm认证

上文中的native用户(fooadmin/foo.123)因为是admin用户，所以可以直接认证成功。

当然还可以单独建个kibana_native用户来认证

curl -u fooadmin:foo.123 -XPOST 'http://localhost:9200/_shield/user/kibana_native' -d '

{

  "password" : "kibana.123",

  "roles" : [ "kibana4_server","kibana4_indices" ]

}

'

还是native认证方便，用户无需所有节点手动同步