简介
Elastalert是用python2写的一个报警框架(目前支持python2.6和2.7，不支持3.x)，github地址为 https://github.com/Yelp/elastalert。他提供不同场景的规则配置，若觉得规则、告警不满足需求时，可以用python编写插件Adding a New Rule Type、Adding a New Alerter。
Elastalert内置的告警方式：
- JIRA
- OpsGenie
- Commands
- HipChat
- MS Teams
- Slack
- Telegram
- AWS SNS
- VictorOps
- PagerDuty
- Exotel
- Twilio
- Gitter
前置条件
- Elasticsearch
- Python 2.7
- pip
-
pip 是 Python 包管理工具，该工具提供了对Python 包的查找、下载、安装、卸载的功能。 目前如果你在 python.org 下载最新版本的安装包，则是已经自带了该工具。 Python 2.7.9 + 或 Python 3.4+ 以上版本都自带 pip 工具。 pip 官网：https://pypi.org/project/pip/ 你可以通过以下命令来判断是否已安装： pip --version 如果你还未安装，则可以使用以下方法来安装： $ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py # 下载安装脚本 $ sudo python get-pip.py # 运行安装脚本 注意：用哪个版本的 Python 运行安装脚本，pip 就被关联到哪个版本，如果是 Python3 则执行以下命令： $ sudo python3 get-pip.py # 运行安装脚本。 一般情况 pip 对应的是 Python 2.7，pip3 对应的是 Python 3.x。 部分 Linux 发行版可直接用包管理器安装 pip，如 Debian 和 Ubuntu： sudo apt-get install python-pip - sudo easy_install -U setuptools
- sudo yum install python-devel
- python-pip python-dev libffi-dev libssl-dev
安装
直接使用pip安装：
$ pip install elastalert 直接装的官网发布的，不好用，还是下载源码安装
或者克隆源码安装：
$ git clone https://github.com/Yelp/elastalert.git
安装模块：
$ pip install "setuptools>=11.3" or sudo ....
$ python setup.py install
根据不同的Elasticsearch版本安装elasticsearch-py。
Elasticsearch 5.0+:
$ pip install "elasticsearch>=5.0.0"
Elasticsearch 2.X:
$ pip install "elasticsearch<3.0.0"
补充：
若出现如下问题：
error: six 1.9.0 is installed but six>=1.10.0 is required by set(['jira'])
解决办法：
$ pip install --upgrade six
更新的位置不是 python lib找的位置：/usr/lib/python2.7/site-packages/pkg_resources /root/.local/lib/python2.7/site-packages 1， sudo pip install --upgrade six DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. Requirement already up-to-date: six in /root/.local/lib/python2.7/site-packages (1.12.0) 2， elastalert-create-index Traceback (most recent call last): File "/usr/bin/elastalert-create-index", line 6, in <module> from pkg_resources import load_entry_point File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3241, in <module> @_call_aside File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3225, in _call_aside f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3254, in _initialize_master_working_set working_set = WorkingSet._build_master() File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 585, in _build_master return cls._build_from_requirements(__requires__) File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 598, in _build_from_requirements dists = ws.resolve(reqs, Environment()) File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 791, in resolve raise VersionConflict(dist, req).with_context(dependent_req)
解决方法：
pip uninstall sixpip install --upgrade six
卸载后重新安装就行。
然后继续安装即可。
如果在windows 64平台报错：执行
pip install python-magic-bin==0.4.14
修复https://stackoverflow.com/questions/18374103/exception-valuefailed-to-find-libmagic-check-your-installation-in-windows-7
E:elastalertexample_rules> python -m elastalert.elastalert --verbose --rule config.yaml
Traceback (most recent call last):
File "C:Python27lib
unpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "C:Python27lib
unpy.py", line 72, in _run_code
exec code in run_globals
File "C:Python27libsite-packageselastalertelastalert.py", line 21, in <module>
from alerts import DebugAlerter
File "C:Python27libsite-packageselastalertalerts.py", line 31, in <module>
from thehive4py.api import TheHiveApi
File "C:Python27libsite-packages hehive4pyapi.py", line 7, in <module>
import magic
File "C:Python27libsite-packagesmagic.py", line 181, in <module>
raise ImportError('failed to find libmagic. Check your installation')
ImportError: failed to find libmagic. Check your installation
重命名配置文件：
$ cp config.yaml.example config.yaml
打开配置文件config.yaml：
$ vim config.yaml
设置es_host、es_host指向你的Elasticsearch，然后保存退出。
配置文件解析
此处的配置文件是指config.yaml文件。
rules_folder：ElastAlert将加载规则配置文件的地方，它将尝试加载文件夹中的每个.yaml文件。
run_every：ElastAlert查询Elasticsearch的频率。
buffer_time：是查询窗口的大小，从每个查询运行的时间向后延伸。对于其中use_count_query或use_terms_query设置为true的规则，此值将被忽略。
es_host：是Elasticsearch集群的地址，ElastAlert将存储有关其状态、查询运行、警报和错误的数据。每个规则也可以设置不同的elasticsearch主机进行查询。
es_port：Elasticsearch对应的端口。
use_ssl： （可选的）是否使用TLS;连接到es_host；设置为True或False。
verify_certs： （可选的）是否验证TLS证书; 设置为True或False，默认是True。
client_cert： （可选的）PEM证书的路径。
client_key： （可选的） 作为客户端密钥使用的私钥文件的路径。
ca_certs： （可选的） 用于验证SSL连接的CA证书的路径。
es_username： （可选的） 用于连接Elasticsearch的basic-auth用户名。
es_password： （可选的） 用于连接Elasticsearch的密码。
es_url_prefix： （可选的） Elasticsearch端点的URL前缀。
es_send_get_body_as： （可选的） 查询Elasticsearch方法- GET，POST或source，默认是GET。
writeback_index：是ElastAlert将存储数据的索引名称。
alert_time_limit： 是失败警报的重试窗口。
创建Elasticsearch索引
elastalert-create-index这个命令会在elasticsearch创建索引，便于ElastAlert将有关其查询及其警报的信息和元数据保存回Elasticsearch。这不是必须的步骤，但是强烈建议创建。因为对于审计，测试很有用，并且重启elastalert不影响计数和发送alert。默认情况下，创建的索引叫 elastalert_status。
$ elastalert-create-index
New index name (Default elastalert_status)
Name of existing index to copy (Default None)
New index elastalert_status created
Done!
创建成功之后，我们可以在Elasticsearch中看到所创建的索引elastalert_status：
$ curl 'localhost:9200/_cat/indices?v'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open .kibana _4Op-xRCSzyC5nI2wZSPvw 1 1 2 0 8.4kb 8.4kb
yellow open elastalert_status ZTetmp8mTS6F28aIBJmJOg 5 1 0 0 810b 810b
elastalert_status
ElastAlert 根据elastalert_status去确定首次启动的时候在什么时间范围内去查询，以避免重复查询。对于每个规则，它将从最近的结束时间开始查询。包括：
- @timestamp：文件上传到Elasticsearch的时间。这是在运行查询并且已经处理结果之后。
- rule_name：相应规则的名称。
- starttime：查询的开始时间戳。
- endtime：查询结束时间戳。
- hits：查询结果的数量。
- matches：处理命中后规则返回的匹配数。请注意，这并不一定意味着警报被触发。
- time_taken：此查询运行所需的秒数。
测试Demo
本次elastalert安装目录为：/opt/elastalert。
配置SMTP
在/opt/elastalert目录下创建文件smtp_auth_file.yaml，内容如下：
#邮箱用户名
user: example@email.com
#不是邮箱密码，是设置的SMTP密码
password: XXXXX
配置规则
在example_rules目录下修改配置文件，如下：
$ cd /opt/elastalert/example_rules
$ vim example_frequency.yaml
修改后的配置文件：
# Alert when the rate of events exceeds a threshold
# (Optional)
# Elasticsearch host
# es_host: elasticsearch.example.com
# (Optional)
# Elasticsearch port
# es_port: 9200
# (OptionaL) Connect with SSL to Elasticsearch
#use_ssl: True
# (Optional) basic-auth username and password for Elasticsearch
#es_username: someusername
#es_password: somepassword
# (Required)
# Rule name, must be unique
name: Example rule
# (Required)
# Type of alert.
# the frequency rule type alerts when num_events events occur with timeframe time
type: frequency
# (Required)
# Index to search, wildcard supported
index: logstash-2017.08.28
# (Required, frequency specific)
# Alert when this many documents matching the query occur within a timeframe
num_events: 1
# (Required, frequency specific)
# num_events must occur within this amount of time to trigger an alert
timeframe:
minutes: 1
# (Required)
# A list of Elasticsearch filters used for find events
# These filters are joined with AND and nested in a filtered query
# For more info: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html
#匹配日志中field字段值为value的日志数据，若不想进行过滤可以写成“filter: []”（这种方式可以用在日志数量的告警中）
filter:
- query:
query_string:
query: "field: value"
#SMTP configration
smtp_host: smtp.163.com
smtp_port: 25
#SMTP auth
smtp_auth_file: /opt/elastalert/smtp_auth_file.yaml
email_reply_to: example@email.com
from_addr: example@email.com
# (Required)
# The alert is use when a match is found
alert:
- "email"
# (required, email specific)
# a list of email addresses to send alerts to
email:
- "test@email.com"
保存修改并退出。
补充：
上述规则表示：在elastalert执行的一分钟内，出现一条有field值为value日志，则触发告警，并且告警通知将以email的形式从example@email.com邮箱发送给test@email.com。
规则配置解析：
-
es_host、es_port：应该指向我们要查询的Elasticsearch集群。
-
name：是这个规则的唯一名称。如果两个规则共享相同的名称，ElastAlert将不会启动。
-
type：每个规则都有不同的类型，可能会采用不同的参数。该frequency类型表示“在timeframe时间内匹配成功次数超过num_events发出警报”。有关其他类型的信息，请参阅规则类型。
-
index：要查询的索引的名称。
-
num_events：此参数特定于frequency类型，是触发警报时的阈值。
-
timeframe：timeframe是num_events必须发生的时间段。
-
filter：是用于过滤结果的Elasticsearch过滤器列表。有关
详细信息，请参阅编写过滤规则。 -
alert：警报。有关警报类型的更多信息，请参阅警报。电子邮件警报需要SMTP服务器才能发送邮件。默认情况下，它将尝试使用localhost。这可以通过smtp_host选项更改。
-
email：是要发送警报的地址列表。
还有许多其他可选配置选项，请参阅常见配置选项。
myrule.yaml和config.yaml
# From example_rules/example_frequency.yaml
es_host: xxx.xxx
es_port: 9200
name: Example rule
type: frequency
index: myindex
timestamp_field: mytime
num_events: 1
timeframe:
minutes: 50
filter:
- term:
"carColor" : "blue"
alert:
- "email"
email:
- "elastalert@example.com"
# This is the folder that contains the rule yaml files # Any .yaml file will be loaded as a rule rules_folder: example_rules # How often ElastAlert will query Elasticsearch # The unit can be anything from weeks to seconds run_every: minutes: 1 # ElastAlert will buffer results from the most recent # period of time, in case some log sources are not in real time buffer_time: minutes: 15 # The Elasticsearch hostname for metadata writeback # Note that every rule can have its own Elasticsearch host es_host: 211.100.75.204 # The Elasticsearch port es_port: 9200 # The AWS region to use. Set this when using AWS-managed elasticsearch #aws_region: us-east-1 # The AWS profile to use. Use this if you are using an aws-cli profile. # See http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html # for details #profile: test # Optional URL prefix for Elasticsearch #es_url_prefix: elasticsearch # Connect with TLS to Elasticsearch #use_ssl: True # Verify TLS certificates #verify_certs: True # GET request with body is the default option for Elasticsearch. # If it fails for some reason, you can pass 'GET', 'POST' or 'source'. # See http://elasticsearch-py.readthedocs.io/en/master/connection.html?highlight=send_get_body_as#transport # for details #es_send_get_body_as: GET # Option basic-auth username and password for Elasticsearch #es_username: someusername #es_password: somepassword # Use SSL authentication with client certificates client_cert must be # a pem file containing both cert and key for client #verify_certs: True #ca_certs: /path/to/cacert.pem #client_cert: /path/to/client_cert.pem #client_key: /path/to/client_key.key # The index on es_host which is used for metadata storage # This can be a unmapped index, but it is recommended that you run # elastalert-create-index to set a mapping writeback_index: elastalert_status # If an alert fails for some reason, ElastAlert will retry # sending the alert until this time period has elapsed alert_time_limit: days: 2
更全的示例：
# Alert when the rate of events exceeds a threshold # (Optional) # Elasticsearch host es_host: 110.10.21.77 # (Optional) # Elasticsearch port es_port: 9200 # (OptionaL) Connect with SSL to Elasticsearch #use_ssl: True # (Optional) basic-auth username and password for Elasticsearch #es_username: someusername #es_password: somepassword # (Required) # Rule name, must be unique name: system error # (Required) # Type of alert. # the frequency rule type alerts when num_events events occur with timeframe time type: frequency # (Required) # Index to search, wildcard supported index: task* attach_related: true # (Required, frequency specific) # Alert when this many documents matching the query occur within a timeframe num_events: 1 # (Required, frequency specific) # num_events must occur within this amount of time to trigger an alert timeframe: hours: 1 timestamp_field: "@timestamp" # (Required) # A list of Elasticsearch filters used for find events # These filters are joined with AND and nested in a filtered query # For more info: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html filter: - terms: "Level": ["fatal", "error"] #use_kibana_dashboard: afAlertDashboard use_kibana4_dashboard: https://nodejsgbl.italkbb.com/kibana/app/kibana#/dashboard/3bde48d0-9880-11e9-b5d5-2df46b09dea6 # (Required) # The alert is use when a match is found alert: - "email" # (required, email specific) # a list of email addresses to send alerts to email: - "xxxx@net263.com"
- "xxxx@163.com" smtp_host: smtp.263.net smtp_port: 25 smtp_auth_file: ../smtp_auth_file.yaml email_reply_to: No.reply@net263.com from_addr: ElastAlert@net263.com #cc: aaa@net263.com email_format: html #alert_text_type: exclude_fields #alert_text_only alert_subject: "Alert: System {0} occurred {1} times." alert_subject_args: - Level #来自index里面json内容 - "num_hits" alert_text: "<pre> You can access all of error log on Kibana: <a href={0}> Aijia server alert dashboard</a><br>" alert_text_args: - kibana_link
{"indexId":"1000001","mytime": "2019-01-14T17:40:17+08:00","carColor":"blue","@timestamp":"2018-12-18T07:48:17","cost":1001,"indexName":"default_log_index" }
{ "indexName" : "af-test", "code" : "1", "msg" : "testMsg", "position" : "AHC.User.ReadService", "eventType" : "alarm","eventTimestamp" : "2019-01-14T17:40:17+08:00", "@timestamp" : "2019-01-14T17:44:24.6172215+08:00"}
key要区分大小写，value 写过滤条件时候，全部小写，否则查询不到。如 term: "msg":"testmsg" 才行
测试规则
命令如下：
$ elastalert-test-rule example_rules/example_frequency.yaml
能够检查规则是否正确，有关详细信息测试部分。
运行
运行elastalert：
$ cd /opt/elastalert $ python -m elastalert.elastalert --verbose --rule example_frequency.yamles_debug、es_debug_trace trace后面接文件名，里面会输出查询语句。
或者运行指定目录下所有yaml文件：
elastalert --verbose --config ./config.yaml --start 2019-06-27T06:00:03
其中config.yaml里面存放着rules的目录。仔细看好目录名称不要写错。
调试elasticsearch 的 查询 语句是否有问题，可以加上参数
trace文件打印出查询语句：
curl -XGET 'http://localhost:9200/?pretty' -d '' "query": { "bool": { "filter": { "bool": { "must": [ "gt": "2019-07-03T09:30:38.664486Z", "lte": "2019-07-03T09:32:03.508890Z" } } }, { "query_string": { "query": "NOT AggregatedHealthState.keyword: Ok" } } ] } } } }, "sort": [ { "@timestamp": { "order": "asc" } } ] }
给Elasticsearch添加测试数据：
$ curl -X POST "http://127.0.0.1:9200/logstash-2017.08.28/test" -d '{
"@timestamp": "2017-08-28T10:54:41.000Z",
"field": "value"
}'
注意：上述的@timestamp是UTC时间，也就是说这条数据是在2017-08-28 18:54:41产生的。
Alert
可在邮箱中看到如下图所示的邮件：
收到邮件中会包含当前符合告警规则的日志数据。
参考资料
- ElastAlert - Easy & Flexible Alerting With Elasticsearch
- Rule Types and Configuration Options
- Writing Filters For Rules
-
https://www.cnblogs.com/zhaishaomin/p/7417306.html
https://blog.csdn.net/pujiaolin/article/details/52252950?locationNum=3
https://blog.csdn.net/gamer_gyt/article/details/52917116
https://blog.csdn.net/davinciyxw/article/details/79104498
https://blog.csdn.net/vbaspdelphi/article/details/54311764
http://www.freebuf.com/sectool/164591.html
alert还有kibana插件,以及elasticalert还可以容器方式启动
参考:
https://github.com/bitsensor/elastalert-kibana-plugin
插件下载慢:
https://git.bitsensor.io/front-end/elastalert-kibana-plugin/-/jobs/10874/artifacts/raw/artifact/elastalert-5.6.4-latest.zip
然后
./bin/kibana-plugin install file:///usr/local/src/elastalert-5.6.4-latest.zip安装后效果:
微信报警：https://github.com/anjia0532/elastalert-wechat-plugin
钉钉报警：https://github.com/xuyaoqiang/elastalert-dingtalk-plugin
告警规则
告警模板
告警时间
告警方式
-----
参考:https://xizhibei.github.io/2017/11/19/alerting-with-elastalert/
https://github.com/chenryn/ELKstack-guide-cn/blob/master/elasticsearch/other/elastalert.md
安装elastalert（按照官网的操作走）
先切换到自己的工作目录
我这里是 /usr/local/dev/
git 克隆源码
git clone https://github.com/Yelp/elastalert.git
cd elastalert
1
2
3
安装
sudo python setup.py install
1
出错了
Running blist-1.3.6/setup.py -q bdist_egg –dist-dir /tmp/easy_install-Gc6gbe/blist-1.3.6/egg-dist-tmp-Ik7LL2
The required version of setuptools (>=1.1.6) is not available,
and can’t be installed while this script is running. Please
install a more recent version first, using
‘easy_install -U setuptools’.
(Currently using setuptools 0.9.8 (/usr/lib/python2.7/site-packages))
ok按照它提示的，，，执行：
sudo easy_install -U setuptools
1
再执行安装（sudo python setup.py install），又出错了
warning: no files found matching 'blist.rst'
blist/_blist.c:38:20: 致命错误：Python.h：没有那个文件或目录
#include<Python.h>
1
2
3
这其实是缺少开发包，执行命令：
sudo yum install python-devel
1
再执行安装（sudo python setup.py install）
到这里基本能安装成功。
有时候会报一些包找不到的错误，可能是网络问题，重复执行安装命令即可（我在阿里云上遇到了）。
继续执行安装命令
pip install -r requirements.txt
1
找不到pip ，先安装pip并更新
sudo yum -y install python-pip
sudo pip install --upgrade pip
1
2
安装完继续
sudo pip install -r requirements.txt
1
在elasticsearch中创建elastalert的日志索引
sudo elastalert-create-index
1
根据自己的情况，填入elasticsearch的相关信息，关于
elastalert_status部分直接回车默认的即可。
如下所示：
Enter elasticsearch host: log.example.com
Enter elasticsearch port: 9200
Use SSL? t/f: t
Enter optional basic-auth username (or leave blank): es_admin
Enter optional basic-auth password (or leave blank):
Enter optional Elasticsearch URL prefix (prepends a string to the URL of every request):
New index name? (Default elastalert_status)
Name of existing index to copy? (Default None)
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
New index elastalert_status created
Done!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
3.创建配置文件
sudo cp config.yaml.example config.yaml
sudo vi config.yaml
1
2
3
根据自己的具体情况进行修改
比如我这里的elasticsearch是ssl 并需要用户登陆的，配置大致如下：
run_every:
minutes: 1
buffer_time:
minutes: 15
es_host: log.example.com
es_port: 9200
use_ssl: True
es_send_get_body_as: GET
es_username: es_admin
es_password: es_password
writeback_index: elastalert_status
alert_time_limit:
days: 2
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
以同样的方式配置规则
cd example_rules/
sudo cp example_frequency.yaml my_rule.yaml
sudo vi my_rule.yaml
1
2
3
4
5
这里就是根据具体的elasticsearch的信息进行配置
es_host: log.example.com
es_port: 9200
use_ssl: True
es_username: es_admin
es_password: es_password
#name属性要求唯一，这里最好能标示自己的产品
name: My-Product Exception Alert
#类型，我选择任何匹配的条件都发送邮件警告
type: any
#需要监控的索引，支持通配
index: logstash-*
#下面两个随意配置
num_events: 50
timeframe:
hours: 4
#根据条件进行过滤查询（这里我只要出现异常的日志，并且排除业务异常（自定义异常））
filter:
- query:
query_string:
query: "message: *exception* AND message: (!*BusinessException*) AND message: (!*ServiceException*)"
#email的警告方式
alert:
- "email"
#增加邮件内容，这里我附加一个日志访问路径
alert_text: "Ref Log https://log.example.com:5601/app/kibana"
#SMTP协议的邮件服务器相关配置（我这里是腾讯企业邮箱）
smtp_host: smtp.exmail.qq.com
smtp_port: 25
#用户认证文件，需要user和password两个属性
smtp_auth_file: smtp_auth_file.yaml
email_reply_to: no-reply@example.com
from_addr: no-reply@example.com
#需要接受邮件的邮箱地址列表
email:
- "user1@example.com"
- "user1@example.com"
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
接下来创建smtp_auth_file.yaml
sudo touch smtp_auth_file.yaml
sudo vi smtp_auth_file.yaml
##配置文件内容
user: "no-reply@example.com"
password: "password"
1
2
3
4
5
6
现在可以简单测试一下配置文件是否正确
sudo elastalert-test-rule ./my_rule.yaml
1
如果有问题，就检查配置文件。
然后启动运行一下试试：
sudo python -m elastalert.elastalert --verbose --rule my_rule.yaml
1
然后故意产生一下异常试试，比如
if(true){
throw new NullPointerException("测试日志异常监控服务");
}
1
2
3
执行一下，一般没啥问题，邮件应该就能收到了，有问题就检查配置文件。
4.我是要监控服务，所以要让elastalert以服务的形式运行
首先创建elastalert的配置目录和相关文件
sudo mkdir /etc/elastalert
cd /etc/elastalert
-- 复制配置文件
sudo cp /usr/local/dev/elastalert/config.yaml config.yaml
sudo mkdir rules
cd rules
-- 复制规则文件
sudo cp /usr/local/dev/elastalert/example_rules/my_rule.yaml my_rule.yaml
-- 复制邮件用户认证文件
sudo cp /usr/local/dev/elastalert/example_rules/smtp_auth_file.yaml smtp_auth_file.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
接下来修改配置文件
修改 config.yaml 中
rules_folder: /etc/elastalert/rules
1
修改 my_rule.yaml中
smtp_auth_file: /etc/elastalert/rules/umu_smtp_auth_file.yaml
1
接下来就是创建systemd服务了
cd /etc/systemd/system
sudo touch elastalert.service
sudo vi elastalert.service
1
2
3
4
elastalert.service 内容
[Unit]
Description=elastalert
After=elasticsearch.service
[Service]
Type=simple
User=root
Group=root
Restart=on-failure
WorkingDirectory=/usr/local/dev/elastalert
ExecStart=/usr/bin/elastalert --config /etc/elastalert/config.yaml --rule /etc/elastalert/rules/my_rule.yaml
[Install]
WantedBy=multi-user.target
1
2
3
4
5
6
7
8
9
10
11
12
13
14
保存退出。
启动服务
sudo systemctl start elastalert
1
查看状态
sudo systemctl status elastalert
1
不出意外的话，你将看到绿色的Active: active (running)
至此，日志的异常监控服务就搭建好了，好的，收工！
---------------------
作者：梦得溪
来源：CSDN
原文：https://blog.csdn.net/pujiaolin/article/details/52252950
版权声明：本文为博主原创文章，转载请附上博文链接！
安装elastalert（按照官网的操作走）
先切换到自己的工作目录 我这里是 /usr/local/dev/ git 克隆源码
git clone https://github.com/Yelp/elastalert.git
cd elastalert123安装
sudo python setup.py install1出错了 Running blist-1.3.6/setup.py -q bdist_egg –dist-dir /tmp/easy_install-Gc6gbe/blist-1.3.6/egg-dist-tmp-Ik7LL2 The required version of setuptools (>=1.1.6) is not available, and can’t be installed while this script is running. Please install a more recent version first, using ‘easy_install -U setuptools’.
(Currently using setuptools 0.9.8 (/usr/lib/python2.7/site-packages))
ok按照它提示的，，，执行：
sudo easy_install -U setuptools1再执行安装（sudo python setup.py install），又出错了
warning: no files found matching 'blist.rst'blist/_blist.c:38:20: 致命错误：Python.h：没有那个文件或目录 #include<Python.h>123这其实是缺少开发包，执行命令：
sudo yum install python-devel1再执行安装（sudo python setup.py install） 到这里基本能安装成功。 有时候会报一些包找不到的错误，可能是网络问题，重复执行安装命令即可（我在阿里云上遇到了）。
继续执行安装命令
pip install -r requirements.txt1找不到pip ，先安装pip并更新
sudo yum -y install python-pipsudo pip install --upgrade pip12安装完继续
sudo pip install -r requirements.txt1在elasticsearch中创建elastalert的日志索引
sudo elastalert-create-index1根据自己的情况，填入elasticsearch的相关信息，关于 elastalert_status部分直接回车默认的即可。 如下所示：
Enter elasticsearch host: log.example.comEnter elasticsearch port: 9200Use SSL? t/f: tEnter optional basic-auth username (or leave blank): es_adminEnter optional basic-auth password (or leave blank):Enter optional Elasticsearch URL prefix (prepends a string to the URL of every request):New index name? (Default elastalert_status)Name of existing index to copy? (Default None)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)New index elastalert_status createdDone!1234567891011121314151617181920212223243.创建配置文件
sudo cp config.yaml.example config.yaml
sudo vi config.yaml123根据自己的具体情况进行修改
比如我这里的elasticsearch是ssl 并需要用户登陆的，配置大致如下：
run_every: minutes: 1
buffer_time: minutes: 15
es_host: log.example.com
es_port: 9200
use_ssl: True
es_send_get_body_as: GET
es_username: es_admin
es_password: es_password
writeback_index: elastalert_status
alert_time_limit: days: 212345678910111213141516171819202122以同样的方式配置规则
cd example_rules/
sudo cp example_frequency.yaml my_rule.yaml
sudo vi my_rule.yaml12345这里就是根据具体的elasticsearch的信息进行配置
es_host: log.example.comes_port: 9200use_ssl: Truees_username: es_admines_password: es_password#name属性要求唯一，这里最好能标示自己的产品name: My-Product Exception Alert#类型，我选择任何匹配的条件都发送邮件警告type: any#需要监控的索引，支持通配index: logstash-*#下面两个随意配置num_events: 50timeframe: hours: 4#根据条件进行过滤查询（这里我只要出现异常的日志，并且排除业务异常（自定义异常））filter:- query: query_string: query: "message: *exception* AND message: (!*BusinessException*) AND message: (!*ServiceException*)"#email的警告方式alert:- "email"
#增加邮件内容，这里我附加一个日志访问路径alert_text: "Ref Log https://log.example.com:5601/app/kibana"#SMTP协议的邮件服务器相关配置（我这里是腾讯企业邮箱）smtp_host: smtp.exmail.qq.comsmtp_port: 25#用户认证文件，需要user和password两个属性smtp_auth_file: smtp_auth_file.yamlemail_reply_to: no-reply@example.comfrom_addr: no-reply@example.com
#需要接受邮件的邮箱地址列表email:- "user1@example.com"- "user1@example.com"1234567891011121314151617181920212223242526272829303132333435363738接下来创建smtp_auth_file.yaml
sudo touch smtp_auth_file.yamlsudo vi smtp_auth_file.yaml
##配置文件内容user: "no-reply@example.com"password: "password"123456现在可以简单测试一下配置文件是否正确
sudo elastalert-test-rule ./my_rule.yaml1如果有问题，就检查配置文件。 然后启动运行一下试试：
sudo python -m elastalert.elastalert --verbose --rule my_rule.yaml1然后故意产生一下异常试试，比如
if(true){ throw new NullPointerException("测试日志异常监控服务");}123执行一下，一般没啥问题，邮件应该就能收到了，有问题就检查配置文件。
4.我是要监控服务，所以要让elastalert以服务的形式运行 首先创建elastalert的配置目录和相关文件
sudo mkdir /etc/elastalertcd /etc/elastalert-- 复制配置文件
sudo cp /usr/local/dev/elastalert/config.yaml config.yamlsudo mkdir rulescd rules
-- 复制规则文件sudo cp /usr/local/dev/elastalert/example_rules/my_rule.yaml my_rule.yaml
-- 复制邮件用户认证文件sudo cp /usr/local/dev/elastalert/example_rules/smtp_auth_file.yaml smtp_auth_file.yaml12345678910111213接下来修改配置文件 修改 config.yaml 中
rules_folder: /etc/elastalert/rules1修改 my_rule.yaml中
smtp_auth_file: /etc/elastalert/rules/umu_smtp_auth_file.yaml1接下来就是创建systemd服务了
cd /etc/systemd/systemsudo touch elastalert.service
sudo vi elastalert.service1234elastalert.service 内容
[Unit]Description=elastalertAfter=elasticsearch.service
[Service]Type=simpleUser=rootGroup=rootRestart=on-failureWorkingDirectory=/usr/local/dev/elastalertExecStart=/usr/bin/elastalert --config /etc/elastalert/config.yaml --rule /etc/elastalert/rules/my_rule.yaml
[Install]WantedBy=multi-user.target1234567891011121314保存退出。 启动服务
sudo systemctl start elastalert1查看状态
sudo systemctl status elastalert1不出意外的话，你将看到绿色的Active: active (running)
至此，日志的异常监控服务就搭建好了，好的，收工！--------------------- 作者：梦得溪 来源：CSDN 原文：https://blog.csdn.net/pujiaolin/article/details/52252950 版权声明：本文为博主原创文章，转载请附上博文链接！