MAXWELL系列（一）-利用maxwell 解析binlog 到 redis

今天猪脚是maxwell，zendesk公司开源 https://github.com/zendesk/maxwell

先看架构，和他竞争的有 Debezium Connector for MySQL

废话不多说，搭建目标任务 mysql的binlog 到redis （192.168.0.1 ~~~~~~~192.168.0.3）

1：下载 https://github.com/zendesk/maxwell/releases/download/v1.22.0/maxwell-1.22.0.tar.gz

2： 安装java ，配置好java环境变量 ，解压maxwell-1.22.0.tar.gz（因为是java 写的）

3：mv  maxwell-1.22.0  /usr/local   && ln -s maxwell  maxwell-1.22.0

4:  配置原库（192.168.0.1）添加账号，(后面要用）.

mysql> select user,host from mysql.user where user='canal';
+-------+------+
| user  | host |
+-------+------+
| canal | %    |
+-------+------+
1 row in set (0.01 sec)

mysql> show grants for canal@'%';
+---------------------------------------------------------------------------+
| Grants for canal@% |
+---------------------------------------------------------------------------+
| GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'canal'@'%' |
+---------------------------------------------------------------------------+

5:配置伪目标mysql库：（maxwell的主机）为什么叫伪目标库？？？因为就是模拟的从库行为。

mysql> select user,host from mysql.user where user='maxwell';
+---------+-----------+
| user    | host      |
+---------+-----------+
| maxwell | localhost |
+---------+-----------+
1 row in set (0.00 sec)

mysql> show grants for maxwell@'localhost';
+--------------------------------------------------------------+
| Grants for maxwell@localhost                                 |
+--------------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO 'maxwell'@'localhost'         |
+--------------------------------------------------------------+
2 rows in set (0.00 sec)

 6：配置maxwell：就是他了！

config.properties

[root@redis-binlog-reportform maxwell]# pwd;ls  -la config.properties
/usr/local/maxwell
-rw-r--r-- 1 root root 10505 May  8 23:19 config.properties

配置redis入口

#           *** redis ***

redis_host=192.168.0.3
redis_port=xxxx
redis_auth=XXXX
redis_database=0
redis_pub_channel=maxwell
redis_list_key=maxwell
redis_type=lpush

配置源库入口

replication_host=192.168.0.2
replication_user=canal
replication_password=xxx
replication_port=3306

配置伪从库入口

# mysql login info
host=localhost
user=maxwell
password=xxx
port=3306

7 启动maxwell

nohup bin/maxwell --config=config.properties &

8 查看进程及日志

MaxwellConfig - disabling bootstrapping; not available when using a separate replication host.
18:23:29,734 WARN  MaxwellMetrics - Metrics will not be exposed: metricsReportingType not configured.
18:23:30,153 INFO  Maxwell - Maxwell v1.22.0 is booting (MaxwellRedisProducer), starting at Position[BinlogPosition[mysql_bin.003563:231462217], lastHeartbeat=0]
18:23:30,334 INFO  MysqlSavedSchema - Restoring schema id 1 (last modified at Position[BinlogPosition[mysql_bin.003555:753826693], lastHeartbeat=0])
18:23:30,537 INFO  BinlogConnectorReplicator - Setting initial binlog pos to: mysql_bin.003563:231462217

9：查看redis 接收情况

[root@redis-binlog-reportform maxwell]# redis-cli -p 9001
127.0.0.1:9001> auth xxxx
OK
127.0.0.1:9001> 
127.0.0.1:9001> 
127.0.0.1:9001> 
127.0.0.1:9001> type maxwell
list
127.0.0.1:9001> 
127.0.0.1:9001> llen maxwell
(integer) 53900946
127.0.0.1:9001> llen maxwell
(integer) 53901342
127.0.0.1:9001> llen maxwell
(integer) 53901028
127.0.0.1:9001> llen maxwell
(integer) 53900745
127.0.0.1:9001>