消息中间件metaq

消息中间件metaq安装并注册到zookper集群

项目地址

https://github.com/killme2008/Metamorphosis

Memorphosis是一个消息中间件，它是linkedin开源MQ——kafka的Java版本，针对淘宝内部应用做了定制和优化。Metamorphosis的设计原则

• 消息都是持久的，保存在磁盘
• 吞吐量第一
• 消费状态保存在客户端
•  分布式，生产者、服务器和消费者都可分布

Metamorphosis的部署结构

[root@q tools]# tar xf metaq-server-1.4.6.2.tar.gz 
[root@q tools]# cd taobao/
[root@q taobao]# ls
COPYING.txt  metamorphosis-server-wrapper  README.md

[root@q taobao]# cd metamorphosis-server-wrapper/
[root@q metamorphosis-server-wrapper]# ls
bin  conf  lib  provided

其中bin目录包含启动脚本（目前只有linux下的启动脚本），conf下为配置文件，lib下为meta服务器的所有jar包,logs为日志所在目录。

[root@q metamorphosis-server-wrapper]# cd conf/
[root@q conf]# ls
async_slave.properties   jettyBroker.properties  notifySlave.properties  samsa_master.properties  timetunnel.properties
gregor_slave.properties  notifyadaper            sample.server.ini       server.ini
[root@q conf]# vim server.ini 

[system]
brokerId=0
numPartitions=1
serverPort=8123
dashboardHttpPort=8120
unflushThreshold=0
unflushInterval=10000
maxSegmentSize=1073741824
maxTransferSize=1048576
deletePolicy=delete,168
deleteWhen=0 0 6,18 * * ?
flushTxLogAtCommit=1
stat=true

;; Update consumers offsets to current max offsets when consumers offsets are out of range of current broker's messages.
;; It must be false in production.But recommend to be true in development or test.
updateConsumerOffsets=true

[zookeeper]
zk.zkConnect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
zk.zkSessionTimeoutMs=30000
zk.zkConnectionTimeoutMs=30000
zk.zkSyncTimeMs=5000

;; Topics section
[topic=test]

[topic=meta-test]

[topic=xl]   #自定义的消息xl的队列

启动

[root@q conf]# cd ..
[root@q metamorphosis-server-wrapper]# ls
bin  conf  lib  logs  provided
[root@q metamorphosis-server-wrapper]# cd bin/
[root@q bin]# sh metaServer.sh start &

查看状态

[root@q bin]# sh metaServer.sh stats

测试

[root@q bin]# ./zkCli.sh -server 127.0.0.1:2181 [zk: 127.0.0.1:2181(CONNECTING) 0] ls /
2017-01-01 19:19:42,232 [myid:] - INFO  [main-SendThread(127.0.0.1:2181):ClientCnxn$SendThread@1235] - Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x159591602330000, negotiated timeout = 30000
WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[zookeeper, meta]  #在此目录生成meta
[zk: 127.0.0.1:2181(CONNECTED) 1] ls /meta
[brokers]
[zk: 127.0.0.1:2181(CONNECTED) 2] ls /meta/brokers
[topics-sub, ids, topics-pub, topics]
 [zk: 127.0.0.1:2181(CONNECTED) 3] ls /meta/brokers/topics
[meta-test, test, xl]
[zk: 127.0.0.1:2181(CONNECTED) 4] ls /meta/brokers/ids  #brok节点 
[0]
[zk: 127.0.0.1:2181(CONNECTED) 5] get  /meta/brokers/ids/0
meta://192.168.179.161:8123
cZxid = 0x200000046
ctime = Sun Jan 01 16:44:09 CST 2017
mZxid = 0x200000048
mtime = Sun Jan 01 16:44:09 CST 2017
pZxid = 0x200000049
cversion = 2
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 27
numChildren = 2
[zk: 127.0.0.1:2181(CONNECTED) 6]

部署另外一台

在同一台模拟部署

[root@q tools]# cp -rf taobao/ taobao1
[root@q bin]# pwd
/home/zsq/tools/taobao1/metamorphosis-server-wrapper/bin
[root@q bin]# ls
env.bat  env.sh  localMetaServer.bat  log4j.properties  metaServer.bat  metaServer.sh  tools_log4j.properties

修改脚本metaServer.sh的pid文件

PID_FILE="$PID_DIR/.run1.pid"

修改env.sh，默认是9123端口

export JMX_PORT=9124

启动

[root@q bin]# sh metaServer.sh start $

[zk: 127.0.0.1:2181(CONNECTED) 12] ls  /meta/brokers/ids  #有两个broker就表面集群部署成功
[0, 1]

[zk: 127.0.0.1:2181(CONNECTED) 18] get  /meta/brokers/ids/1
meta://192.168.179.161:8124
cZxid = 0x200000092
ctime = Sun Jan 01 21:05:26 CST 2017
mZxid = 0x200000094
mtime = Sun Jan 01 21:05:26 CST 2017
pZxid = 0x200000095
cversion = 2
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 27
numChildren = 2
[zk: 127.0.0.1:2181(CONNECTED) 19]

消息中间件metaq的配置启动及相关维护操作