zoukankan      html  css  js  c++  java
  • Maxwell入门

    1 下载tar包

    2 解压

      tar -zvxf maxwell-1.19.4.tar.gz

    3 修改配置

      cd maxwell-1.19.4

      cp config.properties.example config.properties

    根据自己mysql以及kafka配置  

    # tl;dr config
    log_level=info

    producer=kafka
    kafka.bootstrap.servers=hdp1:9092,hdp2:9092,hdp3:9092,hdp4:9092

    # mysql login info
    host=127.0.0.1
    user=maxwell
    password=pwd123456

    4 修改mysql的binlog配置  根据自己机器配置

    vi /etc/my.cnf

    $ vi my.cnf

    [mysqld]

    server_id=1

    log-bin=master

    binlog_format=row

     

    5 创建同步用户

    mysql> GRANT ALL on maxwell.* to 'maxwell'@'%' identified by 'XXXXXX';
    mysql> GRANT SELECT, REPLICATION CLIENT, REPLICATION SLAVE on *.* to 'maxwell'@'%';
    
    mysql>flush privileges;

    6 重启mysql查看bin log是否修改成功

    service mysqld restart

    show variables like '%binlog%'

    7 测试一下maxwell 

    [root@hdp1 /mnt/software/maxwell-1.19.4]#bin/maxwell --user='maxwell' --password='pwd123456' --host='127.0.0.1' --producer=stdout 

    #插入数据 根据自己mysql情况测试

    mysql> INSERT INTO `cmcc_offset` (`topics`, `partId`, `offsets`, `groupId`, `create_time`, `updater`, `update_time`, `is_delete`)

     成功收到数据

     

    8 单独的kafka producer配置

    单独copy指定版本kafka的client的jar包到lib目录下, 我自己的kafka版本是1.1.0的

    启动kafka的producer命令,需要单独制定kafka的version

    bin/maxwell --user='maxwell' --password='pwd123456' --host='127.0.0.1'  --producer=kafka --kafka_version=1.1.0 --kafka.bootstrap.servers=hdp1:9092,hdp2:9092,hdp3:9092,hdp4:9092 --kafka_topic=maxwell

    遇到问题  Kafka的K大写了,改成小写就OK了

    java.lang.RuntimeException: Unknown producer type: Kafka
    at com.zendesk.maxwell.MaxwellContext.getProducer(MaxwellContext.java:372) ~[maxwell-1.19.4.jar:1.19.4]
    at com.zendesk.maxwell.Maxwell.startInner(Maxwell.java:186) ~[maxwell-1.19.4.jar:1.19.4]
    at com.zendesk.maxwell.Maxwell.start(Maxwell.java:156) ~[maxwell-1.19.4.jar:1.19.4]
    at com.zendesk.maxwell.Maxwell.main(Maxwell.java:246) ~[maxwell-1.19.4.jar:1.19.4]
    01:08:02,127 INFO TaskManager - Stopped all tasks
    java.lang.RuntimeException: Unknown producer type: Kafka
    at com.zendesk.maxwell.MaxwellContext.getProducer(MaxwellContext.java:372)
    at com.zendesk.maxwell.Maxwell.startInner(Maxwell.java:186)
    at com.zendesk.maxwell.Maxwell.start(Maxwell.java:156)
    at com.zendesk.maxwell.Maxwell.main(Maxwell.java:246)
    [root@hdp1 /mnt/software/maxwell-1.19.4]#xcall.sh jps

    其他细节详情请看官网http://maxwells-daemon.io/quickstart/

     

     

     

     
  • 相关阅读:
    how to pass a Javabean to server In Model2 architecture.
    What is the Web Appliation Archive, abbreviation is "WAR"
    Understaning Javascript OO
    Genetic Fraud
    poj 3211 Washing Clothes
    poj 2385 Apple Catching
    Magic Star
    关于memset的用法几点
    c++ 函数
    zoj 2972 Hurdles of 110m
  • 原文地址:https://www.cnblogs.com/QuestionsZhang/p/10271865.html
Copyright © 2011-2022 走看看