zoukankan      html  css  js  c++  java
  • kafka 常用命令

    Kafka


    解压,进入kafka目录下的bin目录


    启动zk

    nohup ./zookeeper-server-start.sh ../config/zookeeper.properties &

    nohup ./bin/zookeeper-server-start.sh ./config/zookeeper.properties &


    启动kafka

    nohup ./kafka-server-start.sh ../config/server.properties &

    nohup ./bin/kafka-server-start.sh ./config/server.properties &


    创建topic

    ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

    ./kafka-topics.sh --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test2 --create


    列出topic

    ./kafka-topics.sh --list --zookeeper localhost:2181

    ./kafka-topic --zookeeper localhost:2181 --list


    启动生产者并发送消息

    ./kafka-console-producer.sh --broker-list localhost:9092 --topic zyl

    输入消息

    nihao

    china


    另外开个终端,启动消费者接受消息

    ./kafka-console-consumer.sh --zookeeper localhost:2181 --topic zyl --from-beginning

    可以接受到

    nihao

    china



    另外开个终端,可以看到有4个进程(分别为zk、kafka、控制台消费者、控制台生产者)

    [root@aa bin]# jps | grep -v Jps

    9802 QuorumPeerMain

    10392 Kafka

    10956 ConsoleConsumer

    10887 ConsoleProducer



    使用集群3个节点(多个节点也同理,加上之前的,我一共弄了4个节点)

    拷贝并修改配置文件

    cp server.properties server1.properties

    cp server.properties server2.properties

    cp server.properties server3.properties


    nohup ./kafka-server-start.sh ../config/server1.properties &

    nohup ./kafka-server-start.sh ../config/server2.properties &

    nohup ./kafka-server-start.sh ../config/server3.properties &


    [root@aa bin]# ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic my-replicated-topic(注意,指定的是3)

    Created topic "my-replicated-topic".

    [root@aa bin]# ./kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-replicated-topic

    Topic:my-replicated-topic PartitionCount:1 ReplicationFactor:3 Configs:

    Topic: my-replicated-topic Partition: 0 Leader: 2 Replicas: 2,3,0 Isr: 2,3,0



    [root@aa bin]# ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 4 --partitions 1 --topic my-replicated-topic

    Error while executing topic command Topic "my-replicated-topic" already exists.

    kafka.common.TopicExistsException: Topic "my-replicated-topic" already exists.

    at kafka.admin.AdminUtils$.createOrUpdateTopicPartitionAssignmentPathInZK(AdminUtils.scala:171)

    at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:156)

    at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:86)

    at kafka.admin.TopicCommand$.main(TopicCommand.scala:50)

    at kafka.admin.TopicCommand.main(TopicCommand.scala)


    [root@aa bin]# ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 4 --partitions 1 --topic my-replicated-topic2

    Created topic "my-replicated-topic2".

    [root@aa bin]# ./kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-replicated-topic2

    Topic:my-replicated-topic2 PartitionCount:1 ReplicationFactor:4 Configs:

    Topic: my-replicated-topic2 Partition: 0 Leader: 1 Replicas: 1,2,3,0 Isr: 1,2,3,0



    [root@aa bin]# ./kafka-topics.sh --describe --zookeeper localhost:2181 --topic zyl

    Topic:zyl PartitionCount:1 ReplicationFactor:1 Configs:

    Topic: zyl Partition: 0 Leader: 0 Replicas: 0 Isr: 0




    启动生产者并发送消息

    ./kafka-console-producer.sh --broker-list localhost:9092 --topic my-replicated-topic


    另外开个终端,启动消费者接受消息

    ./kafka-console-consumer.sh --zookeeper localhost:2181 --from-beginning --topic my-replicated-topic



    另外开个终端,可以看到有9个进程(分别为1个zk、4个kafka、2个控制台消费者、2个控制台生产者)

    [root@aa ~]# jps | grep -v Jps

    9802 QuorumPeerMain

    10392 Kafka

    11547 Kafka

    12007 ConsoleProducer

    10956 ConsoleConsumer

    10887 ConsoleProducer

    11469 Kafka

    12054 ConsoleConsumer

    11710 Kafka



    查看单独的进程:

    ps -ef | grep server.properties | grep -v grep

    ps -ef | grep "server1.properties" | grep -v grep



    测试leader

    杀掉进程,然后--describe查看

    [root@aaa bin]# ./kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-replicated-topic

    Topic:my-replicated-topic PartitionCount:1 ReplicationFactor:3 Configs:

    Topic: my-replicated-topic Partition: 0 Leader: 2 Replicas: 2,3,0 Isr: 2,3,0

    可以看到leader是2.

     

     [root@aa bin]# ./kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-replicated-topic2

    Topic:my-replicated-topic2 PartitionCount:1 ReplicationFactor:4 Configs:

    Topic: my-replicated-topic2 Partition: 0 Leader: 1 Replicas: 1,2,3,0 Isr: 1,2,3,0

    可以看到leader是2.


    [root@aa bin]# ps -ef | grep server1.properties | grep -v grep




    杀掉某个kafka进程

    kill -9 11469




     ./kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-replicated-topic2

    Topic:my-replicated-topic2 PartitionCount:1 ReplicationFactor:4 Configs:

    Topic: my-replicated-topic2 Partition: 0 Leader: 2 Replicas: 1,2,3,0 Isr: 2,3,0

    [root@aa bin]# ./kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-replicated-topic

    Topic:my-replicated-topic PartitionCount:1 ReplicationFactor:3 Configs:

    Topic: my-replicated-topic Partition: 0 Leader: 2 Replicas: 2,3,0 Isr: 2,3,0

    可以看到my-replicated-topic2的leader变成了2。


    问题:

    1.zookeeper-shell.sh如何使用

    2.能不能通过kafka-server-stop.sh停止某个kafka呢











    Command must include exactly one action: --list, --describe, --create or --alter

    Option                                  Description

    ------                                  -----------

    --alter                                 Alter the configuration for the topic.

    --config <name=value>                   A topic configuration override for the

                                              topic being created or altered.

    --create                                Create a new topic.

    --deleteConfig <name>                   A topic configuration override to be

                                              removed for an existing topic

    --describe                              List details for the given topics.

    --help                                  Print usage information.

    --list                                  List all available topics.

    --partitions <Integer: # of partitions> The number of partitions for the topic

                                              being created or altered (WARNING:

                                              If partitions are increased for a

                                              topic that has a key, the partition

                                              logic or ordering of the messages

                                              will be affected

    --replica-assignment                    A list of manual partition-to-broker

      <broker_id_for_part1_replica1 :         assignments for the topic being

      broker_id_for_part1_replica2 ,          created or altered.

      broker_id_for_part2_replica1 :

      broker_id_for_part2_replica2 , ...>

    --replication-factor <Integer:          The replication factor for each

      replication factor>                     partition in the topic being created.

    --topic <topic>                         The topic to be create, alter or

                                              describe. Can also accept a regular

                                              expression except for --create option

    --topics-with-overrides                 if set when describing topics, only

                                              show topics that have overridden

                                              configs

    --unavailable-partitions                if set when describing topics, only

                                              show partitions whose leader is not

                                              available

    --under-replicated-partitions           if set when describing topics, only

                                              show under replicated partitions

    --zookeeper <urls>                      REQUIRED: The connection string for

                                              the zookeeper connection in the form

                                              host:port. Multiple URLS can be

                                              given to allow fail-over.

  • 相关阅读:
    spring cloud 和 阿里微服务spring cloud Alibaba
    为WPF中的ContentControl设置背景色
    java RSA 解密
    java OA系统 自定义表单 流程审批 电子印章 手写文字识别 电子签名 即时通讯
    Hystrix 配置参数全解析
    spring cloud 2020 gateway 报错503
    Spring Boot 配置 Quartz 定时任务
    Mybatis 整合 ehcache缓存
    Springboot 整合阿里数据库连接池 druid
    java OA系统 自定义表单 流程审批 电子印章 手写文字识别 电子签名 即时通讯
  • 原文地址:https://www.cnblogs.com/jing1617/p/7449223.html
Copyright © 2011-2022 走看看