zoukankan      html  css  js  c++  java
  • Kafka Broker | Command-line Options and Procedure(转发)

    原文:

    https://data-flair.training/blogs/kafka-broker/

    In this Apache Kafka tutorial, we are going to learn Kafka Broker.

    Kafka Broker manages the storage of messages in the topic(s).

    If Apache Kafka has more than one broker, that is what we call a Kafka cluster.

    Moreover, in this Kafka Broker Tutorial, we will learn how to start Kafka Broker and Kafka command-line Option.


    So, let’s start Apache Kafka Broker.

    What is Kafka Broker?

    A Kafka broker is also known as Kafka server and a Kafka node. These all names are its synonyms.

    In simple words, a broker is a mediator between two. However, Kafka broker is more precisely described as a Message Broker which is responsible for mediating the conversation between different computer systems, guaranteeing delivery of the message to the correct parties.

    Read Apache Kafka Security | Need and Components of Kafka

    Hence, the Kafka cluster typically consists of multiple brokers.

    Kafka Cluster uses Zookeeper for maintaining the cluster state.

    A single Broker can handle thousands of reads and writes per second. Whereas, if there is no performance impact, each broker can handle TB of messages.

    In addition, to be very sure that ZooKeeper performs broker leader election.


    Kafka Broker

    Basically, a broker in Kafka is modeled as KafkaServer, which hosts topics.

    Here, given topics are always partitioned across brokers, in a cluster a single broker hosts topic partitions of one or more topics actually, even when a topic is only partitioned to just a single partition.

    In addition, producers send a message to a broker, after receiving broker stores them on disk keyed by unique offset.

    Moreover, by topic, partition and offset a broker allows consumers to fetch messages.

    However, brokers can create a cluster just by sharing information with each other directly or indirectly using Zookeeper.

    Also, we can say Kafka cluster has exactly one broker which acts as the Controller. By using kafka-server-start.sh script, we can start a single Kafka broker.

    How to Start Kafka Broker?

    • Start Zookeeper.
    1. ./bin/zookeeper-server-start.sh config/zookeeper.properties

    We can start a Kafka server, only when Zookeeper is up and running(that will connect to Zookeeper).

    1. ./bin/kafka-server-start.sh config/server.properties
    2. kafka-server-start.sh script

    Hence, kafka-server-start.sh starts a broker.

    1. $ ./bin/kafka-server-start.sh
    2. USAGE: ./bin/kafka-server-start.sh [-daemon] server.properties [--override property=value]*

    Note- make sure that Zookeeper is up and running before you run kafka-server-start.sh.

    Moreover, use zookeeper-server-start shell script.

    Have a look at Apache Kafka + Spark Streaming Integration

    kafka-server-start.sh uses config/log4j.properties for logging configuration that we can override using KAFKA_LOG4J_OPTS environment variable.

    1. KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:config/log4j.properties"

    kafka-server-start.sh accepts KAFKA_HEAP_OPTS and EXTRA_ARGS environment variables.

    Kafka Command-line Options

    -name

    Defaults to kafkaServer when in daemon mode.

    -loggc

    Enabled when in daemon mode.

    -daemon

    Enables daemon mode.

    –override property=value

    A value that should override the value set for property in server.properties file.

    Let’s revise Apache Kafka Streams | Stream Processing Topology

    1. $ ./bin/kafka-server-start.sh config/server.properties --override broker.id=100
    2. ...
    3. INFO [KafkaServer id=100] started (kafka.server.KafkaServer)

    Conclusion

    Hence, we have learned the complete Kafka Broker, along with commands to start it. However, if any doubt occurs, feel free to ask in the comment section.

    See also – 

    Kafka- serialization and deserialization

    For reference

  • 相关阅读:
    白话https
    网络分层体系结构
    软件度量
    【Java】itext根据模板生成pdf(包括图片和表格)
    【java】Freemarker 动态生成word(带图片表格)
    压力测试-接口关联
    压力测试-录制脚本
    jmeter性能测试2:基础功能介绍
    jmeter性能测试1:目录解析
    桃花运
  • 原文地址:https://www.cnblogs.com/panpanwelcome/p/13534341.html
Copyright © 2011-2022 走看看