zoukankan      html  css  js  c++  java
  • SUSE中搭建kafka

     搭建环境:

    • JDK: java version 1.8.0_221
    • zookeeper:zookeeper-3.5.2
    • kafka: kafka-2.11-1.1.0

    一、安装JDK

    由于需要java环境,所以我们需先安装JDK

    1、下载JDK并解压

    mkdir /usr/java     //在usr下创建java
    cp jdk-8u221-linux-x64.tar.gz /usr/java/     //将下载好的JDK文件copy到/usr/java目录
    cd /usr/java
    tar -xzvf jdk-8u221-linux-x64.tar.gz    //解压到当前目录

    2、配置环境变量

    vi /etc/profile

    将下面内容添加到文件最后:

    JAVA_HOME=/usr/java/jdk1.8.0_221
    CLASSPATH=$JAVA_HOME/lib/
    PATH=$PATH:$JAVA_HOME/bin
    export PATH JAVA_HOME CLASSPATH

    保存后退出,并执行

    source /etc/profile  //使环境变量生效

    3、查看安装情况

    java -version

    有时会遇到:

    每次打开一个终端,都要输入source /etc/profile 环境变量才会生效!

    解决方法:

    vi ~/.bashrc
    将  source /etc/profile 添加到bashrc文件中

     

     二、安装kafka 1.1.0

    kafka下载地址:http://kafka.apache.org/downloads

    1、(这个安装是真滴难受,各种报错...)

    下载好后,解压压缩包,并进入其目录

    tar -xzvf kafka_2.11-1.1.0.tgz
    cd kafka_2.11-1.1.0

    2、这里,需先在一个终端启动zookeeper服务

    bin/zookeeper-server-start.sh config/zookeeper.properties &  //

    我用的kafka自带的zookeeper-server-start.sh启动zookeeper服务,然后就各种报错启动不来kafka,最后我直接去官网上下载zookeeper,重新自己安装

    安装方法可以参考文章:https://www.jianshu.com/p/7dcec9758614

    3、然后打开另一个终端,再启动kafka服务

    bin/kafka-server-start.sh config/server.properties

    4、测试一下

    创建名为“test”的 topic,并查看

    linux-ym54:/usr/local/kafka/kafka_2.11-1.1.0 # bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
    Created topic "test".
    linux-ym54:/usr/local/kafka/kafka_2.11-1.1.0 # bin/kafka-topics.sh --list --zookeeper localhost:2181
    test

     发送消息:

    bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test

     接收消息:

    bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning

     

     有时会遇到:

    • 在启动zookeeper服务时,会出现
    INFO Closed socket connection for client /127.0.0.1:48452 which had sessionid 0x15698f5ac360001 (org.apache.zookeeper.server.NIOServerCnxn)

    That is not an error. The topic details are fetched from Zookeeper. Hence the client (invoked by kafka-topics.sh) first connects to Zookeeper, then establishes a session, gets the data and then disconnects at the end. This is the expected behavior of any clients that will get some data from Zookeeper.

    • 在启动kafka服务时,会出现
    [2019-08-10 09:59:58,527] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
    java.net.UnknownHostException: linux-ym54: linux-ym54: Name or service not known
        at java.net.InetAddress.getLocalHost(InetAddress.java:1506)
        at kafka.server.KafkaServer$$anonfun$3.apply(KafkaServer.scala:387)
        at kafka.server.KafkaServer$$anonfun$3.apply(KafkaServer.scala:385)
        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
        at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
        at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
        at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
        at scala.collection.AbstractTraversable.map(Traversable.scala:104)
        at kafka.server.KafkaServer.createBrokerInfo(KafkaServer.scala:385)
        at kafka.server.KafkaServer.startup(KafkaServer.scala:253)
        at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
        at kafka.Kafka$.main(Kafka.scala:92)
        at kafka.Kafka.main(Kafka.scala)
    Caused by: java.net.UnknownHostException: linux-ym54: Name or service not known
        at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
        at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
        at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
        at java.net.InetAddress.getLocalHost(InetAddress.java:1501)
        ... 13 more

    修改 /etc/hosts文件,

    把 “ your ip     localhost ” 添加进去

    三、kafka在windows中的安装

    可以参考文章:https://blog.csdn.net/u012050154/article/details/76270655

    有时会遇到:

    xshell连接不上的问题!

    • 检查ssh是否开启
    service sshd status   
    • 检查22端口是否开放
    • 最后检查发现是防火墙导致的

    关闭防火墙,然后重启。

    linux-ym54:/home/ilk/Desktop # systemctl list-dependencies |grep firewall
    ●   ├─SuSEfirewall2.service
    ●   ├─SuSEfirewall2_init.service
    linux-ym54:/home/ilk/Desktop # chkconfig SuSEfirewall2 off
    Removed symlink /etc/systemd/system/multi-user.target.wants/SuSEfirewall2.service.
    Removed symlink /etc/systemd/system/multi-user.target.wants/SuSEfirewall2_init.service.
    Removed symlink /etc/systemd/system/SuSEfirewall2_setup.serv
  • 相关阅读:
    UOJ 30 【CF Round #278】Tourists
    CodeForces 1070F Katya and Segments Sets
    Django系列3:Model的简单对接
    Django系列2:MTV简单流程
    python virtualenv
    vscode使用
    Django 系列1:Django简介
    vue-cli3/4 vue ui创建elementui项目
    npm安装参数
    vue系列11:vuex
  • 原文地址:https://www.cnblogs.com/xdjun/p/11314284.html
Copyright © 2011-2022 走看看