zoukankan      html  css  js  c++  java
  • IDEA kafka producer数据输出缓慢 和 kafka consumer 报错的处理

    问题1.

    IDEA 中Kafa_Producer程序数据输出缓慢 但不报错

    问题2.

    Kafa_Consumer程序报错:

    17/11/10 11:31:11 ERROR ReceiverTracker: Deregistered receiver for stream 0: Error starting receiver 0 - org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server 'localhost:2181' with timeout of 10000 ms

    at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1233)
        at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:157)
        at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:131)
        at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:103)
        at kafka.utils.ZkUtils$.apply(ZkUtils.scala:85)
        at kafka.consumer.ZookeeperConsumerConnector.connectZk(ZookeeperConsumerConnector.scala:194)
        at kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:142)
        at kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:159)
        at kafka.consumer.Consumer$.create(ConsumerConnector.scala:112)
        at org.apache.spark.streaming.kafka.KafkaReceiver.onStart(KafkaInputDStream.scala:100)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
        at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:607)
        at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:597)
        at org.apache.spark.SparkContext$$anonfun$34.apply(SparkContext.scala:2173)
        at org.apache.spark.SparkContext$$anonfun$34.apply(SparkContext.scala:2173)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
        at org.apache.spark.scheduler.Task.run(Task.scala:108)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:335)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

    解决:

    1.需要启动Zookeeper服务:./bin/zookeeper-server-start.sh config/zookeeper.properties

    2.启动Kafka服务:./bin/kafka-server-start.sh config/server.properties
  • 相关阅读:
    How to install tcpping on Linux.md
    当前服务器的并发连接数查看
    Windows 系统下安装 dig 命令
    paping使用来测试联通&网站由于tcp协议导致的无法通信问题超时问题
    SSH反向连接及Autossh
    设置Windows Azure Linux虚拟机中的root账户
    用UltraISO制作CentOS U盘安装盘
    CentOS7 修改网卡名称为eth0
    iOS开发--绘图教程
    ios开发--网页中调用JS与JS注入
  • 原文地址:https://www.cnblogs.com/soyo/p/7813778.html
Copyright © 2011-2022 走看看