zoukankan      html  css  js  c++  java
  • Flink1.4.0连接Kafka0.10.2时遇到的问题

      Flink1.4.0连接部署在Linux上的Kafka0.10.2时,报如下异常:  

    org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase  - Async Kafka commit failed.
    org.apache.kafka.clients.consumer.CommitFailedException: 
    Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member.
    This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms,
    which typically implies that the poll loop
    is spending too much time message processing.
    You can address this either by increasing the session timeout
    or by reducing the maximum size of batches returned in poll() with max.poll.records. at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:770) at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:716) at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:784) at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:765) at org.apache.kafka.clients.consumer.internals.RequestFuture$1.onSuccess(RequestFuture.java:186) at org.apache.kafka.clients.consumer.internals.RequestFuture.fireSuccess(RequestFuture.java:149) at org.apache.kafka.clients.consumer.internals.RequestFuture.complete(RequestFuture.java:116) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler.fireCompletion(ConsumerNetworkClient.java:493) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.firePendingCompletedRequests(ConsumerNetworkClient.java:322) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:253) at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1047) at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:995) at org.apache.flink.streaming.connectors.kafka.internal.KafkaConsumerThread.run(KafkaConsumerThread.java:247)

      根据提示,设置如下配置后,问题解决:

            "session.timeout.ms":10000
  • 相关阅读:
    SQL 分页存储
    ubuntu下删除openjdk,改用sun jdk
    LCD 驱动的整体分析。
    关于IT行业人员吃的都是青春饭?[透彻讲解]
    【转载】I.MX25上摄像头调试总结
    wifi 移植
    函数指针的透彻分析
    关于module_param()宏 (转)
    problem with aptget update ubuntu 11.10
    YUV 格式讲解
  • 原文地址:https://www.cnblogs.com/liugh/p/9348503.html
Copyright © 2011-2022 走看看