zoukankan      html  css  js  c++  java
  • Flink1.4.0连接Kafka0.10.2时遇到的问题

      Flink1.4.0连接部署在Linux上的Kafka0.10.2时,报如下异常:  

    org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase  - Async Kafka commit failed.
    org.apache.kafka.clients.consumer.CommitFailedException: 
    Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member.
    This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms,
    which typically implies that the poll loop
    is spending too much time message processing.
    You can address this either by increasing the session timeout
    or by reducing the maximum size of batches returned in poll() with max.poll.records. at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:770) at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:716) at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:784) at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:765) at org.apache.kafka.clients.consumer.internals.RequestFuture$1.onSuccess(RequestFuture.java:186) at org.apache.kafka.clients.consumer.internals.RequestFuture.fireSuccess(RequestFuture.java:149) at org.apache.kafka.clients.consumer.internals.RequestFuture.complete(RequestFuture.java:116) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler.fireCompletion(ConsumerNetworkClient.java:493) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.firePendingCompletedRequests(ConsumerNetworkClient.java:322) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:253) at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1047) at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:995) at org.apache.flink.streaming.connectors.kafka.internal.KafkaConsumerThread.run(KafkaConsumerThread.java:247)

      根据提示,设置如下配置后,问题解决:

            "session.timeout.ms":10000
  • 相关阅读:
    【开发工具 idea】值得推荐的15款idea插件
    【python pip】一招解决pip下载过慢问题
    【python pip】一招解决移动python安装路径pip不可用问题
    xmake新增对Qt编译环境支持
    xmake入门,构建项目原来可以如此简单
    xmake新增对Cuda代码编译支持
    不同编译器对预编译头文件的处理
    xmake v2.1.9版本发布,增加可视化图形菜单配置
    xmake-vscode插件开发过程记录
    xmake-vscode插件开发过程记录
  • 原文地址:https://www.cnblogs.com/liugh/p/9348503.html
Copyright © 2011-2022 走看看