zoukankan      html  css  js  c++  java
  • Flink1.4.0连接Kafka0.10.2时遇到的问题

      Flink1.4.0连接部署在Linux上的Kafka0.10.2时,报如下异常:  

    org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase  - Async Kafka commit failed.
    org.apache.kafka.clients.consumer.CommitFailedException: 
    Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member.
    This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms,
    which typically implies that the poll loop
    is spending too much time message processing.
    You can address this either by increasing the session timeout
    or by reducing the maximum size of batches returned in poll() with max.poll.records. at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:770) at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:716) at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:784) at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:765) at org.apache.kafka.clients.consumer.internals.RequestFuture$1.onSuccess(RequestFuture.java:186) at org.apache.kafka.clients.consumer.internals.RequestFuture.fireSuccess(RequestFuture.java:149) at org.apache.kafka.clients.consumer.internals.RequestFuture.complete(RequestFuture.java:116) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler.fireCompletion(ConsumerNetworkClient.java:493) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.firePendingCompletedRequests(ConsumerNetworkClient.java:322) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:253) at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1047) at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:995) at org.apache.flink.streaming.connectors.kafka.internal.KafkaConsumerThread.run(KafkaConsumerThread.java:247)

      根据提示,设置如下配置后,问题解决:

            "session.timeout.ms":10000
  • 相关阅读:
    fixed与sticky的区别
    自我介绍以及web课程目标
    DOM&BOM
    web中常用单位的使用
    Oracle 使用 DBLINK详解(转载) 挪威
    Sql server 无法删除用户的处理办法(转载) 挪威
    ICMP类型
    makefile笔记
    [笔记]Makefile wildcard
    在Visual Studio 2005下配置WinPcap开发环境
  • 原文地址:https://www.cnblogs.com/liugh/p/9348503.html
Copyright © 2011-2022 走看看