zoukankan      html  css  js  c++  java
  • Flink1.4.0连接Kafka0.10.2时遇到的问题

      Flink1.4.0连接部署在Linux上的Kafka0.10.2时,报如下异常:  

    org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase  - Async Kafka commit failed.
    org.apache.kafka.clients.consumer.CommitFailedException: 
    Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member.
    This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms,
    which typically implies that the poll loop
    is spending too much time message processing.
    You can address this either by increasing the session timeout
    or by reducing the maximum size of batches returned in poll() with max.poll.records. at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:770) at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:716) at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:784) at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:765) at org.apache.kafka.clients.consumer.internals.RequestFuture$1.onSuccess(RequestFuture.java:186) at org.apache.kafka.clients.consumer.internals.RequestFuture.fireSuccess(RequestFuture.java:149) at org.apache.kafka.clients.consumer.internals.RequestFuture.complete(RequestFuture.java:116) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler.fireCompletion(ConsumerNetworkClient.java:493) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.firePendingCompletedRequests(ConsumerNetworkClient.java:322) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:253) at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1047) at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:995) at org.apache.flink.streaming.connectors.kafka.internal.KafkaConsumerThread.run(KafkaConsumerThread.java:247)

      根据提示,设置如下配置后,问题解决:

            "session.timeout.ms":10000
  • 相关阅读:
    OLAP、OLTP的介绍和比较
    Python发行版本Anaconda的安装说明:基于Anaconda2-4.3.1-Windows-x86_64
    Hive学习:Hive连接JOIN用例详解
    Linux学习一:图解CentOS 6.5安装步骤
    Linux学习二:基于CentOS 6.5的网络配置
    一个恒等式.jpg
    centos 7 The path "" is not a valid path to the 3.10.0-229.el7.x86_64 kernel headers. Would you like to change it?
    Redis从入门到精通
    solr服务的搭建
    SpringMvc的运行流程
  • 原文地址:https://www.cnblogs.com/liugh/p/9348503.html
Copyright © 2011-2022 走看看