zoukankan      html  css  js  c++  java
  • kafka CommitFailedException rebalanced poll

    1、异常详情

    org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records. 

    2、原因

    kafka等待消费者返回消费消息成功与否的响应时间超时。

    3、解决方法

    配置参数:session.timeout.ms:是消费者心跳响应的周期,若在这段时间内,没心跳响应,kafka的broker就会移除这个consumer然后rebalance。max.poll.records的大小应该置为一个合理的范围,最好可以在session.timeout.ms的范围内处理完所有的请求。

    The timeout used to detect consumer failures when using Kafka's group management facility. The consumer sends periodic heartbeats to indicate its liveness to the broker. If no heartbeats are received by the broker before the expiration of this session timeout, then the broker will remove this consumer from the group and initiate a rebalance. Note that the value must be in the allowable range as configured in the broker configuration by group.min.session.timeout.ms and group.max.session.timeout.ms.

  • 相关阅读:
    numpy数组(一)
    Iterator和Iterable区别:
    jupyter notebook安装相关问题
    Selenium+Headless Firefox配置
    最长不重复子串
    deprecated conversion from string constant to 'char*
    c++求字符串
    原型模式(Prototype)
    工厂方法模式(Factory Method)
    素数序列的生成及其应用(采用了自研的高效算法)
  • 原文地址:https://www.cnblogs.com/heihei1990/p/13892654.html
Copyright © 2011-2022 走看看