zoukankan      html  css  js  c++  java
  • 【解决了一个问题】腾讯云中使用ckafka生产消息时出现“kafka server: Message contents does not match its CRC.”错误

    初始化的主要代码如下:

    	config := sarama.NewConfig()
    	config.Producer.RequiredAcks = sarama.WaitForAll // Wait for all in-sync replicas to ack the message
    	config.Producer.Retry.Max = 10                   // Retry up to 10 times to produce the message
    	config.Producer.Return.Successes = true
    	config.Producer.Compression = sarama.CompressionSnappy // Compress messages
    	config.Producer.Partitioner = sarama.NewRandomPartitioner
    	config.Producer.Return.Errors = true
    	producer, err := sarama.NewSyncProducer(brokerList, config)
    	if err != nil {
    		log.Printf("sarama.NewSyncProducer fail:%+v
    ", err)
    		return err
    	}
    

    生产消息的主要代码如下:

    	msg := sarama.ProducerMessage{
    		Topic: monitorInstance.topic,
    		Value: sarama.StringEncoder(resportString),
    		//Key: sarama.StringEncoder("monitor_data"),
    	}
    	partition, offset, err := (*producer).SendMessage(&msg)
    	if err != nil {
    		log.Printf("kafka SendMessage fail:%+v
    ", err)
    		return
    	}
    	log.Printf("Your data is stored with unique identifier important/%d/%d
    ", partition, offset)
    

    尝试了以下办法都不行:
    1.把producer从async修改为sync
    2.增加msg中的key
    3.尝试更换topic

    最后发现是这个问题:
    config.Producer.Compression = sarama.CompressionSnappy
    注释掉这一行后正常。

    难道就不能加压缩?问了腾讯云的工程师,得到了解决:
    config.Version = sarama.V2_1_0_0
    config.Producer.Compression = sarama.CompressionSnappy

  • 相关阅读:
    哈夫曼(Huffman)编码
    面向对象的3个基本要素和5个基本设计原则(整理)
    面向对象设计原则OO
    Java多线程中start()和run()的区别
    HBase入门
    SparkGraphXTest.scala
    IntellijIdea中常用的快捷键
    SparkSQLTest.scala
    SparkStreamingTest.scala
    (转)理解POST和PUT的区别,顺便提下RESTful
  • 原文地址:https://www.cnblogs.com/ahfuzhang/p/12737304.html
Copyright © 2011-2022 走看看