启动报错信息
[2020-08-19T10:54:34,504][ERROR][logstash.outputs.kafka ] Unable to create Kafka producer from given configuration {:kafka_error_message=>org.apache.kafka.common.config.ConfigException: Invalid value 32768 for configuration receive.buffer.bytes: Expected value to be a 32-bit integer, but it was a java.lang.Long, :cause=>nil}
官网提示
logstash6.4.0配置output到kafka会有这个bug,可以通过更新logstash-output-kafka插件解决
https://www.elastic.co/guide/en/logstash/6.7/logstash-6-4-0.html
官网给出执行下面的命令解决:
bin/logstash-plugin update logstash-output-kafka
但是执行更新一直等待卡住没有响应,于是...
先查看已安装的插件
本人的目录如下,并能看到已安装 logstash-output-kafka-7.1.2
/work/logstash-6.4.0/vendor/bundle/jruby/2.3.0/gems
或者执行命令查看已安装插件
bin/logstash-plugin list --verbose
执行命令如果遇到报错
Gem::LoadError: You have already activated jruby-openssl 0.10.4, but your Gemfile requires jruby-openssl 0.10.1. Prepending `bundle exec` to your command may solve this.
原因 :
1. logstash 已激活使用 jruby-openssl 0.10.4
2. /work/logstash-6.4.0/ 目录有2个文件(与bin同级):Gemfile 和 Gemfile.lock 却还是 jruby-openssl 0.10.1
解决:
参考国内 RubyGems 镜像:https://gems.ruby-china.com/ 可以找到语法
添加/修改,上面提到的2个文件 :
vim Gemfile
添加/修改内容为: gem 'jruby-openssl', '~> 0.10.4' vim Gemfile.lock
修改: jruby-openssl (0.10.1-java) 改为 jruby-openssl (0.10.4-java)
安装7.1.3版本的 logstash-output-kafka 插件
$ cd /work/logstash-6.4.0/bin $ ./logstash-plugin install --version 7.1.3 logstash-output-kafka Validating logstash-output-kafka-7.1.3 Installing logstash-output-kafka Installation successful