zoukankan      html  css  js  c++  java
  • 想买保时捷的运维李先生 求救求救求救求救

    问题来源

    kafka机器调到了64G的内存,还是报内存不足,后经排查是监控脚本telnet导致的,但是作为脚本的当事人不太能接受这个事实,然后自己一直找答案也没找出来,有以下几个问题在脑袋里一直没解决,也没啥思路。

    待解决的问题

    1)telnet kafka_ip 9092,为什么telnet kafka的端口,也会写入kafka?
    2)telnet是发送了什么数据给kafka,有218762506 bytes这么大?
    3)kafka的配置文件socket.request.max.bytes=104857600与内存设置参数-Xmx3G -Xms3G -XX:MaxDirectMemorySize=3G的关系是什么,怎么调优?
    4)kafka会将与9092相关的数据都写入到topic吗?
    5)telnet以Ctrl + C结束与按两下回车键结束的区别?

    问题描述

    1)使用telnet kafka_ip 9092监控kafka的端口是否存在,频率为30s
    2)kafka里偶尔会报错(telnet ip port 按两下enter)

    [2020-09-29 02:32:04,182] WARN [SocketServer brokerId=0] Unexpected error from /x.x.x.x; closing connection (org.apache.kafka.common.network.Selector)
    org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 218762506 larger than 104857600)
    at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:104)
    at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:335)
    at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:296)
    at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:560)
    at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:496)
    at org.apache.kafka.common.network.Selector.poll(Selector.java:425)
    at kafka.network.Processor.poll(SocketServer.scala:678)
    at kafka.network.Processor.run(SocketServer.scala:583)
    at java.lang.Thread.run(Thread.java:748)

    抓包数据:

    02:45:51.826114 IP ip-172-X-X-X.xxxxxxx.compute.internal.38900 > ip-172-X-X-X.xxxxxxx.compute.internal.XmlIpcRegSvc: Flags [S], seq 1800606878, win 26883, options [mss 8961,sackOK,TS val 3743494567 ecr 0,nop,wscale 7], length 0
    02:45:51.826238 IP ip-172-X-X-X.xxxxxxx.compute.internal.38900 > ip-172-X-X-X.xxxxxxx.compute.internal.XmlIpcRegSvc: Flags [.], ack 814657885, win 211, options [nop,nop,TS val 3743494567 ecr 1556371964], length 0
    02:45:53.417857 IP ip-172-X-X-X.xxxxxxx.compute.internal.38900 > ip-172-X-X-X.xxxxxxx.compute.internal.XmlIpcRegSvc: Flags [P.], seq 0:2, ack 1, win 211, options [nop,nop,TS val 3743496159 ecr 1556371964], length 2
    02:45:54.754650 IP ip-172-X-X-X.xxxxxxx.compute.internal.38900 > ip-172-X-X-X.xxxxxxx.compute.internal.XmlIpcRegSvc: Flags [P.], seq 2:4, ack 1, win 211, options [nop,nop,TS val 3743497495 ecr 1556373556], length 2
    02:45:54.755762 IP ip-172-X-X-X.xxxxxxx.compute.internal.38900 > ip-172-X-X-X.xxxxxxx.compute.internal.XmlIpcRegSvc: Flags [F.], seq 4, ack 2, win 211, options [nop,nop,TS val 3743497497 ecr 1556374893], length 0
    
    02:41:50.268763 IP ip-172-X-X-X.xxxxxxx.compute.internal.58708 > ip-172-X-X-X.xxxxxxx.compute.internal.XmlIpcRegSvc: Flags [S], seq 346111553, win 26883, options [mss 8961,sackOK,TS val 3743253009 ecr 0,nop,wscale 7], length 0
    02:41:50.269183 IP ip-172-X-X-X.xxxxxxx.compute.internal.58708 > ip-172-X-X-X.xxxxxxx.compute.internal.XmlIpcRegSvc: Flags [.], ack 3586979315, win 211, options [nop,nop,TS val 3743253009 ecr 1556045408], length 0
    02:41:58.695326 IP ip-172-X-X-X.xxxxxxx.compute.internal.58708 > ip-172-X-X-X.xxxxxxx.compute.internal.XmlIpcRegSvc: Flags [P.], seq 0:2, ack 1, win 211, options [nop,nop,TS val 3743261435 ecr 1556045408], length 2
    02:42:01.342115 IP ip-172-X-X-X.xxxxxxx.compute.internal.58708 > ip-172-X-X-X.xxxxxxx.compute.internal.XmlIpcRegSvc: Flags [P.], seq 2:4, ack 1, win 211, options [nop,nop,TS val 3743264082 ecr 1556053834], length 2
    02:42:01.343416 IP ip-172-X-X-X.xxxxxxx.compute.internal.58708 > ip-172-X-X-X.xxxxxxx.compute.internal.XmlIpcRegSvc: Flags [F.], seq 4, ack 2, win 211, options [nop,nop,TS val 3743264083 ecr 1556056481], length 0

    2)kafka里偶尔会报错(telnet ip port 按Control C)

    [2020-09-29 02:30:41,878] WARN [SocketServer brokerId=0] Unexpected error from /x.x.x.x; closing connection (org.apache.kafka.common.network.Selector)
    org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = -720899)
    at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:102)
    at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:335)
    at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:296)
    at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:560)
    at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:496)
    at org.apache.kafka.common.network.Selector.poll(Selector.java:425)
    at kafka.network.Processor.poll(SocketServer.scala:678)
    at kafka.network.Processor.run(SocketServer.scala:583)
    at java.lang.Thread.run(Thread.java:748)

    抓包数据:

    02:42:40.625070 IP ip-172-X-X-X.xxxxxxx.compute.internal.45020 > ip-172-X-X-X.xxxxxxx.compute.internal.XmlIpcRegSvc: Flags [S], seq 2552278719, win 26883, options [mss 8961,sackOK,TS val 3743303365 ecr 0,nop,wscale 7], length 0
    02:42:40.625428 IP ip-172-X-X-X.xxxxxxx.compute.internal.45020 > ip-172-X-X-X.xxxxxxx.compute.internal.XmlIpcRegSvc: Flags [.], ack 428181904, win 211, options [nop,nop,TS val 3743303366 ecr 1556095764], length 0
    02:42:42.682674 IP ip-172-X-X-X.xxxxxxx.compute.internal.45020 > ip-172-X-X-X.xxxxxxx.compute.internal.XmlIpcRegSvc: Flags [P.], seq 0:5, ack 1, win 211, options [nop,nop,TS val 3743305423 ecr 1556095764], length 5
    
    02:46:51.833208 IP ip-172-X-X-X.xxxxxxx.compute.internal.55998 > ip-172-X-X-X.xxxxxxx.compute.internal.XmlIpcRegSvc: Flags [S], seq 676331509, win 26883, options [mss 8961,sackOK,TS val 3743554574 ecr 0,nop,wscale 7], length 0
    02:46:51.833323 IP ip-172-X-X-X.xxxxxxx.compute.internal.55998 > ip-172-X-X-X.xxxxxxx.compute.internal.XmlIpcRegSvc: Flags [.], ack 4163492978, win 211, options [nop,nop,TS val 3743554574 ecr 1556431971], length 0
    02:46:52.838892 IP ip-172-X-X-X.xxxxxxx.compute.internal.55998 > ip-172-X-X-X.xxxxxxx.compute.internal.XmlIpcRegSvc: Flags [P.], seq 0:5, ack 1, win 211, options [nop,nop,TS val 3743555580 ecr 1556431971], length 5
  • 相关阅读:
    MyBatis与spring面试题-转载
    122. 买卖股票的最佳时机 II(贪心策略)
    121. 买卖股票的最佳时机
    120. 三角形最小路径和
    236. 二叉树的最近公共祖先(快手面试)
    b,b+树区别
    119. 杨辉三角 II
    118. 杨辉三角
    检查型异常(Checked Exception)与非检查型异常(Unchecked Exception)
    Redis
  • 原文地址:https://www.cnblogs.com/lemon-le/p/13889578.html
Copyright © 2011-2022 走看看