zoukankan      html  css  js  c++  java
  • Flume监控指标项

    配置监控

    1. 修改flume.env.sh

      export JAVA_OPTS=”-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=5445 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false”
      
    2. 使用以下脚本

      flume-ng agent 
      --name a1 
      --conf $FLUME_HOME/conf 
      --conf-file $FLUME_HOME/conf/exec-memory-hdfs-partition.conf 
      -Dflume.root.logger=INFO,console 
      -Dflume.monitoring.type=http 
      -Dflume.monitoring.port=1234
      
      

    指标项整理

    OS监控项

    objectName 指标项 说明
    java.lang:type=OperatingSystem FreePhysicalMemorySize 空闲物理内存
    java.lang:type=OperatingSystem SystemCpuLoad 系统CPU利用率
    java.lang:type=OperatingSystem ProcessCpuLoad 进程CPU利用率
    java.lang:type=GarbageCollector,name=PS Scavenge CollectionCount GC次数

    json数据格式

    {
        "SOURCE.src-1":{
            "OpenConnectionCount":"0",
            "Type":"SOURCE",
            "AppendBatchAcceptedCount":"1355",
            "AppendBatchReceivedCount":"1355",
            "EventAcceptedCount":"28286",
            "AppendReceivedCount":"0",
            "StopTime":"0",
            "StartTime":"1442566410435",
            "EventReceivedCount":"28286",
            "AppendAcceptedCount":"0"
        },
        "CHANNEL.ch-1":{
            "EventPutSuccessCount":"28286",
            "ChannelFillPercentage":"0.0",
            "Type":"CHANNEL",
            "StopTime":"0",
            "EventPutAttemptCount":"28286",
            "ChannelSize":"0",
            "StartTime":"1442566410326",
            "EventTakeSuccessCount":"28286",
            "ChannelCapacity":"1000000",
            "EventTakeAttemptCount":"313734329512"
        },
        "SINK.sink-1":{
            "Type":"SINK",
            "ConnectionClosedCount":"0",
            "EventDrainSuccessCount":"28286",
            "KafkaEventSendTimer":"482493",
            "BatchCompleteCount":"0",
            "ConnectionFailedCount":"0",
            "EventDrainAttemptCount":"0",
            "ConnectionCreatedCount":"0",
            "BatchEmptyCount":"0",
            "StopTime":"0",
            "RollbackCount":"9",
            "StartTime":"1442566411897",
            "BatchUnderflowCount":"0"
        }
    }
    

    指标项说明

    source监控项

    objectName
    (会随实际情况而变化)
    指标项 说明
    org.apache.flume.source:type=r1 OpenConnectionCount 目前与客户端或sink保持连接的总数量
    org.apache.flume.source:type=r1 AppendBatchAcceptedCount 成功提交到channel的批次的总数量
    org.apache.flume.source:type=r1 AppendBatchReceivedCount 接收到事件批次的总数量
    org.apache.flume.source:type=r1 AppendAcceptedCount 逐条录入的次数
    org.apache.flume.source:type=r1 AppendReceivedCount 每批只有一个事件的事件总数量
    org.apache.flume.source:type=r1 EventAcceptedCount 成功写出到channel的事件总数量
    org.apache.flume.source:type=r1 EventReceivedCount 目前为止source已经接收到的事件总数量
    org.apache.flume.source:type=r1 StartTime source启动时的毫秒值时间
    org.apache.flume.source:type=r1 StopTime source停止时的毫秒值时间,为0表示一直在运行

    channel监控项

    objectName
    (会随实际情况而变化)
    指标项 说明
    org.apache.flume.channel:type=c1 EventPutAttemptCount Source尝试写入Channe的事件总次数
    org.apache.flume.channel:type=c1 EventPutSuccessCount 成功写入channel且提交的事件总次数
    org.apache.flume.channel:type=c1 EventTakeAttemptCount sink尝试从channel拉取事件的总次数。
    org.apache.flume.channel:type=c1 EventTakeSuccessCount sink成功从channel读取事件的总数量
    org.apache.flume.channel:type=c1 ChannelSize 目前channel中事件的总数量
    org.apache.flume.channel:type=c1 ChannelCapacity channel的容量
    org.apache.flume.channel:type=c1 ChannelFillPercentage channel已填入的百分比
    org.apache.flume.channel:type=c1 StartTime channel启动时的毫秒值时间
    org.apache.flume.channel:type=c1 StopTime channel停止时的毫秒值时间,为0表示一直在运行

    sink监控项

    objectName
    (会随实际情况而变化)
    指标项 说明
    org.apache.flume.sink:type=k1 ConnectionCreatedCount 创建的连接数量
    org.apache.flume.sink:type=k1 ConnectionClosedCount 关闭的连接数量
    org.apache.flume.sink:type=k1 ConnectionFailedCount 由于错误关闭的连接数量
    org.apache.flume.sink:type=k1 BatchEmptyCount 批量处理event的个数为0的数量-表示source写入数据的速度比sink处理数据的速度慢
    org.apache.flume.sink:type=k1 BatchUnderflowCount 批量处理event的个数小于批处理大小的数量
    org.apache.flume.sink:type=k1 BatchCompleteCount 批量处理event的个数等于批处理大小的数量
    org.apache.flume.sink:type=k1 EventDrainAttemptCount sink尝试写出到存储的事件总数量
    org.apache.flume.sink:type=k1 EventDrainSuccessCount sink成功写出到存储的事件总数量
    org.apache.flume.sink:type=k1 StartTime channel启动时的毫秒值时间
    org.apache.flume.sink:type=k1 StopTime channel停止时的毫秒值时间,为0表示一直在运行
  • 相关阅读:
    LeetCode Subsets II
    LeetCode Rotate Image
    LeetCode Palidrome Number
    LeetCode Generate Parentheses
    LeetCode Maximum Subarray
    LeetCode Set Matrix Zeroes
    LeetCode Remove Nth Node From End of List
    Linux Loop设备 使用
    Linux 文件系统大小调整
    LeetCode N-Queens II
  • 原文地址:https://www.cnblogs.com/fengzzi/p/10033739.html
Copyright © 2011-2022 走看看