zoukankan      html  css  js  c++  java
  • kafka性能测试代码

     bin/kafka-producer-perf-test.sh --num-records 5000000 --record-size 5000 
     --topic kafkatopic2   
      --broker-list 
    
    
    bin/kafka-run-class.sh org.apache.kafka.clients.tools.ProducerPerformance kafkatopic2 50000000 100 -1 acks=1 
     bootstrap.servers=bdnodedn03.smpv.com:6667,bdnodedn08.smpv.com:6667,bdnodedn09.smpv.com:6667 
      buffer.memory=67108864 batch.size=8196
    
    
    Producer
    Setup
    bin/kafka-topics.sh --zookeeper bdnodedn01.smpv.com:2181 --create --topic test-rep-one --partitions 6 --replication-factor 1
    bin/kafka-topics.sh --zookeeper bdnodedn01.smpv.com:2181 --create --topic test-rep-two --partitions 6 --replication-factor 3
    
    Single thread, no replication
    bin/kafka-run-class.sh org.apache.kafka.tools.ProducerPerformance --print-metrics --topic test-rep-one 
    --num-records 6000000 --throughput 100000 
    --record-size 100 --producer-props bootstrap.servers=bdnodedn03.smpv.com:6667 
    buffer.memory=67108864 batch.size=8196
    
    Single-thread, async 3x replication
    bin/kafka-run-class.sh org.apache.kafka.tools.ProducerPerformance --print-metrics --topic test-rep-two 
    --num-records 6000000 --throughput 100000 --record-size 100 --producer-props bootstrap.servers=bdnodedn03.smpv.com:6667 
    acks=1 buffer.memory=67108864 batch.size=8196
    
    Single-thread, sync 3x replication
    bin/kafka-run-class.sh org.apache.kafka.tools.ProducerPerformance --print-metrics --topic test-rep-two 
    --num-records 6000000 --throughput 100000 --record-size 100 
    --producer-props bootstrap.servers=bdnodedn03.smpv.com:6667 acks=-1 buffer.memory=67108864 batch.size=8196
    
    Effect of message size
    
    for i in 10 100 1000 10000 100000;
    do
    echo ""
    echo $i
    bin/kafka-run-class.sh org.apache.kafka.tools.ProducerPerformance --print-metrics 
    --topic test-rep-two --num-records  $((1000*1024*1024/$i)) --throughput 100000 
    --record-size $i --producer-props bootstrap.servers=bdnodedn03.smpv.com:6667 acks=1 buffer.memory=67108864 batch.size=8196
    done;
    
    
    Consumer
    
    Consumer throughput
    bin/kafka-run-class.sh kafka.tools.ConsumerPerformance  --broker-list bdnodedn03.smpv.com:6667 
    --messages 6000000 --threads 1 
    --topic test-rep-two --print-metrics
    
    3 Consumers
    
    On three servers, run:
    bin/kafka-run-class.sh kafka.tools.ConsumerPerformance  --broker-list bdnodedn03.smpv.com:6667 
    --messages 6000000 --threads 1 
    --topic test-rep-two --print-metrics
    

      

  • 相关阅读:
    为何总是在无聊的事中浪费生命
    Objective-C Polymorphism
    [MONGODB]: WHEN ARBITER REQUIRED FOR REPLICA SET
    巴菲特已四度战胜股灾 称A股长期仍将向上(2015年09月14日)
    visual studio 2012 has stopped working
    [2015-04-17] 2015年的股市比2008年还危险……
    Using HiveServer2
    有了Hadoop MapReduce, 为什么还要Spark?
    HIVE: Map Join Vs Common Join, and SMB
    ORA-12170:TNS:连接超时
  • 原文地址:https://www.cnblogs.com/huaxiaoyao/p/11216232.html
Copyright © 2011-2022 走看看