zoukankan      html  css  js  c++  java
  • vc-mysql-sniffer统计MySQL的SQL分布

    有时候我们需要统计线上的SQL执行情况,比如想知道哪条SQL执行最频繁,我们可以开启general_log,然后进行统计,但是general_log开启非常损耗性能,那么我们可以使用vc-mysql-sniffer来代替,该工具是编译好的二进制,下载即可使用。下载地址:

    https://www.vividcortex.com/resources/network-analyzer-for-mysql

    我这里修改了网上的一个脚本,分析vc-mysql-sniffer抓取到的结果统计频繁执行的SQL。

    #!/usr/bin/python
    #coding:utf8
    # python analysis-vc-log.py 3315 | sort | uniq -c | sort -nr |head -n 10
    import re
    import sys
    import os
    import commands
    vc_sniffer_time=5
    port=sys.argv[1]
    vc_cmd=""" /usr/bin/timeout %s  /data/software/vc-mysql-sniffer -binding="[::]:%s"  > /tmp/tmp_vc_mysql_%s.txt """ % (vc_sniffer_time,port,port)
    outtext = commands.getoutput(vc_cmd)
    cmd=""" grep -Ev '# Time:|# User@Host' /tmp/tmp_vc_mysql_%s.txt |sed 's/# Query_time.*/myxxxxx/g' |awk BEGIN{RS=EOF}'{gsub(/\n/," ");print}'|awk BEGIN{RS=EOF}'{gsub(/myxxxxx/,"\n");print}' >/tmp/vc_mysql_%s.txt""" % (port,port)
    outtext = commands.getoutput(cmd)
    file="/tmp/vc_mysql_%s.txt" % (port)
    logFo = open(file)
    for line in logFo:
        line = re.sub(r"
    ","",line)
        lineMatch = re.match(r".*",line)
        if lineMatch:
            lineTmp = lineMatch.group(0)
            lineTmp = lineTmp.lower()
            # remove extra space
            lineTmp = re.sub(r"s+", " ",lineTmp)
            # replace values (value) to values (x)
            lineTmp = re.sub(r"valuess*(.*?)", "values (x)",lineTmp)
            # replace filed = 'value' to filed = 'x'
            lineTmp = re.sub(r"(=|>|<|>=|<=)s*('|").*?2","\1 'x'",lineTmp)
            # replace filed = value to filed = x
            lineTmp = re.sub(r"(=|>|<|>=|<=)s*[0-9]+","\1 x",lineTmp)
            # replace like 'value' to like 'x'
            lineTmp = re.sub(r"likes+('|").*?1","like 'x'",lineTmp)
            # replace in (value) to in (x)
            lineTmp = re.sub(r"ins+(.*?)","in (x)",lineTmp)
            # replace between '...' and '...' to between 'x' and 'x'
            lineTmp = re.sub(r"betweens+('|").*?1s+ands+1.*?1","between 'x' and 'x' ",lineTmp)
            # replace between ... and ... to between x and x
            lineTmp = re.sub(r"betweens+[0-9]+s+ands+[0-9]+","between x and x ",lineTmp)
            # replace limit x,y to limit
            lineTmp = re.sub(r"limit.*","limit",lineTmp)
            print lineTmp
    logFo.close()
    python analysis-vc-log.py 3310 | sort | uniq -c | sort -nr |head -n 10

    抓取mysql 3310端口,统计执行最频繁的10条SQL。结果我这里就不贴上来了。有兴趣的同学自己测试。

    参考资料:

    https://www.centos.bz/2015/02/analysis-mysql-general-log/

  • 相关阅读:
    SqlHelper使用详解
    c# 分布式事务以及MSDTC
    DataGridView隔行显示不同的颜色
    C# WinForm开发系列 DataGridView
    使用SQLHelper类调用带输出、返回参数的存储过程
    MySQLHelper类使用说明
    搜索页面———心得
    侨光卡路网站的工作点滴记录
    服务地图功能的开发心得
    成功项目经理手册
  • 原文地址:https://www.cnblogs.com/gomysql/p/8267070.html
Copyright © 2011-2022 走看看