zoukankan      html  css  js  c++  java
  • 第四模块 :微服务调用链监控CAT架构和实践

     

    采样率:每一个请求为都进行记录,或者100次请求为记录50次

     各个开源框架都满足opentracing的标准,只要使用opentracing标准埋点的客户端,可以使用不同的客户端去展示,opentracing对客户端调用的标准、tracing的标准进行了规范化

    69.调用链监控产品和比较~1.mp4

    1、调用链的可视化,上面三款产品都支持

    2、服务之间的依赖关系,Pinpoint做的最好,图形化展示直观形象

    3、埋点 cat和zipnkin都是侵入式的,Pinpoint的是aop的不需要修改代码

    4、实例的cpu、内存的情况,Pinpoint有

    71.CAT 典型报表~1.mp4

    cat的 报表丰富是cat的最大的优点

     

     cat会把它接入的应用程序对于的报错情况,实时的做成一个报错大盘,以1分钟为单位展示出来

    cat是支持业务监控的,例如支付成功次数等,但是对于业务指标,最好使用专门的metris监控工具去做,cat最大的长处是调用链监控

     

    cat除了全链路追踪功能之外,还有其他功能

    transaction报表

    整个应用在1小时内发生了多少次调用,有多少次是失败的,整个应用的最小延迟,最大延迟,平均值,如果95line和99line和平均值相差不大,说明该系统很稳定,qps就是服务的调用频率

    点击该应用可以看到该应用下接口的性能统计情况

    例如一个url请求超过了50毫秒。我们可以进行搜索,查看对于的具体信息

     

    72.CAT 告警简介~1.mp4

     73.CAT 架构设计~1.mp4

    原始的调用链trace数据存储在hdfs中,聚合的报表数据存储在mysql中,按照分钟、小时的报表聚合数据是存储在mysql中的

     74.【实验】CAT 本地部署~1.mp4

    cat的安装,第一步首先安装cat的源码

     这里按照视频上的,我们安装2.0.0版本

    进入改目录执行下面的操作:进行maven打包的操作

    mvn clean install -DskipTests

    打包成功之后war包位于cat-home目录下,我们将cat-alpha-2.0.0.war重新命名为cat.war

    打包完成之后接下来要进行数据库的操作

     数据库的脚本在这个目录下

     创建一个cat的数据库

    然后执行cat.sql 的脚本

    我们cat运行在d盘,这里需要强调的是在tomcat所在的盘根目录需要有data文件夹,这里我们把tomcat放在d盘目录,因此我们在d盘建立下面的目录D:dataappdatascat

    我们将下面的三个xml文件拷贝到D:dataappdatascat目录下

    接下来修改client.xml的配置文件

    <?xml version="1.0" encoding="utf-8"?>
    
    <config mode="client" xmlns:xsi="http://www.w3.org/2001/XMLSchema" xsi:noNamespaceSchemaLocation="config.xsd">
        <servers>
            <!-- Local mode for development -->
            <server ip="192.168.43.80" port="2280" http-port="2281" />
            <!-- If under production environment, put actual server address as list. -->
            <!-- 
                <server ip="192.168.7.71" port="2280" /> 
                <server ip="192.168.7.72" port="2280" /> 
            -->
        </servers>
    </config>

    这里client要配置server的地址,port是上传文件到server的TCP端口是2280,2281是cat页面访问的端口,这里要写本地机器的IP地址,不要写成127.0.0.1

    <!-- ip:部署CAT应用的服务器IP

    port:CAT服务端接收客户端数据的端口(不允许更改)

    http-port:CAT应用部署到的容器的端口(tomcat的端口)

    -->

    <server ip="10.10.10.121" port="2280" http-port="8080" />

    浏览器打开http://localhost:2281/cat 可以看到cat监控的界面 

    只有客户端配置了这个配置文件,客户端才能正确连接到CAT服务器端

    接下来配置datasources.xml,这里是cat访问mysql数据的配置

    <?xml version="1.0" encoding="utf-8"?>

    <data-sources>
    <data-source id="cat">
    <maximum-pool-size>3</maximum-pool-size>
    <connection-timeout>1s</connection-timeout>
    <idle-timeout>10m</idle-timeout>
    <statement-cache-size>1000</statement-cache-size>
    <properties>
    <driver>com.mysql.jdbc.Driver</driver>
    <url><![CDATA[jdbc:mysql://192.168.43.80:3306/cat]]></url>
    <user>root</user>
    <password>123456</password>
    <connectionProperties><![CDATA[useUnicode=true&autoReconnect=true]]></connectionProperties>
    </properties>
    </data-source>
    <data-source id="app">
    <maximum-pool-size>3</maximum-pool-size>
    <connection-timeout>1s</connection-timeout>
    <idle-timeout>10m</idle-timeout>
    <statement-cache-size>1000</statement-cache-size>
    <properties>
    <driver>com.mysql.jdbc.Driver</driver>
    <url><![CDATA[jdbc:mysql://192.168.43.80:3306/cat]]></url>
    <user>root</user>
    <password>123456</password>
    <connectionProperties><![CDATA[useUnicode=true&autoReconnect=true]]></connectionProperties>
    </properties>
    </data-source>
    </data-sources>

     接下来我们要配置server.xml

    <?xml version="1.0" encoding="utf-8"?>

    <!-- local-mode:是否为本地开发模式。建议在开发环境以及生产环境都设置为false

    hdfs-machine:是否启用HDFS存储

    job-machine:是否为报告工作机(开启生成汇总报告和统计报告,只需要一台服务机开启此功能)

    alert-machine:是否为报警机(开启各类报警监听, 只需要一台服务机开启此功能)

    -->

    控制台 - 提供给业务人员进行数据查看【默认所有的cat节点都可以作为控制台,不可配置】
    消费机 - 实时接收业务数据,实时处理,提供实时分析报表【默认所有的cat节点都可以作为消费机,不可配置】
    任务机(job-machine) - 做一些离线的任务,合并天、周、月等报表 【可以配置】
    告警端(alert-machine) - 启动告警线程,进行规则匹配,发送告警(目前仅支持单点部署)【可以配置】

    local-mode: 建议在开发环境以及生产环境时,都设置为false
    hdfs-machine: 定义是否启用 HDFS 存储方式,默认为 false
    job-machine: 定义当前服务是否为报告工作机(开启生成汇总报告和统计报告的任务,只需要一台服务机开启此功能),默认为 false
    alert-machine: 定义当前服务是否为报警机(开启各类报警监听,只需要一台服务机开启此功能),默认为 false;
    storage: 定义数据存储配置信息
    local-report-storage-time: 定义本地报告文件存放时长,单位为(天)
    local-logivew-storage-time: 定义本地日志文件存放时长,单位为(天)
    local-base-dir: 定义本地数据存储目录,建议直接使用/data/appdatas/cat/bucket目录
    hdfs: 定义HDFS配置信息
    server-uri: 定义HDFS服务地址
    console: 定义服务控制台信息
    remote-servers: 定义HTTP服务列表,(远程监听端同步更新服务端信息即取此值)

    <?xml version="1.0" encoding="utf-8"?>
    
    <!-- Configuration for development environment-->
    <config local-mode="false" hdfs-machine="false" job-machine="true" alert-machine="true">
        
        <storage  local-base-dir="/data/appdatas/cat/bucket/" max-hdfs-storage-time="15" local-report-storage-time="7" local-logivew-storage-time="7">
        
        </storage>
        
        <console default-domain="Cat" show-cat-domain="true">
            <remote-servers>192.168.43.80:2281</remote-servers>        
        </console>
            
    </config>

    这里本地开发模式,不适用hdfs存储,启动聚合,启动报警,远程页面来访问server进行数据查询的短路时2281

    接下来我们要将打包出来的cat.war包部署到tomcat中

    这里因为上面配置的页面访问http-port端口是2281,这里tomcat中server.xml的端口地址也要配置为2281

    接下来,我们把cat.war包部署到webapps目录下

    接下来我们启动tomcat,我们在D:dataapplogscat下可以看到启动对于的目录,这个日志目录一定要保证具有可读写的权限

    启动tomcat的时候要查看日志里面没有错误的信息

    [08-26 15:28:28.598] [INFO] [ServerConfigManager] Loading configuration file(D:dataappdatascatserver.xml) ...
    [08-26 15:28:28.696] [INFO] [ServerConfigManager] CAT server is running with hdfs,false
    [08-26 15:28:31.585] [INFO] [ServerConfigManager] CAT server is running with alert,false
    [08-26 15:28:31.585] [INFO] [ServerConfigManager] CAT server is running with job,true
    [08-26 15:28:31.590] [INFO] [ServerConfigManager] <?xml version="1.0" encoding="utf-8"?>
    <config local-mode="false" job-machine="true" alert-machine="false" hdfs-machine="false" send-machine="true">
       <storage local-base-dir="/data/appdatas/cat/bucket/" upload-thread="5" max-hdfs-storage-time="15" local-report-storage-time="7" local-logivew-storage-time="7">
       </storage>
       <consumer>
          <long-config default-url-threshold="1000" default-sql-threshold="100" default-service-threshold="50">
          </long-config>
       </consumer>
       <console default-domain="Cat" show-cat-domain="true">
          <remote-servers>192.168.43.80:2281</remote-servers>
       </console>
    </config>
    
    [08-26 15:28:33.287] [INFO] [TcpSocketReceiver] start netty server!
    [08-26 15:28:33.295] [INFO] [CatClientModule] Current working directory is D:apache-tomcat-8.5.45in
    [08-26 15:28:33.340] [INFO] [DefaultClientConfigManager] Global config file(dataappdatascatclient.xml) found.
    [08-26 15:28:33.344] [INFO] [DefaultClientConfigManager] Find domain name cat from app.properties.
    [08-26 15:28:33.381] [INFO] [DefaultTransportManager] Remote CAT servers: [/127.0.0.1:2280]
    [08-26 15:28:35.683] [INFO] [ChannelManager] Connected to CAT server at /127.0.0.1:2280
    [08-26 15:28:35.686] [INFO] [ChannelManager] success when init CAT server, new active holderactive future :/127.0.0.1:2280 index:0 ip:127.0.0.1 server config:null
    [08-26 15:28:35.690] [INFO] [CatThreadListener] Thread group(cat) created.
    [08-26 15:28:35.692] [INFO] [TcpSocketSender] Starting thread(cat-TcpSocketSender) ...
    [08-26 15:28:35.693] [INFO] [TcpSocketSender] Starting thread(cat-TcpSocketSender-ChannelManager) ...
    [08-26 15:28:35.695] [INFO] [TcpSocketSender] Starting thread(cat-merge-atomic-task) ...
    [08-26 15:28:35.737] [INFO] [CatClientModule] Starting thread(cat-StatusUpdateTask) ...
    [08-26 15:28:36.008] [INFO] [DefaultDataSourceProvider] Loading data sources from dataappdatascatdatasources.xml ...
    [08-26 15:28:36.707] [INFO] [JdbcDataSource] Connecting to JDBC data source(cat) with properties(driver=com.mysql.jdbc.Driver, url=jdbc:mysql://127.0.0.1:3306/cat?useUnicode=true&autoReconnect=true, user=root) ...
    [08-26 15:28:37.729] [INFO] [JdbcDataSource] Connected to JDBC data source(cat).
    [08-26 15:28:37.932] [INFO] [ServerFilterConfigManager] Starting thread(cat-Server-Filter-Config-Reload) ...
    [08-26 15:28:38.290] [INFO] [LocalMessageBucketManager] Starting thread(cat-LocalMessageBucketManager-BlockDumper) ...
    [08-26 15:28:38.294] [INFO] [LocalMessageBucketManager] Starting thread(cat-LocalMessageBucketManager-OldMessageMover) ...
    [08-26 15:28:38.304] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-0) ...
    [08-26 15:28:38.304] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-4) ...
    [08-26 15:28:38.305] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-8) ...
    [08-26 15:28:38.306] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-12) ...
    [08-26 15:28:38.306] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-3) ...
    [08-26 15:28:38.306] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-7) ...
    [08-26 15:28:38.307] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-11) ...
    [08-26 15:28:38.307] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-16) ...
    [08-26 15:28:38.308] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-15) ...
    [08-26 15:28:38.308] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-1) ...
    [08-26 15:28:38.308] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-5) ...
    [08-26 15:28:38.309] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-9) ...
    [08-26 15:28:38.309] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-17) ...
    [08-26 15:28:38.310] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-13) ...
    [08-26 15:28:38.310] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-19) ...
    [08-26 15:28:38.311] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-2) ...
    [08-26 15:28:38.311] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-6) ...
    [08-26 15:28:38.312] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-10) ...
    [08-26 15:28:38.312] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-14) ...
    [08-26 15:28:38.313] [INFO] [LocalMessageBucketManager] Starting thread(cat-Message-Gzip-18) ...
    [08-26 15:28:38.845] [INFO] [Period] Starting 10 tasks in period [2019-08-26 15:00:00, 2019-08-26 15:59:59]
    [08-26 15:28:38.846] [INFO] [CatThreadListener] Thread group(Cat-RealtimeConsumer) created.
    [08-26 15:28:38.848] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-ProblemAnalyzer-15-0) ...
    [08-26 15:28:38.850] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-TopAnalyzer-15-0) ...
    [08-26 15:28:38.850] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-MetricAnalyzer-15-0) ...
    [08-26 15:28:38.851] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-HeartbeatAnalyzer-15-0) ...
    [08-26 15:28:38.852] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-ProblemAnalyzer-15-1) ...
    [08-26 15:28:38.860] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-CrossAnalyzer-15-0) ...
    [08-26 15:28:38.860] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-StorageAnalyzer-15-0) ...
    [08-26 15:28:38.861] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-TransactionAnalyzer-15-0) ...
    [08-26 15:28:38.861] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-CrossAnalyzer-15-1) ...
    [08-26 15:28:38.861] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-DumpAnalyzer-15-1) ...
    [08-26 15:28:38.862] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-StateAnalyzer-15-0) ...
    [08-26 15:28:38.862] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-EventAnalyzer-15-1) ...
    [08-26 15:28:38.862] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-DumpAnalyzer-15-0) ...
    [08-26 15:28:38.862] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-EventAnalyzer-15-0) ...
    [08-26 15:28:38.862] [INFO] [Period] Starting thread(Cat-RealtimeConsumer-TransactionAnalyzer-15-1) ...
    [08-26 15:28:38.863] [INFO] [RealtimeConsumer] Starting thread(cat-RealtimeConsumer-PeriodManager) ...
    [08-26 15:28:38.961] [INFO] [CatHomeModule] Starting thread(cat-Config-Reload) ...
    [08-26 15:28:39.891] [INFO] [CatHomeModule] Starting thread(cat-Task-Consumer) ...
    [08-26 15:28:39.897] [INFO] [MVC] MVC is starting at /cat
    [08-26 15:28:40.131] [INFO] [CatThreadListener] Thread pool(Cat-ModelService) created.
    [08-26 15:28:41.129] [INFO] [TopologyGraphManager] Starting thread(cat-TopologyGraphReload) ...
    [08-26 15:28:41.358] [INFO] [UrlPatternConfigManager] Starting thread(cat-UrlPattern-Config-Reload) ...
    [08-26 15:28:41.960] [INFO] [AggregationConfigManager] Starting thread(cat-Aggreation-Config-Reload) ...
    [08-26 15:28:42.434] [INFO] [MVC] MVC started at /cat
    [08-26 15:29:00.515] [INFO] [Worker] Starting thread(Cat-ModelService-0) ...
    [08-26 15:29:45.770] [WARN] [TcpSocketSender] Netty write buffer is full! Attempts: 1
    [08-26 15:30:00.501] [INFO] [Worker] Starting thread(Cat-ModelService-1) ...
    [08-26 15:30:05.745] [INFO] [TcpSocketSender] success when init CAT server, new active holderactive future :/127.0.0.1:2280 index:0 ip:127.0.0.1 server config:127.0.0.1:2280;
    [08-26 15:31:00.502] [INFO] [Worker] Starting thread(Cat-ModelService-2) ...
    [08-26 15:32:00.502] [INFO] [Worker] Starting thread(Cat-ModelService-3) ...
    [08-26 15:33:00.503] [INFO] [Worker] Starting thread(Cat-ModelService-4) ...

    上面没有错误的信息

    我们在页面上可以看到访问的信息为:

    这样cat就已经启动成功了,接下来我们要进行全局路由的设置

    点击页面上的配置按钮

    需要进行登录,登录的密码默认是用户名是catadmin,密码是admin

     点击客户端路由

    这里客户端能够访问的server地址只有一台,我们只需要将127.0.0.1替换成当前机器的IP地址就可以了

    <?xml version="1.0" encoding="utf-8"?>
    <router-config backup-server="192.168.43.80" backup-server-port="2280">
       <default-server id="192.168.43.80" weight="1.0" port="2280" enable="true"/>
    </router-config>

    点击cat'上面的state

    如果看到

    这才说明我们的cat服务器安装成功。

  • 相关阅读:
    对post提交数据Content-Type的理解
    预加载的实现方式
    ajax跨域简单请求与复杂请求
    web前端图片模糊到清晰的实现过程
    vue子组件调用父组件的方法
    vue子组件如何向父组件传值
    如何实现下拉弹出框渐渐弹出
    DynamicObject扩展--实现JSON和DynamicObject的序列化与反序列化
    解决 Bash On Windows 下载慢或无法下载的问题
    NPOI扩展--判断指定单元格是否为合并单元格和输出该单元格的行列跨度(维度)
  • 原文地址:https://www.cnblogs.com/kebibuluan/p/11411907.html
Copyright © 2011-2022 走看看