sentinel dashboard 持久化规则，监控数据+nacos和mysql

1. Sentinel是阿里巴巴开源的分布式系统的流量防卫组件，Sentinel 把流量作为切入点，从流量控制，熔断降级，系统负载保护等多个维度保护服务的稳定性。
2. 结合目前项目做了sentinel-dashboard数据持久化的改造，参考生产环境使用sentinel。https://github.com/alibaba/Sentinel/wiki/%E5%9C%A8%E7%94%9F%E4%BA%A7%E7%8E%AF%E5%A2%83%E4%B8%AD%E4%BD%BF%E7%94%A8-Sentinel。
3. 克隆代码地址 https://github.com/alibaba/Sentinel.git。选择1.4.0版本（其他版本修改比较大）。
   1. 规则持久化

<dependency>

    <groupId>com.alibaba.csp</groupId>

    <artifactId>sentinel-datasource-nacos</artifactId>

</dependency>

原来的<scope>test</scope>去掉

1. 自定义规则数据持久化

将项目测试下面的代码移动到正式代码里。如

添加com.taobao.csp.sentinel.dashboard.config.NacosConfig的配置

@Bean

public Converter<String, List<SystemRuleEntity>> systemRuleEntityDecoder() {

    return s -> JSON.parseArray(s, SystemRuleEntity.class);

}

@Bean

    public ConfigService nacosConfigService() throws Exception {

        Properties properties = new Properties();

        properties.put("serverAddr",address);

//      命名空间暂时不用，还是使用默认的public，

// 因为客服端sentinel-datasource-nacos的jar包目前只能设置地址，还不能设置命名空间

//       properties.put("namespace","938ed9f5-b24b-4491-a41c-a2560dc42919");



        return ConfigFactory.createConfigService(properties);

    }

添加com.taobao.csp.sentinel.dashboard.config.NacosConfigUtil的配置

主要是一些静态属性添加，目前添加了限流文件和系统规则文件。

1. 持久化限流规则接口重新实现

a)   推送数据

@Component("flowRuleNacosPublisher")

public class FlowRuleNacosPublisher implements DynamicRulePublisher<List<FlowRuleEntity>> {



    @Autowired

    private ConfigService configService;

    @Autowired

    private Converter<List<Map<String,Object>>,String> convert;



    @Override

    public void publish(String app, List<FlowRuleEntity> rules) throws Exception {

        AssertUtil.notEmpty(app, "app name cannot be empty");

        if (rules == null) {

            return;

        }

        List<Map<String,Object>> ruleList= new ArrayList<>();

        rules.forEach(obj->{

            Map<String,Object> map = new HashMap<>();

            map.put("resource",obj.getResource());

            map.put("controlBehavior",obj.getControlBehavior());

            map.put("count",obj.getCount());

            map.put("grade",obj.getGrade());

            map.put("limitApp",obj.getLimitApp());

            map.put("strategy",obj.getStrategy());

            ruleList.add(map);

        });

        configService.publishConfig(app+NacosConfigUtil.FLOW_DATA_ID_POSTFIX,NacosConfigUtil.GROUP_ID,convert.convert(ruleList));

    }

}

b)   从nacos获取数据

@Component("flowRuleNacosProvider")

public class FlowRuleNacosProvider implements DynamicRuleProvider<List<FlowRuleEntity>> {



    @Autowired

    private ConfigService configService;

    @Autowired

    private Converter<String, List<FlowRuleEntity>> converter;



    @Override

    public List<FlowRuleEntity> getRules(String appName) throws Exception {

        String rules = configService.getConfig(appName + NacosConfigUtil.FLOW_DATA_ID_POSTFIX,

            NacosConfigUtil.GROUP_ID, 3000);

        if (StringUtil.isEmpty(rules)) {

            return new ArrayList<>();

        }

        return converter.convert(rules);

    }

}

c)   修改com.taobao.csp.sentinel.dashboard.view.FlowControllerV2

1. 系统规则的修改与上面限流规则修改差多。
2. 系统监控的数据持久化采（采用mysql+jpa）

1)   添mysql和jpa所需要的jar包

<dependency>

    <groupId>mysql</groupId>

    <artifactId>mysql-connector-java</artifactId>

    <version>8.0.11</version>

</dependency>

<dependency>

    <groupId>org.springframework.boot</groupId>

    <artifactId>spring-boot-starter-data-jpa</artifactId>

    <version>2.0.4.RELEASE</version>

</dependency>

2)   application.properties文件的修改

spring.datasource.url=jdbc:mysql://localhost:3307/edu_operation_platform?useSSL=false&useUnicode=true&characterEncoding=UTF8&allowMultiQueries=true

spring.datasource.username=root

spring.datasource.password=Abcd1234

spring.datasource.driver-class-name= com.mysql.jdbc.Driver



# spring data jpa

spring.jpa.hibernate.ddl-auto=none

spring.jpa.hibernate.use-new-id-generator-mappings=false

spring.jpa.database-platform=org.hibernate.dialect.MySQLDialect

spring.jpa.show-sql=true

3)   自定义监控数据存储

com.taobao.csp.sentinel.dashboard.repository.metric;



import com.alibaba.csp.sentinel.util.StringUtil;

import com.taobao.csp.sentinel.dashboard.datasource.entity.MetricEntity;

import com.taobao.csp.sentinel.dashboard.datasource.entity.MetricPO;



import org.springframework.beans.BeanUtils;

import org.springframework.stereotype.Component;

import org.springframework.transaction.annotation.Transactional;

import org.springframework.util.CollectionUtils;





import java.time.Instant;

import java.util.ArrayList;

import java.util.Date;

import java.util.HashMap;

import java.util.List;

import java.util.Map;

import java.util.stream.Collectors;



import javax.persistence.EntityManager;

import javax.persistence.PersistenceContext;

import javax.persistence.Query;



@Transactional

@Component("jpaMetricsRepository")

public class JpaMetricsRepository implements  MetricsRepository<MetricEntity> {

    @PersistenceContext

    private EntityManager em;

    @Override

    public void save(MetricEntity entity) {

        if (entity == null || StringUtil.isBlank(entity.getApp())) {

            return;

        }

        MetricPO metricPO = new MetricPO();

        BeanUtils.copyProperties(entity, metricPO);

        metricPO.setTimestamp(entity.getTimestamp());

        metricPO.setGmtCreate(entity.getGmtCreate());

        metricPO.setGmtModified(entity.getGmtModified());

        em.persist(metricPO);



    }



    @Override

    public synchronized void saveAll(Iterable<MetricEntity> metrics) {

        if (metrics == null) {

            return;

        }

        metrics.forEach(this::save);

    }



    @Override

    public List<MetricEntity> queryByAppAndResourceBetween(String app, String resource, long startTime, long endTime) {

        List<MetricEntity> results = new ArrayList<MetricEntity>();

        if (StringUtil.isBlank(app)) {

            return results;

        }



        if (StringUtil.isBlank(resource)) {

            return results;

        }



        StringBuilder hql = new StringBuilder();

        hql.append("FROM MetricPO");

        hql.append(" WHERE app=:app");

        hql.append(" AND resource=:resource");

        hql.append(" AND timestamp>=:startTime");

        hql.append(" AND timestamp<=:endTime");



        Query query = em.createQuery(hql.toString());

        query.setParameter("app", app);

        query.setParameter("resource", resource);

        query.setParameter("startTime", Date.from(Instant.ofEpochMilli(startTime)));

        query.setParameter("endTime", Date.from(Instant.ofEpochMilli(endTime)));



        List<MetricPO> metricPOs = query.getResultList();

        if (CollectionUtils.isEmpty(metricPOs)) {

            return results;

        }



        for (MetricPO metricPO : metricPOs) {

            MetricEntity metricEntity = new MetricEntity();

            BeanUtils.copyProperties(metricPO, metricEntity);

            results.add(metricEntity);

        }



        return results;

    }



    @Override

    public List<String> listResourcesOfApp(String app) {

        List<String> results = new ArrayList<>();

        if (StringUtil.isBlank(app)) {

            return results;

        }

        StringBuilder hql = new StringBuilder();

        hql.append("FROM MetricPO");

        hql.append(" WHERE app=:app");

        hql.append(" AND timestamp>=:startTime");



        long startTime = System.currentTimeMillis() - 1000 * 60;

        Query query = em.createQuery(hql.toString());

        query.setParameter("app", app);

        query.setParameter("startTime", Date.from(Instant.ofEpochMilli(startTime)));

        List<MetricPO> metricPOs = query.getResultList();

        if (CollectionUtils.isEmpty(metricPOs)) {

            return results;

        }

        List<MetricEntity> metricEntities = new ArrayList<MetricEntity>();

        for (MetricPO metricPO : metricPOs) {

            MetricEntity metricEntity = new MetricEntity();

            BeanUtils.copyProperties(metricPO, metricEntity);

            metricEntities.add(metricEntity);

        }

        Map<String, MetricEntity> resourceCount = new HashMap<>(32);

        for (MetricEntity metricEntity : metricEntities) {

            String resource = metricEntity.getResource();

            if (resourceCount.containsKey(resource)) {

                MetricEntity oldEntity = resourceCount.get(resource);

                oldEntity.addPassQps(metricEntity.getPassQps());

                oldEntity.addRtAndSuccessQps(metricEntity.getRt(), metricEntity.getSuccessQps());

                oldEntity.addBlockQps(metricEntity.getBlockQps());

                oldEntity.addExceptionQps(metricEntity.getExceptionQps());

                oldEntity.addCount(1);

            } else {

                resourceCount.put(resource, MetricEntity.copyOf(metricEntity));

            }

        }

        return resourceCount.entrySet()

            .stream()

            .sorted((o1, o2) -> {

                MetricEntity e1 = o1.getValue();

                MetricEntity e2 = o2.getValue();

                int t = e2.getBlockQps().compareTo(e1.getBlockQps());

                if (t != 0) {

                    return t;

                }

                return e2.getPassQps().compareTo(e1.getPassQps());

            })

            .map(Map.Entry::getKey)

            .collect(Collectors.toList());

    }



}

4)   修改com.taobao.csp.sentinel.dashboard.metric. MetricFetcher类对监控数据吃久化的实现类的注册：

修改com.taobao.csp.sentinel.dashboard.view. MetricController:

添加实体类：

@Entity

@Table(name = "sentinel_metric")

public class MetricPO implements Serializable {



    private static final long serialVersionUID = 7200023615444172715L;



    /**id，主键*/

    @Id

    @GeneratedValue

    @Column(name = "id")

    private Long id;



    /**创建时间*/

    @Column(name = "gmt_create")

    private Date gmtCreate;



    /**修改时间*/

    @Column(name = "gmt_modified")

    private Date gmtModified;



    /**应用名称*/

    @Column(name = "app")

    private String app;



    /**统计时间*/

    @Column(name = "timestamp")

    private Date timestamp;



    /**资源名称*/

    @Column(name = "resource")

    private String resource;



    /**通过qps*/

    @Column(name = "pass_qps")

    private Long passQps;



    /**成功qps*/

    @Column(name = "success_qps")

    private Long successQps;



    /**限流qps*/

    @Column(name = "block_qps")

    private Long blockQps;



    /**发送异常的次数*/

    @Column(name = "exception_qps")

    private Long exceptionQps;



    /**所有successQps的rt的和*/

    @Column(name = "rt")

    private Double rt;



    /**本次聚合的总条数*/

    @Column(name = "_count")

    private Integer count;



    /**资源的hashCode*/

    @Column(name = "resource_code")

    private Integer resourceCode;



    public static long getSerialVersionUID() {

        return serialVersionUID;

    }



    public Long getId() {

        return id;

    }



    public void setId(Long id) {

        this.id = id;

    }



    public Date getGmtCreate() {

        return gmtCreate;

    }



    public void setGmtCreate(Date gmtCreate) {

        this.gmtCreate = gmtCreate;

    }



    public Date getGmtModified() {

        return gmtModified;

    }



    public void setGmtModified(Date gmtModified) {

        this.gmtModified = gmtModified;

    }



    public String getApp() {

        return app;

    }



    public void setApp(String app) {

        this.app = app;

    }



    public Date getTimestamp() {

        return timestamp;

    }



    public void setTimestamp(Date timestamp) {

        this.timestamp = timestamp;

    }



    public String getResource() {

        return resource;

    }



    public void setResource(String resource) {

        this.resource = resource;

    }



    public Long getPassQps() {

        return passQps;

    }



    public void setPassQps(Long passQps) {

        this.passQps = passQps;

    }



    public Long getSuccessQps() {

        return successQps;

    }



    public void setSuccessQps(Long successQps) {

        this.successQps = successQps;

    }



    public Long getBlockQps() {

        return blockQps;

    }



    public void setBlockQps(Long blockQps) {

        this.blockQps = blockQps;

    }



    public Long getExceptionQps() {

        return exceptionQps;

    }



    public void setExceptionQps(Long exceptionQps) {

        this.exceptionQps = exceptionQps;

    }



    public Double getRt() {

        return rt;

    }



    public void setRt(Double rt) {

        this.rt = rt;

    }



    public Integer getCount() {

        return count;

    }



    public void setCount(Integer count) {

        this.count = count;

    }



    public Integer getResourceCode() {

        return resourceCode;

    }



    public void setResourceCode(Integer resourceCode) {

        this.resourceCode = resourceCode;

    }



// getter setter省略

}

       数据库创建表sql：

-- 创建监控数据表

CREATE TABLE `sentinel_metric` (

  `id` INT NOT NULL AUTO_INCREMENT COMMENT 'id，主键',

  `gmt_create` DATETIME COMMENT '创建时间',

  `gmt_modified` DATETIME COMMENT '修改时间',

  `app` VARCHAR(100) COMMENT '应用名称',

  `timestamp` DATETIME COMMENT '统计时间',

  `resource` VARCHAR(500) COMMENT '资源名称',

  `pass_qps` INT COMMENT '通过qps',

  `success_qps` INT COMMENT '成功qps',

  `block_qps` INT COMMENT '限流qps',

  `exception_qps` INT COMMENT '发送异常的次数',

  `rt` DOUBLE COMMENT '所有successQps的rt的和',

  `_count` INT COMMENT '本次聚合的总条数',

  `resource_code` INT COMMENT '资源的hashCode',

  INDEX app_idx(`app`) USING BTREE,

  INDEX resource_idx(`resource`) USING BTREE,

  INDEX timestamp_idx(`timestamp`) USING BTREE,

  PRIMARY KEY (`id`)

) ENGINE=INNODB DEFAULT CHARSET=utf8;

1. 客服端改造
   1. 引入nacos对数据源支持的jar
   2. 重新实现InitFunc接口。

<dependency>

    <groupId>com.alibaba.csp</groupId>

    <artifactId>sentinel-datasource-nacos</artifactId>

    <version>1.4.0</version>

</dependency>

@Component

public class NacosInit implements InitFunc {

    private static String address;



    @Value("${nacos.address}")

    public void setAddress(String address) {

        this.address = address;

    }



    @Override

    public void init() throws Exception {

        Converter<String, List<FlowRule>> flowParser = source -> JSON.parseObject(source,new TypeReference<List<FlowRule>>() {});

        ReadableDataSource<String, List<FlowRule>> flowSource = new NacosDataSource<>(address, NacosConfigUtil.GROUPID, NacosConfigUtil.FLOW_RULE, flowParser);

        FlowRuleManager.register2Property(flowSource.getProperty());



        Converter<String, List<SystemRule>> systemParser = source -> JSON.parseObject(source,new TypeReference<List<SystemRule>>() {});

        ReadableDataSource<String, List<SystemRule>> systemSource = new NacosDataSource<>(address, NacosConfigUtil.SYSTEM_RULE, NacosConfigUtil.GROUPID, systemParser);

        SystemRuleManager.register2Property(systemSource.getProperty());

    }

}

配置静态属性：

public final class NacosConfigUtil {

    public NacosConfigUtil() {

    }



    public static final  String GROUPID="hbd_rule";

    public static final  String FLOW_RULE="auth-server_flow_rule.yml";

    public static final  String SYSTEM_RULE="auth-server_system_rule.yml";

}

注入新实现的类

创建一个叫com.alibaba.csp.sentinel.init.InitFunc 的文件，内容是你实现InitFunc类的路径，如com.hanboard.educloud.auth.config.NacosInit 放在resources/META-INF/services

打包启动项目，就行了......

项目github地址：https://github.com/chenshijun007/sentinel-nacos-mysql.git