Prometheus+Grafana实用案例

1、订单业务

10分钟内下单失败率

sum(rate(requests_error_total{application="springboot-actuator-prometheus-test"}[10m])) 
/ sum(rate(order_request_count_total{application="springboot-actuator-prometheus-test"}[10m])) * 100

然后是统计订单总金额：

order_amount_sum_sum{application="springboot-actuator-prometheus-test"}

最终结果

https://segmentfault.com/a/1190000021639286

1、创建 Prometheus 监控管理类PrometheusCustomMonitor

这里面我们自定义了三个metrics：

• requests_error_total: 下单失败次数
• order_request_count：下单总次数
• order_amount_sum：下单金额统计

@Component
public class PrometheusCustomMonitor {

    /**
     * 记录请求出错次数
     */
    private Counter requestErrorCount;

    /**
     * 订单发起次数
     */
    private Counter orderCount;

    /**
     * 金额统计
     */
    private DistributionSummary amountSum;

    private final MeterRegistry registry;

    @Autowired
    public PrometheusCustomMonitor(MeterRegistry registry) {
        this.registry = registry;
    }

    @PostConstruct
    private void init() {
        requestErrorCount = registry.counter("requests_error_total", "status", "error");
        orderCount = registry.counter("order_request_count", "order", "test-svc");
        amountSum = registry.summary("order_amount_sum", "orderAmount", "test-svc");
    }

    public Counter getRequestErrorCount() {
        return requestErrorCount;
    }

    public Counter getOrderCount() {
        return orderCount;
    }

    public DistributionSummary getAmountSum() {
        return amountSum;
    }
}

2、新增/order接口

当 flag="1"时，抛异常，模拟下单失败情况。
在接口中统计order_request_count和order_amount_sum

@RestController
public class TestController {

    @Resource
    private PrometheusCustomMonitor monitor;
    
    //....

    @RequestMapping("/order")
    public String order(@RequestParam(defaultValue = "0") String flag) throws Exception {
        // 统计下单次数
        monitor.getOrderCount().increment();
        if ("1".equals(flag)) {
            throw new Exception("出错啦");
        }
        Random random = new Random();
        int amount = random.nextInt(100);
        // 统计金额
        monitor.getAmountSum().record(amount);
        return "下单成功, 金额: " + amount;
    }
}

PS：实际项目中，采集业务监控数据的时候，建议使用AOP的方式记录，不要侵入业务代码。不要像我Demo中这样写。

3、新增全局异常处理器GlobalExceptionHandler

统计下单失败次数requests_error_total：

@ControllerAdvice
public class GlobalExceptionHandler {

    @Resource
    private PrometheusCustomMonitor monitor;

    @ResponseBody
    @ExceptionHandler(value = Exception.class)
    public String handle(Exception e) {
        monitor.getRequestErrorCount().increment();
        return "error, message: " + e.getMessage();
    }
}

4、测试：

启动项目，访问http://localhost:8080/order和http://localhost:8080/order?flag=1模拟下单成功和失败的情况，然后我们访问http://localhost:8080/actuator/prometheus，可以看到我们自定义指标已经被/prometheus端点暴露出来了：

# HELP requests_error_total  
# TYPE requests_error_total counter
requests_error_total{application="springboot-actuator-prometheus-test",status="error",} 41.0
# HELP order_request_count_total  
# TYPE order_request_count_total counter
order_request_count_total{application="springboot-actuator-prometheus-test",order="test-svc",} 94.0
# HELP order_amount_sum  
# TYPE order_amount_sum summary
order_amount_sum_count{application="springboot-actuator-prometheus-test",orderAmount="test-svc",} 53.0
order_amount_sum_sum{application="springboot-actuator-prometheus-test",orderAmount="test-svc",} 2701.0