在Marathon 上部署 cAdvisor + InfluxDB + Grafana Docker监控

关于 Docker 容器的监控，google cAdvisor 是个很好的工具，但是它默认只显示实时数据，不储存历史数据。为了存储和显示历史数据、自定义展示图，可以把将cAdvisor与InfluxDB、Grafana 集成起来。

需要的镜像：

tutum/influxdb

google/cadvisor

grafana/grafana

安装：有8台mesos-slave,都需要运行一个cAdvisor docker实例对docker进行监控，然后需要运行一个InfluxDB docker实例存储cAdvisor 产生的数据，然后运行一个Grafana docker实例将InfluxDB数据库里面的数据取出来展示到图上面。

部署 InfluxDB

• InfluxDB只需要一个实例；
• UI 通过 marathon-lb 的虚拟主机发布；
• 数据端口 8086 通过 servicePort 发布到 marathon-lb所在的slaves;
• servicePort需要设置为固定值，比如：28086，以便于cAdvisor和Grafana连接;
• 数据目录 /data 映射到 nfs共享目录；

{
  "id": "/influxdb",
  "cmd": null,
  "cpus": 0.3,
  "mem": 1024,
  "disk": 0,
  "instances": 1,
  "acceptedResourceRoles": [],
  "container": {
    "type": "DOCKER",
    "volumes": [
      {
        "containerPath": "/etc/localtime",
        "hostPath": "/etc/localtime",
        "mode": "RO"
      },
      {
        "containerPath": "/data",
        "hostPath": "/home/nfs/InfluxDB",
        "mode": "RW"
      }
    ],
    "docker": {
      "image": "10.80.163.110:5000/influxdb",
      "network": "BRIDGE",
      "portMappings": [
        {
          "containerPort": 8083,
          "hostPort": 0,
          "servicePort": 10010,
          "protocol": "tcp",
          "labels": {}
        },
        {
          "containerPort": 8086,
          "hostPort": 0,
          "servicePort": 28086,
          "protocol": "tcp",
          "labels": {}
        }
      ],
      "privileged": false,
      "parameters": [],
      "forcePullImage": false
    }
  },
  "labels": {
    "HAPROXY_0_VHOST": "influxdb.osp.cloud",
    "HAPROXY_GROUP": "external"
  },
  "portDefinitions": [
    {
      "port": 10010,
      "protocol": "tcp",
      "name": "default",
      "labels": {}
    },
    {
      "port": 28086,
      "protocol": "tcp",
      "labels": {}
    }
  ]
}

　　

创建监控数据库

打开 http://influxdb.osp.cloud设置 Host 和 Port 分别为http://influxdb.osp.cloud 和 28086。

 为每个mesos slave创建一个单独的数据库，分别为：cadvisor112, cadvisor113, ...

部署 cAdvisor

• 每个mesos slave都要部署一个实例；
• UI 通过marathon-lb的虚拟主机发布；
• 设置 storage_drive 为 influxdb；

{
  "id": "/cadvisor112",
  "cmd": null,
  "cpus": 0.1,
  "mem": 256,
  "disk": 0,
  "instances": 1,
  "constraints": [
    [
      "hostname",
      "CLUSTER",
      "10.80.163.112"
    ]
  ],
  "acceptedResourceRoles": [
    "*"
  ],
  "container": {
    "type": "DOCKER",
    "volumes": [
      {
        "containerPath": "/etc/localtime",
        "hostPath": "/etc/localtime",
        "mode": "RO"
      },
      {
        "containerPath": "/rootfs",
        "hostPath": "/",
        "mode": "RO"
      },
      {
        "containerPath": "/var/run",
        "hostPath": "/var/run",
        "mode": "RW"
      },
      {
        "containerPath": "/sys",
        "hostPath": "/sys",
        "mode": "RO"
      },
      {
        "containerPath": "/var/lib/docker",
        "hostPath": "/var/lib/docker",
        "mode": "RO"
      },
      {
        "containerPath": "/cgroup",
        "hostPath": "/cgroup",
        "mode": "RO"
      }
    ],
    "docker": {
      "image": "10.80.163.110:5000/cadvisor",
      "network": "BRIDGE",
      "portMappings": [
        {
          "containerPort": 8080,
          "hostPort": 0,
          "servicePort": 10011,
          "protocol": "tcp",
          "labels": {}
        }
      ],
      "privileged": false,
      "parameters": [],
      "forcePullImage": false
    }
  },
  "labels": {
    "HAPROXY_0_VHOST": "cadvisor112.zyyt.osp.cloud",
    "HAPROXY_GROUP": "external"
  },
  "portDefinitions": [
    {
      "port": 10011,
      "protocol": "tcp",
      "name": "default",
      "labels": {}
    }
  ],
  "args": [
    "-storage_driver",
    "influxdb",
    "-storage_driver_host",
    "influxdb.osp.cloud:28086",
    "-storage_driver_db",
    "cadvisor112"
  ]
}

　　

查看cAdvisor UI：

http://cadvisor112.zyyt.osp.cloud

部署 Grafana

• 只需要部署一个实例；
• UI 通过 marathon-lb 虚拟主机发布；
• 数据目录 /var/lib/grafana 映射到 nfs 共享存储，以便于持久化存储； 

{
  "id": "/grafana",
  "cmd": null,
  "cpus": 0.3,
  "mem": 512,
  "disk": 0,
  "instances": 1,
  "constraints": [
    [
      "hostname",
      "CLUSTER",
      "10.80.163.112"
    ]
  ],
  "acceptedResourceRoles": [
    "*"
  ],
  "container": {
    "type": "DOCKER",
    "volumes": [
      {
        "containerPath": "/etc/localtime",
        "hostPath": "/etc/localtime",
        "mode": "RO"
      },
      {
        "containerPath": "/var/lib/grafana",
        "hostPath": "/home/nfs/GrafanaData",
        "mode": "RW"
      }
    ],
    "docker": {
      "image": "10.80.163.110:5000/grafana:2.0.2",
      "network": "BRIDGE",
      "portMappings": [
        {
          "containerPort": 3000,
          "hostPort": 0,
          "servicePort": 10012,
          "protocol": "tcp",
          "labels": {}
        }
      ],
      "privileged": false,
      "parameters": [],
      "forcePullImage": false
    }
  },
  "labels": {
    "HAPROXY_0_VHOST": "grafana.zyyt.osp.cloud",
    "HAPROXY_GROUP": "external"
  },
  "portDefinitions": [
    {
      "port": 10012,
      "protocol": "tcp",
      "name": "default",
      "labels": {}
    }
  ]
}

创建数据分析图

打开 Grafana UI：

http://grafana.zyyt.osp.cloud

设置数据源：

• 类型：InfluxDB
• URL：http://influxdb.osp.cloud:28086
• Access：direct
• Database：选择一个slave的数据库，如：cadvisor112

创建graph：

效果图：

参考： http://www.mamicode.com/info-detail-1393800.html