以下是关于使用 vmagent替换prometheus 的简单使用，可以实现push 模式的metrics 处理
环境准备
- docker-compose 文件
version: "3"
services:
vmstorage:
image: victoriametrics/vmstorage
ports:
- 8482:8482
- 8400:8482
- 8401:8482
volumes:
- ./strgdata:/storage
command:
- '--storageDataPath=/storage'
vmagent:
image: victoriametrics/vmagent
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
ports:
- 8429:8429
command:
- -promscrape.config=/etc/prometheus/prometheus.yml
- -remoteWrite.basicAuth.username=dalong-insert-account-1
- -remoteWrite.basicAuth.password=dalong
- -remoteWrite.url=http://vmauth:8427
vmauth:
image: victoriametrics/vmauth
volumes:
- "./config.yaml:/etc/victoriametrics/config.yaml"
command:
- '-auth.config=/etc/victoriametrics/config.yaml'
ports:
- 8427:8427
vminsert:
image: victoriametrics/vminsert
command:
- '--storageNode=vmstorage:8400'
ports:
- 8480:8480
vmselect:
image: victoriametrics/vmselect
command:
- '--storageNode=vmstorage:8401'
ports:
- 8481:8481
grafana:
image: grafana/grafana
ports:
- 3000:3000
- 配置说明
以上demo集成了vmauth（安全处理）可以参考https://www.cnblogs.com/rongfengliang/p/12937022.html
vmauth 配置
users:
- username: "dalong-select-account-1"
password: "dalong"
url_prefix: "http://vmselect:8481/select/1/prometheus"
- username: "dalong-insert-account-1"
password: "dalong"
url_prefix: "http://vminsert:8480/insert/1/prometheus"
vmagent 配置,实际上就是prometheus 的配置移除了remote_wreite
global:
scrape_interval: 1s
evaluation_interval: 1s
scrape_configs:
- job_name: 'prometheus'
static_configs:
- targets: ['prometheus:9090']
- job_name: 'vminsert'
static_configs:
- targets: ['vminsert:8480']
- job_name: 'vmselect'
static_configs:
- targets: ['vmselect:8481']
- job_name: 'vmstorage'
static_configs:
- targets: ['vmstorage:8482']
因为我们启用了安全配置，所以需要添加一些额外的参数 如下：
-promscrape.config=/etc/prometheus/prometheus.yml -remoteWrite.basicAuth.username=dalong-insert-account-1 -remoteWrite.basicAuth.password=dalong -remoteWrite.url=http://vmauth:8427
- vmagent支持的命令
vmagent-20200521-152405-tags-v1.35.6-cluster-0-gdcbdc009f
Usage of /vmagent-prod:
-csvTrimTimestamp duration
Trim timestamps when importing csv data to this duration. Minimum practical duration is 1ms. Higher duration (i.e. 1s) may be used for reducing disk space usage for timestamp data (default 1ms)
-dryRun
Whether to check only config files without running vmagent. The following files are checked: -promscrape.config, -remoteWrite.relabelConfig, -remoteWrite.urlRelabelConfig . See also -promscrape.config.dryRun
-enableTCP6
Whether to enable IPv6 for listening and dialing. By default only IPv4 TCP is used
-envflag.enable
Whether to enable reading flags from environment variables additionally to command line. Command line flag values have priority over values from environment vars. Flags are read only from command line if this flag isn't set
-envflag.prefix string
Prefix for environment variables if -envflag.enable is set
-fs.disableMmap
Whether to use pread() instead of mmap() for reading data files. By default mmap() is used for 64-bit arches and pread() is used for 32-bit arches, since they cannot data files bigger than 2^32 bytes in memory
-graphiteListenAddr string
TCP and UDP address to listen for Graphite plaintext data. Usually :2003 must be set. Doesn't work if empty
-graphiteTrimTimestamp duration
Trim timestamps for Graphite data to this duration. Minimum practical duration is 1s. Higher duration (i.e. 1m) may be used for reducing disk space usage for timestamp data (default 1s)
-http.disableResponseCompression
Disable compression of HTTP responses for saving CPU resources. By default compression is enabled to save network bandwidth
-http.maxGracefulShutdownDuration duration
The maximum duration for graceful shutdown of HTTP server. Highly loaded server may require increased value for graceful shutdown (default 7s)
-http.pathPrefix string
An optional prefix to add to all the paths handled by http server. For example, if '-http.pathPrefix=/foo/bar' is set, then all the http requests will be handled on '/foo/bar/*' paths. This may be useful for proxied requests. See https://www.robustperception.io/using-external-urls-and-proxies-with-prometheus
-http.shutdownDelay duration
Optional delay before http server shutdown. During this dealy the servier returns non-OK responses from /health page, so load balancers can route new requests to other servers
-httpListenAddr string
TCP address to listen for http connections. Set this flag to empty value in order to disable listening on any port. This mode may be useful for running multiple vmagent instances on the same server. Note that /targets and /metrics pages aren't available if -httpListenAddr='' (default ":8429")
-import.maxLineLen int
The maximum length in bytes of a single line accepted by /api/v1/import (default 104857600)
-influxListenAddr string
TCP and UDP address to listen for Influx line protocol data. Usually :8189 must be set. Doesn't work if empty
-influxMeasurementFieldSeparator string
Separator for '{measurement}{separator}{field_name}' metric name when inserted via Influx line protocol (default "_")
-influxSkipSingleField
Uses '{measurement}' instead of '{measurement}{separator}{field_name}' for metic name if Influx line contains only a single field
-influxTrimTimestamp duration
Trim timestamps for Influx line protocol data to this duration. Minimum practical duration is 1ms. Higher duration (i.e. 1s) may be used for reducing disk space usage for timestamp data (default 1ms)
-insert.maxQueueDuration duration
The maximum duration for waiting in the queue for insert requests due to -maxConcurrentInserts (default 1m0s)
-loggerFormat string
Format for logs. Possible values: default, json (default "default")
-loggerLevel string
Minimum level of errors to log. Possible values: INFO, WARN, ERROR, FATAL, PANIC (default "INFO")
-loggerOutput string
Output for the logs. Supported values: stderr, stdout (default "stderr")
-maxConcurrentInserts int
The maximum number of concurrent inserts. Default value should work for most cases, since it minimizes the overhead for concurrent inserts. This option is tigthly coupled with -insert.maxQueueDuration (default 16)
-maxInsertRequestSize int
The maximum size in bytes of a single Prometheus remote_write API request (default 33554432)
-memory.allowedPercent float
Allowed percent of system memory VictoriaMetrics caches may occupy. Too low value may increase cache miss rate, which usually results in higher CPU and disk IO usage. Too high value may evict too much data from OS page cache, which will result in higher disk IO usage (default 60)
-opentsdbHTTPListenAddr string
TCP address to listen for OpentTSDB HTTP put requests. Usually :4242 must be set. Doesn't work if empty
-opentsdbListenAddr string
TCP and UDP address to listen for OpentTSDB metrics. Telnet put messages and HTTP /api/put messages are simultaneously served on TCP port. Usually :4242 must be set. Doesn't work if empty
-opentsdbTrimTimestamp duration
Trim timestamps for OpenTSDB 'telnet put' data to this duration. Minimum practical duration is 1s. Higher duration (i.e. 1m) may be used for reducing disk space usage for timestamp data (default 1s)
-opentsdbhttp.maxInsertRequestSize int
The maximum size of OpenTSDB HTTP put request (default 33554432)
-opentsdbhttpTrimTimestamp duration
Trim timestamps for OpenTSDB HTTP data to this duration. Minimum practical duration is 1ms. Higher duration (i.e. 1s) may be used for reducing disk space usage for timestamp data (default 1ms)
-promscrape.config string
Optional path to Prometheus config file with 'scrape_configs' section containing targets to scrape. See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config for details
-promscrape.config.dryRun
Checks -promscrape.config file for errors and unsupported fields and then exits. Returns non-zero exit code on parsing errors and emits these errors to stderr. Pass -loggerLevel=ERROR if you don't need to see info messages in the output
-promscrape.config.strictParse
Whether to allow only supported fields in '-promscrape.config'. This option may be used for errors detection in '-promscrape.config' file
-promscrape.configCheckInterval duration
Interval for checking for changes in '-promscrape.config' file. By default the checking is disabled. Send SIGHUP signal in order to force config check for changes
-promscrape.consulSDCheckInterval consul_sd_configs
Interval for checking for changes in consul. This works only if consul_sd_configs is configured in '-promscrape.config' file. See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#consul_sd_config for details (default 30s)
-promscrape.disableCompression
Whether to disable sending 'Accept-Encoding: gzip' request headers to scrape targets. This may reduce CPU usage on scrape targets at the cost of higher network bandwidth utilization
-promscrape.discovery.concurrency int
The maximum number of concurrent requests to Prometheus autodiscovery API (Consul, Kubernetes, etc.) (default 500)
-promscrape.discovery.concurrentWaitTime duration
The maximum duration for waiting to perform API requests if more than -promscrape.discovery.concurrency requests are simultaneously performed (default 1m0s)
-promscrape.dnsSDCheckInterval dns_sd_configs
Interval for checking for changes in dns. This works only if dns_sd_configs is configured in '-promscrape.config' file. See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#dns_sd_config for details (default 30s)
-promscrape.ec2SDCheckInterval ec2_sd_configs
Interval for checking for changes in ec2. This works only if ec2_sd_configs is configured in '-promscrape.config' file. See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#ec2_sd_config for details (default 1m0s)
-promscrape.fileSDCheckInterval duration
Interval for checking for changes in 'file_sd_config'. See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#file_sd_config for details (default 30s)
-promscrape.gceSDCheckInterval gce_sd_configs
Interval for checking for changes in gce. This works only if gce_sd_configs is configured in '-promscrape.config' file. See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#gce_sd_config for details (default 1m0s)
-promscrape.kubernetesSDCheckInterval kubernetes_sd_configs
Interval for checking for changes in Kubernetes API server. This works only if kubernetes_sd_configs is configured in '-promscrape.config' file. See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#kubernetes_sd_config for details (default 30s)
-promscrape.maxScrapeSize int
The maximum size of scrape response in bytes to process from Prometheus targets. Bigger responses are rejected (default 16777216)
-promscrape.suppressScrapeErrors
Whether to suppress scrape errors logging. The last error for each target is always available at '/targets' page even if scrape errors logging is suppressed
-remoteWrite.basicAuth.password value
Optional basic auth password to use for -remoteWrite.url. If multiple args are set, then they are applied independently for the corresponding -remoteWrite.url
-remoteWrite.basicAuth.username value
Optional basic auth username to use for -remoteWrite.url. If multiple args are set, then they are applied independently for the corresponding -remoteWrite.url
-remoteWrite.bearerToken value
Optional bearer auth token to use for -remoteWrite.url. If multiple args are set, then they are applied independently for the corresponding -remoteWrite.url
-remoteWrite.flushInterval duration
Interval for flushing the data to remote storage. Higher value reduces network bandwidth usage at the cost of delayed push of scraped data to remote storage. Minimum supported interval is 1 second (default 1s)
-remoteWrite.label value
Optional label in the form 'name=value' to add to all the metrics before sending them to -remoteWrite.url. Pass multiple -remoteWrite.label flags in order to add multiple flags to metrics before sending them to remote storage
-remoteWrite.maxBlockSize int
The maximum size in bytes of unpacked request to send to remote storage. It shouldn't exceed -maxInsertRequestSize from VictoriaMetrics (default 33554432)
-remoteWrite.maxDiskUsagePerURL int
The maximum file-based buffer size in bytes at -remoteWrite.tmpDataPath for each -remoteWrite.url. When buffer size reaches the configured maximum, then old data is dropped when adding new data to the buffer. Buffered data is stored in ~500MB chunks, so the minimum practical value for this flag is 500000000. Disk usage is unlimited if the value is set to 0
-remoteWrite.queues int
The number of concurrent queues to each -remoteWrite.url. Set more queues if a single queue isn't enough for sending high volume of collected data to remote storage (default 1)
-remoteWrite.relabelConfig string
Optional path to file with relabel_config entries. These entries are applied to all the metrics before sending them to -remoteWrite.url. See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config for details
-remoteWrite.sendTimeout duration
Timeout for sending a single block of data to -remoteWrite.url (default 1m0s)
-remoteWrite.showURL
Whether to show -remoteWrite.url in the exported metrics. It is hidden by default, since it can contain sensistive auth info
-remoteWrite.tlsCAFile value
Optional path to TLS CA file to use for verifying connections to -remoteWrite.url. By default system CA is used. If multiple args are set, then they are applied independently for the corresponding -remoteWrite.url
-remoteWrite.tlsCertFile value
Optional path to client-side TLS certificate file to use when connecting to -remoteWrite.url. If multiple args are set, then they are applied independently for the corresponding -remoteWrite.url
-remoteWrite.tlsInsecureSkipVerify
Whether to skip tls verification when connecting to -remoteWrite.url
-remoteWrite.tlsKeyFile value
Optional path to client-side TLS certificate key to use when connecting to -remoteWrite.url. If multiple args are set, then they are applied independently for the corresponding -remoteWrite.url
-remoteWrite.tlsServerName value
Optional TLS server name to use for connections to -remoteWrite.url. By default the server name from -remoteWrite.url is used. If multiple args are set, then they are applied independently for the corresponding -remoteWrite.url
-remoteWrite.tmpDataPath string
Path to directory where temporary data for remote write component is stored (default "vmagent-remotewrite-data")
-remoteWrite.url value
Remote storage URL to write data to. It must support Prometheus remote_write API. It is recommended using VictoriaMetrics as remote storage. Example url: http://<victoriametrics-host>:8428/api/v1/write . Pass multiple -remoteWrite.url flags in order to write data concurrently to multiple remote storage systems
-remoteWrite.urlRelabelConfig value
Optional path to relabel config for the corresponding -remoteWrite.url
-version
Show VictoriaMetrics version
- 启动
docker-compose up -d
集成使用
- 查看target
curl -i http://localhost:8429/targets
效果
- 集成grafana
参考配置 
- 数据查询
说明
以上是一个简单的集成使用，我们可以基于vmagent 实现pull 模式到push 模式的转变，同时对于一些因为特殊原因可能pull 的是很有用的
同时我们也可以使用vmagent 替换prometheus，而且vmagent 支持多种模式的数据处理，还是很强大的
参考资料
https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/vmagent
https://www.cnblogs.com/rongfengliang/p/12937022.html