zoukankan      html  css  js  c++  java
  • elasticsearch _cat命令详解 (秒懂+史上最全)

    文章很长,而且持续更新,建议收藏起来,慢慢读! Java 高并发 发烧友社群:疯狂创客圈(总入口) 奉上以下珍贵的学习资源:


    推荐:入大厂 、做架构、大力提升Java 内功 的 精彩博文

    入大厂 、做架构、大力提升Java 内功 必备的精彩博文 2021 秋招涨薪1W + 必备的精彩博文
    1:Redis 分布式锁 (图解-秒懂-史上最全) 2:Zookeeper 分布式锁 (图解-秒懂-史上最全)
    3: Redis与MySQL双写一致性如何保证? (面试必备) 4: 面试必备:秒杀超卖 解决方案 (史上最全)
    5:面试必备之:Reactor模式 6: 10分钟看懂, Java NIO 底层原理
    7:TCP/IP(图解+秒懂+史上最全) 8:Feign原理 (图解)
    9:DNS图解(秒懂 + 史上最全 + 高薪必备) 10:CDN图解(秒懂 + 史上最全 + 高薪必备)
    11: 分布式事务( 图解 + 史上最全 + 吐血推荐 ) 12:seata AT模式实战(图解+秒懂+史上最全)
    13:seata 源码解读(图解+秒懂+史上最全) 14:seata TCC模式实战(图解+秒懂+史上最全)

    Java 面试题 30个专题 , 史上最全 , 面试必刷 阿里、京东、美团... 随意挑、横着走!!!
    1: JVM面试题(史上最强、持续更新、吐血推荐) 2:Java基础面试题(史上最全、持续更新、吐血推荐
    3:架构设计面试题 (史上最全、持续更新、吐血推荐) 4:设计模式面试题 (史上最全、持续更新、吐血推荐)
    17、分布式事务面试题 (史上最全、持续更新、吐血推荐) 一致性协议 (史上最全)
    29、多线程面试题(史上最全) 30、HR面经,过五关斩六将后,小心阴沟翻船!
    9.网络协议面试题(史上最全、持续更新、吐血推荐) 更多专题, 请参见【 疯狂创客圈 高并发 总目录

    SpringCloud 精彩博文
    nacos 实战(史上最全) sentinel (史上最全+入门教程)
    SpringCloud gateway (史上最全) 更多专题, 请参见【 疯狂创客圈 高并发 总目录

    背景:

    下一个视频版本,从架构师视角,尼恩为大家打造高可用、高并发中间件的原理与实操。

    目标:通过视频和博客的方式,为各位潜力架构师,彻底介绍清楚架构师必须掌握的高可用、高并发环境,包括但不限于:

    • 高可用、高并发nginx架构的原理与实操

    • 高可用、高并发mysql架构的原理与实操

    • 高可用、高并发nacos架构的原理与实操

    • 高可用、高并发rocketmq架构的原理与实操

    • 高可用、高并发es架构的原理与实操

    • 高可用、高并发minio架构的原理与实操

    在这里插入图片描述

    why 高可用、高并发中间件的原理与实操:

    • 实际的开发过程中,很多小伙伴聚焦crud开发,环境出了问题,都不能启动。

    • 作为架构师,或者未来想走向高端开发,或者做架构,必须掌握高可用、高并发中间件的原理,掌握其实操。

    本系列博客的具体内容,请参见 Java 高并发 发烧友社群:疯狂创客圈

    ES-cat相关命令汇总

    
    _cat/allocation         #查看单节点的shard分配整体情况
    _cat/shards          #查看各shard的详细情况
    _cat/shards/{index}     #查看指定分片的详细情况
    _cat/master          #查看master节点信息
    _cat/nodes           #查看所有节点信息
    _cat/indices         #查看集群中所有index的详细信息
    _cat/indices/{index}      #查看集群中指定index的详细信息
    _cat/segments        #查看各index的segment详细信息,包括segment名, 所属shard, 内存(磁盘)占用大小, 是否刷盘
    _cat/segments/{index}#查看指定index的segment详细信息
    _cat/count           #查看当前集群的doc数量
    _cat/count/{index}   #查看指定索引的doc数量
    _cat/recovery        #查看集群内每个shard的recovery过程.调整replica。
    _cat/recovery/{index}#查看指定索引shard的recovery过程
    _cat/health          #查看集群当前状态:红、黄、绿
    _cat/pending_tasks   #查看当前集群的pending task
    _cat/aliases         #查看集群中所有alias信息,路由配置等
    _cat/aliases/{alias} #查看指定索引的alias信息
    _cat/thread_pool     #查看集群各节点内部不同类型的threadpool的统计信息,
    _cat/plugins         #查看集群各个节点上的plugin信息
    _cat/fielddata       #查看当前集群各个节点的fielddata内存使用情况
    _cat/fielddata/{fields}     #查看指定field的内存使用情况,里面传field属性对应的值
    _cat/nodeattrs              #查看单节点的自定义属性
    _cat/repositories           #输出集群中注册快照存储库
    _cat/templates              #输出当前正在存在的模板信息
    
    

    Cat 命令公共参数

    • Help 查看帮助
    GET _cat/master?help
    
    id   |   | node id    
    host | h | host name  
    ip   |   | ip address 
    node | n | node name   
    
    • Verbose 显示列名
    GET _cat/master?v
    
    id                     host  ip              node
    AhlyPtZYTta1AVH_7mUSbQ node1 192.168.113.101 master-1
    
    • Headers 只显示特定列
    GET _cat/master?v&h=host,ip,node
    
    host  ip              node
    node1 192.168.113.101 master-1
    
    • 数字格式化
    bytes=kb store.size以kb输出
    
    GET _cat/indices?v&h=index,docs.count,store.size&bytes=kb
    
    index                           docs.count store.size
    .monitoring-es-6-2018.08.25            441        721
    .monitoring-kibana-6-2018.08.25         18        205
    .kibana                                  1          7
    .security-6                              3         19
    
    • Format 输出格式
    ##支持的输出格式有json,test,yaml等
    ##默认以text格式输出
    ##以json格式输出 format=json&pretty
    
    GET _cat/indices?v&h=index,docs.count,store.size&bytes=kb&format=json&pretty
    
    [
      {
        "index": ".monitoring-es-6-2018.08.25",
        "docs.count": "476",
        "store.size": "1114"
      },
      {
        "index": ".monitoring-kibana-6-2018.08.25",
        "docs.count": "21",
        "store.size": "72"
      },
      {
        "index": ".kibana",
        "docs.count": "1",
        "store.size": "7"
      },
      {
        "index": ".security-6",
        "docs.count": "3",
        "store.size": "19"
      }
    ]
    
    ##以yaml格式输出 format=yaml&pretty
    GET _cat/indices?v&h=index,docs.count,store.size&bytes=kb&format=yaml&pretty
    
    ---
    - index: ".monitoring-es-6-2018.08.25"
      docs.count: "509"
      store.size: "979"
    - index: ".monitoring-kibana-6-2018.08.25"
      docs.count: "24"
      store.size: "142"
    - index: ".kibana"
      docs.count: "1"
      store.size: "7"
    - index: ".security-6"
      docs.count: "3"
      store.size: "19"
    
    • Sort 排序
    ##按index升序,docs.count降序
    GET _cat/indices?v&h=index,docs.count,store.size&bytes=kb&format=json&pretty&s=index,docs.count:desc
    
    [
      {
        "index": ".kibana",
        "docs.count": "1",
        "store.size": "7"
      },
      {
        "index": ".monitoring-es-6-2018.08.25",
        "docs.count": "608",
        "store.size": "1068"
      },
      {
        "index": ".monitoring-kibana-6-2018.08.25",
        "docs.count": "33",
        "store.size": "163"
      },
      {
        "index": ".security-6",
        "docs.count": "3",
        "store.size": "19"
      }
    ]
    

    查看集群健康状态

    GET _cat/health?v&h=cluster,status
    
    cluster status
    my-elk  green
    

    查看集群节点和磁盘剩余

    ##集群节点
    GET _cat/nodes?v
    
    ip              heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
    192.168.113.103           60          93   5    0.54    0.53     0.55 di        -      data-2
    192.168.113.104           12          87   7    0.72    0.77     0.94 -         -      client-1
    192.168.113.101           13          90   4    0.35    0.32     0.37 m         *      master-1
    192.168.113.102           46          91 100    2.81    2.84     3.44 di        -      data-1
    
    ##磁盘剩余
    GET _cat/nodes?v&h=ip,node.role,name,disk.avail
    
    ip              node.role name     disk.avail
    192.168.113.102 di        data-1        2.4gb
    192.168.113.103 di        data-2        6.5gb
    192.168.113.104 -         client-1        5gb
    192.168.113.101 m         master-1      6.5gb
    

    查看集群master节点

    GET _cat/master?v
    
    id                     host  ip              node
    AhlyPtZYTta1AVH_7mUSbQ node1 192.168.113.101 master-1
    

    查看分配

    ##查看每个数据节点上的分片数(shards),以及每个数据节点磁盘剩余
    GET _cat/allocation?v
    
    shards disk.indices disk.used disk.avail disk.total disk.percent host  ip              node
         4        8.2mb      14gb      3.2gb     17.2gb           81 node2 192.168.113.102 data-1
         4        5.7mb     9.9gb      7.3gb     17.2gb           57 node3 192.168.113.103 data-2
    

    查看被挂起任务

    GET _cat/pending_tasks?v
    
    insertOrder timeInQueue priority source
    

    查看每个节点正在运行的插件

    GET _cat/plugins?v
    
    name     component          version
    data-2   x-pack-core        6.2.4
    data-2   x-pack-deprecation 6.2.4
    data-2   x-pack-graph       6.2.4
    data-2   x-pack-logstash    6.2.4
    data-2   x-pack-ml          6.2.4
    data-2   x-pack-monitoring  6.2.4
    data-2   x-pack-security    6.2.4
    data-2   x-pack-upgrade     6.2.4
    data-2   x-pack-watcher     6.2.4
    client-1 x-pack-core        6.2.4
    client-1 x-pack-deprecation 6.2.4
    client-1 x-pack-graph       6.2.4
    client-1 x-pack-logstash    6.2.4
    client-1 x-pack-ml          6.2.4
    client-1 x-pack-monitoring  6.2.4
    client-1 x-pack-security    6.2.4
    client-1 x-pack-upgrade     6.2.4
    client-1 x-pack-watcher     6.2.4
    master-1 x-pack-core        6.2.4
    master-1 x-pack-deprecation 6.2.4
    master-1 x-pack-graph       6.2.4
    master-1 x-pack-logstash    6.2.4
    master-1 x-pack-ml          6.2.4
    master-1 x-pack-monitoring  6.2.4
    master-1 x-pack-security    6.2.4
    master-1 x-pack-upgrade     6.2.4
    master-1 x-pack-watcher     6.2.4
    data-1   x-pack-core        6.2.4
    data-1   x-pack-deprecation 6.2.4
    data-1   x-pack-graph       6.2.4
    data-1   x-pack-logstash    6.2.4
    data-1   x-pack-ml          6.2.4
    data-1   x-pack-monitoring  6.2.4
    data-1   x-pack-security    6.2.4
    data-1   x-pack-upgrade     6.2.4
    data-1   x-pack-watcher     6.2.4
    

    查看每个节点的自定义属性

    GET /_cat/nodeattrs?v
    
    node     host  ip              attr              value
    data-2   node3 192.168.113.103 ml.machine_memory 1028517888
    data-2   node3 192.168.113.103 ml.max_open_jobs  20
    data-2   node3 192.168.113.103 ml.enabled        true
    client-1 node4 192.168.113.104 ml.machine_memory 1856888832
    client-1 node4 192.168.113.104 ml.max_open_jobs  20
    client-1 node4 192.168.113.104 ml.enabled        true
    master-1 node1 192.168.113.101 ml.machine_memory 1028517888
    master-1 node1 192.168.113.101 ml.max_open_jobs  20
    master-1 node1 192.168.113.101 ml.enabled        true
    data-1   node2 192.168.113.102 ml.machine_memory 1028517888
    data-1   node2 192.168.113.102 ml.max_open_jobs  20
    data-1   node2 192.168.113.102 ml.enabled        true
    

    查看索引分片的恢复视图

    ##索引分片的恢复视图,包括正在进行和先前已完成的恢复
    ##只要索引分片移动到群集中的其他节点,就会发生恢复事件
    GET _cat/recovery/.kibana?v&format=json&pretty
    
    [
      {
        "index": ".kibana",
        "shard": "0",
        "time": "446ms",
        "type": "empty_store",
        "stage": "done",
        "source_host": "n/a",
        "source_node": "n/a",
        "target_host": "node2",
        "target_node": "data-1",
        "repository": "n/a",
        "snapshot": "n/a",
        "files": "0",
        "files_recovered": "0",
        "files_percent": "0.0%",
        "files_total": "0",
        "bytes": "0",
        "bytes_recovered": "0",
        "bytes_percent": "0.0%",
        "bytes_total": "0",
        "translog_ops": "0",
        "translog_ops_recovered": "0",
        "translog_ops_percent": "100.0%"
      },
      {
        "index": ".kibana",
        "shard": "0",
        "time": "2s",
        "type": "peer",
        "stage": "done",
        "source_host": "node2",
        "source_node": "data-1",
        "target_host": "node3",
        "target_node": "data-2",
        "repository": "n/a",
        "snapshot": "n/a",
        "files": "1",
        "files_recovered": "1",
        "files_percent": "100.0%",
        "files_total": "1",
        "bytes": "230",
        "bytes_recovered": "230",
        "bytes_percent": "100.0%",
        "bytes_total": "230",
        "translog_ops": "0",
        "translog_ops_recovered": "0",
        "translog_ops_percent": "100.0%"
      }
    ]
    

    查看每个数据节点上fielddata当前占用的堆内存

    全文检索用倒排索引非常合适;但过滤、分组聚合、排序这些操作,正排索引更合适。

    ES中引入了fielddata的数据结构用来做正排索引。如果需要对某一个字段排序、分组聚合、过滤,则可将字段设置成fielddata。

    默认情况下:

    text类型的字段是不能分组及排序的,如需要则需要开启该字段的fielddata=true,但是这样耗费大量的内存,不建议这么使用。

    keyword类型默认可分组及排序。

    fielddata默认是采用懒加载的机制加载到堆内存中。当某个字段基数特别大,可能会出现OOM。

    GET _cat/fielddata?v&h=node,field,size
    
    node   field                      size
    data-1 kibana_stats.kibana.uuid     0b
    data-1 kibana_stats.kibana.status   0b
    data-2 kibana_stats.kibana.uuid     0b
    data-2 kibana_stats.kibana.status   0b
    
    ##对某一字段进行查看
    GET _cat/fielddata?v&h=node,field,size&fields=kibana_stats.kibana.uuid
    
    node   field                    size
    data-2 kibana_stats.kibana.uuid   0b
    data-1 kibana_stats.kibana.uuid   0b
    

    查看注册的快照仓库

    GET _cat/repositories?v
    
    id type
    

    查看快照仓库下的快照

    ##可将ES中的一个或多个索引定期备份到如HDFS、S3等更可靠的文件系统,以应对灾难性的故障
    ##第一次快照是一个完整拷贝,所有后续快照则保留的是已存快照和新数据之间的差异
    ##当出现灾难性故障时,可基于快照恢复
    
    GET _cat/snapshots/repo1?v
    

    查看每个节点线程池的统计信息

    ##查看每个节点bulk线程池的统计信息
    ## actinve(活跃的),queue(队列中的)和 reject(拒绝的)
    GET _cat/thread_pool/bulk?v&format=json&pretty
    
    [
      {
        "node_name": "data-2",
        "name": "bulk",
        "active": "0",
        "queue": "0",
        "rejected": "0"
      },
      {
        "node_name": "client-1",
        "name": "bulk",
        "active": "0",
        "queue": "0",
        "rejected": "0"
      },
      {
        "node_name": "master-1",
        "name": "bulk",
        "active": "0",
        "queue": "0",
        "rejected": "0"
      },
      {
        "node_name": "data-1",
        "name": "bulk",
        "active": "0",
        "queue": "0",
        "rejected": "0"
      }
    ]
    

    查看索引

    GET _cat/indices/.monitoring*?v&h=index,health
    
    index                           health
    .monitoring-es-6-2018.08.25     green
    .monitoring-kibana-6-2018.08.25 green
    

    查看别名

    GET _cat/aliases?v&h=alias,index
    
    alias     index
    .security .security-6
    

    查看索引模板

    GET _cat/templates?v&format=json&pretty
    
    [
      {
        "name": "logstash-index-template",
        "index_patterns": "[.logstash]",
        "order": "0",
        "version": null
      },
      {
        "name": "security_audit_log",
        "index_patterns": "[.security_audit_log*]",
        "order": "2147483647",
        "version": null
      },
      {
        "name": ".monitoring-kibana",
        "index_patterns": "[.monitoring-kibana-6-*]",
        "order": "0",
        "version": "6020099"
      },
      {
        "name": ".watches",
        "index_patterns": "[.watches*]",
        "order": "2147483647",
        "version": null
      },
      {
        "name": ".monitoring-beats",
        "index_patterns": "[.monitoring-beats-6-*]",
        "order": "0",
        "version": "6020099"
      },
      {
        "name": ".ml-notifications",
        "index_patterns": "[.ml-notifications]",
        "order": "0",
        "version": "6020499"
      },
      {
        "name": ".ml-anomalies-",
        "index_patterns": "[.ml-anomalies-*]",
        "order": "0",
        "version": "6020499"
      },
      {
        "name": ".ml-state",
        "index_patterns": "[.ml-state]",
        "order": "0",
        "version": "6020499"
      },
      {
        "name": "security-index-template",
        "index_patterns": "[.security-*]",
        "order": "1000",
        "version": null
      },
      {
        "name": ".watch-history-7",
        "index_patterns": "[.watcher-history-7*]",
        "order": "2147483647",
        "version": null
      },
      {
        "name": ".ml-meta",
        "index_patterns": "[.ml-meta]",
        "order": "0",
        "version": "6020499"
      },
      {
        "name": ".monitoring-alerts",
        "index_patterns": "[.monitoring-alerts-6]",
        "order": "0",
        "version": "6020099"
      },
      {
        "name": ".monitoring-logstash",
        "index_patterns": "[.monitoring-logstash-6-*]",
        "order": "0",
        "version": "6020099"
      },
      {
        "name": ".triggered_watches",
        "index_patterns": "[.triggered_watches*]",
        "order": "2147483647",
        "version": null
      },
      {
        "name": ".monitoring-es",
        "index_patterns": "[.monitoring-es-6-*]",
        "order": "0",
        "version": "6020099"
      },
      {
        "name": "kibana_index_template:.kibana",
        "index_patterns": "[.kibana]",
        "order": "0",
        "version": null
      }
    ]
    

    查看单个或某类或整个集群文档数

    ##整个集群文档数
    GET _cat/count?v
    epoch      timestamp count
    1535185950 16:32:30  3008
    
    ##某类索引文档数
    GET _cat/count/.monitoring*?v
    epoch      timestamp count
    1535186185 16:36:25  2162
    

    注意:只包含实际文档数,不包括尚未清除的已删除文档。

    查看每个索引的分片

    GET _cat/shards?v&format=json&pretty&s=index
    [
      {
        "index": ".kibana",
        "shard": "0",
        "prirep": "p",
        "state": "STARTED",
        "docs": "1",
        "store": "4kb",
        "ip": "192.168.113.102",
        "node": "data-1"
      },
      {
        "index": ".kibana",
        "shard": "0",
        "prirep": "r",
        "state": "STARTED",
        "docs": "1",
        "store": "4kb",
        "ip": "192.168.113.103",
        "node": "data-2"
      },
      {
        "index": ".monitoring-es-6-2018.08.25",
        "shard": "0",
        "prirep": "p",
        "state": "STARTED",
        "docs": "2557",
        "store": "1.6mb",
        "ip": "192.168.113.102",
        "node": "data-1"
      },
      {
        "index": ".monitoring-es-6-2018.08.25",
        "shard": "0",
        "prirep": "r",
        "state": "STARTED",
        "docs": "2557",
        "store": "1.5mb",
        "ip": "192.168.113.103",
        "node": "data-2"
      },
      {
        "index": ".monitoring-kibana-6-2018.08.25",
        "shard": "0",
        "prirep": "p",
        "state": "STARTED",
        "docs": "211",
        "store": "110.1kb",
        "ip": "192.168.113.102",
        "node": "data-1"
      },
      {
        "index": ".monitoring-kibana-6-2018.08.25",
        "shard": "0",
        "prirep": "r",
        "state": "STARTED",
        "docs": "211",
        "store": "110.1kb",
        "ip": "192.168.113.103",
        "node": "data-2"
      },
      {
        "index": ".security-6",
        "shard": "0",
        "prirep": "p",
        "state": "STARTED",
        "docs": "3",
        "store": "9.8kb",
        "ip": "192.168.113.102",
        "node": "data-1"
      },
      {
        "index": ".security-6",
        "shard": "0",
        "prirep": "r",
        "state": "STARTED",
        "docs": "3",
        "store": "9.8kb",
        "ip": "192.168.113.103",
        "node": "data-2"
      }
    ]
    

    查看每个索引的segment

    GET _cat/segments/.kibana?v&format=json&pretty
    
    [
      {
        "index": ".kibana",
        "shard": "0",
        "prirep": "p",
        "ip": "192.168.113.102",
        "segment": "_2",
        "generation": "2",
        "docs.count": "1",
        "docs.deleted": "0",
        "size": "3.7kb",
        "size.memory": "1346",
        "committed": "true",
        "searchable": "true",
        "version": "7.2.1",
        "compound": "true"
      },
      {
        "index": ".kibana",
        "shard": "0",
        "prirep": "r",
        "ip": "192.168.113.103",
        "segment": "_2",
        "generation": "2",
        "docs.count": "1",
        "docs.deleted": "0",
        "size": "3.7kb",
        "size.memory": "1346",
        "committed": "true",
        "searchable": "true",
        "version": "7.2.1",
        "compound": "true"
      }
    ]
    

    参考文献

    https://blog.csdn.net/wangpei1949/article/details/82287444

  • 相关阅读:
    逐级汇总示例(循环逐级累计法).sql
    UdtSsn.cs
    2013届大华股份 软件算法类试题 D卷
    TCP与UDP区别
    浙江大华 研发类试题
    百度面试经历
    Contiki入门学习
    09网易校园招聘笔试题
    (转载)利用webkit抓取动态网页和链接
    2012.9.23 搜狗笔试
  • 原文地址:https://www.cnblogs.com/crazymakercircle/p/15575624.html
Copyright © 2011-2022 走看看