zoukankan      html  css  js  c++  java
  • fluentd 收集 k8s 到 elasticsearch

    1. 部署elasticsearch + kibana

    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: es-data-pv
    spec:
      capacity:
        storage: 10Gi
      accessModes:
       - ReadWriteOnce
      storageClassName: nfs
      nfs:
        server: 192.168.0.250
        path: /var/nfs/es-data
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: es-data-pvc
    spec:
      accessModes:
        - ReadWriteOnce
      storageClassName: "nfs"
      resources:
        requests:
          storage: 10Gi
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: es-deployment
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: single-es
      template:
        metadata:
          labels:
            app: single-es
        spec:
          initContainers:
          - name: init-sysctl
            image: busybox
            imagePullPolicy: IfNotPresent
            command: ["sysctl", "-w", "vm.max_map_count=262144"]
            securityContext:
              privileged: true
          containers:
          - name: single-es
            image: docker.elastic.co/elasticsearch/elasticsearch:7.5.2
            ports:
              - containerPort: 9200
                name: http
              - containerPort: 9300
                name: transport
            env:
              - name: discovery.type
                value: single-node
            volumeMounts:
              - mountPath: /usr/share/elasticsearch/data
                name: es-data
          volumes:
            - name: es-data
              persistentVolumeClaim:
                claimName: es-data-pvc 
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: single-es-svc
    spec:
      selector:
        app: single-es
      type: NodePort
      ports:
      - name: http
        port: 9200
        targetPort: 9200
        nodePort: 30092
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: kibana-deployment
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: kibana
      template:
        metadata:
          labels:
            app: kibana
        spec:
          containers:
          - name: kibana
            image: docker.elastic.co/kibana/kibana:7.5.2
            ports:
            - name: http
              containerPort: 5601
            env:
            - name: ELASTICSEARCH_HOSTS
              value: http://single-es-svc:9200
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: kibana-svc
    spec:
      selector:
        app: kibana
      type: NodePort
      ports:
      - name: http
        port: 5601
        targetPort: 5601
        nodePort: 30561
    

    2. 部署fluentd

    参考网站:https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch
    修改 fluentd-es-configmap.yaml 中的

    output.conf: |-
        <match **>
          @id elasticsearch
          @type elasticsearch
          @log_level info
          type_name _doc
          include_tag_key true
          host elasticsearch-logging
          port 9200
          logstash_format true
          <buffer>
            @type file
            path /var/log/fluentd-buffers/kubernetes.system.buffer
            flush_mode interval
            retry_type exponential_backoff
            flush_thread_count 2
            flush_interval 5s
            retry_forever
            retry_max_interval 30
            chunk_limit_size 2M
            total_limit_size 500M
            overflow_action block
          </buffer>
        </match>
    

    match 中的es的host改为部署的es的服务名 elasticsearch-logging -> single-es-svc.default // {服务名.命名空间},如果 es 与 fluentd 在一个命名空间下可省略。

  • 相关阅读:
    iOS 面试题搜集
    iOS 常用第三方类库、完整APP示例
    iOS 键盘遮挡输入 解决办法
    iOS UIColor RGB HEX
    iOS APP性能优化
    iOS Swift 数组 交换元素的两种方法
    iOS CoreData primitive accessor
    iOS Start developing ios apps (OC) pdf
    iOS 传值方式
    iOS IB_DESIGNABLE IBInspectable @IBDesignable @IBInspectable 加速UI开发
  • 原文地址:https://www.cnblogs.com/wh-blog/p/12239109.html
Copyright © 2011-2022 走看看