[k8s]一些技巧性的yaml和dockerfile-docker学习思路

我会把一些dockerfile和yaml的技巧性东西不定期搜集到这里

docker学习思路

三部曲:

• 1,vm会搭建服务
• 2,docker会跑服务
• 3,k8s集群会调度该服务

存储

• vm1 vm2 共享存储,vm1的容器挂了直接调度到vm2(docker存储驱动实现)

网络

• 物理机安装flannel,使两台docker上容器能通(物理机/容器搭建etcd集群)
• 物理机搭建openvswitch,实现两台docker

监控

• cadvisor会跑
• 物理机安装grafana,物理机安装promethus(实现nodeexplore和mysqlexplore)
• 容器跑promethus +cadvisor+grafana
• 容器跑cadvisor+influendb+grafana
• 容器跑elk,物理机跑filebeat搜集容器日志

pod的一些技巧

参考: https://jimmysong.io/kubernetes-handbook/appendix/tricks.html

• 在容器中获取 Pod 的IP
• 指定容器的启动参数
• 让Pod调用宿主机的docker能力
• 使用Init container初始化应用配置
• 使容器内时间与宿主机同步
• 在Pod中获取宿主机的主机名、namespace等
• 配置Pod使用外部DNS

dockerfile

tomcat 启动

EXPOSE 8080
CMD ["catalina.sh", "run"]

nginx日志和启动

# forward request and error logs to docker log collector
RUN ln -sf /dev/stdout /var/log/nginx/access.log 
	&& ln -sf /dev/stderr /var/log/nginx/error.log

EXPOSE 80

STOPSIGNAL SIGTERM

CMD ["nginx", "-g", "daemon off;"]

yaml 让busybox当计数器

apiVersion: v1
kind: Pod
metadata:
  name: counter
spec:
  containers:
  - name: count
    image: busybox
    args: [/bin/sh, -c, 'i=0; while true; do echo "$i: $(date)"; i=$((i+1)); sleep 1; done']

相当于

docker run -d --name=b1 busybox  i=0; while true; do echo "$i: $(date)"; i=$((i+1)); sleep 1; done

docker run -d 
           --log-driver=fluentd 
           --log-opt fluentd-address=localhost:24224 
           --log-opt tag="log-test-container-A" 
           busybox sh -c 'while true; do echo "This is a log message from container A"; sleep 10; done;'

yaml 让busybox执行一个脚本

参考: https://kubernetes.io/docs/concepts/cluster-administration/logging/

apiVersion: v1
kind: Pod
metadata:
  name: counter
spec:
  containers:
  - name: count
    image: busybox
    args:
    - /bin/sh
    - -c
    - >
      i=0;
      while true;
      do
        echo "$i: $(date)" >> /var/log/1.log;
        echo "$(date) INFO $i" >> /var/log/2.log;
        i=$((i+1));
        sleep 1;
      done
    volumeMounts:
    - name: varlog
      mountPath: /var/log
  - name: count-log-1
    image: busybox
    args: [/bin/sh, -c, 'tail -n+1 -f /var/log/1.log']
    volumeMounts:
    - name: varlog
      mountPath: /var/log
  - name: count-log-2
    image: busybox
    args: [/bin/sh, -c, 'tail -n+1 -f /var/log/2.log']
    volumeMounts:
    - name: varlog
      mountPath: /var/log
  volumes:
  - name: varlog
    emptyDir: {}

yaml 让busybox睡5min

参考: https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#dns-policy

apiVersion: v1
kind: Pod
metadata:
  name: busybox
  namespace: default
spec:
  containers:
  - image: busybox
    command:
      - sleep
      - "3600"
    imagePullPolicy: IfNotPresent
    name: busybox
  restartPolicy: Always

dockerfile拷贝目录的一个坑

参考: https://stackoverflow.com/questions/26504846/copy-directory-to-other-directory-at-docker-using-add-command

## 拷贝文件不需要写目标
FROM centos
COPY 2.txt /usr/local/

## 拷贝目录则需要这样写,目标,不然拷贝不进去
FROM centos
COPY mysql /usr/local/mysql

设置容器的TZ另一种办法

## override default time zone (Etc/UTC) if TZ variable is set
if [ ! -z "$TZ" ]; then
  ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
fi

k8s集群里pod高负载访问apiserver的8080(api对外地址)

因为api对内地址是443
集群api如果有3台,则需要负载访问,咋办呢? 自定义svc,endpoint,实现对外负载(如果只有一个apiserver,则直连即可)

kind: Endpoints
apiVersion: v1
metadata:
  name: kube-apiserver-http
  namespace: kube-public
subsets:
- addresses:
  - ip: 192.168.x.132
  - ip: 192.168.x.133
  - ip: 192.168.x.134
  ports:
  - name: http
    port: 8080
    protocol: TCP

kind: Service
apiVersion: v1
metadata:
  labels:
    app-name: kube-apiserver-http
  name: kube-apiserver-http
  namespace: kube-public
spec:
  ports:
  - name: http
    port: 80
    targetPort: 8080
    protocol: TCP
  sessionAffinity: ClientIP

command写法和容器privileged模式设置(cni部署网络)/容器获取宿主机(外界/或集群ns的名字)参数

参考: https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml
https://feisky.gitbooks.io/kubernetes/network/flannel/#cni集成
http://cizixs.com/2017/05/23/container-network-cni

        command: [ "/opt/bin/flanneld", "--ip-masq", "--kube-subnet-mgr" ]
        securityContext:
          privileged: true

        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace

参考

    env:
    - name: MY_POD_NAME
      valueFrom:
        fieldRef:
          fieldPath: metadata.name
    - name: MY_POD_NAMESPACE
      valueFrom:
        fieldRef:
          fieldPath: metadata.namespace
    - name: MY_POD_IP
      valueFrom:
        fieldRef:
          fieldPath: status.podIP