zoukankan html css js c++ java

Kubernetes（十三） CI／CD（1）

本文通过在kubernetes上启动Jenkins服务，并将宿主机上的docker、docker.sock挂载到Jenkins容器中，实现在Jenkins容器中直接打镜像的形式实现CI功能。

Kubernetes 集群的安装请参考kubernetes安装

部署Jenkins

下载Jenkins对应的镜像
```
docker pull jenkins/jenkins:2.221
```

将jenkins镜像上传到自己的私有镜像仓库中

docker tag jenkins/jenkins:2.221 192.168.0.107/k8s/jenkins:2.221

docker push 	192.168.0.107/k8s/jenkins:2.221

编写启动Jenkins的yml文件

cat > jenkins.yml << EOF
kind: PersistentVolume
apiVersion: v1
metadata:
  name: jenkins
  labels:
    type: local
    app: jenkins
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: /opt/k8s/yml/jenkins/data
---

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: jenkins-claim
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
---

apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins
  namespace: default
automountServiceAccountToken: true
---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: Jenkins-cluster-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: jenkins
  namespace: default
---

apiVersion: v1
kind: Service
metadata:
  name: jenkins
  labels:
    app: jenkins
spec:
  ports:
    - port: 80
      targetPort: 8080
      nodePort: 8888
      name: jenkins
    - port: 50000
      targetPort: 50000
      nodePort: 50000
      name: agent
  selector:
    app: jenkins
    tier: jenkins
  type: NodePort
---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkins
  labels:
    app: jenkins
spec:
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: jenkins
      tier: jenkins
  template:
    metadata:
      labels:
        app: jenkins
        tier: jenkins
    spec:
      serviceAccountName: jenkins
      containers:
      - image: 192.168.0.107/k8s/jenkins:2.221
        imagePullPolicy: IfNotPresent
        name: jenkins
        securityContext:
          privileged: true
          runAsUser: 0
        volumeMounts:
        - name: kubeconfig
          mountPath: /var/jenkins_home/.kube
        - name: docker
          mountPath: /var/run/docker.sock
        - name: docker-bin
          mountPath: /usr/bin/docker
        - name: jenkins-persistent-storage
          mountPath: /var/jenkins_home
        ports:
        - containerPort: 8080
          name: jenkins
        - containerPort: 50000
        	name: agent
      volumes:
      - name: kubeconfig
        emptyDir: {}
      - name: docker
        hostPath:
          path: /var/run/docker.sock
      - name: docker-bin
        hostPath:
          path: /opt/k8s/bin/docker
      - name: jenkins-persistent-storage
        persistentVolumeClaim:
          claimName: jenkins-claim
EOF

其中ServiceAccount和ClusterRoleBinding是为了后续步骤在kubernets集群中启动pod完成构建任务而创建
为了能在Jenkins容器内部直接使用dokcer命令打镜像，此处直接将宿主机上的docker命令以及docker.sock挂载到Jenkins中，没有在Jenkins容器中进行docker-ce的安装
Jenkins容器设置成特权用户执行，并把执行用户Id设置成0（root用户），原因参看遇到问题一节

启动Jenkins

mkdir -p /opt/k8s/yml/jenkins/data
chmod -R 777 /opt/k8s/yml/jenkins/data
kubectl create -f jenkins.yml

启动后首次登陆密码可在日志中查看，或通过如下命令获取

kubectl exec -it `kubectl get pods  --selector=app=jenkins --output=jsonpath={.items..metadata.name}` cat /var/jenkins_home/secrets/initialAdminPassword

安装插件

安装：git-parameter、git-client、git、pipeline相关插件,可在jenkins插件管理界面上选择安装，如果下载失败，可以查看对应软件的版本从https://mirrors.tuna.tsinghua.edu.cn/jenkins/plugins下载后放到Jenkins工作目录下的plugins目录下。

验证Jenkins

创建git工程，在gitlab上创建一个简单的hello-ci工程，功能是：基于nginx镜像打一个自己的镜像，替换其中的欢迎页 index.html

工程代码

index.html

<html>
    <p><h2 style="font-family:sans-serif">Hello from ci! You've successfully built and run the Hello-ci app.</h2> </p>
    <p style="font-family:sans-serif">The Hello-ci app is a modified version of the <a href="https://hub.docker.com/_/nginx/">nginx web server image</a>. If you open up the <b>kubernetes-ci-cd/hello-ci/DockerFile</b>, you will note several things:</p>
    <p style="font-family:sans-serif">welcome to ci </p>
</html>

Dockerfile

FROM 192.168.0.107/k8s/nginx:1.9.1

COPY index.html /usr/share/nginx/html/index.html

EXPOSE 80

Jenkinsfile

node {


	    properties([parameters([[$class: 'GitParameterDefinition', branch: '', branchFilter: '.*', defaultValue: '', description: '', name: 'release_version', quickFilterEnabled: false, selectedValue: 'NONE', sortMode: 'NONE', tagFilter: '*', type: 'PT_BRANCH_TAG']])])
	
	    checkout scm
	    
	    stage ("edit parameters") {
	    
	        echo "release_version:${release_version}"
	        
	        real_version = release_version.replaceAll("origin/","")
	        
	        echo "real_version:${real_version}"
	    
	    }
	    
	    imageName = "192.168.0.107/k8s/hello-ci:${real_version}"
	    
	    
	    stage ("docker login") {
	        sh "docker login -u admin -p Harbor12345 192.168.0.107"
	    }
	    
	    stage ("Build") {
	    
	        sh "docker build -t ${imageName} application"
	    }
	    stage ("Push") {
	
	        sh "docker push ${imageName}"
	    }
	    
	   
	
	}

采用脚本形式编辑pipeline，也可以采用声明方式
properties属性指定此Jenkins工程是参数化构建，构建参数是 branch或者是tag
根据选定的分支或tag决定打出的镜像对应的版本号

创建Jenkins工程
在Jenkins界面新建一个item，名称hello-pipeline，类型选择:流水线(pipeline)

配置工程为参数化构建，参数列表中选择gitparameter,类型为branchortag
- 此步骤也可不执行，首次执行工程后Jenkins会根据Jenkinsfile中的内容自动把工程变成参数化构建，但是这样第一次就不能选择对应的版本，所以此处加了一个配置
设置工程路径，对应的脚本路径

执行构建，选择一个分支，以master为例，执行构建

构建日志

Started by user admin
Lightweight checkout support not available, falling back to full checkout.
Checking out git http://192.168.0.107:9090/ci-cd/hello-ci.git into /var/jenkins_home/workspace/hello-pipeline@script to read application/Jenkinsfile
Cloning the remote Git repository

 ...
 
[Pipeline] Start of Pipeline

...

[Pipeline] stage
[Pipeline] { (edit parameters)
[Pipeline] echo
release_version:origin/master
[Pipeline] echo
real_version:master
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (docker login)
[Pipeline] sh
+ docker login -u admin -p Harbor12345 192.168.0.107
...

Login Succeeded
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Build)
[Pipeline] sh
+ docker build -t 192.168.0.107/k8s/hello-ci:master application
...

Successfully built b2b4f45901a6
Successfully tagged 192.168.0.107/k8s/hello-ci:master
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Push)
[Pipeline] sh
+ docker push 192.168.0.107/k8s/hello-ci:master
The push refers to repository [192.168.0.107/k8s/hello-ci]

...

4fc9a49e07e9: Pushed
master: digest: sha256:a90710b35388915d2b01dfc6173da996f8191be2a850b9c8453534e85c91a7f9 size: 3012
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

可以看到根据我们选择的master分枝，打出来了一个192.168.0.107/k8s/hello-ci:master的镜像

验证构建好的镜像文件

编写启动镜像的文件

cat > hello-ci.yml << EOF
apiVersion: v1
kind: Service
metadata:
  name: hello-ci
  labels:
    app: hello-ci
spec:
  type: NodePort
  selector:
    app: hello-ci
  ports:
  - name: http
    port: 8089
    targetPort: 80
    nodePort: 8089
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-ci-deployment
spec:
  selector:
    matchLabels:
      app: hello-ci
  replicas: 1
  template:
    metadata:
      labels:
        app: hello-ci
    spec:
      containers:
      - name: hello-ci
        image: 192.168.0.107/k8s/hello-ci:master
        ports:
        - containerPort: 80
EOF

启动hello-ci
```
kubectl create -f hello-ci.yml
```
访问界面

向Jenkins中追加slave node

配置slave节点（在slave节点上执行）

为Jenkins执行用户（按照本文创建的Jenkins用户是root（runAsUser: 0 配置）生成可信赖的认证key，（如果已经生成过，可以直接拿来用）
```
cd ~/.ssh
ssh-keygen -t rsa -C "admin@example.com"

cat id_rsa.pub >> authorized_keys

chmod 700 authorized_keys

service sshd restart
```
在Jenkins界面

Manage Jenkins -> Manage Nodes -> New Node追加一个node

点击OK后进入node配置界面

Credentials追加
保存后Jenkins就会自动的launch 对应的slave，并检查节点上的环境，如是否有java，如果没有就尝试去下载安装（因为现在oracle下载jdk需要登陆，此步骤不会自动成功，所以需要提前在slave节点上安装好jdk工具）

遇到问题

长时间处于Please wait while Jenkins is getting ready to work ...

修改hudson.model.UpdateCenter.xml文件


# 删除
#https://updates.jenkins-ci.org/update-center.json

#追加
<url>https://mirrors.tuna.tsinghua.edu.cn/jenkins/updates/update-center.json</url>

jenkinsfile中执行 sh "docker ..."相关命令时出错
```
dial unix /var/run/docker.sock: connect: permission denied
```
因为宿主机上docker是以root身份启动的，每次启动docker服务会生成docker.sock，此时docker.sock默认对应的用户和用户组都是root的
```
root@master:/opt/k8s/yml/jenkins# ls -l /var/run/docker.sock
```

srw-rw---- 1 root root 0 2月 19 20:11 /var/run/docker.sock

```

而默认的Jenkins镜像是以jenkins用户执行

```
root@master:/opt/k8s/yml/jenkins/data/plugins# kubectl exec -it jenkins-798d66fc78-x9zbr bash

jenkins@jenkins-798d66fc78-x9zbr:/$ 

```

所以不具有访问/var/run/docker.sock的权限，解决方法是在jenkins对应的container的securityContext属性中追加<font color=red>runAsUser: 0 </font>配置

jenkinsfile执行中docker pull和docker push镜像时没有权限访问私有仓库，
```
docker push 192.168.0.107/k8s/hello-ci:v1.0.0
...
denied: requested access to the resource is denied
```
可以先在宿主机上执行好docker login,然后把认证后的/root/.docker/config.json挂载到jenkins容器中，或者在Jenkinsfile中追加docker login的步骤
service 和容器没有启动 50000端口，而agent和master之间通信用的是这个端口，造成一直出错,
```
SEVERE: http://192.168.0.107:8888/ provided port:50000 is not reachable
```

java.io.IOException: http://192.168.0.107:8888/ provided port:50000 is not reachable
at org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver.resolve(JnlpAgentEndpointResolver.java:303)
at hudson.remoting.Engine.innerRun(Engine.java:527)
at hudson.remoting.Engine.run(Engine.java:488)

```

查看全文

相关阅读:
Java并发编程：volatile关键字解析(转）
Java并发编程 Synchronized及其实现原理
 (a++)+(++a)=?
Java集合中那些类是线程安全的
 Servlet 与 CGI 的比较
 JAVA中常见的锁以及其特性
 总结一下hashMap和hashtable方面的知识点
 Which statement is true?
下面哪段程序能够正确的实现了GBK编码字节流到UTF-8编码字节流的转换：
Kafka 详解(转)

原文地址：https://www.cnblogs.com/gaofeng-henu/p/12345141.html