zoukankan      html  css  js  c++  java
  • [k8s]jenkins配合kubernetes插件实现k8s集群构建的持续集成

    另一个结合harbor自动构建镜像的思路: 即code+baseimage一体的方案

    - 程序员将代码提交到代码仓库gitlab
    - 钩子触发jenkins master启动一次构建
    - jenkins master从k8s申请一个jenkins slave编译容器
    - 在容器内编译完成以后,获得最终产物
    - 将最终产物通过dockerfile生成生产部署镜像(这里省略了测试,其实部署镜像需要测试通过)
    - 将生产镜像推送到harbor镜像仓库
    - jenkins slave生命周期结束,k8s销毁slave容器
    - 一次构建完成
    

    k8s持续集成的一个思路:

    这里要说的是部署部分

    注: 这只是一个持续集成思想.本篇按照这个思想来搞,在我的环境里我为了速度快当然还有别的因素我用这种方案,用的很6.
    其他思路:

    • 可以将code+image打在一起做升级
    • 可以rbac+环境+ns+supervisor 每个开发一个环境这样搞
      后面我抽空一一实现下.

    这篇文章思路:

    手动构建war包(集成测试)-->本地tomcat测试通过(功能测试)-->k8s容器化tomcat(pv+deploy+svc+ingress)-->将war包拖入k8s的tomcat测试.

    jenkins jnlp镜像构建(mvn+git+kubectl)-> jnlp镜像测试,确保可被server动态调度-->配置war包的pipeline测试.

    注: 本篇jenkins server部署在vm上,非docker部署,jenkins-jnlp-slave是容器化自动创建的.
    其他内容参考: 容器ci索引: http://www.cnblogs.com/iiiiher/p/8026689.html

    构建jnlp镜像的dockerfile

    • 准备dockerfile所需文件
    git clone https://github.com/jenkinsci/docker-jnlp-slave.git
    cd docker-jnlp-slave
    
    $ ls
    Dockerfile  jenkins-slave  kubectl  README.md
    
    

    构建mvn3.5.2+git+kubectl的镜像

    基于jenkinsci/slave:alpine的基础镜像
    参考: https://github.com/jenkinsci/docker-slave/blob/master/Dockerfile
    https://github.com/jenkinsci/docker-jnlp-slave/blob/master/Dockerfile
    https://hub.docker.com/r/jenkinsci/slave/tags/

    alpine-git安装参考:
    https://hub.docker.com/r/alpine/git/~/dockerfile/

    $ cat Dockerfile 
    FROM jenkinsci/slave:alpine
    
    USER root
    RUN apk add --no-cache curl tar bash
    
    ## Install Maven
    ARG MAVEN_VERSION=3.5.2
    ARG USER_HOME_DIR="/root"
    ARG SHA=707b1f6e390a65bde4af4cdaf2a24d45fc19a6ded00fff02e91626e3e42ceaff
    ARG BASE_URL=https://apache.osuosl.org/maven/maven-3/${MAVEN_VERSION}/binaries
    
    RUN mkdir -p /usr/share/maven /usr/share/maven/ref 
      && curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL}/apache-maven-${MAVEN_VERSION}-bin.tar.gz 
      && echo "${SHA}  /tmp/apache-maven.tar.gz" | sha256sum -c - 
      && tar -xzf /tmp/apache-maven.tar.gz -C /usr/share/maven --strip-components=1 
      && rm -f /tmp/apache-maven.tar.gz 
      && ln -s /usr/share/maven/bin/mvn /usr/bin/mvn 
      && apk --update add git openssh 
      && rm -rf /var/lib/apt/lists/* 
      && rm /var/cache/apk/* 
      && mkdir /src /target 
      && chown jenkins.jenkins /src /target
    
    ENV MAVEN_HOME /usr/share/maven
    ENV MAVEN_CONFIG "$USER_HOME_DIR/.m2"
    
    # install kubectl
    COPY kubectl /usr/local/bin/kubectl
    
    ## install jenkins-slave
    COPY jenkins-slave /usr/local/bin/jenkins-slave
    USER jenkins
    WORKDIR /home/jenkins
    ENTRYPOINT ["jenkins-slave"]
    

    镜像已可以从dockerhub下载:

    docker pull lanny/mvn-git-kubectl-jnlp:3.5.2
    

    测试jnlp镜像

    主要看他能否用jenkins-server动态调用跑起来
    当然首先安装jenkins kubernetes插件,新建一朵云:
    参考: http://www.cnblogs.com/iiiiher/p/7979336.html

    配置项目: 选择pipeline script

    podTemplate(name: 'maotai-dev', cloud: 'kubernetes',
      namespace: 'kube-public', label: 'maotai-dev',
      serviceAccount: 'default', containers: [
      containerTemplate(
          name: 'jnlp',
          image: 'lanny/mvn-git-kubectl-jnlp:3.5.2',
          args: '${computer.jnlpmac} ${computer.name}',
          ttyEnabled: true,
          privileged: false,
          alwaysPullImage: false)
      ],
    ) {
    
      node('maotai-dev') {
        stage('git-clone') {
          container('jnlp') {
              sh """
              date +%F;
              sleep 30;
              """
          }
        }
      }
    }
    
    

    点击构建-->显示构建成功

    构建成功后jnlp镜像随着构建结束自动删除.

    tomcat java-helloworld项目

    kubernetes插件的pipeline使用:
    参考:
    https://github.com/jenkinsci/kubernetes-plugin
    https://help.aliyun.com/document_detail/56336.html?spm=5176.doc56336.6.851.wAqCzu

    javahelloworld代码: https://github.com/lannyMa/trucks ,构建可形成helloworld的war包.可以部署在tomcat用于测试.

    jenkins项目配置: 新建项目 test-pipeline

    podTemplate(name: 'maotai-dev', cloud: 'kubernetes',
      namespace: 'kube-public', label: 'maotai-dev',
      serviceAccount: 'default', containers: [
      containerTemplate(
          name: 'jnlp',
          image: 'lanny/mvn-git-kubectl-jnlp:3.5.2',
          args: '${computer.jnlpmac} ${computer.name}',
          ttyEnabled: true,
          privileged: false,
          alwaysPullImage: false)
      ],
      volumes: [
        persistentVolumeClaim(mountPath: '/tmp/', claimName: 'spring-pvc')
      ]) {
      node('maotai-dev') {
        stage('git-clone') {
          container('jnlp') {
              sh """
              git clone https://github.com/lannyMa/trucks.git
              """
          }
        }
    
        stage('mvn-package') {
          container('jnlp') {
              sh """
              cd trucks && mvn clean package && cp -rpf target/*.war /tmp/
              """
          }
        }
    
        stage('restart') {
          container('jnlp') {
              sh """
              pod_name=`kubectl -s 192.168.x.x:8080 -n kube-public get pods -l name=maotai-dev -o name | cut -d"/" -f2`
              kubectl -s kube-apiserver-http.kube-public -n kube-public delete pod $pod_name
              """
          }
        }
      }
    }
    
    

    配置tomcat项目

    tomcat-pvc.yaml #前提是配置好stroragecalss: 参考: http://www.cnblogs.com/iiiiher/p/7988803.html

    kind: PersistentVolumeClaim
    apiVersion: v1
    metadata:
      name: spring-pvc
      namespace: kube-public
    spec:
      storageClassName: "managed-nfs-storage"
      accessModes:
        - ReadOnlyMany
      resources:
        requests:
          storage: 100Mi
    

    tomcat-deploy.yaml

    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
      name: spring
      namespace: kube-public
    spec:
      replicas: 1
      template:
        metadata:
          labels:
            name: spring
            name: maotai-dev  #这里标签设置需注意,因为jenkins配置kubectl的stage时需要根据标签过滤重启它: kubectl -s 192.168.x.x:8080 -n kube-public get pods -l name=spring -o name | cut -d"/" -f2
        spec:
          containers:
          - name: spring
            image: tomcat:latest
            imagePullPolicy: IfNotPresent
            ports:
            - name: web
              containerPort: 8080
            volumeMounts:
            - mountPath: /usr/local/tomcat/webapps
              name: spring-folder
          volumes:
          - name: spring-folder
            persistentVolumeClaim:
              claimName: spring-pvc
    

    执行成功:

    k8s集群容器化tomcat项目

    • 容器化tomcat项目: 配置k8s集群的tomcat 包含了 pvc+deploy+svc+ingress
    • 做法:
      • 先手动编译项目,本次tomcat测试通过
      • 集成到k8s集群的tomcat,测试,确保项目可以正常运行

    tomcat-svc.yaml

    apiVersion: v1
    kind: Service
    metadata:
      name: spring
      namespace: kube-public
      labels:
        name: spring
    spec:
      ports:
        - name: web
          port: 8080
          targetPort: web
      selector:
        name: spring
    
    

    tomcat-ingress.yaml #前提是已配置好了ingress,nginx-ingress配置参考:http://www.cnblogs.com/iiiiher/p/8006801.html

    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: spring
      namespace: kube-public
    spec:
      rules:
      - host: spring.maotai.net
        http:
          paths:
          - path: /
            backend:
              serviceName: spring
              servicePort: web
    
    

    创建好后确保能够访问:

    接下来需要手动编译,确保项目通过集成测试(可成功编译),功能测试(部署tomcat后可访问)

    项目代码: https://github.com/lannyMa/trucks.git
    mvn配置改源等参考: https://github.com/lannyMa/java-helloword.git

    确保没问题后将war包放到上一步创建的pv里.我的是nfs,直接到nfs-server上把war包托上去,然后重启tomcat,测试效果.

  • 相关阅读:
    cf B. Sereja and Suffixes
    cf E. Dima and Magic Guitar
    cf D. Dima and Trap Graph
    cf C. Dima and Salad
    最短路径问题(floyd)
    Drainage Ditches(网络流(EK算法))
    图结构练习—BFSDFS—判断可达性(BFS)
    Sorting It All Out(拓扑排序)
    Power Network(最大流(EK算法))
    Labeling Balls(拓扑)
  • 原文地址:https://www.cnblogs.com/iiiiher/p/8026555.html
Copyright © 2011-2022 走看看