Jenkins+Docker+SpringCloud微服务持续集成(集群版)

Jenkins+Docker+SpringCloud微服务持续集成(集群版)

作者：运维人在路上

个人博客：https://www.cnblogs.com/hujinzhong

微信公众号：运维人在路上

Bilibili账号：https://space.bilibili.com/409609392

个人简介：本人是一枚大型电商平台的运维工程师，对开发及运维有一定了解，现分享技术干货，欢迎大家交流！

一、单机版存在的问题及优化

1.1、存在的问题

单机版部署文章：https://www.cnblogs.com/hujinzhong/p/14571041.html

上诉部署方案存在的问题：

• 1）一次只能选择一个微服务部署，还要手动修改端口，比较麻烦
• 2）只有一台生产部署服务器
• 3）每个微服务只有一个实例，容错率低

1.2、优化

1）在一个Jenkins工程中可以选择多个微服务同时发布

2）在一个Jenkins工程中可以选择多台生产服务器同时部署

3）每个微服务都是以集群高可用形式部署

二、集群高可用部署

2.1、部署流程说明

2.2、详细部署步骤

2.2.1、修改所有微服务配置

1）注册中心配置修改

# 集群版
spring:
  application:
    name: EUREKA-HA

---
server:
  port: 10086
spring:
  # 指定profile=eureka-server1
  profiles: eureka-server1
eureka:
  instance:
    # 指定当profile=eureka-server1时，主机名是eureka-server1
    hostname: 10.0.0.103
  client:
    service-url:
      # 将自己注册到eureka-server1、eureka-server2这个Eureka上面去
      defaultZone: http://10.0.0.103:10086/eureka/,http://10.0.0.104:10086/eureka/

---
server:
  port: 10086
spring:
  profiles: eureka-server2
eureka:
  instance:
    hostname: 10.0.0.104
  client:
    service-url:
      defaultZone: http://10.0.0.103:10086/eureka/,http://10.0.0.104:10086/eureka/

注意：在启动微服务的时候，加入参数spring.profiles.active来读取对应的配置

2）其他微服务配置

除了Eureka注册中心以外，其他微服务配置都需要加入所有Eureka服务

server:
  port: 9001
spring:
  application:
    name: tensquare-admin-service #指定服务名
  datasource:
    driverClassName: com.mysql.jdbc.Driver
    url: jdbc:mysql://10.0.0.102:3306/tensquare_user?characterEncoding=UTF8
    username: root
    password: Root@123
  jpa:
    database: mysql
    show-sql: true

#Eureka配置
eureka:
  client:
    service-url:
      defaultZone: http://10.0.0.103:10086/eureka,http://10.0.0.104:10086/eureka
  instance:
    lease-renewal-interval-in-seconds: 5 # 每隔5秒发送一次心跳
    lease-expiration-duration-in-seconds: 10 # 10秒不发送就过期
    prefer-ip-address: true


 # jwt参数
jwt:
  config:
    key: itcast
    ttl: 1800000

2.2.2、设计Jenkins集群项目的构建参数

1）安装Extended Choice Parameter插件，支持多选框

2）创建流水线项目

3）配置流水线，添加参数

添加字符串参数：分支名称

添加多选框参数：项目名称

最后效果：

2.2.3、微服务构建镜像上传Harbor

修改Jenkinsfile

//gitlab的凭证
def git_auth = "cf17ff40-5824-4b2d-bdd5-784560255001"
//gitlab仓库地址
def git_url = "git@10.0.0.101:dianchou_group/tensqure_back.git"
//镜像的版本号
def tag = "latest"
//Harbor的url地址
def harbor_url = "10.0.0.101:85"
//镜像库项目名称
def harbor_project = "dianchou"
//Harbor的登录凭证ID
def harbor_auth = "86beb97d-7de6-4dc2-ab10-bc199d9eda97"

node {
    //获取当前选择的项目名称
    def selectedProjectNames = "${project_name}".split(",")

    stage('拉取代码') {
        checkout([$class: 'GitSCM', branches: [[name: "*/${branch}"]], extensions: [], userRemoteConfigs: [[credentialsId: "${git_auth}", url: "${git_url}"]]])
    }

    stage('代码审查') {
         sh "echo ${selectedProjectNames}"
        //定义当前Jenkins的SonarQubeScanner工具
        def scannerHome = tool 'sonarqube-scanner'
        for(int i=0;i<selectedProjectNames.length;i++){
            //tensquare_eureka_server@10086
            def projectInfo = selectedProjectNames[i];
            //当前遍历的项目名称
            def currentProjectName = "${projectInfo}".split("@")[0]
            //当前遍历的项目端口
            def currentProjectPort = "${projectInfo}".split("@")[1]

            //引用当前JenkinsSonarQube环境
            withSonarQubeEnv('sonarqube') {
                 sh """
                         cd ${currentProjectName}
                         ${scannerHome}/bin/sonar-scanner
                 """
            }
        }
    }

    stage('编译，安装公共子工程') {
        sh "mvn -f tensquare_common clean install"
    }

    stage('编译，打包微服务工程，上传镜像') {
       for(int i=0;i<selectedProjectNames.length;i++){
            //tensquare_eureka_server@10086
            def projectInfo = selectedProjectNames[i];
            //当前遍历的项目名称
            def currentProjectName = "${projectInfo}".split("@")[0]
            //当前遍历的项目端口
            def currentProjectPort = "${projectInfo}".split("@")[1]

            sh "mvn -f ${currentProjectName} clean package dockerfile:build"
            //定义镜像名称
            def imageName = "${currentProjectName}:${tag}"
            //给镜像打标签
            sh "docker tag ${imageName} ${harbor_url}/${harbor_project}/${imageName}"
            //登录harbor并上传镜像
            withCredentials([usernamePassword(credentialsId:  "${harbor_auth}", passwordVariable: 'password', usernameVariable: 'username')]) {
                //登录
                sh "docker login -u ${username} -p ${password} ${harbor_url}"
                //上传镜像
                sh "docker push ${harbor_url}/${harbor_project}/${imageName}"
                sh "echo 上传镜像成功"
            }
            //删除本地镜像
            sh "docker rmi -f ${imageName}"
            sh "docker rmi -f ${harbor_url}/${harbor_project}/${imageName}"
       }
    }

    stage('部署应用') {

    }
}

2.2.4、远程部署多服务器

1）jenkins配置远程服务器

拷贝公钥到远程服务器

[root@jenkins ~]# ssh-copy-id 10.0.0.103
[root@jenkins ~]# ssh-copy-id 10.0.0.104

系统配置->添加远程服务器

2）安装docker并配置

[root@hadoop104 ~]# cat /etc/docker/daemon.json 
{
  "registry-mirrors": ["https://t09ww3n9.mirror.aliyuncs.com"],
  "insecure-registries": ["10.0.0.101:85"]
}

3）添加参数

添加多选框：部署服务器

最终效果：

4）修改Jenkinfile

//gitlab的凭证
def git_auth = "cf17ff40-5824-4b2d-bdd5-784560255001"
//gitlab仓库地址
def git_url = "git@10.0.0.101:dianchou_group/tensqure_back.git"
//镜像的版本号
def tag = "latest"
//Harbor的url地址
def harbor_url = "10.0.0.101:85"
//镜像库项目名称
def harbor_project = "dianchou"
//Harbor的登录凭证ID
def harbor_auth = "86beb97d-7de6-4dc2-ab10-bc199d9eda97"

node {
    //获取当前选择的项目名称
    def selectedProjectNames = "${project_name}".split(",")
    //获取当前选择的服务器名称
    def selectedServers = "${publish_server}".split(",")

    stage('拉取代码') {
        checkout([$class: 'GitSCM', branches: [[name: "*/${branch}"]], extensions: [], userRemoteConfigs: [[credentialsId: "${git_auth}", url: "${git_url}"]]])
    }

    stage('代码审查') {
         sh "echo ${selectedProjectNames}"
        //定义当前Jenkins的SonarQubeScanner工具
        def scannerHome = tool 'sonarqube-scanner'
        for(int i=0;i<selectedProjectNames.length;i++){
            //tensquare_eureka_server@10086
            def projectInfo = selectedProjectNames[i];
            //当前遍历的项目名称
            def currentProjectName = "${projectInfo}".split("@")[0]
            //当前遍历的项目端口
            def currentProjectPort = "${projectInfo}".split("@")[1]

            //引用当前JenkinsSonarQube环境
            withSonarQubeEnv('sonarqube') {
                 sh """
                         cd ${currentProjectName}
                         ${scannerHome}/bin/sonar-scanner
                 """
            }
        }
    }

    stage('编译，安装公共子工程') {
        sh "mvn -f tensquare_common clean install"
    }

    stage('编译，打包微服务工程，上传镜像') {
       for(int i=0;i<selectedProjectNames.length;i++){
            //tensquare_eureka_server@10086
            def projectInfo = selectedProjectNames[i];
            //当前遍历的项目名称
            def currentProjectName = "${projectInfo}".split("@")[0]
            //当前遍历的项目端口
            def currentProjectPort = "${projectInfo}".split("@")[1]

            sh "mvn -f ${currentProjectName} clean package dockerfile:build"
            //定义镜像名称
            def imageName = "${currentProjectName}:${tag}"
            //给镜像打标签
            sh "docker tag ${imageName} ${harbor_url}/${harbor_project}/${imageName}"
            //登录harbor并上传镜像
            withCredentials([usernamePassword(credentialsId:  "${harbor_auth}", passwordVariable: 'password', usernameVariable: 'username')]) {
                //登录
                sh "docker login -u ${username} -p ${password} ${harbor_url}"
                //上传镜像
                sh "docker push ${harbor_url}/${harbor_project}/${imageName}"
                sh "echo 上传镜像成功"
            }
            //删除本地镜像
            sh "docker rmi -f ${imageName}"
            sh "docker rmi -f ${harbor_url}/${harbor_project}/${imageName}"

            //遍历所有服务器，分别部署
            for(int j=0;j<selectedServers.length;j++){
                   //获取当前遍历的服务器名称
                   def currentServerName = selectedServers[j]

                   //加上的参数格式：--spring.profiles.active=eureka-server1/eureka-server2
                   def activeProfile = "--spring.profiles.active="

                   //根据不同的服务名称来读取不同的Eureka配置信息
                   if(currentServerName=="master_server"){
                      activeProfile = activeProfile+"eureka-server1"
                   }else if(currentServerName=="slave_server"){
                      activeProfile = activeProfile+"eureka-server2"
                   }

                   //部署应用
                   sshPublisher(publishers: [sshPublisherDesc(configName: "${currentServerName}", transfers: [sshTransfer(cleanRemote: false, excludes: '', execCommand: "/opt/jenkins_shell/deployCluster.sh $harbor_url $harbor_project $currentProjectName $tag $currentProjectPort $activeProfile", execTimeout: 120000, flatten: false, makeEmptyDirs: false, noDefaultExcludes: false, patternSeparator: '[, ]+', remoteDirectory: '', remoteDirectorySDF: false, removePrefix: '', sourceFiles: '')], usePromotionTimestamp: false, useWorkspaceInPromotion: false, verbose: false)])
                   echo "部署成功"
            }
       }
    }

}

5）编写deployCluster.sh部署脚本

[root@hadoop104 jenkins_shell]# pwd
/opt/jenkins_shell
[root@hadoop104 jenkins_shell]# cat deployCluster.sh
#! /bin/sh
#接收外部参数
harbor_url=$1
harbor_project_name=$2
project_name=$3
tag=$4
port=$5
profile=$6

imageName=$harbor_url/$harbor_project_name/$project_name:$tag

echo "$imageName"

#查询容器是否存在，存在则删除
containerId=`docker ps -a | grep -w ${project_name}:${tag}  | awk '{print $1}'`

if [ "$containerId" !=  "" ] ; then
    #停掉容器
    docker stop $containerId

    #删除容器
    docker rm $containerId
	
	echo "成功删除容器"
fi

#查询镜像是否存在，存在则删除
imageId=`docker images | grep -w $project_name  | awk '{print $3}'`

if [ "$imageId" !=  "" ] ; then
      
    #删除镜像
    docker rmi -f $imageId
	
	echo "成功删除镜像"
fi


# 登录Harbor
docker login -u dianchou -p Dianchou123 $harbor_url
# 下载镜像
docker pull $imageName

# 启动容器
docker run -di -p $port:$port $imageName $profile

echo "容器启动成功"

6）最终结果

2.2.5、Nginx+Zuul集群实现高可用网关

1）安装nginx并配置

[root@tomcat ~]# cat /etc/nginx/conf.d/tensquare_font.conf 
upstream zuulServer {
    server 10.0.0.103:10020 weight=1;
    server 10.0.0.104:10020 weight=1;
}
server {
    listen       85;
    server_name  _;
    root   /usr/share/nginx/html;

    location / {
	proxy_pass http://zuulServer/;
    }

}

[root@tomcat ~]# nginx -t
[root@tomcat ~]# nginx -s reload

2）修改前端代码访问后台的访问地址，设置为nginx的负载均衡地址