zoukankan      html  css  js  c++  java

  • Jenkins+Ansible+Gitlab：通过curl自动推送文件&gitlab自动触发构建

    Jenkins通过curl自动推送文件&gitlab自动触发构建

    需求

    通过url参数，自动将gitlab文件推送到指定环境的机器目录下

    可用变量

    http://server/env-vars.html

    #执行脚本，查看输出内容
echo $BRANCH_NAME
#For a multibranch project, this will be set to the name of the branch being built, for example in case you wish to deploy to production from master but not from feature branches; if corresponding to some kind of change request, the name is generally arbitrary (refer to CHANGE_ID and CHANGE_TARGET).
echo $CHANGE_ID
#For a multibranch project corresponding to some kind of change request, this will be set to the change ID, such as a pull request number, if supported; else unset.
echo $CHANGE_URL
#For a multibranch project corresponding to some kind of change request, this will be set to the change URL, if supported; else unset.
echo $CHANGE_TITLE
#For a multibranch project corresponding to some kind of change request, this will be set to the title of the change, if supported; else unset.
echo $CHANGE_AUTHOR
#For a multibranch project corresponding to some kind of change request, this will be set to the username of the author of the proposed change, if supported; else unset.
echo $CHANGE_AUTHOR_DISPLAY_NAME
#For a multibranch project corresponding to some kind of change request, this will be set to the human name of the author, if supported; else unset.
echo $CHANGE_AUTHOR_EMAIL
#For a multibranch project corresponding to some kind of change request, this will be set to the email address of the author, if supported; else unset.
echo $CHANGE_TARGET
#For a multibranch project corresponding to some kind of change request, this will be set to the target or base branch to which the change could be merged, if supported; else unset.
echo $CHANGE_BRANCH
#For a multibranch project corresponding to some kind of change request, this will be set to the name of the actual head on the source control system which may or may not be different from BRANCH_NAME. For example in GitHub or Bitbucket this would have the name of the origin branch whereas BRANCH_NAME would be something like PR-24.
echo $CHANGE_FORK
#For a multibranch project corresponding to some kind of change request, this will be set to the name of the forked repo if the change originates from one; else unset.
echo $BUILD_NUMBER
#The current build number, such as "153"
echo $BUILD_ID
#The current build ID, identical to BUILD_NUMBER for builds created in 1.597+, but a YYYY-MM-DD_hh-mm-ss timestamp for older builds
echo $BUILD_DISPLAY_NAME
#The display name of the current build, which is something like "#153" by default.
echo $JOB_NAME
#Name of the project of this build, such as "foo" or "foo/bar".
echo $JOB_BASE_NAME
#Short Name of the project of this build stripping off folder paths, such as "foo" for "bar/foo".
echo $BUILD_TAG
#String of "jenkins-${JOB_NAME}-${BUILD_NUMBER}". All forward slashes ("/") in the JOB_NAME are replaced with dashes ("-"). Convenient to put into a resource file, a jar file, etc for easier identification.
echo $EXECUTOR_NUMBER
#The unique number that identifies the current executor (among executors of the same machine) that’s carrying out this build. This is the number you see in the "build executor status", except that the number starts from 0, not 1.
echo $NODE_NAME
#Name of the agent if the build is on an agent, or "master" if run on master
echo $NODE_LABELS
#Whitespace-separated list of labels that the node is assigned.
echo $WORKSPACE
#The absolute path of the directory assigned to the build as a workspace.
echo $JENKINS_HOME
#The absolute path of the directory assigned on the master node for Jenkins to store data.
echo $JENKINS_URL
#Full URL of Jenkins, like http://server:port/jenkins/ (note: only available if Jenkins URL set in system configuration)
echo $BUILD_URL
#Full URL of this build, like http://server:port/jenkins/job/foo/15/ (Jenkins URL must be set)
echo $JOB_URL
#Full URL of this job, like http://server:port/jenkins/job/foo/ (Jenkins URL must be set)
echo $GIT_COMMIT
#The commit hash being checked out.
echo $GIT_PREVIOUS_COMMIT
#The hash of the commit last built on this branch, if any.
echo $GIT_PREVIOUS_SUCCESSFUL_COMMIT
#The hash of the commit last successfully built on this branch, if any.
echo $GIT_BRANCH
#The remote branch name, if any.
echo $GIT_LOCAL_BRANCH
#The local branch name being checked out, if applicable.
echo $GIT_CHECKOUT_DIR
#The directory that the repository will be checked out to. This contains the value set in Checkout to a sub-directory, if used.
echo $GIT_URL
#The remote URL. If there are multiple, will be GIT_URL_1, GIT_URL_2, etc.
echo $GIT_COMMITTER_NAME
#The configured Git committer name, if any, that will be used for FUTURE commits from the current workspace. It is read from the Global Config user.name Value field of the Jenkins Configure System page.
echo $GIT_AUTHOR_NAME
#The configured Git author name, if any, that will be used for FUTURE commits from the current workspace. It is read from the Global Config user.name Value field of the Jenkins Configure System page.
echo $GIT_COMMITTER_EMAIL
#The configured Git committer email, if any, that will be used for FUTURE commits from the current workspace. It is read from the Global Config user.email Value field of the Jenkins Configure System page.
echo $GIT_AUTHOR_EMAIL
#The configured Git author email, if any, that will be used for FUTURE commits from the current workspace. It is read from the Global Config user.email Value field of the Jenkins Configure System page.

    工作目录 WORKSPACE

    配置

    安装jenkins、gitlab、ansible

    jenkins安装buildWithParameter插件、gitlab插件

    image

    image

    1.创建流水线项目

    image

    2.勾选 This project is parameterized

    image

    我添加了四个参数

    1.env：指定环境的主机组（prod、test）
2.path：文件路径（我设置默认值为${WORKSPACE} 即为工作目录，jenkins会将仓库克隆到此目录）
3.file：文件名
4.branch：分支名
5.dpath：目标主机目录（我设置默认值为/tmp，确保此目录jenkins账户有权限）

    3.添加gitlab源码

    添加需要添加gitlab用户名密码

    image

    4.添加token，用于使用url访问

    image

    5.配置构建执行的playbook

    image

    5.1.选择要执行的playbook文件（会在${WORKSPACE}中寻找该文件，即gitlab仓库目录下）

    5.2.填写hosts文件，即ansible主机组，可使用本地hosts文件

    [test]
10.130.97.116

[prod]
10.130.98.187

    5.3.填写需要给playbook传递的变量

    Extra Variables，可以使用This project is parameterized中设置的变量，用${变量名}引用

    image

    5.4.gitlab仓库创建playbook文件

    playbook中可以直接使用传递进来的变量

    image

    6.用curl触发jenkins

    在linux中使用curl

    curl http://10.130.97.94:8888/job/1111/buildWithParameters -X POST -d env=test -d token=jbc -d file=1.txt -d branch=master -d dpath=/tmp -v

    -X POST 使用post
-d 传递参数
-v 显示http信息

    image

    jenkins：

    image

    gitlab自动触发构建

    1.jenkins配置gitlab webhook

    1.1jenkins配置

    jenkins有许多触发项可选，但是目前只有使用下列选项测试成功

    Approved Merge Requests (EE-only)
Comments
Comment (regex) for triggering a build	
Allowed branches

    image

    image

    1.2gitlab配置

    image

    2.测试

    gitlab中新建一个分支，修改其中的文件

    image

    image

    image

    使用此方法变量需要设置默认值，否则执行playbook会出现错误

    3.jenkins完整配置

    image

    image

    总结

    利用jenkins+ansible+gitlab实现CICD
    通过可带参数触发实现差异性构建
    利用gitlab作为版本仓库及权限控制中心
    使用ansible自动化执行

    未涉及：回滚、健康检查

    可以利用git show获取comment信息，从而实现comment中描述构建参数。

    问题

    1.ERROR: Error fetching remote repo 'origin'&No such device or address

    报错内容

    Started by user admin
Running as SYSTEM
Building in workspace /var/lib/jenkins/workspace/1111
using credential 6de1623b-eb77-4f23-a603-ae730013d2b1
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url http://10.130.97.94:10000/root/test1.git # timeout=10
Fetching upstream changes from http://10.130.97.94:10000/root/test1.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials 80808080
 > git fetch --tags --progress http://10.130.97.94:10000/root/test1.git +refs/heads/*:refs/remotes/origin/* # timeout=10
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from http://10.130.97.94:10000/root/test1.git
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:899)
	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1114)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1145)
	at hudson.scm.SCM.checkout(SCM.java:504)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1209)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
	at hudson.model.Run.execute(Run.java:1816)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress http://10.130.97.94:10000/root/test1.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: /var/lib/jenkins/workspace/1111@tmp/jenkins-gitclient-pass6522325890806363235.sh:行3: cat: 未找到命令
error: unable to read askpass response from '/var/lib/jenkins/workspace/1111@tmp/jenkins-gitclient-pass6522325890806363235.sh'
fatal: could not read Username for 'http://10.130.97.94:10000': No such device or address

	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2372)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1985)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:80)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:563)
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:897)
	... 11 more
ERROR: Error fetching remote repo 'origin'
Finished: FAILURE

    解决方法：

    在请求串中加入身份信息即可，格式为：
https://[userName]:[password]@github.com/[username]/project.git

    image

    GitLab could not read Username for 'ip地址':No such device or address

    2.ansible 执行报错：No authentication methods available

    ansible 执行报错：No authentication methods available，本地可以执行

    报错信息

    [1111] $ ansible test -m copy -a "src=/var/lib/jenkins/workspace/1111/2.txt dest=/home" -f 5
10.130.97.116 | UNREACHABLE! => {
    "changed": false, 
    "msg": "No authentication methods available", 
    "unreachable": true
}
FATAL: command execution failed

    解决方法一

    在项目配置中，构建里的ansible选项，Credentials中添加目的主机的用户名密码即可

    image

    引入新问题

    image

    缺少-p

    image

    该方法暂时放弃

    解决方法二

    jenkins集成pipeline,ansible流程

    配置/etc/sudoers

    jenkins  ALL=(ALL) NOPASSWD: ALL

    修改/etc/passwd

    jenkins:x:990:985:Jenkins Automation Server:/var/lib/jenkins:/bin/bash

    切换用户jenkins
    su - jenkins

    生成ssh key

    ssh-keygen

    复制ssh key

    ssh-copy-id ip

    对方主机需要有该账号，useradd jenkins，设置密码，passwd jenkins。需注意目标主机jenkins权限问题。

    若yml中涉及到remote_user : root，需要去掉这句话，否则会报：fatal: [10.130.97.116]: UNREACHABLE! => {"changed": false, "msg": "Failed to authenticate: Authentication failed.", "unreachable": true}。原因无发用root身份登录。

    3.skipping: no hosts matched

    原因：没有指定hosts

    解决方法

    image

    或者

    使用-i hosts指定hosts文件

    ansible-playbook build.yml -i /etc/ansible/hosts -f 5 -e env=test -e file=321.txt -e path=/tmp
  • 相关阅读:
    XML-Signature 语法和签名
    ZooKeeper相关资料集锦
    分布式锁
    spring-boot 知识集锦
    Spring boot druid 多数据源配置
    常见 SQL 语句的加锁分析
    fastjson反序列化多层嵌套泛型类与java中的Type类型
    Clean ThreadLocals
    java AOP Before, After, AfterReturning, AfterThrowing, or Around 注解
    java 线程间的通信 （wait / notify / notifyAll）
  • 原文地址：https://www.cnblogs.com/boxker/p/12145062.html
Copyright © 2011-2022 走看看