ubuntu 16.04.2 源码安装gitlab并且利用runner持续集成

参考原档：https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md#using-https

本章只写重要的

Gitlab安装包括设置一下组件：

1. Packages / Dependencies
2. Ruby
3. Go
4. Node
5. System Users
6. Database
7. Redis
8. GitLab
9. Nginx

1. Packages / Dependencies（可有可无的更新）

# run as root!
apt-get update -y
apt-get upgrade -y
apt-get install sudo -y

安装所需的软件包（需要编译Ruby和Ruby gem的本机扩展）：

sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate python-docutils pkg-config cmake

如果要使用Kerberos进行用户身份验证，请安装libkrb5-dev： 

sudo apt-get install libkrb5-dev

注意：如果你不知道Kerberos是什么，你可以假设你不需要它。

确保您安装了正确版本的Git

# Install Git
sudo apt-get install -y git-core

# Make sure Git is version 2.8.4 or higher
git --version

系统打包Git是否太老了？从源文件中删除并编译。 

# Remove packaged Git
sudo apt-get remove git-core

# Install dependencies
sudo apt-get install -y libcurl4-openssl-dev libexpat1-dev gettext libz-dev libssl-dev build-essential

# Download and compile from source
cd /tmp
curl --remote-name --progress https://www.kernel.org/pub/software/scm/git/git-2.8.4.tar.gz
echo '626e319f8a24fc0866167ea5f6bf3e2f38f69d6cb2e59e150f13709ca3ebf301  git-2.8.4.tar.gz' | shasum -a256 -c - && tar -xzf git-2.8.4.tar.gz
cd git-2.8.4/
./configure
make prefix=/usr/local all

# Install into /usr/local/bin
sudo make prefix=/usr/local install

# When editing config/gitlab.yml (Step 5), change the git -> bin_path to /usr/local/bin/git

注意：为了接收邮件通知，请确保安装邮件服务器。默认情况下，Debian随exim4一起提供，但Ubuntu并没有出现问题。推荐的邮件服务器是后缀，您可以安装它：

sudo apt-get install -y postfix

然后选择“Internet站点”，然后按确认确认主机名。

2.Ruby

注意：目前支持的Ruby（MRI）版本为2.3.x。GitLab 9.0支持Ruby 2.1.x。

查看Ruby的版本：

sudo ruby -v

删除旧的Ruby 1.8如果存在：

sudo apt-get remove ruby1.8

下载Ruby并编译它： 

mkdir /tmp/ruby && cd /tmp/ruby
curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.3.tar.gz
echo '1014ee699071aa2ddd501907d18cbe15399c997d  ruby-2.3.3.tar.gz' | shasum -c - && tar xzf ruby-2.3.3.tar.gz
cd ruby-2.3.3
./configure --disable-install-rdoc
make
sudo make install

然后安装Bundler Gem： 

sudo gem install bundler --no-ri --no-rdoc

3.Go

# Remove former Go installation folder
sudo rm -rf /usr/local/go

curl --remote-name --progress https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz
echo '1862f4c3d3907e59b04a757cfda0ea7aa9ef39274af99a784f5be843c80c6772  go1.8.3.linux-amd64.tar.gz' | shasum -a256 -c - && 
  sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz
sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/
rm go1.8.3.linux-amd64.tar.gz

4.Node

由于GitLab 8.17，GitLab需要使用node> = v4.3.0来编译javascript资源，而且使用“>> = v0.17.0来管理javascript依赖关系。在许多发行版中，官方软件包版本提供的版本已过期，因此我们需要通过以下命令进行安装：

# install node v7.x
curl --location https://deb.nodesource.com/setup_7.x | sudo bash -
sudo apt-get install -y nodejs

# install yarn
curl --location https://yarnpkg.com/install.sh | bash -

如果如上述安装时出现问题，就去官网查看相应的地方，会有解决方法，相信我，我成功了。

node:https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions

yarn:https://yarnpkg.com/en/docs/install/

5.System Users

git为GitLab 创建一个用户：

sudo adduser --disabled-login --gecos 'GitLab' git

6.Database

我们建议使用PostgreSQL数据库。对于MySQL，请查看MySQL安装指南,官网相应的地方也有链接。

1.安装数据库包：

sudo apt-get install -y postgresql postgresql-client libpq-dev postgresql-contrib

2.为GitLab创建数据库用户：

sudo -u postgres psql -d template1 -c "CREATE USER git CREATEDB;"

3.创建pg_trgm扩展名（GitLab 8.6+需要）：

sudo -u postgres psql -d template1 -c "CREATE EXTENSION IF NOT EXISTS pg_trgm;"

4. 创建GitLab生产数据库并授予数据库所有权限： 

sudo -u postgres psql -d template1 -c "CREATE DATABASE gitlabhq_production OWNER git;"

5.尝试使用新用户连接到新数据库：

sudo -u git -H psql -d gitlabhq_production

6.检查pg_trgm分机是否启用：

SELECT true AS enabled
FROM pg_available_extensions
WHERE name = 'pg_trgm'
AND installed_version IS NOT NULL;

如果分机启用，则会产生以下输出：　

enabled
---------
 t
(1 row)

7.退出数据库会话：

gitlabhq_production> q

7.Redis：

GitLab至少需要Redis 2.8。 

如果您使用的是Debian 8或Ubuntu 14.04及更高版本，那么您可以简单地安装Redis 2.8： 

sudo apt-get install redis-server

如果您使用的是Debian 7或Ubuntu 12.04，请遵循备用Redis安装的特殊文档（官网对应处）。完成后，请遵循指南的其余部分。

# Configure redis to use sockets
sudo cp /etc/redis/redis.conf /etc/redis/redis.conf.orig

# Disable Redis listening on TCP by setting 'port' to 0
sed 's/^port .*/port 0/' /etc/redis/redis.conf.orig | sudo tee /etc/redis/redis.conf

# Enable Redis socket for default Debian / Ubuntu path
echo 'unixsocket /var/run/redis/redis.sock' | sudo tee -a /etc/redis/redis.conf

# Grant permission to the socket to all members of the redis group
echo 'unixsocketperm 770' | sudo tee -a /etc/redis/redis.conf

# Create the directory which contains the socket
mkdir /var/run/redis
chown redis:redis /var/run/redis
chmod 755 /var/run/redis

# Persist the directory which contains the socket, if applicable
if [ -d /etc/tmpfiles.d ]; then
  echo 'd  /var/run/redis  0755  redis  redis  10d  -' | sudo tee -a /etc/tmpfiles.d/redis.conf
fi

# Activate the changes to redis.conf
sudo service redis-server restart

# Add git to the redis group
sudo usermod -aG redis git

8.GitLab

# We'll install GitLab into home directory of the user "git"
cd /home/git

克隆来源

# Clone GitLab repository
sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 8-7-stable gitlab

版本不要装最新的，最好是差1~2个大版本。　

# Go to GitLab installation folder
cd /home/git/gitlab

# Copy the example GitLab config
sudo -u git -H cp config/gitlab.yml.example config/gitlab.yml

# Update GitLab config file, follow the directions at top of file
sudo -u git -H editor config/gitlab.yml
#修改host:localhost 为host:你的域名或者ip，我改的是192.168.222.131
#修改bin_path:/usr/bin/git 为bin_path:/usr/local/bin/git

# Copy the example secrets file
sudo -u git -H cp config/secrets.yml.example config/secrets.yml
sudo -u git -H chmod 0600 config/secrets.yml

# Make sure GitLab can write to the log/ and tmp/ directories
sudo chown -R git log/
sudo chown -R git tmp/
sudo chmod -R u+rwX,go-w log/
sudo chmod -R u+rwX tmp/

# Make sure GitLab can write to the tmp/pids/ and tmp/sockets/ directories
sudo chmod -R u+rwX tmp/pids/
sudo chmod -R u+rwX tmp/sockets/

# Create the public/uploads/ directory
sudo -u git -H mkdir public/uploads/

# Make sure only the GitLab user has access to the public/uploads/ directory
# now that files in public/uploads are served by gitlab-workhorse
sudo chmod 0700 public/uploads

# Change the permissions of the directory where CI job traces are stored
sudo chmod -R u+rwX builds/

# Change the permissions of the directory where CI artifacts are stored
sudo chmod -R u+rwX shared/artifacts/

# Change the permissions of the directory where GitLab Pages are stored
sudo chmod -R ug+rwX shared/pages/

# Copy the example Unicorn config
sudo -u git -H cp config/unicorn.rb.example config/unicorn.rb

# Find number of cores
nproc

# Enable cluster mode if you expect to have a high load instance
# Set the number of workers to at least the number of cores
# Ex. change amount of workers to 3 for 2GB RAM server
sudo -u git -H editor config/unicorn.rb

# Copy the example Rack attack config
sudo -u git -H cp config/initializers/rack_attack.rb.example config/initializers/rack_attack.rb

# Configure Git global settings for git user
# 'autocrlf' is needed for the web editor
sudo -u git -H git config --global core.autocrlf input

# Disable 'git gc --auto' because GitLab already runs 'git gc' when needed
sudo -u git -H git config --global gc.auto 0

# Enable packfile bitmaps
sudo -u git -H git config --global repack.writeBitmaps true

# Configure Redis connection settings
sudo -u git -H cp config/resque.yml.example config/resque.yml

# Change the Redis socket path if you are not using the default Debian / Ubuntu configuration
sudo -u git -H editor config/resque.yml

重要注意事项：请确保同时编辑gitlab.yml和unicorn.rb匹配您的设置。　　

配置GitLab数据库设置 

# PostgreSQL only:
sudo -u git cp config/database.yml.postgresql config/database.yml

# MySQL only:
sudo -u git cp config/database.yml.mysql config/database.yml

# MySQL and remote PostgreSQL only:
# Update username/password in config/database.yml.
# You only need to adapt the production settings (first part).
# If you followed the database guide then please do as follows:
# Change 'secure password' with the value you have given to $password
# You can keep the double quotes around the password
sudo -u git -H editor config/database.yml

# PostgreSQL and MySQL:
# Make config/database.yml readable to git only
sudo -u git -H chmod o-rwx config/database.yml

Install Gems

 注意：从捆绑器1.5.2开始，您可以调用bundle install -jN（N处理器内核的数量），并享受并行宝石安装，并在完成时间内可以有显着差异（约60％）。检查你的核心数量nproc。有关更多信息，请查看此信息。首先确保你有一个bundle => 1.5.2（run bundle -v），因为它解决了在1.5.2 中修复的一些问题。

# For PostgreSQL (note, the option says "without ... mysql")（过程会比较慢，不要急，会成功的，网上说的那些都是骗人的。）
sudo -u git -H bundle install --deployment --without development test mysql aws kerberos

# Or if you use MySQL (note, the option says "without ... postgres")
sudo -u git -H bundle install --deployment --without development test postgres aws kerberos

注意：如果要使用Kerberos进行用户身份验证，请kerberos在上述--without选项中省略。

　

安装GitLab Shell　

GitLab Shell是专门为GitLab开发的SSH访问和存储库管理软件。

# Run the installation task for gitlab-shell (replace `REDIS_URL` if needed):
sudo -u git -H bundle exec rake gitlab:shell:install REDIS_URL=unix:/var/run/redis/redis.sock RAILS_ENV=production SKIP_STORAGE_VALIDATION=true

# By default, the gitlab-shell config is generated from your main GitLab config.
# You can review (and modify) the gitlab-shell config as follows:
sudo -u git -H editor /home/git/gitlab-shell/config.yml（此处修改时需要将gitlab_url:http://127.0.0.1:8080/）

安装gitlab-workhorse

GitLab-Workhorse使用GNU Make。以下命令行将安装GitLab-Workhorse，/home/git/gitlab-workhorse其中是推荐的位置。 

sudo -u git -H bundle exec rake "gitlab:workhorse:install[/home/git/gitlab-workhorse]" RAILS_ENV=production（这个安装方式会出现问题，然后我从网上找了另一个方式）
在gitlab的上层目录git下安装：
sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-workhorse.git
cd gitlab-workhorse 
sudo -u git -H make

您可以通过将其作为额外参数来指定不同的Git仓库： 

sudo -u git -H bundle exec rake "gitlab:workhorse:install[/home/git/gitlab-workhorse,https://example.com/gitlab-workhorse.git]" RAILS_ENV=production

初始化数据库并激活高级功能 

sudo -u git -H bundle exec rake gitlab:setup RAILS_ENV=production

# Type 'yes' to create the database tables.

# When done you see 'Administrator account created:'

注意：您可以通过在环境变量提供他们设置管理员/ root密码和电子邮件，GITLAB_ROOT_PASSWORD并GITLAB_ROOT_EMAIL分别如下所示。如果您没有设置密码（并且设置为默认密码），请等待公开GitLab到公共网络，直到安装完成，并且您已经首次登录到服务器。在第一次登录时，您将被迫更改默认密码。

sudo -u git -H bundle exec rake gitlab:setup RAILS_ENV=production GITLAB_ROOT_PASSWORD=yourpassword GITLAB_ROOT_EMAIL=youremail（这个可以不设置，无所谓）

安装Init脚本　　

下载init脚本（将是/etc/init.d/gitlab）： 

sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab

如果您使用非默认文件夹或用户副本进行安装并编辑默认文件：

sudo cp lib/support/init.d/gitlab.default.example /etc/default/gitlab

如果您将GitLab安装在另一个目录或默认设置之外，您应该更改这些设置/etc/default/gitlab。不要编辑，/etc/init.d/gitlab因为它将在升级时更改。

 使GitLab启动时启动： 

sudo update-rc.d gitlab defaults 21

安装Gitaly 

# Fetch Gitaly source with Git and compile with Go
sudo -u git -H bundle exec rake "gitlab:gitaly:install[/home/git/gitaly]" RAILS_ENV=production

您可以通过将其作为额外参数来指定不同的Git仓库： 

sudo -u git -H bundle exec rake "gitlab:gitaly:install[/home/git/gitaly,https://example.com/gitaly.git]" RAILS_ENV=production

接下来，确保配置gitaly： 

# Restrict Gitaly socket access
sudo chmod 0700 /home/git/gitlab/tmp/sockets/private
sudo chown git /home/git/gitlab/tmp/sockets/private

# If you are using non-default settings you need to update config.toml
cd /home/git/gitaly
sudo -u git -H editor config.toml

如果你安装出错，请看后面信息，我安装的是gitlab8-7：Gitaly（在GitLab 9.0中引入）是一种为Git存储库提供高级RPC访问权限的服务。从GitLab 9.3起，它仍然是一个可选的组件，范围有限。

访问Git存储库（gitlab-rails，gitlab-shell，gitlab-workhorse）的GitLab组件充当Gitaly的客户端。最终用户无法直接访问Gitaly。

安装Logrotate 

sudo cp lib/support/logrotate/gitlab /etc/logrotate.d/gitlab

检查应用程序状态 

检查GitLab及其环境是否正确配置： 

sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production

编译资产　　

sudo -u git -H yarn install --production --pure-lockfile
sudo -u git -H bundle exec rake gitlab:assets:compile RAILS_ENV=production NODE_ENV=production（这个有可能会出错，但是问题不大）
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production（这个不错，如果没有尝试需要修复的信息就完成了）

编译GetText PO文件　

sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production（这个也是有问题的没事）

启动您的GitLab实例 

sudo service gitlab start
# or
sudo /etc/init.d/gitlab restart

Nginx　

注意： Nginx是GitLab正式支持的Web服务器。如果您不能或不想使用Nginx作为Web服务器，请查看GitLab配方。

 安装

sudo apt-get install -y nginx

站点配置 

复制示例站点配置：

sudo cp lib/support/nginx/gitlab /etc/nginx/sites-available/gitlab
sudo ln -s /etc/nginx/sites-available/gitlab /etc/nginx/sites-enabled/gitlab

确保编辑配置文件以匹配您的设置。另外，请确保您将路径与GitLab相匹配，特别是如果为“git”用户以外的用户安装： 

# Change YOUR_SERVER_FQDN to the fully-qualified
# domain name of your host serving GitLab.
#
# Remember to match your paths to GitLab, especially
# if installing for a user other than 'git'.
#
# If using Ubuntu default nginx install:
# either remove the default_server from the listen line
# or else sudo rm -f /etc/nginx/sites-enabled/default
sudo editor /etc/nginx/sites-available/gitlab（可以该一下server_name，我改的是本机的ip）

测试：

sudo nginx -t

如果此时报错如下：

nginx: [emerg] a duplicate default server for 0.0.0.0:80 in /etc/nginx/sites-enabled/gitlab:29
nginx: configuration file /etc/nginx/nginx.conf test failed

解决方法：

　　bill@bill:/home/git/gitlab$ sudo rm -rf /etc/nginx/sites-enabled/default
　　bill@bill:/home/git/gitlab$ sudo nginx -t
　　nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
　　nginx: configuration file /etc/nginx/nginx.conf test is successful

最后再检查一下：

sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production
并且重启服务：
sudo service nginx restart
sudo service gitlab restart
还有就是一定要关闭防火墙，除非你自己制定规则，省去麻烦直接关闭吧
sudo service ufw stop

接下来就是搭建gitlab-runner，持续集成

可以参考：https://gitlab.com/gitlab-org/gitlab-ci-multi-runner

https://docs.gitlab.com/runner/register/index.html

1.添加GitLab的官方资料库：

# For Debian/Ubuntu
curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-ci-multi-runner/script.deb.sh | sudo bash

# For RHEL/CentOS
curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-ci-multi-runner/script.rpm.sh | sudo bash

2.安装gitlab-ci-multi-runner：

# for DEB based systems
root@host:# apt-get install gitlab-ci-multi-runner=1.11.2

# for RPM based systems
root@host:# yum install gitlab-ci-multi-runner-1.11.2-1

3.注册

sudo gitlab-ci-multi-runner register
注册就可以看看官网的资料：https://docs.gitlab.com/runner/register/index.html

4.一些操作

参考：https://help.aliyun.com/document_detail/52857.html，还有自己新建一个项目的时候也会给出参数便于你设置。