gitlab mysql 坑,GitLab搭建及踩坑

服务器环境:

OS: Ubuntu18.04-LTS

CPU: 2核

内存: 4G

Docker搭建GitLab

拉取gitlab-ce镜像

root@iZ2ze26pixxe9t9kmg6tvhZ:# docker pull gitlab/gitlab-ce

创建gitlab容器的数据映射目录

其中config目录映射gitlab容器的配置文件目录,

data目录映射gitlab容器的数据目录,

logs目录映射gitlab容器的日志目录

root@iZ2ze26pixxe9t9kmg6tvhZ:/home# mkdir gitlab

root@iZ2ze26pixxe9t9kmg6tvhZ:/home# cd gitlab

root@iZ2ze26pixxe9t9kmg6tvhZ:/home/gitlab# mkdir data config logs

root@iZ2ze26pixxe9t9kmg6tvhZ:/home/gitlab# ls

config data logs

root@iZ2ze26pixxe9t9kmg6tvhZ:/home/gitlab#

创建gitlab容器

gitlab容器一共暴露了三个端口: 22/80/443

由于未使用SSL证书, 443端口可以不映射

22端口用于ssh传输, 也可以不映射

只使用容器暴露的80端口, 映射到宿主机的83端口

root@iZ2ze26pixxe9t9kmg6tvhZ:/home/gitlab# docker run -d -p 83:80 --name mygitlab -- restart always -v /home/gitlab/config/:/etc/gitlab -v /home/gitlab/data/:/var/opt/gitlab -v /home/gitlab/logs/:/var/log/gitlab gitlab/gitlab-ce:latest

修改gitlab的配置文件

gitlab的配置文件为容器内的/etc/gitlab/gitlab.rb

由于映射到了容器外的/home/gitlab/config目录下, 可以直接在容器外修改:

root@iZ2ze26pixxe9t9kmg6tvhZ:/home/gitlab/config# vim gitlab.rb

gitlab.rb配置文件默认是全注释的.

修改配置文件中对外暴露的地址:

## GitLab URL

##! URL on which GitLab will be reachable.

##! For more details on configuring external_url see:

##! https://docs.gitlab.com/omnibus/settings/configuration.html

#configuring-the-external-url-for-gitlab

##!

##! Note: During installation/upgrades, the value of the environment variable

##! EXTERNAL_URL will be used to populate/replace this value.

##! On AWS EC2 instances, we also attempt to fetch the public hostname/IP

##! address from AWS. For more details, see:

##! https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html

external_url 'http://gitlab.zack.net.cn'

修改宿主机的nginx配置文件, 添加端口转发规则

新增一条端口转发规则, 将主机名为gitlab.zack.net.cn的80端口请求转发到gitlab容器映射的83端口,

然后重载nginx配置文件

server{

listen 80;

server_name gitlab.zack.net.cn;

location /{

# HOST配置以及域名传递

proxy_set_header Host $host;

proxy_set_header X-Real-IP $remote_addr;

proxy_set_header REMOTE-HOST $remote_addr;

# 不使用服务端端口响应

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

# 转发到83端口

proxy_pass http://127.0.0.1:83/;

}

}

重启gitlab容器进行访问

root@iZ2ze26pixxe9t9kmg6tvhZ:/home/gitlab# docker restart mygitlab

等待重启后访问http://gitlab.zack.net.cn, 设置root账户密码,然后登陆gitlab

Gitlab踩坑

服务器内存不足导致的卡顿问题

gitlab一般要求4核4G内存的起步配置, 对内存消耗较大, 这也是功能强大的原因.由于服务器为2核4G, 同时运行的还有其他服务, 导致gitlab运行期间服务器卡顿, 每次pull/push操作都会失去响应几分钟, CPU占用100%, 内存占满, ssh都会被强行断开连接.

解决办法:设置一个较大的内存交换区, 在内存紧张时将不活跃的内存区域释放出来, 存到交换区

查看系统swap区

这里是已经设置过的swap区, 大小为4G, 云服务器swap区默认没有或者较小

root@iZ2ze26pixxe9t9kmg6tvhZ:~# sudo swapon -s

Filename Type Size Used Priority

/mnt/swap file 4194304 84224 -2

root@iZ2ze26pixxe9t9kmg6tvhZ:~#

禁用swap区

如果存在swap区, 可以先禁用

如果swap没有, 无需禁用,直接创建即可

root@iZ2ze26pixxe9t9kmg6tvhZ:~# sudo swapoff /swapfile

创建swap区文件

swap区大小为 bs * count

一般情况下为/mnt/swap, 也有可能为/data/swap

root@iZ2ze26pixxe9t9kmg6tvhZ:~# dd if=/dev/zero of=/mnt/swap bs=512 count=8388616

将创建的swap分区文件做成swap分区

root@iZ2ze26pixxe9t9kmg6tvhZ:~# mkswap /mnt/swap

查看内核参数vm.swappiness

root@iZ2ze26pixxe9t9kmg6tvhZ:~# cat /proc/sys/vm/swappiness

如果为0, 根据实际需要设置为60

root@iZ2ze26pixxe9t9kmg6tvhZ:~# sysctl -w vm.swappiness=60

启用swap分区

root@iZ2ze26pixxe9t9kmg6tvhZ:~# swapon /mnt/swap

再次检查swap分区情况

root@iZ2ze26pixxe9t9kmg6tvhZ:~# sudo swapon -s

Filename Type Size Used Priority

/mnt/swap file 4194304 84224 -2

root@iZ2ze26pixxe9t9kmg6tvhZ:~#

内存消耗过大导致的卡顿问题

随着gitlab服务启动的时间越来越长, 其内存占用越来越大, 内存也越来越紧张

未启动gitlab时

root@iZ2ze26pixxe9t9kmg6tvhZ:~# free -h

total used free shared buff/cache available

Mem: 3.9G 958M 2.3G 304K 621M 2.7G

Swap: 4.0G 98M 3.9G

root@iZ2ze26pixxe9t9kmg6tvhZ:~#

刚启动时, 内存还够用

root@iZ2ze26pixxe9t9kmg6tvhZ:~# free -h

total used free shared buff/cache available

Mem: 3.9G 2.3G 182M 27M 1.3G 1.2G

Swap: 4.0G 98M 3.9G

root@iZ2ze26pixxe9t9kmg6tvhZ:~#

使用一段时间后, 内存开始紧张

root@iZ2ze26pixxe9t9kmg6tvhZ:~# free -h

total used free shared buff/cache available

Mem: 3.9G 3.1G 171M 40M 554M 552MSwap: 4.0G 101M 3.9G

root@iZ2ze26pixxe9t9kmg6tvhZ:~#

查看系统资源使用情况, 发现gitlab占用了超过80%的内存

root@iZ2ze26pixxe9t9kmg6tvhZ:~# ps aux|head -1;ps aux|grep -v PID|sort -rn -k +4|head

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND

root 18176 0.1 15.2 3544740 616416 ? Sl Apr15 2:06 java -Djava.security.egd=file:/dev/./urandom -jar -Duser.timezone=GMT+08 /app.jar

998 5460 0.5 15.1 1096440 611368 ? Sl 15:31 0:02 unicorn worker[3] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru

998 4774 0.2 14.4 1036824 584224 ? Sl 15:27 0:01 unicorn worker[1] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru

998 5851 0.4 14.0 1036824 568900 ? Sl 15:34 0:00 unicorn worker[2] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru

998 30297 0.0 13.9 1029252 562264 ? Sl Apr15 0:49 unicorn master -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru

998 6200 0.3 13.8 1034660 559924 ? Sl 15:37 0:00 unicorn worker[0] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru

998 30153 0.7 13.6 1177528 552084 ? Ssl Apr15 9:33 sidekiq 5.2.7 gitlab-rails [0 of 8 busy]

999 1333 0.0 4.8 1598884 195808 ? Ssl Apr15 1:05 mysqld

992 30150 0.5 4.6 689920 187392 ? Ssl Apr15 6:23 /opt/gitlab/embedded/bin/prometheus --web.listen-address=localhost:9090 --storage.tsdb.path=/var/opt/gitlab/prometheus/data --config.file=/var/opt/gitlab/prometheus/prometheus.yml

998 30316 0.1 1.9 1342916 77756 ? Sl Apr15 1:35 ruby /opt/gitlab/embedded/service/gitaly-ruby/bin/gitaly-ruby 398 /var/opt/gitlab/gitaly/internal_sockets/ruby.1

root@iZ2ze26pixxe9t9kmg6tvhZ:~#

解决办法: 限制gitlab对内存的使用

修改映射到宿主机的gitlab配置文件

root@iZ2ze26pixxe9t9kmg6tvhZ:~# vim /home/gitlab/config/gitlab.rb

# 修改可以使用的缓存大小200MB-300MB

unicorn['worker_memory_limit_min'] = "200 * 1 << 20"

unicorn['worker_memory_limit_max'] = "300 * 1 << 20"

# 减少sidekiq并发数

sidekiq['concurrency'] = 6

# 修改数据库缓存大小

postgresql['shared_buffers'] = "128MB"

# 修改数据库并发数

postgresql['max_worker_processes'] = 6

# 超时时间

unicorn['worker_timeout'] = 60

# 修改为和cpu核心数一致

unicorn['work_processes'] = 2