docker版的gitlab优化

使用gitlab一年多,发现网页访问的时候,速度越来越慢,有些卡顿的现象,于是查看内存

[root@registry ~]# free -h
              total        used        free      shared  buff/cache   available
Mem:           7.6G        7.1G        181M        6.8M        413M        161M
Swap:           19G        4.4G         15G

分配多少内存都是被占光了

查看占内存的前20个进程

[root@registry ~]# ps -aux | sort -k4nr  |head -n20
systemd+  4260  3.9  6.6 1026812 531160 ?      Ssl  9月02 2524:28 sidekiq 5.0.0 gitlab-rails [0 of 25 busy]
systemd+  4764  0.1  6.2 840588 498768 ?       Sl   9月02  70:20 unicorn worker[5] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+  9700  0.1  5.9 840556 476208 ?       Sl   9月02  72:20 unicorn worker[7] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+ 27966  0.0  5.8 891788 467032 ?       Sl   9月02  60:43 unicorn worker[4] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+  4750  0.1  5.8 825200 472484 ?       Sl   9月02  68:57 unicorn worker[1] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+ 19789  0.1  5.7 851824 461636 ?       Sl   9月02  63:13 unicorn worker[3] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+ 22428  0.1  5.7 855920 462124 ?       Sl   9月02  68:55 unicorn worker[12] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+  4754  0.1  5.7 833424 458452 ?       Sl   9月02  72:37 unicorn worker[2] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+  4779  0.1  5.7 916332 463660 ?       Sl   9月02  69:55 unicorn worker[9] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+ 30351  0.1  5.6 852844 455156 ?       Sl   9月02  63:57 unicorn worker[10] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+  4773  0.1  5.6 836460 455892 ?       Sl   9月02  75:59 unicorn worker[8] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+ 15228  0.0  4.9 941808 393312 ?       Sl   9月04  56:43 unicorn worker[0] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+ 15474  0.0  4.8 887692 390956 ?       Sl   9月02  61:23 unicorn worker[11] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+ 18992  0.0  4.0 842604 325980 ?       Sl   9月16  38:43 unicorn worker[6] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+  4350  0.0  3.8 780916 309892 ?       Sl   9月02  29:29 unicorn master -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
200       4472  0.4  2.6 7893360 210820 ?      Ssl  9月02 305:54 /opt/java/bin/java -Dnexus-work=/sonatype-work -Dnexus-webapp-context-path=/nexus -Xms256m -Xmx768m -cp conf/:lib/* -server -Djava.net.preferIPv4Stack=true org.sonatype.nexus.bootstrap.Launcher ./conf/jetty.xml ./conf/jetty-requestlog.xml
systemd+  4223  0.6  1.7 2411336 140684 ?      Ssl  9月02 395:01 /opt/gitlab/embedded/bin/ruby /opt/gitlab/embedded/bin/gitlab-mon web -c /var/opt/gitlab/gitlab-monitor/gitlab-monitor.yml
992       4267  2.2  1.1 1272904 91580 ?       Ssl  9月02 1421:55 /opt/gitlab/embedded/bin/prometheus -web.listen-address=localhost:9090 -storage.local.path=/var/opt/gitlab/prometheus/data -storage.local.chunk-encoding-version=2 -storage.local.target-heap-size=190243389 -config.file=/var/opt/gitlab/prometheus/prometheus.yml
root      2755  0.5  0.2 2071148 18200 ?       Ssl  9月02 355:10 /usr/bin/dockerd
992       4224  0.3  0.1 1070980 12296 ?       Ssl  9月02 238:47 /opt/gitlab/embedded/bin/node_exporter -web.listen-address=localhost:9100 -collector.textfile.directory=/var/opt/gitlab/node-exporter/textfile_collector
[root@registry ~]#

发现gitlab的master有一个,worker却有12个之多。

使用自制的脚本,查看CPU信息

[root@registry shell]# vi checkCPU.sh
cpuname=$(cat /proc/cpuinfo | grep name | cut -f2 -d: | uniq -c)
physical=$(cat /proc/cpuinfo | grep "physical id" | sort -u | wc -l)
processor=$(cat /proc/cpuinfo | grep "processor" | wc -l)
cpucores=$(cat /proc/cpuinfo  | grep "cpu cores" | uniq)
siblings=$(cat /proc/cpuinfo  | grep "siblings"  | uniq)

echo "* * * * * CPU Information * * * * *"
echo "(CPU型号)cpu name : $cpuname"
echo "(物理CPU个数)physical id is : $physical"
echo "(逻辑CPU个数)processor is : $processor"
echo "(CPU内核数)cpu cores is : $cpucores"
echo "(单个物理CPU的逻辑CPU数)siblings is : $siblings"

结果:

[root@registry shell]# chmod 777 checkCPU.sh 
[root@registry shell]# ./checkCPU.sh 
* * * * * CPU Information * * * * *
(CPU型号)cpu name :      12  Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz
(物理CPU个数)physical id is : 3
(逻辑CPU个数)processor is : 12
(CPU内核数)cpu cores is : cpu cores   : 4
(单个物理CPU的逻辑CPU数)siblings is : siblings        : 4
[root@registry shell]# 

逻辑CPU个数12个,gitlab的worker就启动12个进程。

 

参考:https://blog.csdn.net/ouyang_peng/article/details/84066417

解决方法:

找到配置文件 gitlab.rb

################################################################################
## GitLab Unicorn
##! Tweak unicorn settings.
##! Docs: https://docs.gitlab.com/omnibus/settings/unicorn.html
################################################################################

# unicorn['worker_timeout'] = 60
###! Minimum worker_processes is 2 at this moment
###! See https://gitlab.com/gitlab-org/gitlab-ce/issues/18771
# unicorn['worker_processes'] = 2
unicorn['worker_processes'] = 4

启用unicorn['worker_processes'] = 4

然后重启gitlab的容器:

docker restart gitlab

优化后,

查看内存占用情况:

[root@registry ~]# free -h
              total        used        free      shared  buff/cache   available
Mem:           7.6G        3.5G        3.4G         14M        744M        3.6G
Swap:           19G        729M         18G

再次查看内存占用的前20个进程

[root@registry config]# ps -aux | sort -k4nr  |head -n20
systemd+ 16108  0.3  6.4 784104 516400 ?       Sl   11:34   0:30 unicorn worker[0] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+ 16111  0.4  6.4 770732 518064 ?       Sl   11:34   0:36 unicorn worker[1] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+ 16114  0.3  6.4 774848 514248 ?       Sl   11:34   0:35 unicorn worker[2] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+ 18809  0.3  6.4 772780 518484 ?       Sl   11:50   0:28 unicorn worker[3] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+ 15805  4.1  6.3 767600 505564 ?       Ssl  11:33   6:21 sidekiq 5.0.0 gitlab-rails [0 of 25 busy]
systemd+ 15931  0.4  4.5 600692 362812 ?       Sl   11:33   0:45 unicorn master -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
200       4472  0.4  2.6 7893360 209944 ?      Ssl  9月02 306:34 /opt/java/bin/java -Dnexus-work=/sonatype-work -Dnexus-webapp-context-path=/nexus -Xms256m -Xmx768m -cp conf/:lib/* -server -Djava.net.preferIPv4Stack=true org.sonatype.nexus.bootstrap.Launcher ./conf/jetty.xml ./conf/jetty-requestlog.xml
992      15783  1.3  0.8 962052 72076 ?        Ssl  11:33   2:02 /opt/gitlab/embedded/bin/prometheus -web.listen-address=localhost:9090 -storage.local.path=/var/opt/gitlab/prometheus/data -storage.local.chunk-encoding-version=2 -storage.local.target-heap-size=190243389 -config.file=/var/opt/gitlab/prometheus/prometheus.yml
systemd+ 15791  0.5  0.3 311568 29600 ?        Ssl  11:33   0:49 /opt/gitlab/embedded/bin/ruby /opt/gitlab/embedded/bin/gitlab-mon web -c /var/opt/gitlab/gitlab-monitor/gitlab-monitor.yml
992      15793  0.3  0.2 1049436 21344 ?       Ssl  11:33   0:32 /opt/gitlab/embedded/bin/node_exporter -web.listen-address=localhost:9100 -collector.textfile.directory=/var/opt/gitlab/node-exporter/textfile_collector
libstor+ 15784  0.1  0.2 773784 19312 ?        Ssl  11:33   0:13 /opt/gitlab/embedded/bin/postgres_exporter -web.listen-address=localhost:9187 -extend.query-path=/var/opt/gitlab/postgres-exporter/queries.yaml
root     15758  0.0  0.2  96216 21916 ?        Sl   11:33   0:00 /opt/gitlab/embedded/bin/ruby /opt/gitlab/embedded/bin/omnibus-ctl gitlab /opt/gitlab/embedded/service/omnibus-ctl tail
root      2755  0.5  0.2 2071148 22244 ?       Ssl  9月02 356:03 /usr/bin/dockerd
systemd+ 15790  0.0  0.2 787804 20584 ?        Ssl  11:33   0:01 /opt/gitlab/embedded/bin/gitaly /var/opt/gitlab/gitaly/config.toml
systemd+ 15794  0.0  0.2 1060584 16384 ?       Ssl  11:33   0:02 /opt/gitlab/embedded/bin/gitlab-workhorse -listenNetwork unix -listenUmask 0 -listenAddr /var/opt/gitlab/gitlab-workhorse/socket -authBackend http://localhost:8080 -authSocket /var/opt/gitlab/gitlab-rails/sockets/gitlab.socket -documentRoot /opt/gitlab/embedded/service/gitlab-rails/public -pprofListenAddr  -secretPath /opt/gitlab/embedded/service/gitlab-rails/.gitlab_workhorse_secret -config config.toml
libstor+ 15806  0.0  0.1  48932 12228 ?        Ss   11:33   0:01 /opt/gitlab/embedded/bin/postgres -D /var/opt/gitlab/postgresql/data
libstor+ 15927  0.1  0.1  58916  8164 ?        Ss   11:33   0:12 postgres: gitlab-psql postgres [local] idle
libstor+ 16163  0.0  0.1  65076 13212 ?        Ss   11:34   0:00 postgres: gitlab gitlabhq_production [local] idle
libstor+ 16267  0.0  0.1  65200 13304 ?        Ss   11:35   0:00 postgres: gitlab gitlabhq_production [local] idle
libstor+ 16269  0.0  0.1  65120 12972 ?        Ss   11:35   0:01 postgres: gitlab gitlabhq_production [local] idle

 

其他项的优化,请查看gitlab的配置文件说明

查看gitlab容器这个应用占用了多少个进程:

[root@dev ~]# docker top gitlab
UID                 PID                 PPID                C                   STIME               TTY                 TIME                CMD
systemd+            2814                15931               0                   10月16               ?                   00:13:33            unicorn worker[3] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
libstor+            2965                15806               0                   10月16               ?                   00:00:25            postgres: gitlab gitlabhq_production [local] idle
libstor+            5906                15806               0                   10:20               ?                   00:00:00            postgres: gitlab gitlabhq_production [local] idle
libstor+            5907                15806               0                   10:20               ?                   00:00:00            postgres: gitlab gitlabhq_production [local] idle
libstor+            5908                15806               0                   10:20               ?                   00:00:00            postgres: gitlab gitlabhq_production [local] idle
libstor+            5971                15806               1                   10:20               ?                   00:00:00            postgres: gitlab gitlabhq_production [local] idle
libstor+            5972                15806               0                   10:20               ?                   00:00:00            postgres: gitlab gitlabhq_production [local] idle
libstor+            5973                15806               0                   10:20               ?                   00:00:00            postgres: gitlab gitlabhq_production [local] idle
systemd+            5974                15781               0                   10:20               ?                   00:00:00            sleep 1
libstor+            7338                15806               0                   10月16               ?                   00:00:00            postgres: gitlab gitlabhq_production [local] idle
libstor+            11098               15806               0                   10月18               ?                   00:00:00            postgres: gitlab gitlabhq_production [local] idle
root                15107               15089               0                   10月16               ?                   00:00:00            /bin/bash /assets/wrapper
root                15174               15107               0                   10月16               ?                   00:00:07            runsvdir -P /opt/gitlab/service log: ...........................................................................................................................................................................................................................................................................................................................................................................................................
root                15198               15174               0                   10月16               ?                   00:00:00            runsv sshd
root                15199               15198               0                   10月16               ?                   00:00:03            svlogd -tt /var/log/gitlab/sshd
root                15200               15198               0                   10月16               ?                   00:00:15            /usr/sbin/sshd -D -f /assets/sshd_config -e
root                15757               15107               0                   10月16               ?                   00:00:00            /bin/bash /opt/gitlab/bin/gitlab-ctl tail
root                15758               15757               0                   10月16               ?                   00:00:00            /opt/gitlab/embedded/bin/ruby /opt/gitlab/embedded/bin/omnibus-ctl gitlab /opt/gitlab/embedded/service/omnibus-ctl tail
root                15760               15174               0                   10月16               ?                   00:00:00            runsv redis
root                15761               15174               0                   10月16               ?                   00:00:00            runsv postgresql
root                15762               15174               0                   10月16               ?                   00:00:00            runsv unicorn
root                15763               15174               0                   10月16               ?                   00:00:00            runsv sidekiq
root                15764               15174               0                   10月16               ?                   00:00:00            runsv gitaly
root                15765               15174               0                   10月16               ?                   00:00:00            runsv gitlab-workhorse
root                15766               15174               0                   10月16               ?                   00:00:00            runsv nginx
root                15767               15174               0                   10月16               ?                   00:00:00            runsv logrotate
root                15768               15174               0                   10月16               ?                   00:00:00            runsv prometheus
root                15769               15174               0                   10月16               ?                   00:00:00            runsv node-exporter
root                15770               15174               0                   10月16               ?                   00:00:00            runsv redis-exporter
root                15771               15174               0                   10月16               ?                   00:00:00            runsv postgres-exporter
root                15772               15174               0                   10月16               ?                   00:00:00            runsv gitlab-monitor
root                15773               15760               0                   10月16               ?                   00:00:05            svlogd -tt /var/log/gitlab/redis
root                15774               15766               0                   10月16               ?                   00:00:00            svlogd -tt /var/log/gitlab/nginx
root                15775               15762               0                   10月16               ?                   00:00:00            svlogd -tt /var/log/gitlab/unicorn
root                15776               15768               0                   10月16               ?                   00:01:21            svlogd -tt /var/log/gitlab/prometheus
root                15777               15771               0                   10月16               ?                   00:00:00            svlogd -tt /var/log/gitlab/postgres-exporter
root                15778               15770               0                   10月16               ?                   00:00:00            svlogd -tt /var/log/gitlab/redis-exporter
root                15779               15764               0                   10月16               ?                   00:00:00            svlogd -tt /var/log/gitlab/gitaly
polkitd             15780               15760               1                   10月16               ?                   01:56:23            /opt/gitlab/embedded/bin/redis-server 127.0.0.1:0
systemd+            15781               15762               0                   10月16               ?                   00:05:46            /bin/bash /opt/gitlab/embedded/bin/gitlab-unicorn-wrapper
root                15782               15766               0                   10月16               ?                   00:00:00            nginx: master process /opt/gitlab/embedded/sbin/nginx -p /var/opt/gitlab/nginx
992                 15783               15768               1                   10月16               ?                   01:32:15            /opt/gitlab/embedded/bin/prometheus -web.listen-address=localhost:9090 -storage.local.path=/var/opt/gitlab/prometheus/data -storage.local.chunk-encoding-version=2 -storage.local.target-heap-size=190243389 -config.file=/var/opt/gitlab/prometheus/prometheus.yml
libstor+            15784               15771               0                   10月16               ?                   00:10:44            /opt/gitlab/embedded/bin/postgres_exporter -web.listen-address=localhost:9187 -extend.query-path=/var/opt/gitlab/postgres-exporter/queries.yaml
polkitd             15785               15770               0                   10月16               ?                   00:06:31            /opt/gitlab/embedded/bin/redis_exporter -web.listen-address=localhost:9121 -redis.addr=unix:///var/opt/gitlab/redis/redis.socket
root                15786               15772               0                   10月16               ?                   00:00:14            svlogd -tt /var/log/gitlab/gitlab-monitor
root                15787               15767               0                   10月16               ?                   00:00:00            svlogd -tt /var/log/gitlab/logrotate
root                15788               15769               0                   10月16               ?                   00:00:00            svlogd -tt /var/log/gitlab/node-exporter
root                15789               15765               0                   10月16               ?                   00:00:02            svlogd -tt /var/log/gitlab/gitlab-workhorse
systemd+            15790               15764               0                   10月16               ?                   00:01:23            /opt/gitlab/embedded/bin/gitaly /var/opt/gitlab/gitaly/config.toml
systemd+            15791               15772               0                   10月16               ?                   00:37:08            /opt/gitlab/embedded/bin/ruby /opt/gitlab/embedded/bin/gitlab-mon web -c /var/opt/gitlab/gitlab-monitor/gitlab-monitor.yml
992                 15793               15769               0                   10月16               ?                   00:26:19            /opt/gitlab/embedded/bin/node_exporter -web.listen-address=localhost:9100 -collector.textfile.directory=/var/opt/gitlab/node-exporter/textfile_collector
systemd+            15794               15765               0                   10月16               ?                   00:02:09            /opt/gitlab/embedded/bin/gitlab-workhorse -listenNetwork unix -listenUmask 0 -listenAddr /var/opt/gitlab/gitlab-workhorse/socket -authBackend http://localhost:8080 -authSocket /var/opt/gitlab/gitlab-rails/sockets/gitlab.socket -documentRoot /opt/gitlab/embedded/service/gitlab-rails/public -pprofListenAddr -secretPath /opt/gitlab/embedded/service/gitlab-rails/.gitlab_workhorse_secret -config config.toml
root                15795               15763               0                   10月16               ?                   00:00:00            svlogd -tt /var/log/gitlab/sidekiq
root                15796               15761               0                   10月16               ?                   00:00:00            svlogd -tt /var/log/gitlab/postgresql
systemd+            15805               15763               3                   10月16               ?                   04:34:40            sidekiq 5.0.0 gitlab-rails [0 of 25 busy]
libstor+            15806               15761               0                   10月16               ?                   00:00:59            /opt/gitlab/embedded/bin/postgres -D /var/opt/gitlab/postgresql/data
systemd+            15856               15782               0                   10月16               ?                   00:00:00            nginx: worker process
systemd+            15857               15782               0                   10月16               ?                   00:00:00            nginx: worker process
systemd+            15858               15782               0                   10月16               ?                   00:00:00            nginx: worker process
systemd+            15859               15782               0                   10月16               ?                   00:00:01            nginx: worker process
systemd+            15860               15782               0                   10月16               ?                   00:00:03            nginx: worker process
systemd+            15861               15782               0                   10月16               ?                   00:00:00            nginx: worker process
systemd+            15862               15782               0                   10月16               ?                   00:00:00            nginx: worker process
systemd+            15863               15782               0                   10月16               ?                   00:00:01            nginx: worker process
systemd+            15864               15782               0                   10月16               ?                   00:00:02            nginx: worker process
systemd+            15865               15782               0                   10月16               ?                   00:00:05            nginx: worker process
systemd+            15866               15782               0                   10月16               ?                   00:00:05            nginx: worker process
systemd+            15867               15782               0                   10月16               ?                   00:00:09            nginx: worker process
systemd+            15868               15782               0                   10月16               ?                   00:00:04            nginx: cache manager process
root                15870               15758               0                   10月16               ?                   00:00:00            sh -c find /var/log/gitlab -type f -not -path */sasl/* | grep -E -v '(config|lock|@|gzip|tgz|gz)' | xargs tail --follow=name --retry
root                15873               15870               0                   10月16               ?                   00:00:00            xargs tail --follow=name --retry
root                15887               15873               0                   10月16               ?                   00:01:31            tail --follow=name --retry /var/log/gitlab/sshd/current /var/log/gitlab/gitlab-shell/gitlab-shell.log /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2017-08-01-09-11-44.log /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2017-08-01-09-12-51.log /var/log/gitlab/gitlab-rails/production.log /var/log/gitlab/gitlab-rails/application.log /var/log/gitlab/gitlab-rails/githost.log /var/log/gitlab/redis/state /var/log/gitlab/redis/current /var/log/gitlab/postgresql/state /var/log/gitlab/postgresql/current /var/log/gitlab/logrotate/current /var/log/gitlab/unicorn/unicorn_stderr.log /var/log/gitlab/unicorn/unicorn_stdout.log /var/log/gitlab/unicorn/state /var/log/gitlab/unicorn/current /var/log/gitlab/sidekiq/state /var/log/gitlab/sidekiq/current /var/log/gitlab/gitaly/state /var/log/gitlab/gitaly/current /var/log/gitlab/gitlab-workhorse/state /var/log/gitlab/gitlab-workhorse/current /var/log/gitlab/nginx/current /var/log/gitlab/nginx/error.log /var/log/gitlab/nginx/gitlab_access.log /var/log/gitlab/nginx/gitlab_error.log /var/log/gitlab/nginx/access.log /var/log/gitlab/prometheus/state /var/log/gitlab/prometheus/current /var/log/gitlab/node-exporter/state /var/log/gitlab/node-exporter/current /var/log/gitlab/redis-exporter/state /var/log/gitlab/redis-exporter/current /var/log/gitlab/postgres-exporter/state /var/log/gitlab/postgres-exporter/current /var/log/gitlab/gitlab-monitor/state /var/log/gitlab/gitlab-monitor/current
libstor+            15911               15806               0                   10月16               ?                   00:01:13            postgres: checkpointer process
libstor+            15912               15806               0                   10月16               ?                   00:00:41            postgres: writer process
libstor+            15913               15806               0                   10月16               ?                   00:00:11            postgres: wal writer process
libstor+            15914               15806               0                   10月16               ?                   00:00:11            postgres: autovacuum launcher process
libstor+            15915               15806               0                   10月16               ?                   00:01:51            postgres: stats collector process
libstor+            15927               15806               0                   10月16               ?                   00:10:05            postgres: gitlab-psql postgres [local] idle
systemd+            15931               15107               0                   10月16               ?                   00:03:08            unicorn master -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+            16111               15931               0                   10月16               ?                   00:22:51            unicorn worker[1] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+            16114               15931               0                   10月16               ?                   00:21:35            unicorn worker[2] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
libstor+            16163               15806               0                   10月16               ?                   00:00:37            postgres: gitlab gitlabhq_production [local] idle
libstor+            16269               15806               0                   10月16               ?                   00:00:39            postgres: gitlab gitlabhq_production [local] idle
libstor+            17931               15806               0                   10月18               ?                   00:00:00            postgres: gitlab gitlabhq_production [local] idle
libstor+            18715               15806               0                   10月16               ?                   00:00:00            postgres: gitlab gitlabhq_production [local] idle
root                30484               15767               0                   09:33               ?                   00:00:00            /bin/sh /opt/gitlab/embedded/bin/gitlab-logrotate-wrapper
systemd+            30740               15931               0                   10月17               ?                   00:14:25            unicorn worker[0] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
libstor+            30755               15806               0                   10月17               ?                   00:00:25            postgres: gitlab gitlabhq_production [local] idle
root                32090               30484               0                   09:43               ?                   00:00:00            sleep 3000
[root@dev ~]#

GitLab由乌克兰程序员DmitriyZaporozhets和ValerySizov开发,它使用Ruby语言写成。后来,一些部分用Go语言重写。

Gitlab使用Unicorn开启多进程模型。Unicorn 作为 Web 服务使用了多进程的模型,通过一个 master 进程来管理多个 worker 进程,其中 master 进程不负责处理客户端的 HTTP 请求,多个 worker 进程监听同一组 Socket。

unicorn-io-mode

一组 worker 进程在监听 Socket 时,如果发现当前的 Socket 有等待处理的请求时就会在当前的进程中直接通过 #process_client 方法处理,整个过程会阻塞当前的进程,而多进程阻塞 I/O 的方式没有办法接受慢客户端造成的性能损失,只能通过反向代理 nginx 才可以解决这个问题。

unicorn-multi-processes

使用ps命令结合管道符 | 查询到每个unicorn进程的PID(占用操作系统的和占用docker容器的)

[root@dev ~]# ps -aux |grep unicorn
systemd+  2814  0.2  7.1 897900 574752 ?       Sl   10月16  13:35 unicorn worker[3] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
root      7487  0.0  0.0 112660   976 pts/0    S+   10:30   0:00 grep --color=auto unicorn
root     15762  0.0  0.0   4236   360 ?        Ss   10月16   0:00 runsv unicorn
root     15775  0.0  0.0   4380   360 ?        S    10月16   0:00 svlogd -tt /var/log/gitlab/unicorn
systemd+ 15781  0.0  0.0  18096  1648 ?        Ss   10月16   5:46 /bin/bash /opt/gitlab/embedded/bin/gitlab-unicorn-wrapper
root     15887  0.0  0.0   4404   688 ?        S    10月16   1:31 tail --follow=name --retry /var/log/gitlab/sshd/current /var/log/gitlab/gitlab-shell/gitlab-shell.log 
~~~
systemd+ 15931  0.0  5.3 707188 432180 ?       Sl   10月16   3:09 unicorn master -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+ 16111  0.3  6.8 820076 545992 ?       Sl   10月16  22:52 unicorn worker[1] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+ 16114  0.3  7.1 820076 570364 ?       Sl   10月16  21:36 unicorn worker[2] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
systemd+ 30740  0.2  6.8 861036 550096 ?       Sl   10月17  14:26 unicorn worker[0] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
[root@dev ~]# docker exec -it gitlab /bin/bash
root@dev:/#  ps -aux |grep unicorn
root       376  0.0  0.0   4236   360 ?        Ss   Oct16   0:00 runsv unicorn
root       389  0.0  0.0   4380   360 ?        S    Oct16   0:00 svlogd -tt /var/log/gitlab/unicorn
git        395  0.0  0.0  18096  1648 ?        Ss   Oct16   5:47 /bin/bash /opt/gitlab/embedded/bin/gitlab-unicorn-wrapper
root       501  0.0  0.0   4404   688 ?        S    Oct16   1:32 tail --follow=name --retry /var/log/gitlab/sshd/current /var/log/gitlab/gitlab-shell/gitlab-shell.log 
~~~
git        518  0.0  5.3 707188 432436 ?       Sl   Oct16   3:09 unicorn master -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git        654  0.3  6.9 820076 553640 ?       Sl   Oct16  22:57 unicorn worker[1] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git        657  0.3  7.1 820076 570472 ?       Sl   Oct16  21:40 unicorn worker[2] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git       5223  0.2  7.2 897900 584108 ?       Sl   Oct16  13:38 unicorn worker[3] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
root     10683  0.0  0.0  11276   980 ?        S+   02:51   0:00 grep --color=auto unicorn
git      29468  0.2  6.8 861036 550096 ?       Sl   Oct16  14:29 unicorn worker[0] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
root@dev:/#  netstat -an | grep 29468  
bash: netstat: command not found
root@dev:/# exit
[root@dev ~]# netstat -an | grep 15762
[root@dev ~]# netstat -an | grep 15931
[root@dev ~]# netstat -an | grep 30740
[root@dev ~]# netstat -an | grep 16111
[root@dev ~]# netstat -an | grep 16114

netstat -nap | grep [pid]             #得到PID后,使用netstat命令查询端口占用

 

线程和进程、程序、应用程序之间的关系
看到一种说法是“一个程序至少有一个进程,一个进程至少有一个线程”,这种把程序与进程,进程与线程的关系混淆的说法是错误的。

程序(program)只能有一个进程,一个进程就是一个程序。有人说,我打开一个程序,比如chrome,有十多个进程呢,这是咋回事。那就是十多个程序,操作系统给他们分配了彼此独立的内存,相互执行不受彼此约束,分配同样时间的CPU。对于用户而言,他们是一个整体,我们通常称之为应用程序(application)。对于计算机而言,一个进程就是一个程序,多个进程(比如一个浏览器的多个进程)对计算机而言就是多个不同的程序,它不会把它们理解为一个完整的“程序”。

其实进程之间的关系只有父子关系,没有主从关系,他们之间是并行独立的。但是线程之间是有主从关系的,而且他们共享的是同一个内存块(包括程序、数据和堆栈)。

打个比方,进程之间是父子关系,父进程fork子进程,就好比你养了个儿子。这个子进程会拷贝一份内存块,把程序和数据都复制过去,你儿子跟你长的也很像。但是一旦出生下来了,你们就是两个独立的个体,法律上都是平等的。子进程之后就完全独立了,父进程与子进程之间的关系,与其他进程的关系都是一样的,平等的,谁也管不着谁了,他们也只能采用进程间通信才能相互了解。父亲死了,儿子还活着;父进程over了,子进程可以照样活的好好的。(除非程序员认定有一个进程over了,其他进程没有存在的意义,比如浏览器负责渲染的进程如果down掉了,其他进程自动kill掉)。

但是进程的不同线程的关系可不是这样的。进程可以由多个线程组成,这称之为多线程程序,他们之间的关系就好比你的大脑与四肢和身体其他部分的关系一样。大脑就是主线程,其他部分就是子线程。子线程由主线程派生,而依附于主线程。主线程一旦over,进程就over了,其他子线程更是over了。他们的内存和数据都是同一份,没有进行隔离(既方便,也危险),不需要额外的通信函数。

父亲死了,儿子依旧活下去,你头断了,人还能活吗?

了解进程与线程的关系,就要了解他们的区别。一个计算机可以有多个进程,这称之为多任务,他们共享的是CPU,硬盘,打印机,显示器,但他们的内存是独立的,所以需要进程间通信,这是计算机发展的第一步。一个进程可以有多个线程,这称之为多线程,他们除了共享进程间的共享内容之外,还共享内存,这是计算机发展的第二步,主要是为了满足并行运算时共享数据,无需额外的通信。

所以正确的结论是:一个程序(program)就是一个正在执行的进程,而每个进程,可以是单线程的,也可以是多线程的。一个应用程序(application)通常由多个程序组成。

还是强调下程序(program)和应用程序(application)的区别。一个程序就是一个进程,永远不要说一个程序可能有多个进程。你打开一个应用程序(比如chrome),会有十多个进程,对于计算机而言,它们都是独立的。好比,父亲和儿子在外人看来是一家人,但是对于法律上来说,就是独立的法人。

 

 

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值