Nginx tcp 协议的反向代理和负载均衡

六：Nginx tcp 负载均衡

6.1：主配置文件添加 stream 配置段

• 编辑nginx.conf，添加 stream 配置段：

[root@node106 ~]# vim /apps/nginx/conf/nginx.conf
stream {
  log_format stream '$remote_addr [$time_local] '
                 '$protocol $status $bytes_sent $bytes_received '
                 '$session_time "$upstream_addr" '
                 '"$upstream_bytes_sent" "$upstream_bytes_received" "$upstream_connect_time"';
  include /apps/nginx/stream_conf.d/*.conf;
}

• 创建 stream 配置文件目录：

  以实现 stream 相关的配置文件模块化；

[root@node106 ~]# mkdir /apps/nginx/stream_conf.d/

• 检查配置文件并重载nginx：

[root@node106 ~]# nginx -t
nginx: the configuration file /apps/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /apps/nginx/conf/nginx.conf test is successful
[root@node106 ~]# systemctl reload nginx

6.2：Redis 负载均衡

6.2.1：后端服务器安装 Redis

• node107 和 node108 分别安装 Redis；

~]# yum install redis -y

• 配置 redis 监听在所有 IP：

~]# vim /etc/redis.conf

• 启动 redis：

~]# systemctl enable redis
Created symlink from /etc/systemd/system/multi-user.target.wants/redis.service to /usr/lib/systemd/system/redis.service.
~]# systemctl start redis

~]# ss -tnl | grep 6379
LISTEN     0      511          *:6379                     *:* 

• 进入redis，分别设置 name 为 node107 和 node108：

  以便验证负载均衡的效果；

[root@node107 ~]# redis-cli
127.0.0.1:6379> set name node107
OK
127.0.0.1:6379> get name
"node107"

[root@node108 ~]# redis-cli
127.0.0.1:6379> set name node108
OK
127.0.0.1:6379> get name
"node108"

6.2.2：Nginx 配置 Redis 代理

• 编辑配置文件：

[root@node106 ~]# vim /apps/nginx/stream_conf.d/redis.conf 
upstream redis {
  server 192.168.1.107:6379 weight=1 fail_timeout=5s max_fails=3;
  server 192.168.1.108:6379 weight=1 fail_timeout=5s max_fails=3;
}

server {
  listen 192.168.1.106:6379;
  proxy_connect_timeout 3s;
  proxy_timeout 30s;
  proxy_pass redis;
  access_log /data/nginx/logs/stream_redis_access.log stream;
}

• 检查配置文件并重载nginx：

[root@node106 ~]# nginx -t
nginx: the configuration file /apps/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /apps/nginx/conf/nginx.conf test is successful
[root@node106 ~]# systemctl reload nginx

• 查看监听端口：

[root@node106 ~]# ss -tnl | grep 6379
LISTEN     0      511    192.168.1.106:6379                     *:*

• 访问验证：

#  首次调度到 192.168.1.107：
[root@node105 ~]# redis-cli -h 192.168.1.106
192.168.1.106:6379> get name
"node107"
192.168.1.106:6379> quit

#  第二次调度到 192.168.1.108：
[root@node105 ~]# redis-cli -h 192.168.1.106
192.168.1.106:6379> get name
"node108"

• 查看日志：

[root@node106 ~]# tail -f /data/nginx/logs/stream_redis_access.log
192.168.1.105 [05/Dec/2020:13:54:56 +0800] TCP 200 9941 40 4.376 "192.168.1.107:6379" "40" "9941" "0.001"
192.168.1.105 [05/Dec/2020:13:55:30 +0800] TCP 200 9941 40 32.544 "192.168.1.108:6379" "40" "9941" "0.001"

6.3：MySQL 负载均衡

6.3.1：后端服务器安装 MariaDB

• 分别在 node107 和node108 上安装 mariadb-server：

~]# yum install mariadb-server -y

• 启动mariadb：

~]# systemctl start mariadb 
~]# systemctl enable mariadb 

• 授权远程登录用户 test：

MariaDB [(none)]> GRANT ALL ON *.* TO 'test'@'192.168.1.%' IDENTIFIED BY '123456';
MariaDB [(none)]> FLUSH PRIVILEGES;

• 分别创建 node107 和 node108 数据库：

  以便验证负载均衡效果；

MariaDB [(none)]> CREATE DATABASE node107;

MariaDB [(none)]> CREATE DATABASE node108;

6.3.2：Nginx 配置 MySQL 代理

• 编辑配置文件：

[root@node106 ~]# vim /apps/nginx/stream_conf.d/mysql.conf
upstream mysql {
  server 192.168.1.107:3306 weight=1 fail_timeout=5s max_fails=3;
  server 192.168.1.108:3306 weight=1 fail_timeout=5s max_fails=3;
}

server {
  listen 192.168.1.106:3306;
  proxy_connect_timeout 3s;
  proxy_timeout 30s;
  proxy_pass mysql;
  access_log /data/nginx/logs/stream_mysql_access.log stream;
}

• 检查配置文件并重载nginx：

[root@node106 ~]# nginx -t
nginx: the configuration file /apps/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /apps/nginx/conf/nginx.conf test is successful
[root@node106 ~]# systemctl reload nginx

• 验证监听端口：

[root@node106 ~]# ss -tnl | grep 3306
LISTEN     0      511    192.168.1.106:3306                     *:* 

• 访问验证：

#  首次调度到 192.168.1.107：
[root@node105 ~]# mysql -h192.168.1.106 -utest -p
Enter password: 
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 4
Server version: 5.5.68-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> SHOW DATABASES;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| node107            |
| performance_schema |
| test               |
+--------------------+
5 rows in set (0.01 sec)

MariaDB [(none)]> QUIT
Bye

# 第二次调度到 192.168.1.108
[root@node105 ~]# mysql -h192.168.1.106 -utest -p
Enter password: 
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 5
Server version: 5.5.68-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> SHOW DATABASES;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| node108            |
| performance_schema |
| test               |
+--------------------+
5 rows in set (0.01 sec)

• 查看日志：

[root@node106 ~]# tail -f /data/nginx/logs/stream_mysql_access.log 
192.168.1.105 [05/Dec/2020:14:07:18 +0800] TCP 200 361 145 17.681 "192.168.1.107:3306" "145" "361" "0.002"
192.168.1.105 [05/Dec/2020:14:07:57 +0800] TCP 200 361 140 34.734 "192.168.1.108:3306" "140" "361" "0.001"