Cluster概念
系统扩展方式:
Scale Up:向上扩展,增强
Scale Out:向外扩展,增加设备,调度分配
Cluster:集群,为解决某个特定问题将多台计算机组合起来形成的单个系统。
Linux Cluster类型:
LB:Load Balancing,负载均衡
HA:High Availability,高可用
SPOF(single point of failure)
MTBF(mean time between failure)平均无故障时间
MTTR(mean time to restoration) repair平均恢复前时间
A=MTBF/(MTBF+MTTR) (0,1):99%,99.5%,99.9%,99.99%,99.999%
HPC:High-performance computing,高性能 www.top500.org
分布式系统:
分布式存储:云盘
分布式计算:hadoop,spark
LB Cluster的实现:
硬件:F5 Big-IP
Citrix Netscaler
A10 A10
软件:
LVS:Linux Virtual Server
nginx:支持七层调度
haproxy:支持七层调度
ats:apache traffic server,yahoo捐助
perlbal:perl编写
pound:
基于工作的协议层次划分:
传输层(通用):DPORT
lvs:
nginx:steam
haproxy:mode tcp
应用层(专用):针对特定协议,自定义的请求模型分类
proxy server:
http:nginx,httpd,haproxy(mode http),…
fastcgi:nginx,httpd,…
mysql:mysl-proxy,…
会话保持:负载均衡
1)session sticky:同一用户调度固定服务器
Source IP:LVS sh算法(对某一特定服务而言)
Cookie
2)session replication:每台服务器拥有全部session,session multicast cluster
3)session server:专门的session服务器 memcached,redis
HA集群实现方案:
keepalived:vrrp协议
ais:应用接口协议
heartbeat
cman+rgmanager(RHCS)
coresync_pacemaker
LVS
VS:virtual server,负责调度
RS:Real Server ,负责真正提供服务
L4:四层路由器或交换机
工作原理:VS根据请求报文的目标IP和目标协议及端口将其调度转发至某RS,根据调用算法挑选RS
iptables/netfilter
iptables:用户空间的管理工具
netfilter:内核空间上的框架
流入:PREROUTING–>INPUT
流出:OUTPUT–>POSTROUTING
转发:PREROUTING–>FORWARD–>POSTROUTING
DNAT:目标地址转换;PREROUTING
LVS集群类型中的术语:
VS:Virtual Server,Director Server(DS)
Dispatcher(调度器),Load Balancer
RS:Real Server(lvs),upsteam server(nginx)
backend server(haproxy)
CIP:Client IP
VIP:Virtual Server IP (VS外网的IP)
DIP:Director Server IP (VS内网的IP)
RIP:Real Server IP
访问流程:CIP <–> VIP==DIP <–> RIP
lvs:ipvsadm/ipvs
ipvsadm:用户空间的命令行工具,规则管理器;用于管理集群服务及RealServer
ipvs:工作于内核空间netfilter的INPUT钩子上的框架
lvs集群的类型:
lvs-nat: 修改请求报文的目标IP,多目标IP的DNAT
lvs-dr: 操纵封装新的MAC地址
lvs-tun: 在原请求IP报文之外新加一个IP首部
lvs-fullnat: 修改请求报文的源和目标IP
![](https://i-blog.csdnimg.cn/blog_migrate/67475aad59fdbb210fc1bbf2cdabdd20.png)
![](https://i-blog.csdnimg.cn/blog_migrate/95a3161d038358d00ec056f1295a56bc.png)
![](https://i-blog.csdnimg.cn/blog_migrate/c83854f6c0922c4802eba17c077c0460.png)
IP 地址网段 10.10.10.0/24 255.255.255.0
6 台虚拟机
10.10.10.11
10.10.10.12
10.10.10.13
10.10.10.14
10.10.10.15
10.10.10.16
vip 10.10.10.100
cip 10.10.10.240
VMware
14 pro
网络类型:仅主机模式
操作系统配置
硬件选项
2 U
1GB
20GB
2 块网卡
操作系统配置
桌面版安装 PXE 桌面
LVS:DR模型
第一步:
关闭iptables
[root@localhost ~]# service iptables stop
[root@localhost ~]# chkconfig iptables off
关闭selinux
[root@localhost ~]# vim /etc/selinux/config
[root@localhost ~]# setenforce 0
配置网络
[root@localhost ~]# vim /etc/sysconfig/network-scripts/eth0
IPADDR=10.10.10.11
NETMASK=255.255.255.0
[root@localhost ~]# service iptables restart
挂载光盘
[root@localhost ~]# mkdir /mnt/cdrom
[root@localhost ~]# mount -t iso9660 /dev/cdrom /mnt/cdrom
配置yum
[root@localhost ~]# cd /etc/yum.repos.d/
[root@localhost ~]# mkdir back
[root@localhost ~]# mv * back/
[root@localhost ~]# cp -a back/CentOS-Mdeia.repo .
[root@localhost ~]# vim CentOS-Mdeia.repo
[rhel6]
name=rhel6
baseurl=file:///mnt/cdrom
enabled=1
gpgcheck=0
[root@localhost ~]# yum clean all
[root@localhost ~]# yum -y install gcc gcc-c++ lrzsz
拍摄快照
-------------------------------------------
C:
10.10.10.240
D:
eth0 10.10.10.11
eth0:0 10.10.10.100
RS1:
eth0 10.10.10.12
lo:0 10.10.10.100
RS2:
eth0 10.10.10.13
lo:0 10.10.10.100
负载调度器
--------------------------------------------
第二步:
10.10.10.11
[root@localhost ~]# service NetworkManager stop && chkconfig NetworkManager off
[root@localhost ~]# cd /etc/sysconfig/network-scripts/
[root@localhost ~]# cp -a ifcfg-eth0 ifcfg-eth0:0
[root@localhost ~]# vim ifcfg-eth0:0 或者 vim !$
DEVICE=eth0:0
TYPE=Ethernet
ONBOOT=yes
BOOTPROTO=static
IPADDR=10.10.10.100
NETMASK=255.255.255.0
[root@localhost ~]# ifup eth0:0 或者 service network restart
[root@localhost ~]# vim /etc/sysctl.conf
# ARP -
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.eth0.send_redirects = 0
刷新内核参数
[root@localhost ~]# sysctl -p
安装ipvsadm
[root@localhost ~]# yum install -y ipvsadm
[root@localhost ~]# ipvsadm -Ln
[root@localhost ~]# modprobe ip_vs
添加规则
[root@localhost ~]# ipvsadm -A -t 10.10.10.100:80 -s rr
[root@localhost ~]# ipvsadm -a -t 10.10.10.100:80 -r 10.10.10.12:80 -g
[root@localhost ~]# ipvsadm -a -t 10.10.10.100:80 -r 10.10.10.13:80 -g
[root@localhost ~]# ipvsadm -Ln
[root@localhost ~]# service ipvsadm save
[root@localhost ~]# chkconfig ipvsadm on
查看当前连接状态
[root@localhost ~]# ipvsadm -Ln --stats
---------------------------------------------------
10.10.10.12
[root@localhost ~]# service NetworkManager stop && chkconfig NetworkManager off
Stopping NetworkManager daemon: [ OK ]
[root@localhost ~]# service httpd start
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using localhost.localdomain for ServerName
[ OK ]
[root@localhost ~]# echo 'this is server 1' >> /var/www/html/index.html
[root@localhost ~]# curl localhost
this is server 1
[root@localhost ~]# cd /etc/sysconfig/network-scripts/
[root@localhost ~]# cp -a ifcfg-lo ifcfg-lo:0
vim !$
DEVICE=lo:0
IPADDR=10.10.10.100
NETMASK=255.255.255.255 #修改这三项即可,其他配置不需要修改
[root@localhost ~]# vim /etc/sysctl.conf
# LVS - arp control
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
[root@localhost ~]# sysctl -p
重启回环借口
[root@localhost ~]# ifup lo:0
添加路由
[root@localhost ~]# route add -host 10.10.10.100 dev lo:0
[root@localhost ~]# route -n
添加到开机自启
echo "route add -host 10.10.10.100 dev lo:0" >> /etc/rc.local
-------------------------------------------------
10.10.10.13
[root@localhost ~]# service NetworkManager stop && chkconfig NetworkManager off
Stopping NetworkManager daemon: [ OK ]
[root@localhost ~]# service httpd start
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using localhost.localdomain for ServerName
[ OK ]
[root@localhost ~]# echo 'this is server 2' >> /var/www/html/index.html
[root@localhost ~]# curl localhost
this is server 2
[root@localhost ~]# cd /etc/sysconfig/network-scripts/
[root@localhost ~]# cp -a ifcfg-lo ifcfg-lo:0
vim !$
DEVICE=lo:0
IPADDR=10.10.10.100
NETMASK=255.255.255.255 #修改这三项即可,其他配置不需要修改
[root@localhost ~]# vim /etc/sysctl.conf
# LVS - arp control
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
[root@localhost ~]# sysctl -p
重启回环借口
[root@localhost ~]# ifup lo:0
添加路由
[root@localhost ~]# route add -host 10.10.10.100 dev lo:0
[root@localhost ~]# route -n
添加到开机自启
echo "route add -host 10.10.10.100 dev lo:0" >> /etc/rc.local
-------------------------------------
[root@localhost ~]# modprobe ip_vs
[root@localhost ~]# rpm -ivh ipvsadm-1.261.... 或者 yum install ipvsadm
[root@localhost ~]# shutdown -h now
---
cp:
-a:此参数的效果和同时指定"-dpR"参数相同;
-d:当复制符号连接时,把目标文件或目录也建立为符号连接,并指向与源文件或目录连接的原始文件或目录;
-p:保留源文件或目录的属性;
-R/r:递归处理,将指定目录下的所有文件与子目录一并处理;
-s:对源文件建立符号连接,而非复制文件;
-l:对源文件建立硬连接,而非复制文件;
---
LVS:NAT模型
C
20.20.20.22
D
eth0:20.20.20.11
eth1:10.10.10.11
RS1
eth0:10.10.10.12
RS2
eth0:10.10.10.13
----------------------------
D
配置eth0网卡
eth0:20.20.20.11
[root@localhost ~]# vim ifcfg-eth1
IPADDR=20.20.20.11
NETMASK=255.255.255.0
eth1:10.10.10.11
配置eth1网卡
[root@localhost ~]# vim ifcfg-eth1
IPADDR=10.10.10.11
NETMASK=255.255.255.0
挂载光盘
[root@localhost ~]# mount -t iso9660 /dev/cdrom /mnt/cdrom
安装ipvs管理工具ipvsadm
[root@localhost ~]# yum install -y ipvsadm
开启转发功能
[root@localhost ~]# vim /etc/sysctl.conf
net.ipv4.ip_forward = 1
[root@localhost ~]# sysctl -p
添加一个防火墙规则
[root@localhost ~]# iptables -F
[root@localhost ~]# iptables -t nat -A POSTROUTING -S 10.10.10.0/24 -o eth0 -j SNAT --to-source 20.20.20.11
[root@localhost ~]# iptables -t nat -L
保存iptables规则
[root@localhost ~]# service iptables save
开机启动iptables
[root@localhost ~]# chkconfig iptables on
配置LVS规则
[root@localhost ~]# ipvsadm -A -t 20.20.20.11:80 -s rr
[root@localhost ~]# ipvsadm -a -t 20.20.20.11:80 -r 10.10.10.12:80 -m
[root@localhost ~]# ipvsadm -a -t 20.20.20.11:80 -r 10.10.10.13:80 -m
[root@localhost ~]# ipvsadm -Ln
保存LVS规则
[root@localhost ~]# service ipvsadm save
[root@localhost ~]# chkconfig ipvsadm on
---------------------------
RS1
[root@localhost ~]# service NetworkManager stop && chkconfig NetworkManager off
Stopping NetworkManager daemon: [ OK ]
[root@localhost ~]# chkconfig httpd on
[root@localhost ~]# service httpd start
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using localhost.localdomain for ServerName
[ OK ]
[root@localhost ~]# echo 'this is server 1' >> /var/www/html/index.html
[root@localhost ~]# curl localhost
设置默认网关
echo "GATEWAY=10.10.10.11" >> /etc/sysconfig/network-scripts/ifcfg-eth0
# 或者 route add default gw
service network restart
---------------------------
RS2
[root@localhost ~]# service NetworkManager stop && chkconfig NetworkManager off
Stopping NetworkManager daemon: [ OK ]
[root@localhost ~]# chkconfig httpd on
[root@localhost ~]# service httpd start
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using localhost.localdomain for ServerName
[ OK ]
[root@localhost ~]# echo 'this is server 1' >> /var/www/html/index.html
[root@localhost ~]# curl localhost
设置默认网关
echo "GATEWAY=10.10.10.11" >> /etc/sysconfig/network-scripts/ifcfg-eth0
# 或者 route add default gw
service network restart