高可用服务之Keepalived高可用LVS集群

最新推荐文章于 2022-09-20 17:31:39 发布

adasdlulu

最新推荐文章于 2022-09-20 17:31:39 发布

阅读量159

点赞数

原文链接：https://www.cnblogs.com/qiuhom-1874/p/13659428.html

版权

前文我们聊了下keepalived的邮件通知相关配置，回顾请参考https://www.cnblogs.com/qiuhom-1874/p/13645163.html；今天我们来说说keepalived高可用LVS集群；

我们知道一个服务通常是以一个套接字形式对外提供服务，所谓套接字就是ip+端口；前面的博客中我们主要聊到了keepalived对ip地址的高可用，但通常对ip地址高可用没有多大实质的作用，重要的是我们高可用的ip地址后端对应的服务才是根本，这一篇博客主要讲怎么利用keepalived高可用LVS集群，生成ipvs规则，以及对LVS集群的rs做健康状态检测；

环境说明

名称 ip地址端口
keepalived-node01(master) 192.168.0.41
keepalived-node02(backup) 192.168.0.42
LVS-RS1 192.168.0.43 80
LVS-RS2 192.168.0.44 80
VIP 192.168.0.111 80

准备LVS集群RS1和RS2

1、安装webserver

1
yum install nginx -y
　　提示：rs1和rs2上都要安装nginx服务，用于后端rs提供的服务；

2、提供测试页

3、启动rs1和rs2上的nginx服务

4、编写修改内核参数，并配置vip给RS1和RS2的脚本

View Code
　　提示：以上脚本主要实现了两个参数，给定start参数就把对应的内核参数修改以后，并把vip配置到指定的接口；给stop参数就把vip从指定的端口上删除，并还原内核参数的设定；

在rs1和rs2上执行设置内核参数的脚本

提示：到此后端两个RS的环境就准备好了；

配置keepalived，生成lvs规则

完整的配置

View Code
　　提示：virtual_server用于定义LVS对外集群ip地址和端口（vip），用大括号括起来，其中delay_loop用于指定对后端rs做健康状态检查的时间间隔；lb_algo/lvs_sched用于指定lvs的调度算法，常用的算法有rr，wrr，lc，wlc，lblc，sh，dh；lb_kind/lvs_method用于指定lvs集群的类型，常用的类型有DR，NAT，TUN，需注意这里的类型的值必须大写，否则服务有异常；有关LVS集群类型的相关说明请参考https://www.cnblogs.com/qiuhom-1874/p/12327506.html；protocol用于指定4层协议，常用的4层协议有TCP ,UDP,SCTP，需注意这里的值必须大写；sorry_server用于指定，当后端RS都宕机情况下，临时对用户说sorry的服务器地址和端口；real_server：用来定义后端RS的相关配置，其中weight用于指定当前rs的权重，nb_get_retry用指定对rs检测的重试次数，如果在指定的次数上都监测失败就标记该RS为下线状态，并从当前集群中下线；delay_before_retry用于指定重试之前延迟的时间；connect_timeout用于指定对rs检测的超时时长；HTTP_GET 用于配置对rs的检查方法，HTTP_GET表示应用层http检测，其中path用于指定检测到uri，status_code用于指定对指定URI检测到状态码，通常为200；以上virtual_server的配置在node02上也是相同的配置；

安装sorryserver，并配置测试页面

启动keepalived

提示：到此基于keepalived高可用LVS集群就配置完成了；

验证：用浏览器对VIP访问，看看是否能够访问到后端RS提供的页面？

提示：我们用浏览器访问VIP并没有访问到后端的rs提供的页面；其中的原因是我们在配置keepalived时，开启了严格遵守vrrp协议，所以启动keepalived它默认会自动生成iptables规则，禁止任何地址访问VIP；

提示：解决办法用iptables -F清空iptables规则；这种清空iptables规则的方式只是临时的方式，重启以后，或者vip飘逸后，对应的规则又会生成，永久解决办法是在keepalived的配置文件，禁用它自动生成iptabels规则；

提示：在/etc/keepalived/keepalived.conf的global_defs中加上vrrp_iptables这个配置，这个配置表示禁用自动生成iptables规则；当然我们也可配置不严格遵守vrrp协议，把vrrp_strict去掉也行；选择其中一种方式即可；

验证：重启keepalived，看看对应iptables规则是否还会生成？

提示：可以看到现在vip所在节点的iptables规则就没有在自动生成了，对于node02也是相同的配置，重启keepalived即可解决自动生成iptables规则的问题；

现在在用浏览器访问vip，看看是否能够访问到后端RS提供的页面？

提示：可以看到我们在浏览器上访问VIP是可以正常访问到后端rs提供的页面；

验证：把node01上的keepalived停掉，看看node02是否会自动将vip配置上对应的接口？用浏览器是否还会访问到rs提供的页面呢？

提示：可以看到当node01的keepalived宕机以后，对应vip会自动飘逸到node02上去，并且在客户端访问VIP几乎不受影响；

验证：在node01上查看ipvs规则，看看是否都生成了ipvs规则呢？

提示：可以看到node01上并没有生成ipvs规则，原因是keepalived停掉了，对应的ipvs规则也就删除了；node02上的keepalived是活跃状态，所以对应ipvs规则也是有keepalived自动生成；

验证：把rs1的web服务停掉，看看keepalived是否会检测到rs1不再线，从而把rs1自动从集群踢出去呢？

提示：可以看到当rs1故障以后，keepalived会检测到rs1故障，然后把rs1从集群中提出去，所以我们在ipvs规则表中就没有rs1；

验证：把rs2停掉，看看对应的sorryserver是否会被激活？

提示：可以看到把rs2停掉以后，对应ipvs规则表中就没有RS2，并且它会把我们之前配置的sorryserver 的地址和端口配置上；

验证：用浏览器访问VIP看看对应相应的内容是否是vip所在节点的sorryserver提供的页面呢？
https://www.chinaacc.com/ask/detail/free/18161503
https://www.chinaacc.com/ask/detail/free/18157199
https://www.chinaacc.com/ask/detail/free/18157028
https://www.chinaacc.com/ask/detail/free/18156970
https://www.chinaacc.com/ask/detail/free/18156754
https://www.chinaacc.com/ask/detail/free/18156668
https://www.chinaacc.com/ask/detail/free/18156563
https://www.chinaacc.com/ask/detail/free/18156495
https://www.chinaacc.com/ask/detail/free/18156441
https://www.chinaacc.com/ask/detail/free/18156233
https://www.chinaacc.com/ask/detail/free/18156123
https://www.chinaacc.com/ask/detail/free/18155993
https://www.chinaacc.com/ask/detail/free/18155949
https://www.chinaacc.com/ask/detail/free/18155834
https://www.chinaacc.com/ask/detail/free/18155744
https://www.chinaacc.com/ask/detail/free/18155662
https://www.chinaacc.com/ask/detail/free/18155572
https://www.chinaacc.com/ask/detail/free/18155481
https://www.chinaacc.com/ask/detail/free/18155359
https://www.chinaacc.com/ask/detail/free/18155273
https://www.chinaacc.com/ask/detail/free/18155192
https://www.chinaacc.com/ask/detail/free/18155056
https://www.chinaacc.com/ask/detail/free/18154982
https://www.chinaacc.com/ask/detail/free/18154898
https://www.chinaacc.com/ask/detail/free/18154798
https://www.chinaacc.com/ask/detail/free/18154702
https://www.chinaacc.com/ask/detail/free/18154612
https://www.chinaacc.com/ask/detail/free/18154546
https://www.chinaacc.com/ask/detail/free/18154476
https://www.chinaacc.com/ask/detail/free/18154384
https://www.chinaacc.com/ask/detail/free/18154271
https://www.chinaacc.com/ask/detail/free/18154184
https://www.chinaacc.com/ask/detail/free/18154109
https://www.chinaacc.com/ask/detail/free/18154068
https://www.chinaacc.com/ask/detail/free/18153979
https://www.chinaacc.com/ask/detail/free/18153845
https://www.chinaacc.com/ask/detail/free/18153710
https://www.chinaacc.com/ask/detail/free/18153622
https://www.chinaacc.com/ask/detail/free/18153510
https://www.chinaacc.com/ask/detail/free/18153362
https://www.chinaacc.com/ask/detail/free/18153309
https://www.chinaacc.com/ask/detail/free/18153222
https://www.chinaacc.com/ask/detail/free/18153148
https://www.chinaacc.com/ask/detail/free/18153097
https://www.chinaacc.com/ask/detail/free/18153008
https://www.chinaacc.com/ask/detail/free/18152850
https://www.chinaacc.com/ask/detail/free/18152803
https://www.chinaacc.com/ask/detail/free/18152720
https://www.chinaacc.com/ask/detail/free/18152479
https://www.chinaacc.com/ask/detail/free/18152364
https://www.chinaacc.com/ask/detail/free/18152191
https://www.chinaacc.com/ask/detail/free/18152066
https://www.chinaacc.com/ask/detail/free/18152018
https://www.chinaacc.com/ask/detail/free/18151980
https://www.chinaacc.com/ask/detail/free/18151829
https://www.chinaacc.com/ask/detail/free/18151719
https://www.chinaacc.com/ask/detail/free/18151616
https://www.chinaacc.com/ask/detail/free/18151565
https://www.chinaacc.com/ask/detail/free/18151496
https://www.chinaacc.com/ask/detail/free/18151296
https://www.chinaacc.com/ask/detail/free/18151131
https://www.chinaacc.com/ask/detail/free/18151002
https://www.chinaacc.com/ask/detail/free/18150879
https://www.chinaacc.com/ask/detail/free/18150770
https://www.chinaacc.com/ask/detail/free/18150574
https://www.chinaacc.com/ask/detail/free/18150535
https://www.chinaacc.com/ask/detail/free/18150371
https://www.chinaacc.com/ask/detail/free/18150125
https://www.chinaacc.com/ask/detail/free/18150029
https://www.chinaacc.com/ask/detail/free/18149818
https://www.chinaacc.com/ask/detail/free/18149692
https://www.chinaacc.com/ask/detail/free/18149588
https://www.chinaacc.com/ask/detail/free/18149402
https://www.chinaacc.com/ask/detail/free/18149178
https://www.chinaacc.com/ask/detail/free/18149071
https://www.chinaacc.com/ask/detail/free/18148993
https://www.chinaacc.com/ask/detail/free/18148847
https://www.chinaacc.com/ask/detail/free/18148672
https://www.chinaacc.com/ask/detail/free/18148601
https://www.chinaacc.com/ask/detail/free/18148550
https://www.chinaacc.com/ask/detail/free/18148411
https://www.chinaacc.com/ask/detail/free/18148295
https://www.chinaacc.com/ask/detail/free/18148173
https://www.chinaacc.com/ask/detail/free/18148040
https://www.chinaacc.com/ask/detail/free/18147989
https://www.chinaacc.com/ask/detail/free/18147859
https://www.chinaacc.com/ask/detail/free/18147802
https://www.chinaacc.com/ask/detail/free/18147733
https://www.chinaacc.com/ask/detail/free/18147580
https://www.chinaacc.com/ask/detail/free/18147503
https://www.chinaacc.com/ask/detail/free/18147419
https://www.chinaacc.com/ask/detail/free/18147163
https://www.chinaacc.com/ask/detail/free/18147111
https://www.chinaacc.com/ask/detail/free/18147035
https://www.chinaacc.com/ask/detail/free/18146870
https://www.chinaacc.com/ask/detail/free/18146807
https://www.chinaacc.com/ask/detail/free/18146758
https://www.chinaacc.com/ask/detail/free/18146679
https://www.chinaacc.com/ask/detail/free/18146621
https://www.chinaacc.com/ask/detail/free/18146480
https://www.chinaacc.com/ask/detail/free/18146373
https://www.chinaacc.com/ask/detail/free/18146268
https://www.chinaacc.com/ask/detail/free/18146161
https://www.chinaacc.com/ask/detail/free/18146103
https://www.chinaacc.com/ask/detail/free/18145940
https://www.chinaacc.com/ask/detail/free/18145838
https://www.chinaacc.com/ask/detail/free/18145647
https://www.chinaacc.com/ask/detail/free/18145527
https://www.chinaacc.com/ask/detail/free/18145203
https://www.chinaacc.com/ask/detail/free/18145094
https://www.chinaacc.com/ask/detail/free/18145039
https://www.chinaacc.com/ask/detail/free/18144897
https://www.chinaacc.com/ask/detail/free/18144784
https://www.chinaacc.com/ask/detail/free/18144674
https://www.chinaacc.com/ask/detail/free/18144540
https://www.chinaacc.com/ask/detail/free/18144449
https://www.chinaacc.com/ask/detail/free/18144393
https://www.chinaacc.com/ask/detail/free/18144027
https://www.chinaacc.com/ask/detail/free/18143930
https://www.chinaacc.com/ask/detail/free/18143869
https://www.chinaacc.com/ask/detail/free/18143521
https://www.chinaacc.com/ask/detail/free/18143419
https://www.chinaacc.com/ask/detail/free/18143258
https://www.chinaacc.com/ask/detail/free/18143044
https://www.chinaacc.com/ask/detail/free/18142809
https://www.chinaacc.com/ask/detail/free/18141095
https://www.chinaacc.com/ask/detail/free/18140833
https://www.chinaacc.com/ask/detail/free/18140767
https://www.chinaacc.com/ask/detail/free/18140608
https://www.chinaacc.com/ask/detail/free/18140565
https://www.chinaacc.com/ask/detail/free/18140476
https://www.chinaacc.com/ask/detail/free/18140432
https://www.chinaacc.com/ask/detail/free/18140321
https://www.chinaacc.com/ask/detail/free/18140238
https://www.chinaacc.com/ask/detail/free/18140193
https://www.chinaacc.com/ask/detail/free/18140141
https://www.chinaacc.com/ask/detail/free/18140098
https://www.chinaacc.com/ask/detail/free/18140060
https://www.chinaacc.com/ask/detail/free/18139969
https://www.chinaacc.com/ask/detail/free/18139695
https://www.chinaacc.com/ask/detail/free/18139643
https://www.chinaacc.com/ask/detail/free/18139540
https://www.chinaacc.com/ask/detail/free/18139486
https://www.chinaacc.com/ask/detail/free/18139450
https://www.chinaacc.com/ask/detail/free/18139305
https://www.chinaacc.com/ask/detail/free/18139257
https://www.chinaacc.com/ask/detail/free/18139216
https://www.chinaacc.com/ask/detail/free/18139135
https://www.chinaacc.com/ask/detail/free/18139100
https://www.chinaacc.com/ask/detail/free/18139014
https://www.chinaacc.com/ask/detail/free/18138955
https://www.chinaacc.com/ask/detail/free/18138922
https://www.chinaacc.com/ask/detail/free/18138885
https://www.chinaacc.com/ask/detail/free/18138782
https://www.chinaacc.com/ask/detail/free/18138698
https://www.chinaacc.com/ask/detail/free/18138653
https://www.chinaacc.com/ask/detail/free/18138592
https://www.chinaacc.com/ask/detail/free/18138547
https://www.chinaacc.com/ask/detail/free/18138512
https://www.chinaacc.com/ask/detail/free/18138493
https://www.chinaacc.com/ask/detail/free/18138422
https://www.chinaacc.com/ask/detail/free/18138353
https://www.chinaacc.com/ask/detail/free/18138306
https://www.chinaacc.com/ask/detail/free/18138263
https://www.chinaacc.com/ask/detail/free/18138230
https://www.chinaacc.com/ask/detail/free/18138187
https://www.chinaacc.com/ask/detail/free/18138099
https://www.chinaacc.com/ask/detail/free/18138060
https://www.chinaacc.com/ask/detail/free/18138021
https://www.chinaacc.com/ask/detail/free/18137938
https://www.chinaacc.com/ask/detail/free/18137892
https://www.chinaacc.com/ask/detail/free/18137806
https://www.chinaacc.com/ask/detail/free/18137766
https://www.chinaacc.com/ask/detail/free/18137727
https://www.chinaacc.com/ask/detail/free/18137683
https://www.chinaacc.com/ask/detail/free/18137624
https://www.chinaacc.com/ask/detail/free/18137582
https://www.chinaacc.com/ask/detail/free/18137508
https://www.chinaacc.com/ask/detail/free/18137445
https://www.chinaacc.com/ask/detail/free/18137416
https://www.chinaacc.com/ask/detail/free/18137342
https://www.chinaacc.com/ask/detail/free/18137292
https://www.chinaacc.com/ask/detail/free/18137262
https://www.chinaacc.com/ask/detail/free/18137182
https://www.chinaacc.com/ask/detail/free/18137086
https://www.chinaacc.com/ask/detail/free/18136972
https://www.chinaacc.com/ask/detail/free/18136877
https://www.chinaacc.com/ask/detail/free/18136829
https://www.chinaacc.com/ask/detail/free/18136786
https://www.chinaacc.com/ask/detail/free/18136732
https://www.chinaacc.com/ask/detail/free/18136679
https://www.chinaacc.com/ask/detail/free/18136556
https://www.chinaacc.com/ask/detail/free/18136485
https://www.chinaacc.com/ask/detail/free/18136440
https://www.chinaacc.com/ask/detail/free/18136384
https://www.chinaacc.com/ask/detail/free/18136303
https://www.chinaacc.com/ask/detail/free/18136161
https://www.chinaacc.com/ask/detail/free/18136111
https://www.chinaacc.com/ask/detail/free/18136027
https://www.chinaacc.com/ask/detail/free/18135991
https://www.chinaacc.com/ask/detail/free/18135946
https://www.chinaacc.com/ask/detail/free/18135897
https://www.chinaacc.com/ask/detail/free/18135682
https://www.chinaacc.com/ask/detail/free/18135642
https://www.chinaacc.com/ask/detail/free/18135612
https://www.chinaacc.com/ask/detail/free/18135531
https://www.chinaacc.com/ask/detail/free/18135494
https://www.chinaacc.com/ask/detail/free/18135425
https://www.chinaacc.com/ask/detail/free/18135341
https://www.chinaacc.com/ask/detail/free/18135277
https://www.chinaacc.com/ask/detail/free/18135233
https://www.chinaacc.com/ask/detail/free/18135126
https://www.chinaacc.com/ask/detail/free/18135066
https://www.chinaacc.com/ask/detail/free/18134977
https://www.chinaacc.com/ask/detail/free/18134933
https://www.chinaacc.com/ask/detail/free/18134816
https://www.chinaacc.com/ask/detail/free/18134710
https://www.chinaacc.com/ask/detail/free/18134663
https://www.chinaacc.com/ask/detail/free/18134598
https://www.chinaacc.com/ask/detail/free/18134544
https://www.chinaacc.com/ask/detail/free/18134490
https://www.chinaacc.com/ask/detail/free/18134430
https://www.chinaacc.com/ask/detail/free/18134361
https://www.chinaacc.com/ask/detail/free/18134321
https://www.chinaacc.com/ask/detail/free/18134239
https://www.chinaacc.com/ask/detail/free/18134195
https://www.chinaacc.com/ask/detail/free/18134123
https://www.chinaacc.com/ask/detail/free/18134082
https://www.chinaacc.com/ask/detail/free/18134028
https://www.chinaacc.com/ask/detail/free/18133961
https://www.chinaacc.com/ask/detail/free/18133894
https://www.chinaacc.com/ask/detail/free/18133850
https://www.chinaacc.com/ask/detail/free/18133811
https://www.chinaacc.com/ask/detail/free/18133762
https://www.chinaacc.com/ask/detail/free/18133708
https://www.chinaacc.com/ask/detail/free/18133650
https://www.chinaacc.com/ask/detail/free/18133614
https://www.chinaacc.com/ask/detail/free/18133246
https://www.chinaacc.com/ask/detail/free/18133203
https://www.chinaacc.com/ask/detail/free/18133153
https://www.chinaacc.com/ask/detail/free/18133092
https://www.chinaacc.com/ask/detail/free/18133056
https://www.chinaacc.com/ask/detail/free/18132947
https://www.chinaacc.com/ask/detail/free/18132873
https://www.chinaacc.com/ask/detail/free/18132788
https://www.chinaacc.com/ask/detail/free/18132730
https://www.chinaacc.com/ask/detail/free/18132669
https://www.chinaacc.com/ask/detail/free/18132621
https://www.chinaacc.com/ask/detail/free/18132575
https://www.chinaacc.com/ask/detail/free/18132524
https://www.chinaacc.com/ask/detail/free/18132472
https://www.chinaacc.com/ask/detail/free/18132397
https://www.chinaacc.com/ask/detail/free/18132322
https://www.chinaacc.com/ask/detail/free/18132231
https://www.chinaacc.com/ask/detail/free/18132133
https://www.chinaacc.com/ask/detail/free/18132060
https://www.chinaacc.com/ask/detail/free/18131936
https://www.chinaacc.com/ask/detail/free/18131881
https://www.chinaacc.com/ask/detail/free/18131799
https://www.chinaacc.com/ask/detail/free/18131730
https://www.chinaacc.com/ask/detail/free/18131678
https://www.chinaacc.com/ask/detail/free/18131654
https://www.chinaacc.com/ask/detail/free/18131577
https://www.chinaacc.com/ask/detail/free/18131523
https://www.chinaacc.com/ask/detail/free/18131441
https://www.chinaacc.com/ask/detail/free/18131343
https://www.chinaacc.com/ask/detail/free/18131271
https://www.chinaacc.com/ask/detail/free/18131131
https://www.chinaacc.com/ask/detail/free/18129151
https://www.chinaacc.com/ask/detail/free/18119019
https://www.chinaacc.com/ask/detail/free/18118867
https://www.chinaacc.com/ask/detail/free/18118798
https://www.chinaacc.com/ask/detail/free/18118723
https://www.chinaacc.com/ask/detail/free/18118646
https://www.chinaacc.com/ask/detail/free/18118583
https://www.chinaacc.com/ask/detail/free/18118525
https://www.chinaacc.com/ask/detail/free/18118354
https://www.chinaacc.com/ask/detail/free/18118092
https://www.chinaacc.com/ask/detail/free/18117680
https://www.chinaacc.com/ask/detail/free/18117459
https://www.chinaacc.com/ask/detail/free/18117399
https://www.chinaacc.com/ask/detail/free/18116536
https://www.chinaacc.com/ask/detail/free/18116212
https://www.chinaacc.com/ask/detail/free/18116007
https://www.chinaacc.com/ask/detail/free/18115934
https://www.chinaacc.com/ask/detail/free/18115761
https://www.chinaacc.com/ask/detail/free/18115568
https://www.chinaacc.com/ask/detail/free/18115462
https://www.chinaacc.com/ask/detail/free/18115412
https://www.chinaacc.com/ask/detail/free/18115368
https://www.chinaacc.com/ask/detail/free/18115338
https://www.chinaacc.com/ask/detail/free/18115238
https://www.chinaacc.com/ask/detail/free/18115150
https://www.chinaacc.com/ask/detail/free/18115113
https://www.chinaacc.com/ask/detail/free/18114988
https://www.chinaacc.com/ask/detail/free/18114838
https://www.chinaacc.com/ask/detail/free/18114781
https://www.chinaacc.com/ask/detail/free/18114721
https://www.chinaacc.com/ask/detail/free/18114653
https://www.chinaacc.com/ask/detail/free/18114608
https://www.chinaacc.com/ask/detail/free/18114545
https://www.chinaacc.com/ask/detail/free/18114506
https://www.chinaacc.com/ask/detail/free/18114450
https://www.chinaacc.com/ask/detail/free/18114412
https://www.chinaacc.com/ask/detail/free/18114138
https://www.chinaacc.com/ask/detail/free/18112949
https://www.chinaacc.com/ask/detail/free/18112897
https://www.chinaacc.com/ask/detail/free/18112845
https://www.chinaacc.com/ask/detail/free/18112770
https://www.chinaacc.com/ask/detail/free/18112744
https://www.chinaacc.com/ask/detail/free/18112701
https://www.chinaacc.com/ask/detail/free/18112670
https://www.chinaacc.com/ask/detail/free/18112611
https://www.chinaacc.com/ask/detail/free/18112582
https://www.chinaacc.com/ask/detail/free/18112537
https://www.chinaacc.com/ask/detail/free/18112496
https://www.chinaacc.com/ask/detail/free/18112429
https://www.chinaacc.com/ask/detail/free/18112341
https://www.chinaacc.com/ask/detail/free/18112290
https://www.chinaacc.com/ask/detail/free/18112236
https://www.chinaacc.com/ask/detail/free/18112158
https://www.chinaacc.com/ask/detail/free/18112133
https://www.chinaacc.com/ask/detail/free/18112014
https://www.chinaacc.com/ask/detail/free/18111978
https://www.chinaacc.com/ask/detail/free/18111930
https://www.chinaacc.com/ask/detail/free/18111863
https://www.chinaacc.com/ask/detail/free/18111811
https://www.chinaacc.com/ask/detail/free/18111765
https://www.chinaacc.com/ask/detail/free/18111730
https://www.chinaacc.com/ask/detail/free/18111702
https://www.chinaacc.com/ask/detail/free/18111668
https://www.chinaacc.com/ask/detail/free/18111588
https://www.chinaacc.com/ask/detail/free/18111558
https://www.chinaacc.com/ask/detail/free/18111521
https://www.chinaacc.com/ask/detail/free/18111477
https://www.chinaacc.com/ask/detail/free/18111436
https://www.chinaacc.com/ask/detail/free/18111249
https://www.chinaacc.com/ask/detail/free/18111216
https://www.chinaacc.com/ask/detail/free/18111164
https://www.chinaacc.com/ask/detail/free/18111115
https://www.chinaacc.com/ask/detail/free/18111046
https://www.chinaacc.com/ask/detail/free/18110992
https://www.chinaacc.com/ask/detail/free/18110960
https://www.chinaacc.com/ask/detail/free/18110909
https://www.chinaacc.com/ask/detail/free/18110877
https://www.chinaacc.com/ask/detail/free/18110831
https://www.chinaacc.com/ask/detail/free/18110806
https://www.chinaacc.com/ask/detail/free/18110769
https://www.chinaacc.com/ask/detail/free/18110740
https://www.chinaacc.com/ask/detail/free/18110699
https://www.chinaacc.com/ask/detail/free/18110663
https://www.chinaacc.com/ask/detail/free/18110615
https://www.chinaacc.com/ask/detail/free/18110578
https://www.chinaacc.com/ask/detail/free/18110538
https://www.chinaacc.com/ask/detail/free/18110497
https://www.chinaacc.com/ask/detail/free/18110474
https://www.chinaacc.com/ask/detail/free/18110441
https://www.chinaacc.com/ask/detail/free/18110406
https://www.chinaacc.com/ask/detail/free/18110369
https://www.chinaacc.com/ask/detail/free/18110326
https://www.chinaacc.com/ask/detail/free/18110269
https://www.chinaacc.com/ask/detail/free/18110204
https://www.chinaacc.com/ask/detail/free/18110149
https://www.chinaacc.com/ask/detail/free/18110097
https://www.chinaacc.com/ask/detail/free/18110065
https://www.chinaacc.com/ask/detail/free/18110031
https://www.chinaacc.com/ask/detail/free/18109996
https://www.chinaacc.com/ask/detail/free/18109961
https://www.chinaacc.com/ask/detail/free/18109920
https://www.chinaacc.com/ask/detail/free/18109876
https://www.chinaacc.com/ask/detail/free/18109838
https://www.chinaacc.com/ask/detail/free/18109802
https://www.chinaacc.com/ask/detail/free/18109772
https://www.chinaacc.com/ask/detail/free/18109724
https://www.chinaacc.com/ask/detail/free/18109678
https://www.chinaacc.com/ask/detail/free/18109611
https://www.chinaacc.com/ask/detail/free/18109575
https://www.chinaacc.com/ask/detail/free/18109538
https://www.chinaacc.com/ask/detail/free/18109481
https://www.chinaacc.com/ask/detail/free/18109433
https://www.chinaacc.com/ask/detail/free/18109410
https://www.chinaacc.com/ask/detail/free/18109383
https://www.chinaacc.com/ask/detail/free/18109356
https://www.chinaacc.com/ask/detail/free/18109307
https://www.chinaacc.com/ask/detail/free/18109264
https://www.chinaacc.com/ask/detail/free/18109218
https://www.chinaacc.com/ask/detail/free/18109180
https://www.chinaacc.com/ask/detail/free/18109149
https://www.chinaacc.com/ask/detail/free/18109110
https://www.chinaacc.com/ask/detail/free/18109065
https://www.chinaacc.com/ask/detail/free/18109022
https://www.chinaacc.com/ask/detail/free/18108993
https://www.chinaacc.com/ask/detail/free/18108968
https://www.chinaacc.com/ask/detail/free/18108930
https://www.chinaacc.com/ask/detail/free/18108892
https://www.chinaacc.com/ask/detail/free/18108851
https://www.chinaacc.com/ask/detail/free/18108823
https://www.chinaacc.com/ask/detail/free/18108795
https://www.chinaacc.com/ask/detail/free/18108765
https://www.chinaacc.com/ask/detail/free/18108728
https://www.chinaacc.com/ask/detail/free/18108689
https://www.chinaacc.com/ask/detail/free/18108641
https://www.chinaacc.com/ask/detail/free/18108605
https://www.chinaacc.com/ask/detail/free/18108551
https://www.chinaacc.com/ask/detail/free/18108502
https://www.chinaacc.com/ask/detail/free/18108451
https://www.chinaacc.com/ask/detail/free/18108425
https://www.chinaacc.com/ask/detail/free/18108345
https://www.chinaacc.com/ask/detail/free/18108292
https://www.chinaacc.com/ask/detail/free/18108265
https://www.chinaacc.com/ask/detail/free/18108219
https://www.chinaacc.com/ask/detail/free/18108178
https://www.chinaacc.com/ask/detail/free/18108131
https://www.chinaacc.com/ask/detail/free/18108083
https://www.chinaacc.com/ask/detail/free/18108053
https://www.chinaacc.com/ask/detail/free/18108022
https://www.chinaacc.com/ask/detail/free/18108002
https://www.chinaacc.com/ask/detail/free/18107945
https://www.chinaacc.com/ask/detail/free/18107910
https://www.chinaacc.com/ask/detail/free/18107866
https://www.chinaacc.com/ask/detail/free/18107834
https://www.chinaacc.com/ask/detail/free/18107770
https://www.chinaacc.com/ask/detail/free/18107741
https://www.chinaacc.com/ask/detail/free/18107709
https://www.chinaacc.com/ask/detail/free/18107674
https://www.chinaacc.com/ask/detail/free/18107634
https://www.chinaacc.com/ask/detail/free/18107604
https://www.chinaacc.com/ask/detail/free/18107563
https://www.chinaacc.com/ask/detail/free/18107513
https://www.chinaacc.com/ask/detail/free/18107466
https://www.chinaacc.com/ask/detail/free/18107438
https://www.chinaacc.com/ask/detail/free/18107375
https://www.chinaacc.com/ask/detail/free/18107335
https://www.chinaacc.com/ask/detail/free/18107302
https://www.chinaacc.com/ask/detail/free/18107248
https://www.chinaacc.com/ask/detail/free/18107200
https://www.chinaacc.com/ask/detail/free/18107154
https://www.chinaacc.com/ask/detail/free/18107118
https://www.chinaacc.com/ask/detail/free/18107080
https://www.chinaacc.com/ask/detail/free/18107044
https://www.chinaacc.com/ask/detail/free/18107013
https://www.chinaacc.com/ask/detail/free/18106962
https://www.chinaacc.com/ask/detail/free/18106904
https://www.chinaacc.com/ask/detail/free/18106867
https://www.chinaacc.com/ask/detail/free/18106825
https://www.chinaacc.com/ask/detail/free/18106792
https://www.chinaacc.com/ask/detail/free/18106760
https://www.chinaacc.com/ask/detail/free/18106725
https://www.chinaacc.com/ask/detail/free/18106697
https://www.chinaacc.com/ask/detail/free/18106652
https://www.chinaacc.com/ask/detail/free/18106625
https://www.chinaacc.com/ask/detail/free/18106588
https://www.chinaacc.com/ask/detail/free/18106550
https://www.chinaacc.com/ask/detail/free/18106516
https://www.chinaacc.com/ask/detail/free/18106493
https://www.chinaacc.com/ask/detail/free/18106439
https://www.chinaacc.com/ask/detail/free/18106414
https://www.chinaacc.com/ask/detail/free/18106353
https://www.chinaacc.com/ask/detail/free/18106303
https://www.chinaacc.com/ask/detail/free/18106259
https://www.chinaacc.com/ask/detail/free/18106222
https://www.chinaacc.com/ask/detail/free/18106187
https://www.chinaacc.com/ask/detail/free/18106151
https://www.chinaacc.com/ask/detail/free/18106108
https://www.chinaacc.com/ask/detail/free/18106033
https://www.chinaacc.com/ask/detail/free/18105987
https://www.chinaacc.com/ask/detail/free/18105932
https://www.chinaacc.com/ask/detail/free/18105895
https://www.chinaacc.com/ask/detail/free/18105861
https://www.chinaacc.com/ask/detail/free/18105821
https://www.chinaacc.com/ask/detail/free/18105768
https://www.chinaacc.com/ask/detail/free/18105734
https://www.chinaacc.com/ask/detail/free/18105705
https://www.chinaacc.com/ask/detail/free/18105661
https://www.chinaacc.com/ask/detail/free/18105635
https://www.chinaacc.com/ask/detail/free/18105595
https://www.chinaacc.com/ask/detail/free/18105535
https://www.chinaacc.com/ask/detail/free/18105500
https://www.chinaacc.com/ask/detail/free/18105444
https://www.chinaacc.com/ask/detail/free/18105389
https://www.chinaacc.com/ask/detail/free/18105343
https://www.chinaacc.com/ask/detail/free/18105315
https://www.chinaacc.com/ask/detail/free/18105275
https://www.chinaacc.com/ask/detail/free/18105207
https://www.chinaacc.com/ask/detail/free/18105181
https://www.chinaacc.com/ask/detail/free/18105139
https://www.chinaacc.com/ask/detail/free/18105106
https://www.chinaacc.com/ask/detail/free/18105071
https://www.chinaacc.com/ask/detail/free/18105031
https://www.chinaacc.com/ask/detail/free/18104969
https://www.chinaacc.com/ask/detail/free/18104939
https://www.chinaacc.com/ask/detail/free/18104904
https://www.chinaacc.com/ask/detail/free/18104865
https://www.chinaacc.com/ask/detail/free/18104833
https://www.chinaacc.com/ask/detail/free/18104799
https://www.chinaacc.com/ask/detail/free/18104762
https://www.chinaacc.com/ask/detail/free/18104714
https://www.chinaacc.com/ask/detail/free/18104682
https://www.chinaacc.com/ask/detail/free/18104642
https://www.chinaacc.com/ask/detail/free/18104581
https://www.chinaacc.com/ask/detail/free/18104535
https://www.chinaacc.com/ask/detail/free/18104507
https://www.chinaacc.com/ask/detail/free/18104464
https://www.chinaacc.com/ask/detail/free/18104428
https://www.chinaacc.com/ask/detail/free/18104384
https://www.chinaacc.com/ask/detail/free/18104343
https://www.chinaacc.com/ask/detail/free/18104313
https://www.chinaacc.com/ask/detail/free/18104252
https://www.chinaacc.com/ask/detail/free/18104207
https://www.chinaacc.com/ask/detail/free/18104148
https://www.chinaacc.com/ask/detail/free/18104097
https://www.chinaacc.com/ask/detail/free/18104069
https://www.chinaacc.com/ask/detail/free/18104022
https://www.chinaacc.com/ask/detail/free/18103982
https://www.chinaacc.com/ask/detail/free/18103910
https://www.chinaacc.com/ask/detail/free/18103821
https://www.chinaacc.com/ask/detail/free/18103789
https://www.chinaacc.com/ask/detail/free/18103730
https://www.chinaacc.com/ask/detail/free/18103691
https://www.chinaacc.com/ask/detail/free/18103592
https://www.chinaacc.com/ask/detail/free/18103561
https://www.chinaacc.com/ask/detail/free/18103515
https://www.chinaacc.com/ask/detail/free/18103423
https://www.chinaacc.com/ask/detail/free/18103386
https://www.chinaacc.com/ask/detail/free/18103344
https://www.chinaacc.com/ask/detail/free/18103300
https://www.chinaacc.com/ask/detail/free/18103255
https://www.chinaacc.com/ask/detail/free/18103217
https://www.chinaacc.com/ask/detail/free/18103178
https://www.chinaacc.com/ask/detail/free/18103107
https://www.chinaacc.com/ask/detail/free/18103052
https://www.chinaacc.com/ask/detail/free/18102988
https://www.chinaacc.com/ask/detail/free/18102942
https://www.chinaacc.com/ask/detail/free/18102885
https://www.chinaacc.com/ask/detail/free/18102848
https://www.chinaacc.com/ask/detail/free/18102802
https://www.chinaacc.com/ask/detail/free/18102745
https://www.chinaacc.com/ask/detail/free/18102682
https://www.chinaacc.com/ask/detail/free/18102653
https://blog.csdn.net/ccwangxiao/article/details/108564659
https://blog.csdn.net/ccwangxiao/article/details/108564674
https://blog.csdn.net/ccwangxiao/article/details/108564685
https://blog.csdn.net/ccwangxiao/article/details/108564702
https://blog.csdn.net/ccwangxiao/article/details/108564718
https://blog.csdn.net/ccwangxiao/article/details/108564734
https://blog.csdn.net/ccwangxiao/article/details/108564754
https://blog.csdn.net/ccwangxiao/article/details/108564763