【MYsql】Maxscale中间件使用

背景:
  1. 规划
    192.168.6.114 mysql master
    192.168.6.115
    mysql salve1
    192.168.6.119 mysql slave2
    192.168.6.121 maxscale

    搭建114 115 119的一主两丛结构并开启半同步复制,和搭建mha一样
  2. 在搭建完主从后我们还需要在数据库建一个用于monitor的用户,同时建好后面测试的用户:
    mysql> grant all privileges on *.* to 'max'@'%' identified by 'ESBecs00';   ---为了省事就用这一个用户吧 






1安装maxscale

  1. 下载链接:https://downloads.mariadb.com/files/MaxScale/1.3.0-beta/centos/5/x86_64
  2. 本次博客安装的是
  3. maxscale-beta-1.3.0-1.centos5.x86_64.rpm 该版本

2 修改配置文件

  1. 安装完后会在/etc/目录下生成模板配置文件
  2. [root@localhost etc]# ls maxscale.cnf.template     
    maxscale.cnf.template

  3. 写一个配置文件(每个版本的配置文件不一样,最好是cp模板配置文件来修改)
    1. [root@localhost etc]# more maxscale.cnf
    2. # MaxScale documentation on GitHub:
    3. # https://github.com/mariadb-corporation/MaxScale/blob/master/Documentation/Documentation-Contents.md

    4. # Global parameters
    5. #
    6. # Number of threads is autodetected, uncomment for manual configuration
    7. # Complete list of configuration options:
    8. # https://github.com/mariadb-corporation/MaxScale/blob/master/Documentation/Getting-Started/Configuration-Guide.md

    9. [maxscale]
    10. threads=8

    11. # Server definitions
    12. #
    13. # Set the address of the server to the network
    14. # address of a MySQL server.
    15. #

    16. [server1]
    17. type=server
    18. address=192.168.6.114
    19. port=3306
    20. protocol=MySQLBackend

    21. [server2]
    22. type=server
    23. address=192.168.6.115
    24. port=3306
    25. protocol=MySQLBackend

    26. [server3]
    27. type=server
    28. address=192.168.6.119
    29. port=3306
    30. protocol=MySQLBackend

    31. # Monitor for the servers
    32. #
    33. # This will keep MaxScale aware of the state of the servers.
    34. # MySQL Monitor documentation:
    35. # https://github.com/mariadb-corporation/MaxScale/blob/master/Documentation/Monitors/MySQL-Monitor.md

    36. [MySQL Monitor]
    37. type=monitor
    38. module=mysqlmon
    39. servers=server1,server2,server3
    40. user=max
    41. passwd=ESBecs00
    42. monitor_interval=10000

    43. # Service definitions
    44. #
    45. # Service Definition for a read-only service and
    46. # a read/write splitting service.
    47. #

    48. # https://github.com/mariadb-corporation/MaxScale/blob/master/Documentation/Routers/ReadConnRoute.md

    49. [Read-Only Service]                       ###只读服务
    50. type=service
    51. router=readconnroute
    52. servers=server1,server2,server3
    53. user=max
    54. passwd=ESBecs00
    55. router_options=slave

    56. # https://github.com/mariadb-corporation/MaxScale/blob/master/Documentation/Routers/ReadWriteSplit.md

    57. [Read-Write Service]                       ####写服务
    58. type=service
    59. router=readwritesplit
    60. servers=server1
    61. user=max
    62. passwd=ESBecs00
    63. max_slave_connections=100%

    64. # https://github.com/mariadb-corporation/MaxScale/blob/master/Documentation/Reference/MaxAdmin.md

    65. [MaxAdmin Service]
    66. type=service
    67. router=cli


    68. [Read-Only Listener]
    69. type=listener
    70. service=Read-Only Service
    71. protocol=MySQLClient
    72. port=4008                                  ##读服务启动监听 端口4008

    73. [Read-Write Listener]
    74. type=listener
    75. service=Read-Write Service
    76. protocol=MySQLClient
    77. port=4006                                  ####写服务启动监听 端口

    78. [MaxAdmin Listener]
    79. type=listener
    80. service=MaxAdmin Service
    81. protocol=maxscaled
    82. port=6603                                   ###管理端口



3 启动maxscale服务

  1.   [root@localhost etc]# /etc/init.d/maxscale start
    Starting MaxScale:  found maxscale (pid 
    14126) 正在运行...                                         [确定]
  2. 查看message日志
    1. [root@localhost ~]# tail -f /var/log/messages
    2. Jul 29 17:04:33 localhost maxscale[14035]: Configuration file: /etc/maxscale.cnf
    3. Jul 29 17:04:33 localhost maxscale[14035]: Log directory: /var/log/maxscale               ------日志目录
    4. Jul 29 17:04:33 localhost maxscale[14035]: Data directory: /var/lib/maxscale/data
    5. Jul 29 17:04:33 localhost maxscale[14035]: Module directory: /usr/lib64/maxscale
    6. Jul 29 17:04:33 localhost maxscale[14035]: Service cache: /var/cache/maxscale
    7. Jul 29 17:04:33 localhost maxscale[14035]: Initialise CLI router module V1.0.0.
    8. Jul 29 17:04:33 localhost maxscale[14035]: Loaded module cli: V1.0.0 from /usr/lib64/maxscale/libcli.so
    9. Jul 29 17:04:33 localhost maxscale[14035]: Initializing statemend-based read/write split router module.
    10. Jul 29 17:04:33 localhost maxscale[14035]: Loaded module readwritesplit: V1.0.2 from /usr/lib64/maxscale/libreadwritesplit.so
    11. Jul 29 17:04:33 localhost maxscale[14035]: Initialise readconnroute router module V1.1.0.
    12. Jul 29 17:04:33 localhost maxscale[14035]: Loaded module readconnroute: V1.1.0 from /usr/lib64/maxscale/libreadconnroute.so
    查看日志:
  3. [root@localhost maxscale]# tail -f maxscale1.log 
    1. MariaDB Corporation MaxScale    /var/log/maxscale/maxscale1.log Fri Jul 29 17:09:46 2016
      -----------------------------------------------------------------------
      2016-07-29 17:09:46   notice : Configuration file: /etc/maxscale.cnf
      2016-07-29 17:09:46   notice : Log directory: /var/log/maxscale
      2016-07-29 17:09:46   notice : Data directory: /var/lib/maxscale/data
      2016-07-29 17:09:46   notice : Module directory: /usr/lib64/maxscale
      2016-07-29 17:09:46   notice : Service cache: /var/cache/maxscale
      2016-07-29 17:09:46   notice : Initialise CLI router module V1.0.0.
      2016-07-29 17:09:46   notice : Loaded module cli: V1.0.0 from /usr/lib64/maxscale/libcli.so
      2016-07-29 17:09:46   notice : Initializing statemend-based read/write split router module.
      2016-07-29 17:09:46   notice : Loaded module readwritesplit: V1.0.2 from /usr/lib64/maxscale/libreadwritesplit.so
      2016-07-29 17:09:46   notice : Initialise readconnroute router module V1.1.0.
      2016-07-29 17:09:46   notice : Loaded module readconnroute: V1.1.0 from /usr/lib64/maxscale/libreadconnroute.so
      2016-07-29 17:09:46   notice : Initialise the MySQL Monitor module V1.4.0.
      2016-07-29 17:09:46   notice : Loaded module mysqlmon: V1.4.0 from /usr/lib64/maxscale/libmysqlmon.so
      2016-07-29 17:09:46   notice : MariaDB Corporation MaxScale beta-1.3.0 (C) MariaDB Corporation Ab 2013-2015
      2016-07-29 17:09:46   notice : MaxScale is running in process 14140
      2016-07-29 17:09:46   notice : Loaded 2 MySQL Users for service [Read-Only Service].
      2016-07-29 17:09:46   notice : Loaded module MySQLClient: V1.0.0 from /usr/lib64/maxscale/libMySQLClient.so
      2016-07-29 17:09:46   notice : Listening MySQL connections at 0.0.0.0:4008
      2016-07-29 17:09:46   notice : Loaded 2 MySQL Users for service [Read-Write Service].
      2016-07-29 17:09:46   notice : Listening MySQL connections at 0.0.0.0:4006
      2016-07-29 17:09:46   notice : Loaded module maxscaled: V1.0.0 from /usr/lib64/maxscale/libmaxscaled.so
      2016-07-29 17:09:46   notice : Listening maxscale connections at 0.0.0.0:6603
      2016-07-29 17:09:46   notice : Started MaxScale log flusher.
      2016-07-29 17:09:46   notice : MaxScale started with 8 server threads.
      2016-07-29 17:09:47   notice : A Master Server is now available: 192.168.6.114:3306




4测试

  1. 读负载均衡测试
    1. [root@localhost etc]# mysql -umax -pESBecs00 -h192.168.6.121 -P4008 -e "show tables" chenliang;
    2. Warning: Using a password on the command line interface can be insecure.
    3. +---------------------+
    4. | Tables_in_chenliang |
    5. +---------------------+
    6. | test115 |
    7. +---------------------+
    8. [root@localhost etc]# mysql -umax -pESBecs00 -h192.168.6.121 -P4008 -e "show tables" chenliang;
    9. Warning: Using a password on the command line interface can be insecure.
    10. +---------------------+
    11. | Tables_in_chenliang |
    12. +---------------------+
    13. | test119 |
    14. +---------------------+
    15. [root@localhost etc]# mysql -umax -pESBecs00 -h192.168.6.121 -P4008 -e "show tables" chenliang;
    16. Warning: Using a password on the command line interface can be insecure.
    17. +---------------------+
    18. | Tables_in_chenliang |
    19. +---------------------+
    20. | test115 |
    21. +---------------------+
    22. [root@localhost etc]# mysql -umax -pESBecs00 -h192.168.6.121 -P4008 -e "show tables" chenliang;
    23. Warning: Using a password on the command line interface can be insecure.
    24. +---------------------+
    25. | Tables_in_chenliang |
    26. +---------------------+
    27. | test119 |
    28. +---------------------+

  2. 写测试
    1. [root@localhost etc]# mysql -umax -pESBecs00 -h192.168.6.121 -P4006 -e "create table tt as select * from mysql.user" chenliang;    --创建一个表
    2. Warning: Using a password on the command line interface can be insecure.
    3. [root@localhost etc]# mysql -umax -pESBecs00 -h192.168.6.121 -P4008 -e "show tables" chenliang;                                     ---同步到slave1
    4. Warning: Using a password on the command line interface can be insecure.
    5. +---------------------+
    6. | Tables_in_chenliang |
    7. +---------------------+
    8. | test115 |
    9. | tt |
    10. +---------------------+
    11. [root@localhost etc]# mysql -umax -pESBecs00 -h192.168.6.121 -P4008 -e "show tables" chenliang;                                     ---同步到slave2了,很明显写在了主库上,可以开generlog去验证
    12. Warning: Using a password on the command line interface can be insecure.
    13. +---------------------+
    14. | Tables_in_chenliang |
    15. +---------------------+
    16. | test119 |
    17. | tt |
    18. +---------------------+


5 后台管理

  1. [root@localhost ~]# maxadmin -pmariadb
  2. MaxScale>
  3. MaxScale>
  4. MaxScale> list servers
  5. Servers.
  6. -------------------+-----------------+-------+-------------+--------------------
  7. Server | Address | Port | Connections | Status
  8. -------------------+-----------------+-------+-------------+--------------------
  9. server1 | 192.168.6.114 | 3306 | 0 | Master, Running
  10. server2 | 192.168.6.115 | 3306 | 0 | Slave, Running
  11. server3 | 192.168.6.119 | 3306 | 0 | Slave, Running
  12. -------------------+-----------------+-------+-------------+--------------------
  13. MaxScale> list services
  14. Services.
  15. --------------------------+----------------------+--------+---------------
  16. Service Name | Router Module | #Users | Total Sessions
  17. --------------------------+----------------------+--------+---------------
  18. Read-Only Service | readconnroute | 1 | 3
  19. Read-Write Service | readwritesplit | 1 | 1
  20. MaxAdmin Service | cli | 5 | 5
  21. --------------------------+----------------------+--------+---------------

故障测试之:一台slave,stop

  1. 原始状态
    1. MaxScale> list servers
    2. Servers.
    3. -------------------+-----------------+-------+-------------+--------------------
    4. Server | Address | Port | Connections | Status
    5. -------------------+-----------------+-------+-------------+--------------------
    6. server1 | 192.168.6.114 | 3306 | 0 | Master, Running
    7. server2 | 192.168.6.115 | 3306 | 0 | Slave, Running
    8. server3 | 192.168.6.119 | 3306 | 0 | Slave, Running
    9. -------------------+-----------------+-------+-------------+--------------------

    停掉119的slave,stop slave;查看状态
  2. MaxScale> list servers
    Servers.
    -------------------+-----------------+-------+-------------+--------------------
    Server             | Address         | Port  | Connections | Status              
    -------------------+-----------------+-------+-------------+--------------------
    server1            | 192.168.6.114   |  3306 |           0 | Master, Running
    server2            | 192.168.6.115   |  3306 |           0 | Slave, Running
    server3            | 192.168.6.119   |  3306 |           0 | Running
    -------------------+-----------------+-------+-------------+--------------------

  3. 此时读负载测试:全部落到正常的那台slave
    1. [root@node4 ~]# mysql -umax -pESBecs00 -h192.168.6.121 -P4008 -e "show tables" chenliang;
      +---------------------+
      | Tables_in_chenliang |
      +---------------------+
      | test115             | 
      | tt                  | 
      | tt2                 | 
      +---------------------+
      [root@node4 ~]# mysql -umax -pESBecs00 -h192.168.6.121 -P4008 -e "show tables" chenliang;
      +---------------------+
      | Tables_in_chenliang |
      +---------------------+
      | test115             | 
      | tt                  | 
      | tt2                 | 
      +---------------------+
      [root@node4 ~]# mysql -umax -pESBecs00 -h192.168.6.121 -P4008 -e "show tables" chenliang;
      +---------------------+
      | Tables_in_chenliang |
      +---------------------+
      | test115             | 
      | tt                  | 
      | tt2                 | 
      +---------------------+


恢复119 在测试:负载均衡正常了
  1. MaxScale> list servers
  2. Servers.
  3. -------------------+-----------------+-------+-------------+--------------------
  4. Server | Address | Port | Connections | Status
  5. -------------------+-----------------+-------+-------------+--------------------
  6. server1 | 192.168.6.114 | 3306 | 0 | Master, Running
  7. server2 | 192.168.6.115 | 3306 | 0 | Slave, Running
  8. server3 | 192.168.6.119 | 3306 | 0 | Slave, Running
  9. -------------------+-----------------+-------+-------------+--------------------
[root@node4 ~]# mysql -umax -pESBecs00 -h192.168.6.121 -P4008 -e "show tables" chenliang;
+---------------------+
| Tables_in_chenliang |
+---------------------+
| test119             | 
| tt                  | 
| tt2                 | 
+---------------------+
[root@node4 ~]# mysql -umax -pESBecs00 -h192.168.6.121 -P4008 -e "show tables" chenliang;
+---------------------+
| Tables_in_chenliang |
+---------------------+
| test115             | 
| tt                  | 
| tt2                 | 
+---------------------+





故障测试之:两台slave全部stop

  1. 停掉1156,119的slave;stop slave
  2. 查看状态
    1. MaxScale> list servers
    2. Servers.
    3. -------------------+-----------------+-------+-------------+--------------------
    4. Server | Address | Port | Connections | Status
    5. -------------------+-----------------+-------+-------------+--------------------
    6. server1 | 192.168.6.114 | 3306 | 0 | Running
    7. server2 | 192.168.6.115 | 3306 | 0 | Running
    8. server3 | 192.168.6.119 | 3306 | 0 | Running
    9. -------------------+-----------------+-------+-------------+--------------------

  3. 负载测试:此时负载就出问题,会报错,起码得保持一台slave,存活才行
    1. [root@node4 ~]# mysql -umax -pESBecs00 -h192.168.6.121 -P4008 -e "show tables" chenliang;
    2. ERROR 1045 (28000): failed to create new session

    说明从服务器全部失效后,会导致 master 也无法识别,使整个数据库服务都失效了。


    对于 slave 全部失效的情况,能否让 master 还可用?这样至少可以正常提供数据库服务。


    这需要修改 MaxScale 的配置,告诉 MaxScale 我们需要一个稳定的 master。

    处理过程


    先恢复两个 slave,让集群回到正常状态,登陆两个 slave 的MySQL。

    mysql> start slave;


    修改 MaxScale 配置文件,添加新的配置。

    vi /etc/maxscale.cnf


    找到 [MySQL Monitor] 部分,添加:

    detect_stale_master=true


    保存退出,然后重启 MaxScale。


    验证


    停掉两台 slave ,查看 MaxScale 服务器状态。

    1. MaxScale> list servers
    2. Servers.
    3. -------------------+-----------------+-------+-------------+--------------------
    4. Server | Address | Port | Connections | Status
    5. -------------------+-----------------+-------+-------------+--------------------
    6. server1 | 192.168.6.114 | 3306 | 0 | Master, Stale Status, Running
    7. server2 | 192.168.6.115 | 3306 | 0 | Running
    8. server3 | 192.168.6.119 | 3306 | 0 | Running
    9. -------------------+-----------------+-------+-------------+--------------------


    负载测试:
    1. [root@node4 ~]# mysql -umax -pESBecs00 -h192.168.6.121 -P4008 -e "show tables" chenliang;
    2. +---------------------+
    3. | Tables_in_chenliang |
    4. +---------------------+
    5. | test114 |
    6. | tt |
    7. | tt2 |
    8. +---------------------+
    9. [root@node4 ~]# mysql -umax -pESBecs00 -h192.168.6.121 -P4008 -e "show tables" chenliang;
    10. +---------------------+
    11. | Tables_in_chenliang |
    12. +---------------------+
    13. | test114 |
    14. | tt |
    15. | tt2 |
    16. +---------------------+
    17. [root@node4 ~]# mysql -umax -pESBecs00 -h192.168.6.121 -P4008 -e "show tables" chenliang;
    18. +---------------------+
    19. | Tables_in_chenliang |
    20. +---------------------+
    21. | test114 |
    22. | tt |
    23. | tt2 |
    24. +---------------------+




补充:可将读写分离配置在一个port上

  1. 配置文件
    1. [maxscale]
    2. threads=80

    3. [server1]
    4. type=server
    5. address=192.168.6.114
    6. port=3306
    7. protocol=MySQLBackend

    8. [server2]
    9. type=server
    10. address=192.168.6.115
    11. port=3306
    12. protocol=MySQLBackend

    13. [server3]
    14. type=server
    15. address=192.168.6.119
    16. port=3306
    17. protocol=MySQLBackend


    18. [MySQL Monitor]
    19. type=monitor
    20. module=mysqlmon
    21. servers=server1,server2,server3
    22. user=max
    23. passwd=ESBecs00
    24. monitor_interval=10000
    25. detect_stale_master=true


    26. #[Read-Only Service]
    27. #type=service
    28. #router=readconnroute
    29. #servers=server1,server2,server3
    30. #user=max
    31. #passwd=ESBecs00
    32. #router_options=slave


    33. [Read-Write Service]
    34. type=service
    35. router=readwritesplit
    36. servers=server1,server3,server2      -----都配置在这即可
    37. user=max
    38. passwd=ESBecs00
    39. max_slave_connections=100%


    40. [MaxAdmin Service]
      type=service
      router=cli




      #[Read-Only Listener]
      #type=listener
      #service=Read-Only Service
      #protocol=MySQLClient
      #port=4008


      [Read-Write Listener]
      type=listener
      service=Read-Write Service
      protocol=MySQLClient
      port=4006


      [MaxAdmin Listener]
      type=listener
      service=MaxAdmin Service
      protocol=maxscaled
      port=6603
    41. #####只需要将 read-only 读负载均衡的部分注释掉,只保留read-write部分即可
    重启测试
    1. 写操作
    2. [root@node4 ~]# mysql -uchenliang -pESBecs00 -h192.168.6.121 -P4006  -e "create table tt3 as select * from tt" chenliang;
    3. 检查114,115,119都存在
    4. mysql> show tables;
      +---------------------+
      | Tables_in_chenliang |
      +---------------------+
      | test119             |
      | tt                  |
      | tt2                 |
      | tt3                 |
      +---------------------+
      4 rows in set (0.00 sec)

     读测试:落在了slave上
    1. [root@node4 ~]# mysql -uchenliang -pESBecs00 -h192.168.6.121 -P4006 -e "show tables" chenliang;
    2. +---------------------+
    3. | Tables_in_chenliang |
    4. +---------------------+
    5. | test115 |
    6. | tt |
    7. | tt2 |
    8. | tt3 |
    9. +---------------------+
    10. [root@node4 ~]# mysql -uchenliang -pESBecs00 -h192.168.6.121 -P4006 -e "show tables" chenliang;
    11. +---------------------+
    12. | Tables_in_chenliang |
    13. +---------------------+
    14. | test115 |
    15. | tt |
    16. | tt2 |
    17. | tt3 |
    18. +---------------------+








对比:
实验一:实现了读写分离,和读的负载均衡,但是需要连接两个不同的端口,不方便
实验二:实现了读写分离,但读负载均衡不能实现,只需要链接一个端口即可,比较方便




思考:

  1. 1.假设程序用户是chenliang  对 chengliang业务库具有所有权,通过chenliang 这个角色能否做到读写分离呢?
    1. 直接使用
    2. [root@localhost etc]# mysql -uchenliang -pESBecs00 -h192.168.6.121 -P4006  -e "create table tt2 as select * from tt" chenliang;
    [root@localhost etc]# mysql -uchenliang -pESBecs00 -h192.168.6.121 -P4008  -e "show tables" chenliang;
    Warning: Using a password on the command line interface can be insecure.
    +---------------------+
    | Tables_in_chenliang |
    +---------------------+
    | test119             |
    | tt                  |
    | tt2                 |
    +---------------------+
    [root@localhost etc]# mysql -uchenliang -pESBecs00 -h192.168.6.121 -P4008  -e "show tables" chenliang;                                         
    Warning: Using a password on the command line interface can be insecure.
    +---------------------+
    | Tables_in_chenliang |
    +---------------------+
    | test115             |
    | tt                  |
    | tt2                 |
    +---------------------+

  1. 完美

2 若直接将程序用户写到配置文件的读写分离部分可以吗?

  1. [Read-Only Service]
  2. type=service
  3. router=readconnroute
  4. servers=server1,server2,server3
  5. user=chenliang
  6. passwd=ESBecs00
  7. router_options=slave

  8. # https://github.com/mariadb-corporation/MaxScale/blob/master/Documentation/Routers/ReadWriteSplit.md

  9. [Read-Write Service]
  10. type=service
  11. router=readwritesplit
  12. servers=server1
  13. user=chenliang
  14. passwd=ESBecs00
  15. max_slave_connections=100%

重新启动观察日志会报错:
  1. or table 'user'
  2. 2016-07-29 17:40:44 warning: Read-Only Service: User 'chenliang' is missing SELECT privileges on mysql.db table. Database name will be ignored in authentication. MySQL error message: SELECT command denied to user 'chenliang'@'192.168.6.121' for table 'db'
  3. 2016-07-29 17:40:44 error : Read-Only Service: Inadequate user permissions for service. Service not started.
  4. 2016-07-29 17:40:44 error : Failed to start service 'Read-Only Service'.
  5. 2016-07-29 17:40:44 error : Read-Write Service: User 'chenliang' is missing SELECT privileges on mysql.user table. MySQL error message: SELECT command denied to user 'chenliang'@'192.168.6.121' for table 'user'
  6. 2016-07-29 17:40:44 warning: Read-Write Service: User 'chenliang' is missing SELECT privileges on mysql.db table. Database name will be ignored in authentication. MySQL error message: SELECT command denied to user 'chenliang'@'192.168.6.121' for table 'db'
  7. 2016-07-29 17:40:44 error : Read-Write Service: Inadequate user permissions for service. Service not started.
  8. 2016-07-29 17:40:44 error : Failed to start service 'Read-Write Service'.
  9. 2016-07-29 17:40:44 notice : Loaded module maxscaled: V1.0.0 from /usr/lib64/maxscale/libmaxscaled.so

  10. 很明显:读写分离的配置用户需要对mysql.user表具有查询权限
建议:为了省事就将[Read-Only Service][Read-Write Service][MySQL Monitor]部分用同一个账户 算了(非应用程序user),可以加密处理,让别人看不到明文密码








通过二进制包安装,源码包安装,密码加密等参考文章
http://www.zhdba.com/zhangyue/2014/07/11/maxscale初探/
http://blog.sina.com.cn/s/blog_534360f50102ver7.html

优酷土豆资深工程师:MySQL高可用之MaxScale与MHA

http://dbaplus.cn/news-11-558-1.html

MaxScale:实现MySQL读写分离与负载均衡的中间件利器

http://mp.weixin.qq.com/s?__biz=MzI4NTA1MDEwNg==&mid=2650756387&idx=1&sn=c0dce4a24c85307cd4c64c8183bffa36&scene=0#wechat_redirect




来自 “ ITPUB博客 ” ,链接:http://blog.itpub.net/29096438/viewspace-2122737/,如需转载,请注明出处,否则将追究法律责任。

转载于:http://blog.itpub.net/29096438/viewspace-2122737/

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值