salt mysql_saltstack应用之批量部署mysql

应用环境：

使用salt写了个自动部署mysql的配置配置管理文件，由于mysql的有版本号，端口等不确定属性，需要使用pillar来单独配置每个minion的属性。

一，原始方法

例如，需要在salt id为10.1.1.1-centos.game.web的服务器上部署mysql，版本号为5.5.25，实例有3个，3306，3307,3308， 正常情况下的流程是这样的：

1，在/srv/salt/top.sls中添加配置信息,确保mysql对应的配置被加载

base:

10.1.1.1-centos.game.web:

- centos.public_services.mysql

1

2

3

base:

10.1.1.1-centos.game.web:

-centos.public_services.mysql

2,在/srv/pillar/top.sls配置minion对应的sls文件位置(ps:salt文件中不能再带"."，否则会报错)

base:

10.1.1.1-centos.game.web:

- custom.10-1-1-1-centos-game-web

1

2

3

base:

10.1.1.1-centos.game.web:

-custom.10-1-1-1-centos-game-web

3,新建/srv/pillar/custom/10-1-1-1-centos-game-web.sls,内容如下:

mysql:

ports:

- 3306

- 3307

- 3308

version:  '5_5_25'

1

2

3

4

5

6

mysql:

ports:

-3306

-3307

-3308

version: '5_5_25'

4,执行同步命令

salt 10.1.1.1-centos.game.web state.highstate -v -t 300

1

salt 10.1.1.1-centos.game.webstate.highstate-v-t300

二，改进方法

oh,shit,说好的自动化呢，怎么还要这么多步骤，这可不行！

利用py模式的sls配置文件(其实就是python脚本，只要返回yaml格式的字典文件就好了)，我们可以将以上的操作简化成1步，思路如下：

1，/srv/pillar/top.sls中编写配置：

base:

'*':

- custom

1

2

3

base:

'*':

-custom

2，使用py模式编写/srv/pillar/custom/init.sls，自动读取pillar配置，例如salt id是：10.1.1.1-centos.game.web，那么project为game，然后根据获取的pillar_root组合成路径/srv/pillar/custom/game/10.1.1.1-centos.game.web.yaml,利用yaml模块从文件中读取信息，返回字典

3，在/srv/salt/top.sls文件中匹配所有的minion

‘*’：

- centos.public_services

1

2

‘*’：

-centos.public_services

4，/srv/salt/centos/public_services/init.sls文件使用py模式编写，配置会获取对应的minion的pillar信息，如果包含mysql配置信息且配置正确的话，则返回mysql实例的配置。

那现在要怎么使用呢，很简单，例如你的id为10.1.1.1-centos.game.web，首先在/srv/pillar/custom/目录下建个game目录(从salt id获取的项目名)，然后在game目录先新建文件10.1.1.1-centos.game.web.yaml,里面写上配置信息：

mysql:

ports:

- 3306

- 3307

- 3308

version:  '5_5_25'

最后执行命令:

1

2

3

4

5

6

7

8

mysql:

ports:

-3306

-3307

-3308

version: '5_5_25'

最后执行命令:

salt 10.1.1.1-centos.game.web state.highstate -v -t 300

1

salt 10.1.1.1-centos.game.webstate.highstate-v-t300

静静的等待执行完成就好了！

三，具体代码

/srv/pillar/custom/init.sls

#!py

#coding:utf-8

"""

返回minion对应的pillar信息

"""

import yaml

import os

def run():

"""

首先获取请求的id，从id中获取project，例如id是：1.2.3.4-centos.game.web，那么project为game

然后根据获取的pillar_root组合成路径/srv/pillar/custom/game/1.2.3.4-centos.game.web.yaml,利用yaml模块从文件中读取信息，返回字典

"""

config={}

id=__opts__['id']

project=id.split('-')[-1].split('.')[1]

pillar_root=__opts__['pillar_roots']['base'][0]

path='%s/custom/%s/%s.yaml'%(pillar_root,project,id)

if os.path.isfile(path):

s=open(path).read()

config=yaml.load(s)

return config

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

#!py

#coding:utf-8

"""

返回minion对应的pillar信息

"""

importyaml

importos

defrun():

"""

首先获取请求的id，从id中获取project，例如id是：1.2.3.4-centos.game.web，那么project为game

然后根据获取的pillar_root组合成路径/srv/pillar/custom/game/1.2.3.4-centos.game.web.yaml,利用yaml模块从文件中读取信息，返回字典

"""

config={}

id=__opts__['id']

project=id.split('-')[-1].split('.')[1]

pillar_root=__opts__['pillar_roots']['base'][0]

path='%s/custom/%s/%s.yaml'%(pillar_root,project,id)

ifos.path.isfile(path):

s=open(path).read()

config=yaml.load(s)

returnconfig

/srv/salt/centos/public_services/init.sls

#!py

#coding: utf-8

import subprocess

class MY_ERROR(Exception):

def __init__(self,value):

self.value=value

def __str__(self):

return self.value

def mysql():

"""

1,检查是minion中pillar是否有mysql参数,以及参数是否合法

如果参数没有问题，则返回对应版本的include配置

pillar e.q.

mysql:

ports:

- 3306

- 3307

version: '5_5_25'

"""

mysql_sls_path='centos.public_services.mysql.'

#必要的参数

required_keys=['version','ports']

if __pillar__.has_key('mysql'):

mysql_d=__pillar__['mysql']

#不存在必要的键值对则返回None

for key in required_keys:

if not mysql_d.has_key(key) or str(mysql_d[key]).strip()=="":

raise MY_ERROR('key error! key: %s'%(str(key)))

#判断port是否合法

for port in mysql_d['ports']:

if not port or not 1024

raise MY_ERROR('mysql ports value error: %s'%(str(mysql_d['ports'])))

#组合配置参数

cfg=mysql_sls_path+str(mysql_d['version'][0])

return cfg

return None

def run():

config={}

config['include']=[]

#mysql

mysql_cfg=mysql()

if mysql_cfg:

config['include'].append(mysql_cfg)

if config['include']==[]:

return {}

return config

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

#!py

#coding: utf-8

importsubprocess

classMY_ERROR(Exception):

def__init__(self,value):

self.value=value

def__str__(self):

returnself.value

defmysql():

"""

1,检查是minion中pillar是否有mysql参数,以及参数是否合法

如果参数没有问题，则返回对应版本的include配置

pillar e.q.

mysql:

ports:

- 3306

- 3307

version: '5_5_25'

"""

mysql_sls_path='centos.public_services.mysql.'

#必要的参数

required_keys=['version','ports']

if__pillar__.has_key('mysql'):

mysql_d=__pillar__['mysql']

#不存在必要的键值对则返回None

forkeyinrequired_keys:

ifnotmysql_d.has_key(key)orstr(mysql_d[key]).strip()=="":

raiseMY_ERROR('key error! key: %s'%(str(key)))

#判断port是否合法

forportinmysql_d['ports']:

ifnotportornot1024

raiseMY_ERROR('mysql ports value error: %s'%(str(mysql_d['ports'])))

#组合配置参数

cfg=mysql_sls_path+str(mysql_d['version'][0])

returncfg

returnNone

defrun():

config={}

config['include']=[]

#mysql

mysql_cfg=mysql()

ifmysql_cfg:

config['include'].append(mysql_cfg)

ifconfig['include']==[]:

return{}

returnconfig

/srv/salt/centos/public_services/mysql/5/init.sls

include:

- centos.public_services.mysql.5.packet

- centos.public_services.mysql.5.my_cnf

- centos.public_services.mysql.5.instance

1

2

3

4

include:

-centos.public_services.mysql.5.packet

-centos.public_services.mysql.5.my_cnf

-centos.public_services.mysql.5.instance

/srv/salt/centos/public_services/mysql/5/packet.sls

mysql软件包要放到相应的目录中，具体位置参考下面配置

{%set version=pillar['mysql']['version']%}

mysql:

user.present:

- home: /home/mysql

- shell: /sbin/nologin

/usr/local/nagios/libexec/check_safe -u:

cmd.wait:

- watch:

- user: mysql

/usr/local/src/mysql-{{version.replace('_','.')}}.tar.gz:

file.managed:

- source: salt://centos/public_services/mysql/5/{{version}}/mysql-{{version.replace('_','.')}}.tar.gz

tar -xf mysql-{{version.replace('_','.')}}.tar.gz -C /usr/local/:

cmd.run:

- cwd: /usr/local/src

- unless: ls -l /usr/local/ | grep -e ".* mysql-{{version.replace('_','.')}}$"

/usr/local/mysql-{{version.replace('_','.')}}:

file.directory:

- user: mysql

- group: mysql

- recurse:

- user

- group

/data/mysql_log:

file.directory:

- makedirs: True

- user: mysql

- group: mysql

- recurse:

- user

- group

/data/log-bin:

file.directory:

- makedirs: True

- user: mysql

- group: mysql

- recurse:

- user

- group

/usr/bin/mysql:

file.symlink:

- target: /usr/local/mysql-{{version.replace('_','.')}}/bin/mysql

- unless: ls -l /usr/bin | grep -e " mysql$"

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

{%setversion=pillar['mysql']['version']%}

mysql:

user.present:

-home:/home/mysql

-shell:/sbin/nologin

/usr/local/nagios/libexec/check_safe-u:

cmd.wait:

-watch:

-user:mysql

/usr/local/src/mysql-{{version.replace('_','.')}}.tar.gz:

file.managed:

-source:salt://centos/public_services/mysql/5/{{version}}/mysql-{{version.replace('_','.')}}.tar.gz

tar-xfmysql-{{version.replace('_','.')}}.tar.gz-C/usr/local/:

cmd.run:

-cwd:/usr/local/src

-unless:ls-l/usr/local/|grep-e".* mysql-{{version.replace('_','.')}}$"

/usr/local/mysql-{{version.replace('_','.')}}:

file.directory:

-user:mysql

-group:mysql

-recurse:

-user

-group

/data/mysql_log:

file.directory:

-makedirs:True

-user:mysql

-group:mysql

-recurse:

-user

-group

/data/log-bin:

file.directory:

-makedirs:True

-user:mysql

-group:mysql

-recurse:

-user

-group

/usr/bin/mysql:

file.symlink:

-target:/usr/local/mysql-{{version.replace('_','.')}}/bin/mysql

-unless:ls-l/usr/bin|grep-e" mysql$"

/srv/salt/centos/public_services/mysql/5/my_cnf.sls

注意修改最后的初始化密码

my.cnf配置文件需要放到对应的目录中，my.cnf文件中要设置对应的模板变量:

port= {{port}} #还有其它的和端口相关的配置都要改成{{port}}

basedir=/usr/local/mysql-{{version}} #还有其它的和版本相关的配置都要改成{{version}}

1

2

port={{port}}#还有其它的和端口相关的配置都要改成{{port}}

basedir=/usr/local/mysql-{{version}}#还有其它的和版本相关的配置都要改成{{version}}

#!py

#coding:utf-8

"""

生成my.cnf配置文件，如果文件已存在，不作任何修改

"""

import os

def run():

config={}

version=__pillar__['mysql']['version']

for port in __pillar__['mysql']['ports']:

port=str(port)

if not os.path.isfile('/data/mysql_data_%s/my.cnf'%(port)):

config['/data/mysql_data_%s/my.cnf'%(port)]={

'file.managed':[

{'source':'salt://centos/public_services/mysql/5/%s/my.cnf'%(version)},

{'template':'jinja'},

{'context':{'port':port,'version':'%s'%(version.replace('_','.'))}},

{'require':[{'file':'/data/mysql_data_%s'%(port)}]},

],

}

config['chown mysql.mysql /data/mysql_data_%s/my.cnf'%(port)]='cmd.run'

return config

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

#!py

#coding:utf-8

"""

生成my.cnf配置文件，如果文件已存在，不作任何修改

"""

importos

defrun():

config={}

version=__pillar__['mysql']['version']

forportin__pillar__['mysql']['ports']:

port=str(port)

ifnotos.path.isfile('/data/mysql_data_%s/my.cnf'%(port)):

config['/data/mysql_data_%s/my.cnf'%(port)]={

'file.managed':[

{'source':'salt://centos/public_services/mysql/5/%s/my.cnf'%(version)},

{'template':'jinja'},

{'context':{'port':port,'version':'%s'%(version.replace('_','.'))}},

{'require':[{'file':'/data/mysql_data_%s'%(port)}]},

],

}

config['chown mysql.mysql /data/mysql_data_%s/my.cnf'%(port)]='cmd.run'

returnconfig

/srv/salt/centos/public_services/mysql/5/instance.sls

mysql启动脚本需要放到对应的目录中，启动脚本中要设置对应的模板变量:

basedir=/usr/local/mysql-{{version}} #还有其它的和版本相关的配置都要改成{{version}}

datadir=/data/mysql_data_{{port}} #还有其它的和端口相关的配置都要改成{{port}}

1

2

basedir=/usr/local/mysql-{{version}}#还有其它的和版本相关的配置都要改成{{version}}

datadir=/data/mysql_data_{{port}}#还有其它的和端口相关的配置都要改成{{port}}

{% for port in pillar['mysql']['ports'] %}

/data/mysql_data_{{port}}:

file.directory:

- makedirs: True

- user: mysql

- group: mysql

- recurse:

- user

- group

/etc/init.d/mysqld_{{port}}:

file.managed:

- source: salt://centos/public_services/mysql/5/{{pillar['mysql']['version']}}/mysql.service

- user: root

- group: root

- mode: 755

- template: jinja

- context:

port: {{port}}

version: {{pillar['mysql']['version'].replace('_','.')}}

chkconfig --add mysqld_{{port}};chkconfig --level 345 mysqld_{{port}} on;:

cmd.run:

- unless: chkconfig --list | grep mysqld_{{port}}

#初始化库

init_mysql_{{port}}:

cmd.run:

- name: /usr/local/mysql-{{pillar['mysql']['version'].replace('_','.')}}/scripts/mysql_install_db --user=mysql --basedir=/usr/local/mysql-{{pillar['mysql']['version'].replace('_','.')}} --datadir=/data/mysql_data_{{port}}/

- unless: ls -l /data/mysql_data_{{port}} | grep -e ".* mysql$"

/etc/init.d/mysqld_{{port}} start:

cmd.wait:

- watch:

- cmd: init_mysql_{{port}}

/usr/local/mysql-{{pillar['mysql']['version'].replace('_','.')}}/bin/mysqladmin -uroot -S /tmp/mysql_{{port}}.sock password 'yourmysqlpasswd!':

cmd.wait:

- watch:

- cmd: init_mysql_{{port}}

{%endfor%}

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

{%forportinpillar['mysql']['ports']%}

/data/mysql_data_{{port}}:

file.directory:

-makedirs:True

-user:mysql

-group:mysql

-recurse:

-user

-group

/etc/init.d/mysqld_{{port}}:

file.managed:

-source:salt://centos/public_services/mysql/5/{{pillar['mysql']['version']}}/mysql.service

-user:root

-group:root

-mode:755

-template:jinja

-context:

port:{{port}}

version:{{pillar['mysql']['version'].replace('_','.')}}

chkconfig--addmysqld_{{port}};chkconfig--level345mysqld_{{port}}on;:

cmd.run:

-unless:chkconfig--list|grepmysqld_{{port}}

#初始化库

init_mysql_{{port}}:

cmd.run:

-name:/usr/local/mysql-{{pillar['mysql']['version'].replace('_','.')}}/scripts/mysql_install_db--user=mysql--basedir=/usr/local/mysql-{{pillar['mysql']['version'].replace('_','.')}}--datadir=/data/mysql_data_{{port}}/

-unless:ls-l/data/mysql_data_{{port}}|grep-e".* mysql$"

/etc/init.d/mysqld_{{port}}start:

cmd.wait:

-watch:

-cmd:init_mysql_{{port}}

/usr/local/mysql-{{pillar['mysql']['version'].replace('_','.')}}/bin/mysqladmin-uroot-S/tmp/mysql_{{port}}.sockpassword'yourmysqlpasswd!':

cmd.wait:

-watch:

-cmd:init_mysql_{{port}}

{%endfor%}

四，总结

这个方法不仅可以用在mysql，同样的nginx，redis等都可以举一反三，配置文件编写完成后，仅仅需要简单的在pillar中添加几个变量就可以轻松批量部署了。