ceph-deploy源码分析（三）——mon模块 <转>

ceph-deploy源码分析（三）——mon模块

原文: http://www.hl10502.com/2017/06/19/ceph-deploy-mon/#more

ceph-deploy的mon.py模块是用来管理mon守护进程。

mon 子命令格式如下

ceph-deploy mon [-h] {add,create,create-initial,destroy} ...

• create: 创建mon
• add: 添加mon到集群，添加之前先在ceph.conf配置public_network
• create-initial: 创建mon并初始化
• destroy: 删除mon，如果集群只有一个mon则不能删除

在创建mon时，建议使用 ceph-deploy mon create-initial 来创建。

mon管理

make函数priority为30，子命令设置的默认函数为mon函数。

@priority(30)
def make(parser):
    """
    Ceph MON Daemon management
    """
    parser.formatter_class = ToggleRawTextHelpFormatter
    mon_parser = parser.add_subparsers(dest='subcommand')
    mon_parser.required = True
    mon_add = mon_parser.add_parser(
        'add',
        help=('R|Add a monitor to an existing cluster:\n'
              '\tceph-deploy mon add node1\n'
              'Or:\n'
              '\tceph-deploy mon add --address 192.168.1.10 node1\n'
              'If the section for the monitor exists and defines a `mon addr` that\n'
              'will be used, otherwise it will fallback by resolving the hostname to an\n'
              'IP. If `--address` is used it will override all other options.')
    )
    mon_add.add_argument(
        '--address',
        nargs='?',
    )
    mon_add.add_argument(
        'mon',
        nargs=1,
    )
    mon_create = mon_parser.add_parser(
        'create',
        help=('R|Deploy monitors by specifying them like:\n'
              '\tceph-deploy mon create node1 node2 node3\n'
              'If no hosts are passed it will default to use the\n'
              '`mon initial members` defined in the configuration.')
    )
    mon_create.add_argument(
        '--keyrings',
        nargs='?',
        help='concatenate multiple keyrings to be seeded on new monitors',
    )
    mon_create.add_argument(
        'mon',
        nargs='*',
    )
    mon_create_initial = mon_parser.add_parser(
        'create-initial',
        help=('Will deploy for monitors defined in `mon initial members`, '
              'wait until they form quorum and then gatherkeys, reporting '
              'the monitor status along the process. If monitors don\'t form '
              'quorum the command will eventually time out.')
    )
    mon_create_initial.add_argument(
        '--keyrings',
        nargs='?',
        help='concatenate multiple keyrings to be seeded on new monitors',
    )
    mon_destroy = mon_parser.add_parser(
        'destroy',
        help='Completely remove Ceph MON from remote host(s)'
    )
    mon_destroy.add_argument(
        'mon',
        nargs='+',
    )
    parser.set_defaults(
        func=mon,
    )

mon子命令

mon 函数，mon有四个subcmd为create、add、destroy、create-initial，分别对应mon_create、mon_add、mon_destroy、mon_create_initial四个函数

def mon(args):
    if args.subcommand == 'create':
        mon_create(args)
    elif args.subcommand == 'add':
        mon_add(args)
    elif args.subcommand == 'destroy':
        mon_destroy(args)
    elif args.subcommand == 'create-initial':
        mon_create_initial(args)
    else:
        LOG.error('subcommand %s not implemented', args.subcommand)

创建mon

命令行格式为： ceph-deploy mon create [node1] [node2] [node3] …

mon_create函数创建mon

• args参数校验
• 调用hosts.get函数获取操作系统版本信息，检查是否安装ceph包，目前支持的操作系统为centos/debian/fedora/rhel/suse，如果需要修改代码支持其他操作系统，可以从hosts入手修改，比如增加支持XenServer
• 调用相应操作系统模块下的create函数创建mon，比如操作系统为centos，即hosts/centos下的mon模块
• 调用mon_status函数检测mon状态
• 调用catch_mon_errors函数获取mon的错误信息，写入logger

• def mon_create(args):
      # 获取配置文件
      cfg = conf.ceph.load(args)
      if not args.mon:
          # 参数没指定mon，调用get_mon_initial_members函数从配置文件获取mon_initial_members作为mon
          args.mon = get_mon_initial_members(args, error_on_empty=True, _cfg=cfg)
      if args.keyrings:
          monitor_keyring = concatenate_keyrings(args)
      else:
          keyring_path = '{cluster}.mon.keyring'.format(cluster=args.cluster)
          try:
              # 获取ceph.mon.keyring文件信息
              monitor_keyring = files.read_file(keyring_path)
          except IOError:
              LOG.warning('keyring (%s) not found, creating a new one' % keyring_path)
              new_mon_keyring(args)
              monitor_keyring = files.read_file(keyring_path)
      LOG.debug(
          'Deploying mon, cluster %s hosts %s',
          args.cluster,
          ' '.join(args.mon),
          )
      errors = 0
      # 循环mon
      for (name, host) in mon_hosts(args.mon):
          try:
              # TODO add_bootstrap_peer_hint
              LOG.debug('detecting platform for host %s ...', name)
              # 获取操作系统版本信息，检查是否安装ceph包，如果需要修改代码支持其他操作系统，可以从hosts入手修改
              distro = hosts.get(
                  host,
                  username=args.username,
                  callbacks=[packages.ceph_is_installed]
              )
              LOG.info('distro info: %s %s %s', distro.name, distro.release, distro.codename)
              rlogger = logging.getLogger(name)
              # ensure remote hostname is good to go
              hostname_is_compatible(distro.conn, rlogger, name)
              rlogger.debug('deploying mon to %s', name)
              # 创建mon，调用hosts目录的相应操作系列目录，比如系统是centos，那就是hosts/centos下的mon模块
              distro.mon.create(distro, args, monitor_keyring)
              # tell me the status of the deployed mon
              time.sleep(2)  # give some room to start
              # 检测mon的状态
              mon_status(distro.conn, rlogger, name, args)
              # 获取mon的错误信息，写入logger
              catch_mon_errors(distro.conn, rlogger, name, cfg, args)
              distro.conn.exit()
          except RuntimeError as e:
              LOG.error(e)
              errors &#