TiDB DM部署及使用

1.在中控机上安装依赖包

yum -y install wget
yum -y install epel-release git curl sshpass && \
yum -y install python-pip

2.在中控机上创建 tidb 用户，并生成 SSH 密钥

useradd -m -d /home/tidb tidb
passwd tidb
visudo
tidb ALL=(ALL) NOPASSWD: ALL
su - tidb
ssh-keygen -t rsa

3.下载 及安装DM-Ansible 至中控机

wget https://download.pingcap.org/dm-ansible-v1.0.3.tar.gz
mv dm-ansible-v1.0.3 dm-ansible
cd /home/tidb/dm-ansible
sudo pip install -r ./requirements.txt

4.在中控机上配置 SSH 互信和 sudo 规则
（如果内部服务器有安全策略，禁止root登录也可以手动配置互信和sudo规则）

cd /home/tidb/dm-ansible
vi hosts.ini

替换为自己的IP

ansible-playbook -i hosts.ini create_users.yml -u root -k

5.下载 DM 及监控组件安装包至中控机

ansible-playbook local_prepare.yml

6.编辑 inventory.ini 配置文件(此处示例为dm-master和dm-worker在同一台服务器192.168.0.1 上游mysql IP为192.168.0.2 ，mysql_password为上游mysql数据库加密后的密码。使用dmctl进行密码加密，配置文件中的密码都要使用加密密码，例如密码123456

cd /home/tidb/dm-ansible/resources/bin
./dmctl -encrypt 123456

)
vi /home/tidb/dm-ansible/inventory.ini

## DM modules
[dm_master_servers]
dm_master ansible_host=192.168.0.1

[dm_worker_servers]
dm-worker1 ansible_host=192.168.0.1 source_id="mysql-replica-01" server_id=101 mysql_host=192.168.0.2 mysql_user=ovation mysql_password=as1sdkfgmgOpUAhrPY0vOsg  mysql_port=3306

[dm_portal_servers]
dm_portal ansible_host=192.168.0.1

## Monitoring modules
[prometheus_servers]
prometheus ansible_host=192.168.0.1

[grafana_servers]
grafana ansible_host=192.168.0.1

[alertmanager_servers]
alertmanager ansible_host=192.168.0.1

## Global variables
[all:vars]
cluster_name = test-cluster

ansible_user = tidb

dm_version = v1.0.3

deploy_dir = /data/dm/

grafana_admin_user = "admin"
grafana_admin_password = "admin"

7.部署及启动DM集群

ansible-playbook deploy.yml 
ansible-playbook start.yml

8.配置同步任务
（以单库全表同步为例,下游tidb为192.168.0.3）
在任意文件夹例如/home/tidb下创建任务配置文件
vi /home/tidb/task.yaml

---
name: ova # global unique
task-mode: all  # full/incremental/all

target-database:
  host: "192.168.0.3"
  port: 4000
  user: "test_user"
  password: "NiU2SLoENJosasd12bEQFcnOLQ4dgII="
mysql-instances:             # one or more source database, config more source database for sharding merge
  -
    source-id: "mysql-replica-01"
    black-white-list: "instance"

    # number of threads to use for dump data
    mydumper-thread: 4

    # number of threads to use for load data
    loader-thread: 16

    # number of threads to use for replicate binlog
    syncer-thread: 16
    syncer-config-name: "global"
black-white-list:
  instance:
    do-dbs: ["test"]        # white list of upstream schemas needs to be replicated, regular expression (starts with ~) is supported
    do-tables:                        # white list of upstream tables needs to be replicated, (db-name, tbl-name) regular expression (starts with ~) is supported
    - db-name: "test"
      tbl-name: "~.*"
syncers:
  global:
    worker-count: 64
    batch: 5000
    max-retry: 100
enable-heartbeat: true

9.使用dmctl启动同步任务

/home/tidb/dm-ansible/resources/bin//dmctl --master-addr 192.168.0.1:8261
start-task /home/tidb/task.yaml

查看详细任务状态，正常状态 result 为 true，worker 内的 binlog 位置一致，同步过程中也会展示同步百分比
query-status taskname(本例为test)