中奖纪录侧
数据量:千万级
实施方案:数据库层面迁移,新数据一一匹配
实施脚本:
1、sql整理出需要导出的数据:
select a,b,c,d,e,f,g
from table_1
into outfile '/data/aaa/haha.csv'
fields terminated by ',' optionally enclosed by '\''
lines terminated by '\r\n';
2、使用shell分割记录结果,分批导入
split -a 3 -l 20000 haha.csv target/table2_
for i in `ls -1 target `
do
mysql -s -N -p'xxxxxxx' DPEmidas -e "load data infile '/data/aaa/target/$i'
IGNORE into table table2
fields terminated by ',' optionally enclosed by '\''
lines terminated by '\r\n'"
mv /data/aaa/target/$i /data/aaa/complete/
sleep 1
done
3、
获取 所有项目的git地址:
git/config文件如下:
[core]
repositoryformatversion = 0
filemode = false
bare = false
logallrefupdates = true
symlinks = false
ignorecase = true
[remote "origin"]
url = ssh://git@git.dianpingoa.com/emidas/emidas-decorate.git
fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
remote = origin
merge = refs/heads/master
[branch "feature/search"]
remote = origin
merge = refs/heads/feature/search
[branch "beta"]
remote = origin
merge = refs/heads/beta
需要获取每一个url 那一行后面的地址,正则需要匹配 空格符 url = xxxx
具体正则 (^\s*)url = (.*),sed命令中需要转义 \ $ * + ( )
sed -n 's/\(^\s*\)url = \(.*\)/\2/p' */.git/config > test.txt
https://blog.csdn.net/z69183787/article/details/53693739