基因数据处理65之bwa处理500bp和1000bp的记录

最新推荐文章于 2021-08-29 15:26:24 发布

KeepLearningBigData

最新推荐文章于 2021-08-29 15:26:24 发布

阅读量842

点赞数

分类专栏：基因数据处理文章标签：基因数据处理 Adam Spark

本文链接：https://blog.csdn.net/xubo245/article/details/78848854

版权

基因数据处理专栏收录该内容

122 篇文章 9 订阅

订阅专栏

xubo@xubo:~/xubo/data/alignment/cs-bwamem$ bwa aln bwaindex/GRCH38chr1L3556522.fasta g38l500N10000.fq >g38l500N10000.bwa.sai
[bwa_aln] 17bp reads: max_diff = 2
[bwa_aln] 38bp reads: max_diff = 3
[bwa_aln] 64bp reads: max_diff = 4
[bwa_aln] 93bp reads: max_diff = 5
[bwa_aln] 124bp reads: max_diff = 6
[bwa_aln] 157bp reads: max_diff = 7
[bwa_aln] 190bp reads: max_diff = 8
[bwa_aln] 225bp reads: max_diff = 9
[bwa_aln_core] calculate SA coordinate... 22.19 sec
[bwa_aln_core] write to the disk... 0.00 sec
[bwa_aln_core] 10000 sequences have been processed.
[main] Version: 0.7.12-r1039
[main] CMD: /home/xubo/xubo/tools/bwa-0.7.12/bwa aln bwaindex/GRCH38chr1L3556522.fasta g38l500N10000.fq
[main] Real time: 22.506 sec; CPU: 22.427 sec
xubo@xubo:~/xubo/data/alignment/cs-bwamem$ bwa samse bwaindex/GRCH38chr1L3556522.fasta g38l500N10000.bwa
g38l500N10000.bwamem.sam  g38l500N10000.bwa.sai     
xubo@xubo:~/xubo/data/alignment/cs-bwamem$ bwa samse bwaindex/GRCH38chr1L3556522.fasta g38l500N10000.bwa.sai g38l500N10000.fq >g38l500N10000.bwa.sam
[bwa_aln_core] convert to sequence coordinate... 0.32 sec
[bwa_aln_core] refine gapped alignments... 0.20 sec
[bwa_aln_core] print alignments... 0.05 sec
[bwa_aln_core] 10000 sequences have been processed.
[main] Version: 0.7.12-r1039
[main] CMD: /home/xubo/xubo/tools/bwa-0.7.12/bwa samse bwaindex/GRCH38chr1L3556522.fasta g38l500N10000.bwa.sai g38l500N10000.fq
[main] Real time: 0.826 sec; CPU: 0.631 sec
xubo@xubo:~/xubo/data/alignment/cs-bwamem$ samtools flagstat g38l500N10000.
g38l500N10000.bwamem.sam  g38l500N10000.bwa.sai     g38l500N10000.bwa.sam     g38l500N10000.fq          g38l500N10000.log         g38l500N10000.snap1.sam   
xubo@xubo:~/xubo/data/alignment/cs-bwamem$ samtools flagstat g38l500N10000.bwa.sam
10000 + 0 in total (QC-passed reads + QC-failed reads)
0 + 0 secondary
0 + 0 supplementary
0 + 0 duplicates
8270 + 0 mapped (82.70% : N/A)
0 + 0 paired in sequencing
0 + 0 read1
0 + 0 read2
0 + 0 properly paired (N/A : N/A)
0 + 0 with itself and mate mapped
0 + 0 singletons (N/A : N/A)
0 + 0 with mate mapped to a different chr
0 + 0 with mate mapped to a different chr (mapQ>=5)
xubo@xubo:~/xubo/data/alignment/cs-bwamem$ bwa aln bwaindex/GRCH38chr1L3556522.fasta g38l500N100000.fq >g38l500N100000.bwa.sai
[bwa_aln] 17bp reads: max_diff = 2
[bwa_aln] 38bp reads: max_diff = 3
[bwa_aln] 64bp reads: max_diff = 4
[bwa_aln] 93bp reads: max_diff = 5
[bwa_aln] 124bp reads: max_diff = 6
[bwa_aln] 157bp reads: max_diff = 7
[bwa_aln] 190bp reads: max_diff = 8
[bwa_aln] 225bp reads: max_diff = 9
[bwa_aln_core] calculate SA coordinate... 216.90 sec
[bwa_aln_core] write to the disk... 0.01 sec
[bwa_aln_core] 100000 sequences have been processed.
[main] Version: 0.7.12-r1039
[main] CMD: /home/xubo/xubo/tools/bwa-0.7.12/bwa aln bwaindex/GRCH38chr1L3556522.fasta g38l500N100000.fq
[main] Real time: 217.920 sec; CPU: 217.581 sec
xubo@xubo:~/xubo/data/alignment/cs-bwamem$ bwa samse bwaindex/GRCH38chr1L3556522.fasta g38l500N100000.bwa.sai g38l500N100000.fq >g38l500N100000.bwa.sam
[bwa_aln_core] convert to sequence coordinate... 0.77 sec
[bwa_aln_core] refine gapped alignments... 1.58 sec
[bwa_aln_core] print alignments... 0.47 sec
[bwa_aln_core] 100000 sequences have been processed.
[main] Version: 0.7.12-r1039
[main] CMD: /home/xubo/xubo/tools/bwa-0.7.12/bwa samse bwaindex/GRCH38chr1L3556522.fasta g38l500N100000.bwa.sai g38l500N100000.fq
[main] Real time: 5.167 sec; CPU: 3.399 sec
xubo@xubo:~/xubo/data/alignment/cs-bwamem$ samtools flagstat g38l500N100000.bwa.sam
100000 + 0 in total (QC-passed reads + QC-failed reads)
0 + 0 secondary
0 + 0 supplementary
0 + 0 duplicates
82229 + 0 mapped (82.23% : N/A)
0 + 0 paired in sequencing
0 + 0 read1
0 + 0 read2
0 + 0 properly paired (N/A : N/A)
0 + 0 with itself and mate mapped
0 + 0 singletons (N/A : N/A)
0 + 0 with mate mapped to a different chr
0 + 0 with mate mapped to a different chr (mapQ>=5)
xubo@xubo:~/xubo/data/alignment/cs-bwamem$ bwa aln bwaindex/GRCH38chr1L3556522.fasta g38l1000N10000.fq >g38l1000N10000.bwa.sai
[bwa_aln] 17bp reads: max_diff = 2
[bwa_aln] 38bp reads: max_diff = 3
[bwa_aln] 64bp reads: max_diff = 4
[bwa_aln] 93bp reads: max_diff = 5
[bwa_aln] 124bp reads: max_diff = 6
[bwa_aln] 157bp reads: max_diff = 7
[bwa_aln] 190bp reads: max_diff = 8
[bwa_aln] 225bp reads: max_diff = 9
[bwa_aln_core] calculate SA coordinate... 2.55 sec
[bwa_aln_core] write to the disk... 0.00 sec
[bwa_aln_core] 10000 sequences have been processed.
[main] Version: 0.7.12-r1039
[main] CMD: /home/xubo/xubo/tools/bwa-0.7.12/bwa aln bwaindex/GRCH38chr1L3556522.fasta g38l1000N10000.fq
[main] Real time: 2.873 sec; CPU: 2.823 sec
xubo@xubo:~/xubo/data/alignment/cs-bwamem$ bwa samse bwaindex/GRCH38chr1L3556522.fasta g38l1000N10000.bwa.sai g38l1000N10000.fq >g38l1000N10000.bwa.sam
[bwa_aln_core] convert to sequence coordinate... 0.28 sec
[bwa_aln_core] refine gapped alignments... 0.08 sec
[bwa_aln_core] print alignments... 0.06 sec
[bwa_aln_core] 10000 sequences have been processed.
[main] Version: 0.7.12-r1039
[main] CMD: /home/xubo/xubo/tools/bwa-0.7.12/bwa samse bwaindex/GRCH38chr1L3556522.fasta g38l1000N10000.bwa.sai g38l1000N10000.fq
[main] Real time: 0.847 sec; CPU: 0.521 sec
xubo@xubo:~/xubo/data/alignment/cs-bwamem$ samtools flagstat g38l1000N10000.bwa.sam
10000 + 0 in total (QC-passed reads + QC-failed reads)
0 + 0 secondary
0 + 0 supplementary
0 + 0 duplicates
798 + 0 mapped (7.98% : N/A)
0 + 0 paired in sequencing
0 + 0 read1
0 + 0 read2
0 + 0 properly paired (N/A : N/A)
0 + 0 with itself and mate mapped
0 + 0 singletons (N/A : N/A)
0 + 0 with mate mapped to a different chr
0 + 0 with mate mapped to a different chr (mapQ>=5)
xubo@xubo:~/xubo/data/alignment/cs-bwamem$ bwa aln bwaindex/GRCH38chr1L3556522.fasta g38l1000N100000.fq >g38l1000N100000.bwa.sai
[bwa_aln] 17bp reads: max_diff = 2
[bwa_aln] 38bp reads: max_diff = 3
[bwa_aln] 64bp reads: max_diff = 4
[bwa_aln] 93bp reads: max_diff = 5
[bwa_aln] 124bp reads: max_diff = 6
[bwa_aln] 157bp reads: max_diff = 7
[bwa_aln] 190bp reads: max_diff = 8
[bwa_aln] 225bp reads: max_diff = 9
[bwa_aln_core] calculate SA coordinate... 27.39 sec
[bwa_aln_core] write to the disk... 0.01 sec
[bwa_aln_core] 100000 sequences have been processed.
[main] Version: 0.7.12-r1039
[main] CMD: /home/xubo/xubo/tools/bwa-0.7.12/bwa aln bwaindex/GRCH38chr1L3556522.fasta g38l1000N100000.fq
[main] Real time: 28.528 sec; CPU: 28.452 sec
xubo@xubo:~/xubo/data/alignment/cs-bwamem$ bwa samse bwaindex/GRCH38chr1L3556522.fasta g38l1000N100000.bwa.sai g38l1000N010000.fq >g38l1000N100000.bwa.sam
[bwa_seq_open] fail to open file 'g38l1000N010000.fq' : No such file or directory
xubo@xubo:~/xubo/data/alignment/cs-bwamem$ bwa samse bwaindex/GRCH38chr1L3556522.fasta g38l1000N100000.bwa.sai g38l1000N100000.fq >g38l1000N100000.bwa.sam
[bwa_aln_core] convert to sequence coordinate... 0.33 sec
[bwa_aln_core] refine gapped alignments... 0.46 sec
[bwa_aln_core] print alignments... 0.58 sec
[bwa_aln_core] 100000 sequences have been processed.
[main] Version: 0.7.12-r1039
[main] CMD: /home/xubo/xubo/tools/bwa-0.7.12/bwa samse bwaindex/GRCH38chr1L3556522.fasta g38l1000N100000.bwa.sai g38l1000N100000.fq
[main] Real time: 4.901 sec; CPU: 2.306 sec
xubo@xubo:~/xubo/data/alignment/cs-bwamem$ samtools flagstat g38l1000N100000.bwa.sam
100000 + 0 in total (QC-passed reads + QC-failed reads)
0 + 0 secondary
0 + 0 supplementary
0 + 0 duplicates
7798 + 0 mapped (7.80% : N/A)
0 + 0 paired in sequencing
0 + 0 read1
0 + 0 read2
0 + 0 properly paired (N/A : N/A)
0 + 0 with itself and mate mapped
0 + 0 singletons (N/A : N/A)
0 + 0 with mate mapped to a different chr
0 + 0 with mate mapped to a different chr (mapQ>=5)
xubo@xubo:~/xubo/data/alignment/cs-bwamem$

参考

【1】https://github.com/xubo245/AdamLearning
【2】https://github.com/bigdatagenomics/adam/ 
【3】https://github.com/xubo245/SparkLearning
【4】http://spark.apache.org
【5】http://stackoverflow.com/questions/28166667/how-to-pass-d-parameter-or-environment-variable-to-spark-job  
【6】http://stackoverflow.com/questions/28840438/how-to-override-sparks-log4j-properties-per-driver

研究成果：

【1】 [BIBM] Bo Xu, Changlong Li, Hang Zhuang, Jiali Wang, Qingfeng Wang, Chao Wang, and Xuehai Zhou, "Distributed Gene Clinical Decision Support System Based on Cloud Computing", in IEEE International Conference on Bioinformatics and Biomedicine. (BIBM 2017, CCF B)
【2】 [IEEE CLOUD] Bo Xu, Changlong Li, Hang Zhuang, Jiali Wang, Qingfeng Wang, Xuehai Zhou. Efficient Distributed Smith-Waterman Algorithm Based on Apache Spark (CLOUD 2017, CCF-C).
【3】 [CCGrid] Bo Xu, Changlong Li, Hang Zhuang, Jiali Wang, Qingfeng Wang, Jinhong Zhou, Xuehai Zhou. DSA: Scalable Distributed Sequence Alignment System Using SIMD Instructions. (CCGrid 2017, CCF-C).
【4】more: https://github.com/xubo245/Publications

Help

If you have any questions or suggestions, please write it in the issue of this project or send an e-mail to me: xubo245@mail.ustc.edu.cn
Wechat: xu601450868
QQ: 601450868