基因数据处理101之SparkBWA本地运行配置和实例

最新推荐文章于 2022-11-18 21:48:49 发布
最新推荐文章于 2022-11-18 21:48:49 发布
阅读量1.2k 收藏
点赞数
分类专栏： 基因数据处理
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/xubo245/article/details/79029710
版权

1.修改Makefile.common：

将

LIBBWA_LIBS = -lrt
改为

LIBBWA_LIBS = -lrt -lz

不然会报错误【5】

2.make之后修改java.library.path

步骤：

vi /etc/profile

加入

export LD_LIBRARY_PATH=/home/hadoop/xubo/tools/SparkBWA/build:$LD_LIBRARY_PATH

使生效：

source /etc/profile

3.本地运行脚本：

spark-submit --class SparkBWA \
--master local \
SparkBWA.jar \
-algorithm mem -reads paired \
-index /home/hadoop/xubo/ref/GRCH38L1Index/GRCH38chr1L3556522.fasta \
-partitions 3 \
/xubo/alignment/sparkBWA/GRCH38chr1L3556522N10L50paired1.fastq /xubo/alignment/sparkBWA/GRCH38chr1L3556522N10L50paired2.fastq \
/xubo/alignment/output/sparkBWA/datatestLocalGRCH38chr1L3556522N10L50paired12

4.运行结果：

hadoop@Master:~/xubo/tools/SparkBWA/build$ ./pairedGRCH38L1Local.sh 
[Java_BwaJni_bwa_1jni] Arg 0 'bwa'
[Java_BwaJni_bwa_1jni] Algorithm found 1 'mem'
[Java_BwaJni_bwa_1jni] Arg 1 'mem'
[Java_BwaJni_bwa_1jni] Filename parameter -f found 2 '-f'
[Java_BwaJni_bwa_1jni] Arg 2 '-f'
[Java_BwaJni_bwa_1jni] Filename found 3 '/home/hadoop/cloud/workspace/tmpSparkBWA_GRCH38chr1L3556522N10L50paired1.fastq-3-NoSort-local-1466761250475-0.sam'
[Java_BwaJni_bwa_1jni] Arg 3 '/home/hadoop/cloud/workspace/tmpSparkBWA_GRCH38chr1L3556522N10L50paired1.fastq-3-NoSort-local-1466761250475-0.sam'
[Java_BwaJni_bwa_1jni] Arg 4 '/home/hadoop/xubo/ref/GRCH38L1Index/GRCH38chr1L3556522.fasta'
[Java_BwaJni_bwa_1jni] Arg 5 '/home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD0_1'
[Java_BwaJni_bwa_1jni] Arg 6 '/home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD0_2'
[Java_BwaJni_bwa_1jni] option[0]: bwa.
[Java_BwaJni_bwa_1jni] option[1]: mem.
[Java_BwaJni_bwa_1jni] option[2]: /home/hadoop/xubo/ref/GRCH38L1Index/GRCH38chr1L3556522.fasta.
[Java_BwaJni_bwa_1jni] option[3]: /home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD0_1.
[Java_BwaJni_bwa_1jni] option[4]: /home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD0_2.
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 6 sequences (300 bp)...
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (0, 2, 0, 0)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] skip orientation FR as there are not enough pairs
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::mem_process_seqs] Processed 6 reads in 0.002 CPU sec, 0.002 real sec
[main] Version: 0.7.12-r1044
[main] CMD: bwa mem /home/hadoop/xubo/ref/GRCH38L1Index/GRCH38chr1L3556522.fasta /home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD0_1 /home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD0_2
[main] Real time: 0.438 sec; CPU: 9.435 sec
[Java_BwaJni_bwa_1jni] Return code from BWA 0.
[Stage 3:>                                                          (0 + 1) / 3][Java_BwaJni_bwa_1jni] Arg 0 'bwa'
[Java_BwaJni_bwa_1jni] Algorithm found 1 'mem'
[Java_BwaJni_bwa_1jni] Arg 1 'mem'
[Java_BwaJni_bwa_1jni] Filename parameter -f found 2 '-f'
[Java_BwaJni_bwa_1jni] Arg 2 '-f'
[Java_BwaJni_bwa_1jni] Filename found 3 '/home/hadoop/cloud/workspace/tmpSparkBWA_GRCH38chr1L3556522N10L50paired1.fastq-3-NoSort-local-1466761250475-1.sam'
[Java_BwaJni_bwa_1jni] Arg 3 '/home/hadoop/cloud/workspace/tmpSparkBWA_GRCH38chr1L3556522N10L50paired1.fastq-3-NoSort-local-1466761250475-1.sam'
[Java_BwaJni_bwa_1jni] Arg 4 '/home/hadoop/xubo/ref/GRCH38L1Index/GRCH38chr1L3556522.fasta'
[Java_BwaJni_bwa_1jni] Arg 5 '/home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD1_1'
[Java_BwaJni_bwa_1jni] Arg 6 '/home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD1_2'
[Java_BwaJni_bwa_1jni] option[0]: bwa.
[Java_BwaJni_bwa_1jni] option[1]: mem.
[Java_BwaJni_bwa_1jni] option[2]: /home/hadoop/xubo/ref/GRCH38L1Index/GRCH38chr1L3556522.fasta.
[Java_BwaJni_bwa_1jni] option[3]: /home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD1_1.
[Java_BwaJni_bwa_1jni] option[4]: /home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD1_2.
[Stage 3:===================>                                       (1 + 1) / 3][M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 8 sequences (400 bp)...
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (0, 4, 0, 0)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] skip orientation FR as there are not enough pairs
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::mem_process_seqs] Processed 8 reads in 0.002 CPU sec, 0.001 real sec
[main] Version: 0.7.12-r1044
[main] CMD: bwa mem /home/hadoop/xubo/ref/GRCH38L1Index/GRCH38chr1L3556522.fasta /home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD1_1 /home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD1_2
[main] Real time: 0.440 sec; CPU: 10.097 sec
[Java_BwaJni_bwa_1jni] Return code from BWA 0.
[Java_BwaJni_bwa_1jni] Arg 0 'bwa'
[Java_BwaJni_bwa_1jni] Algorithm found 1 'mem'
[Java_BwaJni_bwa_1jni] Arg 1 'mem'
[Java_BwaJni_bwa_1jni] Filename parameter -f found 2 '-f'
[Java_BwaJni_bwa_1jni] Arg 2 '-f'
[Java_BwaJni_bwa_1jni] Filename found 3 '/home/hadoop/cloud/workspace/tmpSparkBWA_GRCH38chr1L3556522N10L50paired1.fastq-3-NoSort-local-1466761250475-2.sam'
[Java_BwaJni_bwa_1jni] Arg 3 '/home/hadoop/cloud/workspace/tmpSparkBWA_GRCH38chr1L3556522N10L50paired1.fastq-3-NoSort-local-1466761250475-2.sam'
[Java_BwaJni_bwa_1jni] Arg 4 '/home/hadoop/xubo/ref/GRCH38L1Index/GRCH38chr1L3556522.fasta'
[Java_BwaJni_bwa_1jni] Arg 5 '/home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD2_1'
[Java_BwaJni_bwa_1jni] Arg 6 '/home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD2_2'
[Java_BwaJni_bwa_1jni] option[0]: bwa.
[Java_BwaJni_bwa_1jni] option[1]: mem.
[Java_BwaJni_bwa_1jni] option[2]: /home/hadoop/xubo/ref/GRCH38L1Index/GRCH38chr1L3556522.fasta.
[Java_BwaJni_bwa_1jni] option[3]: /home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD2_1.
[Java_BwaJni_bwa_1jni] option[4]: /home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD2_2.
[Stage 3:=======================================>                   (2 + 1) / 3][M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 6 sequences (300 bp)...
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (0, 1, 0, 0)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] skip orientation FR as there are not enough pairs
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::mem_process_seqs] Processed 6 reads in 0.002 CPU sec, 0.002 real sec
[main] Version: 0.7.12-r1044
[main] CMD: bwa mem /home/hadoop/xubo/ref/GRCH38L1Index/GRCH38chr1L3556522.fasta /home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD2_1 /home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD2_2
[main] Real time: 0.429 sec; CPU: 10.584 sec
[Java_BwaJni_bwa_1jni] Return code from BWA 0.

5.数据源：

pair1：

@chr1_114727112_114727587_0:0:0_0:0:0_0/1
AGTACTTGAACTGTGCTAGATCATACACCAAATTATCCTGCATTGTTAAG
+
22222222222222222222222222222222222222222222222222
@chr1_59526946_59527392_0:0:0_1:0:0_1/1
ACTGAAGGTAGAGATGCAGGAATACAGTTACCTGTGCAACTATGACTCTA
+
22222222222222222222222222222222222222222222222222
@chr1_21862138_21862675_1:0:0_2:0:0_2/1
GCCCCAGCCATTAGGCCAAATTTACCAGAAGCCTTTCAGGGTTGCAATCC
+
22222222222222222222222222222222222222222222222222
@chr1_192732894_192733437_1:0:0_1:0:0_3/1
AATAGACAACACGAAGAACAGCTGTGAGCAATACAATTAGAAACTTTTTT
+
22222222222222222222222222222222222222222222222222
@chr1_246769496_246770014_0:0:0_2:0:0_4/1
TAATCTGTACGACAAACCCCCATGTCACTTTACCTCTATAACAAACCTGG
+
22222222222222222222222222222222222222222222222222
@chr1_89997487_89998004_0:0:0_2:0:0_5/1
CCTCAGCCTCCCTAGTAGCTGGGACTACAGGCACGCACCACCAGGCCCGG
+
22222222222222222222222222222222222222222222222222
@chr1_100741557_100742038_2:0:0_1:0:0_6/1
GTTCTCTTATATATTCTGAATAGACATTCTTTATGGAAAATACATTTAGC
+
22222222222222222222222222222222222222222222222222
@chr1_6197792_6198312_2:0:0_0:0:0_7/1
TTCAGTCAGTCGGAAAAACAAGATTAACATAACCAGAAACGTCCTAGGTA
+
22222222222222222222222222222222222222222222222222
@chr1_216218212_216218601_0:0:0_3:0:0_8/1
AATATAGTAAGATAACTTTAGTGCAACTTAAATTTCTTGGACCCAAAGTT
+
22222222222222222222222222222222222222222222222222
@chr1_2670698_2671257_2:0:0_0:0:0_9/1
ACCCACACGCCCATGTGAGCCTCTGACAGCCTGGAACAGCACGCGCAAGC
+
22222222222222222222222222222222222222222222222222

paird2：

@chr1_114727112_114727587_0:0:0_0:0:0_0/2
CAGAATGAAAACAATCTCAAGAACAAAAACCAATAAAAACAACTATAGTT
+
22222222222222222222222222222222222222222222222222
@chr1_59526946_59527392_0:0:0_1:0:0_1/2
CATTGTAAGCACTCAACAAGTGTTAGCTACTCCCAGTTGGAAGCTAGAAT
+
22222222222222222222222222222222222222222222222222
@chr1_21862138_21862675_1:0:0_2:0:0_2/2
GATGGATGTAGTTTTTAATTTATCCAATTGCCTATTCATGGATGTTTAGG
+
22222222222222222222222222222222222222222222222222
@chr1_192732894_192733437_1:0:0_1:0:0_3/2
AATAAAATATCATAGGACTAGGAGGCTTAAACAACATTTATTCCTCGCAG
+
22222222222222222222222222222222222222222222222222
@chr1_246769496_246770014_0:0:0_2:0:0_4/2
ACTTGGATTAGTGCCTGGCACATGGTGTAAGCACTTACTAAGTTTCAACG
+
22222222222222222222222222222222222222222222222222
@chr1_89997487_89998004_0:0:0_2:0:0_5/2
ACCCAATTATCTGCAAAACTGAGCATATTTAAAACAAATAATTACCAATA
+
22222222222222222222222222222222222222222222222222
@chr1_100741557_100742038_2:0:0_1:0:0_6/2
AAATGCTTGAACCCGGGAGGCAGAGGTTGCAGTGAGCCAAGATCATGCCA
+
22222222222222222222222222222222222222222222222222
@chr1_6197792_6198312_2:0:0_0:0:0_7/2
GGACAGGCACCTAGAATACTTTGCAACTCCCTTCATGAGAAGGTGGATGA
+
22222222222222222222222222222222222222222222222222
@chr1_216218212_216218601_0:0:0_3:0:0_8/2
TCATTTATCAGCCTTTTTAAGGGTGTTGAATGATTACCCAAGAGATCAGA
+
22222222222222222222222222222222222222222222222222
@chr1_2670698_2671257_2:0:0_0:0:0_9/2
TGTTACAGGCTGTCAGAGGCTCACCTGGGCATGTGGGTGCTGTTCCAGTC
+
22222222222222222222222222222222222222222222222222

6.部分数据结果：

@SQ SN:chr1 LN:248956422
@PG ID:bwa  PN:bwa  VN:0.7.12-r1044 CL:bwa mem /home/hadoop/xubo/ref/GRCH38L1Index/GRCH38chr1L3556522.fasta /home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD0_1 /home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD0_2
chr1_100741557_100742038_2:0:0_1:0:0_6  65  chr1    100741557   60  50M =   35207765    -65533793   GTTCTCTTATATATTCTGAATAGACATTCTTTATGGAAAATACATTTAGC  22222222222222222222222222222222222222222222222222  NM:i:2  MD:Z:24A13T11   AS:i:40 XS:i:0
chr1_100741557_100742038_2:0:0_1:0:0_6  129 chr1    35207765    0   50M =   100741557   65533793    AAATGCTTGAACCCGGGAGGCAGAGGTTGCAGTGAGCCAAGATCATGCCA  22222222222222222222222222222222222222222222222222  NM:i:1  MD:Z:2T47   AS:i:47 XS:i:47
chr1_114727112_114727587_0:0:0_0:0:0_0  81  chr1    114727538   60  50M =   114727112   -476    CTTAACAATGCAGGATAATTTGGTGTATGATCTAGCACAGTTCAAGTACT  22222222222222222222222222222222222222222222222222  NM:i:0  MD:Z:50 AS:i:50 XS:i:0
chr1_114727112_114727587_0:0:0_0:0:0_0  161 chr1    114727112   60  50M =   114727538   476 CAGAATGAAAACAATCTCAAGAACAAAAACCAATAAAAACAACTATAGTT  22222222222222222222222222222222222222222222222222  NM:i:0  MD:Z:50 AS:i:50 XS:i:0
chr1_192732894_192733437_1:0:0_1:0:0_3  81  chr1    192733388   60  50M =   192732894   -544    AAAAAAGTTTCTAATTGTATTGCTCACAGCTGTTCTTCGTGTTGTCTATT  22222222222222222222222222222222222222222222222222  NM:i:1  MD:Z:21T28  AS:i:45 XS:i:0
chr1_192732894_192733437_1:0:0_1:0:0_3  161 chr1    192732894   60  50M =   192733388   544 AATAAAATATCATAGGACTAGGAGGCTTAAACAACATTTATTCCTCGCAG  22222222222222222222222222222222222222222222222222  NM:i:1  MD:Z:22T27  AS:i:45 XS:i:0

第二部分：

@SQ SN:chr1 LN:248956422
@PG ID:bwa  PN:bwa  VN:0.7.12-r1044 CL:bwa mem /home/hadoop/xubo/ref/GRCH38L1Index/GRCH38chr1L3556522.fasta /home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD1_1 /home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD1_2
chr1_216218212_216218601_0:0:0_3:0:0_8  97  chr1    216218212   60  50M =   216218552   387 AATATAGTAAGATAACTTTAGTGCAACTTAAATTTCTTGGACCCAAAGTT  22222222222222222222222222222222222222222222222222  NM:i:0  MD:Z:50 AS:i:50 XS:i:0
chr1_216218212_216218601_0:0:0_3:0:0_8  145 chr1    216218552   60  47M3S   =   216218212   -387    TCTGATCTCTTGGGTAATCATTCAACACCCTTAAAAAGGCTGATAAATGA  22222222222222222222222222222222222222222222222222  NM:i:1  MD:Z:1G45   AS:i:45 XS:i:0
chr1_6197792_6198312_2:0:0_0:0:0_7  97  chr1    6197792 60  50M =   6198263 521 TTCAGTCAGTCGGAAAAACAAGATTAACATAACCAGAAACGTCCTAGGTA  22222222222222222222222222222222222222222222222222  NM:i:2  MD:Z:29G13A6    AS:i:40 XS:i:0
chr1_6197792_6198312_2:0:0_0:0:0_7  145 chr1    6198263 60  50M =   6197792 -521    TCATCCACCTTCTCATGAAGGGAGTTGCAAAGTATTCTAGGTGCCTGTCC  22222222222222222222222222222222222222222222222222  NM:i:0  MD:Z:50 AS:i:50 XS:i:0
chr1_59526946_59527392_0:0:0_1:0:0_1    81  chr1    59527343    60  50M =   59526946    -447    TAGAGTCATAGTTGCACAGGTAACTGTATTCCTGCATCTCTACCTTCAGT  22222222222222222222222222222222222222222222222222  NM:i:1  MD:Z:20A29  AS:i:45 XS:i:0
chr1_59526946_59527392_0:0:0_1:0:0_1    161 chr1    59526946    60  50M =   59527343    447 CATTGTAAGCACTCAACAAGTGTTAGCTACTCCCAGTTGGAAGCTAGAAT  22222222222222222222222222222222222222222222222222  NM:i:0  MD:Z:50 AS:i:50 XS:i:19
chr1_246769496_246770014_0:0:0_2:0:0_4  81  chr1    246769965   51  50M =   246769496   -519    CCAGGTTTGTTATAGAGGTAAAGTGACATGGGGGTTTGTCGTACAGATTA  22222222222222222222222222222222222222222222222222  NM:i:2  MD:Z:0G13T35    AS:i:44 XS:i:28
chr1_246769496_246770014_0:0:0_2:0:0_4  161 chr1    246769496   60  50M =   246769965   519 ACTTGGATTAGTGCCTGGCACATGGTGTAAGCACTTACTAAGTTTCAACG  22222222222222222222222222222222222222222222222222  NM:i:0  MD:Z:50 AS:i:50 XS:i:19

第三部分：

@SQ SN:chr1 LN:248956422
@PG ID:bwa  PN:bwa  VN:0.7.12-r1044 CL:bwa mem /home/hadoop/xubo/ref/GRCH38L1Index/GRCH38chr1L3556522.fasta /home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD2_1 /home/hadoop/cloud/workspace/tmplocal-1466761250475-RDD2_2
chr1_89997487_89998004_0:0:0_2:0:0_5    97  chr1    89997487    22  50M =   89997955    518 CCTCAGCCTCCCTAGTAGCTGGGACTACAGGCACGCACCACCAGGCCCGG  22222222222222222222222222222222222222222222222222  NM:i:0  MD:Z:50 AS:i:50 XS:i:43 XA:Z:chr1,-26514021,50M,2;
chr1_89997487_89998004_0:0:0_2:0:0_5    145 chr1    89997955    60  50M =   89997487    -518    TATTGGTAATTATTTGTTTTAAATATGCTCAGTTTTGCAGATAATTGGGT  22222222222222222222222222222222222222222222222222  NM:i:2  MD:Z:22C3T23    AS:i:40 XS:i:19
chr1_2670698_2671257_2:0:0_0:0:0_9  97  chr1    2679088 0   50M =   2666592 -12448  ACCCACACGCCCATGTGAGCCTCTGACAGCCTGGAACAGCACGCGCAAGC  22222222222222222222222222222222222222222222222222  NM:i:2  MD:Z:13G34C1    AS:i:43 XS:i:43
chr1_2670698_2671257_2:0:0_0:0:0_9  145 chr1    2666592 0   50M =   2679088 12448   GACTGGAACAGCACCCACATGCCCAGGTGAGCCTCTGACAGCCTGTAACA  22222222222222222222222222222222222222222222222222  NM:i:0  MD:Z:50 AS:i:50 XS:i:50
chr1_21862138_21862675_1:0:0_2:0:0_2    97  chr1    21862138    60  50M =   21862626    538 GCCCCAGCCATTAGGCCAAATTTACCAGAAGCCTTTCAGGGTTGCAATCC  22222222222222222222222222222222222222222222222222  NM:i:1  MD:Z:2A47   AS:i:47 XS:i:0
chr1_21862138_21862675_1:0:0_2:0:0_2    145 chr1    21862626    60  50M =   21862138    -538    CCTAAACATCCATGAATAGGCAATTGGATAAATTAAAAACTACATCCATC  22222222222222222222222222222222222222222222222222  NM:i:2  MD:Z:20G18G10   AS:i:40 XS:i:0

附录
错误：
错误【5】：

libbwa.so: undefined symbol: gzdopen

错误【6】

java.lang.UnsatisfiedLinkError: no bwa in java.library.path

参考

【1】https://github.com/xubo245/AdamLearning
【2】https://github.com/bigdatagenomics/adam/ 
【3】https://github.com/xubo245/SparkLearning
【4】http://spark.apache.org
【5】http://stackoverflow.com/questions/28166667/how-to-pass-d-parameter-or-environment-variable-to-spark-job  
【6】http://stackoverflow.com/questions/28840438/how-to-override-sparks-log4j-properties-per-driver

研究成果：

【1】 [BIBM] Bo Xu, Changlong Li, Hang Zhuang, Jiali Wang, Qingfeng Wang, Chao Wang, and Xuehai Zhou, "Distributed Gene Clinical Decision Support System Based on Cloud Computing", in IEEE International Conference on Bioinformatics and Biomedicine. (BIBM 2017, CCF B)
【2】 [IEEE CLOUD] Bo Xu, Changlong Li, Hang Zhuang, Jiali Wang, Qingfeng Wang, Xuehai Zhou. Efficient Distributed Smith-Waterman Algorithm Based on Apache Spark (CLOUD 2017, CCF-C).
【3】 [CCGrid] Bo Xu, Changlong Li, Hang Zhuang, Jiali Wang, Qingfeng Wang, Jinhong Zhou, Xuehai Zhou. DSA: Scalable Distributed Sequence Alignment System Using SIMD Instructions. (CCGrid 2017, CCF-C).
【4】more: https://github.com/xubo245/Publications

Help

If you have any questions or suggestions, please write it in the issue of this project or send an e-mail to me: xubo245@mail.ustc.edu.cn
Wechat: xu601450868
QQ: 601450868
KeepLearningBigData
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
samtools and bwa
03-11
samtools and bwa参数例子
Xtensa可配置处理器原理和两个成功应用实例.pdf
09-30
Xtensa可配置处理器原理和两个成功应用实例.pdf
华为 eNSP配置实例详解
10-31
通过与华为合作伙伴共同组织，现提供完整的华为 eNSP关于 路由器、交换机设备，配置单臂路由、链路聚合、三层交换机的实例，包含拓朴图、配置命令及详细设置，亲测可用。希望有需要的朋友下载，交流。
MySQL数据库入门之多实例配置方法详解
01-19
本文实例讲述了MySQL数据库入门之多实例配置方法。分享给大家供大家参考，具体如下： 前面介绍了相关的基础命令操作：MySQL数据库基础篇之入门基础命令 所有的操作都是基于单实例的，mysql多实例在实际生产环境也是非常实用的，因为必须要掌握。 1、什么是多实例 多实例就是一台服务器上开启多个不同的服务端口（默认3306），运行多个mysql的服务进程，这此服务进程通过不同的socket监听不同的服务端口来提供各在的服务，所有实例之间共同使用一套MYSQL的安装程序，但各自使用不同的配置文件、启动程序、数据文件，在逻辑上是相对独立的。 多实例主要作用是：充分利用现有的服务器硬件资源，为不同的服
xulrunner下载、安装、配置和实例
05-18
附件中包含 xulrunner-1.8.1.2pre.en-US.win32.zip xulrunner的安装和配置实例.docx（详细说明） tyapp.rar（实例的源码）
基因数据处理54之bwa-mem运行paird-end（1千万条100bp的reads）
Keep Learning
12-18 1928
指令：```hadoop@Master:~/cloud/adam/xubo/data/GRCH38Sub/cs-bwamem$ bwa mem GRCH38BWAindex/GRCH38chr1L3556522.fasta g38L100c10000000Nhs20Paired1.fq g38L100c10000000Nhs20Paired2.fq >g38L100c10000000Nhs20Pai
实验记录 | 6/30-7/4
weixin_40640700的博客
07-05 513
参考链接：https://gatk.broadinstitute.org/hc/en-us/articles/360056969692-Mutect2 参照官网得mutect2的示例代码，运行（随意）： ./gatk Mutect2 -R /home/xxzhang/workplace/QBRC/geneome/hg19/hg19.fa -I /home/xxzhang/workplace/output/tumor.bam -O mutect.vcf 出现错误： Error: Invalid or c
Spark问题14之Spark stage retry问题
Keep Learning
03-06 5305
更多代码请见：https://github.com/xubo245基因数据处理系列之SparkBWA1.解释1.1 简述当partitions超过节点数量的时候Lost executor的问题，已经提交到SparkBWA中，https://github.com/citiususc/SparkBWA/issues/35另外发现，tmp里面有临时文件没有删除，而且stage retry未解决2.记录完整
基因数据处理106之bwa-mem运行paird-end（1千万条100bp的reads g38L100c10000000Nhs20Paired12）
Keep Learning
01-11 1065
脚本：hadoop@Master:~/xubo/project/alignment/sparkBWA$ cat g38L100c10000000Nhs20Paired12Bwamem.sh echo "start" startTime4=`date +"%s.%N"` time4=`date +"%Y%m%d%H%M%S"` #spark-submit --cla
BWA比对及Samtools提取目标序列
weixin_56827666的博客
05-13 4587
今天想看一下自己的序列里面会不会有某细菌基因组存在，主要使用BWA和Samtools：bwa主要用于将低差异度的短序列与参考基因组进行比对。主要包含三种比对算法：backtrack、SW和MEM，第一种只支持短序列比对（<100bp），后两种支持长序列比对(70bp~1M)，并支持分割比对（split alignment）。MEM算法是最新的也是官方推荐的。BWA-MEM 是一种新的比对算法，用于将测序 reads 或者组装后 contigs 比对至大型参考基因组，例如人参考基因组。它会自动选择局部比
Springboot mybatis多数据源配置项目实例
11-19
Springboot mybatis 多数据源配置 项目实例·················
linux中bwa使用程序,序列比对BWA（Docker篇）
weixin_34281191的博客
05-10 1916
BWA，即Burrows-Wheeler-Alignment Tool。BWA 是一种能够将差异度较小的序列比对到一个较大的参考基因组上的软件包。首先通过BWT(Burrows-Wheeler Transformation，BWT压缩算法)为大型参考基因组建立索引，然后将reads比对到基因组。特点是快速、准确、省内存。它有三个不同的算法：BWA-backtrack:是用来比对 Illumina...
基因数据处理109之SparkBWA运行成功的部分log
Keep Learning
06-02 656
数据来自： http://219.219.220.149:18080/history/application_1466866953605_0001/executors/ http://219.219.220.248:8042/node/containerlogs/container_1466866953605_0001_01_000003/hadoop/stderr/?start=0 S...
基因数据处理102之SparkBWA本地运行100万条paired-reads实例
Keep Learning
01-11 1095
脚本：spark-submit --class SparkBWA \ --master local \ --archives bwa.zip \ SparkBWA.jar \ -algorithm mem -reads paired \ -index /home/hadoop/xubo/ref/GRCH38L1Index/GRCH38chr1L3556522.fasta \ -partitions
基因数据处理93之sparkBWA安装和使用
Keep Learning
01-04 1701
1.安装 git clone https://github.com/citiususc/SparkBWA.git cd SparkBWA make 2.使用： 报错：没有成功 hadoop@Master:~/xubo/project/alignment/sparkBWA$ ./run.sh Error: Must specify a primary resource (JAR or P
bwa和bowtie2使用方法
win_win223的博客
11-18 2844
bwa和bowtie比对
BWA的使用
hs6605015的博客
10-01 4780
一、简介 BWA，即Burrows-Wheeler-Alignment Tool。BWA 是一种能够将差异度较小的序列比对到一个较大的参考基因组上的软件包。它由三个不同的算法： BWA-backtrack: 是用来比对 Illumina 的序列的，reads 长度最长能到 100bp。 BWA-SW: 用于比对 long-read ，支持的长度为 70bp-1Mbp；同时支持剪接性比对。 BWA-MEM: 推荐使用的算法，支持较长的read长度，同时支持剪接性比对（split alignments)，但是B
BWA使用说明
热门推荐
小猪打呼噜
06-16 1万+
BWA使用说明
BWA用法笔记
wangprince2017
11-04 5986
BWA下载与安装 $ wget https://sourceforge.net/projects/bio-bwa/files/bwa-0.7.17.tar.bz2 $ tar xvfj bwa-0.7.17.tar.bz2 $ cd bwa-0.7.17 $ make $./bwa BWA简单比对 # 建立索引 # 建立索引可以通过子命令调用两种算法 # -is IS...
基于springboot的多数据源配置 代码实例下载
最新发布
11-20
基于Spring Boot的多数据源配置是一种常见的需求，它可以让我们在一个应用程序中使用多个数据库，从而更好地管理数据。如果你想要学习如何在Spring Boot应用中配置多数据源，可以通过以下步骤获取代码实例： 1. 首先，你可以在GitHub上搜索“spring-boot-multi-datasource”，会找到一些开源的多数据源配置的示例项目。 2. 选择一个你感兴趣的示例项目，并点击下载按钮将其下载到本地。 3. 解压下载的压缩包，进入项目目录，你会看到项目的结构和源代码文件。 4. 打开项目中的配置文件（一般是application.properties或者application.yml），这些文件中包含了多数据源配置的相关信息，比如数据库连接信息、驱动类等。 5. 导入项目到你喜欢的集成开发工具（比如IntelliJ IDEA或Eclipse），然后运行项目。 6. 在浏览器中输入http://localhost:8080（这是项目的默认端口），如果能够看到页面正常显示，就说明项目已经成功运行并且多数据源配置生效了。 通过以上步骤，你可以获取一个基于Spring Boot的多数据源配置的代码实例，并且学习如何在自己的项目中进行多数据源配置。祝你学习顺利！

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值