基因数据处理53之cs-bwamem集群版运行paird-end（1千万条100bp的reads）

最新推荐文章于 2021-07-05 09:31:18 发布
最新推荐文章于 2021-07-05 09:31:18 发布
阅读量776 收藏
点赞数
分类专栏： 基因数据处理 文章标签： 基因数据处理
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/xubo245/article/details/78829270
版权
  1. art：

    art_illumina -ss HS20 -i GRCH38BWAindex/GRCH38chr1L3556522.fna -p -l 100 -m 200 -s 10  -c 10000000 -o g38L100c10000000Nhs20Paired

2.fastq上传到hdfs


    spark-submit  --class cs.ucla.edu.bwaspark.BWAMEMSpark --master spark://219.219.220.149:7077  /home/hadoop/xubo/tools/cloud-scale-bwamem-0.2.1/target/cloud-scale-bwamem-0.2.0-assembly.jar upload-fastq   1 21 g38L100c10000000Nhs20Paired1.fq g38L100c10000000Nhs20Paired2.fq /xubo/data/alignment/cs-bwamem/fastq/g38L100c10000000Nhs20Paired12.fastq

3.配置：
需要将fasta dispatch到各个节点，每个节点需要有cs-bwamem的jar包，jni搞全局路径

4.align：


    spark-submit --executor-memory 4g --class cs.ucla.edu.bwaspark.BWAMEMSpark --total-executor-cores 20 --master spark://219.219.220.149:7077  --conf spark.driver.host=219.219.220.149 --conf spark.driver.cores=4 --conf spark.driver.maxResultSize=4g --conf spark.storage.memoryFraction=0.7  --conf spark.akka.threads=2 --conf spark.akka.frameSize=1024 /home/hadoop/xubo/tools/cloud-scale-bwamem-0.2.1/target/cloud-scale-bwamem-0.2.0-assembly.jar cs-bwamem -bfn 1 -bPSW 1 -sbatch 10 -bPSWJNI 1  -oChoice 2 -oPath hdfs://219.219.220.149:9000/xubo/data/alignment/cs-bwamem/fastq/g38L100c10000000Nhs20Paired12F2.adam -localRef 1 -jniPath /home/hadoop/xubo/tools/cloud-scale-bwamem-0.2.1/target/jniNative.so -isSWExtBatched 1  1 /home/hadoop/cloud/adam/xubo/data/GRCH38Sub/cs-bwamem/GRCH38BWAindex/GRCH38chr1L3556522.fasta  /xubo/data/alignment/cs-bwamem/fastq/g38L100c10000000Nhs20Paired12.fastq

5.merge：

spark-submit --executor-memory 6g --class cs.ucla.edu.bwaspark.BWAMEMSpark --total-executor-cores 20 --master spark://219.219.220.149:7077  --conf spark.driver.host=219.219.220.149 --conf spark.driver.cores=4 --conf spark.driver.maxResultSize=6g --conf spark.storage.memoryFraction=0.7  --conf spark.akka.threads=2 --conf spark.akka.frameSize=1024 /home/hadoop/xubo/tools/cloud-scale-bwamem-0.2.1/target/cloud-scale-bwamem-0.2.0-assembly.jar merge hdfs://219.219.220.149:9000 /xubo/data/alignment/cs-bwamem/fastq/g38L100c10000000Nhs20Paired12F2.adam /xubo/data/alignment/cs-bwamem/fastq/g38L100c10000000Nhs20Paired12F2.merge.adam

6.统计：


    package org.bdgenomics.avocado.cli

    import org.apache.spark.sql.SQLContext
    import org.apache.spark.{SparkConf, SparkContext}

    /**
      * Created by xubo on 2016/5/27.
      * 从hdfs下载经过avocado匹配好的数据
      * run:success
      */
    object parquetRead2csbwamemDup {
      def main(args: Array[String]) {
        val conf = new SparkConf().setMaster("local[4]").setAppName(this.getClass().getSimpleName().filter(!_.equals('$')))
        val sc = new SparkContext(conf)
        val sqlContext = new SQLContext(sc)
        println("start:")
        //    val file = "hdfs://219.219.220.149:9000/xubo/16.adam/0"
        //    val file = "hdfs://219.219.220.149:9000/xubo/data/alignment/cs-bwamem/fastq/g38L100c10000000Nhs20.merge.adam/"
        val file = "hdfs://219.219.220.149:9000/xubo/data/alignment/cs-bwamem/fastq/g38L100c10000000Nhs20Paired12F2.merge.adam/"


        val df3 = sqlContext.read.option("mergeSchema", "true").parquet(file)
        //    df3.printSchema()
        df3.show()
        println(df3.count())
        var j = 1
        //    for (i <- df3) {
        //      println(j + ":" + i)
        //      j += 1
        //    }
        //    println("distinct:" + df3.distinct.count())
        sc.stop
        println("end")
      }
    }

结果：


    scala>     println(df3.count())
    18512773
    scala> println("distinct:" + df3.distinct.count())
    distinct:18512773

附录：
部分记录：


    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.56094168004282, 9.939863106970265)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.56094168004282 9.939863106970265
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.3929171883384, 9.895888368589208)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.3929171883384 9.895888368589208
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.5008221332571, 9.893101752855983)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.5008221332571 9.893101752855983
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.50341742708892, 9.938821490084942)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.50341742708892 9.938821490084942
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.43662172257996, 9.956950171714343)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.43662172257996 9.956950171714343
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.5294684266857, 9.903683785238687)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.5294684266857 9.903683785238687
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.40190347187567, 9.937420104198324)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.40190347187567 9.937420104198324
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.45073240443014, 10.004935473704805)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.45073240443014 10.004935473704805
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.48925135028793, 9.872713569624056)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.48925135028793 9.872713569624056
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.40273561658512, 9.90951049687331)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.40273561658512 9.90951049687331
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.46782169411932, 9.994092930915187)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.46782169411932 9.994092930915187
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.4832529043789, 9.916498387414416)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.4832529043789 9.916498387414416
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.51942615912995, 9.980650734352077)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.51942615912995 9.980650734352077
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.42283366088975, 10.025277342223173)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.42283366088975 10.025277342223173
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.56863795443184, 9.884954572476323)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.56863795443184 9.884954572476323
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.54418222714128, 9.937606729429495)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.54418222714128 9.937606729429495
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.51789788933257, 9.908230182966792)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.51789788933257 9.908230182966792
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.49175824175825, 9.953633581247528)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.49175824175825 9.953633581247528
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.57901688501767, 9.95075215935561)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.57901688501767 9.95075215935561
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.47238496713348, 9.984373782742304)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.47238496713348 9.984373782742304
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.46635524016688, 9.968761617488683)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.46635524016688 9.968761617488683
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.38825547732228, 9.92027510738216)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.38825547732228 9.92027510738216
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.4851967207246, 9.937352774317741)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.4851967207246 9.937352774317741
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.55340706386016, 9.817553555623133)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.55340706386016 9.817553555623133
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.44619450694668, 9.920089825716484)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.44619450694668 9.920089825716484
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.5452329749104, 9.926594228354224)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.5452329749104 9.926594228354224
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.45703727621714, 9.948262102053539)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.45703727621714 9.948262102053539
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.4996774887121, 9.889445679734042)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.4996774887121 9.889445679734042
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.44845544695727, 10.04637179033811)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.44845544695727 10.04637179033811
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.54115079081086, 9.861927310088358)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.54115079081086 9.861927310088358
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.50392688847637, 9.923422645866905)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.50392688847637 9.923422645866905
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.47992413398225, 9.938359869833343)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.47992413398225 9.938359869833343
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.58786034019695, 9.881771439959131)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.58786034019695 9.881771439959131
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.53944973771544, 9.928751173123029)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.53944973771544 9.928751173123029
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.51932833154697, 9.921047819212657)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.51932833154697 9.921047819212657
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.5051575828961, 9.955724349633446)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.5051575828961 9.955724349633446
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.45987665323875, 9.934862525129335)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.45987665323875 9.934862525129335
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.41697601370646, 9.924344174968864)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.41697601370646 9.924344174968864
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.4504008016032, 10.037075657682921)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.4504008016032 10.037075657682921
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.53644939965696, 9.99170757810556)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.53644939965696 9.99170757810556
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.48251923421006, 9.977411795227688)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.48251923421006 9.977411795227688
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.50750643408637, 9.996386292381906)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.50750643408637 9.996386292381906
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.43163108941235, 9.93712833366825)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.43163108941235 9.93712833366825
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.4462753788962, 9.969547663168125)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.4462753788962 9.969547663168125
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.564805511698, 9.981653834837397)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.564805511698 9.981653834837397
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.36601983305985, 9.935551164661065)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.36601983305985 9.935551164661065
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.5214522629195, 9.948307577425197)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.5214522629195 9.948307577425197
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.4983700519434, 9.945836975580079)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.4983700519434 9.945836975580079
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.55142989753293, 9.949033183252514)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.55142989753293 9.949033183252514
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.57046907934694, 9.951195165671601)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.57046907934694 9.951195165671601
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.5627628483855, 9.940990579395123)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.5627628483855 9.940990579395123
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.57557661362418, 9.88907098626934)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.57557661362418 9.88907098626934
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.43170531280145, 9.97153529406552)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.43170531280145 9.97153529406552
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.50119476443524, 9.993324406109274)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.50119476443524 9.993324406109274
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.53636818766986, 9.949249012517377)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.53636818766986 9.949249012517377
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.56565548290277, 9.982952754471368)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.56565548290277 9.982952754471368
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.44116064703562, 9.922618947362455)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.44116064703562 9.922618947362455
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.48944921316166, 9.932084984684073)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.48944921316166 9.932084984684073
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.3957828750534, 9.897098539777856)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.3957828750534 9.897098539777856
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.46895465769603, 9.819260659759172)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.46895465769603 9.819260659759172
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.5120634807163, 9.905862404052492)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.5120634807163 9.905862404052492
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.50274353310056, 9.979749125168556)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.50274353310056 9.979749125168556
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.49695820211852, 9.940759507891329)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.49695820211852 9.940759507891329
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.42262032467997, 9.919691202862072)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.42262032467997 9.919691202862072
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.49617638650656, 9.973742669699334)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.49617638650656 9.973742669699334
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.57427937915742, 9.952728233473406)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.57427937915742 9.952728233473406
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 198, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.48755438271164, 9.944608296251676)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.48755438271164 9.944608296251676
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.52546792398914, 10.000808728139058)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.52546792398914 10.000808728139058
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (192, 199, 205)
    low and high boundaries for computing mean and std.dev: (166, 231)
    mean and std.dev: (198.49446902654867, 9.999977060170599)
    low and high boundaries for proper pairs: (153, 244)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 153 244 0 198.49446902654867 9.999977060170599
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Load FASTQ files                                                                
    @Worker1
    skip orientation FF as there are not enough pairs                               
    analyzing insert size distribution for orientation
    (25, 50, 75) percentile: (191, 198, 205)
    low and high boundaries for computing mean and std.dev: (163, 233)
    mean and std.dev: (198.28641975308642, 10.159216233717126)
    low and high boundaries for proper pairs: (149, 247)
    skip orientation RF as there are not enough pairs
    skip orientation RR as there are not enough pairs
    @MemPeStat
    pes(0): 0 0 1 0.0 0.0
    pes(1): 149 247 0 198.28641975308642 10.159216233717126
    pes(2): 0 0 1 0.0 0.0
    pes(3): 0 0 1 0.0 0.0
    Summary:
    Worker1 Time: 815792
    Calculate Metrics Time: 2093
    Worker2 Time: 688463
    16/06/03 18:07:19 ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerExecutorMetricsUpdate(4,WrappedArray())
    16/06/03 18:07:19 WARN QueuedThreadPool: 1 threads could not be stopped
    CS-BWAMEM Finished!!!
    Jun 3, 2016 5:41:47 PM INFO: parquet.hadoop.ParquetInputFormat: Total input paths to process : 21
    Jun 3, 2016 5:41:57 PM INFO: parquet.hadoop.ParquetInputFormat: Total input paths to process : 21
    hadoop@Master:~/cloud/adam/xubo/data/GRCH38Sub/cs-bwamem$

参考

【1】https://github.com/xubo245/AdamLearning
【2】https://github.com/bigdatagenomics/adam/ 
【3】https://github.com/xubo245/SparkLearning
【4】http://spark.apache.org

研究成果：

【1】 [BIBM] Bo Xu, Changlong Li, Hang Zhuang, Jiali Wang, Qingfeng Wang, Chao Wang, and Xuehai Zhou, "Distributed Gene Clinical Decision Support System Based on Cloud Computing", in IEEE International Conference on Bioinformatics and Biomedicine. (BIBM 2017, CCF B)
【2】 [IEEE CLOUD] Bo Xu, Changlong Li, Hang Zhuang, Jiali Wang, Qingfeng Wang, Xuehai Zhou. Efficient Distributed Smith-Waterman Algorithm Based on Apache Spark (CLOUD 2017, CCF-C).
【3】 [CCGrid] Bo Xu, Changlong Li, Hang Zhuang, Jiali Wang, Qingfeng Wang, Jinhong Zhou, Xuehai Zhou. DSA: Scalable Distributed Sequence Alignment System Using SIMD Instructions. (CCGrid 2017, CCF-C).
【4】more: https://github.com/xubo245/Publications

Help

If you have any questions or suggestions, please write it in the issue of this project or send an e-mail to me: xubo245@mail.ustc.edu.cn
Wechat: xu601450868
QQ: 601450868
KeepLearningBigData
关注
专栏目录
NGS数据格式梳理02-SAM/BAM格式最详细解读
qq_21478261的博客
05-08 4967
全网最全介绍SAM|BAM文件文章。
whole-genome-sequencing Data Analysis 学习笔记1 基本概念
cs_mary的博客
02-01 3029
Follow Jimmy:Gene Analysis 探索全基因组测序数据 1.参考文章 Systematic Characteristic Exploration of the Chimeras Generated in Multiple Displacement Amplification through Next Generation Sequencing Data Reanalysis
基因数据处理107之cs-bwamem运行paird-end（1千万条100bp的reads g38L100c10000000Nhs20Paired1T20160625）
Keep Learning
06-02 663
1.upload: hadoop@Master:~/xubo/project/alignment/cs-bwamem$ cat align.sh #spark-submit --class cs.ucla.edu.bwaspark.BWAMEMSpark --master spark://219.219.220.149:7077 /home/hadoop/xubo/tools/cloud-...
基因数据处理116之重新运行SparkBWA Yarn集群模式
Keep Learning
06-03 564
更多代码请见：https://github.com/xubo245 基因数据处理系列 1.解释 很久没运行SparkBWA了，系统文件有点多，重新运行。 2.代码： end hadoop@Master:~/disk2/xubo/project/alignment/sparkBWA$ vi g38L100c100000Nhs20Paired12SparkBWAYarnPartit...
基因数据处理54之bwa-mem运行paird-end（1千万条100bp的reads）
Keep Learning
12-18 2016
指令：```hadoop@Master:~/cloud/adam/xubo/data/GRCH38Sub/cs-bwamem$ bwa mem GRCH38BWAindex/GRCH38chr1L3556522.fasta g38L100c10000000Nhs20Paired1.fq g38L100c10000000Nhs20Paired2.fq >g38L100c10000000Nhs20Pai
实验记录 | 6/30-7/4
weixin_40640700的博客
07-05 559
参考链接：https://gatk.broadinstitute.org/hc/en-us/articles/360056969692-Mutect2 参照官网得mutect2的示例代码，运行（随意）： ./gatk Mutect2 -R /home/xxzhang/workplace/QBRC/geneome/hg19/hg19.fa -I /home/xxzhang/workplace/output/tumor.bam -O mutect.vcf 出现错误： Error: Invalid or c
RNA-seq Experimental Design and Quality Control
Al_Liz_Well
06-16 798
RNA-seq Experimental design and Quanlity Control
whole-genome-sequencing Data Analysis 学习笔记5 全基因组测序数据的变异位点
热门推荐
cs_mary的博客
03-02 1万+
变异为什么是相对的？有的变异现象是由于生殖细胞内的遗传物质的改变引起的，因而能够遗传给后代，属于可遗传的变异。 可遗传的变异有三种来源：基因突变，基因重组，染色体变异。由于DNA分子中发生碱基对增添、缺失或改变，而引起的基因结构的改变，就叫做基因突变，特点是普遍，随机，频率低，多有害，不定向基因重组是指在生物体进行有性生殖的过程中，控制不同性状的基因的重新组合。基因的自由组合定律告诉我们，在生物体
Linux设备驱动编程第三版-笔记
u011961033的专栏
10-16 1408
第1章 设备驱动简介 1.1 驱动程序的角色 机制:提供什么能力. 策略:如何使用这些能力. 1.2. 划分内核     内核的角色可以划分:     一:进程管理       二:内存管理     三:文件系统     四:设备控制     五:网络     1.2.1. 可加载模块 1.3. 设备和模块的分类 字符设备: 块设备: 网络接口   第 2 章...
whole-genome-sequencing Data Analysis 学习笔记4 变异位点注释数据库
cs_mary的博客
02-28 3214
linux下如何使用sftp命令 sftp> get /var/www/fuyatao/index.php /home/fuyatao/ 这条语句将从远程主机的 /var/www/fuyatao/目录下将 index.php 下载到本地 /home/fuyatao/目录下。sftp> put /home/fuyatao/downloads/Linuxgl.pdf /var/www/fuya
基因数据处理100之bwamem算法处理100万条paired-reads数据GRCH38chr1L3556522N1000000L100paired12
Keep Learning
01-11 1794
运行记录：hadoop@Master:~/xubo/ref/GRCH38L1Index/pe$ bwa mem ../GRCH38chr1L3556522.fasta GRCH38chr1L3556522N1000000L100paired1.fastq GRCH38chr1L3556522N1000000L100paired2.fastq >GRCH38chr1L3556522N1000000L1
基因数据处理101之SparkBWA本地运行配置和实例
Keep Learning
01-11 1270
1.修改Makefile.common：将LIBBWA_LIBS = -lrt 改为LIBBWA_LIBS = -lrt -lz不然会报错误【5】2.make之后修改java.library.path步骤：vi /etc/profile 加入export LD_LIBRARY_PATH=/home/hadoop/xubo/tools/SparkBWA/build:$LD_LIBRARY_PATH
基因数据处理102之SparkBWA本地运行100万条paired-reads实例
Keep Learning
01-11 1124
脚本：spark-submit --class SparkBWA \ --master local \ --archives bwa.zip \ SparkBWA.jar \ -algorithm mem -reads paired \ -index /home/hadoop/xubo/ref/GRCH38L1Index/GRCH38chr1L3556522.fasta \ -partitions
基因数据处理115之重新运行SparkBWA本地模式
Keep Learning
06-03 1029
更多代码请见：https://github.com/xubo245 基因数据处理系列 1.记录 hadoop@Master:~/disk2/xubo/tools/SparkBWA/build$ ./pairedGRCH38L1Local.sh [Stage 3:&gt; (...
scheduler.LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerExecuto...
weixin_34130269的博客
06-22 2474
引起这个问题的原因是因为spark没有关闭 转载于:https://www.cnblogs.com/xiaoma0529/p/7066543.html
Spark详解（十）：SparkShuffle机制原理分析
Master-TJ的个人博客
02-24 1126
1. Spark Shuffle简介 在Hadoop的MapReduce框架中Shuffle是连接Map和Reduce的桥梁，Map的输出要用到Reduce中必须经过Shuffle这个环节。由于Shuffle阶段涉及到磁盘的读写和网络传输，因此Shuffle的性能高低直接影响到整个程序的性能和吞吐量。 Spakr Shuffle定义：Shuffle在中文的意思是“洗牌、混洗”的意思，在MapRe...
测序数据质量控制
weixin_30251829的博客
12-31 5562
　　基于边合成边测序（Sequencing By Synthesis，SBS）技术，Illumina HiSeq2500高通量测序平台对cDNA文库进行测序，能够产出大量的高质量Reads，测序平台产出的这些Reads或碱基称为原始数据（Raw Data），其大部分碱基质量打分能达到或超过Q30。Raw Data通常以FASTQ格式提供，每个测序样品的Raw Data包括两个FASTQ文件，分别包...
将长序列打断成100bp-300bp的reads方法或者软件有哪些
最新发布
04-05
有以下几种方法或软件可以将长序列打断成100bp-300bp的reads： 1. Trimmomatic：Trimmomatic是一款广泛使用的序列处理软件，其中包括将长序列打断为短reads的功能。使用命令如下： ``` java -jar trimmomatic.jar ...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值