Keep Learning

Adam学习8之数据获取：例如：NA21144和NA12878ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000_genomes_project/data/GIH/NA21144ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000_genomes_projec

环境: adam-2.10.0.19 clean package test install1.Ubuntu下对adam进行mvn clean package -DskipTestsxubo@xubo:~/cloud/adam-2.10-0.19-git$ mvn clean package -DskipTests[INFO] Scanning for projects...[INFO]

编译时adam-cli会报错，git目录找不到，可以参考http://blog.csdn.net/xubo245/article/details/51177616解决1.mvn installD:\1win7\java\jdk\bin\java -Dmaven.multiModuleProjectDirectory=D:\all\idea\adam-adam-parent_2.10-0.19.0 -

Adam学习25之读取sam生成的alignmentRecord含recordGroupDictionary1.代码：package org.bdgenomics.adam.testimport java.nio.file.Filesimport org.apache.spark.{SparkConf, SparkContext}import org.bdgenomics.adam.rdd.ADA

对已经匹配或者没有匹配的reads进行排序源码： def adamSortReadsByReferencePosition(): RDD[AlignmentRecord] = SortReads.time { log.info("Sorting reads by reference position") // NOTE: In order to keep unmapped reads

sparkTest("test score") { val poorClippedReads = for (i <- 0 until 5) yield { createMappedRead("1", 44, 142, numClippedBases = 2, avgPhredScore = 30, readName = "poorClipped%d".format(i))

1.问题：1.1 描述当读入fastq文件后，需要进行转换，比如collect或者读取属性，会出现没有序列化的问题1.2 问题代码：package org.dsw.coreimport org.apache.spark.{SparkContext, SparkConf}import org.bdgenomics.adam.rdd.ADAMContext._/** * Created by xu

1.fragment 为10000：val fq1 = ac.loadFasta(file1, 10000)：hadoop@Master:~/cloud/testByXubo/spark/GRCH38$ ./cluster.sh fq0.count:45850077

Adam环境搭建（含window下eclipse配置）环境：集群：Ubuntu14.04 +Spark 1.5.2 +scala2.10本地：window7 64 +eclipse4.3.2+scala2.10.41.Adam安装：参考【1】$ git clone https://github.com/bigdatagenomics/adam.git$ cd adam

Adam学习3之ADAMContext类没有找到问题：hadoop@Mcnode1:~/cloud/adam/xubo/testAdam34/kmer$ ./submitJob.sh Exception in thread "main" java.lang.NoClassDefFoundError: org/bdgenomics/adam/rdd/ADAMContext

Adam学习之6源码解读kmer.scala代码：package testAdamimport org.apache.spark._import org.bdgenomics.adam.rdd.ADAMContextimport org.bdgenomics.adam.projections.{AlignmentRecordField, Projection}object kmer

adam-2.10-0.19在开始用idea编译有问题，主要是adam-core和adam-cli中的pom中引进的插件版本有问题：<plugin> <groupId>org.scoverage</groupId> <artifactId>maven-scoverage-plugin</artifactId> <version>

结果为：[INFO] Scanning for projects...[INFO] ------------------------------------------------------------------------[INFO] Reactor Build Order:[INFO] [INFO] ADAM_2.10[INFO] ADAM_2.10: Core[INFO] AD

之前老在eclipse下无法用maven编译成功，在直接使用maven时候（参考【1】）就可以在eclipse下用maven编译打包了：pom.xml->run as ->build->mvn clean package -DskipTests  记录：[INFO] Scanning for projects...[INFO] ------------------

问题：将单节点的内存从512M加到4096M时运行时间不减少反而增加，不解，待解决。是不是单个测试用例的偶然性？hadoop@Master:~/cloud/testByXubo/spark/hs38DH/package$ ./cluster.sh fq0.count:105887

ADAM学习之4集群安装（未成功）1.基本：$ git clone https://github.com/bigdatagenomics/adam.git$ cd adam$ export MAVEN_OPTS="-Xmx512m -XX:MaxPermSize=256m"2.安装maven3.配置pom.xml

Adam学习之7代码kmer.scala完善（统计和SaveAsFile）代码：package testAdamimport org.apache.spark._import org.bdgenomics.adam.rdd.ADAMContextimport org.bdgenomics.adam.projections.{AlignmentRecordField, Proj

Adam学习2之adam-shell使用环境：集群：Ubuntu14.04 +Spark 1.5.2 +scala2.10//本地：window7 64 +eclipse4.3.2+scala2.10.4代码：import org.bdgenomics.adam.rdd.ADAMContextimport org.bdgenomics.adam

0.代码（读取方法）：package org.bdgenomics.adamLocal.algorithms.testimport org.apache.spark.SparkConfimport org.apache.spark.SparkContextimport org.apache.spark.sql.SQLContextimport org.bdgenomics.adam.

更多Adam学习代码等资料请见：https://github.com/xubo245/AdamLearning1.Fasta格式在Adam里面的Avro模式是NucleotideContigFragment，具体在packageorg.bdgenomics.formats.avro包下，这个包在bdg-formats项目下，更多的可以看git：https://github.com/

1.参考2中提高了具体的，cp过来：{"readNum": null, "contig": null, "start": null, "oldPosition": null, "end": null, "mapq": null, "readName": "chrUn_KN707606v1_decoy_1204_1728_0:0:0_1:0:0_0", "sequence": "CTCCTCGC

1.SAM/BAM和Fastq一样的Avro，不详细就写了2，数据：{"readNum": 0, "contig": {"contigName": "chrUn_KN707606v1_decoy", "contigLength": 2200, "contigMD5": null, "referenceURL": null, "assembly": null, "species":

Adam学习11之Adam2.10-0.19搭建问题解决From：https://github.com/bigdatagenomics/adam/issues/1001install by mvn clean package -DskipTestserror：[INFO] [INFO] ADAM_2.10 ...............................

安装Adam——2.10-0.191.配置基本环境：jdk1.7scala-2.10.4maven 3.3.9查看版本：D:\all\eclipse432\adam-2.10-0.19-git-bin\adam-2.10-0.19-git>javac -versionjavac 1.7.0_25D:\all\eclipse432\adam-2

第三种方法和第二种方法可以顺利执行，统计出来GRCH38（GCA_000001405.15_GRCh38_full_analysis_set.fna）的信息：fq0.count:45850077 Method 3=> Length:321202 sum3:3.209457