基因数据处理41之mango使用失败

原创 2016年05月30日 13:30:58
hadoop@Master:~/xubo/tools/adam-2.10-0.19-git/bin$ mango-submit /xubo/mango/snap/datatest.fa.adam -read_files /xubo/mango/snap/datatest2.sort.bam.adam  -var_files /xubo/mango/snap/datatest2.flt.vcf.adam -feat_file /xubo/mango/snap/datatest.sorted.bed -port 9992
Using SPARK_SUBMIT=/home/hadoop/cloud/spark-1.5.2//bin/spark-submit
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Command body threw exception:
java.io.FileNotFoundException: File does not exist: /xubo/mango/snap/datatest2.sort.bam.adam/_seqdict.avro
    at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
    at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1891)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1832)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1812)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1784)
    at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:542)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:362)
    at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033)

Exception in thread "main" java.io.FileNotFoundException: File does not exist: /xubo/mango/snap/datatest2.sort.bam.adam/_seqdict.avro
    at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
    at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1891)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1832)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1812)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1784)
    at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:542)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:362)
    at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033)

    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
    at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
    at org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:1222)
    at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1210)
    at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1200)
    at org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:271)
    at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:238)
    at org.apache.hadoop.hdfs.DFSInputStream.<init>(DFSInputStream.java:231)
    at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1498)
    at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:302)
    at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:298)
    at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
    at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:298)
    at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:766)
    at org.bdgenomics.adam.rdd.ADAMContext.loadAvro(ADAMContext.scala:399)
    at org.bdgenomics.adam.rdd.ADAMContext.loadParquetAlignments(ADAMContext.scala:472)
    at org.bdgenomics.mango.models.AlignmentRecordMaterialization.getFileReference(AlignmentRecordMaterialization.scala:104)
    at org.bdgenomics.mango.models.AlignmentRecordMaterialization.loadADAMSample(AlignmentRecordMaterialization.scala:97)
    at org.bdgenomics.mango.cli.VizReads.initAlignments$1(VizReads.scala:442)
    at org.bdgenomics.mango.cli.VizReads.run(VizReads.scala:406)
    at org.bdgenomics.utils.cli.BDGSparkCommand$class.run(BDGCommand.scala:54)
    at org.bdgenomics.mango.cli.VizReads.run(VizReads.scala:392)
    at org.bdgenomics.utils.cli.BDGCommandCompanion$class.main(BDGCommand.scala:32)
    at org.bdgenomics.mango.cli.VizReads$.main(VizReads.scala:65)
    at org.bdgenomics.mango.cli.VizReads.main(VizReads.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:674)
    at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
    at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): File does not exist: /xubo/mango/snap/datatest2.sort.bam.adam/_seqdict.avro
    at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
    at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1891)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1832)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1812)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1784)
    at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:542)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:362)
    at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033)

    at org.apache.hadoop.ipc.Client.call(Client.java:1468)
    at org.apache.hadoop.ipc.Client.call(Client.java:1399)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
    at com.sun.proxy.$Proxy16.getBlockLocations(Unknown Source)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getBlockLocations(ClientNamenodeProtocolTranslatorPB.java:254)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
    at com.sun.proxy.$Proxy17.getBlockLocations(Unknown Source)
    at org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:1220)
    ... 31 more
May 30, 2016 6:47:19 PM INFO: org.apache.parquet.hadoop.ParquetInputFormat: Total input paths to process : 1
May 30, 2016 6:47:21 PM WARNING: org.apache.parquet.hadoop.ParquetRecordReader: Can not initialize counter due to context is not a instance of TaskInputOutputContext, but is org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl
May 30, 2016 6:47:22 PM INFO: org.apache.parquet.hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1 records.
May 30, 2016 6:47:22 PM INFO: org.apache.parquet.hadoop.InternalParquetRecordReader: at row 0. reading next block
May 30, 2016 6:47:22 PM INFO: org.apache.parquet.hadoop.InternalParquetRecordReader: block read in memory in 39 ms. row count = 1
May 30, 2016 6:47:23 PM INFO: org.apache.parquet.hadoop.ParquetInputFormat: Total input paths to process : 1
May 30, 2016 6:47:23 PM WARNING: org.apache.parquet.hadoop.ParquetRecordReader: Can not initialize counter due to context is not a instance of TaskInputOutputContext, but is org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl
May 30, 2016 6:47:23 PM INFO: org.apache.parquet.hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1 records.
May 30, 2016 6:47:23 PM INFO: org.apache.parquet.hadoop.InternalParquetRecordReader: at row 0. reading next block
May 30, 2016 6:47:23 PM INFO: org.apache.parquet.hadoop.InternalParquetRecordReader: block read in memory in 4 ms. row count = 1
May 30, 2016 6:47:23 PM INFO: org.apache.parquet.hadoop.ParquetInputFormat: Total input paths to process : 1
May 30, 2016 6:47:23 PM WARNING: org.apache.parquet.hadoop.ParquetRecordReader: Can not initialize counter due to context is not a instance of TaskInputOutputContext, but is org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl
May 30, 2016 6:47:23 PM INFO: org.apache.parquet.hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1 records.
May 30, 2016 6:47:23 PM INFO: org.apache.parquet.hadoop.InternalParquetRecordReader: at row 0. reading next block
May 30, 2016 6:47:23 PM INFO: org.apache.parquet.hadoop.InternalParquetRecordReader: block read in memory in 3 ms. row count = 1
版权声明:本文为博主原创文章,欢迎转载和交流!源码交流:https://github.com/xubo245/

相关文章推荐

hive和impala查询数据对比

首先impala查询数据,更像rdbms一样(mysql)。 --1.impala连接  [root@MASTER01 ~]# impala-shell -islave03; Starting I...

第62课:SparkSQL下的Parquet使用最佳实践和代码实践学习笔记

第62课:SparkSQL下的Parquet使用最佳实践和代码实践学习笔记 本期内容: 1 SparkSQL下的Parquet使用最佳实践 2 SparkSQL下的Parquet实战   一...
  • slq1023
  • slq1023
  • 2016年04月02日 20:21
  • 2137

基因数据处理39之mango安装记录

更多代码请见:https://github.com/xubo245/AdamLearning1解释 mango安装记录 mango主要是在adam上实现 可视化平台 A scalable gen...

基因数据处理26之bcftools安装和使用

1.下载:https://github.com/samtools/bcftools2.安装 make make install3.结合samtools使用 对排序好的bam数据用samtools...

基因数据处理48之ART使用实例

相关参数请见上一篇1.使用实例1: hadoop@Master:~/cloud/adam/xubo/data/GRCH38Sub/cs-bwamem$ art_illumina -ss HS20 -i...

基因数据处理17之使用scala对BWA运行结果进行各阶段程序时间提取和统计求和

提取代码: package test import scala.io.Source import java.io.File._ import java.io.PrintWriter object...

基因数据处理31之avocado运行avocado-cli中的avocado问题3-变异识别找不到RecordGroupSample(null)

读入的read为: val fqFile = "hs38DHSE1L100F1.sam"读取结果:cleanedReads.count:1 {"readNum": 0, "contig": {"con...

基因数据处理52之cs-bwamem集群版运行(1千万条100bp的reads)

1.art生成模拟序列: art_illumina -ss HS20 -i GRCH38BWAindex/GRCH38chr1L3556522.fna -l 100 -c 10000000 -o g3...

基因数据处理21之BWASW算法ref分块建立索引然后比对(ref切分为四段,read为250条)

1.时间分析 对ref为单条染色体进行比对,第一次比对在3-5s不等,对chr1-4比对,在20s左右 连续比对多次后,对单染色体比对降到1s左右,chr1-4降到2s左右 不懂为什么比一次比对...

基因数据处理20之BWASW算法ref分块建立索引然后比对

SRR003161h20.fastq 三次测试: hadoop@Master:~/cloud/adam/xubo/data/GRCH38Sub$ bwa bwasw GRCH38chr1L35565...
内容举报
返回顶部
收藏助手
不良信息举报
您举报文章:基因数据处理41之mango使用失败
举报原因:
原因补充:

(最多只允许输入30个字)