1. http://dbis.informatik.uni-freiburg.de/forschung/projekte/DiPoS/RDFPath.html下载可执行的jar包,放到集群的master机器上
2. NT格式的数据上传到HDFS上
3.hadoop jar RDFPath.jar query.Run query-name dataset cyclemode
4. 先载入数据?
hadoop jar RDFPath.jar loader.Dblp RDFPath/data/lubm1.nt 0
(1/4) Streaming Data from HDFSHDFS Path: hdfs://node01.dbcluster1.tju.edu.cn:9000/user/yueqixin/RDFPath/data/lubm1.nt
Parser: ExNTripleParser
Initializing Berkeley DB
Dictionary Encoding (Subjects) : false
Dictionary Encoding (Objects) : false
Dictionary Encoding (Predicates): false
Writing tuples to local folder: RDFPath/tmp/
Dictionary Entries: 0
Created local folder: /home/yueqixin/RDFPath/tmp/www.w3.org/1999/02/22-rdf-syntax-ns#type>
Created local folder: /home/yueqixin/RDFPath/tmp/www.w3.org/2002/07/owl#imports>
Created local folder: /home/yueqixin/RDFPath/tmp/#name>
Created local folder: /home/yueqixin/RDFPath/tmp/#subOrganizationOf>
Created local folder: /home/yueqixin/RDFPath/tmp/#teacherOf>
Created local folder: /home/yueqixin/RDFPath/tmp/#undergraduateDegreeFrom>
Created local folder: /home/yueqixin/RDFPath/tmp/#mastersDegreeFrom>
Created local folder: /home/yueqixin/RDFPath/tmp/#doctoralDegreeFrom>
Created local folder: /home/yueqixin/RDFPath/tmp/#worksFor>
Created local folder: /home/yueqixin/RDFPath/tmp/#emailAddress>
Created local folder: /home/yueqixin/RDFPath/tmp/#telephone>
Created local folder: /home/yueqixin/RDFPath/tmp/#researchInterest>
Created local folder: /home/yueqixin/RDFPath/tmp/#headOf>
Created local folder: /home/yueqixin/RDFPath/tmp/#memberOf>
Created local folder: /home/yueqixin/RDFPath/tmp/#takesCourse>
Created local folder: /home/yueqixin/RDFPath/tmp/#advisor>
Created local folder: /home/yueqixin/RDFPath/tmp/#publicationAuthor>
Created local folder: /home/yueqixin/RDFPath/tmp/#teachingAssistantOf>
Writing complete
(2/4) Copying Triples from local to HDFS:
Local Path: RDFPath/tmp/
HDFS Path: hdfs://node01.dbcluster1.tju.edu.cn:9000/user/yueqixin/RDFPath/lubm1.nt/
(4/4) Creating SequenceFiles of PathObjects
Predicates (Counter): 18
Predicates (Edges): #emailAddress> #memberOf> #takesCourse> #name> #doctoralDegreeFrom> #subOrganizationOf> #teachingAssistantOf> //www.w3.org/2002/07/owl#imports> #teacherOf> #telephone> #researchInterest> #publicationAuthor> #headOf> #advisor> #undergraduateDegreeFrom> #worksFor> //www.w3.org/1999/02/22-rdf-syntax-ns#type> #mastersDegreeFrom>
Creating Reversing Paths: false
Distributing Dictionary: false
Exception in thread "main" java.io.IOException: No FileSystem for scheme: null
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2660)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(FileInputFormat.java:520)
at path.Assignment.<init>(Assignment.java:102)
at loader.InputProcessor.process(InputProcessor.java:68)
at loader.InputProcessor.<init>(InputProcessor.java:40)
at loader.Dblp.main(Dblp.java:157)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
似乎是把lubm1.nt转换为某种中间格式的数据,但是发生了错误:Exception in thread "main" java.io.IOException: No FileSystem for scheme: null 查资料应该是由于Hadoop的版本不兼容,毕竟集群是hadoop2.7.4,而RDFPath代码是2011年实现的了,那会才是haoop0.23.0?
看官网的介绍,测试用例是在CDH3上执行的,CDH3版本是基于Apache hadoop 0.20.2改进的,可以确定是版本不兼容吧。
还在学习中……