贪心学院第二个课题,学习了知识图谱的相关的知识,并认识了neo4j这个图数据库:
本周作业 : 将上图中知识关系图谱中列的关系使用neo4j数据库将关系表示出来
实现思路 : 使用neo4j这种图数据库,创建不同的节点和节点对应的多维度的关系,最后把上面职业规划发展的展示图中能够展示所有的复杂关系。过程经历了九九八十一难,遇到问题需要一个个解决掉……
走来就遇到的问题1:
9月11日上完课以后,使用老师给的链接把neo4jwindows版本下载下来,本机本身已安装好的JDK1.8(之前学习周老师的Java接口自动化测试课程时就安装过了,已经好久没用,它正安静的睡大觉,11号又被我唤醒了),但是我去了bin目录下双击neo4j.bat,可是却一直报错说找不到Neo4j-Management.psd1这个文件,可是我看到它明明是存在的,看到这个报错信息,我可郁闷了,我没接触过这个数据库,这报错信息多坑人啊。
这里我有在百度网上查过,解决方法是需要把eo4j.ps1文件里Import-Module "$PSScriptRootNeo4j-Management.psd1"改为自己安装neo4j的路径,并且需要使用neo4j.bat install-service来安装neo4j服务 安装好后,再双击neo4j.bat就可以愉快的玩耍起来了!
遇到的问题2:
周老师讲的课还是很生动的,本人当时感觉应该可以做出来,就是会慢一点,想法很美好,可现实却很残酷!我建了两个节点和关系忘了把节点的name属性加上了,只写了节点名称,就形成了节点没有name属性,只有id值的尴尬场景。我连create都没写顺溜,就更不会修改了。所以我想办法把之前没有name属性的节点和节点间关系删除掉了,再从头来过:
通过match语句把要删除的节点和节点间关系全部查出来,后面跟着需要删除的节点和关系
match (python:`Python基础`)-[officeRelation:pythonWithAutoOffice]->(AutoOffice:`自动化办公`) delete python,AutoOffice,officeRelation
提示我Deleted 2 nodes, deleted 1 relationship, completed after 7 ms. 我总算松了一口气,可以正式开始了!
创建过程记录:
第一步,创建Knowledge节点:
1.1开始创建Python基础、requests、bs4、scrapy、MySQL、MongoDB知识节点和他们之间的关系,具体的语句如下:
create (p:Knowledge{name:"Python基础 "}),
(r:Knowledge{name:"requests"}),
(bs:Knowledge{name:"bs4"}),
(s:Knowledge{name:"scrapy"}),
(mysql:Knowledge{name:"MySQL"}),
(mongo:Knowledge{name:"MongoDB"}),
(p)-[studyPath1:pythonStudyPath1]->(r),
(r)-[studyPath2:pythonStudyPath2]->(bs),
(bs)-[studyPath3:pythonStudyPath3]->(s),
(s)-[studyPath4:pythonStudyPath4]->(mysql),
(mysql)-[studyPath5:pythonStudyPath5]->(mongo)
执行以上语句得到Added 6 labels, created 6 nodes, set 6 properties, created 5 relationships, completed after 2466 ms.,就表示执行成功了,在Database Information 这里能看到6个Node Labels 节点标签,5个Relationship Types关系类型
我们通过match语句查询出刚才创建的所有的节点和节点间关系:
MATCH (n) RETURN n LIMIT 25 [limit 25 最多显示25条]
这里发现创建出来的关系和老师的图谱上显示的并不同,后来仔细看了一下图谱,发现应该是scrapy下面除了和mysql有关联关系,还有和mongoDB也是有关系,而我创建的关系中把MongoDB和mysql设置成了上下的关联关系,所以我需要修改一下节点关系,这里我用的比较笨的办法,先把原来MySQL指向MongoDB的关系删除掉,再从已有的scrapy建个指向MongoDB的关系:
执行过程:
1) 首先把原来MySQL指向MongoDB的关系删除掉:
match (s:Knowledge) where s.name="MySQL"
match (m:Knowledge) where m.name="MongoDB"
match (s)-[path5:pythonStudyPath5]->(m)
delete path5
查询到MySQL-MongoDB的关系后,删除它
2)在原来的scrapy节点上创建一个关系指向MongoDB
首先查到已存在的scrapy节点信息,然后查到已存在的MongoDB节点信息
最后在两个节点中创建节点间关系:
match (s:Knowledge) where s.name="scrapy"
match(m:Knowledge) where m.name="MongoDB"
create (s)-[path5:pythonStudyPath5]->(m) return s , path5 , m
最后显示成:
第二步:
2.1.按照上面的方法继续创建别的Knowledge节点及关系,创建的语句:
create (file:Knowledge{name:"文件处理"}),
(flask:Knowledge{name:"Flak"}),
(django:Knowledge{name:"Django"}),
(file)-[path6:pythonStudyPath6]->(flask),(file)-[path7:pythonStudyPath7]->(django)
共创建了三个节点,一个关系
2.2 创建Python基础指向文件处理的关系的语句如下:
match (python:Knowledge) where id(python)=58
match (file:Knowledge) where file.name="文件处理"
create (python)-[path8:pythonStudyPath8]->(file) return python , path8 , file
2.3创建Django指向MySQL的关系语句如下:
match (django:Knowledge) where django.name="Django"
match (mysql:Knowledge) where mysql.name="MySQL"
create (django)-[path9:pythonStudyPath9]->(mysql) return mysql , path9, django
2.4创建Flask指向MySQL的关系语句如下:
match (flask:Knowledge) where flask.name="Flask"
match (mysql:Knowledge) where mysql.name="MySQL"
create (flask)-[path10:pythonStudyPath10]->(mysql) return mysql , path10, flask
整体的结构如下:
2.5创建知识节点Hbase,Hive以及Python基础和Hbase,Hive之间关系语句如下:
create (:Knowledge{name:"Hbase"}),(:Knowledge{name:"Hive"})
match (python:Knowledge) where id(python)=58
match(hbase:Knowledge) where hbase.name="Hbase"
create (python)-[path11:pythonStudyPath11]->(hbase) return python ,path11,hbase
match (python:Knowledge) where id(python)=58
match(hive:Knowledge) where hive.name="Hive"
create (python)-[path12:pythonStudyPath12]->(hive) return python ,hive,path12
2.6创建以Java为基础的知识节点Java、Storm、flume、Kafka、Hadoop、Flink
create (:Knowledge{name:"Java"}),
(:Knowledge{name:"Hadoop"}),
(:Knowledge{name:"flume"}),
(:Knowledge{name:"Storm"}),
(:Knowledge{name:"Kafka"}),
(:Knowledge{name:"Flink"})
2.7创建从Java到其他5个节点的关系语句如下:
match (j:Knowledge) where j.name="Java"
match(h:Knowledge)where h.name="Hadoop"
match (flume:Knowledge) where flume.name="flume"
match(s:Knowledge)where s.name="Storm"
match(k:Knowledge)where k.name="Kafka"
match(F:Knowledge)where F.name="Flink"
create(j)-[:javaStudyPath1]->(h)
create(j)-[:javaStudyPath2]->(flume)
create(j)-[:javaStudyPath3]->(s)
create(j)-[:javaStudyPath4]->(k)
create(j)-[:javaStudyPath5]->(F)
2.8创建scala、spark知识节点及之间关系的语句如下:
create (scala:Knowledge{name:"Scala"}),
(spark:Knowledge{name:"Spark"}),
match (j:Knowledge) where j.name="Scala"
match(h:Knowledge)where h.name="Spark"
create (j)-[path:javaStudyPath6]->(h)
return j,h,path
2.9创建高数、统计、机器学习、深度学习节点及之间关系语句如下:
create (math:Knowledge{name:"高数"}),
(cou:Knowledge{name:"统计"}),
(machine:Knowledge{name:"机器学习"}),
(deep:Knowledge{name:"深度学习"}),
(math)-[path1:mathStudyPath1]->(cou),
(cou)-[path2:mathStudyPath2]->(machine),
(machine)-[path3:mathStudyPath3]->(deep)
return math,cou,machine,deep
2.10 补充Python基础和hbase和Mysql 、MongoDB数据库之间的关系语句如下:
match (python:Knowledge)where id(python)=58
match (mysql:Knowledge) where mysql.name="MySQL"
match (mongo:Knowledge) where mongo.name="MongoDB"
create (python)-[path1:dbStudyPath1]->(mysql)
create (python)-[path2:dbStudyPath2]->(mongo)
return python,mongo,path1,path2
第三步,创建Job节点信息:
3.1创建名为自动化办公的Job类型节点,并将自动化办公节点和Python基础关联起来,具体语句如下:
match (p:Knowledge) where id(p)=58
create (auto:Job{name:"自动化办公"})
match (auto:Job) where auto.name=”自动化办公”
create (auto)-[autoPath1:autoOfficeStudyPath1]->(p)
创建pytone基础和文件处理的关系的语句如下:
match (p:Knowledge) where id(p)=58
match(file:Knowledge) where file.name="文件处理"
create (p)-[autoPath2:autoOfficeStudyPath2]->(file)
3.2创建名为Web开发的Job节点,将它和Python基础及Linux的知识节点相关联,具体的语句如下:
match(python:Knowledge)where id(python)=58
create (web:Job{name:"Web开发"})
create (linux:Knowledge{name:"Linux"})
create (web)-[webPath1:webStudyPath]->(python)
create (web)-[webPath2:webStudyPath2]->(linux)
这里老是会忘记写return语句
3.3创建名为爬虫的Job节点,并创建它下面的知识节点网络、HTML、Xpath、CSS并与之进行关联,语句如下:
create (spider:Job{name:"爬虫"}),
(html:Knowledge{name:"HTML"}),
(net:Knowledge{name:"网络"}),
(xpath:Knowledge{name:"XPath"}),
(css:Knowledge{name:"CSS"}),
(spider)-[path1:spiderStudyPath1]->(html),
(spider)-[path2:spiderStudyPath2]->(net),
(html)-[path3:spiderStudyPath3]->(css),
(html)-[path4:spiderStudyPath4]->(xpath) return spider,html,net,css,xpath
3.4 补两条关系,爬虫到Linux的关系,机器学习到Linux的关系:
match (spider)where (spider.name="爬虫")
match (linux)where (linux.name="Linux")
create (spider)-[path5:spiderStudyPath5]->(linux) return spider,linux,path5
match (machine:Knowledge)where (machine.name="机器学习")
match (linux:Knowledge)where (linux.name="Linux")
create (machine)-[path:machineStudyPath]->(linux) return machine,linux,path
3.5创建名为数据分析的Job节点及Windows办公软件、所属行业的业务知识的知识节点,
并将数据分析与它下面的两个知识关联,并关联到Linux和统计的知识节点,具体语句如下:
首先节点信息:
create (data:Job{name:"数据分析"}),
(office:Knowledge{name:"Windows办公软件"}),
(business:Knowledge{name:"所属行业的业务知识"})
然后创建节点间关系:
match (data:Job) where data.name="数据分析"
match (linux:Knowledge)where linux.name="Linux"
match (office:Knowledge)where office.name="Windows办公软件"
match (business:Knowledge)where business.name="所属行业的业务知识"
match (cou:Knowledge)where cou.name="统计"
match (python:Knowledge)where id(python)=58
create (data)-[path1:dataAnalysisPath1]->(office),
(data)-[path2:dataAnalysisPath2]->(business),
(data)-[path3:dataAnalysisPath3]->(linux),
(data)-[path4:dataAnalysisPath4]->(cou)
(data)-[path5:dataAnalysisPath5]->(python)
return data,linux , office ,business , cou ,python,path1 ,pah2,path3,path4,path5
3.6创建人工智能Job节点,及它与python基础、高数的关系:
create (ai:Job{name:"人工智能"})
match (ai:Job) where ai.name="人工智能"
match (python:Knowledge)where id(python)=58
match(math:Knowledge) where math.name="高数"
create (ai)-[path1:aiStudyPath1]->(python)
create (ai)-[path2:aiStudyPath2]->(math)
return ai,python,math,path1,path2
第四步,创建岗位方向节点数据:
4.1 创建深度学习知识节点下面的四个岗位方向节点信息,并创建好它们之间的关系,具体语句如下:
match(deep:Knowledge) where deep.name="深度学习"
create (deep)-[path1:deepStudyPath1]->(sense:Knowledge{name:"视觉"})
create (deep)-[path2:deepStudyPath2]->(speech:Knowledge{name:"语音"})
create (deep)-[path3:deepStudyPath3]->(nlp:Knowledge{name:"NLP"})
create (deep)-[path4:deepStudyPath4]->(recommend:Knowledge{name:"推荐"})
return deep,path1,path2,path3,path4
这个语句写的太快,忘记把视觉、语音、NLP、推荐四个节点设置为岗位方向,所以我又把原来的标签信息删除了,修改了这四个节点的知识标签为岗位方向标签:
match (sense:Knowledge)where sense.name="视觉"
remove sense:Knowledge
set sense:Direction return sense
match (speech:Knowledge)where speech.name="语音"
remove speech:Knowledge
set speech:Direction return speech
match (nlp:Knowledge)where nlp.name="NLP"
remove nlp:Knowledge
set nlp:Direction return nlp
match (recommend:Knowledge)where recommend.name="推荐"
remove recommend:Knowledge
set recommend:Direction return recommend
修改完后,关系如下图:
4.2创建大数据这个Job节点,离线计算、实时计算两个岗位方向节点及大数据与Linux、离线计算、实时计算间关系。由于不会使用with,都是创建节点和关系分开写的:
先创建节点:
create (bigdata:Job{name:"大数据"}) ,
(offline:Direction{name:"离线计算"}),
(online:Direction{name:"实时计算"})
再创建关系
match(bigdata:Job) where bigdata.name="大数据"
match(linux:Knowledge) where linux.name="Linux"
match(offline:Direction) where offline.name="离线计算"
match(online:Direction) where online.name="实时计算"
create (bigdata)-[path1:bigdataStudyPath1]->(offline)
create (bigdata)-[path2:bigdataStudyPath2]->(online)
create (bigdata)-[path3:bigdataStudyPath3]->(linux)
return bigdata,linux,offline,online ,path1,path2,path3
4.3创建离线计算岗位方向节点和Hive、Hbase、Hadoop、Scala知识节点的关系语句如下:
match(offline:Direction) where offline.name="离线计算"
match(hive:Knowledge) where hive.name="Hive"
match(hbase:Knowledge) where hbase.name="Hbase"
match(hadoop:Knowledge) where hadoop.name="Hadoop"
match(scala:Knowledge) where scala.name="Scala"
create (offline)-[path1:bigdataStudyPath4]->(hive) ,
(offline)-[path2:bigdataStudyPath5]->(hbase) ,
(offline)-[path3:bigdataStudyPath6]->(hadoop),
(offline)-[path4:bigdataStudyPath7]->(scala)
return hive,offline,hbase,hadoop, scala,path1,path2,path3,path4
4.4补充实时计算 岗位方向节点指向Java知识节点的关系语句如下:
match(online:Direction) where online.name="实时计算"
match(java:Knowledge) where java.name="Java"
create (online)-[path1:bigdataStudyPath8]->(java)
return online,java,path1
最终完成后,数据结构如下图:
本次学习的感受:
图数据库neo4j的学习,一开始会感觉挺难的,做着做着,就感觉没有想象中那么难。我想新的知识刚接触时都会比较晕,一旦会了,剩下就需要细心了。我还挺喜欢看这些小球的,写好了特地把它们重新布了一下局,因为关系比较多,线连的比较乱。还是不如思维导图好看,不过,小球可以被拖动,感觉它们很有灵性的样子!