neo4j 连接 spark

今天使用neo4j连接spark
neo4j版本3.4
spark版本1.6.0
(1) 首先,需要添加jar包 neo4j-spark-connector_2.10-1.0.0-RC1.jar或者添加maven依赖

org.neo4j.spark
neo4j-spark-connector_2.10
1.0.0-RC1

(2) 设置spark连接信息
val conf : SparkConf = new SparkConf().setAppName(“InitSpark”).setMaster(“local[*]”) conf.set(“spark.neo4j.bolt.url”,”bolt://localhost:7687”)
conf.set(“spark.neo4j.bolt.user”,”neo4j”)
conf.set(“spark.neo4j.bolt.password”,”123456”)

简单例子

object Neo4jDataFrameTest { 
 def main(args: Array[String]): Unit = {      
val sQLContext = InitSpark.getSqlContext   
val query =" MATCH p=(m:ITEM)<-[r:rel*]-(n:ITEM) where m.item = '84010420' and  ALL(c in r where c.exdt>'2018-12-15')" +  " with  m.item as mark,length(r) as len, last(r).number as num,reduce(s=1.0 ,x in rels(p)| s*tofloat(x.number)) as nums, last(r).mitem as mitem , last(r) as bom  " +  " return  mark,bom.erpid as erpid, bom.virtual as virtual,mitem,bom.item as item,bom.pono as pono," +  " bom.comment as comment, bom.warehouse as warehouse, bom.exdt as exdt, bom.indt as indt,tofloat(num) as num,nums,len "   
val df = Neo4jDataFrame.withDataType(sQLContext,query,Seq.empty,"mark" ->    StringType,"erpid" ->StringType,"virtual"->LongType,  "mitem"->StringType,"item"->StringType,"pono"->LongType,"comment"->StringType,"warehouse"->StringType,"exdt"->StringType,  "indt" ->StringType,"num"->DoubleType,"nums"-> DoubleType ,"len"->LongType)    
df.show(1000)  }}
object Neo4jConnectSparkGraph {
  def main(args: Array[String]): Unit = {
     val sc = InitSpark.getSC
     val sQLContext = InitSpark.getSqlContext
     val cypher = "match (n:ITEM) return n.item limit 10"
     val neo = Neo4jRowRDD(sc,cypher)
   }
 }
  • 2
    点赞
  • 4
    收藏
    觉得还不错? 一键收藏
  • 10
    评论
Graph Algorithms: Practical Examples in Apache Spark and Neo4j By 作者: Mark Needham – Amy E. Hodler ISBN-10 书号: 1492047686 ISBN-13 书号: 9781492047681 Edition 版本: 1 出版日期: 2019-01-04 pages 页数: (217) Discover how graph algorithms can help you leverage the relationships within your data to develop more intelligent solutions and enhance your machine learning models. You’ll learn how graph analytics are uniquely suited to unfold complex structures and reveal difficult-to-find patterns lurking in your data. Whether you are trying to build dynamic network models or forecast real-world behavior, this book illustrates how graph algorithms deliver value—from finding vulnerabilities and bottlenecks to detecting communities and improving machine learning predictions. This practical book walks you through hands-on examples of how to use graph algorithms in Apache Spark and Neo4j—two of the most common choices for graph analytics. Also included: sample code and tips for over 20 practical graph algorithms that cover optimal pathfinding, importance through centrality, and community detection. Learn how graph analytics vary from conventional statistical analysis Understand how classic graph algorithms work, and how they are applied Get guidance on which algorithms to use for different types of questions Explore algorithm examples with working code and sample datasets from Spark and Neo4j See how connected feature extraction can increase machine learning accuracy and precision Walk through creating an ML workflow for link prediction combining Neo4j and Spark

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 10
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值