解决pyspark环境下GraphFrames报错问题

最新推荐文章于 2023-08-01 23:45:00 发布

九筒-

最新推荐文章于 2023-08-01 23:45:00 发布

阅读量1k

点赞数

文章标签： spark 大数据分布式图论

本文链接：https://blog.csdn.net/weixin_44343282/article/details/126212999

版权

背景

Spark图计算实战：在pyspark环境下使用GraphFrames库

环境

mac os
conda→python=3.8
jupyter notebook
pyspark=3.3.0
graphframes=0.6

代码

from pyspark import SparkConf, SparkContext
from pyspark.sql import SparkSession
from graphframes import GraphFrame

sc = SparkContext()
spark = SparkSession(sc)

# Vertics DataFrame
vertics = spark.createDataFrame([
  ("a", "Alice", 34),
  ("b", "Bob", 36),
  ("c", "Charlie", 37),
  ("d", "David", 29),
  ("e", "Esther", 32),
  ("f", "Fanny", 38),
  ("g", "Gabby", 60)
], ["id", "name",