python获取当前日期的前一天爆炸_python – 在PySpark中爆炸

我想从包含单词列表的DataFrame转换为DataFrame,每个单词都在自己的行中.

如何在DataFrame中的列上进行爆炸？

下面是我的一些尝试示例,您可以在其中取消注释每个代码行并获取以下注释中列出的错误.我在Python 2.7中使用PySpark和Spark 1.6.1.

from pyspark.sql.functions import split, explode

DF = sqlContext.createDataFrame([('cat \n\n elephant rat \n rat cat', )], ['word'])

print 'Dataset:'

DF.show()

print '\n\n Trying to do explode: \n'

DFsplit_explode = (

DF

.select(split(DF['word'], ' '))

# .select(explode(DF['word'])) # AnalysisException: u"cannot resolve 'explode(word)' due to data type mismatch: input to function explode should be array or map type, not StringType;"

# .map(explode) # AttributeError: 'PipelinedRDD' object has no attribute 'show'

# .explode() # AttributeError: 'DataFrame' object has no attribute 'explode'

).show()

# Trying without split

print '\n\n Only explode: \n'

DFsplit_explode = (

DF

.select(explode(DF['word'])) # AnalysisException: u"cannot resolve 'explode(word)' due to data type mismatch: input to function explode should be array or map type, not StringType;"

).show()

请指教