python每行5个数_字数小于5的行数

使用pyspark，

我想找到包含单词数的行数&lt；5

我写了这个代码，但我不知道它有什么问题from pyspark.sql import SparkSession, Row

spark = SparkSession.builder.master("spark://master:7077").appName('test').config(conf=SparkConf()).getOrCreate()

df = spark.read.text('text.txt')

rdd = df.rdd

print(df.count())

rdd1=rdd.filter(lambda line: len((line.split(" "))<5)).collect()

print(rdd1.count())

This is the a small part of the Error

-----------------------------------------------------------------------

Py4JJavaError Traceback (most recent call last)

in ()

9 rdd = df.rdd

10 print(df.count())

---> 11 rdd1=rdd.filter(lambda line: len((line.split(" "))<5)).collect()

12 print(rdd1.count())

13

Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.collectAndServe.

: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 144.0 failed 1 times, most recent failure: Lost task 0.0 in stage 144.0 (TID 144, localhost): org.apache.spark.api.python.PythonException: Traceback (most recent call last):

File "/Users/ff/spark/python/lib/pyspark.zip/pyspark/sql/types.py", line 1497, in __getattr__

idx = self.__fields__.index(item)

ValueError: 'split' is not in list