arrylist spark_Spark error when convert JavaRDD to DataFrame: java.util.Arrays$ArrayList is not a va...

I encountered the same issue several days ago and the only way to solve this problem is the use an array of array. Why ? Here is the response:

An ArrayType is wrapper for Scala Arrays which correspond one-to-one to Java arrays. Java ArrayList is not mapped by default to Scala Array so that's why you get the exception:

java.util.Arrays$ArrayList is not a valid external type for schema of array

Hence, passing directly a String[] sould have work:

RowFactory.create(line.split(" "))

But since create takes as input an Object list as a row may have a columns list, the String[] get interpreted to a list of String columns. That's why a double array of String is required:

RowFactory.create(new String[][] {line.split(" ")})

However, still the mystery of constructing a DataFrame from a Java List of rows in the spark documentation. This is because the SparkSession.createDataFram