Convert a simple one line string to RDD in Spark

问题

I have a simple line:

line = "Hello, world"

I would like to convert it to an RDD with only one element. I have tried

sc.parallelize(line)

But it get:

sc.parallelize(line).collect()
['H', 'e', 'l', 'l', 'o', ',', ' ', 'w', 'o', 'r', 'l', 'd']

Any ideas?

回答1:

try using List as parameter:

sc.parallelize(List(line)).collect()

it returns

res1: Array[String] = Array(hello,world)

回答2:

The below code works fine in Python

sc.parallelize([line]).collect()

['Hello, world']

Here we are passing the parameter "line" as a list.

来源：https://stackoverflow.com/questions/26157620/convert-a-simple-one-line-string-to-rdd-in-spark