Question: How to add new content to a record using Spark

Question

How to add new content to a record using Spark

Answers 1
Added at 2017-01-01 15:01
Tags
Question

I have a text file containing several records like:

 aaaaa
 bbbbb
 ccccc

I want to add a string (ex. "record:") in front of each record, so each of them will be like:

 record:aaaaa
 record:bbbbb
 record:ccccc

I have the code that reads this text file and creates a RDD for this file:

 val aRdd = sc.textFile("/tmp/myFile")

How do I add the string to each record in this RDD using Spark? Many Thanks.

Answers
nr: #1 dodano: 2017-01-01 15:01

By using RDD.map:

val rddWithRecord = sc.textFile("/tmp/myFile").map(currentWord => s"record:$currentWord")
rddWithRecord.print()
Source Show
◀ Wstecz