Friday, 11 January 2019

Spark MR WC

val myfile = sc.textFile("/user/cloudera/input.txt")

val counts = myfile.flatMap(line => line.split(" ")).map(word => (word, 1)).reduceByKey(_ + _)

counts.saveAsTextFile("/home/cloudera/Desktop/output")

No comments:

Post a Comment