Skip to content

Commit

Permalink
Fix warn filter
Browse files Browse the repository at this point in the history
  • Loading branch information
tabdulradi committed Jan 27, 2016
1 parent 13dca89 commit 1a3e765
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ object BasicFilterUnionCombo {
val sc = new SparkContext(conf)
val inputRDD = sc.textFile(args(1))
val errorsRDD = inputRDD.filter(_.contains("error"))
val warningsRDD = inputRDD.filter(_.contains("error"))
val warningsRDD = inputRDD.filter(_.contains("warn"))
val badLinesRDD = errorsRDD.union(warningsRDD)
println(badLinesRDD.collect().mkString("\n"))
}
Expand Down

0 comments on commit 1a3e765

Please sign in to comment.