Here we are going to discuss mapPartitions ,mapPartitionsWithIndex and filter operation. (a) MapPartitions: This transformation is similar to map,but runs separately on each partition (block) of the RDD. MapPartitions can be used as an alternative to map and foreach. Q-1 Remove the specific number from input list or input RDD using mapPartitions Read more…
Here we are going to discuss mapPartitions ,mapPartitionsWithIndex and filter operation. (a) MapPartitions: This transformation is similar to map,but runs separately on each partition (block) of the RDD. mapPartitions() can be used as an alternative to map() and foreach(). Q-1 How to iterate each partitions using mapPartitions transformation and convert each element Read more…
We will discuss and practice each transformations and actions using in spark RDD. Transformations: Map,FlatMap, MapPartition, Filter, Sample, Union, Intersection, Distinct, ReduceByKey, GroupByKey, AggregateByKey, Join, Repartition, Coalesce etc . Actions: Reduce, Collect, Count, First, Take, Foreach, saveAsTextFile etc. Q-1 What all different ways to create the RDD? Ans:
We will discuss and practice below each transformations and actions in spark RDD. Transformations: Map,FlatMap, MapPartition, Filter, Sample, Union, Intersection, Distinct, ReduceByKey, GroupByKey, AggregateByKey, Join, Repartition, Coalesce etc . Actions: Reduce, Collect, Count, First, Take, Foreach, saveAsTextFile etc. Q-1 What all different ways to create the RDD? Ans: Parallelize the Read more…