Spark学习之RDD算子

Transformations算子

1、map(T ⇒ U)

2、flatMap(T ⇒ List[U])

3、filter(T ⇒ Boolean)

4、mapPartitions(List[T] ⇒ List[U])

5、mapPartitionsWithIndex

6、mapValues

7、sample(withReplacement, fraction, seed)

8、union(other)

9、intersection(other)

10、subtract(other, numPartitions)

11、distinct(numPartitions)

12、reduceByKey((V, V) ⇒ V, numPartition)

13、groupByKey()

14、combineByKey()

15、aggregateByKey()

16、foldByKey(zeroValue)((V, V) ⇒ V)

17、join(other, numPartitions)

18、cogroup(other, numPartitions)

19、cartesian(other)

20、sortBy(ascending, numPartitions)

21、partitionBy(partitioner)

22、coalesce(numPartitions)

23、repartition(numPartitions)

24、repartitionAndSortWithinPartitions

Action算子

1、reduce( (T, T) ⇒ U )

2、collect()

3、count()

4、first()

5、take( N )

6、takeSample(withReplacement, fract)

7、fold(zeroValue)( (T, T) ⇒ U )

8、saveAsTextFile(path)

9、saveAsSequenceFile(path)

10、countByKey()

11、foreach( T ⇒ … )