pyspark.RDD.collectWithJobGroup

RDD.collectWithJobGroup(groupId: str, description: str, interruptOnCancel: bool = False) → List[T]

When collect rdd, use this method to specify job group.

New in version 3.0.0.

Deprecated since version 3.1.0: Use pyspark.InheritableThread with the pinned thread mode enabled.

Parameters

groupIdstr

The group ID to assign.

descriptionstr

The description to set for the job group.

interruptOnCancelbool, optional, default False

whether to interrupt jobs on job cancellation.

Returns

list

a list containing all the elements

See also

RDD.collect()

SparkContext.setJobGroup()

pyspark.RDD.collectAsMap pyspark.RDD.combineByKey