org.apache.spark

Class Accumulable<R,T>

Object

org.apache.spark.Accumulable<R,T>

All Implemented Interfaces:

java.io.Serializable

Direct Known Subclasses:

Accumulator

public class Accumulable<R,T>
extends Object
implements java.io.Serializable

A data type that can be accumulated, ie has an commutative and associative "add" operation, but where the result type, R, may be different from the element type being added, T.

You must define how to add data, and how to merge two of these together. For some data types, such as a counter, these might be the same operation. In that case, you can use the simpler Accumulator. They won't always be the same, though -- e.g., imagine you are accumulating a set. You will add items to the set, and you will union two sets together.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
Accumulable(R initialValue, AccumulableParam<R,T> param)

Method Summary

Methods

Modifier and Type	Method and Description
void	add(T term) Add more data to this accumulator / accumulable
long	id()
R	localValue() Get the current value of this accumulator from within a task.
void	merge(R term) Merge two accumulable objects together
void	setValue(R newValue) Set the accumulator's value; only allowed on master
String	toString()
R	value() Access the accumulator's current value; only allowed on master.
R	zero()

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Accumulable

public Accumulable(R initialValue,
           AccumulableParam<R,T> param)

Method Detail

id

public long id()

zero

public R zero()

add

public void add(T term)

Add more data to this accumulator / accumulable

Parameters:

term - the data to add

merge

public void merge(R term)

Merge two accumulable objects together

Normally, a user will not want to use this version, but will instead call add.

Parameters:

term - the other R that will get merged with this

value

public R value()

Access the accumulator's current value; only allowed on master.

localValue

public R localValue()

Get the current value of this accumulator from within a task.

This is NOT the global value of the accumulator. To get the global value after a completed operation on the dataset, call value.

The typical use of this method is to directly mutate the local value, eg., to add an element to a Set.

setValue

public void setValue(R newValue)

Set the accumulator's value; only allowed on master

toString

public String toString()

Overrides:

toString in class Object