org.apache.spark.streaming

Class DStreamGraph

Object

org.apache.spark.streaming.DStreamGraph

All Implemented Interfaces:

java.io.Serializable, Logging

public final class DStreamGraph
extends Object
implements scala.Serializable, Logging

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
DStreamGraph()

Method Summary

Methods

Modifier and Type	Method and Description
void	addInputStream(InputDStream<?> inputStream)
void	addOutputStream(DStream<?> outputStream)
Duration	batchDuration()
boolean	checkpointInProgress()
void	clearCheckpointData(Time time)
void	clearMetadata(Time time)
scala.collection.Seq<Job>	generateJobs(Time time)
InputDStream<?>[]	getInputStreams()
Duration	getMaxInputStreamRememberDuration() Get the maximum remember duration across all the input streams.
DStream<?>[]	getOutputStreams()
ReceiverInputDStream<?>[]	getReceiverInputStreams()
void	remember(Duration duration)
Duration	rememberDuration()
void	restart(Time time)
void	restoreCheckpointData()
void	setBatchDuration(Duration duration)
void	setContext(StreamingContext ssc)
void	start(Time time)
Time	startTime()
void	stop()
void	updateCheckpointData(Time time)
void	validate()
Time	zeroTime()

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

DStreamGraph

public DStreamGraph()

Method Detail

rememberDuration

public Duration rememberDuration()

checkpointInProgress

public boolean checkpointInProgress()

zeroTime

public Time zeroTime()

startTime

public Time startTime()

batchDuration

public Duration batchDuration()

start

public void start(Time time)

restart

public void restart(Time time)

stop

public void stop()

setContext

public void setContext(StreamingContext ssc)

setBatchDuration

public void setBatchDuration(Duration duration)

remember

public void remember(Duration duration)

addInputStream

public void addInputStream(InputDStream<?> inputStream)

addOutputStream

public void addOutputStream(DStream<?> outputStream)

getInputStreams

public InputDStream<?>[] getInputStreams()

getOutputStreams

public DStream<?>[] getOutputStreams()

getReceiverInputStreams

public ReceiverInputDStream<?>[] getReceiverInputStreams()

generateJobs

public scala.collection.Seq<Job> generateJobs(Time time)

clearMetadata

public void clearMetadata(Time time)

updateCheckpointData

public void updateCheckpointData(Time time)

clearCheckpointData

public void clearCheckpointData(Time time)

restoreCheckpointData

public void restoreCheckpointData()

validate

public void validate()

getMaxInputStreamRememberDuration

public Duration getMaxInputStreamRememberDuration()

Get the maximum remember duration across all the input streams. This is a conservative but safe remember duration which can be used to perform cleanup operations.