org.apache.spark.streaming.scheduler

BatchInfo

case class BatchInfo(batchTime: Time, streamIdToInputInfo: Map[Int, StreamInputInfo], submissionTime: Long, processingStartTime: Option[Long], processingEndTime: Option[Long], outputOperationInfos: Map[Int, OutputOperationInfo]) extends Product with Serializable

:: DeveloperApi :: Class having information on completed batches.

batchTime

Time of the batch

streamIdToInputInfo

A map of input stream id to its input info

submissionTime

Clock time of when jobs of this batch was submitted to the streaming scheduler queue

processingStartTime

Clock time of when the first job of this batch started processing

processingEndTime

Clock time of when the last job of this batch finished processing

outputOperationInfos

The output operations in this batch

Annotations
@DeveloperApi()
Source
BatchInfo.scala
Linear Supertypes
Serializable, Serializable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. BatchInfo
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. AnyRef
  7. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new BatchInfo(batchTime: Time, streamIdToInputInfo: Map[Int, StreamInputInfo], submissionTime: Long, processingStartTime: Option[Long], processingEndTime: Option[Long], outputOperationInfos: Map[Int, OutputOperationInfo])

    batchTime

    Time of the batch

    streamIdToInputInfo

    A map of input stream id to its input info

    submissionTime

    Clock time of when jobs of this batch was submitted to the streaming scheduler queue

    processingStartTime

    Clock time of when the first job of this batch started processing

    processingEndTime

    Clock time of when the last job of this batch finished processing

    outputOperationInfos

    The output operations in this batch

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  7. val batchTime: Time

    Time of the batch

  8. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  9. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  10. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  11. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  12. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  13. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  14. final def notify(): Unit

    Definition Classes
    AnyRef
  15. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  16. def numRecords: Long

    The number of recorders received by the receivers in this batch.

  17. val outputOperationInfos: Map[Int, OutputOperationInfo]

    The output operations in this batch

  18. def processingDelay: Option[Long]

    Time taken for the all jobs of this batch to finish processing from the time they started processing.

    Time taken for the all jobs of this batch to finish processing from the time they started processing. Essentially, it is processingEndTime - processingStartTime.

  19. val processingEndTime: Option[Long]

    Clock time of when the last job of this batch finished processing

  20. val processingStartTime: Option[Long]

    Clock time of when the first job of this batch started processing

  21. def schedulingDelay: Option[Long]

    Time taken for the first job of this batch to start processing from the time this batch was submitted to the streaming scheduler.

    Time taken for the first job of this batch to start processing from the time this batch was submitted to the streaming scheduler. Essentially, it is processingStartTime - submissionTime.

  22. val streamIdToInputInfo: Map[Int, StreamInputInfo]

    A map of input stream id to its input info

  23. val submissionTime: Long

    Clock time of when jobs of this batch was submitted to the streaming scheduler queue

  24. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  25. def totalDelay: Option[Long]

    Time taken for all the jobs of this batch to finish processing from the time they were submitted.

    Time taken for all the jobs of this batch to finish processing from the time they were submitted. Essentially, it is processingDelay + schedulingDelay.

  26. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  27. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  28. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Deprecated Value Members

  1. def streamIdToNumRecords: Map[Int, Long]

    Annotations
    @deprecated
    Deprecated

    (Since version 1.5.0) Use streamIdToInputInfo instead

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped