Class

org.apache.spark.mllib.clustering

StreamingKMeans

Related Doc: package clustering

Permalink

class StreamingKMeans extends Logging with Serializable

StreamingKMeans provides methods for configuring a streaming k-means analysis, training the model on streaming, and using the model to make predictions on streaming data. See KMeansModel for details on algorithm and update rules.

Use a builder pattern to construct a streaming k-means analysis in an application, like:

val model = new StreamingKMeans()
  .setDecayFactor(0.5)
  .setK(3)
  .setRandomCenters(5, 100.0)
  .trainOn(DStream)
Annotations
@Since( "1.2.0" )
Source
StreamingKMeans.scala
Linear Supertypes
Serializable, Serializable, Logging, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. StreamingKMeans
  2. Serializable
  3. Serializable
  4. Logging
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new StreamingKMeans()

    Permalink
    Annotations
    @Since( "1.2.0" )

  2. new StreamingKMeans(k: Int, decayFactor: Double, timeUnit: String)

    Permalink
    Annotations
    @Since( "1.2.0" )

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  6. var decayFactor: Double

    Permalink
    Annotations
    @Since( "1.2.0" )

  7. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  8. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  9. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  10. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  11. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  12. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  13. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  14. def isTraceEnabled(): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  15. var k: Int

    Permalink
    Annotations
    @Since( "1.2.0" )

  16. def latestModel(): StreamingKMeansModel

    Permalink

    Return the latest model.

    Return the latest model.

    Annotations
    @Since( "1.2.0" )

  17. def log: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  18. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  19. def logDebug(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  20. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  21. def logError(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  22. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  23. def logInfo(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  24. def logName: String

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  25. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  26. def logTrace(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  27. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  28. def logWarning(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  29. var model: StreamingKMeansModel

    Permalink
    Attributes
    protected

  30. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  31. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  32. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  33. def predictOn(data: JavaDStream[Vector]): JavaDStream[Integer]

    Permalink

    Java-friendly version of predictOn.

    Java-friendly version of predictOn.

    Annotations
    @Since( "1.4.0" )

  34. def predictOn(data: DStream[Vector]): DStream[Int]

    Permalink

    Use the clustering model to make predictions on batches of data from a DStream.

    Use the clustering model to make predictions on batches of data from a DStream.

    data

    DStream containing vector data

    returns

    DStream containing predictions

    Annotations
    @Since( "1.2.0" )

  35. def predictOnValues[K](data: JavaPairDStream[K, Vector]): JavaPairDStream[K, Integer]

    Permalink

    Java-friendly version of predictOnValues.

    Java-friendly version of predictOnValues.

    Annotations
    @Since( "1.4.0" )

  36. def predictOnValues[K](data: DStream[(K, Vector)])(implicit arg0: ClassTag[K]): DStream[(K, Int)]

    Permalink

    Use the model to make predictions on the values of a DStream and carry over its keys.

    Use the model to make predictions on the values of a DStream and carry over its keys.

    K

    key type

    data

    DStream containing (key, feature vector) pairs

    returns

    DStream containing the input keys and the predictions as values

    Annotations
    @Since( "1.2.0" )

  37. def setDecayFactor(a: Double): StreamingKMeans.this.type

    Permalink

    Set the forgetfulness of the previous centroids.

    Set the forgetfulness of the previous centroids.

    Annotations
    @Since( "1.2.0" )

  38. def setHalfLife(halfLife: Double, timeUnit: String): StreamingKMeans.this.type

    Permalink

    Set the half life and time unit ("batches" or "points").

    Set the half life and time unit ("batches" or "points"). If points, then the decay factor is raised to the power of number of new points and if batches, then decay factor will be used as is.

    Annotations
    @Since( "1.2.0" )

  39. def setInitialCenters(centers: Array[Vector], weights: Array[Double]): StreamingKMeans.this.type

    Permalink

    Specify initial centers directly.

    Specify initial centers directly.

    Annotations
    @Since( "1.2.0" )

  40. def setK(k: Int): StreamingKMeans.this.type

    Permalink

    Set the number of clusters.

    Set the number of clusters.

    Annotations
    @Since( "1.2.0" )

  41. def setRandomCenters(dim: Int, weight: Double, seed: Long = Utils.random.nextLong): StreamingKMeans.this.type

    Permalink

    Initialize random centers, requiring only the number of dimensions.

    Initialize random centers, requiring only the number of dimensions.

    dim

    Number of dimensions

    weight

    Weight for each center

    seed

    Random seed

    Annotations
    @Since( "1.2.0" )

  42. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  43. var timeUnit: String

    Permalink
    Annotations
    @Since( "1.2.0" )

  44. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any

  45. def trainOn(data: JavaDStream[Vector]): Unit

    Permalink

    Java-friendly version of trainOn.

    Java-friendly version of trainOn.

    Annotations
    @Since( "1.4.0" )

  46. def trainOn(data: DStream[Vector]): Unit

    Permalink

    Update the clustering model by training on batches of data from a DStream.

    Update the clustering model by training on batches of data from a DStream. This operation registers a DStream for training the model, checks whether the cluster centers have been initialized, and updates the model using each batch of data from the stream.

    data

    DStream containing vector data

    Annotations
    @Since( "1.2.0" )

  47. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  48. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  49. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped