specific

Type Members

1. class OpBinaryEstimatorWrapper[I1 <: FeatureType, I2 <: FeatureType, O <: FeatureType, E <: Estimator[M], M <: Model[M]] extends SwBinaryEstimator[I1, I2, O, M, E]

   

   Wraps a spark ML estimator.

   Wraps a spark ML estimator. This class is meant for Predictor-like Spark algorithms, but which don't inherit from Predictor, for whatever reason. Examples of which include: IsotonicRegression, AFTSurvivalRegression, OneVsRest. Their defining characteristic is that they output a model which takes in 2 columns as input (labels and features) and output one column as result, but don't inherit from Predictor (if it did, should use OpPredictorWrapper instead).

   I1

   first input feature type

   I2

   second input feature type

   O

   output feature type

   E

   spark estimator to wrap

   M

   spark model type returned by spark estimator wrapped

2. class OpEstimatorWrapper[I <: FeatureType, O <: FeatureType, E <: Estimator[M], M <: Model[M]] extends SwUnaryEstimator[I, O, M, E]

   

   Wraps a spark ML estimator.

   Wraps a spark ML estimator. This wrapper is meant for Estimators not already covered by more specific wrappers such as: OpPredictorWrapper. Examples of estimators meant to be wrapped with OpEstimatorWrapper include MinMaxScaler, IDF, VectorIndexer, CountVectorizer, QuantileDiscretizer, StandardScaler, PCA, MaxAbsScaler, Word2Vec, etc. Their defining characteristic is that they output a Model which takes in one column as input and output one column as a result.

   I

   the type of the transformation input feature

   O

   the type of the transformation output feature

   E

   spark estimator to wrap

   M

   spark model type returned by spark estimator wrapped

3. abstract class OpPredictionModel[T <: PredictionModel[Vector, T]] extends OpPredictorWrapperModel[T]

   

   Class that takes in a spark PredictionModel and wraps it into an OP model which returns a Prediction feature

   Class that takes in a spark PredictionModel and wraps it into an OP model which returns a Prediction feature

   T

   type of the model to wrap

4. class OpPredictorWrapper[E <: Predictor[Vector, E, M], M <: PredictionModel[Vector, M]] extends Estimator[OpPredictorWrapperModel[M]] with OpPipelineStage2[RealNN, OPVector, Prediction] with SparkWrapperParams[E]

   

   Wraps a spark ML predictor.

   Wraps a spark ML predictor. Predictors represent supervised learning algorithms (regression and classification) in spark ML that inherit from Predictor, supported models are: org.apache.spark.ml.classification.LogisticRegression org.apache.spark.ml.regression.LinearRegression, org.apache.spark.ml.classification.RandomForestClassifier, org.apache.spark.ml.regression.RandomForestRegressor, org.apache.spark.ml.classification.NaiveBayesModel, org.apache.spark.ml.classification.GBTClassifier, org.apache.spark.ml.regression.GBTRegressor, org.apache.spark.ml.classification.DecisionTreeClassifier org.apache.spark.ml.regression.DecisionTreeRegressor, org.apache.spark.ml.classification.LinearSVC org.apache.spark.ml.classification.MultilayerPerceptronClassifier, org.apache.spark.ml.regression.GeneralizedLinearRegression. Their defining characteristic is that they output a model which takes in 2 columns as input (labels and features) and output one to three column as result.

   E

   spark estimator to wrap

   M

   spark model returned

5. abstract class OpPredictorWrapperModel[M <: PredictionModel[Vector, M]] extends Model[OpPredictorWrapperModel[M]] with SparkWrapperParams[M] with OpTransformer2[RealNN, OPVector, Prediction]

   

6. abstract class OpProbabilisticClassifierModel[T <: ProbabilisticClassificationModel[Vector, T]] extends OpPredictorWrapperModel[T]

   

   Class that takes in a spark ProbabilisticClassifierModel and wraps it into an OP model which returns a Prediction feature

   Class that takes in a spark ProbabilisticClassifierModel and wraps it into an OP model which returns a Prediction feature

   T

   type of the model to wrap

7. class OpTransformerWrapper[I <: FeatureType, O <: FeatureType, T <: InOutTransformer] extends SwUnaryTransformer[I, O, T]

   

   Wraps a spark ML transformer with setable input and output columns.

   Wraps a spark ML transformer with setable input and output columns. Those transformers that fall in this case, include those that inherit from org.apache.spark.ml.UnaryEstimator, as well as others such as OneHotEncoder, org.apache.spark.ml.feature.Binarizer, org.apache.spark.ml.feature.VectorSlicer, org.apache.spark.ml.feature.HashingTF, org.apache.spark.ml.feature.StopWordsRemover, org.apache.spark.ml.feature.IndexToString, org.apache.spark.ml.feature.StringIndexer. Their defining characteristic is that they take one column as input, and output one column as result.

   I

   The type of the input feature

   O

   The type of the output feature (result of transformation)

   T

   type of spark transformer to wrap