Package

com.salesforce.op

evaluators

Permalink

package evaluators

Visibility
  1. Public
  2. All

Type Members

  1. case class BinaryClassificationBinMetrics(BrierScore: Double, binCenters: Seq[Double], numberOfDataPoints: Seq[Long], averageScore: Seq[Double], averageConversionRate: Seq[Double]) extends EvaluationMetrics with Product with Serializable

    Permalink

    Metrics of BinaryClassificationBinMetrics

    Metrics of BinaryClassificationBinMetrics

    BrierScore

    brier score for overall dataset

    binCenters

    center of each bin

    numberOfDataPoints

    total number of data points in each bin

    averageScore

    average score in each bin

    averageConversionRate

    average conversion rate in each bin

  2. case class BinaryClassificationMetrics(Precision: Double, Recall: Double, F1: Double, AuROC: Double, AuPR: Double, Error: Double, TP: Double, TN: Double, FP: Double, FN: Double, thresholds: Seq[Double], precisionByThreshold: Seq[Double], recallByThreshold: Seq[Double], falsePositiveRateByThreshold: Seq[Double]) extends EvaluationMetrics with Product with Serializable

    Permalink

    Metrics of Binary Classification Problem

  3. sealed abstract class ClassificationEvalMetric extends EnumEntry with EvalMetric

    Permalink

    Classification Metrics

  4. sealed trait EvalMetric extends EnumEntry with Serializable

    Permalink

    Eval metric

  5. trait EvaluationMetrics extends JsonLike

    Permalink

    Trait for all different kinds of evaluation metrics

  6. case class MultiClassificationMetrics(Precision: Double, Recall: Double, F1: Double, Error: Double, ThresholdMetrics: ThresholdMetrics) extends EvaluationMetrics with Product with Serializable

    Permalink

    Metrics of MultiClassification Problem

  7. case class MultiMetrics(metrics: Map[String, EvaluationMetrics]) extends EvaluationMetrics with Product with Serializable

    Permalink

    A container for multiple evaluation metrics for evaluators

    A container for multiple evaluation metrics for evaluators

    metrics

    map of evaluation metrics

  8. abstract class OpBinaryClassificationEvaluatorBase[T <: EvaluationMetrics] extends OpClassificationEvaluatorBase[T]

    Permalink

    Base Interface for OpBinaryClassificationEvaluator

  9. abstract class OpEvaluatorBase[T <: EvaluationMetrics] extends Evaluator with OpHasLabelCol[RealNN] with OpHasPredictionValueCol[RealNN] with OpHasPredictionCol

    Permalink

    Base Interface for OpEvaluator to be used in Evaluator creation.

    Base Interface for OpEvaluator to be used in Evaluator creation. Can be used for both OP and spark eval (so with workflows and cross validation).

  10. sealed abstract class OpEvaluatorNames extends EnumEntry with EvalMetric

    Permalink

    GeneralMetrics

  11. trait OpHasLabelCol[T <: FeatureType] extends Params

    Permalink

    Trait for labelCol param

  12. trait OpHasPredictionCol extends Params

    Permalink

    Trait for predictionCol which contains all output results param

  13. trait OpHasPredictionValueCol[T <: FeatureType] extends Params

    Permalink

    Trait for internal flattened predictionCol param

  14. trait OpHasProbabilityCol[T <: FeatureType] extends Params

    Permalink

    Trait for internal flattened probabilityCol Param

  15. trait OpHasRawPredictionCol[T <: FeatureType] extends Params

    Permalink

    Trait for internal flattened rawPredictionColParam

  16. abstract class OpMultiClassificationEvaluatorBase[T <: EvaluationMetrics] extends OpClassificationEvaluatorBase[T]

    Permalink

    Base Interface for OpMultiClassificationEvaluator

  17. abstract class OpRegressionEvaluatorBase[T <: EvaluationMetrics] extends OpEvaluatorBase[T]

    Permalink

    Base Interface for OpRegressionEvaluator

  18. sealed abstract class RegressionEvalMetric extends EnumEntry with EvalMetric

    Permalink

    Regression Metrics

  19. case class RegressionMetrics(RootMeanSquaredError: Double, MeanSquaredError: Double, R2: Double, MeanAbsoluteError: Double) extends EvaluationMetrics with Product with Serializable

    Permalink

    Metrics of Regression Problem

  20. case class SingleMetric(name: String, value: Double) extends EvaluationMetrics with Product with Serializable

    Permalink

    A container for a single evaluation metric for evaluators

    A container for a single evaluation metric for evaluators

    name

    metric name

    value

    metric value

  21. case class ThresholdMetrics(topNs: Seq[Int], thresholds: Seq[Double], correctCounts: Map[Int, Seq[Long]], incorrectCounts: Map[Int, Seq[Long]], noPredictionCounts: Map[Int, Seq[Long]]) extends EvaluationMetrics with Product with Serializable

    Permalink

    Threshold-based metrics for multiclass classification

    Threshold-based metrics for multiclass classification

    Classifications being correct, incorrect, or no classification are defined in terms of the topN and score threshold to be: Correct - score of the true label is in the top N scores AND the score of the true label is >= threshold Incorrect - score of top predicted label >= threshold AND (true label NOT in top N predicted labels OR score of true label < threshold) No prediction - otherwise (score of top predicted label < threshold)

    topNs

    list of topN values (used as keys for the count maps)

    thresholds

    list of threshold values (correspond to thresholds at the indices of the arrays in the count maps)

    correctCounts

    map from topN value to an array of counts of correct classifications at each threshold

    incorrectCounts

    map from topN value to an array of counts of incorrect classifications at each threshold

    noPredictionCounts

    map from topN value to an array of counts of no prediction at each threshold

Value Members

  1. object BinaryClassEvalMetrics extends Enum[ClassificationEvalMetric]

    Permalink

    Binary Classification Metrics

  2. object EvalMetric extends Serializable

    Permalink

    Eval metric companion object

  3. object Evaluators

    Permalink

    Just a handy factory for evaluators

  4. object MultiClassEvalMetrics extends Enum[ClassificationEvalMetric]

    Permalink

    Multi Classification Metrics

  5. object OpEvaluatorNames extends Enum[OpEvaluatorNames] with Serializable

    Permalink

    Contains evaluator names used in logging

  6. object RegressionEvalMetrics extends Enum[RegressionEvalMetric]

    Permalink

    Regression Metrics

Ungrouped