op

Type Members

1. case class Continuous(min: Double, max: Double, mean: Double, variance: Double) extends LabelInfo with Product with Serializable

   

   Summary of label distribution for continuous label

   Summary of label distribution for continuous label

   min

   min value

   max

   max value

   mean

   mean value

   variance

   variance of values

2. case class Discrete(domain: Seq[String], prob: Seq[Double]) extends LabelInfo with Product with Serializable

   

   Summary of label distribution for discrete label

   Summary of label distribution for discrete label

   domain

   sequence of all unique values observed in data

   prob

   probabilities of each unique value observed in data (order is matched to domain order)

3. class EvaluateResult extends OpWorkflowRunnerResult

   

4. case class FeatureHistory(originFeatures: Seq[String], stages: Seq[String]) extends JsonLike with Product with Serializable

   

   History of all stages and origin features used to create a given feature

   History of all stages and origin features used to create a given feature

   originFeatures

   alphabetically ordered names of the raw features this feature was created from

   stages

   sequence of the stageNames applied

5. case class FeatureInsights(featureName: String, featureType: String, derivedFeatures: Seq[Insights], metrics: Seq[RawFeatureFilterMetrics] = Seq.empty, distributions: Seq[FeatureDistribution] = Seq.empty, exclusionReasons: Seq[ExclusionReasons] = Seq.empty, sensitiveInformation: Seq[SensitiveFeatureInformation] = Seq.empty) extends Product with Serializable

   

   Summary of feature insights for all features derived from a given input (raw) feature

   Summary of feature insights for all features derived from a given input (raw) feature

   featureName

   name of raw feature insights are about

   featureType

   type of raw feature insights are about

   derivedFeatures

   sequence containing insights for each feature derived from the raw feature

   metrics

   sequence containing metrics computed in RawFeatureFilter

   distributions

   distribution information for the raw feature (if calculated in RawFeatureFilter)

   exclusionReasons

   exclusion reasons for the raw feature (if calculated in RawFeatureFilter)

   sensitiveInformation

   derived information about sensitive field checks (if performed)

6. class FeaturesResult extends OpWorkflowRunnerResult

   

7. case class GenderDetectionResults(strategyString: String, pctUnidentified: Double) extends JsonLike with Product with Serializable

   

8. case class Insights(derivedFeatureName: String, stagesApplied: Seq[String], derivedFeatureGroup: Option[String], derivedFeatureValue: Option[String], excluded: Option[Boolean] = None, corr: Option[Double] = None, cramersV: Option[Double] = None, mutualInformation: Option[Double] = None, pointwiseMutualInformation: Map[String, Double] = Map.empty, countMatrix: Map[String, Double] = Map.empty, contribution: Seq[Double] = Seq.empty, min: Option[Double] = None, max: Option[Double] = None, mean: Option[Double] = None, variance: Option[Double] = None) extends Product with Serializable

   

   Summary of insights for a derived feature

   Summary of insights for a derived feature

   derivedFeatureName

   name of derived feature

   stagesApplied

   the stageNames of all stages applied to make feature from the raw input feature

   derivedFeatureGroup

   grouping of this feature if the feature is a pivot

   derivedFeatureValue

   value of the feature if the feature is a numeric encoding of a non-numeric feature or bucket

   excluded

   was this derived feature excluded from the model by the sanity checker

   corr

   the correlation of this feature with the label

   cramersV

   the cramersV of this feature with the label (when both label and feature are categorical)

   mutualInformation

   the mutual information for this feature (and all features in its grouping) with the label (categorical features only)

   pointwiseMutualInformation

   the mutual information of this feature with each value of the label (categorical features only)

   countMatrix

   the counts of the occurrence of this feature with each of the label values (categorical features only)

   contribution

   the contribution of this feature to the model (eg feature importance for random forest, weight for logistic regression)

   min

   the min value of this feature

   max

   the max value of this feature

   mean

   the mean value of this feature

   variance

   the variance of this feature

9. trait LabelInfo extends AnyRef

   

   Common trait for Continuous and Discrete

10. case class LabelSummary(labelName: Option[String] = None, rawFeatureName: Seq[String] = Seq.empty, rawFeatureType: Seq[String] = Seq.empty, stagesApplied: Seq[String] = Seq.empty, sampleSize: Option[Double] = None, distribution: Option[LabelInfo] = None) extends Product with Serializable

    

    Summary information about label used in model creation (all fields will be empty if no label is found)

    Summary information about label used in model creation (all fields will be empty if no label is found)

    labelName

    name of label feature

    rawFeatureName

    name of raw features that label is derived from

    rawFeatureType

    types of raw features that label is derived from

    stagesApplied

    the stageNames of all stages applied to label before modeling

    sampleSize

    count of label used to compute distribution information (will be fraction of data corresponding to sample rate in sanity checker)

    distribution

    summary of label distribution (either continuous or discrete)

11. case class ModelInsights(label: LabelSummary, features: Seq[FeatureInsights], selectedModelInfo: Option[ModelSelectorSummary], trainingParams: OpParams, stageInfo: Map[String, Any]) extends Product with Serializable

    

    Summary of all model insights

    Summary of all model insights

    label

    summary of information about the label

    features

    sequence containing insights for each raw feature that fed into the model

    selectedModelInfo

    summary information about model training and winning model from model selector

    trainingParams

    op parameters used in model training

    stageInfo

    all stages and their parameters settings used to create feature output of model keyed by stageName

12. abstract class OpApp extends AnyRef

    

    A simple command line app for running an OpWorkflow with Spark.

    A simple command line app for running an OpWorkflow with Spark. A user needs to implement a run function.

13. abstract class OpAppWithRunner extends OpApp

    

    A simple command line app for running an OpWorkflow with Spark.

    A simple command line app for running an OpWorkflow with Spark. A user needs to implement a runner creation function.

14. final class OpParams extends JsonLike

    

    OpParams for passing in command line information

15. class OpWorkflow extends OpWorkflowCore

    

    Workflow for TransmogrifAI.

    Workflow for TransmogrifAI. Takes the final features that the user wants to generate as inputs and constructs the full DAG needed to generate them from those features lineage. Then fits any estimators in the pipeline dag to create a sequence of transformations that are saved in a workflow model.

16. class OpWorkflowModel extends OpWorkflowCore

    

    Workflow model is a container and executor for the sequence of transformations that have been fit to the data to produce the desired output features

17. class OpWorkflowModelReader extends AnyRef

    

    Reads OpWorkflowModelWriter serialized OpWorkflowModel objects by path and JValue.

    Reads OpWorkflowModelWriter serialized OpWorkflowModel objects by path and JValue. This will only work if the features were serialized in topological order. NOTE: The FeatureGeneratorStages will not be recovered into the Model object, because they are part of each feature.

18. class OpWorkflowModelWriter extends MLWriter

    

    Writes the OpWorkflowModel to json format.

    Writes the OpWorkflowModel to json format. For now we will not serialize the parent of the model

    Note

    The features/stages must be sorted in topological order

19. sealed trait OpWorkflowRunType extends EnumEntry with Serializable

    

20. class OpWorkflowRunner extends Serializable

    

    A class for running an TransmogrifAI Workflow.

    A class for running an TransmogrifAI Workflow. Provides methods to train, score, evaluate and computeUpTo for TransmogrifAI Workflow.

21. case class OpWorkflowRunnerConfig(runType: OpWorkflowRunType = null, paramLocation: Option[String] = None, defaultParams: OpParams = new OpParams(), readLocations: Map[String, String] = Map.empty, writeLocation: Option[String] = None, modelLocation: Option[String] = None, metricsLocation: Option[String] = None) extends JsonLike with Product with Serializable

    

    OpWorkflowRunner configuration container

    OpWorkflowRunner configuration container

    runType

    workflow run type

    paramLocation

    workflow file params location

    defaultParams

    default params to use in case the file params is missing

    readLocations

    read locations

    writeLocation

    write location

    modelLocation

    model location

    metricsLocation

    metrics location

22. trait OpWorkflowRunnerResult extends Serializable

    

23. final class ReaderParams extends JsonLike

    

    Reader params

24. sealed trait ResultFeatureRetention extends EnumEntry with Serializable

    

    Methods of vectorizing text (eg.

    Methods of vectorizing text (eg. to be chosen by statistics computed in SmartTextVectorizer)

25. implicit class RichAnyConcreteFeaturesArray extends RichAnyFeaturesCollection

    

    Enrichment functions for an array of arbitrary features

    Enrichment functions for an array of arbitrary features

    Definition Classes

    RichFeaturesCollection

26. implicit class RichAnyFeaturesArray extends RichAnyFeaturesCollection

    

    Enrichment functions for an array of arbitrary features

    Enrichment functions for an array of arbitrary features

    Definition Classes

    RichFeaturesCollection

27. implicit class RichAnyFeaturesCollection extends AnyRef

    

    Enrichment functions for a collection of arbitrary features

    Enrichment functions for a collection of arbitrary features

    Definition Classes

    RichFeaturesCollection

28. implicit class RichBase64Feature extends AnyRef

    

    Definition Classes

    RichTextFeature

29. implicit class RichBase64MapFeature extends AnyRef

    

    Enrichment functions for Base64Map features.

    Enrichment functions for Base64Map features.

    Definition Classes

    RichMapFeature

30. implicit class RichBinaryFeature extends AnyRef

    

    Enrichment functions for Binary Feature

    Enrichment functions for Binary Feature

    Definition Classes

    RichNumericFeature

31. implicit class RichBooleanMapFeature extends AnyRef

    

    Enrichment functions for OPMap Features with Boolean values

    Enrichment functions for OPMap Features with Boolean values

    Definition Classes

    RichMapFeature

32. implicit class RichComboBoxFeature extends AnyRef

    

    Definition Classes

    RichTextFeature

33. implicit class RichDateFeature extends AnyRef

    

    Enrichment functions for Date Feature

    Enrichment functions for Date Feature

    Definition Classes

    RichDateFeature

34. implicit class RichDateListFeature extends AnyRef

    

    Enrichment functions for DateList Feature

    Enrichment functions for DateList Feature

    Definition Classes

    RichListFeature

35. implicit class RichDateMapFeature extends AnyRef

    

    Enrichment functions for OPMap Features with Date values

    Enrichment functions for OPMap Features with Date values

    Definition Classes

    RichMapFeature

36. implicit class RichDateTimeFeature extends AnyRef

    

    Enrichment functions for DateTime Feature

    Enrichment functions for DateTime Feature

    Definition Classes

    RichDateFeature

37. implicit class RichDateTimeListFeature extends AnyRef

    

    Enrichment functions for DateList Feature

    Enrichment functions for DateList Feature

    Definition Classes

    RichListFeature

38. implicit class RichDateTimeMapFeature extends AnyRef

    

    Enrichment functions for OPMap Features with DateTime values

    Enrichment functions for OPMap Features with DateTime values

    Definition Classes

    RichMapFeature

39. implicit class RichEmailFeature extends AnyRef

    

    Definition Classes

    RichTextFeature

40. implicit class RichEmailMapFeature extends AnyRef

    

    Enrichment functions for EmailMap Features

    Enrichment functions for EmailMap Features

    Definition Classes

    RichMapFeature

41. implicit class RichFeature[A <: FeatureType] extends AnyRef

    

    Enrichment functions for Feature[A]

    Enrichment functions for Feature[A]

    A

    input type

    Definition Classes

    RichFeature

42. implicit class RichGeolocationFeature extends AnyRef

    

    Enrichment functions for Geolocation Feature

    Enrichment functions for Geolocation Feature

    Definition Classes

    RichListFeature

43. implicit class RichGeolocationMapFeature extends AnyRef

    

    Enrichment functions for OPMap Features with Geolocation values

    Enrichment functions for OPMap Features with Geolocation values

    Definition Classes

    RichMapFeature

44. implicit class RichIdFeature extends AnyRef

    

    Definition Classes

    RichTextFeature

45. implicit class RichIntegralFeature[T <: Integral] extends AnyRef

    

    Enrichment functions for Integral Feature

    Enrichment functions for Integral Feature

    Definition Classes

    RichNumericFeature

46. implicit class RichIntegralMapFeature[T <: OPMap[Long]] extends AnyRef

    

    Enrichment functions for OPMap Features with Long values

    Enrichment functions for OPMap Features with Long values

    Definition Classes

    RichMapFeature

47. implicit class RichLocationFeature[T <: Text with Location] extends AnyRef

    

    Definition Classes

    RichLocationFeature

48. implicit class RichMapFeature[T <: OPMap[_]] extends AnyRef

    

    Enrichment functions for OPMap Features

    Enrichment functions for OPMap Features

    Definition Classes

    RichMapFeature

49. implicit class RichMultiPickListMapFeature[T <: OPMap[Set[String]]] extends AnyRef

    

    Enrichment functions for OPMap Features with String values

    Enrichment functions for OPMap Features with String values

    Definition Classes

    RichMapFeature

50. implicit class RichNumericFeature[I <: OPNumeric[_]] extends AnyRef

    

    Enrichment functions for Numeric Feature

    Enrichment functions for Numeric Feature

    I

    input type

    Definition Classes

    RichNumericFeature

51. implicit class RichOPSetFeature[T <: OPSet[_]] extends AnyRef

    

    Enrichment functions for OPSet Feature

    Enrichment functions for OPSet Feature

    Definition Classes

    RichSetFeature

52. implicit class RichPhoneFeature extends AnyRef

    

    Definition Classes

    RichTextFeature

53. implicit class RichPhoneMapFeature extends AnyRef

    

    Enrichment functions for PhoneMap features

    Enrichment functions for PhoneMap features

    Definition Classes

    RichMapFeature

54. implicit class RichPickListFeature extends AnyRef

    

    Definition Classes

    RichTextFeature

55. implicit class RichPredictionFeature extends AnyRef

    

    Enrichment functions for Prediction Features

    Enrichment functions for Prediction Features

    Definition Classes

    RichMapFeature

56. implicit class RichRealFeature[T <: Real] extends AnyRef

    

    Enrichment functions for Real Feature

    Enrichment functions for Real Feature

    Definition Classes

    RichNumericFeature

57. implicit class RichRealMapFeature[T <: OPMap[Double]] extends AnyRef

    

    Enrichment functions for OPMap Features with Double values

    Enrichment functions for OPMap Features with Double values

    Definition Classes

    RichMapFeature

58. implicit class RichRealNNFeature extends AnyRef

    

    Enrichment functions for Real non nullable Feature

    Enrichment functions for Real non nullable Feature

    Definition Classes

    RichNumericFeature

59. implicit class RichSetFeature extends AnyRef

    

    Enrichment functions for MultiPickList Feature

    Enrichment functions for MultiPickList Feature

    Definition Classes

    RichSetFeature

60. implicit class RichStringMapFeature[T <: OPMap[String]] extends AnyRef

    

    Enrichment functions for OPMap Features with String values.

    Enrichment functions for OPMap Features with String values. All are pivoted by default except TextMap and TextAreaMap which are defined specially below.

    Definition Classes

    RichMapFeature

61. implicit class RichTextAreaMapFeature extends AnyRef

    

    Enrichment functions for TextAreaMap Features (they are hashed by default instead of being pivoted)

    Enrichment functions for TextAreaMap Features (they are hashed by default instead of being pivoted)

    Definition Classes

    RichMapFeature

62. implicit class RichTextFeature[T <: Text] extends AnyRef

    

    Definition Classes

    RichTextFeature

63. implicit class RichTextListFeature extends AnyRef

    

    Enrichment functions for TextList Feature

    Enrichment functions for TextList Feature

    Definition Classes

    RichListFeature

64. implicit class RichTextMapFeature extends AnyRef

    

    Enrichment functions for TextMap Features (they are hashed by default instead of being pivoted)

    Enrichment functions for TextMap Features (they are hashed by default instead of being pivoted)

    Definition Classes

    RichMapFeature

65. implicit class RichURLFeature extends AnyRef

    

    Definition Classes

    RichTextFeature

66. implicit class RichURLMapFeature extends AnyRef

    

    Enrichment functions for URLMap Features

    Enrichment functions for URLMap Features

    Definition Classes

    RichMapFeature

67. implicit class RichVectorFeature extends AnyRef

    

    Enrichment functions for Vector Feature

    Enrichment functions for Vector Feature

    Definition Classes

    RichVectorFeature

68. implicit class RichVectorFeaturesCollection extends AnyRef

    

    Enrichment functions for a collection of vector features

    Enrichment functions for a collection of vector features

    Definition Classes

    RichFeaturesCollection

69. class ScoreResult extends OpWorkflowRunnerResult

    

70. sealed abstract class SensitiveFeatureInformation extends JsonLike

    

    A base class for different SensitiveFeatureInformation The following three params are required for every kind of SensitiveFeatureInformation

71. case class SensitiveNameInformation(probName: Double, genderDetectResults: Seq[GenderDetectionResults], probMale: Double, probFemale: Double, probOther: Double, name: String, key: Option[String] = None, actionTaken: Boolean = false) extends SensitiveFeatureInformation with Product with Serializable

    

72. class StreamingScoreResult extends OpWorkflowRunnerResult

    

73. class TrainResult extends OpWorkflowRunnerResult