Text Descriptors Customisation for Text-specific Metrics and Preset

Updates:

1. Added text descriptor TriggerWordsPresence
2. Added an option to specify which text descriptors should be used in the following metrics:

• TextDescriptorsDriftMetric
• TextDescriptorsDistribution
• TextDescriptorsCorrelationMetric

3. Added an option to specify which text descriptors should be used in the TextOverviewPreset

Fixes:
#512
#519
#520
#521
#522

Fix for Correlation Tests

Updates:
Instead of using just Pearson correlation, the following tests automatically choose the correlation method depending on target and prediction types:

• TestTargetPredictionCorrelation
• TestTargetFeaturesCorrelations
• TestPredictionFeaturesCorrelations

Fixes:
#446
#499
#504
#505
#511

Compatibility with Python 3.11

Updated project build to be compatible with python 3.11

Conda-compatible project build

Conda-compatible project build: prebuilt UI is stored in the repo; there is no need to build it locally.

Text data support in Reports and TestSuites

Breaking Changes:

• Python 3.6 is no longer supported

Updates:

• New parameter “text_features” was added to ColumnMapping. text_features parameter takes a list with feature names: “column_mapping.text_features=[’text_feature_1’, ‘text_feature_2’, …, ‘text_feature_k’]”
• The following metrics now support text features:
  • DatasetSummaryMetric()
  • DatasetMissingValuesMetric()
  • ColumnSummaryMetric(column_name="name")
  • ColumnMissingValuesMetric(column_name="name")
  • ColumnRegExpMetric(column_name="name", reg_exp=r".ticket.")
  • ConflictPredictionMetric()
  • ConflictTargetMetric()
  • DatasetCorrelationsMetric()
  • DatasetDriftMetric()
  • DataDriftTable()
  • ColumnDriftMetric(column_name=”name”)
  • TargetByFeaturesTable(columns=columns)
  • ClassificationQualityByFeatureTable()
  • RegressionErrorBiasTable()
• All metric presets now support text features
• All tests based on metrics that support text features also support texts
• The following test presets now support text features:
  • NoTargetPerformanceTestPreset
  • DataStabilityTestPreset
  • DataQualityTestPreset
  • DataDriftTestPreset
• Added metric TextDescriptorsDriftMetric for text data
• Added metric TextDescriptorsDistribution for text data
• Added metric TextDescriptorsCorrelationMetric for text data
• Added TextOverviewPreset(column_name=”name”) for text data. The preset includes:
  • ColumnSummaryMetric
  • TextDescriptorsDistribution
  • TextDescriptorsCorrelation
  • ColumnDriftMetric (if reference dataset is provided)
  • TextDescriptorsDriftMetric (if reference dataset is provided)

Changes:

• Method get_parameters(self) -> Optional[tuple] from Metric(Generic[TResult]) class was updated and became optional. The algorithm to determine metric parameters and create a tuple with metric parameters and its values is updated to cover more parameter types. Since the algorithm has been significantly updated, get_parameters method should be implemented in a custom metric class only if specific behaviour is needed. For most standard scenarios, the default version is sufficient.
• Metric deduplication now includes not only metric calculation parameters but features as well. This helps to deduplicate calculations of the same metrics on top of the same features and as a result, reduce calculation time.

Fixes:

• #484
• #490

Minor fixes for Metrics, Tests and Presets

Updates:
Added Maximum-Mean-Discrepancy (MMD) test #383
Added an example of the integration with Metaflow #468

Fixes:
#463
#471
#472
#473
#474
#476

Stable Metrics, Tests and Presets

Breaking Changes:

NOTE: Dashboards, Profiles, Tabs and Profile Sections are now DEPRECATED and will be completely REMOVED in the nearest releases.

Deleted NumTargetDriftPreset (use TargetDriftPreset instead)
Deleted CatTargetDriftPreset (use TargetDriftPreset instead)

Renamed Parameters:

• classification_threshold -> probas_threshold
  this afects:
  ClassificationQualityMetric , TestAccuracyScore, TestPrecisionScore, TestRecallScore, TestF1Score, TestTPR, TestTNR, TestFPR, TestFNR, TestPrecisionByClass, TestRecallByClass, TestF1ByClass, ClassificationPreset, BinaryClassificationTestPreset

• threshold-> stattest_threshold
  this afects:
  ColumnDriftMetric, TestColumnValueDrift, BinaryClassificationTestPreset, BinaryClassificationTopKTestPreset, MulticlassClassificationTestPreset

• all_features_stattest  -> stattest & all_features_threshold -> stattest_threshold
  this afects:
  DataDriftTable, DatasetDriftMetric, TestNumberOfDriftedColumns, TestShareOfDriftedColumns, DataDriftPreset, TargetDriftPreset, DataDriftTestPreset, NoTargetPerformanceTestPreset

• cat_features_stattest -> cat_stattest  & cat_features_threshold -> cat_stattest_threshold 
  this afects:
  DataDriftTable, DatasetDriftMetric, TestNumberOfDriftedColumns, TestShareOfDriftedColumns, DataDriftPreset, TargetDriftPreset, DataDriftTestPreset, NoTargetPerformanceTestPreset

• num_features_stattest -> num_stattest & num_features_stattest -> num_stattest_threshold 
  this afects:
  DataDriftTable, DatasetDriftMetric, TestNumberOfDriftedColumns, TestShareOfDriftedColumns, DataDriftPreset, TargetDriftPreset, DataDriftTestPreset, NoTargetPerformanceTestPreset

• per_feature_stattest -> per_column_stattest & per_feature_stattest -> per_column_stattest_threshold
  this afects:
  DataDriftTable, DatasetDriftMetric, TestNumberOfDriftedColumns, TestShareOfDriftedColumns, DataDriftPreset, TargetDriftPreset, DataDriftTestPreset, NoTargetPerformanceTestPreset

Renamed Tests:

• TestColumnValueDrift  -> TestColumnDrift
• TestColumnValueRegExp -> TestColumnRegExp 
• TestValueQuantile -> TestColumnQuantile

Updates:

Added top_error parameter to RegressionErrorBiasTable metric #422
Added ClassificationDummyMetric metric #445
Added RegressionDummyMetric metric #445
Added ConflictPredictionMetric metric #455
Added ConflictTargetMetric metric #455

Added API reference DRAFT https://docs.evidentlyai.com/reference/api-reference

Added new Statistical Tests:

• Apps-Singleton test #363
• Total Variation Distance (TVD) #391

Fixes:

#431
#438
#451
#458

Fixes for Metrics and Metric Presets

Breaking Changes:
Metrics Rename:
ClassificationQuality -> ClassificationQualityMetric
ProbabilityDistribution -> ClassificationProbDistribution

Tests Rename:
TestHighlyCorrelatedFeatures -> TestHighlyCorrelatedColumns
TestFeatureValueMin -> TestColumnValueMin
TestFeatureValueMax -> TestColumnValueMax
TestFeatureValueMean -> TestColumnValueMean
TestFeatureValueMedian -> TestColumnValueMedian
TestFeatureValueStd -> TestColumnValueStd
TestNumberOfDriftedFeatures -> TestNumberOfDriftedColumns
TestShareOfDriftedFeatures -> TestShareOfDriftedColumns
TestFeatureValueDrift -> TestColumnValueDrift

Updates:
#400
#373

Fixes:
#371
#421
#419
#418
#417
#416
#415

Metrics & Metric Presets

Breaking Changes:

All Test Presets were renamed.
TestPreset suffix was added to original names:

• NoTargetPerformance -> NoTargetPerformanceTestPreset
• DataQuality -> DataQualityTestPreset
• DataStability -> DataStabilityTestPreset
• DataDrift -> DataDriftTestPreset
• Regression -> RegressionTestPreset
• MulticlassClassification -> MulticlassClassificationTestPreset
• BinaryClassificationTopK -> BinaryClassificationTopKTestPreset
• BinaryClassification -> BinaryClassificationTestPreset

Updates:

Added DataDrift metrics:

• DatasetDriftMetric
• DataDriftTable
• ColumnValuePlot
• TargetByFeaturesTable

Added DataQuality metrics:

• ColumnDistributionMetric
• ColumnQuantileMetric
• ColumnCorrelationsMetric
• ColumnValueListMetric
• ColumnValueRangeMetric
• DatasetCorrelationsMetric

Added DataIntegrity metrics:

• ColumnSummaryMetric
• ColumnMissingValuesMetric
• DatasetSummaryMetric
• DatasetMissingValuesMetric

Added Classification metrics:

• ClassificationQuality
• ClassificationClassBalance
• ClassificationConfusionMatrix
• ClassificationQualityByClass
• ClassificationClassSeparationPlot
• ProbabilityDistribution
• ClassificationRocCurve
• ClassificationPRCurve
• ClassificationPRTable
• ClassificationQualityByFeatureTable

Added Regression metrics:

• RegressionQualityMetric
• RegressionPredictedVsActualScatter
• RegressionPredictedVsActualPlot
• RegressionErrorPlot
• RegressionAbsPercentageErrorPlot
• RegressionErrorDistribution
• RegressionErrorNormality
• RegressionTopErrorMetric
• RegressionErrorBiasTable

Added MetricPresets:

• DataDriftPreset
• DataQualityPreset
• RegressionPreset
• ClassificationPreset

Added New Statistical Tests

• Anderson-Darling test for numerical features
• Cramer Von Mises test for numerical features
• Hellinger distance test for numerical and categorical features
• Mann-Whitney U-rank test for numerical features
• Cressie-Read power divergence test for categorical features

Fixes:
#334
#353
#361
#367

Metrics Generator & Code Checks

Updates:

• Replaced BaseWidgetInfo with helpers: #326
• Added metrics generator for column-based metrics: #323
• Added black and isort: #322

Fixes:

• #340
• #341
• #336