Select dataset containing new test data
Computing ...
Input Data Drift
Get insights on the applicability of a model by exploring how a dataset of new data differs from the original evaluation (test) dataset
Global Drift Score
Sampling first rows per dataset
Drift model
Lower bound
Accuracy
Upper bound
0.45
≤
0.47
≤
0.6
Lower is better.
In order to detect data drift, we train a random forest classifier (the drift model) to discriminate the new data set from the test set. If this classifier has accuracy > 0.5, it implies that test data and new data can be distinguished and that you are observing data drift. You may consider retraining your model in that situation.
Binomial test
Hypothesis tested
There is no drift (accuracy ≤ 0.5)
Significance level
0.05
p-value
0.00020
Conclusion
≤ 0.05 so drift detected
The hypothesis tested is that there is no drift, in which case the expected drift model accuracy is 0.5 (datasets undistinguishable). The observed accuracy might deviate from this expectation and the Binomial test evaluates whether this deviation is statistically significant, modelling the number of correct predictions as a random variable drawn from a Binomial distribution.
The p-value is the probability to observe this particular accuracy (or larger) under the hypothesis of absent drift. If this probability is lower than the significance level (i.e. 5%), it’s then unlikely to be in the situation of absent drift: the hypothesis of no drift is rejected, triggering a drift detection. The significance level indicates the rate of falsely-detected drifts we are ready to accept from the test.
Model Information
Fugacity
Fugacity expresses the difference between the expected "ideal" data your model was trained on and the observed "real" data you are analyzing. We compare the proportion of samples predicted in each class when scoring on both the test and your input datasets.
Predicted probability density chart
This chart represents the probability density estimation for a given prediction class when scoring both the test dataset and your input dataset.
Visually different probability density estimations indicate high data drift.
Feature Drift Overview
The scatter plot shows feature importance for the original model versus feature importance for the
(data classifying) drift model.
This graph should be examined alongside with the drift score ().
For a highly drifted dataset (drift score ~1), if the features most responsible for data drift are of low importance
in the original model (bottom right quadrant), you can expect the behavior of the model to remain the same.
Features in the top right quadrant of this scatter plot are highly drifted (i.e. they are powerful in
distinguishing test data from new observations), but also of high importance for the original model.
In this situation, you can expect the performance of the model to degrade as your model does not apply
to your new observations.