# Concept: Model Lifecycle Management[¶](https://knowledge.dataiku.com/latest/courses/scoring/model-lifecycle/model-mgmt-summary.html#concept-model-lifecycle-management "Permalink to this headline")

In Machine Learning Basics and Scoring Basics, we trained models to predict readmission for hospital patients based on historic patient data. We then deployed a model to the Flow and used the Score recipe to apply the model predictions to new, unseen patient records.

## Active Version of the Model[¶](https://knowledge.dataiku.com/latest/courses/scoring/model-lifecycle/model-mgmt-summary.html#active-version-of-the-model "Permalink to this headline")

The model deployed to the Flow is our first and **active version** of the model, which means that it is the version used when running the Retrain, Score, or Evaluate recipes.

The idea of which model is the “active version” is important because data science projects are highly iterative. Over time, your projects and models will evolve as new labelled data becomes available, or as you come up with new feature engineering strategies for your training data.

Whether you have a batch of new labelled records or are ready to try a new feature strategy, you will need to replace the deployed model with a new, updated version.

In our sample use case, Hospital Readmission, new patients continue to arrive at the hospital. Some are readmitted, and some never return. It is possible that the distributions and relationships between features and the target in the data change over time. Maybe the hospital has changed policies? Maybe a pandemic is keeping patients away? To prevent the model from becoming stale, it is good practice to retrain the model regularly when fresh training data becomes available.

## A Validation Set[¶](https://knowledge.dataiku.com/latest/courses/scoring/model-lifecycle/model-mgmt-summary.html#a-validation-set "Permalink to this headline")

How can we design a Flow that lets us manage the model lifecycle? One way is to adjust the Split recipe to carve out a validation set from the labelled training data.

A **validation set** gives us an unbiased evaluation of model fit.

## Model Retraining[¶](https://knowledge.dataiku.com/latest/courses/scoring/model-lifecycle/model-mgmt-summary.html#model-retraining "Permalink to this headline")

To retrain our model, we’ll run the Train recipe again and train the model on the new, slightly smaller training data that excludes all records in the validation set. To help automate the retraining of our model, we could create a **scenario**. For more information about scenarios, visit Scenarios in the knowledge base, or Automation scenarios in the reference documentation.

Note

We can also retrain the model by selecting the saved model and choosing “Retrain” from the available actions.

After completion of the model training, we can see the new version of the model by clicking on the saved model in the Flow. During retraining, all of the settings in the model design–the algorithm, hyperparameters, and feature handling–will be kept identical to the active model. But the model will be fitted to the new data.

By default, the retrained model becomes the new, active version. We could also change settings to require manual activation of any new model version. From this page, we can activate a new version of the model or roll back to a previous version of the model.

## The Evaluate Recipe[¶](https://knowledge.dataiku.com/latest/courses/scoring/model-lifecycle/model-mgmt-summary.html#the-evaluate-recipe "Permalink to this headline")

But how do we know if the new version of the model is better than the previous one? That’s a job for the **Evaluate** recipe.

As we update the active version of the model, we’ll be able to compare a model’s performance against previous versions using data that was never seen during training.

The Evaluate recipe requires two inputs: a labelled dataset and a deployed prediction model. It produces two outputs: the scored input dataset and another dataset of model metrics.

The scored validation set (*validation\_scored*) produced by the Evaluate recipe is similar to the output of the Score recipe. The “Readmitted” variable was already present, but we can now compare it against the class probabilities and predictions according to the active version of the model.

After running the Evaluate recipe one time, we have only one row in the model metrics dataset. After each update to the active version of the model, we’ll re-run the Evaluate recipe and see a new row of metrics added.

## Retraining for New Historic Data[¶](https://knowledge.dataiku.com/latest/courses/scoring/model-lifecycle/model-mgmt-summary.html#retraining-for-new-historic-data "Permalink to this headline")

With this evaluation plan in place, consider a case sometime in the near future where our historic data has changed. After it travels through our data pipeline, we have new, labelled training and validation sets recording which patients were readmitted.

Once again, let’s retrain the model on the new training data.

The model we just trained on our new data is now the active version of the model. ROC AUC has slightly improved, but we can get a fairer assessment against the validation set.

With a fresh active version of the model, let’s re-run the Evaluate recipe.

We have a new row in the model metrics dataset recording the performance on the newly trained model. The metrics are quite close. It looks like the model is performing the same on the new labelled data as it did on the older data.

Perhaps we should keep this model as the active version? If not, we can always roll back to a previous version.

## What’s Next?[¶](https://knowledge.dataiku.com/latest/courses/scoring/model-lifecycle/model-mgmt-summary.html#whats-next "Permalink to this headline")

In Scoring Basics, you’ve seen how to deploy a model to the Flow to predict new data.

You’ve also seen how Dataiku lets you version your model in order to keep improving it over time, while keeping track of its history.

In Automating the Model Lifecycle, you can learn about working with the model lifecycle in greater detail, including how to automate this cycle with scenarios, metrics, and checks.
