# How to Gain Control of MLOps Processes[¶](https://knowledge.dataiku.com/latest/courses/mlops/prod-concepts/process-governance.html#how-to-gain-control-of-mlops-processes "Permalink to this headline")

## Introduction[¶](https://knowledge.dataiku.com/latest/courses/mlops/prod-concepts/process-governance.html#introduction "Permalink to this headline")

Controlling your models is one of the keys to the successful management and effectiveness of MLOps. Without this control, an organization risks damage caused by an inadvertently harmful model.

Although steps in the MLOps process might be difficult to formalize, the outcome is worthwhile because it provides visibility into progress tracking and decision making.

Let’s look at three categories of model control and governance:

* Audit and documentation

* Human-in-the-loop

* Pre-production verification

### Audit and Documentation[¶](https://knowledge.dataiku.com/latest/courses/mlops/prod-concepts/process-governance.html#audit-and-documentation "Permalink to this headline")

Both internal and external stakeholders will want to be able to ask questions about our deployed models, including what experiments were conducted and why each decision was made. To meet these needs, we should consider keeping a full log of all changes made during development.

Some ways to accomplish this include:

* Create an audit trail. Dataiku includes an audit trail that logs all actions performed by users.

* Create manual documentation, such as in a Wiki.

* Use project version control. In Dataiku, each change we make is automatically recorded in the Git repository.

* Configure audit and query logging for API services in production.

Note

The Model Document Generator automatically generates documentation associated with any trained model. To learn more about workflow documentation, including the Model Document Generator, visit the Knowledge Base article, Workflow Documentation.

### Human in the Loop[¶](https://knowledge.dataiku.com/latest/courses/mlops/prod-concepts/process-governance.html#human-in-the-loop "Permalink to this headline")

To scale our AI projects, we need people and automation. We need machines to allow people to work faster. We need humans in the loop to make sure our AI projects align with our human values and do what we expect so that we continue to trust the output.

One example of keeping humans in the loop is to require sign-offs on model versions before deploying the model from one environment to another, such as from development to test or from test to production.

Here are some ways we can implement human-in-the-loop interactions to avoid pitfalls:

* Document and revisit the goals of the project.

* Incorporate responsible and explainable AI strategies. For example, use interactive visual statistics, individual prediction explanations, and model fairness reports

* Monitor the model for drift

* Run “what-if” scenarios (interactive scoring).

* Perform model evaluation and model validation throughout the MLOps process.

* Implement automated alerts and notifications.

* Implement model version sign-offs.

Note

Enterprise customers can take advantage of Dataiku’s Govern node which allows you to prevent deployment of projects or models without a signoff.

### Preproduction Verification[¶](https://knowledge.dataiku.com/latest/courses/mlops/prod-concepts/process-governance.html#preproduction-verification "Permalink to this headline")

Preproduction verification involves the ways that we can validate our ML models before deploying them into production. Dataiku offers numerous capabilities for implementing preproduction verification in our MLOps process. We can take advantage of these capabilities to:

* Create metrics and checks

* Perform model evaluations

* Compare model evaluations by running model comparisons

* Design models in a development environment and deploy to a production environment (using production deployments and bundles)

Works Cited

Mark Treveil and the Dataiku team. Introducing MLOps: How to Scale Machine Learning in the Enterprise. O’Reilly, 2020.
