# Overview

This project illustrates how Dataiku supports **MLflow model import** and **experiment tracking for code-based model training**.

**MLflow** is an open-source platform developed by Databricks that facilitates the end-to-end management of the machine learning lifecycle. It provides a set of APIs for tracking experiments, packaging code into reproducible runs, and sharing and deploying models across different environments. By combining the strengths of MLflow and Dataiku, organizations can accelerate the machine learning lifecycle, enhance collaboration, and drive innovation.

This project covers the following aspects:

- **Importing** an external MLflow model into Dataiku DSS.
- **Training** custom Python models in Dataiku DSS and tracking experiments for tasks such as tabular regression, classification and multi-output regression.
- **Evaluating** the performance of MLflow models.
- **Deploying** an MLflow Saved Model through an API service.

>  **Notes** : in this project, example notebooks are provided but not visible in the public gallery of projects. Please download the project to have access to these notebooks in  **</>**  >  **Notebooks** . 

The project can be downloaded [here](https://downloads.dataiku.com/public/dss-samples/EX_MLFLOW/).


# Data

Multiple datasets have been used to illustrate the different processes : 

 - For the sections [Import an external MLflow model](flow_zone:1R1sdxJ) and [Experiment Tracking for Tabular Classification](flow_zone:default), the data comes from [UCI](https://archive.ics.uci.edu/dataset/222/bank+marketing). [This dataset](dataset:input-tabular_classification), from a Portuguese bank's direct marketing campaigns, records the outcomes of phone calls made to potential clients. It includes various customer and campaign-related features. The goal is to **predict whether a client will subscribe to a term deposit (binary classification: yes or no)**.
 - For the section [Experiment Tracking for Tabular Pytorch Regression](flow_zone:sUozkQs), the dataset consists of 10 continuous input features labeled _feature_1_ to _feature_10_, along with a continuous target variable _y_. The goal is to **build a regression model to predict the target _y_ from the input features**.
 - For the section [Experiment Tracking for Multi-Output Regression \["Other" prediction type\]](flow_zone:glcaeNZ), [this script](recipe:compute_time_series_generated_data) generates a synthetic time series dataset with 1,000 daily observations. It includes three input features (feature1, feature2, feature3) and two continuous target variables (output1, output2) calculated as noisy linear combinations of the features. The goal of the prediction task is to **train a multi-output regression model to predict both output1 and output2 from the input features**.

# Walkthrough

The project is structured into **five** main parts. First, we cover how to import MLflow models. Next, we demonstrate training custom models in DSS while using MLflow for experiment tracking, across use cases like tabular classification, PyTorch regression, multi-output regression, computer vision, and LLMs. Third, we focus on model explainability. Fourth, we address model performance monitoring. Finally, we show how to deploy MLflow models as API endpoints.

## MLflow Models Import 

In this [section](flow_zone:1R1sdxJ), we assume the ML model has been trained outside DSS. The objective is to **import this external MLflow model into DSS** and **convert it into a Saved Model**, enabling the use of DSS features such as the Evaluate and Score recipes.

![import_mlflow.png](5UIQKKj2oF1g)

<div align=center> <i>Import and Score an External MLflow Model</i> </div>

 **Key steps** :

- A [managed folder](managed_folder:M8tpkIby) points to the directory containing the MLflow model.
- A [Python recipe](recipe:recipe_from_notebook_import_mlflow_models) converts this model into a [Saved Model](saved_model:00D6b8kG). A commented [Jupyter notebook](jupyter_notebook:Importing MLflow models in DSS) is also available.
- An [Evaluate recipe](recipe:evaluate_on_eval_data) computes metrics and drift on an evaluation dataset, outputting a [Model Evaluation Store](model_evaluation_store:1rzFDC1p) and [scored data](dataset:eval_data_evaluated).

> For detailed instructions, refer to the [documentation](https://doc.dataiku.com/dss/latest/mlops/mlflow-models/importing.html).

## MLflow Experiment Tracking

This section highlights how to **train custom models** not available in the Lab, and **track experiments** using the MLflow API. Each use case ends with a Saved Model and a performance evaluation.

In addition, please note that all the experiments are gathered in the  **Experiment Tracking UI** ( _ML tab > Experiment Tracking_ ). 

> For detailed instructions, refer to the [documentation](https://doc.dataiku.com/dss/latest/mlops/experiment-tracking/index.html).

### Tabular Classification

In this [section](flow_zone:default), we work with **marketing campaign data** (see Data) to perform a binary classification task.

![tabular_classification.png](VNqLt9vRpqRm)
<div align=center> <i>Train and Evaluate a Tabular Classification Model</i> </div>

 **Key components** :

- [Input dataset](dataset:input-tabular_classification)
- A [Python recipe](recipe:recipe_from_notebook_xptrackingsklearn) trains a scikit-learn model and tracks experiments with MLflow (see [documentation](https://doc.dataiku.com/dss/latest/mlops/experiment-tracking/tracking.html)). A commented [notebook](jupyter_notebook:Train a classification model on tabular data) is available.
- Outputs: a [Saved Model](saved_model:wO2sOCX4), an [experiments folder](managed_folder:0U5AmTf7), and a tracked experiment.
- Model evaluation is done via an [Evaluate recipe](recipe:evaluate_on_uci_bank_evaluate), results are saved in a [Model Evaluation Store](model_evaluation_store:08Bcoxsm).


### PyTorch Regression 

This [section](flow_zone:sUozkQs) demonstrates **regression using a custom PyTorch model** with experiment tracking.

![pytorch_regression.png](cvxyNg0T3Cjr)
<div align=center> <i>Train and Evaluate a PyTorch Regression Model</i> </div>

 **Key components** :

- [Input dataset](dataset:input-pytorch_regression) contains 10 continuous features and a continuous target variable.
- A [Split recipe](recipe:split_generated_data_dataset) creates train, validation, and test sets.
- A [Python recipe](recipe:recipe_from_notebook_Train_a_PyTorch_regression_model_1) trains a neural network, testing different hyperparameters. Outputs include a [Saved Model](saved_model:ajPPDEF1) and an [experiments folder](managed_folder:pshnkijv).
- A [Scoring recipe](recipe:score_test) generates predictions on test data.


### Multi Output Regression

In this [section](flow_zone:glcaeNZ), we explore **multi-output regression** on generated time series data.

![multi_output.png](RMN1Ao2C8SIN)
<div align=center> <i>Train and Evaluate a Multi-Output Time Series Regression Model</i> </div>

 **Key components** :

- [Input dataset](dataset:input-keras_multioutput_regression) includes a date variable, three features, and two target variables.
- A [Python recipe](recipe:compute_time_series_generated_data) generates the data.
- A [Split recipe](recipe:split_time_series_generated_data) creates train, validation, and test sets.
- A [Python recipe](recipe:recipe_from_notebook_Train_a_Keras_time_series_multioutput_model) trains a Keras model with multiple outputs. Outputs include a [Saved Model](saved_model:pI14W9D9) and an [experiments folder](managed_folder:eXga6z7K).
- Evaluation is done via [score recipe](recipe:score_test_ts), a [Python recipe](recipe:compute_evaluation_metrics), and a [standalone Evaluate recipe](recipe:standalone_evaluate_on_test_ts_scored).


## Model Explainability for MLflow models

All Saved Models, except the one used for multi-output regression, are evaluated using the Evaluate recipe and benefit from DSS’s built-in **explainability and performance insights**.

## MLflow Models Deployment in API service

The [mlflow-service](lambda_service:mlflow-service) showcases how MLflow models can be deployed in DSS as an API. It exposes the four trained models as **separate prediction endpoints**, illustrating DSS’s seamless integration between experiment tracking and real-time deployment.

# Next Steps: Using Your Own Data

The project can be downloaded [here](https://downloads.dataiku.com/public/dss-samples/EX_MLFLOW/).

## Technical Requirements

This project:

- requires **Dataiku DSS ≥ 12.4** to ensure full compatibility with MLflow model import, tracking, and deployment features;
- requires a **Python 3.9 code environment** named `py_39_mlflow_in_dku` with the following packages, as specified in the [documentation](https://doc.dataiku.com/dss/latest/mlops/mlflow-models/limitations.html#import-and-scoring):

```mlflow==2.17.2
mlflow==2.17.2
mlflow[extras]
catboost==1.2.5
scikit-learn==1.0.2
statsmodels==0.13.5
torch==2.1.2
torchmetrics==0.11.0
torchvision==0.16.2
tensorflow==2.13.1
tensorflow-estimator==2.13.0
keras==2.13.1
evaluate==0.4.3
transformers==4.51.3```

To simplify project configuration, set this environment  **at the project level** :
 1. Go to ... > ```Settings``` > ```Code env selection```.
 2. Choose " **Select an environment** " mode.
 3. Pick the newly created environment from the dropdown.
 
## How to reuse this project

Depending on the use case you want to replicate (e.g., tabular classification, PyTorch regression, time series multi-output prediction), input datasets must meet the following conditions:

| Use Case                    | Input Format & Requirements                                                                 |
| --------------------------- | ------------------------------------------------------------------------------------------- |
| **Tabular Classification**  | CSV-style table with labeled target column for classification                               |
| **PyTorch Regression**      | CSV-style table with numeric features and one continuous target column                      |
| **Multi-Output Regression** | Table with time series-style inputs (date + numeric features) and multiple numeric targets  |
| **MLflow Import**           | Folder pointing to a valid MLflow model directory structure (from an external training run) |

All the datasets are stored in filesystem so no remapping will be needed. However you have the option to [change the connection type](https://knowledge.dataiku.com/latest/data-sourcing/connections/concept-connection-changes.html#connection-changes) if you want to rely on a specific data storage type.

To reuse a specific flow (e.g., experiment tracking, model import, deployment), you will need to:

 - Replace the  **input datasets**  with your own data. You can do this by creating new datasets and rewiring existing recipes.
 - Adjust any  **Python recipes**  where features or model architectures are hardcoded to match your own dataset structure.
 - For  **model import** , point the  **managed folder**  to your own MLflow model directory.
 - For  **API deployment** , ensure that your Saved Models meet the compatibility criteria outlined [here](https://doc.dataiku.com/dss/latest/mlops/mlflow-models/limitations.html#import-and-scoring).
 
>  📌 If you're unsure which flow zone to reuse, refer back to the walkthrough section relevant to your use case (e.g., Tabular Classification, PyTorch Regression, etc.).

Keep in mind that Dataiku allows you to reuse elements at different levels. In particular it is possible to:

- [duplicate a whole project](https://knowledge.dataiku.com/latest/getting-started/dataiku-ui/how-to-duplicate-project.html)
- [copy and paste entire subflows](https://knowledge.dataiku.com/latest/collaboration/sharing-projects-assets/how-to-copy-flow-items.html)
- copy and paste recipes and remap input/output datasets
- [copy and paste preparation steps](https://doc.dataiku.com/dss/latest/preparation/copy-steps.html) within a Prepare recipe

# Related resources

- [Masterclass Slide Deck](https://drive.google.com/file/d/1SsNqpo0o_fQ2Ym-wzlJ_S8JvAh-0MDy0/view?usp=sharing)
- [Dataiku Documentation: Importing MLflow models](https://doc.dataiku.com/dss/latest/mlops/mlflow-models/importing.html)
- [Dataiku Documentation: Deploying MLflow models](https://doc.dataiku.com/dss/latest/mlops/experiment-tracking/deploying.html)
- [Dataiku Documentation: Experiment Tracking Concepts](https://doc.dataiku.com/dss/latest/mlops/experiment-tracking/concepts.html)
- [Dataiku Documentation: Tracking Experiments in Code](https://doc.dataiku.com/dss/latest/mlops/experiment-tracking/tracking.html)
- [Dataiku Developer Guide: Dataiku's MLflow Extension](https://developer.dataiku.com/latest/api-reference/python/experiment-tracking.html#experiment-tracking)
- [Dataiku Developer Guide: Machine Learning Tutorials](https://developer.dataiku.com/latest/tutorials/machine-learning/index.html)