# Reporting Scenario Activities[¶](https://knowledge.dataiku.com/latest/courses/o16n/automation/reporting-scenarios.html#reporting-scenario-activities "Permalink to this headline")

Once you’ve created a scenario, you may be interested in receiving updates about scenario activities, such as model training or changes in data quality.

Dataiku DSS provides the ability to add reporters which can send these updates, and actionable messages about scenario activities to users. Reporters can be set up to auto-trigger and send messages to email addresses or messaging channels such as Slack, Hipchat, Microsoft Teams, and Twilio.

Before beginning the hands-on portion of the tutorial, you can watch the following video, which walks through the outline of the steps.

## Let’s Get Started![¶](https://knowledge.dataiku.com/latest/courses/o16n/automation/reporting-scenarios.html#let-s-get-started "Permalink to this headline")

In this tutorial, you will learn the basics of creating reporters that send updates about scenario activities. These reporters will send messages:

* to a channel in Microsoft Teams

* at the beginning of a scenario run

* at the end of a scenario run

We will work with the fictional retailer Haiku T-Shirt’s data.

### Prerequisites[¶](https://knowledge.dataiku.com/latest/courses/o16n/automation/reporting-scenarios.html#prerequisites "Permalink to this headline")

This tutorial assumes that:

* you have completed the Metrics, Checks, & Scenarios lesson.

* you have an account with Microsoft Teams and an existing channel called *Customer Churn Analysis*.

## Create Your Project[¶](https://knowledge.dataiku.com/latest/courses/o16n/automation/reporting-scenarios.html#create-your-project "Permalink to this headline")

From the DSS homepage, navigate to the project you started in the previous lesson. For this tutorial, we will continue to work with the scenario **Rebuild data and retrain model** that was created in the Automation tutorial.

From the **Jobs** menu in the top navigation bar, select **Scenarios** and open the **Rebuild data and retrain model** scenario.

## Create a Reporter to Send a Message When the Scenario Starts[¶](https://knowledge.dataiku.com/latest/courses/o16n/automation/reporting-scenarios.html#create-a-reporter-to-send-a-message-when-the-scenario-starts "Permalink to this headline")

We will create a reporter that sends messages to a channel in Microsoft Teams when the scenario begins. To do this, first obtain the URL of an incoming webhook connector for your channel in Microsoft Teams. You may have to set up the incoming webhook if one doesn’t already exist.

In this tutorial, you will connect DSS to the *Customer Churn Analysis* channel, by means of an incoming webhook connector called *DSS Scenario Reporter*. Using this connection, DSS can post notifications about the scenario’s activities in the channel.

You create reporters on the **Settings** tab of the scenario. Say you want to create a reporter that sends messages to the *Customer Churn Analysis* channel at the beginning of a scenario run. In the **Settings** tab:

* Click **Add Reporter** and select **Microsoft Teams**.

* Name the reporter `Scenario Started`.

* Set the value of “Send on scenario” to **Start**.

* Set the “Run condition” to **Off**. The run condition controls whether the reporter runs at the end of the scenario, therefore, we want it off in this reporter.

* Paste the URL of the webhook connector in the “URL” section. This allows DSS to connect to the *Customer Churn Analysis* channel.

Now, let’s customize the message that the reporter will send when triggered. The “Body” of the reporter contains some default text: `{ "text" : "DSS scenario ${scenarioName}: ${outcome}"}` that consists of message content and variables. You can see a list of the available variables and their descriptions in the “Available variables” section of the reporter.

* Edit the “Body” to make it more descriptive, by specifying the trigger name: `{ "text" : "DSS scenario ${scenarioName} has begun. Triggered by ${triggerName}"}`.

The reporter is now ready to use. To test it, click **Save** and **Run Now**.

When you return to the *Customer Churn Analysis* channel of Microsoft Teams, notice that DSS has sent a message to the channel, stating: “DSS scenario Rebuild data and retrain model has begun. Triggered by Manual trigger”.

## Create a Reporter to Send a Message When the Scenario Ends[¶](https://knowledge.dataiku.com/latest/courses/o16n/automation/reporting-scenarios.html#create-a-reporter-to-send-a-message-when-the-scenario-ends "Permalink to this headline")

It is also useful to send notifications of scenario results, for example: to send a message if a scenario fails. Let’s configure a reporter to do just this.

### Configure the Reporter Settings[¶](https://knowledge.dataiku.com/latest/courses/o16n/automation/reporting-scenarios.html#configure-the-reporter-settings "Permalink to this headline")

To create a reporter that runs when the scenario ends:

* Click **Add Reporter** and select **Microsoft Teams**.

* Name the reporter `Scenario Failed`.

* Keep the default value of “Send on scenario” as **End**.

* Set the “Run condition” to **ON**. Recall that the run condition further controls whether the reporter runs at the end of the scenario. Keep the default value of the run condition: `outcome != ‘SUCCESS’`. This tells the scenario to run at the end of the scenario activity, if the scenario fails.

* Paste the URL of the webhook connector in the “URL” section, so that DSS can connect to the *Customer Churn Analysis* channel.

* Customize the message that the reporter will send when triggered, by editing the “Body” of the reporter to read: `{ "text" : "DSS scenario ${scenarioName}: ${outcome}. Failed on ${firstFailedStepName}"}`. Notice that the “Available variables” section of the reporter consists of a different set of variables that are unique to this kind of reporter (one that sends messages at the end of a scenario activity).

* Save the reporter settings.

### Trigger the Reporter[¶](https://knowledge.dataiku.com/latest/courses/o16n/automation/reporting-scenarios.html#trigger-the-reporter "Permalink to this headline")

Now that you’ve configured the settings for the reporter, the next thing to do is trigger the reporter This way, you can test that it works as it should, when the scenario fails.

In the monitoring with metrics and checks section of the previous Automation tutorial, we enabled the *Records count* metric to automatically compute after building the *Orders\_filtered* dataset.

We also created a check that issued a warning if the **Record count** (representing the number of new customers in the dataset) was less than 170.

Now we will change the **Checks** settings so that the job fails (instead of merely giving a warning), when the number of new customers in the dataset is less than 170. To do this:

* Navigate to the **Status** tab of the *Orders\_filtered* dataset, then click the **Edit** tab and then the **Checks** panel.

* Edit the settings for the *Minimum new customers* check to ensure that the software returns an error if the number of new customers is less than 170:

>

>

> 	+ Set the “Minimum” value to `170` (this is a hard minimum).

> 	+ Turn off the setting for the soft minimum.

> 	+ **Save**

>

The scenario has a trigger that checks the input datasets: *Customers* and *Orders*, and rebuilds the *Orders\_filtered* dataset whenever the input data changes. If you proceed to build the *Orders\_filtered* dataset with the “Build mode” set to **Build required datasets** (the current setting), DSS will check to see if the source datasets have changed, and will update each dataset in the flow as appropriate.

Because we have not changed the input datasets, rebuilding the *Orders\_filtered* dataset in the current build mode would not actually update the dataset. As a result, the *Records count* metric would not auto compute, and the *Minimum new customers* check would not run.

To ensure that the metrics and check run, we must ensure that the *Orders\_filtered* dataset is re-built, by changing the build mode for the dataset to **Force-rebuild the dataset and dependencies**. Specifically:

* Return to the scenario page and navigate to the **Steps** tab.

* Change the “Build mode” in the **Build Filtered Orders** step to **Force-rebuild dataset and dependencies**. This ensures that DSS also rebuilds all datasets leading to the *Orders\_filtered* dataset in the Flow. The metrics and checks also automatically compute and run, once the *Orders\_filtered* dataset is built.

## Test It All[¶](https://knowledge.dataiku.com/latest/courses/o16n/automation/reporting-scenarios.html#test-it-all "Permalink to this headline")

Let’s now test the reporters in our scenario by hitting the **Save** and **Run Now** buttons.

Notice that the scenario *Rebuild data and retrain model* and the job *Build Orders\_filtered* start and soon fail. This is because the checks run as part of a build, and having the check return an ERROR status causes the build to fail.

Upon returning to the *Customer Churn Analysis* channel in Microsoft Teams, we notice that *DSS Scenario Reporter* has sent a couple of messages to the channel. The first is to notify members that the scenario has begun, and soon after, the second message arrives to inform members that the scenario has failed.

## Next Steps[¶](https://knowledge.dataiku.com/latest/courses/o16n/automation/reporting-scenarios.html#next-steps "Permalink to this headline")

Apart from sending simple messages to a Teams channel, DSS supports Microsoft cards via the scenario reporter. These cards are created with JSON text, and they contain components like images, graphs, and more, to present richly formatted content directly into a Microsoft Teams conversation.

We have now created our first couple of scenario reporters. Thanks for following this tutorial!

See also

Reference: Reporting on scenario runs
