# Custom probes and checks[¶](https://doc.dataiku.com/dss/latest/scenarios/custom_metrics_and_checks.html#custom-probes-and-checks "Permalink to this headline")

* Custom probe

+ Return values

+ Partitioned datasets

* Custom check

The predefined probes and checks handle simple cases, and more complex computations can be done using custom probes and custom checks. These are python functions, and run with access to the DSS Python APIs.

## Custom probe[¶](https://doc.dataiku.com/dss/latest/scenarios/custom_metrics_and_checks.html#custom-probe "Permalink to this headline")

A custom python probe is a function taking the dataset or folder as parameter and returning values.

### Return values[¶](https://doc.dataiku.com/dss/latest/scenarios/custom_metrics_and_checks.html#return-values "Permalink to this headline")

The function can return a single value, in which case the metric gets the generic name “value”:

§ def process(dataset):

§ return dataset.get\_dataframe().shape[1]

A custom probe can also compute several values in one pass, and return them as a dictionary of name to value:

§ def process(dataset):

§ df = dataset.get\_dataframe()

§ return {'num\_rows' : df.shape[0], 'num\_cols' : df.shape[1]}

DSS automatically infers the type of the metric’s value, among DOUBLE, BOOLEAN, BIGINT, STRING and ARRAY, but in some cases one wants to explicitly specify the type. For example, to get a ISO-formatted UTC timestamp recognized as a date, one has to pass the metric value under the form of a pair of (value, type)

§ from datetime import datetime as dt

§ from dataiku.metric import MetricDataTypes

§ def process(dataset):

§ now = dt.strftime(dt.now(), '%Y-%m-%dT%H:%M:%SZ')

§ return {'now\_as\_string' : now, 'now\_as\_date' : (now, MetricDataTypes.DATE)}

### Partitioned datasets[¶](https://doc.dataiku.com/dss/latest/scenarios/custom_metrics_and_checks.html#partitioned-datasets "Permalink to this headline")

For partitioned datasets, the function of the custom probe receives a second parameter: the partition on which the computation is requested. When the computation is requested on the full dataset and not on just one partition, the value passed is “ALL”.

## Custom check[¶](https://doc.dataiku.com/dss/latest/scenarios/custom_metrics_and_checks.html#custom-check "Permalink to this headline")

A custom check is a function taking the dataset, folder or saved model as parameter and returning a check outcome.

Note

It is advised to name all custom checks in order to distinguish the values they produce in the checks display, because custom checks can’t auto-generate a meaningful name.

If appropriate, a message can be returned as a second return value.

§ def process(last\_values, dataset):

§ if dataset.name == 'PROJ.a\_dataset':

§ return 'OK'

§ else:

§ return 'ERROR', 'not the expected dataset'

The last values of each metric for the dataset, folder or saved model are passed as the first parameter. This parameter is a dict of metric identifier to metric data point.

§ def process(last\_values):

§ if int(last\_values['basic:COUNT\_FILES'].get\_value()) > 10:

§ return 'OK'

§ else:

§ return 'ERROR', 'not enough files'

Here’s another example of a custom check that compares the max value of the dataset column `order\_date` to the last build date for the dataset. Note that in the *Metrics* tab, the “Column statistics” metric for the dataset column `order\_date` is set to `MAX` in this example. This makes the metric value `last\_values['col\_stats:MAX:order\_date']` available in our custom check.

§ from datetime import datetime, timedelta

§ def process(last\_values):

§ build\_start\_date = last\_values['reporting:BUILD\_START\_DATE'].get\_value()

§ max\_data\_timestamp = last\_values['col\_stats:MAX:order\_date'].get\_value()

§ # turn our values into datetime objects to allow us to use datetime functions

§ build\_date\_datetime = datetime.strptime(build\_start\_date, "%Y-%m-%dT%H:%M:%S.%fZ")

§ max\_timestamp\_datetime = datetime.strptime(max\_data\_timestamp, "%Y-%m-%dT%H:%M:%S.%fZ")

§ # flag if the difference is greater than 1 day

§ if build\_date\_datetime > max\_timestamp\_datetime + timedelta(days=1):

§ return 'ERROR'

§ else:

§ return 'OK', build\_date\_formatted.isoformat()
