# Trace explorer

[![code style: prettier](https://img.shields.io/badge/code_style-prettier-ff69b4.svg?style=flat-square)](https://github.com/prettier/prettier)

## Intro

This repo contains the code for the **Trace Explorer** _Dataiku_ plugin used to run a visual Web Application to explore Dataiku LLM Mesh traces. Requires _Dataiku_ 13.4 or above. The code is based on **Python** / [**Flask**](https://flask.palletsprojects.com/en/stable/) for backend and **Typescript** / [**Vue**](https://vuejs.org/) for the frontend.

## Documentation

For contributor and agent guidance, start here:

- `docs/README.md` (documentation index)
- `AGENTS.md` (mandatory project guardrails)

## Developement

### Standalone

If you want to run your plugin as a standalone outside _Dataiku_, please follow [these instructions](https://design.solutions.dataiku-dss.io/projects/WIKI/wiki/65/Developing%20Dataiku%20Plugins%20in%20Business%20Solutions#run-as-a-standalone-1) before going further.

#### Local configuration

Here is a full example of the config:

```json
// 📍 python-lib/traces_explorer/backend/utils/local_config.json
{
  "webapp_config": {
    "llm_responses_dataset": "history_dataset",
    "llm_responses_column": "llm_context",
    "data_storage": "managed_folder",
    "managed_folder_id": "YOUR_MANAGED_FOLDER_ID"
  },
  "default_project_key": "DEFAULT_PROJECT"
}
```

`webapp_config.data_storage` controls where pasted traces are stored:

- `workload_folder` (default): persisted under workload local folder
- `managed_folder`: persisted in Dataiku managed folder (requires `managed_folder_id`)
- `no_storage`: in-memory only (keeps only the last pasted trace)

> Note: remove `llm_responses_dataset` and `llm_responses_column` fields from the JSON object if you want to emulate an empty **dataset** setup.

### Frontend

The web application is built with [**Vue**](https://vuejs.org/).

#### Installation

You will need to add a `.npmrc` containing your personnal access token (aka **PAT**) in the frontend repo (`resource/frontend`) folder to be able to fetch the private `quasar-ui-bs` package, please follow these [instructions](https://design.solutions.dataiku-dss.io/projects/WIKI/wiki/56/Install%20package%20locally).

Inside the frontend repo, install the dependencies and run the local server [using `pnpm`](https://pnpm.io/installation#using-npm)

```sh
cd resource/frontend
pnpm install
pnpm dev
```

> Note: If you encounter errors during installation after adding the `.npmrc` file, please remove any `yarn.lock` or `package-lock.json` files and refer to [additional troubleshooting](https://design.solutions.dataiku-dss.io/projects/WIKI/wiki/56/Install%20package%20locally#troubleshooting-1) if needed.

### Backend

The backend is powered by [**Flask**](https://flask.palletsprojects.com/en/stable/).

#### Virtual environment and installation

From the root folder of the project, create and activate your virtual environment and install the dependencies with [`uv`](https://design.solutions.dataiku-dss.io/projects/WIKI/wiki/89/Development%20Environment%20Setup#understanding-uv-1):

```sh
uv venv .venv
source .venv/bin/activate
uv pip install -r code-env/python/spec/requirements.txt
```

Then you will need to link the `dataiku` and `dataikuapi` modules to your virtual environment, by specifying the path to your local _Dataiku_ installation :

```sh
ln -s ~/Library/DataScienceStudio/kits/dataiku-dss-13.5.3-osx/python/dataikuapi .venv/lib/python3.10/site-packages/dataikuapi
ln -s ~/Library/DataScienceStudio/kits/dataiku-dss-13.5.3-osx/python/dataiku .venv/lib/python3.10/site-packages/dataiku
```

> **Note:** The usual location provided below is for installations using the `.dmg` installer on macOS. If you installed Dataiku using a different method, or on a different operating system, the path may vary. Please adjust the path accordingly.
It is usually located at `/Users/<YOUR_USERNAME>/Library/DataScienceStudio`.

To start the backend, ensure that you have the environment that you created earlier activated. Then run

```
python ./python-lib/wsgi.py
```

## Backend Flow

```mermaid
flowchart TD
    A[Frontend / API Client] --> B[routes/api/traces.py]
    B --> C[TraceService]

    C --> D[TraceRepository]
    C --> E[TraceParser]
    C --> F[PastedTraceStorage]
    C --> G[dataiku.Dataset / webapp_config]

    F --> H{data_storage mode}
    H --> I[WorkloadTracesFolderService]
    H --> J[ManagedTracesFolderService]
    H --> K[NoStorageTracesFolderService]

    I --> L[(Workload folder files)]
    J --> M[(Managed folder files)]
    K --> N[(In-memory last pasted trace)]
```
