Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 29 additions & 10 deletions documentation/.vuepress/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,35 @@ module.exports = {
],
},

// Custom algorithms
{
title: "Custom algorithms",
path: "/customAlgorithms/",
collapsable: false,
initialOpenGroupIndex: -1,
children: [
{
title: "How to use algorithms",
path: "/customAlgorithms/howToUseAlgorithms.md",
},
{
title: "Integrated algorithms",
path: "/customAlgorithms/integratedAlgorithms.md",
},
{
title: "Algo-providers",
path: "/customAlgorithms/algoProviders/",
children: [
"/customAlgorithms/algoProviders/easyAlgoProvider.md",
"/customAlgorithms/algoProviders/algoProviderTemplates.md",
"/customAlgorithms/algoProviders/customImplementation.md",
"/customAlgorithms/algoProviders/implementInBackend.md",
"/customAlgorithms/algoProviders/addingAlgoProviders.md",
],
},
],
},

// Dashboard
{
title: "Dashboard",
Expand Down Expand Up @@ -130,16 +159,6 @@ module.exports = {
path: "/dashboard/layouts/",
collapsable: true,
},
{
title: "Custom algorithms",
path: "/dashboard/algoProviders/",
collapsable: true,
children: [
"/dashboard/algoProviders/algoProviders.md",
"/dashboard/algoProviders/implementInBackend.md",
"/dashboard/algoProviders/integratedAlgorithms.md",
],
},
{
title: "Data export",
path: "/dashboard/dataExport/",
Expand Down
4 changes: 4 additions & 0 deletions documentation/.vuepress/public/install/addBlock.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
50 changes: 50 additions & 0 deletions documentation/.vuepress/public/install/gears.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions documentation/.vuepress/public/install/heartFile.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
5 changes: 5 additions & 0 deletions documentation/.vuepress/public/install/terminal.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
31 changes: 31 additions & 0 deletions documentation/customAlgorithms/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Adding custom algorithms to DebiAI

Projects have their own specific needs and sometimes the algorithms that are available in DebiAI are not enough. That's why DebiAI allows you to add your own algorithms.

Adding an algorithm to DebiAI allows you to use it with the data directly from the dashboard and analyze the results like any other project data.

Different kinds of algorithms can be added to DebiAI, ranging from data quality to metrics calculation, from data transformation using machine learning models to anomaly detection.

**Learn more:**

<LinkableChoices :choices="[
{
title: 'How to use algorithms?',
description: 'Learn how to use an algorithm from the dashboard and analyze the results',
imageLink: '/getStarted/guide.svg',
linkDestination: './howToUseAlgorithms',
},
{
title: 'Integrated algorithms',
description: 'DebiAI comes with a set of integrated algorithms, learn more about them',
imageLink: '/install/gears.svg',
linkDestination: './integratedAlgorithms',
},
{
title: 'Add your own algorithms',
description: 'Learn about Algo-providers and how to add your own algorithms to DebiAI',
imageLink: '/install/addBlock.svg',
linkDestination: './algoProviders',
},
]"
/>
87 changes: 87 additions & 0 deletions documentation/customAlgorithms/algoProviders/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# DebiAI Algo-providers

An Algo-provider is a service that can respond to the algorithms requests of DebiAI. DebiAI will interact with your Algo-provider in two ways:

- For getting the list of available algorithms
- For running an algorithm and getting the results

## Creating an Algo-provider

We offer different ways to create an Algo-provider:

<LinkableChoices :choices="[
{
title: 'Python module',
description: 'Create an Algo-provider from a single Python function',
imageLink: '/install/python.svg',
linkDestination: '/customAlgorithms/algoProviders/easyAlgoProvider',
tag: 'Recommended'
},
{
title: 'Algo-provider templates',
description: 'Generate an Algo-provider using a pre-built template',
imageLink: '/install/template.svg',
linkDestination: '/customAlgorithms/algoProviders/algoProviderTemplates',
},
{
title: 'Custom implementation',
description: 'Build an Algo-provider from scratch',
imageLink: '/install/build.svg',
linkDestination: '/customAlgorithms/algoProviders/customImplementation',
},
{
title: 'Modify the DebiAI backend',
description: 'Add your algorithm next to the integrated algorithms',
imageLink: '/install/heartFile.svg',
linkDestination: '/customAlgorithms/algoProviders/implementInBackend',
tag: 'Good for contributors'
},
]"
/>

## Algorithms description

You will need to describe your algorithm in a Json format and then create the implementation of your algorithm. After that, DebiAI will be able to understand your algorithm and run it.

Here is, for example, the description of a simple moving average algorithm:

```json
{
"id": "moving_average",
"name": "Moving average",
"description": "Calculate the moving average of a data.",
"tags": ["calculations"],
"author": "DebiAI",
"version": "1.0.0",
"inputs": [
{
"name": "data",
"description": "The data to calculate the moving average on.",
"type": "array",
"arrayType": "number",
"lengthMin": 1,
"lengthMax": 100000
},
{
"name": "periods",
"description": "The number of periods to calculate the moving average on.",
"type": "number",
"default": 3,
"min": 1,
"max": 100
}
],
"outputs": [
{
"name": "moving_average",
"description": "The moving average of the data. Same length as the data",
"type": "array",
"arrayType": "number"
}
]
}
```

As you can see, the input and output of the algorithm are described. This description is used by DebiAI to understand how to run the algorithm and how to display it in the dashboard and what kind of data it needs.

To learn more about the algorithms descriptions, you can read the [Algorithm description documentation](https://github.com/debiai/algo-provider-python-template/blob/main/algo-api/README.md).
126 changes: 126 additions & 0 deletions documentation/customAlgorithms/algoProviders/addingAlgoProviders.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
# Adding Algo-providers to DebiAI

Once you have created and deployed your Algo-provider, you can add it to DebiAI. We offer multiple ways to do so:

<LinkableChoices :choices="[
{
title: 'With debiai-gui',
description: 'If using the cli provided by our python module',
imageLink: '/install/terminal.svg',
elementIdDestination: 'with-debiai-gui'
},
{
title: 'From the dashboard',
description: 'Easiest method, but not persistent',
imageLink: '/install/screen.svg',
elementIdDestination: 'from-the-dashboard'
},
{
title: 'Env. variables',
description: 'Best for Docker deployments',
imageLink: '/install/world.svg',
elementIdDestination: 'connecting-via-environment-variables'
},
{
title: 'Configuration file',
description: 'For development setups',
imageLink: '/install/build.svg',
elementIdDestination: 'connecting-via-configuration-file'
}
]"
/>

## With DebiAI-gui

If you are using the [DebiAI-gui Python package](../../introduction/gettingStarted/installation/README.md#debiai-gui-python-package) to run DebiAI, you can add Algo-providers directly as parameters:

```bash
debiai-gui start -ap http://localhost:4000 http://localhost:8000
```

Once you have created and deployed your Algo-provider, you can add it to DebiAI.

## From the dashboard

You can add your Algo-provider directly from the dashboard. To do so, go to the Algo-providers page from the menu:

![menu](../menu.png)

And click on the "Add a new Algo-provider" button:

![add](../algo_providers_menu_1.png)

You will need to provide the URL of your Algo-provider. This URL should be the root URL of your Algo-provider, for example: `https://my-algo-provider.com/`.

![add](../algo_providers_menu_2.png)

Once you have added your Algo-provider, you will be able to use the algorithms it provides in the Algorithms tab of the analysis dashboard:

![add](../algo_providers_menu_3.png)

## Connecting via Environment Variables

For deployments, you can define environment variables to specify provider URLs.

### Example:

```bash
export DEBIAI_ALGO_PROVIDER_MyAlgoProvider1=http://localhost:3000/
export DEBIAI_ALGO_PROVIDER_MyAlgoProvider2=http://localhost:3010/
```

With Docker:

```bash
docker run -p 3000:3000 \
-e DEBIAI_ALGO_PROVIDER_MyAlgoProvider1=http://localhost:3000/ \
-e DEBIAI_ALGO_PROVIDER_MyAlgoProvider2=http://localhost:3010/ \
debiai/app
```

With Docker Compose:

```yaml
version: "3.8"
services:
debiai:
image: "debiai/app"
ports:
- "3000:3000"
environment:
- DEBIAI_ALGO_PROVIDER_MyAlgoProvider1=http://localhost:3000/
- DEBIAI_ALGO_PROVIDER_MyAlgoProvider2=http://localhost:3010/
```

For a full list fo environment variables, check the [config.env](https://github.com/debiai/DebiAI/blob/main/debiaiServer/config/config.env) file.

## Connecting via Configuration File

You can also configure providers in config.ini:
Example `(debiai/debiaiServer/config/config.ini)`:

```ini
[ALGO_PROVIDERS]
MyAlgoProvider1 = http://localhost:3000/
MyAlgoProvider2 = http://localhost:3010/
```

After editing, restart DebiAI (or rebuild the Docker image if using containers).

::: tip Configuration priority order:

1. DebiAI-gui python module parameters
2. Environment variables
3. Configuration file settings.
:::

If the Algo-provider is accessible and follows the API, DebiAI will list the algorithms in the dashboard analysis page.

::: warning
**DebiAI must be able to access your Algo-provider.**

- If running locally, use `localhost` as the URL.
- If hosted externally, use the **public IP address**.
- When using **Docker**, you may need to use the public IP or `--network host` to access a provider deployed on `localhost`.
More details: [Docker documentation](https://docs.docker.com/network/host/).
:::
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Algo-provider templates

To help you create custom algo-providers, we have created rest api service templates. You can use it as a starting point for your own Algo-provider.

- Python: [Algo-provider Python template](https://github.com/debiai/algo-provider-python-template).

:::tip
Using the [algo-provider Python module](./easyAlgoProvider.md) is the easiest and recommended way to create an Algo-provider for DebiAI if you are using Python. Use the Python template if you want to go further with your Algo-provider.
:::

After creating your Algo-provider, you can add it to DebiAI by providing the URL of your provider. [Learn more on how to add Algo-providers to DebiAI.](./addingAlgoProviders.md#adding-algo-providers-to-debiai)
Loading