# AI-Server

A server that can serve AI models with an API and an authentication system

## Usage

The ai-server project require a conda environement. You can use Docker to deploy it easily.

### Docker

You can easily use docker-compose to run the project. 
Simply go into the project directory and run : 

```bash
docker compose run -d
```

#### Environment Variables

The project use special environement variables :

| Name            | Description                               |
|-----------------|-------------------------------------------|
| MODEL_DIRECTORY | the directory where the models are stored |

#### Volumes

The project might store data inside multiples volumes : 

| Path | Type | Required | Description |
|------|------|----------|-------------|
| /models/ | Volume | True | The place where the models are stored |
| /root/.huggingface/hub | Bind | False | The place where the internal models are cached. Avoid redownloading huge amount of data at every inference |

## Models

A model is an object that can be loaded and do inference.

It is stored inside a directory and must always contain a `config.json` file.

### Configuration

This is a json-structured file with basic information about the model.

It describe : 
- its type (see below)
- its tags
- its interface
- its inputs
- its output mimetype

And other properties depending on the model type.

#### Types

There is for now only a single type of Model : the Python model

##### Python Model

A python model is isolated in a `conda` environement.

To be considered a Python model, you need theses three files : 

| File        | Description                   |
|-------------|-------------------------------|
| config.json | The configuration file        |
| env         | The conda virtual environment |
| model.py    | The model file                | 

###### Configuration

Additionnal fields might be found in the configuration :

...

###### Virtual Environment

You can create a conda virtual environement it with :

`conda create --prefix ./env/ python=<version>`

You can install your requirements inside with `pip3` or `conda`.

###### Internal Model

You need to create a `model.py` file containing a class named `Model`.
This class must be exposed with the Pyro4 library for inter-operability with our main environement.

Here is an example of an internal model :

```python
import Pyro4

import torch
import transformers


@Pyro4.expose
class Model:
    def __init__(self):
        # load the model
        self.pipe = transformers.pipeline(...)

    def infer(self, messages: list[dict]) -> bytes:
        with torch.no_grad():
            outputs = self.pipe(messages)

        return outputs[0]["generated_text"][-1]
```