ISRI/ai-server
10
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fixed the life cycle of the models (they couldn't unload anymore) and simplified the implementation of the Python models

Browse source
This commit is contained in:
faraphel 2025-01-12 21:26:50 +01:00
parent f647c960dd
commit 8bf28e4c48
9 changed files with 96 additions and 111 deletions
Download patch file Download diff file Expand all files Collapse all files

11
samples/models/dummy/model.py
View file

@ -1,11 +1,6 @@
import typing
def load(model) -> None:
pass
def unload(model) -> None:
pass
async def infer(model, messages: list[dict]) -> typing.AsyncIterator[bytes]:
yield messages[-1]["content"].encode("utf-8")
class Model:
async def infer(self, messages: list[dict]) -> typing.AsyncIterator[bytes]:
yield messages[-1]["content"].encode("utf-8")