ISRI/ai-server
10
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

added support for additional more user-friendly interfaces, improved some part of the application loading process to make it a bit simpler

Browse source
This commit is contained in:
faraphel 2025-01-12 12:52:19 +01:00
parent 1a49aa3779
commit f647c960dd
20 changed files with 353 additions and 107 deletions
Download patch file Download diff file Expand all files Collapse all files

2
samples/models/python-bert-2/model.py
View file

@ -16,7 +16,7 @@ def unload(model) -> None:
model.model = None
model.tokenizer = None
def infer(model, prompt: str) -> typing.Iterator[bytes]:
async def infer(model, prompt: str) -> typing.AsyncIterator[bytes]:
inputs = model.tokenizer(prompt, return_tensors="pt")
with torch.no_grad():