integration tests in this repo and in the backend fail randomly, the cause seems to be related to the model download from hf.
rate-limiting is one of the causes (which we can solve like llm2 does, with local caching),
the other issue is that /init and /enable are completed while the model file does not seem to be in place fully, so in a race condition, the embedding process sometimes finds the file, sometimes doesn't:
https://github.com/nextcloud/context_chat/actions/runs/33515757237/job/99882236764#step:33:46
and in yet another case, seems to crash due to OOM?
https://github.com/nextcloud/context_chat/actions/runs/33515757237/job/99882236566#step:34:20
integration tests in this repo and in the backend fail randomly, the cause seems to be related to the model download from hf.
rate-limiting is one of the causes (which we can solve like llm2 does, with local caching),
the other issue is that /init and /enable are completed while the model file does not seem to be in place fully, so in a race condition, the embedding process sometimes finds the file, sometimes doesn't:
https://github.com/nextcloud/context_chat/actions/runs/33515757237/job/99882236764#step:33:46
and in yet another case, seems to crash due to OOM?
https://github.com/nextcloud/context_chat/actions/runs/33515757237/job/99882236566#step:34:20