Currently a few operators are designed to call an Ollama api:
- answer_frames_ollama
- answer_transcription_ollama
Here we propose to send the prompt in OpenAI format instead so the operators remain compatible with Ollama but also support other engines or LLM platforms.
Advantages:
- broader compatibility with inference engines or llm platforms
- reduced complexity & dependencies: we can just make that call with standard Python libraries without a container
- opportunity to move that Openai calling logic into a common ancestor operator
It also presents some difficulties as we need to preserve the following features that where supported directly (or easily) by Ollama API:
Additional sub-tasks:
Currently a few operators are designed to call an Ollama api:
Here we propose to send the prompt in OpenAI format instead so the operators remain compatible with Ollama but also support other engines or LLM platforms.
Advantages:
It also presents some difficulties as we need to preserve the following features that where supported directly (or easily) by Ollama API:
Additional sub-tasks: