SearchSavior · SearchSavior · Sep 5, 2026 · Sep 5, 2026
diff --git a/README.md b/README.md
@@ -11,7 +11,7 @@
 
 **OpenArc** is an inference engine for Intel devices. 
 
-Serve LLMs, VLMs, Whisper, Kokoro-TTS, Qwen-TTS, Qwen-ASR, Embedding and Reranker models over OpenAI compatible endpoints, powered by OpenVINO on your device. Local, private, open source AI.  
+Serve LLMs, VLMs, Whisper, Kokoro-TTS, Qwen-TTS, Qwen-ASR, Embedding and Reranker models over OpenAI compatible endpoints, powered by OpenVINO on your device. Local, private, open source AI. OpenArc enables you to host speech to text, text to speech and an LLM on the same server, at the same time. 
 
 OpenArc is a community-driven effort to make acceleration from OpenVINO easier to access, deploy and leverage for our usecases.
 
@@ -31,6 +31,8 @@ Thanks to everyone on Discord for their continued support!
 
 ## Features
 
+  - 
+  - Support for openvino genai `scheduler_config`
   - NEW! Containerization with Docker #60 by @meatposes
   - NEW! Speculative decoding support for LLMs #57 by @meatposes
   - NEW! Streaming cancellation support for LLMs and VLMs
@@ -46,8 +48,7 @@ Thanks to everyone on Discord for their continued support!
       - `/v1/embeddings`: `qwen3-embedding` #33 by @mwrothbe
       - `/v1/rerank`: `qwen3-reranker` #39 by @mwrothbe
   - `jinja` templating with `AutoTokenizers`
-  - OpenAI Compatible tool calls with streaming and paralell 
-    - tool call parser currently reads "name", "argument" 
+  - OpenAI Compatible tool and reasoning parsing
   - Fully async multi engine, multi task architecture
   - Model concurrency: load and infer multiple models at once
   - Automatic unload on inference failure
@@ -66,7 +67,7 @@ Thanks to everyone on Discord for their continued support!
 
 
 > [!NOTE] 
-> Interested in contributing? Please open an issue before submitting a PR!
+> Interested in contributing? Please discuss with us on discord or open an issue before submitting a PR!
 
 
 ## Acknowledgments

diff --git a/config.yaml b/config.yaml
@@ -11,7 +11,7 @@ startup_models : ${OPENRARC_AUTOLOAD_MODELS}
 models:
   qwen35-08b:
     engine: ovgenai
-    model_type: llm
+    model_type: vlm
     model_path: /mnt/Ironwolf-4TB/Models/OpenVINO/Qwen3.5/Qwen3.5-0.8B-int8_asym-ov/
     device: CPU
     tool_call_parser: qwen35
@@ -107,10 +107,26 @@ models:
       stream_chunk_frames:
       stream_left_context:
 
-
   qwen3_tts_oscar:
     model_type: qwen3_tts_voice_clone
     model_path:
     engine: openvino
     device: GPU.0
-    qwen3_tts_voice_clone_options:
+    qwen3_tts_voice_clone_options:
+      ref_text:
+      x_vector_only:
+      instruct:
+        qwen3_tts_options:
+      max_new_tokens:
+      do_sample:
+      top_k:
+      top_p:
+      temperature:
+      repetition_penalty:
+      subtalker_do_sample:
+      subtalker_top_k:
+      subtalker_top_p:
+      subtalker_temperature:
+      stream: 
+      stream_chunk_frames:
+      stream_left_context:
diff --git a/docs/assets/tokens_as_straight_line_boundary_example.png b/docs/assets/tokens_as_straight_line_boundary_example.png
diff --git a/docs/assets/tokens_as_straight_line_with_checkpoints.png b/docs/assets/tokens_as_straight_line_with_checkpoints.png
diff --git a/docs/assets/unnamed.png b/docs/assets/unnamed.png
diff --git a/docs/configure.md b/docs/configure.md
@@ -1,4 +1,4 @@
-runtime_config
+## runtime_config
 
 
 runtime_config is an OpenArc entrypoint to the *properties* way of configuring openvino runtime. These settings allow users to tune the behavior of openivno runtime without needing to change application logic and are meant to be "portable", requring no code changes. Since OpenArc 

diff --git a/docs/index.md b/docs/index.md
@@ -13,7 +13,6 @@ icon: lucide/chess-rook
 
 Welcome to the OpenArc documentation! 
 
-
 ## Installation
 
 - [Linux](install.md#linux)
@@ -32,6 +31,18 @@ OpenArc includes a command line tool for controlling the server.
 - [openarc bench](commands.md#bench) — Benchmarking tool for LLMs.
 - [openarc tool](commands.md#tool) — OpenVINO utilities.
 
+## Configuration
+
+>Under construction!
+
+- [Advanced openvino properties](configure.md#runtime_config)
+
+
+## Concepts
+
+- [Tool and Reasoning Parsing](tool_use.md#tool-and-reasoning-parsing)
+
+
 ## Models
 
 Models to get you started and where to find more!
@@ -40,7 +51,7 @@ OpenArc is deeply integrated with the Huggingface Ecosytem and has been written
 
 We are working on improving this process with experimental GGUF support coming, as well as a new frontend application similar to LM-Studio!
 
-Below are some models to get started which are known to work. My hugg
+Below are some models to get started which are known to work. My huggingface has many 
 
 - [Model Sources](models.md#sources)
 - [LLMs](models.md#llms)

diff --git a/docs/models.md b/docs/models.md
@@ -100,13 +100,15 @@ If you need help converting a particular model join Discord and we can help you!
 
 ## Model-Specific Instructions
 
-### Qwen3.5/3.6
+### Tool and Reasoning Parsing
 
-**Is Qwen3.5/3.6 supported?**
 
-Qwen3.5 models has unofficial support. However, they do require you to build `openvino` and `openvino.genai` from source. You will also need to install the latest version of `optimum-intel`. 
+OpenArc now supports tool and reasoning parsing for several architectures. Our approach ensures correct openai compatible parsing using openvino genai by leveraging some useful facts about tokens. 
+
+Since Autoregressive language models emit tokens in a continuous stream one by one, we can visualize it this way
+- streaming decoded tokens to a buff
+
 
-To add a model, run the command `openarc add --model-name MODEL_NAME --model-path /path/to/model --model-type vlm --device GPU|CPU --runtime-config '{"ATTENTION_BACKEND": "SDPA"}'`. OpenArc resolves the VLM vision token from the model's `config.json`. Intel is currently working on adding support for Qwen3.5 to utilize the PA attention backend but it has not been merged yet. This currently appears to be much more performant. If you have built `openvino.genai` with the support included, you may change the runtime config parameter to use PA instead.
 
 **How do I control thinking?**
 

diff --git a/docs/tool_use.md b/docs/tool_use.md
@@ -0,0 +1,49 @@
+
+## Tool and Reasoning Parsing
+
+OpenArc now supports "proper" tool and reasoning parsing for several architectures. Our approach ensures correct openai compatible parsing using openvino genai by leveraging some useful facts about tokens and will be the strategy going forward so I figured it would be a good idea to write down how it works. 
+
+These changes took a few months (of inconsistent work :D) to get right due to difficulties in the deepest undocumented parts of openvino and openvino genai. Plus, I just wasn't happy where every attempted implementation ended up. 
+
+To motivate our approach lets talk about tokens to capture some intuition. 
+
+### Tokens as a straight line
+
+Since Autoregressive language models emit tokens in a continuous stream one by one, it is safe to visualize them as a straight line. 
+
+
+Taking some liberties for readability we can visualize tokens as single words in a sequence starting at 1:
+
+![Tokens as a straight line](assets/tokens_as_straight_line.png)
+
+In most tokenizers `<think>` is exactly one token, a fact we can use to check where the boundaries are in the raw token ids, before any algorithm touches them. A key thing is that the `<` can make it impossible to tokenize `<think>` into exactly one id- and because tokens always occur in a sequence determined by probabilities, we cannot easily check where these critical tokens are along our straight line. So, if a model emits a think tag in its reasoning and our code checks text for think tags we are screwed, and need a better way to make sure programs who use the output of ai systems get content in a consumable format. 
+
+If we know that the important tokens who always have a stable id can be used as boundaries to check raw token ids, we can imagine `<think>` tags like a checkpoints along our sequence:
+
+![Tokens as a straight line with checkpoints](assets/tokens_as_straight_line_with_checkpoints.png)
+
+Now that we can be sure where the checkpoints we can write code that slices each section into its content, and safely tokenize each portion directly into buckets which are routed where they need to go.
+
+![Token stream sliced into buckets at boundaries](assets/tokens_as_straight_line_boundary_example.png)
+
+
+So, for all models where this approach can be applied we will use it! The code lives under [src/engine/ov_genai/tool_parse](https://github.com/SearchSavior/OpenArc/tree/main/src/engine/ov_genai/tool_parse). Formally it follows a state machine pattern.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Further reading: 
+
+[How does tokenization work, and why do LLMs usually rely on subword tokenizers such as BPE?](https://sebastianraschka.com/faq/docs/tokenization-bpe.html) - great overview
+