Description:
Problem:
In long conversations (10+ turns), the LLM can suffer from "context drift," where the initial system prompt is too far back in the context window. This can cause the model to "forget" its core instructions, especially the critical rule about conditional Markdown formatting (~ prefix) and the default to be concise.
Proposed Solution:
To combat this, we will implement a "rule re-injection" mechanism. Instead of resending the entire system prompt, we will inject a concise, special "reminder" message into the context at regular intervals.
Implementation Details:
- In
hotkey_manager.py, before sending the messages list to the LLM, check the length of the conversation history.
- If the number of messages is a multiple of 10 (representing 5 full user/assistant turns), inject a new system message right before the latest user prompt.
- The injected message should be a condensed reminder of the core rules, for example:
{
"role": "system",
"content": "[SYSTEM REMINDER: Adhere to core rules. Use Markdown ONLY if prompt starts with ~. Otherwise, PLAIN TEXT ONLY. Be concise by default.]"
}
Acceptance Criteria:
Description:
Problem:
In long conversations (10+ turns), the LLM can suffer from "context drift," where the initial system prompt is too far back in the context window. This can cause the model to "forget" its core instructions, especially the critical rule about conditional Markdown formatting (
~prefix) and the default to be concise.Proposed Solution:
To combat this, we will implement a "rule re-injection" mechanism. Instead of resending the entire system prompt, we will inject a concise, special "reminder" message into the context at regular intervals.
Implementation Details:
hotkey_manager.py, before sending themessageslist to the LLM, check the length of the conversation history.{ "role": "system", "content": "[SYSTEM REMINDER: Adhere to core rules. Use Markdown ONLY if prompt starts with ~. Otherwise, PLAIN TEXT ONLY. Be concise by default.]" }Acceptance Criteria: