Repository navigation
Callback Enhancement: Include intermediate responses #158
Description
Is your feature request related to a problem?
The current callback response only includes the final provider response, leaving out crucial intermediate inputs and validation results, which limits visibility into the API's processing steps.
Describe the solution you'd like
Enhance the callback response to include:
- User’s input (post-input guardrail)
- Input to LLM (after input guardrail)
- Output from LLM
- Response to user (after output guardrail)
Original issue
Which feature or component needs enhancement?
The current callback response only contains the final provider response, omitting important intermediate inputs and validation results. This limits visibility into the API's processing steps and effectiveness.
Describe the current behavior
Post guardrails validation, we only send the final output response, but skip intermediate results i.e input to validators, input to llm, output from llm and post output guardrails response to user.
Describe the enhancement you'd like
Enhance the callback response (post guardrails) to include:
User’s input (post-input guardrail)
Input to LLM (after input guardrail)
Output from LLM
response to user(after output guardrail)
Activity
[-]Guardrails: Enrich guardrails response back to kaapi-backend with intermediate input output results[/-][+]Callback Enhancement: Include intermediate responses[/+]