Agent observability: tracing + eval harness

[PRATHAMESH75]

Context

Generation agents (ontology_generator.py, oasis_profile_generator.py, sentiment_analyzer.py, simulation_config_generator.py) lack structured tracing and regression testing.

Proposed work

• Add structured prompt/response tracing and a small eval/regression suite for the core agents.

Acceptance criteria

• Prompts/responses are logged in a structured, queryable form
• An eval set catches regressions in agent output quality

[HarperZ9]

Useful acceptance criteria I would add before implementation, so tracing and evals do not become a pile of logs:

• log a stable run_id, agent_name, source/input ref, prompt template/version hash, model/config, response ref/hash, tool/action refs if any, and an explicit verdict field
• keep raw prompt/response storage behind a redaction switch; default public fixtures can use hashes plus synthetic cases
• make the eval set include negative fixtures: stale prompt template, missing source, malformed output, empty/ambiguous generation, and changed expected ontology/profile field
• every eval failure should point back to the run record that produced it, not just the final generated file

That gives you two review paths: query traces by agent/run when debugging, and rerun a small fixture pack to see whether a generator regression is real or just expected-output drift.