Skip to content

feat: add council dissent analyzer with auto-deepening research#4

Merged
dobosmarton merged 2 commits intomainfrom
feat/council-dissent-analyzer
Feb 23, 2026
Merged

feat: add council dissent analyzer with auto-deepening research#4
dobosmarton merged 2 commits intomainfrom
feat/council-dissent-analyzer

Conversation

@dobosmarton
Copy link
Copy Markdown
Owner

@dobosmarton dobosmarton commented Feb 23, 2026

Summary

When the Agent Council (multi-model scoring panel) votes on an idea, council members sometimes disagree sharply — one model says GO while another says NO_GO, or they diverge on individual scoring dimensions (e.g., Anthropic rates willingness_to_pay at 90 while OpenAI rates it at 50). Previously these splits were silently aggregated via median and majority vote, losing valuable signal.

The Council Dissent Analyzer detects these disagreements, triggers targeted follow-up research on the contested dimensions, then re-scores with augmented evidence — turning council splits from noise into a research signal for more informed decisions.

How it works

Initial council vote → detect dissent (spread > threshold per dimension)
  → if dissent found:
    → generate targeted follow-up queries via LLM
    → collect follow-up research (Tavily + Perplexity + Firecrawl)
    → re-score with augmented evidence via AgentCouncil
    → attach DissentAnalysis to final PreBuildScore

Changes

  • New models (verdandi/models/scoring.py): DimensionDissent, DissentResolutionRound, DissentAnalysis — all frozen Pydantic models. Added optional dissent_analysis field to PreBuildScore.
  • New config settings (verdandi/config.py): dissent_enabled, dissent_max_rounds, dissent_dimension_threshold, dissent_decision_split_required
  • DissentAnalyzer (verdandi/agents/dissent.py): Core class with detect_dissent(), build_followup_queries(), run_followup_research(), resolve(), and _rescore_with_context(). Maps scoring dimension names to research dimension names for targeted queries.
  • Scoring hook (verdandi/agents/scoring.py): After council.evaluate(), runs dissent resolution when dissent_enabled=True. Also adds mock dissent data to dry-run output.
  • CLI + TUI display: Dissent analysis section in verdandi report and in the TUI experiment detail screen, showing contested dimensions, per-provider scores, resolution rounds, and whether the decision flipped.
  • 19 new tests (tests/test_dissent.py): Detection, dimension mapping, model validation, resolution flow, and settings integration.

Verification

  • ruff check: 0 errors
  • ruff format: clean
  • mypy --strict: 0 errors (94 files)
  • pytest: 460 passed (19 new + 441 existing)

When council members disagree on scoring dimensions (spread > threshold)
or on the GO/NO_GO decision, the analyzer detects these splits, generates
targeted follow-up research queries, collects additional evidence, and
triggers a re-scoring round — turning council disagreement into a
research signal for more informed decisions.
Replace broad pytest.raises(Exception) with specific
pydantic.ValidationError and consolidate threshold/rounds boundary
tests into a single parametrized test case.
@dobosmarton dobosmarton merged commit a1fda0b into main Feb 23, 2026
3 checks passed
@dobosmarton dobosmarton deleted the feat/council-dissent-analyzer branch February 23, 2026 16:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant