Perspective Ensemble

Module: moralstack/runtime/modules/perspective_module.py

The Perspective Ensemble evaluates responses from multiple cognitive perspectives.

For testers and stakeholders: Results (approval per perspective, concerns, suggestions) feed into the aggregated guidance for revision and decision. A very low min_approval can lead to REVISE. Useful for testing sensitive cases (e.g. vulnerable user) where a certain level of caution is expected.

Overview

The Perspective Ensemble:

Simulates how different stakeholders would perceive the response
Identifies concerns specific to each perspective
Generates suggestions to improve the response
Aggregates evaluations into an overall score

On a DCCL MATCH, this module is not invoked. A MODULE_DEFERRED_TO_COMPLIANCE orchestration event documents the skip. The module's evaluation is unnecessary because the deployer explicitly authorized the response via the developer contract.

Prompt structure (prompt-caching reorder, Part A / A5a)

build_perspectives_system_prompt() (moralstack/prompts/perspectives_prompt.py) is now ctx-INDEPENDENT (static only): RISK CONTEXT INTERPRETATION + DEVELOPER CONTRACT EVALUATION + common instructions/JSON schema, byte-identical across every request AND every perspective in a round — the cacheable system prefix.

REQUEST/RESPONSE/RISK CONTEXT (formerly interpolated into the shared system under OPT-2, which made it vary per request and defeated caching) now live in the per-perspective user message, built by build_perspectives_user_prompt(name, instructions, context). This is a deliberate cost/latency trade-off: the draft and risk context are re-sent once per perspective (undoing OPT-2's single-send saving), but cached input tokens are ~90% cheaper, so the reorder is still a net win.

All three call paths (evaluate, _evaluate_single_perspective, evaluate_single) thread the dynamic context explicitly into the per-perspective user message; evaluate_single has no risk parameter, so its DelibContext.risk_score defaults to 0.5 and its user message renders risk_score=0.50 (unchanged behavior — only the message it lives in moved). Verified by tests/test_static_prefix_stability.py.

Standard Perspectives

The default ensemble uses 2 perspectives (user, compliance) to reduce cost and latency (max_perspectives=2 in config). All available perspectives (from DEFAULT_PERSPECTIVES in perspective_module.py):

Perspective	Focus	Weight
Direct User	Satisfaction of original request	1.0
Vulnerable User	Impact on users in fragile situations	1.2
Neutral Observer	Objective and impartial evaluation	1.0
Adversary	Potential abuse by malicious actors	0.8
Compliance	Regulatory and policy compliance	1.0

Weights are used for weighted aggregation; the convergence evaluator uses vulnerable ×1.2, compliance ×1.1, adversary ×0.8 for early-exit threshold calculation.

Output Structure

Perspective

@dataclass
class Perspective:
    id: str              # e.g. "vulnerable_user"
    name: str            # e.g. "Vulnerable User"
    prompt_template: str # Evaluation template
    weight: float        # Weight in aggregation [0, 2]

PerspectiveResult

@dataclass
class PerspectiveResult:
    perspective_id: str
    perspective_name: str
    approval_score: float      # [0, 1]
    concerns: list[str]        # Concerns
    suggestions: list[str]     # Suggestions
    rationale: str             # Reasoning

PerspectiveAggregation

@dataclass
class PerspectiveAggregation:
    results: list[PerspectiveResult]  # Individual results
    overall_score: float              # Weighted average
    min_approval: float               # Most critical perspective
    max_approval: float               # Most favorable perspective
    consensus_level: float            # Agreement [0, 1]
    aggregated_concerns: list[str]    # All concerns
    aggregated_suggestions: list[str] # All suggestions
    recommendation: str               # "proceed" / "revise"

Usage

Initialization

from moralstack.runtime.modules.perspective_module import create_minimal_ensemble

# Default: 2 perspectives (user, compliance) to reduce cost; config from env when not passed
perspectives = create_minimal_ensemble(policy=policy)

Evaluation

result = perspectives.evaluate(
    prompt="User request",
    response="Response to evaluate",
)

print(f"Overall score: {result.overall_score}")
print(f"Min approval: {result.min_approval}")
print(f"Recommendation: {result.recommendation}")

# Details per perspective
for pr in result.results:
    print(f"\n{pr.perspective_name}:")
    print(f"  Approval: {pr.approval_score}")
    print(f"  Concerns: {pr.concerns}")
    print(f"  Suggestions: {pr.suggestions}")

Configuration

EnsembleConfig (in moralstack/runtime/modules/perspective_module.py) controls LLM and ensemble behaviour. When no explicit config is passed (e.g. create_minimal_ensemble(policy) or LLMPerspectiveEnsemble(policy)), config is loaded from environment variables (see Environment Variables).

Environment Variables

All perspective ensemble tuning can be overridden via .env. Variables are read at ensemble creation (CLI and benchmark); empty or missing values use the defaults below. See .env.template for the full list. In application runs (CLI and benchmark), perspective configuration is the single source of configuration — no CLI or code path overrides these variables.

Model (perspective evaluation LLM)

MORALSTACK_PERSPECTIVES_MODEL

Default: (none — uses the same model as the rest of the stack, e.g. OPENAI_MODEL or gpt-4o)
Type: string (OpenAI model id)
Meaning: OpenAI model used only for the perspective ensemble. When set and non-empty, the CLI and benchmark create a dedicated OpenAIPolicy with this model for perspectives; the rest of the stack keeps using OPENAI_MODEL.
Example: MORALSTACK_PERSPECTIVES_MODEL=gpt-4o-mini uses a smaller model for perspective evaluation to reduce cost/latency.

Ensemble behaviour

MORALSTACK_PERSPECTIVES_MAX_RETRIES

Default: 3
Type: int (≥ 1)
Meaning: Number of parse attempts per perspective JSON response before marking that perspective as failed.

Perspective evaluation uses a strict Structured Outputs schema (perspective_response_format() in moralstack/utils/structured_output.py), so the provider enforces the four fields (approval_score, concerns, suggestions, rationale) instead of the client validating the reply after paying for it. It replaces plain json_object mode, which guaranteed only that the reply parsed. PolicyLLM._complete degrades it to {"type": "json_object"} on models outside the supports_json_schema allowlist, so client-side validation stays load-bearing on that path (see docs/modules/openai_params.md). The module records zero parse failures over 4,919 COMPL-AI calls: the schema is preventive here, not a fix.

MORALSTACK_PERSPECTIVES_MAX_TOKENS

Default: 512
Type: int (≥ 1)
Meaning: Maximum tokens for each perspective evaluation LLM response.

MORALSTACK_PERSPECTIVES_TEMPERATURE

Default: 0.1
Type: float (0–2)
Meaning: LLM temperature for perspective evaluations. Low values favour consistent, structured JSON output.

MORALSTACK_PERSPECTIVES_TOP_P

Default: 0.9
Type: float (0–1)
Meaning: Nucleus sampling (top-p) for perspective LLM generation. Controls diversity of token sampling.

MORALSTACK_PERSPECTIVES_PARALLEL_EVALUATION

Default: false
Type: bool (1/0, true/false, yes/no)
Meaning: When true, perspectives are evaluated in parallel (thread pool); when false, sequentially.

MORALSTACK_PERSPECTIVES_MAX_WORKERS

Default: 3
Type: int (≥ 1)
Meaning: Maximum workers for parallel evaluation (used only when MORALSTACK_PERSPECTIVES_PARALLEL_EVALUATION is true).

MORALSTACK_PERSPECTIVES_TIMEOUT_SECONDS

Default: 60.0
Type: float (≥ 1)
Meaning: Timeout in seconds for the full parallel evaluation round.

MORALSTACK_PERSPECTIVES_MAX_PERSPECTIVES

Default: 2
Type: int (≥ 0; 0 = no limit)
Meaning: Maximum number of perspectives to evaluate per request. Default 2 (user, compliance) reduces cost and latency.

MORALSTACK_PERSPECTIVES_CONSERVATIVE_ON_FAILURE

Default: true
Type: bool
Meaning: When a perspective fails (e.g. parse error after retries), treat it conservatively in aggregation.

MORALSTACK_PERSPECTIVES_ENABLE_CACHING

Default: false
Type: bool
Meaning: When true, cache evaluation results by (request, response) to avoid recomputation for identical inputs.

Example Output

PerspectiveAggregation(
    results=[
        PerspectiveResult(
            perspective_id="direct_user",
            perspective_name="Direct User",
            approval_score=0.90,
            concerns=[],
            suggestions=["Include more specific examples"],
            rationale="The response directly addresses the user's question"
        ),
        PerspectiveResult(
            perspective_id="vulnerable_user",
            perspective_name="Vulnerable User",
            approval_score=0.60,
            concerns=[
                "May seem dismissive to someone in distress",
                "Assumes trust in democratic processes"
            ],
            suggestions=[
                "Acknowledge emotional impact",
                "Provide examples of successful change"
            ],
            rationale="Response could be more empathetic"
        ),
        # ... other perspectives
    ],
    overall_score=0.83,
    min_approval=0.60,
    max_approval=0.90,
    consensus_level=0.75,
    aggregated_concerns=[
        "May seem dismissive to someone in distress"
    ],
    aggregated_suggestions=[
        "Acknowledge emotional impact",
        "Provide examples of successful change"
    ],
    recommendation="proceed"
)

Orchestrator Integration

Contribution to Guidance

Perspectives contribute to aggregated guidance:

# Suggestions
[PERSPECTIVES - Suggestions] Vulnerable User: Acknowledge emotional impact

# Concerns
[PERSPECTIVES - Concerns] Vulnerable User: Response may seem dismissive

Revision Decision

if perspectives.min_approval < 0.5:
    # At least one critical perspective
    decision = DecisionType.REVISE

Convergence

# Perspectives contribute to convergence when:
# - overall_score >= 0.8
# - No critical concern
# - No urgent suggestion

Aggregated Score Calculation

overall_score = Σ(approval_i × weight_i) / Σ(weight_i)

# Example:
# Direct User: 0.90 × 0.30 = 0.27
# Vulnerable:  0.60 × 0.25 = 0.15
# Adversary:   0.85 × 0.20 = 0.17
# Neutral:     0.80 × 0.15 = 0.12
# Compliance:  0.95 × 0.10 = 0.095
# Overall = (0.27 + 0.15 + 0.17 + 0.12 + 0.095) / 1.0 = 0.805

Consensus Level

Measures agreement between perspectives:

consensus_level = 1 - (std_dev(approval_scores) / 0.5)
# Values close to 1.0 = high agreement
# Values close to 0.0 = strong disagreement

Caching

The module implements caching:

# Cache based on hash(prompt + response)
# Avoids recomputation for identical inputs

Parallel Execution

Perspectives are evaluated in parallel to reduce latency:

with ThreadPoolExecutor(max_workers=5) as executor:
    futures = [
        executor.submit(evaluate_perspective, perspective)
        for perspective in perspectives
    ]
    results = [f.result() for f in as_completed(futures)]

Constitutional Override

When the Critic reports HARD constitutional violations, perspective approval is capped so that perspectives cannot override the Constitution. The orchestration layer calls:

from moralstack.runtime.modules.perspective_module import apply_constitutional_override

# aggregation: PerspectiveAggregation or EnsembleResult (from state._perspectives_aggregation)
# When EnsembleResult is passed, the inner .aggregation is modified in place.
state._perspectives_aggregation = apply_constitutional_override(
    state._perspectives_aggregation, state.last_critique
)

Parameters: aggregation may be a PerspectiveAggregation or an EnsembleResult; critic_result is the last critic report (or compatible object with violated_hard / violations).
Behaviour: If the critic has HARD violations, weighted_approval is capped to 0.2 on the inner aggregation and a concern string is appended. The same object passed in is returned (type preserved).
Returns: The same object as passed (e.g. EnsembleResult with its .aggregation modified, or PerspectiveAggregation modified).

Conversation Context

For multi-turn requests, perspectives receive the developer contract plus recent conversation history from ProcessedRequest. The prompt declares context_mode=role_serialized_truncated; last 3 turns, and the deliberation runner records CONTEXT_SHAPE_RECORDED plus context-shape metadata on the perspectives LLM call.

FilesExpand file tree

perspectives.md

Latest commit

History