Skip to content

fix: remove @runtime_checkable from protocols; use hasattr checks in validation#51

Merged
bkrabach merged 2 commits intomainfrom
fix/remove-runtime-checkable-protocols
Mar 21, 2026
Merged

fix: remove @runtime_checkable from protocols; use hasattr checks in validation#51
bkrabach merged 2 commits intomainfrom
fix/remove-runtime-checkable-protocols

Conversation

@bkrabach
Copy link
Collaborator

Problem

On Python 3.11, isinstance(tool, Tool) returns False for every conformant tool. The root cause is:

  1. Tool (and all other core protocols) are decorated with @runtime_checkable
  2. A Tool.__protocol_attrs__ override was added (line 174 of interfaces.py) to exclude input_schema from isinstance checks for backward compatibility
  3. __protocol_attrs__ is a Python 3.12-only feature. On Python 3.11, _get_protocol_attrs() walks cls.__dict__ rather than reading that attribute — so __protocol_attrs__ leaks as a spurious required member, and any tool that doesn't expose it as an instance attribute fails the isinstance check

This caused test_tool_without_input_schema_satisfies_isinstance to fail on Python 3.11 before this fix.

The Fix

interfaces.py

  • Remove @runtime_checkable from all 6 protocols: Orchestrator, Provider, Tool, ContextManager, HookHandler, ApprovalProvider
  • Remove the Tool.__protocol_attrs__ hack
  • Remove the now-unused runtime_checkable import

The Protocol classes themselves are kept intact — they remain importable contract documentation with IDE hover support and static type checking via pyright/mypy. @runtime_checkable was only serving the validation/ dev tooling, and it required version-specific workarounds to work.

validation/ (tool, provider, orchestrator, hook, context)

  • Replace all isinstance(obj, Protocol) checks with structural hasattr() helper functions:
    • _implements_tool_interface(obj) — checks name, description, execute
    • _implements_provider_interface(obj) — checks name, get_info, list_models, complete, parse_tool_calls
    • _implements_orchestrator_interface(obj) — checks execute
    • _implements_hook_handler_interface(obj) — checks callable()
    • _implements_context_manager_interface(obj) — checks all 5 required methods
  • Improved error messages now name the missing member(s) instead of just returning a boolean
  • Updated module docstrings

This matches what the kernel runtime already does: session.py, coordinator.py, and amplifier-foundation all use hasattr() duck typing — never isinstance() against Protocol types.

tests/test_interfaces.py

  • Replace the now-failing test_tool_without_input_schema_satisfies_isinstance (which demonstrated the Python 3.11 bug) with test_tool_without_input_schema_satisfies_structural_check using hasattr()
  • Add test_tool_is_not_runtime_checkable — regression guard
  • Add test_orchestrator_is_not_runtime_checkable

tests/test_validation.py

  • Update message string assertions to match updated error message wording ("interface" instead of "protocol")

Verification

  • Python 3.11.14 (venv): 566 passed, 1 skipped ✅
  • cargo test -p amplifier-core: 19 doc-tests passed ✅

Before / After

Scenario Before After
isinstance(tool, Tool) on Python 3.11 ❌ Returns False (due to __protocol_attrs__ leak) N/A — no longer used in validation
isinstance(tool, Tool) on Python 3.12 ✅ Returns True N/A
Validation detects conformant tool ✅ 3.12 only ✅ Python 3.11, 3.12, 3.13
Validation detects non-conformant tool ✅ (with better error messages)
Runtime (session, coordinator) behavior Unchanged — always used duck typing Unchanged

## Root Cause

On Python 3.11, setting Tool.__protocol_attrs__ (added to exclude
input_schema from backward-compat isinstance() checks) causes
_get_protocol_attrs() to include '__protocol_attrs__' itself as a
required protocol member — because Python 3.11's implementation walks
cls.__dict__ rather than reading the class attribute. This makes
isinstance(tool, Tool) return False for every conformant tool on 3.11.

## The Fix

**interfaces.py**
- Remove @runtime_checkable from all 6 protocols (Orchestrator, Provider,
  Tool, ContextManager, HookHandler, ApprovalProvider)
- Remove the Tool.__protocol_attrs__ hack (Python 3.12-only feature that
  breaks silently on 3.11)
- Remove the runtime_checkable import (no longer used)

The Protocol classes themselves are kept intact — they serve as
importable, hoverable contract documentation and enable pyright/mypy
static type checking. @runtime_checkable was only serving the validation
dev tooling, and it required version-specific workarounds to function.

**validation/ (tool, provider, orchestrator, hook, context)**
- Replace all isinstance(obj, Protocol) checks with structural hasattr()
  helper functions (_implements_tool_interface etc.)
- Add clear error messages that name the missing member(s)
- Update module docstrings to document the hasattr approach
- Update __init__.py docstring

This matches what the kernel runtime (session.py, coordinator.py) and
foundation already do: duck-type via hasattr, never isinstance against
Protocol types.

**tests/test_interfaces.py**
- Replace test_tool_without_input_schema_satisfies_isinstance (currently
  FAILING on Python 3.11 due to the __protocol_attrs__ bug) with
  test_tool_without_input_schema_satisfies_structural_check using hasattr
- Add test_tool_is_not_runtime_checkable to prevent regression
- Add test_orchestrator_is_not_runtime_checkable

**tests/test_validation.py**
- Update message assertions to match new 'interface' wording

## Verified

- Python 3.11.14 (venv): 566 passed, 1 skipped
- cargo test -p amplifier-core: 19 doc-tests passed
@bkrabach bkrabach merged commit a49fffe into main Mar 21, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant