feat: add model-compatibility.json registry and models CLI

[jsdevninja]

Description

Closes #329: a machine-readable model compatibility registry plus CLI commands for listing and inspecting entries.

What changed

• Added model-compatibility.json at the repository root with fields model, provider, status (verified | works | unknown | broken), notes, last_tested, and optional parameter_size_b for size filtering.
• Shipped the same registry with the package (src/browser_harness/model-compatibility.json) via setuptools package-data.
• Added browser_harness.model_compatibility (load/validate, --min-size parsing, --model-info resolution).
• Wired browser-harness models list [--min-size <n>b] [--status <s>] and browser-harness --model-info <name> in run.py, documented in --help.
• Added unit tests for parsing, filters, and CLI dispatch.

How to test

uv run pytest tests/unit/test_model_compatibility.py tests/unit/test_run.py
uv run browser-harness models list
uv run browser-harness models list --min-size 35b --status verified
uv run browser-harness --model-info mistral-small

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Adds a machine-readable model compatibility registry and `browser-harness` commands to list and inspect models, addressing #329. Users can filter by parameter size and status and view detailed info for a model.

- **New Features**
  - Added model-compatibility.json (fields: model, provider, status, notes, last_tested, optional parameter_size_b) and bundled it with the package.
  - Introduced `browser_harness.model_compatibility` for loading/validating the registry, size parsing, and model resolution.
  - New CLI: `browser-harness models list [--min-size <n>b] [--status <s>]` and `browser-harness --model-info <name>`, documented in `--help`.
  - Included the registry in packaging via setuptools; added unit tests for parsing, filters, and CLI dispatch.

<sup>Written for commit 2c09eeffb29f2eaef36044f9d3df501b553afcf7. Summary will update on new commits.</sup>

<!-- End of auto-generated description by cubic. -->

[cubic-dev-ai]

5 issues found across 6 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/browser_harness/model-compatibility.json">

<violation number="1" location="src/browser_harness/model-compatibility.json:1">
P2: Adds a second editable registry copy with no sync mechanism, so root and packaged model metadata can drift and make CLI output differ between dev and installed environments.</violation>

<violation number="2" location="src/browser_harness/model-compatibility.json:3">
P2: Invalid Ollama model name format: this entry uses a space instead of the required `model:tag` form.</violation>
</file>

<file name="src/browser_harness/model_compatibility.py">

<violation number="1" location="src/browser_harness/model_compatibility.py:21">
P2: Repo-root registry lookup uses wrong parent level (`parents[2]` instead of `parents[1]`), causing development-time registry changes to be silently ignored.</violation>

<violation number="2" location="src/browser_harness/model_compatibility.py:49">
P2: `parameter_size_b` validation accepts booleans because `bool` is a subclass of `int`, allowing malformed registry entries to pass as numeric sizes.</violation>

<violation number="3" location="src/browser_harness/model_compatibility.py:60">
P2: parse_size_b accepts malformed size tokens because it searches for a substring match instead of validating the entire argument.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.}

[cubic-dev-ai]

P2: Adds a second editable registry copy with no sync mechanism, so root and packaged model metadata can drift and make CLI output differ between dev and installed environments.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/browser_harness/model-compatibility.json, line 1:

<comment>Adds a second editable registry copy with no sync mechanism, so root and packaged model metadata can drift and make CLI output differ between dev and installed environments.</comment>

<file context>
@@ -0,0 +1,58 @@
+[
+  {
+    "model": "qwen3.6 35b-a3b",
</file context>

[cubic-dev-ai]

P2: Invalid Ollama model name format: this entry uses a space instead of the required model:tag form.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/browser_harness/model-compatibility.json, line 3:

<comment>Invalid Ollama model name format: this entry uses a space instead of the required `model:tag` form.</comment>

<file context>
@@ -0,0 +1,58 @@
+[
+  {
+    "model": "qwen3.6 35b-a3b",
+    "provider": "ollama",
+    "status": "verified",
</file context>

[cubic-dev-ai]

P2: parameter_size_b validation accepts booleans because bool is a subclass of int, allowing malformed registry entries to pass as numeric sizes.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/browser_harness/model_compatibility.py, line 49:

<comment>`parameter_size_b` validation accepts booleans because `bool` is a subclass of `int`, allowing malformed registry entries to pass as numeric sizes.</comment>

<file context>
@@ -0,0 +1,199 @@
+                f"expected one of {sorted(VALID_STATUSES)}"
+            )
+        ps = row.get("parameter_size_b")
+        if ps is not None and not isinstance(ps, (int, float)):
+            raise ValueError(f"entry {i} parameter_size_b must be a number or omitted")
+        out.append(row)
</file context>

[cubic-dev-ai]

P2: parse_size_b accepts malformed size tokens because it searches for a substring match instead of validating the entire argument.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/browser_harness/model_compatibility.py, line 60:

<comment>parse_size_b accepts malformed size tokens because it searches for a substring match instead of validating the entire argument.</comment>

<file context>
@@ -0,0 +1,199 @@
+    t = token.strip().lower()
+    if not t:
+        raise ValueError("empty size")
+    m = _SIZE_RE.search(t)
+    if m:
+        return float(m.group(1))
</file context>

Suggested change

	m = _SIZE_RE.search(t)
	m = _SIZE_RE.fullmatch(t)

[cubic-dev-ai]

P2: Repo-root registry lookup uses wrong parent level (parents[2] instead of parents[1]), causing development-time registry changes to be silently ignored.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/browser_harness/model_compatibility.py, line 21:

<comment>Repo-root registry lookup uses wrong parent level (`parents[2]` instead of `parents[1]`), causing development-time registry changes to be silently ignored.</comment>

<file context>
@@ -0,0 +1,199 @@
+    """Load registry JSON: repo root when developing from a src/ checkout (#329), else bundled copy."""
+    here = Path(__file__).resolve().parent
+    if here.name == "browser_harness" and here.parent.name == "src":
+        root = here.parents[2] / "model-compatibility.json"
+        if root.is_file():
+            return root.read_bytes()
</file context>