Use honest NumPy-first vector backends

README.md

@@ -1,7 +1,7 @@
 # HelixAgent
 
 <p align="center">
-  <strong>Durable autonomous-agent runtime in Python: a bounded plan/execute/observe/replan loop with governed tools, approval gates, and SQLite checkpoints for resumable runs. FastAPI service with Prometheus + OpenTelemetry, optional C++ acceleration with Python fallback, and reproducible benchmarks.</strong>
+  <strong>Durable autonomous-agent runtime in Python: a bounded plan/execute/observe/replan loop with governed tools, approval gates, and SQLite checkpoints for resumable runs. FastAPI service with Prometheus + OpenTelemetry, NumPy/BLAS vector operations, optional C++ ctypes interop, pure-Python fallback, and reproducible benchmarks.</strong>
 </p>
 
 <p align="center">
@@ -22,13 +22,13 @@
   <a href="https://helixagent-mzekflcbhda4zdchpyhjum.streamlit.app/"><img src="https://img.shields.io/badge/Live%20demo-Streamlit-FF4B4B?logo=streamlit&logoColor=white" alt="Live Streamlit demo"></a>
 </p>
 
-HelixAgent is a durable Python agent runtime built around a bounded plan/execute/observe/replan loop with governed tools, approval gates, and SQLite checkpoints. Its included planner is deterministic and rule-based; the planner protocol is extensible, but no model provider is implemented. Optional C++ cosine similarity degrades gracefully to a scale-stable Python fallback when the shared library is unavailable.
+HelixAgent is a durable Python agent runtime built around a bounded plan/execute/observe/replan loop with governed tools, approval gates, and SQLite checkpoints. Its included planner is deterministic and rule-based; the planner protocol is extensible, but no model provider is implemented. NumPy is the default cosine-similarity backend. The optional C++ ctypes binding is an FFI demonstration, not a performance claim; a scale-stable Python implementation is retained only for NumPy-unavailable environments.
 
 ## Features
 
 - **Bounded autonomous execution:** A typed plan/execute/observe/replan loop enforces iteration and tool-call budgets.
-- **Deterministic planning and native acceleration:** The typed planner protocol uses a rule-based default, while `ctypes` optionally loads a C++ cosine-similarity library.
-- **Resilient fallbacks:** Python planning and vector implementations keep the agent usable without native artifacts.
+- **Deterministic planning and vector backends:** The typed planner protocol uses a rule-based default; NumPy/BLAS is the default vector path, while the opt-in `ctypes` binding demonstrates C++ interop.
+- **Resilient fallbacks:** The pure-Python vector implementation is used only if the declared NumPy dependency is unavailable.
 - **FastAPI service:** `/`, `/health`, and `/predict` endpoints with generated OpenAPI documentation.
 - **Observability:** Prometheus metrics and OpenTelemetry instrumentation are attached to the API.
 - **Interactive demo:** A Streamlit interface exercises the same agent runtime.
@@ -48,7 +48,7 @@ Client / Streamlit
  Autonomous runtime ----> SQLite checkpoints
    |        |       |
    |        |       +--> Governed tool registry + approval gates
-   |        +----------> C++ vector library -> Python fallback
+   |        +----------> NumPy (BLAS) default -> optional C++ via ctypes (interop demo) -> pure-Python fallback
    +-------------------> Planner protocol -> deterministic default
 ```
 
@@ -62,7 +62,7 @@ durability. This keeps a future model planner from bypassing execution invariant
 | Safety | Pause write/destructive tools for explicit approval | Safer default with additional operator latency |
 | Runaway control | Bound iterations, tool calls, retries, and tool duration | Predictable cost; a valid long task may exhaust its budget |
 | Planner extensibility | Typed `Planner` protocol with rule-based default | Credential-free execution; no model provider is implemented |
-| Native acceleration | Optional C++ cosine similarity with Python fallback | Portable behavior with environment-dependent performance |
+| Vector interop and fallback | NumPy/BLAS default, optional C++ ctypes binding, Python degradation path | Portable behavior; C++ demonstrates FFI and is not claimed to beat BLAS |
 
 Runtime invariants are covered by tests: terminal states are persisted, denied tools are never
 executed, budget exhaustion fails closed, retries are bounded, and timeout responses do not wait
@@ -89,14 +89,29 @@ not production SLOs or cross-hardware claims. Reproduce locally with:
 
 ```bash
 python -m benchmarks.autonomy_runtime --iterations 200 --warmup 20
+python -m benchmarks.vector_ops --output vector-ops-results.json
 ```
 
 See [benchmark methodology and limitations](docs/BENCHMARKS.md) for metric definitions and the
 evaluation boundary. CI also uploads a fresh `benchmark-results.json` artifact on Python 3.11.
 
+## Vector-operations benchmark
+
+The vector benchmark measures the NumPy default, the optional C++ ctypes backend when its shared library is present, and the pure-Python implementation. It reports measurements from the machine that runs it; it does not rank backends or claim that C++ outperforms BLAS.
+
+~~~bash
+python -m benchmarks.vector_ops --sizes 128 1024 10000 --warmup 10 --repetitions 100 --output vector-ops-results.json
+~~~
+
+| Backend | 128 | 1k | 10k |
+|---|---|---|---|
+| NumPy | <fill after running: python -m benchmarks.vector_ops> | <fill after running: python -m benchmarks.vector_ops> | <fill after running: python -m benchmarks.vector_ops> |
+| C++ ctypes (when available) | <fill after running: python -m benchmarks.vector_ops> | <fill after running: python -m benchmarks.vector_ops> | <fill after running: python -m benchmarks.vector_ops> |
+| Pure Python | <fill after running: python -m benchmarks.vector_ops> | <fill after running: python -m benchmarks.vector_ops> | <fill after running: python -m benchmarks.vector_ops> |
+
 ## Evidence boundaries
 
-The CI matrix exercises Python 3.10 and 3.11 quality/tests, container API health, and Streamlit startup; security and supply-chain workflows run separately. Runtime contract coverage includes terminal-run idempotence, approval gating, bounded retries and budgets, persisted failure for unknown tools, and Python vector fallback properties. The C++ path remains optional and environment-dependent, so native-enabled parity is not claimed.
+The CI matrix exercises Python 3.10 and 3.11 quality/tests, container API health, and Streamlit startup; security and supply-chain workflows run separately. Runtime contract coverage includes terminal-run idempotence, approval gating, bounded retries and budgets, persisted failure for unknown tools, and Python vector fallback properties. The C++ path remains optional and environment-dependent; it is an FFI demonstration rather than a claim of better performance than NumPy.
 
 For the full claim-to-evidence map, invariant definitions, and reproducible statistical primitives, see [claims matrix](docs/CLAIMS_MATRIX.md), [runtime invariants](docs/RUNTIME_INVARIANTS.md), and [evaluation notes](benchmarks/eval/README.md).
 

agent/agent_core.py

@@ -1,14 +1,25 @@
-"""HelixAgent compatibility facade and optional native vector integration."""
+""""HelixAgent compatibility facade and honest vector-operation backends."""
 
 from __future__ import annotations
 
 import ctypes
 import logging
 import math
+import os
+from collections.abc import Iterable
 from pathlib import Path
+from typing import Literal
+
+try:
+    import numpy as np
+except ImportError:  # pragma: no cover - NumPy is a declared runtime dependency.
+    np = None
 
 log = logging.getLogger(__name__)
 
+VectorBackend = Literal["auto", "numpy", "cpp"]
+VectorInput = Iterable[float]
+
 _lib_vec = None
 _LIB_PATH = Path(__file__).parent / "cpp" / "libvector.so"
 try:
@@ -24,27 +35,107 @@
     log.warning("Native vector library unavailable: %s", exc)
 
 
-def cosine_sim(left: list[float], right: list[float]) -> float:
-    """Return cosine similarity using the native library when available."""
-    if not left or len(left) != len(right):
+def _coerce_numpy_vectors(left: object, right: object) -> tuple[object, object]:
+    """Return validated, one-dimensional, contiguous float64 NumPy buffers."""
+    if np is None:
+        raise RuntimeError("NumPy is unavailable; use the Python fallback instead")
+    try:
+        left_array = np.ascontiguousarray(left, dtype=np.float64)
+        right_array = np.ascontiguousarray(right, dtype=np.float64)
+    except (TypeError, ValueError) as exc:
+        raise TypeError(
+            "Vectors must be coercible to one-dimensional float64 arrays"
+        ) from exc
+    if left_array.ndim != 1 or right_array.ndim != 1:
+        raise ValueError("Vectors must be one-dimensional")
+    if left_array.size == 0 or left_array.size != right_array.size:
+        raise ValueError("Vectors must be non-empty and have equal dimensions")
+    return left_array, right_array
+
+
+def _coerce_python_vectors(
+    left: VectorInput, right: VectorInput
+) -> tuple[list[float], list[float]]:
+    """Validate iterable inputs without requiring NumPy."""
+    try:
+        left_values = [float(value) for value in left]
+        right_values = [float(value) for value in right]
+    except (TypeError, ValueError) as exc:
+        raise TypeError("Vectors must be iterable numeric values") from exc
+    if not left_values or len(left_values) != len(right_values):
         raise ValueError("Vectors must be non-empty and have equal dimensions")
-    if _lib_vec is not None:
-        array_type = ctypes.c_double * len(left)
-        return float(_lib_vec.cosine_similarity(array_type(*left), array_type(*right), len(left)))
-    # Normalize before accumulating the dot product.  Squaring tiny or huge
-    # components first can underflow or overflow even when cosine is well-defined.
-    left_magnitude = math.hypot(*left)
-    right_magnitude = math.hypot(*right)
+    return left_values, right_values
+
+
+def cosine_similarity_numpy(left: object, right: object) -> float:
+    """Return cosine similarity through contiguous float64 NumPy arrays."""
+    left_array, right_array = _coerce_numpy_vectors(left, right)
+    denominator = np.linalg.norm(left_array) * np.linalg.norm(right_array)
+    if denominator == 0.0:
+        return 0.0
+    return float(np.dot(left_array, right_array) / denominator)
+
+
+def cosine_similarity_cpp(left: object, right: object) -> float:
+    """Return cosine similarity through the opt-in C++ ctypes demonstration."""
+    if _lib_vec is None:
+        raise RuntimeError(
+            "C++ vector backend is unavailable; build agent/cpp/libvector.so first"
+        )
+    left_array, right_array = _coerce_numpy_vectors(left, right)
+    # C++ reads raw double pointers: float32 or non-contiguous buffers would make it
+    # read the wrong bytes, so coercion above is a required FFI boundary guarantee.
+    pointer_type = ctypes.POINTER(ctypes.c_double)
+    return float(
+        _lib_vec.cosine_similarity(
+            left_array.ctypes.data_as(pointer_type),
+            right_array.ctypes.data_as(pointer_type),
+            left_array.size,
+        )
+    )
+
+
+def cosine_similarity_python(left: VectorInput, right: VectorInput) -> float:
+    """Return a scale-stable pure-Python cosine similarity."""
+    left_values, right_values = _coerce_python_vectors(left, right)
+    left_magnitude = math.hypot(*left_values)
+    right_magnitude = math.hypot(*right_values)
     if not left_magnitude or not right_magnitude:
         return 0.0
 
     normalized_dot = math.fsum(
-        (a / left_magnitude) * (b / right_magnitude)
-        for a, b in zip(left, right, strict=True)
+        (first / left_magnitude) * (second / right_magnitude)
+        for first, second in zip(left_values, right_values, strict=True)
     )
     return max(-1.0, min(1.0, normalized_dot))
 
 
+def cpp_backend_available() -> bool:
+    """Return whether the optional C++ shared library loaded successfully."""
+    return _lib_vec is not None
+
+
+def cosine_sim(
+    left: object, right: object, *, backend: VectorBackend | None = None
+) -> float:
+    """Return cosine similarity with an explicit, evidence-based dispatch order.
+
+    NumPy is the default because it is a declared dependency and uses the platform's
+    BLAS-backed vector operations. The C++ backend is opt-in via backend="cpp" or
+    HELIXAGENT_VECTOR_BACKEND=cpp to demonstrate safe native interop; it is not
+    selected as a performance optimization. The pure-Python implementation is used
+    only when NumPy is unavailable, preserving a last-resort degradation path.
+    """
+    selected = backend or os.getenv("HELIXAGENT_VECTOR_BACKEND", "auto").lower()
+    if selected not in {"auto", "numpy", "cpp"}:
+        raise ValueError("Vector backend must be one of: auto, numpy, cpp")
+    if selected == "cpp":
+        return cosine_similarity_cpp(left, right)
+    if np is None:
+        return cosine_similarity_python(left, right)
+    return cosine_similarity_numpy(left, right)
+
+
 class AgenticAssistant:
     """Backward-compatible synchronous interface over the durable autonomous runtime."""
 

benchmarks/vector_ops.py

@@ -0,0 +1,159 @@
+""""Reproducible three-way cosine-similarity backend benchmark.
+
+The report contains measurements from the current machine only. It does not assert
+a winner: NumPy is the default backend, the optional C++ path demonstrates ctypes
+interop, and the Python implementation is a degradation path.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import platform
+import statistics
+import time
+from collections.abc import Callable
+from pathlib import Path
+from typing import Any
+
+import numpy as np
+
+from agent.agent_core import (
+    cosine_similarity_cpp,
+    cosine_similarity_numpy,
+    cosine_similarity_python,
+    cpp_backend_available,
+)
+
+DEFAULT_SIZES = (128, 1_024, 10_000)
+
+
+def percentile(values: list[float], percentile_value: float) -> float:
+    """Return a nearest-rank percentile for a non-empty sample."""
+    ordered = sorted(values)
+    index = max(0, math.ceil(percentile_value * len(ordered)) - 1)
+    return ordered[index]
+
+
+def _time_backend(
+    backend: Callable[[object, object], float],
+    left: object,
+    right: object,
+    *,
+    repetitions: int,
+    warmup: int,
+) -> dict[str, float]:
+    for _ in range(warmup):
+        backend(left, right)
+
+    samples_us: list[float] = []
+    for _ in range(repetitions):
+        started = time.perf_counter_ns()
+        backend(left, right)
+        samples_us.append((time.perf_counter_ns() - started) / 1_000)
+
+    return {
+        "mean_us": round(statistics.fmean(samples_us), 6),
+        "p50_us": round(statistics.median(samples_us), 6),
+        "p95_us": round(percentile(samples_us, 0.95), 6),
+    }
+
+
+def run_benchmark(
+    *,
+    sizes: tuple[int, ...] = DEFAULT_SIZES,
+    repetitions: int = 100,
+    warmup: int = 10,
+    seed: int = 17_290,
+) -> dict[str, Any]:
+    """Measure every available vector backend with deterministic input vectors."""
+    if not sizes or any(size < 1 for size in sizes):
+        raise ValueError("sizes must contain positive dimensions")
+    if repetitions < 1 or warmup < 0:
+        raise ValueError("repetitions must be positive and warmup cannot be negative")
+
+    backends: list[tuple[str, Callable[[object, object], float]]] = [
+        ("numpy", cosine_similarity_numpy),
+        ("python", cosine_similarity_python),
+    ]
+    if cpp_backend_available():
+        backends.insert(1, ("cpp", cosine_similarity_cpp))
+
+    generator = np.random.default_rng(seed)
+    measurements: list[dict[str, Any]] = []
+    for size in sizes:
+        left = np.ascontiguousarray(generator.standard_normal(size), dtype=np.float64)
+        right = np.ascontiguousarray(generator.standard_normal(size), dtype=np.float64)
+        for name, backend in backends:
+            result = _time_backend(
+                backend, left, right, repetitions=repetitions, warmup=warmup
+            )
+            measurements.append(
+                {
+                    "backend": name,
+                    "vector_size": size,
+                    "repetitions": repetitions,
+                    **result,
+                }
+            )
+
+    return {
+        "benchmark": "vector_ops",
+        "scope": "local cosine-similarity backend timings; no claim of cross-host performance",
+        "environment": {
+            "python": platform.python_version(),
+            "platform": platform.platform(),
+            "processor": platform.processor() or "not reported",
+            "numpy": np.__version__,
+        },
+        "workload": {
+            "sizes": list(sizes),
+            "warmup_runs_per_backend": warmup,
+            "measured_runs_per_backend": repetitions,
+            "random_seed": seed,
+            "cpp_backend_available": cpp_backend_available(),
+        },
+        "measurements": measurements,
+    }
+
+
+def print_table(report: dict[str, Any]) -> None:
+    """Print the measured backend timings without ranking or interpretation."""
+    print("backend | vector size | mean (us) | p50 (us) | p95 (us)")
+    print("--- | ---: | ---: | ---: | ---:")
+    for measurement in report["measurements"]:
+        print(
+            f"{measurement['backend']} | {measurement['vector_size']} | "
+            f"{measurement['mean_us']:.6f} | {measurement['p50_us']:.6f} | "
+            f"{measurement['p95_us']:.6f}"
+        )
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--sizes", nargs="+", type=int, default=list(DEFAULT_SIZES))
+    parser.add_argument("--repetitions", type=int, default=100)
+    parser.add_argument("--warmup", type=int, default=10)
+    parser.add_argument("--seed", type=int, default=17_290)
+    parser.add_argument(
+        "--output",
+        type=Path,
+        default=Path("vector-ops-results.json"),
+        help="JSON artifact path written with the measurements from this run",
+    )
+    args = parser.parse_args()
+    report = run_benchmark(
+        sizes=tuple(args.sizes),
+        repetitions=args.repetitions,
+        warmup=args.warmup,
+        seed=args.seed,
+    )
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(json.dumps(report, indent=2) + "\n", encoding="utf-8")
+    print_table(report)
+    print(f"JSON artifact: {args.output}")
+
+
+if __name__ == "__main__":
+    main()