docs: add AUDIT_LANDSCAPE.md — full repository landscape audit - #90
Draft
THOClabs wants to merge 3 commits into
Draft
docs: add AUDIT_LANDSCAPE.md — full repository landscape audit#90THOClabs wants to merge 3 commits into
THOClabs wants to merge 3 commits into
Conversation
Adds a single read-only audit report at the repo root mapping the system, dependencies/supply chain, security surface (with file:line evidence), test/verification coverage, documentation/onboarding, git history/dead weight, three-tier expansion opportunities, and open questions. Every claim carries a file/line/commit citation and a confidence tag (confirmed/inferred/suspected). Findings were gathered by parallel research agents, mechanically snippet-verified against source, and high-severity security findings independently cross-read. No product code is modified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SKZMQ755R7q38SKqJsCQsg
…est-integrity pass Second audit pass adding 52 verified findings covering the ~13 service subsystems the first pass only characterized, plus dedicated bug-class hunts: - §3.7 Async & event-loop correctness: async drivers doing blocking serial/socket/inference I/O on the loop; a fire-and-forget capture task. - §3.8 Driver correctness & data integrity: declination sign loss near the equator, Alpaca constructor arg-shape misuse, fabricated coordinates on error, a mislabeled catalog star, non-atomic history writes, HTML-email injection, LX200 command-injection surface. - §4 correction: the "safety-critical modules have strong behavioral tests" claim holds only at the unit layer — the entire e2e tier and two of three safety-integration suites import zero production code, and the only base SafetyMonitor evaluation test is dead on checkout. Folds in the CI service-container startup-failure refinement. - §5/§6: broader install.sh config-schema divergence; function-level dead code and a phantom tool symbol. Every new citation mechanically snippet-verified; new highs independently cross-read. No product code modified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SKZMQ755R7q38SKqJsCQsg
Executes the repo in a throwaway uv-sync venv to upgrade the audit's load-bearing claims from static reads to observed behavior, and records one defect static analysis missed: the entry point crashes on startup. - NEW: python -m nightwatch.main aborts on every mode but --version with TypeError from setup_logging(level=...) vs the log_level parameter (main.py:308/:325 vs logging_config.py:185) — the process never reaches the orchestrator. - Observed-by-execution: phantom nightwatch.telescope_tools import fails; the four undeclared deps (llama_cpp/anthropic/openai/RPi) absent even with all extras; uv.lock resolves numpy 2.4.6 (breaks astropy → test_plate_solver collection); the /workspaces safety test fails collection; safety unit tests pass 23/23; ruff 2675 / mypy 160 errors all swallowed by CI. Report file only; no product code changed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SKZMQ755R7q38SKqJsCQsg
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a single read-only audit report,
AUDIT_LANDSCAPE.md, at the repo root. It maps what this repository is, how it works, its dependencies and security surface, test/documentation coverage, git history and dead weight, and where it could realistically grow. No product code is changed.The report was produced in two passes: an initial pass over the hot runtime paths, then a deepen pass covering the ~13 service subsystems the first pass only characterized, plus a dedicated async-correctness bug-class hunt, a broken-wiring sweep, and a full (not sampled) test-integrity grade. 121 verified findings total.
Related Issues
None.
Type of Change
Changes Made
AUDIT_LANDSCAPE.md(~8k words, 8 sections + 2 appendices): system map; dependencies/supply chain; security surface (file:line for every claim, incl. §3.7 async correctness and §3.8 driver correctness/data-integrity/injection); test/verification coverage; documentation/onboarding; history/dead weight; three-tier expansion opportunities (quick wins / substantial builds / reimaginings); open questions.Safety Checklist
Not applicable — this PR adds a documentation file only and modifies no telescope/enclosure/safety code. The audit reports on the safety subsystem (§3.3, §3.7, §4.1) but changes nothing. Safety interlocks, weather checks, emergency stop, graceful shutdown, and confirmation prompts are untouched.
Testing
No code changed, so no test run applies. As a report-quality gate, all 162 file:line citations in the document were mechanically verified to resolve to real files and in-range lines; the underlying findings were snippet-verified against source and cross-read for high-severity items (see the report's Appendix A).
Voice Commands
Not applicable — no voice commands added or modified.
Documentation
AUDIT_LANDSCAPE.md)Additional Notes
Headline: the system's components are individually real and tested, but the production entry point never assembles them (
nightwatch/main.py:247starts an empty service registry;nightwatch/voice_pipeline.py:2086imports a non-existent module). The deepen pass added a layer of driver-correctness bugs (declination sign loss near the equator, Alpaca constructor misuse, fabricated coordinates on error, a mislabeled catalog star, non-atomic history writes) and an async event-loop-blocking class, and corrected a first-pass over-claim: the "safety-critical modules have strong behavioral tests" statement holds only at the unit layer — the entire e2e tier and two of three safety-integration suites import zero production code. The single biggest opportunity remains a config→registry service-assembly factory that turns the existing, tested drivers into a runnable system. Section 8 lists open questions that need a human answer.🤖 Generated with Claude Code
https://claude.ai/code/session_01SKZMQ755R7q38SKqJsCQsg