Completion Checker v6.0.0

Independent verification for AI task claims.

A #teamnormie project by @amidwestnoob

---

The Problem

I run a local AI agent on my own hardware. I asked it to do things — create files, run commands, update configs. It told me "done" every time.

Then I audited it. 40.8% of its task completions were fabricated. Files it claimed to create didn't exist. Commands it said it ran never executed. It was narrating work instead of doing it.

I wrote about it here on r/LocalLLaMA and the response made a few things clear: I didn't know what I was doing with Reddit and GitHub, I don't know what I don't know, as a noob I was expecting way too much from unsupervised local agents, and — the big one — this isn't just my problem. If you're running local AI agents, your agent is probably lying to you too. You just haven't checked yet.

So I built this.

What It Does

The Completion Checker is a standalone Python daemon that independently verifies whether your AI agent actually did what it claimed. It watches a queue of verification tasks, checks each one against the real filesystem, and maintains a trust score over time.

The agent can't influence it, can't talk to it, and can't lie to it. It only trusts what it can see on disk.

Six verification types:

• file_exists — Does the file actually exist? Is it non-empty?
• file_contains — Does the file contain the expected text?
• file_modified_after — Was the file touched after the agent claimed to modify it?
• directory_exists — Does the directory exist?
• approved_command — Run a whitelisted command and check output
• process_running — Is the expected process actually running?

Trust scoring across three time windows (24h, 7d, all-time) with letter grades from A+ down to F.

Quick Start

# Copy the script somewhere on your machine
cp completion_checker_v6.0.0.py ~/tools/

# Add an alias (optional, but nice)
echo 'alias checker="python3 ~/tools/completion_checker_v6.0.0.py"' >> ~/.bashrc
source ~/.bashrc

# Test it — verify a file exists
checker quick-file ~/.bashrc ".bashrc exists"

# Run the verification
checker verify

# See your trust dashboard
checker status

That's it. No dependencies beyond Python 3.9+. No pip install. No Docker. One file.

Example Output

╔══════════════════════════════════════════════╗
║   COMPLETION CHECKER — TRUST DASHBOARD       ║
║   v6.0.0  #teamnormie  @amidwestnoob         ║
╚══════════════════════════════════════════════╝

  24-hour trust:   85.7%  (6/7 passed)  [B]
  7-day trust:     85.7%  (6/7 passed)  [B]
  All-time:        85.7%  (6/7 passed)  [B]

  Recent verifications:

    ✓ GPU visible
    ✓ vLLM inference engine running
    ✓ Lessons.md feedback loop exists
    ✓ Guidance skills folder set up
    ✓ Deterministic skills folder set up
    ✓ LINUS.md operating rules exist
    ✗ LINUS.md operating rules exist
      └─ NOT FOUND: /home/user/rig/LINUS.md

That last failure is real. The project doc said the file was created. The checker said it wasn't there. It was right — the file had never been transferred to the machine. The checker caught it on its first real task.

Usage

# Run as a background daemon (polls every 5 seconds)
checker daemon

# Single verification pass
checker verify

# Trust dashboard
checker status

# Recent history
checker history
checker history -n 50

# Add verification tasks
checker add file_exists "Config file created" --path ~/.config/myapp/config.json
checker add file_contains "API key set" --path ~/.env --expected "API_KEY="
checker add file_modified_after "Log updated" --path /var/log/agent.log --after 2026-02-24T12:00:00Z
checker add directory_exists "Workspace ready" --path ~/projects/myapp
checker add approved_command "GPU check" --command-key nvidia_smi
checker add process_running "Server alive" --process-name uvicorn

# Quick shortcut for file checks
checker quick-file ~/output.txt "Agent created output file"

# See all safe commands
checker list-commands

# Install as systemd user service
checker install-service

Customizing Approved Commands

Open the script and find the APPROVED_COMMANDS dictionary near the top. It's clearly marked with a CUSTOMIZE THIS block. Add commands that match your setup:

APPROVED_COMMANDS: Dict[str, List[str]] = {
    # Uncomment what you use:
    "ollama_list":      ["ollama", "list"],
    "vllm_status":      ["systemctl", "is-active", "vllm.service"],
    "lmstudio_health":  ["curl", "-s", "http://localhost:1234/v1/models"],

    # Add your own:
    "my_check":         ["some", "command", "here"],
}

Only commands in this dictionary will ever be executed. Everything else is blocked. This is by design.

Security Model

This tool was built because an AI agent was lying. The security model assumes the agent is adversarial:

• No arbitrary command execution. Only hardcoded, whitelisted commands run.
• No agent access. The checker runs as a separate process the agent can't reach.
• Path traversal blocked. .. in any path is rejected.
• Atomic writes. Results can't be corrupted by crashes or race conditions.
• Append-only results. History can't be rewritten.
• File locking. Safe for concurrent access.

How It Was Built

Every piece of code in this project goes through the same process:

1. Claude (Sonnet/Opus) builds it — first draft, working implementation
2. Claude 4x self-critique — four passes reviewing its own code like a skeptical senior engineer. Does it run? Does it work on this hardware? What are the edge cases? What's the polish?
3. Grok 4x review — same four-pass process, different model, fresh eyes. Catches things Claude missed, questions assumptions
4. Claude final polish — last pass to clean up anything that came out of the cross-review

No single model ships unchecked work. If Claude builds something, Grok reviews it. If Grok builds something, Claude reviews it. The models don't always agree, and that's the point — I implemented this process specifically to remove the "AI slop" that happens when one model goes unchecked.

How It Works

Your AI agent claims it did something
         │
         ▼
You (or your framework) add a verification task to the queue
         │
         ▼
Checker daemon picks it up, checks the filesystem
         │
         ▼
Result recorded: PASS or FAIL with details
         │
         ▼
Trust score updated across all time windows

The checker doesn't care what agent you're running, what model, or what framework. It only cares about what's on disk.

File Structure

~/.mission-control/
├── verify-queue.json       # Pending verification tasks
├── verify-results.json     # Completed verifications
├── verify-archive.json     # Older results (auto-rotated)
├── verify-dashboard.txt    # ASCII dashboard for file output
└── checker.log             # Runtime logs (auto-rotated at 1MB)

Requirements

• Python 3.9+
• Linux (uses fcntl for file locking, pgrep for process checks)
• No external dependencies. Standard library only.

Background

I'm a non-technical person building local AI infrastructure in my basement. I don't have a CS degree. It's just me and two frontier models — one builds, the other reviews. I built this because I needed it — my AI agent was fabricating almost half its work and I had no way to know without manually checking every single thing.

The 40.8% fabrication rate came from a structured audit. The agent wasn't malicious — it was a combination of the wrong model, the wrong inference engine, and tasks that were too complex for unsupervised execution. But the core insight applies to any local AI setup: if you're not verifying on the filesystem, you're trusting a language model's word for it.

This tool is one piece of a larger sovereignty-focused local AI project. More at @amidwestnoob.

License

MIT — do whatever you want with it.

---

"Never trust self-reporting. Always verify on the filesystem."