Pneuma — an evidence-driven runtime for local AI on constrained boards

A small, private language model that runs where the hardware is.

What this project is · Hardware status · Quick start · Measured results · Roadmap · Provenance · Notices

Pneuma is a portable, evidence-driven embedded-AI project. It provides a memory-aware runtime, board capability profiles, model-format guardrails, deployment checks, and a dashboard for inspecting what is actually verified on constrained hardware. The platform boundary is broader than any single board; the measurements below identify the current reference receipt rather than pretending that every port has the same result.

The current measured reference workload is a 28.9M-parameter TinyStories completion model on the ESP32-S3. It combines a compact dense transformer with a large Per-Layer Embeddings (PLE) table stored in flash, allowing the deployable artifact to contain 28.9 million stored parameters without loading all of them into fast RAM. TinyStories is the legacy reference target, not the whole Pneuma product identity.

The project has two parts:

• a Python training, evaluation, quantization, and export pipeline in pneuma/;
• a portable Arduino/PlatformIO adapter and board capability profiles in pneuma/firmware/arduino/;
• a machine-readable board tier matrix in pneuma/board_tiers.json and a strict evidence-receipt validator in pneuma/scripts/validate_board_receipt.py;
• a product contract for the Barista embedded café assistant in pneuma/product/barista_contract.json, plus automated contract, fingerprint, and trust-report gates;
• a browser dashboard for inspecting the memory design, selecting a board board profile, connecting to a device, and streaming generations when the optional Wi-Fi server is enabled.

What makes Pneuma different

Pneuma is a separately branded downstream project in the technical lineage of slvDev/esp32-ai, not an official release of that repository. The relationship and material-by-material scope are documented in NOTICE.md and PROVENANCE.md.

Pneuma's added product boundary is the evidence ladder: a board profile is not support, a product contract is not a trained model, and a host check is not a hardware receipt. The dashboard and release tools keep those states visible instead of collapsing them into one compatibility claim.

What this project is

Pneuma is not a general-purpose assistant. It is a TinyStories completion model and a practical demonstration of memory-aware inference on constrained microcontrollers. Its personality comes from the constraint: the dense compute core stays small, while the larger PLE table lives in flash and is read one token at a time.

The dashboard lets you:

• explore the model in a local demo mode without hardware;
• inspect the SRAM, PSRAM, and flash memory layout;
• choose between ESP32-S3, ESP32-C3, and ESP32-WROOM hardware profiles;
• configure a compatible device by IP address;
• test the device connection and inspect health telemetry;
• stream generated tokens over the device's optional local-network SSE endpoint;
• adjust temperature, top-k, and maximum-token settings.

The current packaged TinyStories firmware exposes five preset prompts on hardware. The repository now includes a verified model-fetch workflow and the ByteLevel-BPE tokenizer asset toolchain needed for the Barista port. Arbitrary text on hardware still requires the model-specific Barista firmware to pass its host and ESP32-S3 validation gates; the dashboard labels that profile as local demo-only until then.

Barista is currently a contract-defined experimental café profile rather than a hardware-ready Pneuma model. Its contract covers short brew guidance, dose checks, sensor readings, device status, explicit actions, evidence fields, and an out-of-scope response. The barista identifier remains stable for tooling compatibility, while independent model assets and a board receipt are still required before making stronger product claims. The contract and source-level trust gates can be run without model files or hardware:

cd pneuma
python3 scripts/validate_product_contract.py
python3 scripts/trust_check.py

The default trust report is intentionally honest about pending model assets and board receipts. Use --strict-assets and --receipt runs/<receipt>.json when running a release gate.

Hardware status

The architecture is intended for compatible ESP32 family boards, but a reference build is not the same thing as universal plug-and-play support.

Board	Status	Meaning
ESP32-S3 N16R8	Verified reference target	The included firmware, partition layout, model image, memory profile, and throughput numbers were measured here.
Other ESP32-S3 PSRAM boards	Compatible port target	Match the flash/PSRAM budget and adapt the board profile if required.
ESP32-C3	Portable port target	Needs a board-specific build and validation of flash, PSRAM, task assumptions, SDK support, and speed.
ESP32-WROOM / classic ESP32	Portable port target	Needs a suitable high-flash/PSRAM variant or a smaller model and board-specific runtime work.
Raspberry Pi Pico (RP2040)	First port target	Adapter smoke test is being established; reduced model and board evidence are still required.

The dashboard's board selector is a hardware-preview and configuration aid. It does not automatically flash a board or transform firmware for a selected profile.

The board matrix uses three tiers: AVR-class boards are command and sensor hosts, RP2040/STM32/ESP32-C3/WROOM boards are small-model port targets, and the ESP32-S3 N16R8 is the verified full-runtime reference. A new board is not supported until its own compile, flash, memory, speed, and output-quality receipt passes validation.

The reference hardware contract is approximately:

1. a 14.9 MB model artifact in a memory-mapped flash partition;
2. about 1.64 MB of tied embedding/output-head data staged in PSRAM;
3. PSRAM for scratch buffers, logits, sampling workspace, and the attention KV cache;
4. at least 512 KB of usable internal SRAM for the dense compute design;
5. platform hooks for flash mapping, PSRAM allocation, timers, tasks, and optional Wi-Fi streaming.

Boards that do not meet this contract may still run a smaller or redesigned model, but should not be described as running the current 28.9M-parameter artifact without a separate measured build. The first selected non-reference target is the Raspberry Pi Pico; its adapter is not a support receipt.

Reference configuration

Metric	ESP32-S3 N16R8 reference
Stored parameters	28.9M
Dense compute core	approximately 559K parameters
Vocabulary	32,768 tokens
Model artifact	14.9 MB, group-wise 4-bit
Memory	512 KB SRAM · 8 MB PSRAM · 16 MB flash
End-to-end generation	approximately 9.5 tok/s
Pure decode measurement	9.72 tok/s · 102.9 ms/model step
Model domain	TinyStories short-fiction generation
Local-only operation	Serial/display inference needs no network

These are measured reference values, not promises for every ESP32 board.

Quick start

1. Export and verify the model

The Python project requires Python 3.12+ and uses uv:

cd pneuma
uv sync
cd src
uv run python export.py
cd ..

cc -O3 -o /tmp/esp32-llm-verify firmware/host_verify/verify.c -lm
/tmp/esp32-llm-verify firmware/model/model.bin firmware/model/golden.txt

Generated model and vocabulary files are intentionally ignored by Git. Create them locally before compiling the firmware.

For a released model, use the guarded workflow instead of copying files by hand:

cd pneuma
scripts/fetch_model.sh tinystories
PORT=/dev/ttyUSB0 scripts/deploy.sh tinystories
python scripts/benchmark_device.py tinystories \
  --port /dev/ttyUSB0 --out runs/tinystories.json --repeat 2

Downloads are staged and checked against pinned SHA-256 hashes and byte counts. Deployment requires an explicit model name and serial port, compiles before flashing, and prints the model fingerprint that the board should report at boot.

2. Build the verified firmware

Install Arduino CLI and Arduino ESP32 core 3.3.10. The following profile is only for the ESP32-S3 N16R8 reference target:

cd pneuma

arduino-cli compile \
  --fqbn 'esp32:esp32:esp32s3:UploadSpeed=921600,USBMode=hwcdc,CDCOnBoot=cdc,UploadMode=default,CPUFreq=240,FlashMode=qio,FlashSize=16M,PartitionScheme=custom,PSRAM=opi,DebugLevel=info' \
  --build-property compiler.optimization_flags=-O3 \
  --build-path /tmp/esp32-llm-build \
  firmware/esp32_llm

Put the exported model at firmware/model/model.bin. For Wi-Fi streaming, configure WIFI_SSID, WIFI_PASS, and USE_WIFI 1 in firmware/esp32_llm/esp32_llm.ino. Do not commit real Wi-Fi credentials.

3. Flash and monitor

Replace /dev/cu.usbmodem2101 with the port reported by your computer:

arduino-cli upload \
  -p /dev/cu.usbmodem2101 \
  --fqbn 'esp32:esp32:esp32s3:UploadSpeed=921600,USBMode=hwcdc,CDCOnBoot=cdc,UploadMode=default,CPUFreq=240,FlashMode=qio,FlashSize=16M,PartitionScheme=custom,PSRAM=opi,DebugLevel=info' \
  --input-dir /tmp/esp32-llm-build \
  firmware/esp32_llm

esptool.py --chip esp32s3 --port /dev/cu.usbmodem2101 --baud 921600 \
  write_flash 0x110000 firmware/model/model.bin

arduino-cli monitor -p /dev/cu.usbmodem2101 --config baudrate=115200

The model partition only needs to be flashed again after a new model export. Firmware-only changes can be uploaded without rewriting the model.

The current reference measurements were taken from one exported model artifact and matching firmware build. The exact artifact checksum is intentionally not published here; reproduce the export locally and record your own checksum when auditing or distributing a model file.

Expected boot diagnostics:

model: V=32768 D=96 L=6 H=4 F=66 P=128
head staged int8: 2.53 MB
PSRAM free after alloc: ~5100 KB

For ESP32-C3, ESP32-WROOM, or another family member, treat the first build as a port. Replace the compile flags, flash mapping, PSRAM mode, task assumptions, and chip argument, then publish board-specific speed, memory, and quality measurements.

4. Connect the dashboard

When Wi-Fi is enabled, the serial monitor prints the device IP address:

Connected. IP: 192.168.1.100
HTTP server listening on port 80

Open the Pneuma dashboard, go to Connection, enter the IP address, and use Test connection. The reference firmware exposes:

Endpoint	Purpose
GET /health	Device status, model name, speed, IP, and active sampling settings
POST /config	Update temperature, top-k, and maximum-token settings
POST /generate	Stream generation as Server-Sent Events

The computer and board must be on the same network. No cloud account or external AI API key is required.

How the memory design works

SRAM   fast, small    dense transformer core and active intermediate values
PSRAM  medium         output head, scratch buffers, logits, and KV cache
FLASH  large          25M-parameter Per-Layer Embeddings lookup table

The dense reasoning core is sized for the internal SRAM budget. The output head and working buffers use PSRAM. The large PLE table remains memory-mapped in flash, and each token reads only a small number of rows rather than loading the entire table into RAM.

The inference loop:

1. primes the model with a preset token sequence;
2. reads the current PLE rows from flash;
3. runs attention, SwiGLU feed-forward layers, and PLE adapters;
4. computes logits with the tied embedding/output head;
5. applies temperature and top-k sampling;
6. emits the selected token to serial, display, and optionally SSE;
7. repeats until the token budget or sequence limit is reached.

The deployable shape is:

vocab      = 32768
d_model    = 96
layers     = 6
heads      = 4
ffn        = 66
ple_dim    = 128

The portable C runtime in pneuma/firmware/common/llm.h reads dimensions from the model header and binds quantized tensors in place. Host verification uses the same forward-pass implementation as the device.

Training and source map

Path	Role
pneuma/src/model.py	Model architecture and PLE layers
pneuma/src/train.py	Training loop
pneuma/src/sample.py	Text sampling
pneuma/src/quantize.py	Group-wise 4-bit post-training quantization
pneuma/src/export.py	Deployable model export
pneuma/src/gen_assets.py	Firmware vocabulary assets
pneuma/data/prepare.py	TinyStories data preparation
pneuma/firmware/host_verify/verify.c	Portable C golden-output check
pneuma/firmware/host_verify/ppl.c	Perplexity comparison for runtime variants
pneuma/firmware/arduino/	Arduino/PlatformIO adapter, profiles, and smoke test
RESULTS.md	Reproducibility and measured hardware report
NOTICE.md	Provenance, privacy, and security notes
LICENSE	Project EULA

Honest limitations

Pneuma is trained on TinyStories, not general instruction data. It is intended to demonstrate memory-efficient local generation, not replace a cloud assistant. It does not reliably:

• answer factual questions;
• follow multi-step instructions;
• write production code;
• perform dependable arithmetic or reasoning;
• maintain a conversation;
• accept arbitrary hardware prompts with the current preset-only firmware.

Short generations of roughly 200–512 tokens are the intended operating range.

Privacy and security

Local serial/display inference does not require a network. If Wi-Fi is enabled, the firmware starts an unauthenticated HTTP server with permissive CORS headers for the dashboard. The current firmware is designed for a trusted local network, not for exposure to the public internet.

Do not port-forward it, place it directly on an untrusted network, or commit Wi-Fi credentials. The /generate endpoint streams generated text over plain HTTP on the local network, and /health exposes device telemetry and the local IP address. See NOTICE.md for the full operational notice.

Attribution

Pneuma is maintained and released by @Wooinxlkz. The source repository is github.com/Wooinxlkz/Pneuma.

Pneuma is a separately branded downstream project in the technical lineage of slvDev/esp32-ai. Upstream-derived code retains its MIT license and attribution; Pneuma-specific additions are identified in PROVENANCE.md. This project is not an official release of, or endorsement by, the upstream repository or author.

TinyStories is the short-fiction dataset by Ronen Eldan and Yuanzhi Li at Microsoft Research (paper). The PLE direction is informed by Google's published Gemma research. Plain-C microcontroller inference work such as llama2.c is cited as research context, not as a claim of shared authorship or copied implementation.

Use of this repository is governed by the project EULA, including its third-party notices and restrictions.