Skip to content

feat: Option C user-defined I/O adapters (static + Android + optional CLI dlopen) #446

Open
Open
feat: Option C user-defined I/O adapters (static + Android + optional CLI dlopen)#446
Labels
enhancementNew feature or request
@justinjoy

Description

@justinjoy
justinjoy
opened on Apr 9, 2026

Summary (한글)

Discussion #366 Option C (사용자 정의 I/O 어댑터)의 구현 방향을 확정한다.

핵심 제약

  • 사용자는 libwirelog를 재빌드하지 않는다. 사용자는 자신의 어댑터 코드만 컴파일하여 설치된 libwirelog에 링크한다.
  • 동적 로딩(dlopen)은 원칙적으로 피하되, wl CLI에서 사용자 어댑터를 쓰려는 경로에서만 명시적 opt-in으로 허용한다.
  • .input 디렉티브 문법은 변경하지 않는다 (파서는 이미 임의 key="value"를 허용).

채택 경로

  • 경로 A (기본): 런타임 등록 레지스트리 + 라이브러리 임베딩. 동적 로딩 불필요.
  • 경로 B (opt-in): wl --load-adapter=path.so 로 사용자 공유 라이브러리에서 진입 심볼을 dlopen하여 레지스트리에 주입.

1. Background

Discussion #366 은 eBPF / pcap / kafka 등 비 CSV 소스를 wirelog 로 공급하는 방법을 논의하며 세 가지 대안을 제시했다. Option C 는 "사용자 정의 I/O 어댑터" 로, 다음 이유로 채택한다.

  • Option A (순수 엔진) 는 모든 통합마다 개별 글루 코드를 요구한다.
  • Option B (네이티브 I/O 내장) 는 SO 크기 (현재 203KB), 플랫폼 의존성, 유지보수 부담을 증가시킨다.
  • Option C 는 코어를 최소로 유지하면서 .input scheme:// 확장을 허용한다.

제약 재확인: 사용자는 libwirelog.so 를 재빌드하지 않는다. 따라서 비평 단계에서 제시되었던 "컴파일 타임 WL_IO_ADAPTERS[] 정적 테이블" 대안은 기각된다. 해당 대안은 wirelog 트리에서 사용자 코드를 함께 빌드해야 성립하는데, 이는 제약 위반이다. 런타임 등록 API 가 유일한 선택지다.

2. Design goals

Goal Decision
User extension without rebuilding wirelog Runtime registry + public wl_io_register_adapter() symbol exported from libwirelog
No ABI leakage of internal types Opaque wl_io_ctx_t wraps all internal access (intern table, schema, params)
Backward compatibility for .input(filename=...) Built-in csv adapter auto-registered; no-scheme defaults to csv
No grammar change Reuse existing key="value" params; introduce reserved key io="scheme"
Minimal binary growth Fixed-size registry (32 slots), no hashmap, ~3KB .text budget
Thread safety Single mutex around registry, one-shot lazy builtin registration via pthread_once equivalent
CLI plugin path Optional wl --load-adapter=PATH flag, dispatched via dlopen; compile-time opt-in (-Dio_plugin_dlopen=true), disabled by default
Android (first-class Path A target) NDK cross-build support, Prefab .aar distribution, JNI_OnLoad registration pattern, opaque platform_ctx slot on wl_io_ctx_t for JavaVM *. No dlopen. See section 7b.

3. Adapter interface (C11)

New internal-but-exported header wirelog/io/io_adapter.h:

#ifndef WL_IO_ADAPTER_H
#define WL_IO_ADAPTER_H

#include <stdint.h>
#include "wirelog/types.h"  /* wirelog_column_type_t */

#ifdef __cplusplus
extern "C" {
#endif

#define WL_IO_ABI_VERSION 1u
#define WL_IO_MAX_ADAPTERS 32

/* Opaque context. Internal layout is private; users access it only through
 * the wl_io_ctx_* accessor functions below. This is the ABI firewall. */
typedef struct wl_io_ctx wl_io_ctx_t;

/* Accessors. All pointers returned are borrowed and valid for the duration
 * of the open()/read() call only. */
const char                  *wl_io_ctx_relation_name(const wl_io_ctx_t *);
uint32_t                     wl_io_ctx_num_cols     (const wl_io_ctx_t *);
wirelog_column_type_t        wl_io_ctx_col_type     (const wl_io_ctx_t *, uint32_t col);

/* Parameter lookup from .input key="value" list (excluding reserved "io"). */
const char                  *wl_io_ctx_param        (const wl_io_ctx_t *, const char *key);

/* String interning, the ONLY sanctioned way for an adapter to produce
 * WIRELOG_TYPE_STRING column values. Returns the int64_t id to place in
 * the row buffer. Returns 0 on failure. Internally wraps wl_intern_put()
 * without exposing wl_intern_t. */
int64_t                      wl_io_ctx_intern_string(wl_io_ctx_t *, const char *utf8);

/* Adapter vtable. The adapter itself is a user-owned static const struct
 * whose lifetime must exceed the registry (the registry stores the
 * pointer, not a copy). */
typedef struct wl_io_adapter {
    uint32_t    abi_version;   /* must equal WL_IO_ABI_VERSION */
    const char *scheme;        /* stable string, e.g. "csv", "pcap"    */
    const char *description;   /* human-readable, for diagnostics       */

    /* Required. Produces a malloc-allocated row-major int64_t buffer of
     * size (*out_nrows) * wl_io_ctx_num_cols(ctx). Ownership transfers to
     * the caller; wirelog will free() it with the same libc free().
     * `user_data` is the same pointer stored on the adapter struct below,
     * passed through verbatim as the LAST argument — per the standard C
     * callback convention (pthread_create, qsort_r, glib, libcurl, etc.).
     * Return 0 on success, -1 on error. */
    int (*read)(wl_io_ctx_t *ctx,
                int64_t   **out_data,
                uint32_t   *out_nrows,
                void       *user_data);

    /* Optional, may be NULL. Pre-flight param validation. Writes at most
     * (errbuf_len - 1) bytes into errbuf on error. `user_data` is the
     * trailing argument, matching read(). Return 0 on ok. */
    int (*validate)(wl_io_ctx_t *ctx,
                    char *errbuf, size_t errbuf_len,
                    void *user_data);

    /* Opaque, passed verbatim as the trailing argument to read()/validate().
     * Lifetime must exceed the adapter's registration. wirelog never
     * dereferences it. This slot exists so that adapters written in
     * languages with non-capturing C function pointers (Swift
     * @convention(c), ObjC bridging, Rust extern "C") can attach
     * per-instance state without globals. */
    void *user_data;
} wl_io_adapter_t;

/* Registration API. Thread-safe. Duplicate scheme returns -1.
 * The adapter pointer must remain valid until wl_io_unregister_adapter()
 * is called or the process exits. Typical usage: pass &static_const_struct. */
int  wl_io_register_adapter  (const wl_io_adapter_t *adapter);
int  wl_io_unregister_adapter(const char *scheme);

/* Introspection (used by session_facts.c; also useful for test harnesses). */
const wl_io_adapter_t *wl_io_find_adapter(const char *scheme);

/* Diagnostic. Thread-local. NULL if no error recorded. */
const char *wl_io_last_error(void);

#ifdef __cplusplus
}
#endif
#endif

Why an opaque context: the critic correctly identified that letting adapters touch wl_intern_t * directly would leak internal ABI permanently. wl_io_ctx_t hides it; the only intern operation exposed is wl_io_ctx_intern_string(), which wraps wl_intern_put(). Adapters never see wl_intern_t, wl_ir_relation_info_t, or wirelog_program_t.

Why ownership transfer on the batch buffer: matching the current wl_csv_read_file_ex() contract (wirelog/io/csv_reader.c) minimizes retrofit work. The rule is simple and documented in the header: adapter malloc, core free, same libc. A future arena-based batch mode is a separate PR.

Why user_data is a first-class adapter field (not optional / deferred): adding a parameter to a vtable function pointer after release is an ABI-breaking change that bumps WL_IO_ABI_VERSION and forces every existing adapter to migrate. Doing it right on day one costs nothing. The motivation is specifically:

  • Swift @convention(c) closures cannot capture context. An iOS adapter written in Swift that needs any per-instance state — an NSBundle reference, a decryption key, a database handle — has no place to store it other than a global variable unless the vtable carries a user_data slot. Requiring globals for Swift interop is an anti-pattern.
  • Android JNI adapters likewise benefit: cached jclass/jmethodID handles for a specific ContentResolver adapter are per-instance, not session-wide.
  • Desktop C adapters such as a pcap adapter holding a pcap_t * currently have no clean place to store the handle except a file-scope static — and that breaks the moment a user registers two pcap adapters with different configurations.
  • Two distinct state scopes exist and both are useful:
    • wl_io_ctx_t.platform_ctx — session-level, platform-wide, set once at session init (e.g., JavaVM * on Android, CFBundleRef on iOS if ever needed). Shared across all adapters in that session.
    • wl_io_adapter_t.user_data — adapter-instance-level, private to one specific registered adapter, passed verbatim to its callbacks. Not shared.

A typical Swift registration pattern using user_data:

import wirelog

class BundleAdapterState {
    let bundle: Bundle
    init(_ b: Bundle) { self.bundle = b }
}

let state = BundleAdapterState(.main)
var adapter = wl_io_adapter_t()
adapter.abi_version = WL_IO_ABI_VERSION
adapter.scheme      = ("bundle" as NSString).utf8String
adapter.read        = bundleRead   // @convention(c) function
adapter.user_data   = Unmanaged.passUnretained(state).toOpaque()
wl_io_register_adapter(&adapter)

Inside bundleRead, the adapter recovers the state via Unmanaged<BundleAdapterState>.fromOpaque(user_data!).takeUnretainedValue(). Without user_data, this pattern is impossible in Swift.

Why abi_version: combined with wl_io_register_adapter() rejecting mismatched versions, this gives us a clean break point if the vtable grows. Since libwirelog.so is the only side that dereferences the struct, a version bump is safe.

4. .input directive dispatch

Reserved key: io="scheme". Parser already accepts any key="value" pair (wirelog/parser/parser.c:1150-1217), so no grammar change. The IR gains one field:

/* wirelog/ir/program.h — append to wl_ir_relation_info_t */
char *input_io_scheme;   /* strdup'd from io="..." param; NULL => "csv" */

Collected in wl_ir_program_collect_metadata() and freed in wl_ir_program_free() alongside input_param_names[]. The relations-array grow path in wirelog/ir/program.c must be audited to confirm shallow-copy does not double-free this pointer — existing input_param_names already establishes the safe pattern.

Dispatch rewrite of wl_session_load_input_files() (wirelog/session_facts.c:51-181): replace the hard-coded CSV branch (lines 116-158) with:

const char *scheme = rel->input_io_scheme ? rel->input_io_scheme : "csv";
const wl_io_adapter_t *adapter = wl_io_find_adapter(scheme);
if (!adapter) {
    fprintf(stderr, "error: unknown io scheme '%s' for relation '%s'\n",
            scheme, rel->name);
    return -1;
}

wl_io_ctx_t *ctx = wl_io_ctx_create_for_relation(sess, prog, rel);
if (!ctx) return -1;

if (adapter->validate) {
    char err[256] = {0};
    if (adapter->validate(ctx, err, sizeof err) != 0) {
        fprintf(stderr, "error: %s adapter validate: %s\n", scheme, err);
        wl_io_ctx_destroy(ctx);
        return -1;
    }
}

int64_t *data = NULL;
uint32_t  nrows = 0;
int rc = adapter->read(ctx, &data, &nrows);
wl_io_ctx_destroy(ctx);

if (rc != 0) { free(data); return -1; }
if (nrows > 0 && data) {
    rc = wl_session_insert(sess, rel->name, data, nrows, rel->column_count);
}
free(data);
if (rc != 0) return -1;

The CSV-specific path-resolution logic currently at session_facts.c:89-113 moves into the built-in CSV adapter (wirelog/io/csv_adapter.c), where it belongs. session_facts.c becomes fully scheme-agnostic.

Backward compatibility rules (airtight):

  • .input R(filename="x.csv") → no io=, defaults to csv, CSV adapter reads filename/delimiter params. Identical to current behavior.
  • .input R(io="csv", filename="x.csv") → same thing, explicit.
  • .input R(io="csv", filename="x.csv", source="...") → source is ignored by CSV adapter; reserved for future adapters.
  • .input R(io="pcap", filename="y.pcap") → dispatches to user-registered pcap adapter, which may choose to interpret filename itself.
  • .input R(filename="foo.pcap") → STILL routes to CSV adapter (not extension-sniffed); will fail at CSV parse, same as today. Documented.
  • io="" or unknown scheme → hard error.

5. Built-in CSV adapter

New file wirelog/io/csv_adapter.c. Implements wl_io_adapter_t with scheme="csv". Its read() callback:

  1. Calls wl_io_ctx_param(ctx, "filename") / "delimiter" to get params.
  2. Replicates the existing cwd-fallback path resolution (session_facts.c:89-113).
  3. Branches on whether any column is WIRELOG_TYPE_STRING (use wl_io_ctx_col_type() accessor).
  4. For integer-only: calls wl_csv_read_file() directly.
  5. For mixed: calls a new wl_csv_read_file_via_ctx() variant (or refactors wl_csv_read_file_ex() to take a callback for interning), ensuring wl_io_ctx_intern_string() is used for STRING cells. This keeps wl_intern_put() off the adapter's visible API.

Auto-registered at first wl_io_find_adapter() call via a pthread_once-based ensure_builtins(). Also called at the top of wl_io_register_adapter() so that user registration cannot race ahead of built-ins.

6. Path A — Library embedding (no dynamic loading)

User workflow:

/* user_main.c — compiled by the user, NOT by wirelog */
#include <wirelog/wirelog.h>
#include <wirelog/io/io_adapter.h>

static int my_pcap_read(wl_io_ctx_t *ctx, int64_t **out_data, uint32_t *out_nrows) {
    const char *path = wl_io_ctx_param(ctx, "filename");
    /* ... open pcap, fill row-major int64_t buffer, intern any strings via
       wl_io_ctx_intern_string(ctx, s) ... */
    return 0;
}

static const wl_io_adapter_t my_pcap_adapter = {
    .abi_version = WL_IO_ABI_VERSION,
    .scheme      = "pcap",
    .description = "libpcap file reader",
    .read        = my_pcap_read,
};

int main(void) {
    wl_io_register_adapter(&my_pcap_adapter);
    /* ... wl_session_create, load program with
       .input packet(io="pcap", filename="cap.pcap"), run, snapshot ... */
}

Build on the user side:

cc -c user_main.c my_pcap_read.c
cc -o user_app user_main.o my_pcap_read.o -lwirelog -lpcap

wirelog is not rebuilt. The user only compiles their own translation units and links against the installed libwirelog.so. This satisfies the constraint exactly.

7. Path B — CLI plugin via dlopen (opt-in only)

The wl CLI driver (wirelog/cli/driver.c) is a pre-built binary that the user cannot re-link without rebuilding wirelog. To use a custom adapter with wl run program.dl, some form of dynamic loading is unavoidable. We accept this as a bounded exception.

Compile-time gate: new meson option -Dio_plugin_dlopen=true (default false). When disabled, wl --load-adapter prints an error and the dlopen code path is not compiled in (no -ldl dependency, no symbol exports). When enabled:

wl --load-adapter=/path/to/libwirelog-pcap.so run program.dl

Contract for the user .so:

/* User's shared library must export exactly one symbol: */
WL_IO_PLUGIN_EXPORT
const wl_io_adapter_t *const *wl_io_plugin_entry(uint32_t *n_out, uint32_t abi_ver);

The CLI calls dlopen → dlsym("wl_io_plugin_entry") → invokes with WL_IO_ABI_VERSION → receives an array of adapter pointers → registers each via wl_io_register_adapter(). On ABI mismatch, the CLI refuses to register and prints a clear message.

This is the only dynamic-loading code path in the project. It lives in wirelog/cli/plugin_loader.c, is gated by WL_IO_PLUGIN_DLOPEN, and is not linked into libwirelog.so itself. Embedders following Path A pay zero cost and have zero dlopen surface.

Why this is acceptable despite the original "no dynamic loading" rule: the rule targets runtime plugin discovery and unrestricted .so hot-loading as a general extension mechanism. The --load-adapter flag is (a) explicit per-invocation, (b) compile-time gated off by default, (c) scoped to the CLI only, (d) does not affect the library ABI or the embedding API. It is the minimum concession to make the CLI usable with user adapters without re-linking the CLI itself.

Users who cannot accept any dlopen at all simply disable -Dio_plugin_dlopen and use Path A exclusively.

7b. Path C — Android support

Android is a first-class Path A target and is explicitly not a Path B target. The analysis below justifies both decisions and enumerates the PR #1 deliverables needed to unblock Android app developers.

7b.1 Why Path A is the only viable path on Android

  • Android API 24+ enforces linker namespace restrictions (SELinux neverallow rules) that reject dlopen on paths outside the app's own private native library directory. Arbitrary plugin .so loading is not permitted for normal apps.
  • There is no wl CLI binary in a normal APK. The user's entry point is Java/Kotlin, not a C main(). The --load-adapter flag has nowhere to run.
  • Therefore: on Android, adapter registration always happens through the runtime registry API called from JNI_OnLoad, with the adapter .c statically compiled into the app's own JNI shared library. No dlopen, no plugin .so separation, no Path B.

The narrow exceptions — rooted devices, platform-signed system apps, debug-only adb shell flows — are called out in docs/android.md but are not first-class supported configurations.

7b.2 Toolchain and distribution

wirelog is already 95% NDK-ready because it is pure C11 with no C++ STL dependency (no libc++_shared / libstdc++ conflict surface), uses only POSIX pthread primitives available since API 21, and gates NEON on aarch64 in meson.build. Two targeted changes are required:

  1. Exclude the CLI from Android builds. New meson option: 
    # meson_options.txt
option('android', type: 'boolean', value: false,
       description: 'Build for Android: skip CLI target, add 16KB page alignment')
    In wirelog/meson.build, guard the wirelog_cli executable definition behind if not get_option('android'). Android apps have no shell entry point for the CLI.
  2. 16KB page-size alignment for Play Store 2026. When -Dandroid=true, append -Wl,-z,max-page-size=16384 to libwirelog's link_args. NDK r27+ defaults to this but earlier NDK versions require it explicitly, and Play Store will reject misaligned shared libraries from 2026 onward.

Cross-compilation is driven by meson cross-files, one per ABI:

File ABI Meson cross-file content
cross/android-arm64.ini arm64-v8a NDK toolchain root, aarch64-linux-android21-clang
cross/android-armv7.ini armeabi-v7a armv7a-linux-androideabi21-clang
cross/android-x86_64.ini x86_64 x86_64-linux-android21-clang (emulator)

Recommended distribution format: Android Prefab .aar, containing per-ABI libwirelog.so plus the public headers (wirelog/*.h, wirelog/io/io_adapter.h) wrapped in a Prefab package. This is the standard AGP 4.1+ format and lets app developers consume wirelog by adding one Maven coordinate plus buildFeatures { prefab true } in their build.gradle. A thin packaging script runs three cross-compiles and assembles the .aar — no changes to meson.build beyond the two above.

Minimum Android API level: 21 (Android 5.0). thread_posix.c uses only pthread_create, pthread_mutex_init, pthread_cond_wait, pthread_cond_broadcast, all of which are API 21 primitives.

7b.3 App-level integration pattern

User's adapter .c is compiled directly into libmyapp.so, not as a separate .so. A separate adapter .so would require dlopen from the app's private lib dir and increase namespace complexity with zero benefit. Single-.so integration is simpler, keeps the adapter private to the app, and eliminates any RTLD visibility issue.

// app/src/main/cpp/myapp_jni.c
#include <jni.h>
#include <android/log.h>
#include "wirelog/wirelog.h"
#include "wirelog/io/io_adapter.h"
#include "my_pcap_adapter.h"   /* user-written, same TU tree */

static const wl_io_adapter_t MY_PCAP = {
    .abi_version = WL_IO_ABI_VERSION,
    .scheme      = "pcap",
    .description = "in-app pcap reader",
    .read        = my_pcap_read,
};

JNIEXPORT jint JNICALL
JNI_OnLoad(JavaVM *vm, void *reserved)
{
    (void)vm; (void)reserved;
    if (wl_io_register_adapter(&MY_PCAP) != 0) {
        __android_log_print(ANDROID_LOG_ERROR, "wirelog",
                            "adapter register failed: %s", wl_io_last_error());
        return JNI_ERR;
    }
    return JNI_VERSION_1_6;
}
# app/src/main/cpp/CMakeLists.txt
cmake_minimum_required(VERSION 3.22)
project(myapp)
find_package(wirelog REQUIRED CONFIG)      # from the Prefab .aar
add_library(myapp SHARED myapp_jni.c my_pcap_adapter.c)
target_link_libraries(myapp wirelog::wirelog android log)
// app/build.gradle
android {
    buildFeatures { prefab true }
    defaultConfig { externalNativeBuild { cmake { path "src/main/cpp/CMakeLists.txt" } } }
}
dependencies { implementation 'com.cleverplant:wirelog-android:x.y.z@aar' }

wirelog is not rebuilt. The app developer compiles only myapp_jni.c + my_pcap_adapter.c. The satisfies the "user builds only their own part" constraint identically to Path A on desktop.

7b.4 File system and .input paths

On Android, /tmp and the process cwd are not useful: app-private data lives under Context.getFilesDir(), and assets inside the APK are accessed through AAssetManager, not fopen. Consequences:

  • The cwd-fallback path resolution currently in wirelog/session_facts.c:89-113 (moving into csv_adapter.c per section 5) is a no-op on Android. Not harmful, but document that Android callers should always pass absolute paths via .input(filename="/absolute/path").
  • The JNI bridge is responsible for resolving Context.getFilesDir() and splicing it into the .input filename before calling wl_session_load_input_files(). The core does not need a new "base directory" parameter; this is purely a caller concern.
  • APK assets become a natural motivating example for a user adapter. A reference android_asset adapter using AAssetManager_open / AAsset_read ships under examples/android/asset_adapter/ — outside the core tree because it depends on <android/asset_manager.h>. Datalog usage: .input events(io="android_asset", filename="events.csv").

7b.5 JNI callbacks and threading

  • Thread ownership: wl_session_load_input_files() and wl_session_step() may be called from any thread. There is no main-thread assertion in session.c, workqueue.c, or thread_posix.c. Apps should invoke wirelog from a background worker (e.g., Kotlin Dispatchers.IO), never from the UI thread.
  • JNI callbacks from adapter read(): an adapter that needs to call back into Java (e.g., ContentResolver query) must capture JavaVM * at JNI_OnLoad time, call AttachCurrentThread at the start of read(), and DetachCurrentThread before returning. This is user responsibility; the core interface does not marshal Java references. Documented in docs/android.md.

7b.6 One core design concession for Android

The wl_io_ctx_t opaque context defined in section 3 gains one optional slot:

/* In io_adapter.h, extend the (internal) struct and add an accessor: */
void       *wl_io_ctx_platform(const wl_io_ctx_t *);         /* NULL on desktop */
int         wl_io_ctx_set_platform(wl_io_ctx_t *, void *);   /* set once at session init */

Use: an Android app that needs JNI callbacks from inside adapter read() calls wl_io_ctx_set_platform(ctx, javaVm) via a session-level setter at startup (e.g., alongside wl_session_create()). Adapters retrieve it via wl_io_ctx_platform(ctx) and cast to JavaVM *. On desktop it is NULL and ignored.

This costs nothing at runtime (void * field + two accessors, ≲ 50 bytes .text), avoids a second parallel "Android adapter interface", and cannot be retrofitted later without bumping WL_IO_ABI_VERSION and breaking plugin consumers. It is the only core interface change Android motivates.

7b.7 PR #1 Android deliverables

Included in PR #1:

  • meson_options.txt — new android boolean option (default false)
  • wirelog/meson.build — guard wirelog_cli on not get_option('android'); conditional -Wl,-z,max-page-size=16384 under android option
  • cross/android-arm64.ini, cross/android-armv7.ini, cross/android-x86_64.ini — NDK meson cross-files
  • wirelog/io/io_adapter.h — wl_io_ctx_platform / wl_io_ctx_set_platform accessors (trivial additions on top of section 3 design)
  • examples/android/asset_adapter/ — reference AAssetManager-based adapter, CMakeLists.txt, README with end-to-end Datalog example
  • docs/android.md — integration guide: Prefab .aar consumption, JNI_OnLoad skeleton, path handling, threading contract, 16KB page-size requirement, API 21 minimum
  • CI: add a cross-compile build smoke test (compile only, no emulator) for arm64-v8a using a pinned NDK version, to prevent regressions on the Android build path

Deferred beyond PR #1:

  • Prebuilt .aar publication to Maven Central / JitPack (requires separate release CI matrix)
  • Full Android emulator-based test execution (AVD spin-up adds significant CI time; not justified before Prefab packaging exists)
  • Adapters beyond the android_asset reference example (ContentResolver, MediaStore, etc.)
  • 16KB page-size runtime verification on a 16KB device (requires Pixel 8+ or emulator config)

7b.8 Android-specific risks

  1. NDK version drift. Pin NDK to a known-good version in CI (e.g., r26d or r27) and bump explicitly. Random breakage from NDK minor releases is a real and recurring problem.
  2. Prefab packaging script is out of meson scope. It lives as a shell script under scripts/package-android-aar.sh and is not part of meson test. Document clearly that the .aar is not produced by ninja install.
  3. 16KB page alignment regressions. Add a CI check that runs readelf -l libwirelog.so | grep LOAD on the Android build and asserts alignment ≥ 0x4000. Cheap and prevents silent Play Store rejections.
  4. JNI thread attach leaks. The adapter author is responsible for balancing AttachCurrentThread/DetachCurrentThread. docs/android.md must show the paired-call pattern and warn about leaked JNI env refs.
  5. Asset-backed file paths conflict with the "always absolute path" recommendation. android_asset://events.csv is not a filesystem path; the adapter interprets it. Document that the "absolute path" guidance applies only to the built-in csv adapter, not to user adapters using custom URI schemes.

7c. Path C (continued) — iOS support

iOS is a first-class Path A target and is explicitly not a Path B target. The analysis below enumerates iOS-specific constraints and the PR #1 deliverables needed to unblock iOS app developers. Structure parallels 7b (Android) but the underlying constraints are different enough that this section is fully self-contained.

7c.1 Why Path A is the only viable path on iOS

App Store Review Guideline 2.5.2 prohibits downloading, interpreting, or executing code not embedded in the originally submitted binary. This eliminates any dlopen-based plugin flow for App Store apps. Technically, dlopen on frameworks already inside the .ipa bundle is not syscall-blocked — the iOS dynamic linker permits it — but Apple's review process rejects apps that load adapter code from non-Apple sources at runtime. For wirelog's "user-written adapter" use case the distinction does not matter: the adapter must be statically compiled into the app binary or into a statically linked framework.

Furthermore, there is no wl CLI on iOS (no shell entry point in a normal app) and no command-line invocation surface. The --load-adapter flag has nowhere to run. Conclusion: on iOS, adapter registration is always via wl_io_register_adapter() called from the app's startup code — typically AppDelegate.application(_:didFinishLaunchingWithOptions:), an @main App initializer, or a dedicated +load method. The adapter .c is statically compiled into the app's own binary or framework. No dynamic loading, no Path B variant.

7c.2 Distribution format — static XCFramework via Swift Package Manager

The recommended distribution artifact is a static-library XCFramework:

wirelog.xcframework/
  ios-arm64/                              libwirelog.a  Headers/
  ios-arm64_x86_64-simulator/             libwirelog.a  Headers/   (fat slice)

Static .a slices are preferred over dynamic frameworks for three reasons:

  1. No dyld load-time overhead — startup cost is zero.
  2. No embedded code-signing complications — dynamic frameworks must be re-signed when embedded in an app and add complexity to the .ipa signing flow.
  3. Zero App Store rejection risk from dynamic library policies — static linkage is unambiguously compliant.

Swift Package Manager is the primary consumer, using Swift 5.3+ binary targets (shipped with Xcode 12 in 2020):

// Package.swift consumer-side
.binaryTarget(
    name: "wirelog",
    url: "https://github.com/justinjoy/wirelog/releases/download/vX.Y.Z/wirelog.xcframework.zip",
    checksum: "sha256-..."
)

CocoaPods remains supported as a secondary path via a vendored podspec that ships a prebuilt libwirelog.a plus the public headers. Target audience: projects that have not migrated to SPM.

A source-only SPM C target is technically feasible (point SPM at the C source tree and let it compile) but forces every integrating app to re-compile wirelog from source, which is unattractive for a Meson-built C11 library with NEON gating and per-architecture cross-files. Reserve as a fallback, not the primary path.

7c.3 Meson cross-compilation for iOS

Minimum deployment target: iOS 13. Rationale: C11, pthread, and the full POSIX subset wirelog relies on are solid; coverage is ~98% of active devices as of 2025; Swift binary targets require iOS 13+ anyway.

cross/ios-arm64.ini:

[binaries]
c = 'clang'
ar = 'ar'

[built-in options]
c_args = ['-target', 'arm64-apple-ios13.0',
          '-isysroot',
          '/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk']
c_link_args = ['-target', 'arm64-apple-ios13.0']

[host_machine]
system = 'darwin'
cpu_family = 'aarch64'
cpu = 'arm64'
endian = 'little'

cross/ios-simulator-arm64.ini uses -target arm64-apple-ios13.0-simulator and iPhoneSimulator.sdk. cross/ios-simulator-x86_64.ini uses -target x86_64-apple-ios13.0-simulator (needed for Intel Mac developers).

Bitcode: deprecated since Xcode 14 (2022). Do not enable it. Explicitly omit any -fembed-bitcode flags from the cross-files. Stale ENABLE_BITCODE=YES in user projects causes hard-to-diagnose build failures — docs/ios.md must warn about this.

arm64e (Pointer Authentication, Apple A12+): Apple-internal ABI only for third-party developers. Skip.

New meson option parallel to android:

# meson_options.txt
option('ios', type : 'boolean', value : false,
       description : 'Build for iOS (disables CLI target, enables iOS-specific link flags)')

In wirelog/meson.build, guard the wirelog_cli executable target on not (get_option('android') or get_option('ios')). iOS has no CLI entry point.

Also under -Dios=true, ensure the build does not apply -fvisibility=hidden to libwirelog.a globally. Hidden symbols are invisible to Swift's Clang module importer and cause link-time failures in SPM consumers. If wirelog/meson.build sets hidden visibility for release builds, add an override when -Dios=true or -Dandroid=true.

7c.4 Swift and Objective-C interop via Clang module map

wirelog is pure C11 with zero C++ sources, so Swift imports it directly via a Clang module map. Ship wirelog/module.modulemap inside each XCFramework slice's Headers/ directory:

// wirelog/module.modulemap
module wirelog {
    umbrella header "wirelog.h"
    header "io/io_adapter.h"
    export *
    link "wirelog"
}

The export * directive makes all C symbols visible to Swift. The link "wirelog" line causes Swift to emit the correct -lwirelog linker directive automatically.

Swift-side registration pattern, using the user_data field defined in section 3:

import wirelog
import Foundation

class BundleAdapterState {
    let bundle: Bundle
    init(_ b: Bundle) { self.bundle = b }
}

// @convention(c) function — cannot capture; recovers state from the
// trailing user_data argument (standard C callback convention).
let bundleRead: @convention(c) (UnsafeMutablePointer<wl_io_ctx_t>?,
                                UnsafeMutablePointer<UnsafeMutablePointer<Int64>?>?,
                                UnsafeMutablePointer<UInt32>?,
                                UnsafeMutableRawPointer?) -> Int32 = {
    ctx, outData, outNrows, userData in
    let state = Unmanaged<BundleAdapterState>.fromOpaque(userData!).takeUnretainedValue()
    // ... use state.bundle to locate the fact file, read rows, fill outData ...
    return 0
}

@main
struct MyApp: App {
    init() {
        let state = BundleAdapterState(.main)
        var adapter = wl_io_adapter_t()
        adapter.abi_version = WL_IO_ABI_VERSION
        adapter.scheme      = ("bundle" as NSString).utf8String
        adapter.read        = bundleRead
        adapter.user_data   = Unmanaged.passUnretained(state).toOpaque()
        _ = wl_io_register_adapter(&adapter)
    }
    var body: some Scene { WindowGroup { ContentView() } }
}

Without the user_data field defined in section 3, this pattern is impossible in Swift. This is why user_data is a mandatory day-one addition, not a follow-up.

7c.5 File system and sandbox

iOS apps are strictly sandboxed. All paths live under NSHomeDirectory():

Subdirectory Purpose Backed up to iCloud
Documents/ User-visible data Yes
Library/Caches/ Regenerable caches No
Library/Application Support/ App-internal state Yes
tmp/ (NSTemporaryDirectory()) Transient scratch No
.app bundle contents Read-only resources N/A

There is no /tmp analogous to Unix, and there is no cwd-relative file access — every relative path is resolved against an undefined base. Rule: resolve absolute paths on the Swift/ObjC side before passing them into wirelog. Example:

guard let url = Bundle.main.url(forResource: "facts", withExtension: "csv") else { return }
let absolutePath = url.path  // /var/containers/.../MyApp.app/facts.csv
// Pass absolutePath into the .input directive or to wl_session_load_input_files

Bundle resources are already absolute file paths, so the built-in CSV adapter works unchanged for reading facts shipped inside the .app — no iOS analogue to Android's android_asset:// reference adapter is needed. The built-in CSV path-resolution logic (currently in wirelog/session_facts.c:89-113, moving into wirelog/io/csv_adapter.c as part of section 5) is effectively a no-op on iOS because the caller always passes absolute paths.

For custom formats — e.g., an encrypted fact store using CommonCrypto — users write a regular adapter with no iOS-specific core change.

7c.6 Threading

wirelog's wl_session_load_input_files is synchronous and contains no main-thread assertions. iOS-specific thread analysis matches Android:

  • Never call wirelog from the main dispatch queue. Blocking the main queue freezes the UI and can trigger watchdog kills.
  • Recommended pattern with Swift concurrency:
Task.detached(priority: .userInitiated) {
    let rc = wl_session_load_input_files(session, program)
    await MainActor.run { /* update UI with results */ }
}
  • Recommended pattern with GCD:
DispatchQueue.global(qos: .userInitiated).async {
    _ = wl_session_load_input_files(session, program)
    DispatchQueue.main.async { /* update UI */ }
}
  • Objective-C++ / Objective-C callers use dispatch_async(dispatch_get_global_queue(QOS_CLASS_USER_INITIATED, 0), ^{ ... }).

No core change is needed to support iOS threading models — the existing wirelog threading model (the caller picks the thread) works unchanged.

7c.7 Symbol visibility and dead-stripping

Two risks specific to static-library iOS builds:

  1. Dead-stripping of wl_io_register_adapter. If an app uses only the built-in CSV adapter and never explicitly calls wl_io_register_adapter, the static-library linker may strip that symbol and the surrounding registry infrastructure. Mitigations:

    • Mark registration-path symbols with __attribute__((used)) in wirelog/io/io_adapter.c.
    • Or maintain an -exported_symbols_list file consumed by the XCFramework packaging script.
    • Recommended: __attribute__((used)) in the source, because it is automatic for any downstream packaging scheme.

  2. Hidden visibility breaking Swift imports. If wirelog/meson.build applies -fvisibility=hidden in release builds, those hidden symbols are invisible to Swift's Clang module importer and cause SPM link-time failures. Audit meson.build and add -fvisibility=default override when -Dios=true (and likewise for -Dandroid=true to be consistent). Document the convention: iOS and Android builds use default visibility; desktop Linux/macOS may use hidden visibility for unexported symbols if the build opts in.

7c.8 PR #1 iOS deliverables

Included in PR #1:

  • meson_options.txt — new ios boolean option (default false)
  • wirelog/meson.build — guard wirelog_cli on not (get_option('android') or get_option('ios')); apply iOS-appropriate c_link_args under -Dios=true; disable -fvisibility=hidden under -Dios=true
  • wirelog/io/io_adapter.c — add __attribute__((used)) to wl_io_register_adapter, wl_io_unregister_adapter, wl_io_find_adapter, and the built-in CSV adapter bootstrap function
  • cross/ios-arm64.ini, cross/ios-simulator-arm64.ini, cross/ios-simulator-x86_64.ini — meson cross-files
  • wirelog/module.modulemap — Clang module map shipped inside each XCFramework slice's Headers/ directory
  • scripts/package-ios-xcframework.sh — out-of-meson packaging script that runs three cross-compiles (ios-arm64, ios-simulator-arm64, ios-simulator-x86_64), assembles a fat simulator slice via lipo, then wraps everything in an XCFramework via xcodebuild -create-xcframework
  • docs/ios.md — integration guide: SPM binary target consumption, Swift import, @convention(c) closure pattern with user_data, sandbox path handling, threading recommendations, App Store policy notes
  • CI: ios-arm64 cross-compile build smoke test using a pinned Xcode version; no simulator runtime (CI cost is prohibitive and a compile-clean check catches >90% of regressions)

Deferred beyond PR #1:

  • macOS Catalyst: plausible near-zero additional cost once the XCFramework pipeline exists. Add ios-arm64_x86_64-maccatalyst slice as a follow-up PR.
  • visionOS (xrOS): Apple's newest platform (2024+). Defer until there is concrete user demand — no current evidence of it.
  • Prebuilt XCFramework publication to a Swift Package Registry or GitHub Release binary asset. Requires a release automation pipeline that does not exist yet.
  • examples/ios/encrypted_adapter/ — optional reference showing a CommonCrypto-backed custom URI scheme using the user_data slot. Nice-to-have, not gate.
  • Full CI runs on iOS simulator — expensive and premature.

7c.9 iOS-specific risks and mitigations

  1. Xcode version drift and SDK minimum bumps. Apple deprecates iOS deployment minimums every one to two years and ships new SDKs with behavioral changes. Pin the Xcode version used by the CI cross-compile job. Plan to re-audit the minimum deployment target annually. Severity: Moderate.
  2. Simulator architecture split. Apple Silicon Macs run the arm64 simulator; Intel Macs run the x86_64 simulator. The XCFramework packaging script must produce a fat simulator slice combining both — shipping only one breaks half the developer base. Severity: Major; mitigated by lipo in the packaging script.
  3. Static library dead-stripping of the registry. See 7c.7; mitigated by __attribute__((used)) on registration-path symbols. Severity: Major without the fix, trivial with it.
  4. App Store rejection for any perceived dynamic loading. Reviewers may flag apps that even contain dlopen symbols. Ensure wirelog/cli/plugin_loader.c is excluded from iOS builds by guarding it on not (get_option('ios') or get_option('android')) in wirelog/cli/meson.build. The plugin loader must not contribute any symbols to libwirelog.a on iOS. Severity: Critical if violated, trivial to guard.
  5. Privacy manifest (PrivacyInfo.xcprivacy, mandatory since 2024). wirelog's core path does not use any of Apple's "required reason APIs" — no file timestamps, no UserDefaults, no system boot time, no disk space queries. No wirelog-level PrivacyInfo.xcprivacy entry is required. However, docs/ios.md must clearly state that apps using wirelog with adapters that access those APIs (e.g., a user adapter reading file timestamps) must declare the reason in their own app-level privacy manifest. Severity: Informational for wirelog, Major for app developers who overlook it.
  6. Code signing of dynamic symbols. Although this design uses static linkage (avoiding the issue), any future shift to dynamic framework distribution must re-sign the framework when embedded. Noted for posterity; not a PR Restructure CI pipeline: staged workflow with static analysis reports #1 risk. Severity: N/A for PR Restructure CI pipeline: staged workflow with static analysis reports #1.
  7. arm64e ABI exposure. Pointer Authentication is an Apple-internal ABI; third-party apps cannot ship arm64e binaries. Do not include an arm64e slice in the XCFramework. Severity: Trivial; just don't add the slice.

8. Thread safety

  • Registry state: static wl_io_adapter_t const *s_registry[WL_IO_MAX_ADAPTERS]; guarded by a single wl_mutex_t from wirelog/thread.h.
  • wl_io_register_adapter() / wl_io_unregister_adapter() take the lock.
  • wl_io_find_adapter() takes the lock briefly, returns the stable pointer (adapters must outlive their registration, which is a documented caller contract).
  • Built-in registration uses a one-shot flag behind the same mutex. No pthread_once per se — a guarded if (!s_initialized) inside the locked critical section is sufficient and portable across the existing thread_posix.c / thread_msvc.c split.
  • .input load runs on the main thread before any worker steps, so there is no contention during normal operation.
  • wl_io_ctx_intern_string() currently relies on wl_intern_put() not being thread-safe. This is fine for Path A/B because load is single-threaded; a header comment makes this explicit. Streaming/multi-thread ingestion is out of scope.

9. File plan

New files

Path Purpose
wirelog/io/io_adapter.h Public-internal header: vtable, registration API, opaque context, error codes
wirelog/io/io_adapter.c Registry storage, mutex, find/register/unregister, built-in bootstrap
wirelog/io/io_ctx.c wl_io_ctx_t impl + accessors (relation name, num_cols, col_type, param, intern_string)
wirelog/io/csv_adapter.c Built-in csv adapter wrapping existing wl_csv_read_file{,_ex}()
wirelog/cli/plugin_loader.c (conditional on -Dio_plugin_dlopen) dlopen + dlsym + register
tests/test_io_adapter.c Registry unit tests (register / find / unknown / duplicate / reset)
tests/test_io_ctx.c Context accessor unit tests + mock adapter end-to-end through .input
docs/io-adapters.md User guide: Path A embedding example, Path B CLI plugin contract, ABI versioning policy
docs/android.md Android integration guide (Prefab .aar, JNI_OnLoad, path handling, threading, 16KB alignment)
cross/android-arm64.ini, cross/android-armv7.ini, cross/android-x86_64.ini NDK meson cross-files, one per ABI
examples/android/asset_adapter/asset_adapter.c, CMakeLists.txt, README.md Reference AAssetManager-backed adapter for APK-embedded facts
cross/ios-arm64.ini, cross/ios-simulator-arm64.ini, cross/ios-simulator-x86_64.ini iOS meson cross-files: device, Apple Silicon simulator, Intel simulator
wirelog/module.modulemap Clang module map bundled inside each XCFramework slice's Headers/ dir; enables Swift import wirelog
scripts/package-ios-xcframework.sh Out-of-meson packaging script: runs three cross-compiles, lipo-fats the simulator slices, runs xcodebuild -create-xcframework
docs/ios.md iOS integration guide: SPM binary target consumption, Swift @convention(c) pattern with trailing user_data, sandbox path handling, threading, App Store policy notes, bitcode warning

Modified files

Path Change
wirelog/ir/program.h (around line 34, append to wl_ir_relation_info_t) Add char *input_io_scheme;
wirelog/ir/program.c Populate input_io_scheme in metadata collection; free in wl_ir_program_free(); audit the relations grow path for shallow-copy safety (mirror existing input_param_names handling)
wirelog/session_facts.c:51-181 Replace hard-coded CSV block with adapter dispatch; #include "io/io_adapter.h"; drop direct csv_reader.h include
wirelog/io/csv_reader.c / .h No change to existing functions. Possibly add a wl_csv_read_file_via_ctx() variant that takes a (void *ctx, int64_t (*intern_cb)(void*, const char*)) pair to keep wl_intern_t out of the adapter interface
wirelog/cli/driver.c (around line 399) Conditional (#ifdef WL_IO_PLUGIN_DLOPEN) --load-adapter CLI flag handling; no change to wl_session_load_input_files() call
wirelog/meson.build (around line 80 — wirelog_io_src) Add io_adapter.c, io_ctx.c, csv_adapter.c
wirelog/meson.build New option io_plugin_dlopen (boolean, default false); conditional -ldl and plugin_loader.c inclusion in CLI exe
wirelog/meson.build Guard wirelog_cli target behind not (get_option('android') or get_option('ios')); append -Wl,-z,max-page-size=16384 to libwirelog link_args when -Dandroid=true; disable -fvisibility=hidden when -Dios=true or -Dandroid=true; apply iOS -target link flags under -Dios=true
wirelog/cli/meson.build Guard plugin_loader.c on not (get_option('ios') or get_option('android')) so that dlopen-related symbols never appear in the iOS or Android libwirelog.a/libwirelog.so
wirelog/io/io_adapter.c Annotate wl_io_register_adapter, wl_io_unregister_adapter, wl_io_find_adapter, and the built-in CSV adapter bootstrap function with __attribute__((used)) so iOS static-library dead-stripping cannot remove them when an app uses only the built-in CSV adapter
meson_options.txt Declare io_plugin_dlopen option; declare android boolean option; declare ios boolean option
tests/meson.build New test_io_adapter and test_io_ctx executables; add new sources to ir_src if indirect dependencies require

Public headers

io_adapter.h is installed under $includedir/wirelog/io/ so that Path A users can #include <wirelog/io/io_adapter.h>. It is the only new exported header. No changes to wirelog.h.

10. Atomic commit plan (TDD per CLAUDE.md)

Each commit must compile and pass the full test suite.

  1. test(#366): Add failing test_io_adapter scaffold
    Creates tests/test_io_adapter.c with the five registry scenarios; wires tests/meson.build. Tests fail to compile → gated behind commit 2 or kept #if 0 until commit 2 lands.
  2. feat(#366): Introduce wl_io_adapter_t interface and registry
    Adds io_adapter.h, io_adapter.c, io_ctx.c (accessors only, no intern yet). Meson wiring. test_io_adapter passes.
  3. feat(#366): Add wl_io_ctx_intern_string accessor
    Wires the intern-table wrapper behind the opaque context. New unit tests in test_io_ctx.c cover string and integer paths.
  4. feat(#366): Built-in CSV adapter auto-registration
    New csv_adapter.c; registers on first lookup. At this point the CSV adapter coexists with the legacy session_facts.c path but is not yet used.
  5. refactor(#366): Dispatch .input loading through io adapter registry
    Adds input_io_scheme to wl_ir_relation_info_t, populates / frees it, rewrites wl_session_load_input_files() to dispatch via wl_io_find_adapter(). Moves path resolution into csv_adapter.c. Existing test_cli .input cases (lines 352, 431, 467 in tests/test_cli.c) are the regression gate.
  6. feat(#366): Optional CLI --load-adapter via dlopen (gated)
    Adds plugin_loader.c, meson option io_plugin_dlopen, CLI flag parsing, ABI-version check. Default build does not link it; CI adds one job with the option enabled plus a sample adapter .so.
  7. docs(#366): I/O adapter user guide
    docs/io-adapters.md with Path A worked example (pcap skeleton, with the compile command) and Path B contract (wl_io_plugin_entry signature, ABI policy).
  8. feat(#366): Android NDK cross-build support and platform_ctx slot
    Adds the three cross/android-*.ini files, meson_options.txt android boolean, wirelog_cli guard, 16KB page alignment link_args, and the wl_io_ctx_platform/wl_io_ctx_set_platform accessors on wl_io_ctx_t. CI smoke job cross-compiles arm64-v8a.
  9. docs(#366): Android integration guide and reference asset adapter
    Adds docs/android.md and the examples/android/asset_adapter/ reference implementation (AAssetManager-backed, compile-only in CI, no emulator run).
  10. feat(#366): iOS cross-build support and Clang module map
    Adds the three cross/ios-*.ini files, meson_options.txt ios boolean, wirelog_cli and plugin_loader.c guards, iOS -target / visibility link flags, __attribute__((used)) annotations on registry symbols, and wirelog/module.modulemap. CI smoke job cross-compiles ios-arm64.
  11. docs(#366): iOS integration guide and XCFramework packaging script
    Adds docs/ios.md and scripts/package-ios-xcframework.sh. The packaging script runs three cross-compiles, lipo-fats the simulator slices, and calls xcodebuild -create-xcframework. Packaging is not part of meson test / ninja install; documented explicitly.

11. Test plan

New tests

  • tests/test_io_adapter.c
    • test_register_and_find: register scheme, find returns same pointer
    • test_find_unknown_scheme: returns NULL, sets wl_io_last_error()
    • test_duplicate_register_error: second register with same scheme fails
    • test_builtin_csv_autoload: find("csv") succeeds before any user registration
    • test_unregister_then_find: after unregister, find returns NULL; re-register works
    • test_abi_version_mismatch: adapter with wrong abi_version is rejected
  • tests/test_io_ctx.c
    • Mock adapter that asserts num_cols, col_type, param lookup, relation_name
    • String-interning path: mock emits a STRING column, verify int64 id round-trips through the session snapshot
    • End-to-end: parse a small program with .input R(io="mock", key="val"), load, snapshot, verify derived tuples
  • (Path B only, conditional) tests/test_plugin_loader.c
    • Build a tiny libmock_adapter.so as a meson target, load via plugin_loader.c, register, use

Regression gates

  • tests/test_csv.c — all existing cases must pass unchanged (CSV reader internals untouched)
  • tests/test_cli.c — .input-related cases (test_run_pipeline_csv_input ~L352, test_run_pipeline_csv_missing_file ~L431, test_run_pipeline_csv_tab_delimiter ~L467) must stay green
  • tests/test_program.c — .input metadata collection still produces the expected params plus the new input_io_scheme field
  • Full meson test -C build must remain ≥ current passing count

TSan: commit 2 onwards must run cleanly under -Db_sanitize=thread; new registry mutex is exercised from the main thread only but the sanitizer catches accidental worker-thread access.

12. Risks and mitigations

  1. Intern ABI leakage (was the critic's Critical). Mitigated by the opaque wl_io_ctx_t and wl_io_ctx_intern_string() wrapper. wl_intern_t * is never visible to adapters. If the intern implementation changes, only io_ctx.c and the CSV reader variant need updating.
  2. Batch buffer ownership traps. Documented contract: adapter malloc, wirelog free. Encoded in the read function's doc comment and enforced by test_io_ctx (leak check under ASan). A follow-up PR may introduce an arena-based variant if real adapters need it.
  3. Registry init order / thread safety. Guarded by a single mutex plus a one-shot built-in registration flag. No C file-scope constructors relied upon. Registration is permitted from any thread; .input load is main-thread-only, matching current behavior.
  4. input_io_scheme double-free on relations grow. Explicit audit of wirelog/ir/program.c grow path is part of commit 5. Mirrors the existing input_param_names handling which already does this correctly.
  5. filename= vs future source= collision. Resolved by the "always route by io=, never sniff" rule documented in section 4.
  6. dlopen surface inviting scope creep. Mitigated by (a) compile-time gate off by default, (b) single export symbol contract (wl_io_plugin_entry), (c) ABI version check, (d) explicit documentation that plugin loading is a CLI-only convenience, not a general extension mechanism.
  7. Backward compatibility for existing Datalog programs. All existing .input(filename="x.csv") usages continue to work unchanged because no io= means csv and the built-in CSV adapter re-uses wl_csv_read_file{,_ex}() unchanged.
  8. Binary size. Estimated .text growth: registry ~1KB, context accessors ~0.7KB, CSV adapter wrapper ~0.8KB, total ≲ 3KB — well under the 5KB budget. plugin_loader.c is excluded from libwirelog.so entirely.

13. Out of scope (explicit)

  • eBPF / pcap / kafka concrete adapter implementations (each is its own follow-up).
  • Streaming / auto-step / back-pressure (WL_IO_AGAIN, periodic callback). Reserved but not implemented.
  • TTL / sliding windows / retraction via adapters.
  • Batch / arena-based buffer ownership model.
  • Multi-adapter chaining or filters.
  • Schema inference from the source.
  • Cross-thread adapter callbacks; intern table is documented as main-thread-only for now.
  • Public ABI stability guarantee across minor versions — WL_IO_ABI_VERSION bumps are permitted and the CLI plugin loader will refuse to load mismatched plugins.
  • Runtime plugin discovery (scanning a directory, env-var search path, etc.). --load-adapter=PATH is the only sanctioned loading entry point.
  • Android: prebuilt .aar publication to Maven Central / JitPack, full emulator-based test runs in CI, adapters beyond the reference android_asset example. Build smoke test on NDK cross-compile is included in PR Restructure CI pipeline: staged workflow with static analysis reports #1; runtime verification is deferred.
  • Android Path B (dlopen-based plugins): permanently out of scope on Android due to linker namespace restrictions. Documented in docs/android.md.

14. Acceptance criteria

  • meson test -C build passes with at least the current test count plus the new test_io_adapter and test_io_ctx executables.
  • meson test -C build-tsan (TSan build) passes.
  • meson test -C build-asan (ASan build) passes, including the new buffer-ownership leak checks.
  • A Path A worked example in docs/io-adapters.md compiles and runs against the installed libwirelog without rebuilding wirelog.
  • With -Dio_plugin_dlopen=true, a sample adapter .so loads via wl --load-adapter=... and processes a .input directive end-to-end.
  • libwirelog.so size growth verified ≤ 5KB against the pre-PR baseline.
  • Existing .input(filename="...") programs produce byte-identical snapshots before and after the refactor.
  • meson setup build-android --cross-file cross/android-arm64.ini -Dandroid=true && ninja -C build-android libwirelog.so succeeds in CI with a pinned NDK version.
  • readelf -l build-android/wirelog/libwirelog.so | grep LOAD shows alignment ≥ 0x4000 (16KB page size).
  • examples/android/asset_adapter/ compiles against the cross-built libwirelog.so headers without errors.
  • meson setup build-ios --cross-file cross/ios-arm64.ini -Dios=true && ninja -C build-ios libwirelog.a succeeds in CI with a pinned Xcode version.
  • nm build-ios/wirelog/libwirelog.a | grep wl_io_register_adapter confirms the symbol is present and not dead-stripped (i.e., __attribute__((used)) is effective).
  • otool -hv build-ios/wirelog/libwirelog.a 2>/dev/null || file build-ios/wirelog/libwirelog.a confirms the archive contains arm64 Mach-O objects.
  • A sample Swift source file referencing wl_io_adapter_t, wl_io_register_adapter, and a @convention(c) read function with the trailing user_data parameter compiles cleanly against wirelog/module.modulemap (verified by an xcrun swiftc -parse check in CI).
  • scripts/package-ios-xcframework.sh produces a valid wirelog.xcframework directory whose Info.plist lists both the ios-arm64 and ios-arm64_x86_64-simulator slices.

15. Open questions for reviewers

  • Is the io_plugin_dlopen meson option name acceptable, or prefer cli_plugin_loader?
  • Should wl_io_plugin_entry return a const array of adapter pointers (current proposal) or take a registration callback void (*register)(const wl_io_adapter_t *) that the plugin invokes? The latter is marginally more flexible for plugins exposing multiple adapters with conditional registration.
  • Should docs/io-adapters.md live under docs/ or under a new docs/extending/ subtree, given further extension docs are planned?
  • Are we comfortable installing wirelog/io/io_adapter.h as a public header now, or should it ship under a wirelog-experimental/ prefix for the first release?

16. Task breakdown, sub-issues, and execution order

This issue is the umbrella for 24 concrete sub-issues, produced jointly by the architect and the critic. The list below is the adversarially-reviewed consolidation: it flips TDD ordering where vanilla plans fail, isolates the session_facts.c dispatch rewrite as the single rollback unit, and adds seven [NEW] tasks that a vanilla plan typically misses (header install wiring, ASan/TSan CI jobs, binary-size regression gate, installed-headers example compile, Android 16KB alignment hard gate, iOS symbol + Swift parse gate, ABI symbol golden file).

16.1 Task list

Issue Title Cat Depends on Block Size
#448 Pin baseline metrics and .input fixture snapshots Foundation — Y S
#449 Install path + WL_PUBLIC visibility macro foundation Foundation — Y S
#450 Failing adapter registry test scaffold (TDD gate) Foundation #449 Y S
#451 wl_io_adapter_t header + registry implementation Foundation #450 Y M
#452 Failing context accessor + intern tests (TDD gate) Foundation #451 Y S
#453 wl_io_ctx_t accessors + intern wrapper + platform_ctx slot Foundation #452 Y M
#454 [NEW] IR field input_io_scheme — isolated commit + ASan grow audit IR #453 Y S
#455 [NEW] Refactor wl_csv_read_file_ex to intern-callback shape CSV #453 Y M
#456 Built-in CSV adapter + auto-registration CSV #454, #455 Y M
#457 Failing .input dispatch test (TDD gate) Integration #456 Y S
#458 session_facts.c dispatch rewrite (rollback unit) Integration #457 Y M
#459 [NEW] ASan + TSan CI gates as discrete jobs CI #458 Y S
#460 [NEW] Binary-size regression gate CI #458 N S
#461 Optional CLI plugin loader (gated, Path B) CLI #458 N M
#462 [NEW] Path A docs example compile-check in CI CI #458 N S
#463 docs/io-adapters.md user guide + CHANGELOG entry Docs #451 N M
#464 Android meson plumbing + NDK cross-files Android #458 N M
#465 [NEW] Android CI smoke + 16KB alignment hard gate CI #464 Y S
#466 Android docs + asset_adapter reference example Docs #464 N M
#467 iOS meson plumbing + cross-files + Clang module map iOS #464 N M
#468 [NEW] iOS CI smoke + symbol + Swift parse gate CI #467 Y M
#469 iOS XCFramework packaging script iOS #467 N L
#470 docs/ios.md integration guide Docs #467 N S
#471 [NEW] CHANGELOG + release notes + open-question resolution + ABI symbol golden file Docs #458, #461, #464, #467 Y S

Sizing: S ≈ 0.5d, M ≈ 1d, L ≈ 2–3d. Blocking=Y means downstream tasks are hard-blocked and cannot proceed with mocks. [NEW] flags tasks a vanilla plan typically misses.

16.2 Execution order (Mermaid dependency graph)

flowchart TD
    classDef foundation fill:#e1f5ff,stroke:#0366d6,color:#000
    classDef csv fill:#fff5e6,stroke:#d19a00,color:#000
    classDef ir fill:#f0e6ff,stroke:#6f42c1,color:#000
    classDef integ fill:#ffe6e6,stroke:#d73a49,color:#000
    classDef ci fill:#e6ffe6,stroke:#28a745,color:#000
    classDef cli fill:#f6f8fa,stroke:#586069,color:#000
    classDef android fill:#e6f9e6,stroke:#2ea44f,color:#000
    classDef ios fill:#e6f0ff,stroke:#0052cc,color:#000
    classDef docs fill:#fff0f5,stroke:#b3005c,color:#000

    I448["#448<br/>Baseline snapshots"]:::foundation
    I449["#449<br/>Install + WL_PUBLIC"]:::foundation
    I450["#450<br/>Registry test scaffold"]:::foundation
    I451["#451<br/>Registry impl"]:::foundation
    I452["#452<br/>Ctx test scaffold"]:::foundation
    I453["#453<br/>Ctx + platform_ctx"]:::foundation
    I454["#454<br/>IR input_io_scheme"]:::ir
    I455["#455<br/>csv_reader callback"]:::csv
    I456["#456<br/>Built-in CSV adapter"]:::csv
    I457["#457<br/>Dispatch test"]:::integ
    I458["#458<br/>session_facts rewrite<br/>ROLLBACK UNIT"]:::integ
    I459["#459<br/>ASan + TSan CI"]:::ci
    I460["#460<br/>Binary-size gate"]:::ci
    I461["#461<br/>CLI plugin loader"]:::cli
    I462["#462<br/>Path A example CI"]:::ci
    I463["#463<br/>io-adapters.md"]:::docs
    I464["#464<br/>Android meson"]:::android
    I465["#465<br/>Android 16KB gate"]:::ci
    I466["#466<br/>Android docs + asset"]:::docs
    I467["#467<br/>iOS meson + modulemap"]:::ios
    I468["#468<br/>iOS CI + Swift parse"]:::ci
    I469["#469<br/>XCFramework script"]:::ios
    I470["#470<br/>docs/ios.md"]:::docs
    I471["#471<br/>CHANGELOG + ABI golden"]:::docs

    I449 --> I450 --> I451 --> I452 --> I453
    I453 --> I454
    I453 --> I455
    I454 --> I456
    I455 --> I456
    I456 --> I457 --> I458
    I448 -.baseline.-> I458
    I451 --> I463

    I458 --> I459
    I458 --> I460
    I458 --> I461
    I458 --> I462
    I458 --> I464

    I464 --> I465
    I464 --> I466
    I464 --> I467

    I467 --> I468
    I467 --> I469
    I467 --> I470

    I458 --> I471
    I461 --> I471
    I464 --> I471
    I467 --> I471
Loading

16.3 Critical path and parallelization strategy

Strict critical path: #449 → #450 → #451 → #452 → #453 → {#454, #455} → #456 → #457 → #458 → #459 → #471. Everything before #458 is strictly sequential (Foundation layer + IR field + CSV callback refactor + CSV adapter + dispatch test). #458 is the rollback unit: it is the only behavioral change and must be revertible without unwinding #451–#456. Every task before #458 is additive (new files, new tests, no behavior change).

#448 (baseline fixture snapshots) is the rollback comparator. It runs in parallel with the entire Foundation chain but must land before #458 so that byte-identical regression can be asserted. It has no hard code dependency — the dashed baseline edge into #458 represents the comparison gate, not a build dependency.

#463 (docs/io-adapters.md) can start immediately after #451 because the public header is stable at that point. This is a parallelization win over a naive "docs last" sequencing.

First fan-out point — after #458. Five parallel tracks open up:

Second fan-out point — after #464. Inside Android, #465 (16KB alignment gate) and #466 (docs + example) run in parallel. Inside iOS (which rebases on #464's meson changes), #468 (CI), #469 (packaging script), and #470 (docs) run in parallel — they share no files.

Why #464 → #467 is a serialization, not a dependency: Android and iOS both edit the same wirelog/meson.build lines (the wirelog_cli guard and meson_options.txt). Running them in parallel guarantees a manual merge conflict. Serialize: Android first, iOS rebases and extends the guard to not (get_option('android') or get_option('ios')). This is noted explicitly because the plan would otherwise look like two independent parallel tracks.

Merge gate at #471. CHANGELOG, release notes, ABI symbol golden file, and resolution of the section 15 open questions all land here. #471 is the release gate and depends on #458 (core), #461 (optional CLI plugin), #464 (Android), #467 (iOS).

Wall-clock estimate with a single developer: ~14 days. With three developers assigned to CLI / Android / iOS after #459 green, post-#458 phase collapses to ~5 days wall-clock, for a total of ~9 days.

Refs: discussion #366 · wirelog/session_facts.c:51-181 · wirelog/ir/program.h:31-45 · wirelog/parser/parser.c:1150-1217 · wirelog/io/csv_reader.{h,c} · wirelog/cli/driver.c:399 · wirelog/meson.build:80

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions