Phase 3: TLS support via AmiSSL

[sidick]

Add TLS as a third mqtt_transport implementation, per docs/ARCHITECTURE.md's roadmap:

• AmiSSL 5.x transport behind an MQTT_TLS tag; soft-fails when amissl.library is absent.
• An OpenSSL host transport for test parity (host unit/smoke tests against a TLS broker).

See docs/ARCHITECTURE.md ("Roadmap" section, Phase 3) and src/core/mqtt_transport.h (the vtable seam this plugs into — do not bypass it).

[sidick]

TLS support via AmiSSL (issue #3)

Context

midge is about to move past v0.1. Issue #3 tracks TLS, the first item on the
post-release roadmap (docs/ARCHITECTURE.md's old Phase 3, now tracked as a
GitHub issue per the "track future work via issues" convention). The project
owner's constraint: TLS must be opt-in, off by default, because most
Amigas (68020/68030-era CPUs) are too slow for a TLS handshake to be a sane
default — it must never be silently attempted or silently preferred over
plaintext.

There's a working reference to build from: ~/src/micropython/ports/amiga/
(amiga_ssl.c/.h + modssl.c) integrates AmiSSL v5 for MicroPython on
Amiga. It proves the AmiSSL v5 "master library" open sequence and the
OpenSSL-shaped API surface, but its I/O model (a BIO_new_bio_pair() pump
for asyncio non-blocking integration) doesn't fit midge, which uses a much
simpler synchronous polling model (WaitSelect(1s) then a blocking call —
see src/amiga/transport_bsdsocket.c). The plan below adapts the library
lifecycle and verification parts of that reference and designs midge's own
I/O shape.

TLS plugs into midge's existing mqtt_transport vtable
(src/core/mqtt_transport.h) — docs/ARCHITECTURE.md already commits to
"adds AmiSSL and OpenSSL implementations behind the same vtable; mqtt_client
doesn't change at all," and this plan honors that: zero changes to
src/core.

Decisions already made with the project owner:

• TLS is opt-in per connection, default off, everywhere (CLI tools and
  mqtt.library).
• Certificate verification defaults on (AmiSSL's bundled CA trust store)
  when TLS is enabled, with an explicit --insecure-style opt-out for
  self-signed/home-broker setups.
• Scope includes both the AmiSSL transport (Amiga) and a host-side OpenSSL
  transport (for CI/test parity against a real TLS broker), matching
  docs/ARCHITECTURE.md's original Phase 3 description.
• No cipher-suite/protocol-floor tuning for CPU-speed reasons in this first
  cut — justification below.

Where TLS plugs in (confirmed by reading the code, not guessed)

Four call sites decide plaintext vs TLS, plus one shared library insertion
point:

1. src/host/pub_main.c / sub_main.c — today call transport_bsd_connect() unconditionally.
2. src/amiga/pub_main.c / sub_main.c (static Amiga CLI, no library) — today call transport_bsdsocket_connect() unconditionally.
3. src/amiga/pub_main_lib.c / sub_main_lib.c (default, library-linked CLI) — never touch a transport; they build a struct MqttConnectOpts and call MQTT_CreateClient()/MQTT_Connect(). TLS intent must flow through that struct.
4. src/library/mqtt_funcs.c's child_connect() (~line 230) is the one factored connect routine, used by both the initial connect and the mco_AutoReconnect loop — the single insertion point inside the library; branching here covers auto-reconnect for free.

tool_opts (src/tools/tool_opts.h) is shared by two hand-maintained parsers
with no sync mechanism (src/host/args.c getopt, src/amiga/args.c
ReadArgs) — a new option means editing both by hand, as every existing option
already does. Default port (1883) is hardcoded before parsing in both; making
it TLS-aware (8883 default when TLS is on and no port given) means switching
to a "was port explicitly set" sentinel.

Milestones (each independently mergeable and testable)

Milestone 1 — Host OpenSSL transport + opt-in plumbing

Proves the vtable seam and flag-threading entirely on the host, no AmiSSL/SDK
dependency yet — fastest to land and get CI feedback from.

• New src/host/transport_openssl.h/.c: openssl_ctx { fd, SSL_CTX*, SSL* },
  transport_openssl_connect(out, ctx, host, port, insecure_skip_verify).
  Plain TCP connect (same shape as transport_bsd_connect(), duplicated per
  the project's existing "hand-maintained in parallel" tolerance rather than
  factored into a shared helper). SSL_CTX_set_verify() mapping
  verify-on/-S insecure to SSL_VERIFY_PEER|FAIL_IF_NO_PEER_CERT /
  SSL_VERIFY_NONE; SSL_CTX_set_default_verify_paths() for the system
  trust store when verifying. SO_RCVTIMEO set the same as
  transport_bsd.c so SSL_read doesn't block indefinitely.
  send/recv translate SSL_ERROR_WANT_READ/WANT_WRITE into "0 bytes,
  not an error" per the vtable contract. On handshake/verify failure, read
  SSL_get_verify_result() before SSL_free() (a real footgun in the
  reference: reading it after free silently returns X509_V_OK).
• tool_opts: add int tls; int tls_insecure; (both default 0).
• src/host/args.c / src/amiga/args.c: add boolean flags — -s/-S
  (getopt: enable TLS / enable TLS+skip verify) and TLS/S/TLSINSECURE/S
  (ReadArgs) — plus the port-default restructuring (0 sentinel, resolved
  after parsing: tls ? 8883 : 1883).
• src/host/pub_main.c/sub_main.c: branch on opts.tls between the two
  host transports.
• Makefile: HOST_SRCS gains the new file; host link lines gain
  -lssl -lcrypto (host OpenSSL is assumed present, like every other host
  build dependency today — no optional-SDK handling needed here).
• CI: new tests/broker/tls-smoke.sh (self-signed cert generated on the
  fly, mirrors tests/broker/smoke.sh's shape) + make broker-tls-smoke
  target, added as its own local job in .github/workflows/ci.yml
  alongside the existing broker-smoke/net-smoke/library-smoke/
  docs-build jobs (all currently gated by the changes job's path
  filters, each a plain make <target> step, not part of the reusable
  five-verb ci job) — reuse the broker change-detection category rather
  than adding a new one.
• Docs: userdocs/CLI-Reference.md gets the -s/-S host flags (Amiga
  flags land in Milestone 3, once they exist).

Verify: make test-host unaffected; new make broker-tls-smoke passes
(verify-on against a provisioned CA, and -S insecure, both round-trip);
default behavior (no -s) is byte-for-byte unchanged.

Milestone 2 — Amiga AmiSSL transport + Makefile SDK plumbing

The real target-side work. Starts with a short de-risking spike (not
merged) before committing to the I/O design:

• Spike: build a throwaway program against the real AmiSSL 5.x SDK, do
  the OpenAmiSSLTags() open sequence against a real per-connection
  bsdsocket.library SocketBase, then drive a handshake using
  SSL_set_fd() + the same WaitSelect-gate-then-blocking-call pattern
  bsdsocket_recv() already uses, under Copperline (throttled toward 68020
  timings) against a real TLS Mosquitto, repeated enough times to catch
  flakiness. Why this matters: the micropython reference tried and
  abandoned a blocking SSL_set_fd client path, citing intermittent
  broken-pipe failures "under the Amiga's slow handshakes" — their stated
  reason (no non-blocking poll-readiness for asyncio) doesn't apply to
  midge, but the underlying reliability concern might still. If the spike
  is reliable, proceed as designed below. If not, fall back to a
  BIO_new_bio_pair() pump adapted to midge's synchronous polling model —
  this only changes transport_amissl.c's internals, not its external
  shape, so nothing else in this plan changes either way.
• New src/amiga/transport_amissl.h/.c, mirroring transport_bsdsocket.h's
  per-connection-ctx discipline exactly (CLAUDE.md's static/extern trap —
  this file lives in mqtt.library's single shared data segment, serving
  multiple simultaneous connections, so no file-scope AmiSSL library
  base globals, unlike the micropython reference which safely uses them
  for its single-interpreter case):

  typedef struct {
      int fd;
      struct Library *socket_base;
      struct Library *amissl_master_base, *amissl_base, *amissl_ext_base;
      void *ssl_ctx;   /* SSL_CTX* */
      void *ssl;       /* SSL* */
      ULONG break_sigmask;
      int ctrl_c;
  } amissl_ctx;
  
  int transport_amissl_connect(mqtt_transport *out, amissl_ctx *ctx,
                                const char *host, uint16_t port,
                                int insecure_skip_verify);

  Connect sequence: open bsdsocket.library + resolve/connect (same as
  transport_bsdsocket_connect(), duplicated) → open amisslmaster.library
  • OpenAmiSSLTags(..., AmiSSL_SocketBase, SocketBase, ...) using this
    connection's own just-opened base → on failure, clean up and return -1
    distinguishably (so callers can report "AmiSSL not installed" rather than
    a generic connect failure) → SSL_CTX_new/set_security_level(0)
    (widens the trust floor only, carried over from the reference, not a
    speed knob) → SSL_CTX_set_verify() per the verify/insecure flag →
    when verifying, SSL_CTX_set_default_verify_paths() wrapped in the
    pr_WindowPtr = (APTR)-1 requester-suppression trick from the reference
    → SSL_set_fd/set_connect_state → handshake loop with
    ERR_clear_error() before each SSL_do_handshake(), gated by the same
    WaitSelect(SIGBREAKF_CTRL_C | break_sigmask) pattern as
    bsdsocket_recv() so a slow multi-round-trip handshake stays
    Ctrl-C-abortable (a requirement the reference didn't need to solve).
    send/recv/close mirror Milestone 1's OpenSSL versions, with recv
    doing the WaitSelect gate first, then SSL_read, translating
    WANT_READ/WANT_WRITE to "0, not an error."
• Makefile: new MIDGE_TLS ?= 1 feature flag (mirrors the reference's
  MICROPY_PY_AMIGA_SSL ?= 1) and AMISSL_SDK_DEV ?= $(BUILD)/amissl-sdk
  override, fetched by a new tools/amiga-fetch-amissl-sdk.sh (write fresh
  against the public AmiSSL SDK release — don't copy micropython's script
  without checking its license first). A Makefile check fails loudly with
  clear remediation only when MIDGE_TLS=1 and a target that needs it
  (m68k's static-tool builds, library) is actually invoked and the SDK
  header is missing — this is the first conditional-optional-SDK idiom
  in this codebase (no existing precedent), modeled on the reference. When
  MIDGE_TLS=0, transport_amissl.c is excluded from the source list
  entirely — a MIDGE_TLS=0 build needs zero AmiSSL headers, not just a
  runtime soft-fail. Note: m68k's library-linked tool builds
  (pub_main_lib.c/sub_main_lib.c) never link AmiSSL directly — only the
  static tools and mqtt.library do (see Milestone 3).
• CI: the AmiSSL SDK needs fetching inside ghcr.io/sidick/amiga-dev:1's
  container before make m68k/make library run — either the image
  bundles it (coordinate with whoever owns sidick/amiga-workflows) or the
  fetch script runs as a cached CI step first. New tests/net/net-tls-smoke.sh
  (Copperline HostSocket → host TLS Mosquitto) + make net-tls-smoke,
  gated the same way net-smoke is today.
• Docs: userdocs/Installation.md "Requirements" gains AmiSSL v5
  (amisslmaster.library), phrased as optional/only-needed-for-TLS.

Verify: make m68k/make library build clean with MIDGE_TLS=1 and
with MIDGE_TLS=0 (the latter needing zero SDK present); new
make net-tls-smoke passes on real cross-compiled code under Copperline;
manual Ctrl-C-during-handshake abort check; manual check that a missing
amisslmaster.library at runtime fails clearly, not silently-plaintext.

Milestone 3 — Wire TLS through the CLI tools and mqtt.library

Connects Milestone 2's transport to every entry point.

• src/amiga/pub_main.c/sub_main.c (static): branch on opts.tls between
  transport_bsdsocket_connect()/transport_amissl_connect(). Under
  MIDGE_TLS=0 builds, requesting -s fails with a clear "no TLS in this
  build" error rather than a silent plaintext fallback.
• src/library/include/libraries/mqtt.h: add to MqttConnectOpts:
  BOOL mco_UseTLS; (default FALSE = today's plaintext behaviour) and
  BOOL mco_TLSInsecureSkipVerify; (ignored unless mco_UseTLS). New
  MQTTERR_NOTLS (-206) for "TLS requested but unavailable" (build-time
  MIDGE_TLS=0 or runtime-missing amisslmaster.library) — distinguishable
  from a generic connect failure.
• src/library/mqtt_funcs.c's child_connect(): branch on
  h->ch_Opts.mco_UseTLS, covering both initial connect and
  auto-reconnect via the existing single-insertion-point factoring.
• src/amiga/pub_main_lib.c/sub_main_lib.c: map opts.tls/
  opts.tls_insecure into co.mco_UseTLS/co.mco_TLSInsecureSkipVerify
  before MQTT_CreateClient() — no transport knowledge at this layer.
• src/amiga/args.c: the TLS/S/TLSINSECURE/S template additions (if not
  already landed alongside Milestone 1's host-side flag work).
• CI: new make library-tls-net-smoke (mirrors library-net-smoke,
  mco_UseTLS=TRUE) including a reconnect-under-TLS check reusing the
  existing broker-restart pattern — proves auto-reconnect re-establishes
  TLS, not silently plaintext.
• src/library/mqtt.doc: document the new fields/error code, then
  make api-reference to regenerate userdocs/mqtt-library-reference.md
  (never hand-edit the generated file, per CLAUDE.md).
• userdocs/CLI-Reference.md: full flag docs for all four binaries,
  including the 8883-default-port behaviour and a caveat paragraph
  (mirroring the existing QoS-1 caveat) on why TLS is opt-in, the AmiSSL
  dependency, and what -S/TLSINSECURE does.

Verify: make library-tls-net-smoke passes (connect + pub/sub + TLS
reconnect, real Copperline); existing net-smoke/library-net-smoke
(plaintext) still pass unmodified, proving the default truly changed
nothing; manual check both Amiga CLI flavours work end-to-end; manual check
a MIDGE_TLS=0 library/CLI combo rejects TLS cleanly.

Milestone 4 — Docs polish and release-readiness

• docs/ARCHITECTURE.md: update the transport-vtable section from
  "Phase 3 adds..." (future tense) to describe what shipped, pointing at
  transport_amissl.c/transport_openssl.c.
• Final userdocs/ pass once flag names are locked, including a short
  "why is TLS opt-in / why can it be slow" explanation for end users.
• Close issue Phase 3: TLS support via AmiSSL #3.

Verify: full CI green (build/test-host/test-target/lint/dist,
plus all the new TLS smoke jobs) with MIDGE_TLS=1 as the default;
make dist/make guide produce a clean release archive with TLS-capable
binaries and updated docs; a real (non-self-signed) end-to-end TLS test
against a public broker if feasible, to validate AmiSSL's bundled CA store
actually works, not just the CI-only self-signed/insecure path.

Why no cipher-suite/protocol-floor tuning in this cut

The reference implementation's only tuning
(SSL_CTX_set_security_level(ctx, 0)) widens acceptance (legacy/small
keys), it doesn't narrow cipher choice for speed. The dominant handshake
cost on a slow 68k is asymmetric crypto driven by whatever cert/key the
broker presents — the client can't cheaply opt out of that while still
verifying, and pinning to "cheap" ciphers would mostly just break
compatibility with modern brokers that refuse legacy ciphers, without
reliably improving wall-clock time. The opt-in default already delivers the
project owner's actual requirement (no user pays this cost unaccountably).
If real on-target timing (Milestones 2/4) shows it's impractically slow
against typical modern broker configs, a targeted follow-up
(SSL_CTX_set_cipher_list() behind an advanced flag) can be added later
without touching the transport vtable or MqttConnectOpts shape.

Explicitly deferred (file as follow-up issues once this plan is approved)

• CA-file override (a CAFILE/--cafile flag for brokers behind a
  private CA not in AmiSSL's bundle). Real scenario for self-hosted setups
  that want verification without -S, but additive — small, non-structural
  change to mco_*/tool_opts/parsers if/when needed. Not in this plan's
  scope.
• Cipher/protocol tuning, per the justification above.
• Host-side OpenSSL transport hardening beyond CI parity (e.g. exposing
  it as a first-class documented feature for host tooling, not just test
  infrastructure) — out of scope; Milestone 1's transport exists to prove
  the design and give CI real coverage, not as a shipped host feature in
  its own right (the host CLI tools are dev/test tooling, not a release
  target).

Critical files

• src/core/mqtt_transport.h — the vtable seam (unchanged)
• src/amiga/transport_bsdsocket.c — the pattern transport_amissl.c follows
• src/library/mqtt_funcs.c (child_connect()) — the one library insertion point
• src/library/include/libraries/mqtt.h — MqttConnectOpts, error codes
• src/tools/tool_opts.h, src/host/args.c, src/amiga/args.c — opt-in flag plumbing
• Makefile — new MIDGE_TLS/AMISSL_SDK_DEV feature-flag idiom
• ~/src/micropython/ports/amiga/amiga_ssl.c, modssl.c — reference only, not part of midge

Next step if approved

Post this plan (or a link to it) as a comment on issue #3, then open
follow-up issues for the two deferred items above, then start Milestone 1
on a branch (main requires a PR now).

[sidick]

Milestone 2 de-risking spike update: migrated the spike's boot image from a
hand-staged sys/ directory to an amibake-built (~/src/amibake,
manifests/AmiSSLSpike.toml) real AmigaOS 3.2.2 + AmiSSL 5.27 install -
Docker being unwedged and amibake existing made that a better foundation
than reusing amiauth's hard drive setup. tests/copperline/amissl-spike/
now boots that image and runs amisslspike (SSL_set_fd + WaitSelect-gated
handshake) against a real TLS Mosquitto.

Finding: at real 68020/14MHz cycle-accurate timing, the AmiSSL
handshake always succeeds, but the immediately-following SSL_write fails
intermittently with SSL_ERROR_SYSCALL (roughly 2 of every 7-9 runs came
back clean). Isolating the variable:

• --jit alone (same nominal 14MHz): 7/7 clean
• --jit --cpu-clock 200: 9/9+ clean
• stock cycle-accurate timing (no --jit): consistently intermittent
  failures

Copperline's --jit is explicitly documented as "not cycle-exact, like an
accelerator card," so this isolates to timing, not throughput - it's
very likely a genuine race in the WaitSelect-gated write-retry loop that
only manifests at real, unaccelerated 68020 speeds. That's notable because
midge's baseline target is a real, unaccelerated 68020 (CLAUDE.md) - so
this isn't just an emulator artifact, it's a signal that
transport_amissl.c may hit this on real target hardware too.

Root cause not yet identified. run.sh now defaults to cycle-accurate
timing (so the bug stays reproducible); SPIKE_FAST=1 ./run.sh opts into
--jit --cpu-clock 200 for a fast "does the wiring work at all" check,
but that mode should not be used to judge real-hardware readiness.

[sidick]

Follow-up with a concrete threshold, plus prior art upstream.

Upstream precedent: jens-maus/amissl#111
is the same symptom shape - a good TLS handshake followed by a broken-pipe-style
failure, fixed by enabling JIT under Amiberry. AmiSSL's maintainer's diagnosis:
"broken pipe usually means the processor is too slow" - closed as "the Amiga
just can't keep up with modern SSL," a CPU-speed limit rather than something
fixable in AmiSSL itself. That matches our spike's shape closely enough that
I don't think there's a code-level fix waiting to be found here either.

Clock-speed threshold sweep (real cycle-accurate --cpu-clock, no
--jit, 150s bench each):

Clock	Result
14MHz	2/6 passed, 4/6 failed
16MHz	4/4 passed
18MHz	4/4 passed
20MHz	5/5 passed
25/33/50/75/100MHz	all clean

So the threshold sits between 14 and 16MHz - a genuinely stock, unaccelerated
68020 (~14MHz, e.g. an A1200's 68EC020) is right at the edge of this race,
while even a modest bump (16MHz+) clears it reliably in every run so far.
Not yet tried on real hardware or against a broker other than local
Mosquitto.

Suggested direction: given the upstream precedent, I'd lean toward
documenting this as a known minimum-practical-spec constraint for TLS
specifically (not disabling ciphers - the upstream issue wasn't
cipher-specific, it reproduced across different sites/ciphers and was
resolved purely by CPU speed) rather than chasing a code fix. Concretely:
keep TLS supported on the 68020 baseline as shipped, but document in
userdocs that reliable TLS in practice wants ~16MHz+ (i.e. any real
accelerator, or later 68030+), and that a genuinely stock 14MHz 68020 may
see intermittent TLS failures under load - same conclusion the AmiSSL
maintainer reached upstream.

[sidick]

Milestone: the Amiga-side TLS transport is now real and wired end-to-end
(commits f13e8b7, 7c7e0c8):

• src/amiga/transport_amissl.c - production AmiSSL mqtt_transport,
  derived from the spike but with per-connection library bases (no
  globals, safe for simultaneous mqtt.library connections) and
  WaitSelect-gated send/recv.
• mqtt.library picks it per-connection via new
  mco_TLS/mco_TLSInsecure fields on MqttConnectOpts, feature-gated
  by MIDGE_AMIGA_TLS (Makefile M68K_HAS_AMISSL +
  make fetch-amissl-sdk, mirroring the host's HOST_HAS_TLS idiom) so
  everything still builds without the SDK.
• The library-linked mqtt_pub/mqtt_sub expose TLS/TLSINSECURE
  switches; the static builds reject TLS rather than silently going
  plaintext. Docs updated (TLS on the Amiga in CLI-Reference,
  mqtt.doc/api reference regenerated).

Verified under Copperline at stock cycle-accurate 68020/14MHz: the real
mqtt_pub TLS TLSINSECURE published through mqtt.library +
transport_amissl to a real TLS Mosquitto (TLSv1.3), message received on
the host. Notably the earlier 14MHz intermittency did NOT reproduce
across 31/31 isolated-transport runs plus the CLI run - the threshold
sweep's failure rates may have been noisier (host-load-dependent) than
the writeup above implied, though the CPU-speed caveat stays documented
as a real risk per the upstream precedent.

Trap worth knowing: without the AmiSSL: assign, OpenAmiSSLTags()
blocks forever on a "please insert volume" DOS requester after the TCP
connect - looks like a silent connection hang from the broker side. Now
documented in CLI-Reference.

Remaining for this issue: an on-target TLS smoke job (needs the
amibake-built OS3.2.2+AmiSSL image, so local-only for now), and the
CA-file override / cipher-suite questions already split out as #13/#14.

[sidick]

On-target TLS verification is now covered too - #16 adds a local-only
`library-tls-smoke` test (`tests/library/libtls.c`/`tls-run.sh`) that
exercises mqtt.library's real `mco_TLS` API end-to-end against a real TLS
Mosquitto under Copperline. Full green run locally (all 8 checks + host
observer received the publish). Not CI-wired since it needs a real
AmigaOS+AmiSSL image (amibake) and a real Kickstart ROM - see
`tests/library/README.md`.

That closes out the remaining item from the last progress comment. What's
left for this issue is just the already-split-out #13 (CA-file override)
and #14 (cipher-suite measurement).

[sidick]

Closing: this issue's own scope (AmiSSL transport, mqtt.library wiring,
CLI flags, on-target verification, docs) is done and merged (commits
f13e8b7, 7c7e0c8, #16, #17). CA-file override (#13) is also done (#18).

Remaining TLS-adjacent work is tracked in its own issue, not here: #14
(cipher-suite measurement for slow 68k handshakes) - worth re-scoping
there given our own finding that the AmiSSL/68020 issue is CPU-speed,
not cipher-specific.