clean up the public include surface

[rdmark]

Move Internal Headers and Symbols Out of the Public Include Surface

Summary

The project currently installs several headers from include/ that expose implementation details such as concrete struct afp_server, struct afp_volume, DSI request queues, mutexes, file descriptors, reconnect state, and low-level AFP protocol helpers. This makes the public API harder to consume from C/C++, leaks unstable ABI details, and turns internal synchronization choices into public header dependencies.

We should split the public API from internal implementation headers. Public headers should expose stable handles, value structs, constants, and supported functions. Internal headers and symbols should live under lib/ and be included only by in-tree implementation code.

Motivation

• Avoid exposing implementation details such as pthread_mutex_t, reconnect guards, request queues, fd state, and AFP session internals.
• Avoid public C++ compatibility issues from internal C constructs or compiler-specific synchronization details.
• Make ABI and API expectations clearer before the project leaves alpha.
• Reduce accidental dependency by external consumers on structs and functions we may need to change freely.
• Make future concurrency fixes easier by keeping threading primitives private.

Current Problem Areas

• include/afp.h exposes concrete internal structs and many low-level functions.
• struct afp_server is public even though callers should generally use serverid_t or library functions.
• struct afp_volume is public even though stateless users generally use volumeid_t.
• Internal helpers such as server lookup, reconnect, AFP command functions, cache helpers, loop helpers, and protocol reply functions are visible through public headers.
• Some headers under include/ are effectively internal, including dsi.h, midlevel.h, utils.h, and parts of map_def.h.

Proposed Direction

1. Keep public headers minimal and stable:

   • include/afp.h
   • include/afpsl.h
   • include/afp_protocol.h
   • include/afp_xattr.h
   • include/libafpclient.h
   • any wire protocol IPC headers that must remain public

2. Move implementation-only declarations to lib/:

   • lib/afp_internal.h
   • lib/dsi.h
   • lib/midlevel.h
   • lib/utils.h or a narrower internal utility header
   • protocol/private reply headers as needed

3. Make server and volume opaque to public consumers:

   • Public:

     struct afp_server;
     struct afp_volume;
     typedef void *serverid_t;
     typedef void *volumeid_t;

   • Internal:
     concrete struct afp_server and struct afp_volume definitions.

4. Keep public value structs public:

   • struct afp_url
   • struct afp_server_basic
   • struct afp_file_info_basic
   • struct afp_volume_summary

Suggested Migration Plan

Phase 1: Inventory

• List all installed headers in include/meson.build.
• Classify each header as public, private, or mixed.
• For mixed headers, list which declarations are actually needed by external users.

Phase 2: Introduce Internal Header

• Add lib/afp_internal.h.
• Move concrete internal structs there:
  • struct afp_server
  • struct afp_volume
  • struct afp_file_info
  • struct afp_rx_buffer
  • struct afp_token
  • internal comment/icon structs if not public
• Move private macros and state constants there:
  • SERVER_STATE_*
  • mount/attach state constants
  • volume_is_readonly
  • reconnect guard helpers

Phase 3: Trim Public afp.h

• Keep only public declarations in include/afp.h.
• Remove concrete server/volume structs.
• Remove low-level AFP command prototypes.
• Remove internal synchronization, reconnect, queue, and loop details.
• Ensure afp.h is self-contained and usable from C and C++.

Phase 4: Move Internal Headers

• Move implementation-only headers from include/ to lib/ where appropriate.
• Update in-tree includes to reference internal headers directly.
• Keep compatibility shims only if needed temporarily, and do not install them.

Phase 5: Tighten Installation

• Update include/meson.build so only public headers are installed.
• Verify meson install does not install internal headers.
• Consider adding a simple CI check that installed headers do not include lib/ internals or expose internal-only symbols.

Phase 6: Compile and Consumer Checks

• Build the full project.
• Run the test suite.
• Add lightweight public-header compile checks:
  • C translation unit including public headers only.
  • C++ translation unit including public headers only.
• Verify consumers can use stateless APIs without AFPCLIENT_INTERNAL.

Acceptance Criteria

• Public headers no longer expose struct afp_server internals.
• Public headers no longer expose struct afp_volume internals.
• Public headers do not expose threading primitives, atomics, DSI queues, fd state, or reconnect state.
• Internal implementation still builds using headers under lib/.
• Installed headers are self-contained and compile in both C and C++.
• Existing Meson tests pass.
• No external consumer needs AFPCLIENT_INTERNAL for supported public APIs.

Non-Goals

• Do not redesign the stateless IPC protocol in this issue.
• Do not remove existing public stateless APIs.
• Do not change runtime behavior except where needed to preserve build correctness after the header split.
• Do not make ABI stability guarantees beyond reducing public exposure of unstable internals.

Open Questions

• Should low-level stateful AFP APIs remain public at all, or should the supported public API be the stateless afpsl surface?
• Should include/afp_server.h remain public, or is it daemon IPC only?
• Should public handles remain void *, or should they become typed opaque pointers?
• Do we want temporary compatibility headers for one release, or can alpha status allow an immediate cleanup?

Notes

This cleanup was prompted by reconnect state synchronization work. Exposing synchronization primitives or atomic storage in public structs creates unnecessary C/C++ compatibility and ABI concerns. Keeping those details internal lets us fix threading issues without making implementation choices part of the public API.

[rdmark]

Make In-Tree Clients Consume libafpclient Like Third Parties

Summary

The internal header cleanup outlined above makes include/afp.h smaller, but it does not yet
create a real public API boundary. The command-line client, stateless daemon,
and FUSE daemon still include lib/afp_internal.h or other implementation-only
headers and directly manipulate internal server, volume, file, reconnect, and
request state.

The goal of the next cleanup stage is for these in-tree consumers to build and
run against the installed public libafpclient API in the same way a third-party
consumer would. Implementation internals should remain available only to
lib/.

Progress

Completed:

• Step 1: Added include/afp_client.h with public opaque server, volume, and
  file handles plus stable public value structs and operations.
• Step 2: Implemented the stateful facade in lib/afp_client.c, wrapping the
  existing internal server, volume, file, metadata, reconnect, status, and
  password-change behavior.
• Step 3: Ported daemon/commands.c to daemon-owned server, volume, and file
  handle records backed by public afp_client_* handles. The stateless IPC wire
  protocol shape remains unchanged, but daemon command handlers no longer
  include private AFP/DSI/midlevel headers or call ml_*, server-list,
  reference-count, reconnect, or raw volume helpers directly.

Verification completed after Step 3:

• meson compile -C builddir
• meson test -C builddir --print-errorlogs
• standalone C and C++ include checks for afp_client.h
• boundary grep checks for private includes and direct internal calls in
  daemon/commands.c

Current next step:

• Step 4: Port FUSE callbacks to public volume and file handles.

Current Problem

The current split is incomplete because public consumers cannot perform the
main stateful client operations through public headers alone. As a result,
in-tree programs still depend on implementation details:

• cmdline/ still uses internal connection and server lifecycle helpers.
• daemon/ still uses internal server lookup, reference counting, volume
  attachment state, and ml_* filesystem helpers.
• fuse/ still stores and dereferences concrete struct afp_volume and
  struct afp_file_info values and calls ml_* directly.
• Multiple non-lib/ targets still include lib/afp_internal.h, dsi.h,
  midlevel.h, utils.h, map_def.h, or uams_def.h.
• Build configuration still gives all targets broad include access and defines
  AFPCLIENT_INTERNAL globally.

Desired Boundary

Only lib/ should include internal headers and compile with
AFPCLIENT_INTERNAL.

The following targets should consume only public installed headers:

• cmdline/
• daemon/
• fuse/
• public-facing tests

They should not dereference struct afp_server, struct afp_volume, or
struct afp_file_info, and they should not call low-level AFP, DSI, reconnect,
server-list, reference-count, cache, or ml_* helpers directly.

Required Structural Changes

1. Add a Public Stateful Client API

Introduce a real public stateful libafpclient facade, either in a new
include/afp_client.h or by expanding include/libafpclient.h.

This API should expose opaque handles and stable value types for:

• server connections
• volume attachments
• open files or file IDs
• directory listing entries
• volume summaries
• mount and mapping options
• authentication/UAM options

It should provide public operations for:

• connect, resume, reconnect, disconnect
• server info and status
• list volumes
• attach and detach volumes
• stat, statfs, readdir
• open, read, write, close
• create, delete, rename, chmod, chown, truncate, utime, mkdir, rmdir
• xattr operations
• FinderInfo and resource fork operations
• password change

Internally, this facade can wrap today’s afp_server_*, afp_connect_volume,
ml_*, and lower-level helpers.

2. Keep ml_* Internal

Do not make midlevel.h the third-party API as-is. It is close to a useful
filesystem abstraction, but it still leaks internal pointer types such as
struct afp_volume * and struct afp_file_info *.

Instead, keep ml_* under the implementation boundary and expose wrappers that
use public opaque volume and file handles, stable integer file IDs, or public
value structs.

3. Refactor the Stateless Daemon Command Implementation

daemon/commands.c should stop doing direct internal server and volume work.

Move or wrap the following behind the public libafpclient API:

• server lookup by pointer, name, address, or signature
• server list traversal
• reference counting
• connected/disconnected state checks
• volume attached/mounted state checks
• reconnect and resume matching
• ml_* filesystem operations
• direct access to volume->server

The daemon should own its IPC request/response framing, but use public
libafpclient handles for AFP state.

4. Refactor FUSE to Use Public Handles

The FUSE daemon should not store struct afp_volume * as private FUSE data or
store struct afp_file_info * as FUSE file handles.

Replace these with public libafpclient volume and file handles, then route FUSE
callbacks through public operations for lookup, open, read, write, close,
metadata, and xattr behavior.

5. Refactor afpcmd and afpgetstatus

The command-line tools should use public connection, status, server-info,
volume, and filesystem operations.

They should not include or depend on:

• lib/afp_internal.h
• dsi.h
• utils.h
• map_def.h
• uams_def.h
• midlevel.h

Any helper currently needed from those headers should either become part of a
small public API or remain hidden behind a public operation.

6. Stop Exposing Raw Internal Pointers as Public IDs

serverid_t and volumeid_t currently behave like raw internal pointers. That
is convenient, but it makes the stateless IPC and in-tree consumers dependent
on internal object addresses.

Replace this with one of:

• opaque public handle structs managed by libafpclient, or
• stable integer/token handles owned by the daemon or client context.

External callers should never need to cast a handle back to an internal
struct afp_server * or struct afp_volume *.

7. Enforce the Boundary in Meson

Split include directories and compile definitions by target:

• lib/ gets access to lib/ and include/, and compiles with
  AFPCLIENT_INTERNAL.
• cmdline/, daemon/, and fuse/ get public include/ only.
• Public-facing tests get public include/ only.
• Internal tests that validate protocol internals may explicitly opt into
  internal headers.

Add a CI or Meson check that fails if non-lib/ sources include private
headers.

Example checks:

rg '#include "lib/afp_internal.h"|#include "afp_internal.h"' cmdline daemon fuse include
rg '#include "dsi.h"|#include "midlevel.h"|#include "utils.h"' cmdline daemon fuse include

Migration Order

1. Done: Define the public stateful libafpclient API and handle model.
2. Done: Implement wrappers in lib/ while leaving existing internal code in
   place.
3. Done: Port daemon/commands.c to public handles and public operations.
4. Port FUSE callbacks to public volume and file handles.
5. Port afpcmd and afpgetstatus.
6. Remove private includes from non-lib/ targets.
7. Restrict Meson include paths and AFPCLIENT_INTERNAL.
8. Add public-header and private-include CI checks.
9. Remove or relocate remaining implementation-only installed headers.

Acceptance Criteria

• cmdline/, daemon/, and fuse/ build without AFPCLIENT_INTERNAL.
• cmdline/, daemon/, and fuse/ do not include headers from lib/.
• Non-lib/ targets do not include dsi.h, midlevel.h, utils.h,
  map_def.h, or uams_def.h unless those headers have been deliberately
  made public and scrubbed of internals.
• Public consumers cannot dereference server, volume, or file internals.
• Public consumers do not call ml_*, low-level AFP command functions, DSI
  helpers, reconnect helpers, server-list functions, or reference-count
  functions.
• Public headers compile standalone in C and C++.
• Full Meson build and tests pass.

Non-Goals

• Do not redesign the stateless IPC wire protocol in the same step.
• Do not rewrite AFP protocol command implementations.
• Do not change runtime behavior except where needed to preserve existing
  semantics behind the public facade.
• Do not promise long-term ABI stability beyond hiding implementation details
  before the project leaves alpha.

[rdmark]

@andylemin I would love to get your thoughts on the architectural direction here!

in short, libafpclient does not have a proper external API, all implementations (the FUSE daemon, stateless library and daemon, afpcmd) are privy to the internal interfaces of the library, notably the midlevel API and various AFP server/volume structures

I started experimenting locally and have a partial restructuring of the library, progress noted in the previous comment, but it's a very invasive refactor so I want to calibrate on the goal before going deeper into the project

obviously, right now is the best time to do this, since we are a fresh fork with a new name :)

right now I'm leaning towards abandoning the idea of turning libafpclient into a true shared library with an external API, and pivot towards the stateless library being the actual 3rd party API, and update the documentation accordingly

[rdmark]

we have run into a real life API boundary situation now: linking libafpclient with the netatalk testsuite revealed that there is at least one direct overlap, dsi_getstatus(), which exist in both libafpclient and libatalk API surface

for this particular issue, I’d start by moving/keeping include/dsi.h private, then rename all DSI implementation symbols to afpc_dsi_*. After that, audit exported symbols with nm -g/objdump -T and shrink the visible set until only the intended API remains.