sml-pkg

A package-manifest resolver, lockfile, and .mlb generator for the sjqtentacles Standard ML ecosystem — plus a thin command-line build driver.

The whole resolver is a pure, deterministic core: parsing an sml.pkg manifest, resolving a dependency graph, serializing a lockfile, and generating .mlb content are all string/data transforms with no FFI, no clock, no randomness, and no network. The same inputs always produce byte-identical output under MLton and Poly/ML.

The impure parts — fetching missing dependencies with git, reading the on-disk vendoring tree, and invoking mlton — are isolated in one small driver module (cli/) and are explicitly not part of the byte-identical guarantee, mirroring how sml-httpc-tool / sml-serve quarantine their I/O.

sml-pkg interoperates with the same on-disk layout the org already uses with smlpkg: an sml.pkg manifest at the repo root and dependencies vendored under lib/github.com/<owner>/<repo>/, each exposing a sources.mlb.

• Pkg.parse — sml.pkg text → typed manifest (errors as values).
• Pkg.resolve — manifest * registry → deterministic topological resolution, with missing-dependency, version-conflict, and cycle detection.
• Pkg.lockfile — resolution → stable, path-sorted sml.lock.
• Pkg.mlb — resolution → .mlb referencing each vendored basis in dependency order.

The sml.pkg manifest grammar

The same format consumed by smlpkg:

package github.com/sjqtentacles/<name>

require {
  github.com/sjqtentacles/<dep>
  github.com/sjqtentacles/<dep> <version>
}

• The package line names the package by import path.
• The require { ... } block lists zero or more dependencies, one per line.
• Each dependency is an import path optionally followed by a single whitespace-separated version token — an integer like 1, or a pseudo-version like 0.0.0-20260621120446+2466b0395.... The token is preserved verbatim.
• Blank lines and (* ... *) whole-line comments (including trailing inline comments) are ignored. Duplicate dependencies are rejected.

The sml.lock format

A stable, reproducible serialization sorted by import path, so it depends only on the resolved dependency set and selected versions — never on traversal order. Unpinned dependencies are written with *:

# sml.lock -- generated by sml-pkg; do not edit by hand.
root github.com/sjqtentacles/<name>

github.com/sjqtentacles/<dep-a> *
github.com/sjqtentacles/<dep-b> 1

The generated .mlb

The basis library, then each resolved dependency's vendored basis in dependency order (dependencies before dependents):

(* Generated by sml-pkg for github.com/sjqtentacles/<name>. Do not edit by hand. *)
$(SML_LIB)/basis/basis.mlb

../lib/github.com/sjqtentacles/<dep>/sources.mlb

API

signature PKG = sig
  datatype ('a, 'e) result = Ok of 'a | Err of 'e

  type require   = { path : string, version : string option }
  type manifest  = { name : string, requires : require list }
  type parseError = { line : int, message : string }

  val parse    : string -> (manifest, parseError) result
  val render   : manifest -> string
  val repoName : string -> string

  type registry = string -> manifest option
  val registryOf : (string * manifest) list -> registry

  type resolved   = { path : string, repo : string,
                      version : string option, deps : string list }
  type resolution = { root : string, order : resolved list }
  datatype resolveError =
      Missing  of { required_by : string, path : string }
    | Cycle    of string list
    | Conflict of { path : string, versions : string list }

  val resolve : manifest * registry -> (resolution, resolveError) result

  val lockfile : resolution -> string

  type mlbConfig = { basisPrefix : string, eachFile : string }
  val defaultMlbConfig : mlbConfig
  val mlb : mlbConfig -> resolution -> string

  val parseErrorToString   : parseError -> string
  val resolveErrorToString : resolveError -> string
end

Resolution semantics

• Transitive closure of the root's require graph (the root itself is not listed in order).
• Deterministic topological order: Kahn's algorithm with a lexicographic-smallest tie-break, so the order is reproducible across runs and compilers.
• Diamonds collapse: a dependency reached by two paths appears exactly once.
• Versions merge per dependency: an explicit version beats an unpinned one; two different explicit versions are a Conflict.
• Missing packages (required but absent from the registry) and cycles (reported in a canonical rotation, smallest path first, loop closed) are returned as Err values, never raised.

Example

Running examples/demo.sml with make example prints:

== parse a manifest ==
package: github.com/sjqtentacles/sml-demo
requires: 2
  - github.com/sjqtentacles/sml-color
  - github.com/sjqtentacles/sml-image @ 1
----------------------------------------
== resolve the graph (diamond) ==
topological order (deps before dependents):
  sml-codec
  sml-color
  sml-inflate
  sml-image @ 1
----------------------------------------
== sml.lock ==
# sml.lock -- generated by sml-pkg; do not edit by hand.
root github.com/sjqtentacles/sml-demo

github.com/sjqtentacles/sml-codec *
github.com/sjqtentacles/sml-color *
github.com/sjqtentacles/sml-image 1
github.com/sjqtentacles/sml-inflate *
----------------------------------------
== generated .mlb ==
(* Generated by sml-pkg for github.com/sjqtentacles/sml-demo. Do not edit by hand. *)
$(SML_LIB)/basis/basis.mlb

../lib/github.com/sjqtentacles/sml-codec/sources.mlb
../lib/github.com/sjqtentacles/sml-color/sources.mlb
../lib/github.com/sjqtentacles/sml-inflate/sources.mlb
../lib/github.com/sjqtentacles/sml-image/sources.mlb

The CLI driver (impure, unguaranteed)

A thin front end built on the vendored sml-cli parser. Build it with make driver (MLton); it produces bin/sml-pkg:

sml-pkg resolve          # read ./sml.pkg, resolve against vendored lib/, write ./sml.lock
sml-pkg lock             # alias for resolve
sml-pkg mlb [--out F]     # print (or write) the dependency .mlb
sml-pkg sync [--dry-run]  # git clone any missing vendored dependencies into lib/
sml-pkg vendor           # alias for sync
sml-pkg build [--out F]   # generate the .mlb and invoke mlton

All dependency math is delegated to the pure structure Pkg; the driver only adds I/O. Safety: shell-outs are restricted to git/mlton, and clones only ever write into the repo's own lib/ subtree — the driver never deletes anything. What the driver can do well today: resolve, lock, and mlb are robust pure transforms over on-disk data; sync clones missing repos and build runs MLton. What it deliberately leaves thin: it does not yet pin to or verify the pseudo-version of a clone, and Poly/ML driving is left to the existing tools/polybuild rather than reimplemented here.

Build & test

Requires MLton and/or Poly/ML.

make test        # build + run the pure-core suite under MLton
make test-poly   # run the suite under Poly/ML
make all-tests   # both (byte-identical)
make example     # build + run the deterministic demo
make driver      # build the impure CLI (bin/sml-pkg)
make smoke       # build the driver, then `sml-pkg resolve` here
make clean

Installing with smlpkg

smlpkg add github.com/sjqtentacles/sml-pkg
smlpkg sync

Reference lib/github.com/sjqtentacles/sml-pkg/pkg.mlb from your own .mlb (MLton / MLKit), or feed sources.mlb to tools/polybuild (Poly/ML).

Layout

sml.pkg                                       smlpkg manifest (requires sml-cli)
Makefile                                      MLton + Poly/ML targets
.github/workflows/ci.yml                      CI: MLton + Poly/ML
lib/github.com/sjqtentacles/
  sml-pkg/
    pkg.sig      PKG signature (pure core)
    pkg.sml      parser + resolver + lockfile + mlb generator
    sources.mlb  ordered source list
    pkg.mlb      public basis
  sml-cli/       vendored byte-for-byte (declarative arg parser)
cli/
  driver.sig     DRIVER signature (the impure edge)
  driver.sml     file I/O, lib/ scan, git/mlton shell-outs
  pkg.sml        sml-cli front end + subcommand dispatch
  pkg.mlb        impure tool basis
examples/
  demo.sml       parse + resolve (diamond) + lockfile + mlb walkthrough
test/
  harness.sml    shared assertion harness
  test.sml       parse / resolve / lockfile / mlb golden vectors (31 checks)
  entry.sml / main.sml
tools/polybuild  Poly/ML build wrapper

Tests

31 deterministic golden checks: parsing every observed sml.pkg shape (leaf, multi-require, integer and pseudo-version pins, comments/blank lines, round-trip via render) and the error cases (missing package, unterminated block, duplicate dep, malformed line, trailing junk); resolving a diamond (shared dependency collapses to one node in topological order), missing detection, cycle detection with canonical rotation, version selection, and version conflict; and byte-exact sml.lock and .mlb output (including the empty-leaf cases and a custom mlbConfig). Run make all-tests to verify identical output under both compilers.

License

MIT. See LICENSE.