Skip to content

Latest commit

 

History

History
353 lines (310 loc) · 19.3 KB

architecture.md

File metadata and controls

353 lines (310 loc) · 19.3 KB

Architecture

Directory Layout

baller/
├── Cargo.toml
├── src/
│   ├── main.rs              # Entry point, OS check, config + context init
│   ├── cli/                 # CLI parsing with clap derive
│   │   ├── mod.rs
│   │   ├── parse.rs         # BallerCommand, CommandTypes enum
│   │   ├── help.rs          # Custom help output
│   │   └── version.rs       # Version output
│   ├── commands/            # Per-command business logic
│   │   ├── mod.rs
│   │   ├── draft.rs         # Install (with recursive dep resolution)
│   │   ├── eject.rs         # Uninstall
│   │   ├── roster.rs        # List/search installed + remote
│   │   ├── freeze.rs        # Pin/unpin packages
│   │   ├── substitute.rs    # Swap packages
│   │   ├── update.rs        # Update non-frozen packages
│   │   ├── rollback.rs      # Undo a failed draft/substitute/update
│   │   ├── sweep.rs         # Clean cache
│   │   ├── referee/         # referee group: audit, check, scan, cache, config, sbom
│   │   └── build.rs         # Assemble a package from a local manifest
│   ├── config/
│   │   ├── mod.rs
│   │   └── config.rs        # BallerConfig, RegistryConfig, HooksConfig, RefereeConfig
│   ├── core/                # Core engine
│   │   ├── mod.rs
│   │   ├── package.rs       # Package / PackageSource structs
│   │   ├── registry.rs      # Multi-source RegistryClient
│   │   ├── downloader.rs    # Download, hash verify, extract
│   │   ├── db.rs            # SQLite state management
│   │   ├── dep_solver.rs    # Dependency resolution, cycle detection
│   │   ├── hooks.rs         # Pre/post hook execution
│   │   └── manifest.rs      # baller.toml/json parser (flat + nested)
│   ├── http/                # HTTP registry clients
│   │   ├── mod.rs           # HttpClient (reqwest wrapper, retries)
│   │   ├── github.rs        # GitHub Releases API
│   │   ├── registry_api.rs  # Baller registry API
│   │   ├── chocolatey.rs    # Chocolatey OData v2 feed
│   │   ├── system.rs        # System Linux package manager (apt/dnf/pacman)
│   │   └── cargo.rs         # crates.io via the local cargo CLI
│   ├── security/            # Referee — the package security layer
│   │   ├── mod.rs           # Referee service, the Phase A gate, GateOutcome
│   │   ├── identity.rs      # Package -> advisory identities
│   │   ├── osv.rs           # OSV API client and wire types
│   │   ├── ranges.rs        # Affected-version range matching
│   │   ├── scoring.rs       # CVSS base scores, risk index, thresholds
│   │   ├── verdict.rs       # Verdicts, matched advisories, package reports
│   │   ├── scan.rs          # Phase B artifact scanner
│   │   └── virustotal.rs    # Optional hash-only VirusTotal lookup
│   ├── platform/            # OS abstraction
│   │   ├── mod.rs
│   │   ├── common.rs        # PlatformManager trait
│   │   ├── linux.rs         # Symlink implementation
│   │   └── windows.rs       # Windows implementation
│   ├── error/
│   │   ├── mod.rs
│   │   └── error.rs         # BallError enum
│   ├── context.rs           # AppContext (shared services container)
│   └── utils/
│       ├── mod.rs
│       ├── fs.rs            # File system utilities
│       └── security.rs      # Hash verification utilities

Startup Flow

  1. main() → entry() OS validation (Linux or Windows only)
  2. create_baller_dir() — ensures ~/.baller/ exists
  3. BallerConfig::parse_config() — reads ~/.baller/baller.conf, falls back to defaults
  4. AppContext::new() — initialises all shared services:
    • Creates ~/.baller/cache/ and ~/.baller/hooks/ dirs
    • HttpClient (reqwest with timeouts + retries)
    • DbManager (SQLite schema init)
    • RegistryClient (source chain respecting per-source enable flags)
    • Downloader (cached download, extraction)
  5. BallerCommand::parse_command() — clap CLI argument parsing
  6. command.execute(&ctx) — dispatches to the appropriate command module

Core Data Structures

BallerConfig

Holds all user-configurable paths (install_dir, db_path, cache_dir, hooks_dir), RegistryConfig (source URLs, order, per-source enabled flags), and HooksConfig (per-type enable flags).

AppContext

Lifetime container for all shared services. Created once at startup, passed by reference to every command function and sub-operation.

Package

The universal package metadata struct used across the entire system:

  • name, version, description, author
  • source (PackageSource enum: GitHub, BallerRegistry, Chocolatey, System { manager }, Cargo { crate_name })
  • download_url, sha256, hash_algorithm, dependencies
  • advisory (optional AdvisoryDeclaration: the OSV ecosystem, name and advisory ids the package declares about itself) and vulnerabilities (OSV-shaped records a registry served with the metadata). Both feed Referee; see referee.md

The hash_algorithm field (e.g., "SHA256", "SHA512") is set by the Chocolatey source to indicate which hash algorithm was used. When present, the downloader decodes the base64 hash and verifies using the specified algorithm. For GitHub/Baller sources, hash_algorithm is None and the downloader defaults to SHA-256 hex verification.

Platform Abstraction

The PlatformManager trait in platform/common.rs defines:

  • get_install_dir() — where binaries go (~/.local/bin on Linux)
  • get_config_dir() — where baller stores data (~/.baller)
  • create_symlink(source, name) — expose a binary in the install dir
  • remove_symlink(name) — remove a binary from the install dir

Each OS implements the trait in its own module. The ActiveManager type alias resolves to the correct implementation at compile time via cfg(target_os).

Registry Fallback Chain

The RegistryClient iterates sources in configurable order (source_order), trying each until one returns successfully. The per-source _enabled booleans act as a filter — disabled sources are skipped entirely.

The default source_order is platform-derived: the Baller registry comes first, the platform's native ecosystem comes next (Chocolatey on Windows; the distro package manager followed by Cargo on Linux), and GitHub is always the last-resort fallback. When all sources fail, the returned error lists every source that was tried and the reason it failed.

Platform Default source_order
Windows baller, chocolatey, github
Linux baller, system, cargo, github
Windows: request → [Baller Registry] ?→ [Chocolatey] ?→ [GitHub]
Linux:   request → [Baller Registry] ?→ [System PM]  ?→ [Cargo] ?→ [GitHub]
                          ↓ failure          ↓ failure     ↓ failure   ↓ failure
                          try next           try next      try next    return error

See docs/registry.md for the per-source details.

Platform Scoping

The chain decides where to look; three layers decide whether what it found can run on this host. Each holds on its own, and together they mean no artifact built for another platform can be installed, whatever the manifest, the --source flag or a registry says:

  1. Source affinity — PackageSource::affinity() (core/package.rs) maps every source to Only(Windows) (Chocolatey), Only(Linux) (System) or Agnostic (GitHub, Baller, Cargo) as an exhaustive match, so a new source cannot be added without deciding its scope. The one gate, ensure_installable() in core/registry.rs, checks that affinity and the package's architectures allow-list. It runs where a source is used, not where it is declared: on build's effective source, on --source before any fetch (ensure_source_supported), and on every package draft, substitute and update would install, before the dry run, any hook or any download. The System host-manager probe (build's ensure_system_source_supported) is a separate capability check and runs after it.
  2. Platform-scoped manifests — [source.linux] / [source.windows] tables are resolved for the host inside ManifestParser, so Package stays flat and no consumer needs selection logic (see docs/manifest.md). The GitHub release selector likewise never picks a .nupkg or .exe asset on a non-Windows host.
  3. Artifact verification — Downloader::download_and_extract reads the header of the binary find_binary_in_dir located: an ELF on Windows or a PE on Linux is a PlatformMismatch; a shebang script or any other file that is not a definite binary passes; an empty file counts as no binary at all. Every install path (build, draft, substitute, both of update's) extracts through this one function, so the check covers them all.

UnsupportedOs and PlatformMismatch are not PackageNotFound, so the dependency resolver propagates them: a mismatch anywhere in the graph aborts the whole command.

Dependency Resolution

resolve_deps() in dep_solver.rs:

  1. BFS traversal from the root package
  2. Fetches metadata for each dependency from the registry
  3. Builds a dependency graph
  4. Validates version constraints (using semver::VersionReq) — every registered constraint is enforced, even one registered after its target was already resolved (diamond/transitive deps and pinned roots), and a constraint string VersionReq::parse rejects raises VersionConflict instead of being relaxed to *
  5. Detects cycles (DFS with White/Gray/Black coloring) — cycles are hard errors unless every node on the cycle is a system package
  6. Topological sort (DFS post-order) for install order, validated against the graph's edges so a cyclic graph can never produce an order silently
  7. Returns packages in dependency-first order

System package resilience: The resolver tolerates common characteristics of system packages, scoped so a genuine resolution failure is never hidden:

  • Virtual packages (e.g., default-dbus-session-bus): a dependency that resolves from no source is skipped only when every package that depends on it is itself system-sourced — a Debian/RPM roll-up name the native manager handles. Anything else lands in ResolveResult.unresolved, and draft/substitute abort with UnresolvedDependencies instead of installing a subset; update stays best-effort and warns per name. PackageNotFound is skippable only through this scoped window — every other error, including NoMatchingAsset and NoBinaryFound, aborts the resolution, so a dependency with no usable build can never be silently dropped from an install.
  • Dependency cycles (e.g., libc6 ↔ libgcc-s1): tolerated only when every node on the cycle is system-sourced; the resolver then falls back to a best-effort ordering. A cycle touching any non-system package raises DependencyCycle and aborts the install.
  • Unparseable versions: If parse_version_flexible() cannot normalize a version to semver, a PackageManagerError is returned.

Version Format Flexibility

Versions from the system package manager (e.g. 2:1.21-76, 8.2.2637-20.fc36) are not strict semver. parse_version_flexible() in dep_solver.rs handles:

  1. Epoch prefix stripping (2:1.21 → 1.21)
  2. Revision suffix stripping (1.21-76 → 1.21, 8.2.2637-20.fc36 → 8.2.2637)
  3. Embedded tag stripping (1.3.dfsg+really1.3.1 → 1.3)
  4. Build metadata stripping (1.0+build → 1.0)
  5. Normalization to 3 segments (1.4.309.0 → 1.4.309, 20240118 → 20240118.0.0, 1.21 → 1.21.0)
  6. Pre-release handling: Versions with semver pre-release tags (e.g., 1.21.76-2) are treated as revision-stripped (-2 is a Debian revision, not a semver pre-release)

Installation Paths

BALLER supports four installation paths, selected automatically based on PackageSource:

  • Archive path (GitHub / BallerRegistry): select the asset built for the host platform → download archive → SHA-256 hex verify → extract → locate the binary → symlink it → record in DB. Locating the binary gates the rest: if the extracted tree holds no executable, the command fails with NoBinaryFound, removes the extract directory and the cached archive, and writes no database row — it never links or records a package it could not install. Downloader::no_binary_error implements that cleanup once, and draft, build, substitute and update all call it. See error-handling.md for how the GitHub asset is chosen.
  • Chocolatey path (Chocolatey/NuGet): download .nupkg archive → SHA-512 base64 decode and verify → extract as zip → record in DB. NuGet .nupkg files are zip archives and are extracted with the zip handler. Archives without a recognized extension (e.g., cached from API URLs) fall back to zip extraction.
  • System path (PackageSource::System): delegate to native package manager via install_system_package() in http/system.rs, which runs sudo apt-get install -y <name> (or dnf/pacman equivalent) and records the package in the DB. No archive is downloaded, no symlink is created.
  • Cargo path (PackageSource::Cargo): delegate to the local toolchain via install_cargo_package() in http/cargo.rs, which runs cargo install <crate> — without sudo, since cargo builds into the user's ~/.cargo/bin — and records the package in the DB. No archive is downloaded, no symlink is created.

The downloader (Downloader::download_and_extract) refuses to handle system and cargo packages, returning a PackageManagerError if invoked on one — this is a defensive check, since the draft command should always dispatch those packages to their own path.

Package Security (Referee)

AppContext carries a Referee alongside the downloader and registry client, built from config.referee and GlobalFlags::no_referee. It runs in two phases:

  • Phase A (Referee::gate) runs in draft, update and substitute on the fully resolved plan, and in build on its one package (after the download URL — and so the installed version — is resolved, or before a Cargo project is compiled), before the install loop and before the pre_install hook. Because no package in the plan has been fetched at that point, a block leaves zero symlinks, zero roster rows and an empty cache wherever in the plan the offending package sits.
  • Phase B (Referee::screen_artifact) runs after download_and_extract and before create_symlink. A blocking finding calls Downloader::purge_download — the cleanup no_binary_error also uses — so the extract directory and the cached archive are both removed. In update it runs before the old extract directory is pruned, leaving a rejected upgrade's predecessor linked and runnable. build's Cargo-project path uses Referee::screen_binary instead: one file, the compiled binary, scanned without purging, because it is the user's own build output.

Verdicts are cached in the referee_cache table of the same SQLite database, keyed by (ecosystem, name, version); only clean and vulnerable are stored. Rows are durable unless [referee] cache_ttl_days is set, in which case a clean row older than the TTL is re-queried; vulnerable rows are exempt. The installed_packages table carries an advisory column (added by migration on open) holding a package's declared advisory identity, so baller referee re-checks it under the same identity the install used.

The modules live in src/security/ and are mapped in referee.md.

Error Handling and Rollback

Partial Failure Rollback

draft, substitute and update install a plan as separate persisted steps (extract, link, record) with no transaction around them, so each notes every step in a commands::rollback::Session as it goes. On a failure the session is unwound newest package first, then the original error is returned. A package new to the run is unlinked, taken off the roster and purged from the cache (extract dir and archive). A package the run found already installed — a draft --force reinstall, a dependency substitute re-fetched, update's root — gets its snapshotted roster row and dependency rows written back and its link pointed at its old binary again. update prunes the old release's extract dir only after the new one is linked and recorded, so that binary is always still there to go back to. Rollback is best-effort (a step that cannot be undone is logged at --verbose) and runs no hooks; what a system package manager or cargo install did is not undone.

Orphan Dependency Cleanup

When ejecting a package, BALLER checks for orphaned dependencies — packages that were installed as transitive dependencies and are no longer required by any remaining installed package. Orphans are automatically removed with a warning message. This relies on the user_installed flag in the database: packages installed as dependencies are marked user_installed = false, while packages explicitly installed via draft are marked user_installed = true.

Confirmation Prompts

Destructive commands (eject, sweep, substitute, inject) prompt for confirmation unless the global --yes / -y flag is passed. --yes is one of the global flags declared on BallerCommand and carried to commands via AppContext::flags (src/context.rs). utils::fs::confirm takes the --json flag too: prompts go to stderr, and under --json it returns ConfirmationRequired instead of prompting.

JSON Output

Under --json every run leaves exactly one JSON document on stdout. Commands print theirs through utils::output::print_json, which also records that a document went out. main reports a failure as utils::output::error_json (command, error, code) on stdout, unless a document was already printed (referee --fail-on), in which case the error goes to stderr as text. Errors raised before clap returns a parsed command still know --json and the subcommand from cli::parse::scan_args, which reads the raw arguments; the same scan rewrites --json with -h/--help/-V/--version, which clap would otherwise answer itself in text, into the JSON-aware help [<command>] and version subcommands. Child processes inherit baller's stdout, so hooks (core::hooks) and the package-manager installs (http::system, http::cargo) take theirs from utils::output::child_stdout(): stderr under --json, inherited otherwise. main sets the mode once with set_json_mode, the way it sets colored's override, so the flag is not threaded through every run_hook caller.

Version Comparison

The update command uses parse_version_flexible() for version comparison, which handles Debian epoch prefixes, revision suffixes, embedded tags, date-based versions, and multi-segment versions — not just strict semver. Falls back to string comparison when both versions cannot be parsed.