Skip to content

cds get: cap GitHub tarball download/extraction size #505

Description

@RonaldHensbergen

Summary

cds get's GitHub tarball download path (_download_github_repository in
cli/getter.py, introduced in #493) has no upper bound on the size of the
archive it downloads and buffers in memory before writing to disk.

with urlopen(request, timeout=30) as response:  # nosec B310
    archive_bytes = response.read()

archive_path = work_dir / "repository.tar.gz"
archive_path.write_bytes(archive_bytes)

response.read() has no size limit, and the subsequent
archive.extractall(extract_root, filter="data") has no cap on total
extracted size either. A malicious or compromised --remote <owner/repo>
(or an unexpectedly huge upstream repository) could return an very large
tarball, causing excessive memory use while buffering the full response body,
and/or exhausting disk space on extraction (zip-bomb-style amplification).

Proposed fix

  • Stream the download in chunks (rather than a single response.read()) and
    enforce a maximum archive size, aborting with a clear GetError if
    exceeded.
  • Consider also capping total extracted size (or per-member size) when
    iterating the tarball, since filter="data" protects against path
    traversal/permission issues but not size amplification.
  • Pick a reasonable default limit (e.g. a few hundred MB) since this is meant
    to fetch a profiles/+modules/ subset of a data-stack repo, not arbitrary
    large binary content.

Context

Raised as a minor, non-blocking observation during review of #493 (which
otherwise correctly uses the safe filter="data" tarfile extraction mode and
scopes network access to a fixed GitHub API host with a 30s timeout).

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:cliCLI commands and behaviorenhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions