Skip to content

ci(container): publish official Docker images to GHCR #379

Description

@Fgeeha

Before you start

  • I have read the Roadmap and this is not already planned there.
  • I have read the FAQ.
  • I have searched existing issues and found no duplicate.

Applies to

Engine

What problem does this solve

FreeToken currently has no official container image published by the project.

Users who want to run ft serve in Docker must assemble their own CUDA, Python,
build-toolchain, and FreeToken environment. This makes deployments less
reproducible and leaves users without a project-maintained image tag to use in
automation.

I checked the Roadmap, FAQ, and existing issues before opening this request.

Proposed solution

Add a project-maintained CUDA development container image and a GitHub Actions
workflow that:

  • builds the image for pull requests and manual workflow runs without publishing;
  • publishes only from the upstream repository on pushes to main and version tags;
  • publishes to ghcr.io/flashml-org/freetoken;
  • provides latest for the default branch plus semantic-version and commit-SHA tags;
  • uses GitHub Actions cache for Buildx layers;
  • documents how to mount a model directory and run ft serve with Docker.

The image should retain nvcc, because FreeToken can compile CUDA kernels at
first use.

Alternatives considered

  • Users maintain local Dockerfiles: flexible, but duplicates setup work and
    produces inconsistent CUDA and dependency combinations.
    • Publish only Python wheels: already useful, but does not provide a
      reproducible deployment image.
    • Use a runtime-only CUDA image: smaller, but unsuitable when JIT compilation
      of FreeToken CUDA kernels is needed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions