Skip to content

Add Hugging face via nexus #78

Description

@JaimieWi

Hugging Face proxy added to the Nexus shared service (tre-shared-service-sonatype-nexus 3.3.6 → 3.3.7)

What changed:

  • New Nexus proxy repository huggingface, defined in huggingface_proxy_conf.json, proxying https://huggingface.co. Uses the huggingface base type/repo type, anonymous read, content/metadata cached for 1440 minutes (24h), negative caching enabled.
  • Egress allow-list updated in locals.tf — added huggingface.co,*.huggingface.co to nexus_allowed_fqdns so the Nexus VM's firewall rules permit the outbound calls to the Hub.
  • Docs updated in nexus.md: added the FQDN row, a "Current Repos" table row, and a new Using Hugging Face section.
  • Version bump in porter.yaml (3.3.6 → 3.3.7) and a CHANGELOG.md entry.

How researchers use it:
Unlike PyPI/Conda/Docker/R, the proxy isn't auto-configured on the Guacamole VM images — a researcher has to set these env vars themselves before using huggingface_hub/transformers/datasets/huggingface-cli:

export HF_ENDPOINT="https://nexus-{TRE_ID}.{LOCATION}.cloudapp.azure.com/repository/huggingface"
export HF_HUB_DOWNLOAD_TIMEOUT=120
export HF_HUB_ETAG_TIMEOUT=1800
export HF_HUB_DISABLE_XET=1
  • HF_ENDPOINT redirects all Hub calls through the Nexus proxy instead of hitting huggingface.co directly.
  • The timeout bumps account for large model/dataset files.
  • HF_HUB_DISABLE_XET=1 is required because Nexus doesn't support Hugging Face's newer Xet transfer protocol — without it, downloads would silently try (and fail) to use Xet.
  • Known limitation: only public models/datasets work. Gated or private repos need bearer-token auth support, which requires Nexus Repository 3.95+; this template currently pins the Nexus server itself to 3.77.2, so that's out of scope until the underlying Nexus version is upgraded.

Possible follow-up (parallel to the MOTD nav-doc idea): since this proxy needs manual env var setup that's easy to forget, it could be pre-baked into the Guacamole VM images (like the PyPI/Conda/Docker proxies already are) instead of relying on researchers to configure it by hand each session.

https://help.sonatype.com/en/create-a-hugging-face-repository.html

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions