docsy(v2): document ephemeral/disk storage in task resources#1198
Open
ppiegaze wants to merge 1 commit into
Open
docsy(v2): document ephemeral/disk storage in task resources#1198ppiegaze wants to merge 1 commit into
ppiegaze wants to merge 1 commit into
Conversation
Add an "Ephemeral storage" section to the task-configuration Resources page explaining the `disk` parameter: what ephemeral storage is, how it maps to the Kubernetes ephemeral-storage resource, its pod-tied lifecycle, that it is a single value (not a request/limit tuple), and its default (unset) behavior. Cross-links the union-only resource- management guidance. Closes the DOC-101 gap (disk was only listed as a one-line parameter with no explanation of the mechanism). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Peeter Piegaze <1153481+ppiegaze@users.noreply.github.com>
GHA build & deploy previewBuilt by
Updated automatically on every push. |
ppiegaze
requested review from
EngHabu,
cosmicBboy,
kumare3 and
samhita-alla
as code owners
Contributor
There was a problem hiding this comment.
Pull request overview
Adds missing conceptual documentation for Flyte SDK’s Resources(disk=...) so users understand what ephemeral storage is, what it applies to, and its lifecycle/risks in Kubernetes-backed execution.
Changes:
- Adds a new “Ephemeral storage” section describing the
diskparameter and its Kubernetesephemeral-storagemapping. - Clarifies what ephemeral storage covers vs. offloaded
flyte.io.File/flyte.io.Dir. - Adds a Union-only cross-link to existing ephemeral storage guidance in resource management.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| **Single value, not a request/limit range.** Unlike `cpu` and `memory`, `disk` takes a single string, not a `(request, limit)` tuple. | ||
|
|
||
| **Default behavior.** If you don't set `disk`, no ephemeral-storage request or limit is applied. The pod can still write to node-local disk, but it may be evicted if the node comes under storage pressure. Tasks doing heavy local data processing should set `disk` explicitly. |
| ) | ||
| ``` | ||
|
|
||
| Under the hood, `disk` maps to the Kubernetes [`ephemeral-storage`](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#local-ephemeral-storage) resource on the task's pod. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an Ephemeral storage section to the task-configuration Resources page (
content/user-guide/task-configuration/resources.md) explaining thediskparameter offlyte.Resources:File/Dirstorage;diskmaps to the Kubernetesephemeral-storageresource;diskis a single value, not a(request, limit)tuple (unlikecpu/memory);Why
Resolves DOC-101 ("How does ephemeral storage work?"). The
diskfield was shipped but only listed as a one-line parameter ("Ephemeral storage —10Gi") with no explanation of the mechanism. Genuine partial-coverage gap.Verify-live evidence (code is authoritative)
flyte._resources.Resources):disk: Optional[str] = None; docstring — "The amount of disk to allocate to the task. This is a string of the form10GiB." Single value, no tuple.flyte/_internal/runtime/resources_serde.py):_get_disk_resource_entry→tasks_pb2.Resources.ResourceName.EPHEMERAL_STORAGE; andpod_spec_from_resourcesmapsephemeral_storage→ephemeral-storage..../task-configuration/resources/): confirmeddiskonly listed, unexplained.Draft — held per docsy publish-gate (feature already shipped in the 2.0.x SDK; ready for review).