Skip to content

fix: list large artifact datasets and explain the listing cap - #749

Closed
devin-ai-integration[bot] wants to merge 18 commits into
mainfrom
devin/1792687492-artifact-listing-cap
Closed

devin-ai-integration[bot] wants to merge 18 commits into
mainfrom
devin/1792687492-artifact-listing-cap

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Fixes OR-585.

Summary

The Artifacts pane listing (local::files::collect_tree) stopped at a global MAX_ENTRIES = 2000 nodes. That count is depth-first and follows read_dir order, so an extracted dataset with 6,750 images in 15 class folders showed only 5 folders, never showed the other 10, and only said "Listing truncated — the folder has more artifacts."

  • MAX_ENTRIES goes from 2,000 to 10,000, so a dataset of this size now lists in full.
  • collect_tree lists all siblings in a folder before descending into any of them. If the cap is hit, every folder it reached still appears (the class folders stay visible and some just list fewer files), and the result is still sorted. Before, an early return skipped the sort.
  • collect_tree recurses with each directory's original entry.path(), stops descending once an entry has been omitted, and only sets truncated when a listable (non-ignored) entry is actually dropped.
  • The UI falls back to 2,000 when an older remote host omits limit.
  • fingerprint (the 2Hz SSE change detector) now hashes the collect_tree output instead of running its own separate depth-first walk (hash_dir, removed). It now covers exactly the nodes that are listed, plus the truncated/partial flags.
  • Folders the cap cut short are marked partial. GET /api/projects/{id}/files?dir=<path> (checked with resolve_contained) lists one folder on its own with a fresh 10,000 budget. In the UI, a partial folder stays collapsed rather than looking empty. Opening it fetches that listing and grafts it into the tree. Opened partial folders are outside the SSE fingerprint, so they poll every 2s while visible. A failed load shows an error or closes the folder, and reopening it retries.
  • ArtifactsListing gains limit. The truncation notice now gives the cap and the on-disk folder: "Showing the first 10,000 artifacts. The rest are still on disk in {dir}." All 9 locales are updated and ui/dist is rebuilt.
  • An opened partial folder whose own ?dir= listing is also capped (more than 10,000 entries directly inside it) gets its own notice naming that folder's on-disk path.

Test plan

  • New capped_listing_keeps_every_sibling_folder unit test (covers sibling preservation, partial, and the fresh budget for a sub-listing) and listing_exactly_at_cap_is_not_truncated, plus the existing local::files tests
  • cargo fmt --check, cargo clippy --all-targets -- -D warnings, pnpm build (i18n lint, typecheck), UI node tests
  • Reproduced on main (Linux, dev slot, a real Claude Code chat creates 15 × 450 PNGs): only 5 of 15 class folders are shown, with the truncation notice
  • Post-fix check on this branch at 3d2dd843 (after merging origin/main), same flow with real Claude Code chats:
    • 15 × 450 = 6,750 PNGs: all 15 class folders are sorted, class_15 lists through img_0449.png, and there's no truncation notice.
    • 15 × 800 = 12,000 PNGs: the notice shows 10,000 and the on-disk dir, and all 15 root folders appear. class_06 was partial with zero children and showed collapsed. Clicking it loaded all of its files (807, through img_0806.png), and the preview works.
    • Live update: with class_06 open, Claude added a new PNG, and it appeared about 1.2s later with no reload.
    • A folder with 10,050 PNGs directly inside it starts partial. Opening it lists 10,000 rows and shows a second notice for <artifacts dir>/wide_10050.
  • Not exercised in the app: the failed-load retry path, polling suppression under a collapsed parent, the missing-limit fallback for older hosts, and a listing at exactly the cap (that case is covered by a unit test)

Bug reproduction on main

Bug reproduction on main

Post-fix functionality on this branch

Post-fix functionality

Link to Devin session: https://app.devin.ai/sessions/8dd441464c614938afa8bc3b73b32c9b
Open in Devin Desktop: https://app.devin.ai/desktop/session/8dd441464c614938afa8bc3b73b32c9b?variant=devin

Raise the artifact listing cap from 2,000 to 10,000 nodes, list each
folder's siblings before descending so a capped tree never drops whole
folders, derive the SSE fingerprint from the same traversal, and state
the cap and on-disk location in the truncation notice.

Fixes OR-585

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@greptile-apps

greptile-apps Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new blocking issue was found.

Summary

Raises the artifact listing cap to 10,000 and keeps sibling folders visible when a listing reaches it.

  • Artifact listings keep sibling folders visible when a tree hits its cap.
  • The Artifacts pane can load more files from a cut-short folder.

No new actionable issues were found. The previous threads were unnumbered, so none require numbered finding entries.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Root artifact listing] --> B{Folder partial?}
  B -->|No| C[Show listed children]
  B -->|Yes| D[Keep folder collapsed]
  D --> E[User opens folder]
  E --> F[Fetch folder with fresh budget]
  F -->|Success| G[Show fetched children]
  G --> H[Poll while open]
  F -->|Failure| I[Show error and allow retry]
Loading

Reviews (20) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." · Reviewed by Greptile

Comment thread src/local/files.rs Outdated
Comment thread src/local/files.rs Outdated
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

Comment thread src/local/files.rs Outdated
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

1 similar comment
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

@devin-ai-integration devin-ai-integration Bot added the awaiting review Devin bugfix PR is open and needs human review before merge label Oct 22, 2026
…fact-listing-cap

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	ui/dist/assets/PdfPreview-Bh6puvd-.js
#	ui/dist/assets/PdfPreview-Cr9nWodS.js
#	ui/dist/assets/PdfPreview-Dn2XGhUv.js
#	ui/dist/assets/index-BwOfMXM0.js
#	ui/dist/assets/index-DHbi4ZWi.js
#	ui/dist/assets/index-DNEZ9j_w.js
#	ui/dist/index.html
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

Comment thread ui/src/components/ArtifactsTab.tsx Outdated
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

…fact-listing-cap

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	ui/dist/assets/PdfPreview-9b5qPNPi.js
#	ui/dist/assets/PdfPreview-Bh6puvd-.js
#	ui/dist/assets/PdfPreview-DHY1H0AM.js
#	ui/dist/assets/index-CQIDX7GA.js
#	ui/dist/assets/index-DNEZ9j_w.js
#	ui/dist/assets/index-DjlqofzY.js
#	ui/dist/index.html
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

Folders the capped listing didn't finish are marked partial; opening one
lists it on its own (?dir=) with a fresh cap, so every file stays reachable
through directory navigation.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

Comment thread ui/src/components/ArtifactsTab.tsx
Comment thread ui/src/components/ArtifactsTab.tsx
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

Comment thread ui/src/components/ArtifactsTab.tsx Outdated
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

Comment thread ui/src/components/ArtifactsTab.tsx
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

…fact-listing-cap

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	ui/dist/assets/PdfPreview-CfnQeP9Q.js
#	ui/dist/assets/PdfPreview-CsvPNokB.js
#	ui/dist/assets/PdfPreview-DHY1H0AM.js
#	ui/dist/assets/index-B1Sbkrve.js
#	ui/dist/assets/index-BsYgd7Kt.js
#	ui/dist/assets/index-CQIDX7GA.js
#	ui/dist/index.html
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

Comment thread src/local/files.rs Outdated
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

…fact-listing-cap

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	ui/dist/assets/PdfPreview-CKyJoJPc.js
#	ui/dist/assets/PdfPreview-CsvPNokB.js
#	ui/dist/assets/PdfPreview-YHPVr7ta.js
#	ui/dist/assets/index-BsYgd7Kt.js
#	ui/dist/assets/index-CV-jxtEm.js
#	ui/dist/assets/index-Cqdzdm_Q.js
#	ui/dist/index.html
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

…fact-listing-cap

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	ui/dist/assets/PdfPreview-Arq-xDe9.js
#	ui/dist/assets/PdfPreview-CKyJoJPc.js
#	ui/dist/assets/PdfPreview-DyW6mz8f.js
#	ui/dist/assets/index-BRHnE0ar.js
#	ui/dist/assets/index-BfOmY9IZ.js
#	ui/dist/assets/index-Cqdzdm_Q.js
#	ui/dist/index.html
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

…fact-listing-cap

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	ui/dist/assets/PdfPreview-Arq-xDe9.js
#	ui/dist/assets/PdfPreview-BESv4u66.js
#	ui/dist/assets/PdfPreview-DMaQ28uR.js
#	ui/dist/assets/index-BWXSOt3p.js
#	ui/dist/assets/index-BfOmY9IZ.js
#	ui/dist/assets/index-CHTdcS2r.js
#	ui/dist/index.html
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

Comment thread ui/src/components/ArtifactsTab.tsx Outdated
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review — bb1671d keeps each opened folder's truncated flag (loadedEntries now returns the whole listing) and shows a notice naming that folder's on-disk path; please re-check finding 1.

…fact-listing-cap

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	ui/dist/assets/PdfPreview-B3N1d7je.js
#	ui/dist/assets/PdfPreview-BESv4u66.js
#	ui/dist/assets/PdfPreview-BvWccRfV.js
#	ui/dist/assets/index-CGSgsrbT.js
#	ui/dist/assets/index-CHTdcS2r.js
#	ui/dist/assets/index-DFE76ylj.js
#	ui/dist/index.html
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

…fact-listing-cap

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	ui/dist/assets/PdfPreview-1jfQSPzX.js
#	ui/dist/assets/PdfPreview-BvWccRfV.js
#	ui/dist/assets/PdfPreview-Drs1tUZu.js
#	ui/dist/assets/index-CE2Gq4kH.js
#	ui/dist/assets/index-CujjRTy1.js
#	ui/dist/assets/index-DFE76ylj.js
#	ui/dist/index.html
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting review Devin bugfix PR is open and needs human review before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants