add pagination & remote search flow for remote storage

[Light2Dark]

📝 Summary

Related to #9662

Screen.Recording.2026-05-28.at.10.24.22.AM.mov

📋 Pre-Review Checklist

• For large changes, or changes that affect the public API: this change was discussed or approved through an issue, on Discord, or the community discussions (Please provide a link if applicable).
• Any AI generated code has been reviewed line-by-line by the human PR author, who stands by it.
• Video or media evidence is provided for any visual changes (optional).

✅ Merge Checklist

• I have read the contributor guidelines.
• Documentation has been updated where applicable, including docstrings for API changes.
• Tests have been added for the changes made.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
marimo-docs	Ready	Preview, Comment	May 28, 2026 8:35am

[cubic-dev-ai]

2 issues found across 16 files

Architecture diagram

sequenceDiagram
    participant UI as React UI (StorageInspector)
    participant Hook as useStorageEntries Hook
    participant State as Jotai Store (state.ts)
    participant API as Server API (commands.py)
    participant Backend as StorageBackend (storage.py)
    participant Obstore as Obstore (S3/GCS)
    participant Fsspec as FsspecFilesystem (local/SFTP)

    Note over UI,Fsspec: Storage List with Pagination

    UI->>Hook: Expand directory or section
    Hook->>State: Check cache (entriesByPath)
    alt Cache miss
        State-->>Hook: No cached entries
        Hook->>API: NEW: listEntries(namespace, prefix, limit, pageToken=null)
        API->>Backend: list_entries(prefix, limit, page_token)
        
        alt Obstore backend
            Backend->>Obstore: list_with_delimiter(prefix)
            Obstore-->>Backend: Raw entries (may be truncated at 1000)
            Backend->>Backend: _paginate_entries(offset=0, limit)
            Backend-->>API: StorageListResult(entries, nextPageToken, mayHaveMore)
        else Fsspec backend
            Backend->>Fsspec: ls(prefix, detail=True)
            Fsspec-->>Backend: File list
            Backend->>Backend: _paginate_entries(offset=0, limit)
            Backend-->>API: StorageListResult(entries, nextPageToken)
        end
        
        API-->>Hook: NEW: StorageEntriesNotification(entries, nextPageToken, mayHaveMore)
        Hook->>State: NEW: setEntries({entries, nextPageToken, mayHaveMore})
        State-->>Hook: entries, hasMore, mayHaveMore
        Hook-->>UI: Render entries + "Load more" button
    else Cache hit
        State-->>Hook: Cached entries + pagination metadata
        Hook-->>UI: Render entries
        alt hasMore (nextPageToken != null)
            UI->>Hook: Show "Load more" button
        else mayHaveMore (no token, but may exist)
            UI->>Hook: Show "May exist more" indicator
        end
    end

    Note over UI,Hook: Load More Click

    alt hasMore is true
        UI->>Hook: loadMore()
        Hook->>Hook: setIsLoadingMore(true)
        Hook->>API: NEW: listEntries(namespace, prefix, limit, pageToken=nextPageToken)
        API->>Backend: list_entries(prefix, limit, page_token)
        
        alt Obstore backend
            Backend->>Obstore: list_with_delimiter(prefix)
            Obstore-->>Backend: Raw entries
            Backend->>Backend: _paginate_entries(offset=parseInt(pageToken), limit)
            Backend-->>API: StorageListResult(entries, nextPageToken, mayHaveMore)
        else Fsspec backend
            Backend->>Fsspec: ls(prefix, detail=True)
            Fsspec-->>Backend: File list
            Backend->>Backend: _paginate_entries(offset=parseInt(pageToken), limit)
            Backend-->>API: StorageListResult(entries, nextPageToken)
        end
        
        API-->>Hook: NEW: StorageEntriesNotification(entries, nextPageToken, mayHaveMore)
        Hook->>State: NEW: setEntries({entries, append:true, nextPageToken, mayHaveMore})
        State-->>Hook: Appended entries + updated metadata
        Hook-->>UI: Render appended entries
        alt Has more pages
            UI->>Hook: Keep "Load more" button
        else No more pages but mayHaveMore
            UI->>Hook: Show "Maybe more" indicator
        end
    end

    Note over Hook,State: Error Handling

    alt Load more fails
        Hook->>Hook: catch error -> setLoadMoreError
        Hook-->>UI: Show error message with retry
        UI->>Hook: loadMore() retry
    end

Loading

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

2 issues found across 6 files (changes from recent commits).

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>