Skip to content

fix(workflows): read complete n8n inventories and validate list responses - #3835

Closed
tang-vu wants to merge 2 commits into
Osmantic:mainfrom
tang-vu:fix/workflows-n8n-pagination-20260908
Closed

tang-vu wants to merge 2 commits into
Osmantic:mainfrom
tang-vu:fix/workflows-n8n-pagination-20260908

Conversation

@tang-vu

@tang-vu tang-vu commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Closes #7293

Summary

Read the complete n8n workflow inventory within one five-second deadline, and validate list response envelopes before publishing workflow or execution rows. Workflows after n8n's first page now appear in the catalog and can be removed through the existing endpoint. A malformed later page cannot publish a partial inventory or authorize deletion from that inventory.

Why this matters

The shipped n8n 2.6.4 handler defaults to 100 workflows per page. Previously ODS treated that first page as the whole inventory. Successful HTTP responses with a non-object root, missing/non-list data, or non-object rows also escaped into catalog matching and execution-history responses, causing exceptions or invalid output.

Inventory reads retain live API-key loading, discard all accumulated rows after an unavailable or malformed page, propagate cancellation, and close the session. Invalid/repeated cursors return 502, including falsey non-string cursors that previously looked like the end of pagination. Malformed execution-list responses use the existing empty-list/error response. This change validates list envelopes and object rows; it does not redefine workflow ownership matching or paginate the limited recent-executions endpoint.

Overlap check

This updates canonical #3835 and incorporates the response-envelope/object-array validation from closed #3681 and #2469. The older pagination variant #5829 is not resubmitted. The current live N8N_API_KEY reader is preserved. Activation, catalog identity, duplicate import, and export changes are outside this scope. Only routers/workflows.py and tests/test_workflow_pagination.py differ from the recorded main base.

Provider contracts were checked against the shipped n8n 2.6.4 workflow handler and its executions handler.

AI Assistance

The human contributor and Codex collaborated on this PR. The contributor provided project direction, scope decisions and environment setup; Codex assisted with analysis, implementation or reconstruction, tests, review and PR preparation. The validation commands were executed by Codex, with results and limitations documented in this PR. The human contributor remains responsible for the final changes and validation claims. Maintainer review and applicable release gates remain required before merge.

Release Lane / Changed Surface

  • Mainline change targeting main.
  • Dashboard API workflow reads and their catalog/removal consumers; medium risk.

Validation

  • Base: b4f015a921c59eeb20317c5f48adf2dfb43b1430.
  • Candidate: d938caab853b702cf91104d24d821d93602010af.
  • Windows x64, CPython 3.14.2, real loopback HTTP fixture using n8n's cursor shape.
  • Incremental red checks before the corresponding fixes: 11 failing inventory cases (one existing pass), six failing execution cases, four failing falsey-cursor cases.
  • Four workflow suites (test_workflow_pagination.py, test_workflows.py, test_workflows_n8n_api_key.py, test_workflow_missing_dependencies.py): 84 passed.
  • Fresh-main replay of the 30-case HTTP fixture: 29 failed, one passed. One failure is the absent timeout compatibility helper; the remaining failures exercise endpoint behavior.
  • Ruff's repository error selection and git diff --check: passed.
  • CI test inventory: passed; .github/workflows/dashboard.yml runs the service's entire tests/ directory under Python 3.11.
  • New commit identity gate: passed for two scoped commits above main.
  • Separate AI review: no blocking findings within this scope.
  • Final candidate GitHub CI: all 68 checks succeeded, including the full Dashboard API suite, all mypy jobs, identity gate and extended contract suite.

The earlier branch carried historical public-beta ancestry rejected by the new commit-identity gate. This update reconstructs only the two changed files on current main, preserving the old branch locally rather than rewriting other contributors' identities. The initial recovery commit 462aa4037b3621d1f8d4d42915bb4838ef1729eb had full-tree equivalence with tested/reviewed 657c385950bd922e986d5482895b45a3b3be8e68. The final candidate additionally fixes Python-version-sensitive timeout imports and annotates pagination parameters and the HTTP fixture state. All four workflow suites were rerun after these type fixes: 84 passed. An isolated mypy 2.4.0 check reported no errors in either changed file; 27 errors in three unrelated imported files remain governed by the repository baseline.

Operational Validation / Limits

Live provider validation at this candidate SHA passed on Ubuntu 24.04.5 / WSL2, Python 3.12.3 and Docker Desktop 29.8.1, using shipped n8nio/n8n:2.6.4@sha256:b962d7f8ba9e990a0c530256d841fdc52312dce32173f29808e29a9430811ad3:

  • Created 101 workflows through the real n8n public API. Its default response contained 100 rows and a cursor; the main-base helper returned 100 while this candidate returned all 101 unique workflows.
  • The ODS HTTP catalog consumer resolved an owned workflow from the later page. Its DELETE consumer removed that exact ID, confirmed by a real n8n 404 and remaining inventory of 100.
  • A real webhook workflow was activated and executed successfully; its genuine execution row reached the ODS HTTP history response.

These checks used an isolated n8n container/volume, a private catalog and FastAPI TestClient with Dashboard auth overridden; outbound aiohttp traffic reached genuine n8n. They do not establish full installed Dashboard browser/auth behavior. The owned fixture container, volume and temporary credentials were removed afterward. Malformed responses and cancellation remain covered by controlled regression fixtures. Completed full-product Portal/model validation and native Python 3.10 remain deferred; no human validation is claimed. No persisted state or schema changes; reverting restores the previous inventory/read behavior.

@tang-vu
tang-vu changed the base branch from main to public-beta September 16, 2026 10:37
This was referenced Sep 16, 2026
@tang-vu
tang-vu force-pushed the fix/workflows-n8n-pagination-20260908 branch from 0429796 to 462aa40 Compare October 4, 2026 09:18
@tang-vu tang-vu changed the title fix(workflows): read the complete n8n inventory within one deadline fix(workflows): read complete n8n inventories and validate list responses Oct 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ODS workflow inventory omits n8n workflows beyond the first page

1 participant