You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
No lower-bound clamp: pageSize=0 or negative bypasses pagination logic.
2. countDocuments(filter) on every list, no upper bound
Every list endpoint runs countDocuments(filter) in parallel with the find. At scale (100k+ docs) this is slower than the find itself. ConnectorController.syncConnectors (workflows/controllers/ConnectorController.ts:272-285) does find().toArray() with no limit — loads the entire catalog into memory.
3. Cron uses env, not Settings
runner/session-manager.ts:24-27 reads VERIFY_CRON_INTERVAL_HOURS once at module load. The Settings document (admin/models/settings.ts:11) stores verifyCronIntervalHours and lets operators update it via PUT /settings. Updates are silent no-ops until pod restart.
Same for Settings.eventRetentionDays — wired into the TTL index at boot, never reapplied.
Use estimatedDocumentCount() when no filter is applied; for filtered queries, allow a ?countMode=skip|estimate|exact query param so callers can opt out.
For syncConnectors, paginate the upload batch or stream.
Re-read Settings.verifyCronIntervalHours on boot; subscribe to settings changes (Mongo change stream OR a periodic poll on a settingsVersion field) and call stopVerifyCron(); startVerifyCron();. For expireAfterSeconds on event_logs, run db.runCommand({collMod: ...}) on settings change.
Severity: Medium
Three related API-surface hygiene issues.
1. Pagination defaults vary
/audit(admin/controllers/AuditController.ts:91-101)/ingest/history(ingest/controllers/IngestController.ts:169-173)/schemas(schemas/controllers/SchemaController.ts:108-112)/workflows(workflows/controllers/WorkflowController.ts:97-101)/connectors(workflows/controllers/ConnectorController.ts:102-106)/runs/history(runner/controllers/HistoryController.ts:36-44)No lower-bound clamp:
pageSize=0or negative bypasses pagination logic.2.
countDocuments(filter)on every list, no upper boundEvery list endpoint runs
countDocuments(filter)in parallel with the find. At scale (100k+ docs) this is slower than the find itself.ConnectorController.syncConnectors(workflows/controllers/ConnectorController.ts:272-285) doesfind().toArray()with no limit — loads the entire catalog into memory.3. Cron uses env, not Settings
runner/session-manager.ts:24-27readsVERIFY_CRON_INTERVAL_HOURSonce at module load. TheSettingsdocument (admin/models/settings.ts:11) storesverifyCronIntervalHoursand lets operators update it viaPUT /settings. Updates are silent no-ops until pod restart.Same for
Settings.eventRetentionDays— wired into the TTL index at boot, never reapplied.Remediation
parsePagination(page, pageSize)util inshared/crud.ts(depends on [Feature] Centralize HTTP client + CRUD scaffolding (kill duplication) #33). Clamp1 <= pageSize <= 100,page >= 1. Apply uniformly.estimatedDocumentCount()when no filter is applied; for filtered queries, allow a?countMode=skip|estimate|exactquery param so callers can opt out.syncConnectors, paginate the upload batch or stream.Settings.verifyCronIntervalHourson boot; subscribe to settings changes (Mongo change stream OR a periodic poll on asettingsVersionfield) and callstopVerifyCron(); startVerifyCron();. ForexpireAfterSecondsonevent_logs, rundb.runCommand({collMod: ...})on settings change.Coupled with #32 (config) and #33 (CRUD helpers).