Add tag-based document indexing allowlist

[MichelleAntunes]

Fixes #278.

Why this matters

Context Chat currently indexes every document matching its mimetype whitelist, with no way to exclude anything. For organizations handling sensitive or confidential material — legal, healthcare, HR, compliance-regulated industries — there's currently no way to control which content gets sent to an LLM for embedding. Everything eligible is indexed by default, which is the wrong default for these environments: it means adopting Context Chat requires either accepting that every eligible document becomes part of the AI's knowledge base, or not adopting it at all.

This PR takes an allow-list approach rather than a block-list one, on purpose: with an allow-list, nothing is exposed to the AI unless someone explicitly marks it as safe to include. A block-list ("index everything except X") is easy to get wrong by omission — someone forgets to exclude a folder, and it's silently indexed. An allow-list fails safe: if you forget to tag something, it simply isn't indexed, which is the safer failure mode for sensitive content.

This also has to work for instances that are already running Context Chat today, with documents already indexed under the existing "index everything" behavior. The feature is opt-in: existing instances keep indexing everything by default, and only change behavior if an admin deliberately switches modes. Nothing changes for anyone who doesn't touch the new setting.

How it works

Admins get a new toggle on the Context Chat admin settings page:

• Index all documents (default, existing behavior, unchanged)
• Index only documents tagged 'AI knowledge'

The tag is a regular Nextcloud system tag, so it uses infrastructure that already exists and that admins/users already understand, rather than introducing a new concept. Tagging a folder covers everything inside it (tag inheritance walks up the file's ancestor folders), so an admin can mark, say, a single top-level "AI knowledge" folder and everything moved into it becomes eligible, without needing to tag every individual file.

Switching modes has real consequences that need to happen automatically, not just at settings-save time:

• Switching to tag-only mode schedules a background cleanup that walks every file already in the index and removes anything that isn't tagged. Otherwise, documents indexed before the switch would silently remain searchable forever, defeating the point of the feature.
• Switching back to "all" triggers a full reindex, since documents that were previously excluded (or never indexed because they didn't have the tag) need to be picked up.

Both of these are background jobs, not something the admin has to wait on synchronously — the admin page's indexing status (already existing in Context Chat) reflects live progress for both flows, so the admin can see when the switch has actually finished taking effect, not just when the setting was saved.

Design decisions worth calling out

Why extend the existing event-listener pattern instead of a new mechanism: Nextcloud already has a system tag mechanism and Context Chat already listens for filesystem events (FileListener.php) to decide what to index. This PR follows that same shape, listening for TagAssignedEvent/TagUnassignedEvent in addition to the existing file events, rather than introducing a parallel, separate indexing pipeline just for tags.

Why the completion-check logic was extracted and rewritten, not just extended: The existing "has initial indexing completed?" check relied on a single global last_enqueued_db_id marker. On instances with multiple users (each with their own storage), that marker gets overwritten by whichever user's crawl job runs last — so a small storage finishing first could make the check falsely report "complete" while a larger storage still had thousands of files queued. This is a pre-existing issue that became directly relevant here because the new tag-cleanup job needed the same completion check, and duplicating the flawed logic would have meant two places to fix later instead of one. The rewritten check (IndexCompletionService) verifies live state instead — are there still crawl jobs of either kind scheduled, are there still files in the queue, are there still pending queue actions — rather than trusting a single stored marker that can go stale.

Why the config cache is cleared explicitly after saving: Nextcloud's app config layer is cached (APCu) and shared across PHP processes, including the long-running background worker process. Without clearing the cache after saving index_mode, the worker could keep operating on the old value for some time after the admin toggled the setting, even though the database already had the new value.

Changes

• lib/Service/FsEventService.php: hasAiKnowledgeTag() checks the "AI knowledge" tag on a node and walks up its ancestors for folder-level inheritance; returns true unconditionally when the app is in "index all" mode, so tag-checking has zero effect on existing installs that don't opt in
• lib/Controller/ConfigController.php: handleIndexModeChange() detects an actual mode change (not just any settings save) and schedules the appropriate background job; also clears the app config cache after saving
• lib/BackgroundJobs/UntaggedCleanupSchedulerJob.php + UntaggedCleanupCrawlJob.php: discover all mounts and remove untagged documents from the index in batches, mirroring the existing SchedulerJob/StorageCrawlJob shape rather than introducing a different batching pattern
• lib/Command/CleanupUntagged.php: occ context_chat:cleanup-untagged command to trigger the same cleanup manually, useful for admins scripting deployment or recovering from an interrupted run
• lib/Service/IndexCompletionService.php: extracted from QueueController for the reasons above, and used by both the normal indexing flow and the new tag-cleanup flow
• src/components/ViewAdmin.vue: admin UI for the mode toggle; the UI update is optimistic (switches immediately for responsiveness) and reverts to the previous value if the save request fails, so the displayed state never lies about what's actually saved

Testing

Tested end-to-end on a real multi-user Nextcloud instance: tagged specific folders and individual files, confirmed tagged documents get indexed and untagged ones get removed, confirmed folder-tag inheritance works correctly for files inside a tagged folder, confirmed switching modes back and forth triggers the correct job each time, confirmed the admin page's status accurately reflects "running" vs "finished" throughout both flows (including across multiple users' storages, which is what surfaced the completion-check issue described above), and confirmed a real Context Chat query only returns tagged sources when tag-only mode is active.

🤖 AI (if applicable)

• The content of this PR was partly or fully generated using AI