Preserve original source URL for synced Knowledge Base documents

[msi-martinsen]

Problem

When OIKB syncs content from external systems such as Confluence into an Open WebUI Knowledge Base, the original source URL is currently lost.

For example, the Confluence connector converts a page into a .txt document and uploads the extracted text, but a user receiving an answer based on that document has no easy way to open the original Confluence page.

This is useful both for verification and for continuing to read the source material - with pictures, diagrams, linked pdfs or other material.

Proposed feature

Allow connectors to provide an optional canonical source_url for each ManifestEntry.

For example:

ManifestEntry(
    filename="Installing the VPN.txt",
    path="IT/Guides",
    checksum="...",
    size=0,
    source_url="https://example.atlassian.net/wiki/pages/viewpage.action?pageId=123456"
)

The URL could then be preserved when the document is uploaded to Open WebUI.

Ideally it would be stored as file/document metadata so Open WebUI can expose it alongside citations and potentially provide an Open original source action.

Connector examples

Confluence

Use the canonical page URL, preferably based on the page ID or the URL returned by the Confluence API.

https://example.atlassian.net/wiki/pages/viewpage.action?pageId=123456

Jira

https://example.atlassian.net/browse/PROJECT-123

GitHub

Link to the original file, issue, wiki page, etc.

SharePoint / Notion / other connectors

Use the canonical browser URL returned by the source API where available.

Possible implementation

Add an optional property to ManifestEntry:

@dataclass(frozen=True, slots=True)
class ManifestEntry:
    filename: str
    path: str
    checksum: str
    size: int
    source_url: str | None = None

Then propagate it through the sync/upload process.

If Open WebUI currently does not support displaying arbitrary source metadata, an optional fallback could be to prepend something like:

Source: https://...

to the extracted document content.

A configuration option such as:

defaults:
  include-source-url: true

could control this behavior.

Why this is useful

A Knowledge Base often acts as a searchable/indexed representation of another authoritative system.

Users should be able to:

1. Get an answer from the indexed content.
2. See which original document/page the answer came from and the attached photos.
3. Open the authoritative source to verify or continue reading.

This would improve provenance and citations for all remote OIKB connectors, not just Confluence.