diff --git a/.claude/agents/docs-campaign.md b/.claude/agents/docs-campaign.md new file mode 100644 index 000000000..c06b5e255 --- /dev/null +++ b/.claude/agents/docs-campaign.md @@ -0,0 +1,163 @@ +--- +name: docs-campaign +description: > + Orchestrates multi-page documentation campaigns for a topic area. + Use when documenting a broad topic that spans multiple DIAL components + and requires systematic research, writing, and auditing. + Triggers on: "document configuration", "docs campaign for X", + "comprehensive docs for X", or any request to systematically + document a topic area across multiple pages. +tools: Read, Edit, Write, Bash, Grep, Glob, WebFetch +model: claude-opus-4-6 +memory: project +skills: + - docs-researcher + - docs-page-writer + - docs-auditor +--- + +# DIAL Documentation Campaign Orchestrator + +You orchestrate multi-page documentation campaigns for the DIAL docs site. A campaign takes a topic area (e.g., "configuration", "DIAL Apps", "deployment") and systematically researches, writes, and audits all pages needed to cover it comprehensively. + +You have three pre-loaded skills: **docs-researcher** (how to investigate source repos), **docs-page-writer** (how to write docs pages), and **docs-auditor** (how to assess quality). Follow their rules when executing each phase — they are your operating procedures. + +## Workspace context + +This is the **documentation and meta repository** for AI DIAL. It is NOT the application code. The platform code lives in 20+ sibling repositories under `https://github.com/epam/ai-dial-*`. + +This repo contains: +- `docs/` — Docusaurus 3 site, published to https://docs.dialx.ai/ +- `docs-planning/` — gap analysis, roadmap, structure, style guide, glossary +- `dial-docker-compose/` — minimal Docker Compose setups for quick start +- `dial-docker-compose-advanced/` — advanced Docker Compose configs +- `dial-cookbook/` — code examples and Jupyter notebooks +- `dial-samples/` — sample DIAL applications +- `dial-sdk` — git submodule (run `git submodule update --init` if needed) + +### Available tools + +Via Bash, you have access to: + +- **`gh` CLI** — GitHub CLI for accessing sibling repos. Preferred for reading files and listing repo trees: + ``` + gh api repos/epam//git/trees/main?recursive=1 -q '.tree[].path' + gh api repos/epam//contents/ -q .content | base64 -d + ``` +- **`agent-browser`** — headless web browser for fetching and parsing docs.dialx.ai pages, browsing GitHub repos, verifying external links, crawling site structure, and comparing rendered docs against source Markdown. Check capabilities: `agent-browser -h` +- **`tree`** — directory structure visualization +- **`git`** — log, blame, diff, file history, contributor listing + +### Docs site + +```bash +cd docs +npm install +npm run start # local dev at http://localhost:3000 +npm run build # production build (onBrokenLinks: 'throw') +``` + +Content: `docs/docs/` (Markdown). Sidebar: `docs/sidebars.js`. Config: `docs/docusaurus.config.js`. + +### Key conventions + +- **DIAL** in all-caps. Never "Dial" or "dial." +- Component names capitalized: **DIAL Core**, **DIAL Chat**, **DIAL Admin**, **DIAL SDK** +- The API is the **Unified API** (not "DIAL API") +- **Application**, **Adapter**, **Interceptor** are distinct concepts — don't conflate +- Deprecated: **Assistant** (archived), **Addon** (abandoned) — don't use in new content + +### Gotchas + +- `dial-sdk` is a git submodule, not a regular folder +- Broken links in docs often mean a sibling repo's README changed +- URL path `/video demos/` has a space (known issue) +- Some docs pages link to GitHub as the authoritative source for config — this is what we're fixing + +## Campaign workflow + +### Phase 1: Scope + +Before doing any research or writing, produce a campaign plan. + +1. Check your agent memory (`MEMORY.md`) for prior work on this topic. If resuming, pick up where you left off. +2. Read `docs-planning/gap-analysis.md` — identify gaps related to the topic. +3. Read `docs-planning/recommended-site-structure.md` — identify all target pages for this topic. +4. Read `CLAUDE.md` — identify which DIAL component repos are involved. +5. List existing docs pages that cover (or partially cover) this topic. +6. Produce a **campaign plan** as a numbered list: + - Pages to create (with target path from the structure document) + - Pages to rewrite (with current path and what needs changing) + - Pages to merge or delete + - Dependency order (which pages should be written first) + - Which components need research + +Save the campaign plan to your agent memory before proceeding. + +### Phase 2: Research + +For each component identified in the scope: + +1. Follow the **docs-researcher** skill procedures (pre-loaded in your context). +2. Use the research mode appropriate to the topic: + - Configuration → Mode 4: Config extraction + - Features → Mode 2: Feature investigation + - Architecture → Mode 5: Architecture mapping + - APIs → Mode 3: API mapping + - Single component → Mode 1: Component deep-dive +3. Save research briefs to `.claude-workspace/research/` using the researcher's naming convention and template. +4. Update `.claude-workspace/research/_index.md`. +5. After completing research for each component, update your agent memory with progress. + +### Phase 3: Write + +For each target page in the campaign plan: + +1. Follow the **docs-page-writer** skill procedures (pre-loaded in your context). +2. Read relevant research briefs as input context. +3. Create or edit pages in `docs/docs/` at the paths specified by the structure document. +4. Add required frontmatter (type, persona, component, last_verified, owner). +5. Update `docs/sidebars.js` for new pages or moved pages. +6. Add redirects in `docs/docusaurus.config.js` if URLs changed. +7. Run `cd docs && npm run build` after each page to catch broken links early. +8. Update agent memory with progress after each page. + +### Phase 4: Audit + +After all pages are written: + +1. Follow the **docs-auditor** skill procedures (pre-loaded in your context). +2. Run a section audit on all pages touched by this campaign. +3. Fix any issues found (terminology, frontmatter, structure, links). +4. Re-run `cd docs && npm run build` to confirm. +5. Save audit results to agent memory. + +### Phase 5: Report + +Return a concise summary to the main conversation: + +- **Created**: list of new pages with paths +- **Modified**: list of edited pages with what changed +- **Research briefs**: list of briefs produced +- **Audit results**: overall quality score, any remaining issues +- **Remaining work**: anything that couldn't be completed (blocked by missing info, needs human decision, etc.) + +## Progress tracking + +Use your agent memory directory to track campaign state across sessions. Maintain: + +- `MEMORY.md` — index of all campaigns and their current phase +- One file per campaign (e.g., `campaign-configuration.md`) tracking: + - Scope: components and pages identified + - Research: which components have been researched, brief file paths + - Writing: which pages have been created/edited + - Audit: results and remaining issues + +When resuming a campaign, read the campaign file first and continue from the last completed step. + +## Important constraints + +- Follow the glossary (`docs-planning/glossary.md`) for all terminology. +- Never link to GitHub READMEs as authoritative sources — bring content on-site. +- If a page doesn't have a clear home in the structure document, stop and report it rather than guessing. +- If research reveals the topic is larger than expected, report the revised scope before continuing. diff --git a/.claude/commands/setup.md b/.claude/commands/setup.md new file mode 100644 index 000000000..7e8c27031 --- /dev/null +++ b/.claude/commands/setup.md @@ -0,0 +1,4 @@ +Goal is to setup environment for the first time. + +Do the following: +1. Run `mkdir -p .claude-workspace/{research,drafts,audits}` to create workspace directory \ No newline at end of file diff --git a/.claude/settings.json b/.claude/settings.json index 0d5f87936..01963d2a5 100644 --- a/.claude/settings.json +++ b/.claude/settings.json @@ -1,8 +1,25 @@ { + "$schema": "https://json.schemastore.org/claude-code-settings.json", "permissions": { "allow": [ "PowerShell(gh issue list --repo epam/ai-dial-admin-frontend --limit 20)", - "PowerShell($issues = Invoke-RestMethod -Uri \"https://api.github.com/repos/epam/ai-dial-admin-frontend/issues?state=open&per_page=20\" -Headers @{ \"User-Agent\" = \"claude-code\" }; $issues | Select-Object number, title, state, @{N='labels';E={\\($_.labels.name\\) -join ','}} | Format-Table -AutoSize -Wrap)" + "PowerShell($issues = Invoke-RestMethod -Uri \"https://api.github.com/repos/epam/ai-dial-admin-frontend/issues?state=open&per_page=20\" -Headers @{ \"User-Agent\" = \"claude-code\" }; $issues | Select-Object number, title, state, @{N='labels';E={\\($_.labels.name\\) -join ','}} | Format-Table -AutoSize -Wrap)", + "Bash(npm run lint)", + "Bash(npm run test *)", + "Read(~/.zshrc)", + "Bash(tree *)", + "Bash(agent-browser *)", + "Bash(ls *)", + "Bash(grep *)", + "Bash(find *)", + "Bash(gh *)", + "Bash(git clone *)" + ], + "deny": [ + "Read(./.env)", + "Read(./.env.*)", + "Read(./node_modules/**)", + "Read(./.docusaurus/**)" ] } -} +} \ No newline at end of file diff --git a/.claude/settings.local.json b/.claude/settings.local.json index 8a57bc3b4..4e9469337 100644 --- a/.claude/settings.local.json +++ b/.claude/settings.local.json @@ -1,7 +1,12 @@ { "permissions": { "allow": [ - "Bash(gh issue *)" + "Bash(gh issue *)", + "Read(//Users/Attila_Torda-Pilisi/.claude/**)", + "Bash(find / -type d -iname \"docs-page-writer\")", + "Bash(find / -type d -iname \"docs-auditor\")", + "Bash(npm run *)", + "Bash(echo \"EXIT=$?\")" ] } } diff --git a/.claude/skills/docs-auditor/SKILL.md b/.claude/skills/docs-auditor/SKILL.md new file mode 100644 index 000000000..6023450ea --- /dev/null +++ b/.claude/skills/docs-auditor/SKILL.md @@ -0,0 +1,323 @@ +--- +name: docs-auditor +description: Audit DIAL documentation pages for quality, compliance, and structural placement. This skill is read-only — it produces assessment reports but does not modify files. Use this skill when the user asks to audit, assess, evaluate, grade, or inventory documentation pages — whether a single page, a section, or the full site. Also triggers on "content inventory," "docs quality check," "check this page," "what's wrong with this doc," "grade this page," "audit the docs," or any request to evaluate documentation against the style guide, structure, or Diátaxis standards. Use for Phase 0 inventory work, ongoing quality checks, and PR reviews of docs content. To fix findings, use the docs-page-writer skill. +--- + +# DIAL Documentation Auditor + +This skill audits documentation pages against the DIAL Style Guide, Recommended Site Structure, and Gap Analysis. It produces structured assessments that feed directly into the content inventory spreadsheet. + +## Tools + +Use these tools for specific audit steps: + +| Step | Tool | Purpose | +|---|---|---| +| Read page content | `Read` | Load the `.md` file for analysis | +| Terminology scan | `Bash` with `grep` | Search for forbidden phrases and naming violations — faster and more reliable than visual scanning | +| Internal link validation | `Bash`: `cd docs && npm run build` | The Docusaurus config has `onBrokenLinks: 'throw'` — the build catches all broken internal links at once | +| Live URL verification | `WebFetch` or `agent-browser` | Check published URLs and external links when needed | +| Sidebar depth check | `Read` on `docs/sidebars.js` | Verify sidebar placement and nesting depth | +| Batch file listing | `Bash` with `find` | List all `.md` files in a section: `find docs/docs/
-name '*.md'` | + +## Planning documents + +Both skills read from `docs-planning/`. Load selectively based on audit mode: + +| Mode | Required documents | +|---|---| +| Single page audit | `glossary.md`, `style-guide.md`, `recommended-site-structure.md` | +| Section audit | All of the above + `gap-analysis.md` | +| Full inventory | All five documents including `improvement-roadmap.md` | + +## Audit modes + +### Single page audit +The user provides a page path (e.g., `docs/docs/platform/core/about-core.md`) or a URL (e.g., `https://docs.dialx.ai/platform/core/about-core`). Produce a full assessment. + +### Section audit +The user names a section (e.g., "audit the Platform > Core section"). Audit every page in that section, produce per-page assessments, then a section summary. + +### Full inventory +The user asks for a complete content inventory. Walk every `.md` file in `docs/docs/`, produce per-page assessments, and output a summary table suitable for a spreadsheet. + +### Prioritization for batch audits + +When auditing a section or the full site, start with pages on the Phase 1 critical path in `docs-planning/improvement-roadmap.md`. These are the highest-traffic, highest-value pages. Audit remaining pages in sidebar order. + +--- + +## Audit procedure + +For each page, run these checks in order. Read the page content first, then evaluate. + +### 1. Identity + +| Field | What to capture | +|---|---| +| **File path** | e.g., `docs/docs/platform/core/about-core.md` | +| **Published URL** | e.g., `https://docs.dialx.ai/platform/core/about-core` | +| **Menu path** | e.g., `Platform > Core > About` | +| **Page title** | The H1 or frontmatter title | +| **URL ↔ menu match** | Do the URL path segments match the sidebar placement? Flag mismatches. | + +### 2. Diátaxis classification + +Read the page and classify it as exactly one of: **tutorial**, **how-to**, **reference**, **explanation**, **user-guide**, **landing**, or **other**. + +Then assess: +- Does the page's **actual content** match its **declared or implied type**? (e.g., a page under "Tutorials" that is actually a configuration reference) +- Does the page try to be **two types at once**? (e.g., explaining concepts mid-tutorial, embedding parameter tables in an explanation) If so, recommend splitting. +- Is the content type appropriate for its **sidebar location**? + +Refer to `references/diataxis-signals.md` for classification heuristics. + +### 3. Frontmatter check + +Check for required fields. Current pages will almost certainly lack these — the point is to flag what needs adding during migration. + +| Field | Required | Notes | +|---|---|---| +| `title` | Yes | Should be imperative for how-tos, noun for reference | +| `type` | Yes | tutorial / how-to / reference / explanation / user-guide | +| `persona` | Yes | end-user / app-dev / devops / admin / evaluator / architect | +| `component` | Yes | core / chat / admin / sdk / adapters / helm / apps | +| `last_verified` | Yes | ISO date | +| `owner` | Yes | GitHub team handle | + +### 4. Content quality + +Score 1–5 using the rubric in `references/quality-rubric.md`. Evaluate across five dimensions: + +- **Completeness** — does the page cover its topic fully, or does it redirect to GitHub / leave gaps? +- **Accuracy** — are code examples runnable? Are version pins current? Are claims verifiable? +- **Clarity** — can the target persona understand this without external help? +- **Self-sufficiency** — can the reader accomplish the task without leaving the page for essential information? +- **Freshness** — are dependencies, screenshots, UI references, and API examples current? + +Overall score = average of the five dimensions, rounded. + +### 5. Terminology compliance + +Scan for violations against two sources: +- **Canonical terms and definitions**: read `docs-planning/glossary.md` — this is the single source of truth for product names, component names, core concepts, deprecated terms, and the canonicalized name for contested terms (e.g., "Agent Builder" vs "application runner"). +- **Audit mechanics** (forbidden phrases, capitalization rules, reporting format): read `references/terminology-checklist.md`. + +Check for: +- **Incorrect product names**: any variant not matching `docs-planning/glossary.md` (e.g., "Dial" instead of "DIAL") +- **Vague component references**: "the backend," "the server," "the frontend" instead of the glossary's canonical component names +- **Deprecated terms**: terms marked as deprecated in the glossary used without a deprecation notice on the page +- **Naming chaos**: terms with multiple variants in the glossary — flag which variant the page uses +- **Forbidden phrases**: per `references/terminology-checklist.md` +- **Inconsistent capitalization**: per `references/terminology-checklist.md` + +Report: number of violations, list of specific instances with line numbers. + +### 6. Structural placement + +Consult `docs-planning/recommended-site-structure.md` and assess: + +- **Current location**: where this page lives now +- **Target location**: where the Structure document says it should live +- **Action needed**: keep / move / merge / split / rewrite / delete / redirect +- **Merge target**: if this page should be merged with another, name the other page +- **Is this a duplicate?**: check against the known duplicate-title pairs listed in `docs-planning/gap-analysis.md` §1.1 + +### 7. Navigation quality + +- **Sidebar depth**: count the levels from root to this page. Flag if > 4. +- **"What's next" links**: does the page end with follow-up links? (Almost certainly not — flag as missing.) +- **Dead-end check**: does the page link forward to any other docs page, or does it just... stop? +- **Misleading title**: does the sidebar label accurately describe the content? (See Gap Analysis §1.5 for known offenders.) +- **GitHub redirect**: does the page link to GitHub as the authoritative source for content that should be on-site? + +### 8. Link health + +- **Internal links**: run `cd docs && npm run build` to detect all broken internal links at once (the config has `onBrokenLinks: 'throw'`). For single-page audits, spot-check links manually. +- **Internal link format**: internal doc-to-doc links must be **relative paths ending in `.md`** (e.g., `../developer-tools/sdk-reference/0.index.md`). Flag absolute root paths (`](/...`) and extension-less internal links (`](../foo/bar)` with no `.md`). +- **External links**: note but don't validate (too slow for batch audits). Use `WebFetch` selectively for suspicious URLs. +- **GitHub-as-authority links**: flag any link to a GitHub README that substitutes for on-site documentation (e.g., "Refer to the AI DIAL Core repository for full configuration details") + +### 9. Code and example freshness + +- **Pinned versions**: are dependencies pinned? How old are the pins? +- **Runnable examples**: could a reader copy-paste and run the code, or are there missing imports, undefined variables, placeholder-only snippets? +- **Language tags**: do code blocks specify a language (` ```python `, ` ```bash `)? +- **Shell prompts**: are there `$` prompts in copyable commands? (Should not be.) + +### 9b. Markdown formatting conventions + +Scan for the site's Markdown-output conventions. Use `grep` for the first two — they are exact and fast. + +- **Admonitions**: flag any `:::` admonition syntax (`:::note`, `:::tip`, `:::warning`, `:::info`). Highlights must use a **bold label line + blockquote** instead (`**Warning**` then `> ...`). Four labels only: Note, Tip, Warning, Deprecated. +- **Host in examples**: flag `localhost` in example URLs and code. Must be `0.0.0.0` (e.g., `http://0.0.0.0:8080`). +- **Bold lead-in labels**: flag a bold label that introduces a multi-sentence block inline on the same line (e.g., `**Verify:** The adapter translates...`). The content must follow on its own paragraph after a blank line. +- **Parallel statements as bullets**: flag a set of consecutive, parallel standalone statements rendered as bare back-to-back paragraphs. They should be a bullet list. + +See `references/terminology-checklist.md` §7 for the grep patterns and reporting format. + +### 10. Video assessment (if applicable) + +If the page references or embeds a video: +- **Is the video supplementary or primary?** If primary (the page says "watch the video to learn X" without written equivalent), flag as content gap. +- **Is there a paired sample repo, sample data, and config?** If not, flag. +- **Is this a tutorial video or a demo?** Tutorial videos should be embedded on tutorial pages. Demos should be in the Demos section. + +### 11. Tutorial structure (tutorials only) + +Skip this check for non-tutorial pages. + +- **Project structure diagram**: does the tutorial include a directory/file tree (fenced code block) showing the complete project layout? Flag if missing. +- **Exact file paths**: are all file references qualified with their path relative to the project root (e.g., `my-app/src/index.ts`, not just `index.ts`)? Scan for bare filenames in step headings and prose. Flag each bare filename without a directory prefix. +- **Tree-to-content consistency**: do all files shown in the project structure tree actually appear in the tutorial steps? Flag files in the tree that are never created, and files created in steps that are missing from the tree. + +--- + +## Output format + +### Single page assessment + +```markdown +## Audit: [Page Title] + +| Field | Value | +|---|---| +| File path | `docs/docs/platform/core/about-core.md` | +| Published URL | https://docs.dialx.ai/platform/core/about-core | +| Menu path | Platform > Core > About | +| URL ↔ menu match | ✅ Match / ⚠️ Mismatch: [details] | + +### Classification +- **Declared type**: (from frontmatter or sidebar context) +- **Actual type**: (your assessment) +- **Mismatch?**: Yes/No — [explanation] +- **Should split?**: Yes/No — [what to split into] + +### Frontmatter +- [ ] title +- [ ] type +- [ ] persona +- [ ] component +- [ ] last_verified +- [ ] owner +Missing: [list] + +### Quality score: X/5 +| Dimension | Score | Notes | +|---|---|---| +| Completeness | X/5 | | +| Accuracy | X/5 | | +| Clarity | X/5 | | +| Self-sufficiency | X/5 | | +| Freshness | X/5 | | + +### Terminology: X violations +- Line N: "the backend" → should be "DIAL Core" +- Line N: "simply" → delete +- (list all) + +### Structural placement +- **Current location**: Platform > Core > About +- **Target location**: Understand DIAL > Architecture highlights (absorbed) +- **Action**: merge +- **Merge target**: Architecture highlights page +- **Known duplicate?**: Yes — "About" ×2 (Gap Analysis §1.1) + +### Navigation +- **Sidebar depth**: 3 ✅ +- **"What's next" links**: ❌ Missing +- **Dead end?**: Yes — no forward links +- **Misleading title?**: Yes — "About" is generic +- **GitHub redirects**: 0 + +### Links +- Internal: X links, Y broken +- Non-relative / missing-`.md` internal links: X instances — [list with line numbers] +- GitHub-as-authority: X instances + +### Formatting conventions +- `:::` admonitions: X instances — [list with line numbers] +- `localhost` in examples: X instances +- Bold lead-in labels without line break: X instances +- Parallel statements not bulleted: [list or "None"] + +### Code freshness +- Pinned versions: [list with ages] +- Runnable: Yes/No +- Language tags: X missing + +### Video +- (assessment or "No video referenced") + +### Tutorial structure (tutorials only) +- **Project structure diagram**: ✅ Present / ❌ Missing +- **Bare filenames (no path)**: X instances — [list] +- **Tree-content mismatches**: [list or "None"] + +### Recommended action +**[KEEP / MOVE / MERGE / SPLIT / REWRITE / DELETE / REDIRECT]** +[One paragraph explaining the recommendation and what specifically needs to happen.] +``` + +### Batch summary (for section or full inventory) + +After auditing multiple pages, produce a summary table: + +```markdown +## Audit summary: [Section name or "Full site"] + +| Page | Quality | Type match | Terminology | Action | Target location | +|---|---|---|---|---|---| +| About Core | 2/5 | ⚠️ Mixed | 4 violations | Merge | §2 Architecture highlights | +| Access Control | 3/5 | ⚠️ Ref as Expl | 2 violations | Merge | §2 Auth & access control | +| ... | | | | | | + +### Key findings +- X pages total audited +- Average quality score: X/5 +- Pages with type mismatch: X +- Pages with missing frontmatter: X (expected: all) +- Pages with "what's next" links: X +- Pages flagged as duplicates: X +- Pages with GitHub-as-authority links: X +- Pages with `:::` admonitions: X +- Pages with non-relative or missing-`.md` internal links: X +- Pages using `localhost` in examples: X +- Pages with stale dependencies: X +- Tutorials missing project structure diagram: X +- Recommended actions: X keep, X move, X merge, X split, X rewrite, X delete +``` + +--- + +## Integration with planning documents + +The auditor reads these files when evaluating: + +| Document | Path | Used for | +|---|---|---| +| Glossary | `docs-planning/glossary.md` | Canonical terms, product names, component names, deprecated terms, contested names | +| Gap Analysis | `docs-planning/gap-analysis.md` | Known issues, duplicate pairs, misleading labels | +| Recommended Structure | `docs-planning/recommended-site-structure.md` | Target placement, merge targets, action recommendations | +| Style Guide | `docs-planning/style-guide.md` | Voice rules, formatting standards, page structure requirements | +| Improvement Roadmap | `docs-planning/improvement-roadmap.md` | Priority context (which fixes matter most) | + +When referencing a known gap, cite it: "This page is affected by Gap Analysis §1.1 (duplicate title: 'About' ×2)." + +## Abbreviated format for batch audits + +For section or full-inventory audits, use the full per-page template only for pages scoring **2/5 or below**. For all other pages, use this one-line format in the summary table: + +``` +| Page title | File path | Quality | Type | Type match | Term. violations | Action | Notes | +``` + +Write batch audit output to a file (e.g., `docs-planning/audit-results/
.md`) rather than inline, to avoid overwhelming the conversation. + +## Frontmatter validation + +The custom frontmatter fields (`type`, `persona`, `component`, `last_verified`, `owner`) are not validated by the Docusaurus build. A missing `type` field won't break the build but will fail this audit. Enforcement comes from skill checks and code review. + +## Remediation + +To fix findings from an audit, invoke the `docs-page-writer` skill with the audit results as context. The typical workflow is: audit → identify issues → write fixes → re-audit to verify. \ No newline at end of file diff --git a/.claude/skills/docs-auditor/references/diataxis-signals.md b/.claude/skills/docs-auditor/references/diataxis-signals.md new file mode 100644 index 000000000..c87bda885 --- /dev/null +++ b/.claude/skills/docs-auditor/references/diataxis-signals.md @@ -0,0 +1,96 @@ +# Diátaxis Classification Signals + +Use these heuristics to classify a page. When signals conflict, the **dominant** pattern wins. + +## Tutorial signals + +The page IS a tutorial if: +- It has numbered steps that build on each other toward a goal +- It starts from a clean state and produces a working result +- It says "you will learn" or "by the end of this tutorial" +- It includes verification after steps ("you should see…") +- It has a "What you learned" section +- The reader creates something that didn't exist before + +The page is NOT a tutorial if: +- Steps are "download and run" with no explanation (that's a quickstart) +- It assumes the reader already knows the domain (that's a how-to) +- Steps are optional or unordered (that's a how-to) +- It describes what something IS rather than guiding the reader to DO (that's an explanation) + +## How-to signals + +The page IS a how-to if: +- Title starts with a verb: "Configure…," "Enable…," "Add…," "Rotate…" +- It assumes the reader already knows what they want to achieve +- It provides steps without explaining why each step matters +- It offers alternatives ("you can also…") +- It addresses a specific, bounded task + +The page is NOT a how-to if: +- It teaches concepts along the way (that's a tutorial) +- It exhaustively lists every parameter (that's reference) +- It discusses design decisions or trade-offs (that's explanation) + +## Reference signals + +The page IS a reference if: +- It's organized as a catalog: parameters, endpoints, config keys, error codes +- It uses tables with type/default/description columns +- Every entry is documented, even obvious ones +- Ordering is systematic (alphabetical, structural), not priority-based +- Voice is neutral and declarative — no "you should" +- It includes complete type information + +The page is NOT a reference if: +- It explains WHY a setting exists (that's explanation) +- It walks through configuration step-by-step (that's how-to) +- It selectively highlights "important" settings (that's how-to or explanation) + +## Explanation signals + +The page IS an explanation if: +- It discusses WHY something is the way it is +- It covers trade-offs, alternatives, and design decisions +- It uses phrases like "this approach was chosen because…" +- It includes architecture diagrams showing relationships +- It compares DIAL to other tools or approaches +- It has no runnable steps + +The page is NOT an explanation if: +- It includes setup instructions (those belong in how-to) +- It catalogs parameters (that's reference) +- It guides the reader through building something (that's tutorial) + +## User guide signals + +The page IS a user guide if: +- It documents a UI: "click this button," "navigate to this menu" +- It's organized by product feature, not by learning goal +- The reader is an end user, not a developer or operator +- Screenshots dominate + +Use `type: user-guide` in frontmatter. User guides are distinct from how-tos (task-oriented for technical users) and tutorials (learning-oriented from zero). The Chat User Guide section is the primary example in DIAL docs. + +## Common misclassification patterns in DIAL docs + +These are the most frequent mistakes found during the Gap Analysis: + +| Page looks like | But is actually | Why it's misclassified | +|---|---|---| +| "Tutorial" (in the Tutorials section) | Configuration reference | It documents JSON schema fields, not guided steps | +| "Tutorial" (in the Tutorials section) | User guide | It's a comprehensive product manual, not a learning experience | +| "Tutorial" (in the Tutorials section) | How-to | It assumes familiarity and provides task-oriented steps without teaching | +| "Platform" explanation | Detailed reference | It lists authorization rules, config files, and API endpoints | +| "Platform" explanation | Lightweight landing page | It's 3 paragraphs that link to the Tutorials section for real content | +| Quickstart | Tutorial | It has numbered steps but no explanation, no verification, no learning outcomes | + +## Mixed pages — how to spot them + +A page is mixed (needs splitting) if it: +- Starts with "What is X" (explanation) then switches to "How to configure X" (how-to) +- Has a concept section followed by a parameter table (explanation + reference) +- Teaches a concept via guided steps but also lists every configuration option (tutorial + reference) +- Contains a user guide section alongside developer API documentation (user guide + reference) + +**Recommendation when mixed:** note both types and recommend splitting in the audit. \ No newline at end of file diff --git a/.claude/skills/docs-auditor/references/quality-rubric.md b/.claude/skills/docs-auditor/references/quality-rubric.md new file mode 100644 index 000000000..a8b82e0c1 --- /dev/null +++ b/.claude/skills/docs-auditor/references/quality-rubric.md @@ -0,0 +1,119 @@ +# Quality Scoring Rubric + +Score each dimension 1–5. Overall score = average, rounded to nearest integer. + +--- + +## Completeness + +*Does the page cover its topic fully, or does it redirect, leave gaps, or trail off?* + +| Score | Criteria | +|---|---| +| **5** | Comprehensive. Every aspect of the topic is documented on this page. No "refer to GitHub for details." Reader never needs to leave. | +| **4** | Mostly complete. Minor subtopics may link elsewhere, but all core information is on-page. | +| **3** | Partial. Covers the main points but has visible gaps — some settings undocumented, some steps missing, some scenarios not addressed. | +| **2** | Stub-like. Introduces the topic but redirects to GitHub, another page, or external docs for most of the actual content. | +| **1** | Placeholder or link farm. The page is essentially "this exists, see [external link] for details." | + +### DIAL-specific examples +- **Score 2**: Configuration Guide page that says "Refer to the AI DIAL Core repository" for 7 components — all real content is on GitHub +- **Score 3**: Access Control page that covers concepts but doesn't document all object types or configuration options +- **Score 5**: Chat User Guide that comprehensively covers every UI feature + +### Tutorial-specific completeness criteria + +For tutorials, Completeness also requires: +- A project structure tree diagram showing the final directory layout +- Every file referenced with its exact path from the project root +- A verification checkpoint after each substantive step + +A tutorial missing the project structure diagram cannot score above **3** on Completeness. + +--- + +## Accuracy + +*Are code examples runnable? Are version pins current? Are claims verifiable?* + +| Score | Criteria | +|---|---| +| **5** | All code examples run on a clean machine. Version pins are current (within 6 months). All claims are verifiable. | +| **4** | Examples are likely runnable with minor adjustments. Versions are within 12 months. | +| **3** | Examples are plausible but untested or missing imports/setup. Some versions are outdated. | +| **2** | Examples have visible errors (wrong API, missing params, broken imports). Versions are >12 months old. | +| **1** | Examples are clearly broken or use deprecated APIs. Version pins are >2 years old. Factual claims are outdated. | + +### DIAL-specific examples +- **Score 1**: Cookbook examples pinning `openai-python-sdk` versions from 2+ years ago +- **Score 3**: Quick Start that works but pins no versions (will break unpredictably) +- **Score 4**: Deployment guide with current Helm chart versions and working commands + +--- + +## Clarity + +*Can the target persona understand this without external help?* + +| Score | Criteria | +|---|---| +| **5** | Crystal clear for the target persona. Appropriate level of assumed knowledge. Jargon defined on first use or linked to glossary. Headings are scannable. | +| **4** | Clear with minor rough spots. Occasional undefined term or ambiguous sentence. | +| **3** | Understandable with effort. Some sections require re-reading. Assumed knowledge occasionally too high or too low for the target persona. | +| **2** | Confusing. Key concepts unexplained. Structure doesn't help the reader find what they need. Mixed audience signals (writing for developers and end users simultaneously). | +| **1** | Impenetrable. Wall of text, no structure, undefined acronyms, no clear audience. | + +### DIAL-specific examples +- **Score 2**: "Apps Development" section whose first page is DIAL-to-DIAL Adapter — a reader looking to build apps is immediately confused +- **Score 3**: Quick App Configuration Guide — clear JSON schema docs, but the reader can't tell if this is the right place to learn about Quick Apps +- **Score 4**: Chat User Guide — well-structured product manual with clear sections + +--- + +## Self-sufficiency + +*Can the reader accomplish the goal without leaving the page for essential information?* + +| Score | Criteria | +|---|---| +| **5** | Completely self-contained. Everything needed to accomplish the task or understand the concept is on this page. Links are supplementary, not essential. | +| **4** | Nearly self-contained. One or two essential pieces require following a link, but the link is to another docs page (not GitHub). | +| **3** | Partially self-contained. Reader must visit 2–3 other pages or repos to get the full picture. | +| **2** | Heavily dependent on external content. The page makes sense only if you've already read GitHub READMEs or other pages. | +| **1** | Not self-contained at all. The page is essentially a routing hub that links elsewhere for all real content. | + +### DIAL-specific examples +- **Score 1**: DevOps sidebar entry that links directly to GitHub with no on-site content +- **Score 2**: Configuration Guide that sends readers to 7 GitHub repos for actual config documentation +- **Score 4**: Admin Panel User Guide that covers the full admin workflow on-site + +--- + +## Freshness + +*Are dependencies, screenshots, UI references, and API examples current?* + +| Score | Criteria | +|---|---| +| **5** | Everything is current. Dependencies pinned within last 6 months. Screenshots match current UI. API examples use current endpoints and parameters. | +| **4** | Mostly current. Minor staleness (dependency 6–12 months old, screenshot slightly outdated but recognizable). | +| **3** | Noticeably stale. Dependencies 12–18 months old. Some UI references don't match. Still mostly functional. | +| **2** | Significantly stale. Dependencies >18 months old. Screenshots show old UI. Some examples may not work. | +| **1** | Abandonware signals. Dependencies >2 years old. UI completely redesigned since screenshots were taken. Examples are broken. | + +### DIAL-specific examples +- **Score 1**: Cookbook page with 2+ year old `openai-python-sdk` dependency +- **Score 3**: Deployment guide referencing a Helm chart version from 12 months ago +- **Score 5**: Page verified within the last month with current dependency pins + +--- + +## Interpreting the overall score + +| Overall | Interpretation | Typical action | +|---|---|---| +| **5** | Excellent. Meets all standards. | Keep, minor updates only | +| **4** | Good. Needs polish but fundamentally sound. | Keep, update during migration | +| **3** | Acceptable. Usable but has clear gaps. | Rewrite specific sections | +| **2** | Poor. Misleads or frustrates readers. | Major rewrite or merge into a better page | +| **1** | Harmful. Actively wastes reader time or sends them to dead ends. | Delete, redirect, or rebuild from scratch | \ No newline at end of file diff --git a/.claude/skills/docs-auditor/references/terminology-checklist.md b/.claude/skills/docs-auditor/references/terminology-checklist.md new file mode 100644 index 000000000..787eee778 --- /dev/null +++ b/.claude/skills/docs-auditor/references/terminology-checklist.md @@ -0,0 +1,98 @@ +# Terminology Checklist + +This file covers the **audit mechanics** for terminology compliance: what to scan for, how to report violations, severity guide. For the **canonical terms themselves** (product names, component names, core concepts, deprecated terms, contested names), always read `docs-planning/glossary.md` — it is the single source of truth. + +--- + +## What to scan for + +### 1. Product and component name violations + +Read `docs-planning/glossary.md` for the canonical forms. Scan the page for any variant that doesn't match. Common violations: + +- Lowercase or mixed-case product names ("dial," "Dial," "DiAL" instead of "DIAL") +- Informal substitutions ("the backend," "the server," "the frontend" instead of the glossary's canonical component names) +- Vague references ("DIAL API" when "Unified API" is the canonical term) + +### 2. Core concept conflation + +Read the concept definitions in `docs-planning/glossary.md`. Flag when a page uses one concept where it means another (e.g., "adapter" when the page is describing an interceptor, or "application" when it means a model adapter). + +### 3. Deprecated terms without marking + +Read the deprecated terms section in `docs-planning/glossary.md`. If a page uses a deprecated term (e.g., "Assistant," "Addon"), check: is there a deprecation notice on the page? If not, flag as a violation. + +### 4. Naming chaos (contested terms) + +Read `docs-planning/glossary.md` for any term that has multiple variants listed (e.g., "Agent Builder" vs "application runner" vs "Application builders" vs "Builders"). Flag which variant the page uses. Until the glossary canonicalizes one, every occurrence should be noted for later alignment. + +### 5. Forbidden phrases + +These are independent of the glossary. Delete on sight: + +| Phrase | Problem | Fix | +|---|---|---| +| "simply" | Gaslights stuck readers | Delete | +| "just" (as minimizer) | Same | Delete ("just run" → "run") | +| "easily" | Same | Delete | +| "obviously" | Same | Delete | +| "please note that" | Throat-clearing | Start with the thing | +| "it should be noted that" | Same | Start with the thing | +| "our product" | Marketing voice | "DIAL" | +| "AI-powered" | Everything here is AI | Delete | +| "cutting-edge" | Marketing | Delete | +| "best-in-class" | Marketing | Delete | +| "click here" | Inaccessible link text | Describe the destination | +| "as shown in the video" | Text must stand alone | Rewrite to be self-contained | + +### 6. Capitalization violations + +| Context | Rule | Example | +|---|---|---| +| Product names | Always capitalized as defined in glossary | DIAL Core, DIAL Chat | +| Generic concepts mid-sentence | Lowercase | "an application," "an adapter" | +| Start of sentence | Capitalize naturally | "Applications are extensions…" | +| HTTP verbs in reference docs | Uppercase | `GET`, `POST`, `DELETE` | +| Environment variables | Uppercase, underscored, code font | `DIAL_URL`, `DIAL_SERVER_PORT` | +| Config keys in reference docs | Code font | `server.port`, `auth.jwt.jwks-url` | +| Headings | Sentence case | "Configure the adapter" not "Configure The Adapter" | + +### 7. Markdown formatting conventions + +Scan for the site's Markdown-output conventions. The first three are exact grep checks; run them from the repo root against the page (or section): + +```bash +grep -nE ':::' # admonitions — must be bold label + blockquote +grep -n 'localhost' # must be 0.0.0.0 in example hosts/URLs +grep -nE '\]\(/' # absolute internal links — must be relative + .md +``` + +| Convention | Violation | Fix | +|---|---|---| +| Highlights | `:::note` / `:::tip` / `:::warning` / `:::info` admonition | Bold label line + blockquote (`**Warning**` then `> ...`) — labels: Note, Tip, Warning, Deprecated | +| Example host | `localhost` in a URL or code example | `0.0.0.0` (e.g., `http://0.0.0.0:8080`) | +| Internal links | Absolute root path (`](/section/page)`) or missing `.md` extension | Relative path ending in `.md` (`](../section/page.md)`) | +| Bold lead-in | Bold label introducing a block inline (`**Verify:** The adapter...`) | Blank line after the label; content as its own paragraph | +| Lists | Consecutive parallel standalone statements as bare paragraphs | Bullet list (`- ...`) | + +--- + +## How to report violations + +For each violation found, report: + +``` +- Line N: "the backend" → "DIAL Core" (product name — see glossary) +- Line N: "simply" → delete (forbidden phrase) +- Line N: "Builders" → flag for glossary canonicalization (naming chaos) +- Line N: "Addon" used without deprecation notice (deprecated term — see glossary) +``` + +Count total violations per page. Severity guide: + +| Violations | Severity | +|---|---| +| 0 | ✅ Clean | +| 1–3 | Minor — fix during migration | +| 4–8 | Moderate — dedicated cleanup PR | +| 9+ | Major — full terminology pass needed | \ No newline at end of file diff --git a/.claude/skills/docs-page-writer/SKILL.md b/.claude/skills/docs-page-writer/SKILL.md new file mode 100644 index 000000000..3750add91 --- /dev/null +++ b/.claude/skills/docs-page-writer/SKILL.md @@ -0,0 +1,240 @@ +--- +name: docs-page-writer +description: Use this skill whenever creating, editing, or reviewing Markdown documentation pages in the `docs/docs/` directory of the ai-dial repo. Triggers on any task involving writing docs content, editing existing docs pages, creating new documentation, reviewing docs for style compliance, or migrating content between sections. Also use when the user mentions "docs," "documentation," "write a page," "tutorial," "how-to," "reference page," "explanation page," or any Diátaxis content type. Use it even for small edits — the frontmatter, terminology, and structural rules apply to every change. +--- + +# DIAL Documentation Page Writer + +This skill ensures every documentation page in `docs/docs/` follows the DIAL Documentation Style Guide and fits correctly into the Recommended Site Structure. + +## Editing existing pages vs. writing new ones + +Most docs improvement work is editing existing pages, not creating new ones. When editing: + +1. Read the current page with the `Read` tool +2. Compare against the self-check checklist (step 8) +3. Fix what's broken — don't rewrite what's fine +4. Preserve any content that's accurate and well-structured + +Only follow the full step-by-step process below for new pages or major rewrites. + +## Planning documents + +Load selectively based on the task: + +| Task | Required documents | +|---|---| +| Small edit | `docs-planning/glossary.md` (terminology) | +| New page | `glossary.md` + `style-guide.md` + `recommended-site-structure.md` | +| Migration / restructure | All of the above + `gap-analysis.md` + `improvement-roadmap.md` | + +## Step 1: Determine the content type + +Every page is **exactly one** Diátaxis type. Determine which before writing anything: + +| Type | Reader's question | Voice | Structure | +|---|---|---|---| +| **Tutorial** | "Teach me to do this" | Second person, encouraging, low assumption | Prerequisites → Goal → Numbered steps → Verification → What you learned → Next steps | +| **How-to** | "Help me accomplish this task" | Imperative, terse, assumes familiarity | Goal → Prerequisites → Steps → Result → Related tasks | +| **Reference** | "Give me the details" | Third person, declarative, no opinions | Identifier → Description → Parameters → Returns → Errors → Example | +| **Explanation** | "Help me understand why" | Essayistic but precise, can use "we" for design intent | Thesis → Background → Discussion → Implications → Further reading | +| **User guide** | "Show me how to use the UI" | Second person, feature-organized, screenshot-heavy | Feature overview → UI walkthrough → Tips → Next steps | + +**If a page feels like two types, split it into two pages.** + +User guides document a UI for end users. They are distinct from how-tos (which are task-oriented for technical users) and tutorials (which teach from zero). The Chat User Guide section uses this type. + +## Step 2: Add required frontmatter + +Every page must start with: + +```yaml +--- +title: "Configure rate limits" # imperative for how-tos, noun for reference +type: how-to # tutorial | how-to | reference | explanation | user-guide +persona: devops # end-user | app-dev | devops | admin | evaluator | architect +component: core # core | chat | admin | sdk | adapters | helm | apps +last_verified: 2026-04-27 +owner: "@team-handle" +--- +``` + +## Step 3: Write the opening + +The **first 60 words** must answer three questions: +1. What will the reader accomplish or learn? +2. Who is this for? +3. What is the prerequisite knowledge? + +If a reader can't tell from the top of the page whether they're in the right place, the page has failed. + +## Step 4: Follow type-specific structure + +Read the appropriate template in `references/` before writing: +- Tutorial → read `references/template-tutorial.md` +- How-to → read `references/template-howto.md` +- Reference → read `references/template-reference.md` +- Explanation → read `references/template-explanation.md` + +### Tutorial-specific requirements + +When writing tutorials, always include: + +1. **Project structure diagram.** After the setup step, add a "Project structure" heading with a fenced code block showing the complete directory tree the reader will build. Use `text` as the language tag. Example: + + ```text + dial-rag-app/ + ├── app.py + ├── requirements.txt + └── config/ + └── core-config.json + ``` + +2. **Exact file paths.** Every file the reader creates or edits must be referenced by its path relative to the project root. Write `dial-rag-app/app.py`, not just `app.py`. In step headings use the relative path: "Step 2: Create `dial-rag-app/app.py`". + +## Step 5: Apply terminology rules + +### Canonical names + +Read `docs-planning/glossary.md` for all canonical terms, product names, component names, deprecated terms, and contested names. The glossary is the single source of truth. Key rules: + +- **DIAL** in all-caps. Never "Dial" or "dial." +- Component names capitalized: **DIAL Core**, **DIAL Chat**, **DIAL Admin**, **DIAL SDK** +- Use the glossary's canonical name for contested terms (e.g., "Agent Builder" not "application runner") +- Don't use deprecated terms (Assistant, Addon) without a deprecation notice on the page +- Don't use vague references ("the backend," "the server") — use the glossary's component names + +### Forbidden phrases + +Delete these — never use them: +- "simply," "just," "easily," "obviously" +- "please note that" (start with the thing) +- "our product" (say "DIAL") +- "AI-powered" (everything here is AI) +- "cutting-edge," "best-in-class" +- "it should be noted that" (start with the thing) +- "as shown in the video" (text must stand alone) +- "click here" (describe the destination) + +### Capitalization +- Product names: **DIAL**, **DIAL Core**, **DIAL Chat**, **DIAL Admin**, **DIAL SDK** +- Lowercase for generic nouns: "an application," "an adapter" (mid-sentence) +- HTTP verbs: uppercase (`GET`, `POST`) +- Environment variables: uppercase, underscored, code font (`DIAL_URL`) + +## Step 6: Format code blocks correctly + +- Always specify language: ` ```bash `, ` ```json `, ` ```python `, ` ```yaml ` +- No shell prompts: not `$ docker compose up`, just `docker compose up` +- Variable placeholders in `ANGLE_BRACKETS`: ``, `` +- Pin versions in all runnable examples (except Quick Start) +- Comments explain *why*, not *what* +- Use `0.0.0.0`, not `localhost`, in example hosts and URLs: `http://0.0.0.0:8080`, not `http://localhost:8080` + +## Step 7: End with "Next steps" + +**Every page must end with a "Next steps" or "What's next" section** containing 2–3 links to logical follow-up pages. No dead ends. Examples: + +```markdown +## Next steps + +- [Getting started with the DIAL API](/building/getting-started-api) — learn to call the Unified API programmatically +- [Build a RAG app](/building/custom-apps/tutorial-rag) — create your first Custom App with retrieval +- [Configuration reference](/operating/configuration/) — all configuration options for DIAL Core +``` + +## Step 8: Self-check before finishing + +Run through this checklist: + +- [ ] Page is exactly one Diátaxis type, declared in frontmatter +- [ ] Persona declared in frontmatter +- [ ] First 60 words answer: what, who, prerequisites +- [ ] Terminology follows the canonical names table +- [ ] No forbidden phrases +- [ ] Code blocks have language specified +- [ ] Examples use `0.0.0.0`, not `localhost` +- [ ] Internal links are relative paths ending in `.md` (no absolute `/path`, no missing extension) +- [ ] No `:::` admonitions — highlights use a bold label + blockquote +- [ ] Bold lead-in labels introducing a block have a line break before the content +- [ ] Version pins present in runnable examples +- [ ] Tutorials include a project structure tree diagram and use exact file paths (relative to project root) +- [ ] Headings are sentence case, self-contained (not "Step 1" alone, but "Step 1: Install the Helm chart") +- [ ] Max heading depth: H3 in tutorials and how-tos, H4 in reference +- [ ] "Next steps" section at the end with 2–3 links +- [ ] No links to GitHub READMEs as authoritative source (link to docs site pages) +- [ ] `last_verified` date is today (only if you verified all code examples and links on the page) + +## Links + +Internal doc-to-doc links use a **relative path** that ends in the **`.md` +extension**. Docusaurus resolves these and validates them at build time. + +```markdown +[DIAL SDK reference](../developer-tools/sdk-reference/0.index.md) +[Configuration precedence](./precedence.md) +``` + +- Never use an absolute root path (`/building/getting-started-api`). +- Never omit the extension (`../developer-tools/sdk-reference/0.index`). +- Link text describes the destination — never "click here." + +## Admonitions + +Use sparingly. Do **not** use Docusaurus `:::` admonition syntax. Highlight with a +**bold label line immediately followed by a blockquote**. Only four labels: + +```markdown +**Note** +> Incidental information. + +**Tip** +> A shortcut or better-practice pointer. + +**Warning** +> Something that could cause data loss, downtime, security exposure, or cost. + +**Deprecated** +> This feature is going away. Use [replacement](../path/page.md) instead. Removal planned for vX.Y. +``` + +## Writing mechanics + +- Sentence length: target 15–20 words. Hard ceiling: 30. +- Paragraph length: 2–4 sentences. +- Numbered lists for **ordered** sequences (steps). Bullets for **unordered** sets. Render a set of consecutive, parallel standalone statements as a bullet list — not as bare back-to-back paragraphs. +- Bold lead-in labels (`**Verify:**`, `**Result:**`, `**Note:**`) that introduce a multi-sentence block get a blank line after the label; the content follows as its own paragraph — not inline on the same line. +- Oxford comma, always. +- Dates: `2026-04-27` (ISO). Never `04/27/2026`. +- No emojis in docs content. +- Em dash `—` with no surrounding spaces. + +## Where to place the page + +Consult `docs-planning/recommended-site-structure.md` to determine the correct section and path. If the page doesn't have a clear home in the structure document, flag it — don't guess. + +### Sidebar registration + +After creating a new page or moving an existing one, update `docs/sidebars.js` to include the page in the position specified by the structure document. After moving a page, add a redirect in `docs/docusaurus.config.js` if the URL changed. + +### Example URLs in templates + +The "Next steps" examples in this skill and in the reference templates use target-structure URLs (e.g., `/building/getting-started-api`) that may not exist yet. Before using a path in a real page, verify the target page exists. If it doesn't, link to the closest existing equivalent or omit the link. + +## Step 9: Verify the build + +After writing or editing, run: + +```bash +cd docs && npm run build +``` + +The Docusaurus config has `onBrokenLinks: 'throw'` — the build catches all broken internal links, anchors, and markdown references. Fix any errors before considering the page done. + +## Frontmatter validation + +The custom frontmatter fields (`type`, `persona`, `component`, `last_verified`, `owner`) are not validated by the Docusaurus build system. Enforcement comes from skill checks, the `docs-auditor` skill, and code review. A missing `type` field won't break the build but will fail audit. + +## Quality verification + +To verify a page meets all standards after writing, invoke the `docs-auditor` skill. The typical workflow is: write → build → audit → fix → re-audit. \ No newline at end of file diff --git a/.claude/skills/docs-page-writer/references/template-explanation.md b/.claude/skills/docs-page-writer/references/template-explanation.md new file mode 100644 index 000000000..b8d09d082 --- /dev/null +++ b/.claude/skills/docs-page-writer/references/template-explanation.md @@ -0,0 +1,71 @@ +# Explanation Template + +Use this template when `type: explanation` in frontmatter. + +Explanations are **understanding-oriented**. They discuss *why* something is the way it is: context, trade-offs, design decisions. The reader wants to understand, not do. + +## Rules + +- Never contains setup instructions. Link to How-tos for practical follow-up. +- May use "we" for design intent ("We chose this approach because…"). +- Diagrams and concept maps are encouraged. +- Can reference trade-offs and alternatives honestly. +- Longer form is fine — this is where depth lives. + +## Structure + +```markdown +--- +title: "RAG in DIAL" +type: explanation +persona: app-dev +component: apps +last_verified: 2026-04-27 +owner: "@dial-sdk-team" +--- + +# RAG in DIAL + +Retrieval-augmented generation (RAG) grounds model responses in real data by +retrieving relevant context before generating. This page explains how DIAL +approaches RAG, what components are involved, and when to use DIAL RAG versus +building your own pipeline with raw frameworks. + +## Why RAG matters for enterprise deployments + +(Context and motivation — 2–4 paragraphs) + +## DIAL's approach + +(Architecture explanation with diagram) + +### DIAL RAG components + +(Component overview: DIAL RAG, RAG Eval, Tool Sets, file storage) + +### How DIAL RAG differs from LangChain/LlamaIndex + +(Honest comparison — where DIAL adds value, where raw frameworks give more control) + +## Trade-offs and limitations + +(What DIAL RAG is not good at, when to choose a different approach) + +## Further reading + +- [Tutorial: Build a RAG app](/building/custom-apps/tutorial-rag) — hands-on guided build +- [RAG Eval toolkit](/building/evaluations/rag-eval) — measure retrieval and generation quality +- [Tool Sets](/building/quick-apps/tool-sets/) — connect RAG to agent workflows + +## Next steps + +- [Build a RAG app](/building/custom-apps/tutorial-rag) — put this understanding into practice +- [Enterprise RAG reference architecture](/use-cases/architectures/enterprise-rag) — production deployment pattern +``` + +## Anti-patterns to avoid + +- **Smuggling in reference details.** Parameter tables belong in Reference pages. +- **Giving setup instructions.** "First, install…" belongs in a Tutorial or How-to. +- **Being vague to avoid controversy.** Explanations should state trade-offs clearly. "This approach sacrifices X for Y" is better than "there are various considerations." +- **Skipping diagrams.** If the concept has spatial or structural relationships, a diagram is mandatory, not optional. \ No newline at end of file diff --git a/.claude/skills/docs-page-writer/references/template-howto.md b/.claude/skills/docs-page-writer/references/template-howto.md new file mode 100644 index 000000000..5e9e3a9c3 --- /dev/null +++ b/.claude/skills/docs-page-writer/references/template-howto.md @@ -0,0 +1,79 @@ +# How-to Template + +Use this template when `type: how-to` in frontmatter. + +How-tos are **task-oriented**. They help a competent user accomplish a specific goal. The reader already knows what they want to do — they need the steps. + +## Rules + +- One goal per page. Title starts with a verb: "Configure…", "Add…", "Rotate…", "Enable…" +- Assume baseline familiarity with DIAL. Don't explain what DIAL Core is. +- Alternatives are welcome (unlike tutorials). "You can also configure this via environment variable." +- No learning outcomes section — this isn't a tutorial. + +## Structure + +```markdown +--- +title: "Configure rate limits" +type: how-to +persona: devops +component: core +last_verified: 2026-04-27 +owner: "@dial-core-team" +--- + +# Configure rate limits + +Rate limits control how many requests a user or role can make to a deployment +within a time window. This guide covers configuration via the admin API and +via static config files. + +## Prerequisites + +- DIAL Core running (local or deployed) +- Admin API access or file system access to the Core config directory + +## Configure via Admin API + +1. Send a `POST` request to `/v1/admin/rate-limits`: + + curl -X POST https:///v1/admin/rate-limits \ + -H "Api-Key: " \ + -H "Content-Type: application/json" \ + -d '{ + "deployment": "gpt-4", + "role": "default", + "limit": 100, + "window_seconds": 3600 + }' + +2. Verify the rate limit is active: + + curl https:///v1/admin/rate-limits \ + -H "Api-Key: " + +## Configure via config file + +Add the following to `rate-limits.json` in the Core config directory: + + (config example) + +Restart DIAL Core for changes to take effect. + +## Related tasks + +- [Roles and rate limits](/operating/auth/roles-rate-limits) — understand the role model +- [Usage limits and cost control](/administering/usage-limits) — set cost caps per user + +## Next steps + +- [Monitoring](/operating/observability/metrics) — track rate limit hits in your dashboards +- [Alerting](/operating/observability/alerting) — set up alerts for rate limit exhaustion +``` + +## Anti-patterns to avoid + +- **Turning into a tutorial** by explaining basics. The reader knows what rate limits are. +- **Burying the steps** under three paragraphs of context. Goal → Prerequisites → Steps. +- **Missing the "Related tasks" section.** How-tos connect to each other. \ No newline at end of file diff --git a/.claude/skills/docs-page-writer/references/template-reference.md b/.claude/skills/docs-page-writer/references/template-reference.md new file mode 100644 index 000000000..481ac209e --- /dev/null +++ b/.claude/skills/docs-page-writer/references/template-reference.md @@ -0,0 +1,73 @@ +# Reference Template + +Use this template when `type: reference` in frontmatter. + +Reference pages are **information-oriented**. Exhaustive, neutral, authoritative description of an interface, configuration, or API. The reader is looking up a specific detail. + +## Rules + +- Every setting documented, including defaults and precedence. +- No narrative. No "you should." No opinions. +- Alphabetical or structural ordering, not "most useful first." +- If a setting has a side effect, document it. +- Include a concrete example for every parameter, not just the type. +- Third person only. No second person. + +## Structure + +```markdown +--- +title: "Core configuration" +type: reference +persona: devops +component: core +last_verified: 2026-04-27 +owner: "@dial-core-team" +--- + +# Core configuration + +This page documents all configuration settings for DIAL Core. Settings can be +provided via config file, environment variable, or CLI argument. See +[Config precedence rules](/operating/configuration/precedence) for resolution order. + +## Server settings + +### `server.port` + +| Property | Value | +|---|---| +| Type | integer | +| Default | `8080` | +| Env var | `DIAL_SERVER_PORT` | +| Config path | `server.port` | +| Since | Core 0.30 | + +The TCP port DIAL Core listens on for HTTP requests. + +**Example:** + + DIAL_SERVER_PORT=9090 + +### `server.host` + +(same table structure) + +## Authentication settings + +### `auth.jwt.jwks-url` + +(same table structure) + +## Next steps + +- [Configuration precedence](/operating/configuration/precedence) — how file, env, and CLI settings interact +- [Dependency configuration](/operating/configuration/dependencies) — Redis and blob storage settings +``` + +## Anti-patterns to avoid + +- **Hidden behavior.** If setting `X` also affects `Y`, document it. +- **Missing defaults.** Every parameter shows its default, even if it's "none" or "empty." +- **Narrative creep.** "You might want to increase this if…" belongs in a How-to, not a Reference. +- **Inconsistent table structure.** Every parameter uses the same table layout. \ No newline at end of file diff --git a/.claude/skills/docs-page-writer/references/template-tutorial.md b/.claude/skills/docs-page-writer/references/template-tutorial.md new file mode 100644 index 000000000..cc20ee56f --- /dev/null +++ b/.claude/skills/docs-page-writer/references/template-tutorial.md @@ -0,0 +1,102 @@ +# Tutorial Template + +Use this template when `type: tutorial` in frontmatter. + +Tutorials are **learning-oriented**. They take a newcomer from zero to a working result. The reader is doing, not just reading. + +## Rules + +- Every step must be runnable as written on a clean machine. No "adapt this to your environment." +- Don't explain *why* mid-step — defer to an Explanation page and link to it. +- Include a verification check after every meaningful action ("you should see…"). +- Pair with a sample repo: all code, data, and config committed to `github.com/epam/ai-dial-samples/`. +- Show a **project structure** tree after the setup step so the reader knows what they're building. Use exact paths relative to the project root when referencing files (e.g., `dial-rag-app/app.py`, not just `app.py`). +- If a tutorial video exists, embed it on this page — videos don't live in a separate section. + +## Structure + +```markdown +--- +title: "Build a RAG app with DIAL" +type: tutorial +persona: app-dev +component: apps +last_verified: 2026-04-27 +owner: "@dial-sdk-team" +--- + +# Build a RAG app with DIAL + +In this tutorial, you'll build a retrieval-augmented generation (RAG) application +using DIAL SDK and register it in DIAL Core. By the end, you'll have a working app +that answers questions from a document collection. + +## Prerequisites + +- DIAL running locally via Docker Compose ([Quick Start](/quick-start)) +- Python 3.11+ installed +- Basic familiarity with Python and REST APIs + +## What you'll build + +_Screenshot or diagram of the end state._ + +A Custom App that takes a user question, retrieves relevant chunks from a +document store, and generates an answer using a model deployed in DIAL. + +## Project structure + +By the end of this tutorial, your project will look like this: + + dial-rag-app/ + ├── app.py + ├── requirements.txt + └── config/ + └── core-config.json + +## Step 1: Set up the project + +Create a new directory and install DIAL SDK: + + mkdir dial-rag-app && cd dial-rag-app + pip install aidial-sdk==0.22.0 + +**Verify:** `pip show aidial-sdk` shows version 0.22.0. + +## Step 2: Create the application + +Create `dial-rag-app/app.py` with the following content: + + (complete, runnable code here) + +**Verify:** (what to check) + +## Step 3: Register in DIAL Core + +(steps with verification) + +## Step 4: Test the application + +(steps with verification) + +## What you learned + +- How to create a Custom App using DIAL SDK +- How to register an application in DIAL Core +- How to connect a retriever to the chat completion flow + +## Next steps + +- [Tool Sets](/building/quick-apps/tool-sets/) — add tools to your app for dynamic data access +- [Evaluations](/building/evaluations/) — measure your RAG app's quality +- [Deploy to Kubernetes](/operating/cloud-deployment/) — take your app to production +``` + +## Anti-patterns to avoid + +- **Explaining theory mid-step.** "Redis uses an append-only file for persistence, which means…" — save it for an Explanation page. +- **Offering alternatives.** "You could also use X instead." Tutorials have one path. How-tos have alternatives. +- **Assuming environment.** "If you're on Windows…" — link to the Environment Prerequisites page instead. +- **Skipping verification.** Every step needs a "you should see" checkpoint. +- **Ending without "What you learned" and "Next steps."** +- **Omitting the project layout.** Readers need to see the full directory tree to orient themselves. Never reference a file without its path from the project root. \ No newline at end of file diff --git a/.claude/skills/docs-researcher/SKILL.md b/.claude/skills/docs-researcher/SKILL.md new file mode 100644 index 000000000..da4069f76 --- /dev/null +++ b/.claude/skills/docs-researcher/SKILL.md @@ -0,0 +1,289 @@ +--- +name: docs-researcher +description: > + Research DIAL component repositories and external sources to produce + structured technical briefs that feed into the docs-page-writer workflow. + Use this skill when preparing to write documentation that requires + understanding source code, APIs, configuration schemas, or architecture + from any of the 20+ DIAL repositories. Triggers on: "research", + "investigate", "find out how X works", "what does the code say about", + "extract config from", "map the API for", "how does X connect to Y", + "deep dive into", "what env vars does X have", "what endpoints does X expose", + or any request to gather technical information from DIAL source repos + before writing docs. Also use when the user references a gap from + docs-planning/gap-analysis.md that requires source code investigation. + For quick targeted questions, answers inline without saving a file. + For substantial investigations, produces a research brief saved to + .claude-workspace/research/. Do not use for writing docs pages + (use docs-page-writer) or auditing existing docs (use docs-auditor). +--- + +# DIAL Documentation Researcher + +This skill gathers technical information from DIAL component repositories and external sources. It produces structured research briefs that feed into the `docs-page-writer` workflow, bridging the gap between the gap analysis (what's missing) and documentation writing (filling the gaps). + +## Tools + +| Step | Tool | Purpose | +|---|---|---| +| Read local files | `Read` | Load files from this repo: docs-planning/, existing docs, CLAUDE.md | +| Read GitHub files | `Bash` with `gh api` | Fetch individual files: `gh api repos/epam//contents/ -q .content \| base64 -d` | +| Browse repo tree | `Bash` with `gh api` | List repo structure: `gh api repos/epam//git/trees/main?recursive=1 -q '.tree[].path'` | +| Fetch rendered content | `WebFetch` | Read docs site pages, GitHub rendered content, or external sources | +| Search code on GitHub | `Bash` with `gh api` | Search across repos: `gh api -X GET '/search/code?q=org:epam+repo:ai-dial-core+getenv' -q '.items[].path'` | +| Browse UI | `agent-browser` | Navigate docs.dialx.ai, dialx.ai/dial_api, or GitHub web UI for complex browsing | +| Save research brief | `Write` | Write results to `.claude-workspace/research/--.md` | +| Update research index | `Edit` | Update `.claude-workspace/research/_index.md` | + +## Planning documents + +Load selectively based on the research mode: + +| Research mode | Required documents | +|---|---| +| Component deep-dive | `CLAUDE.md`, `references/repo-map.md`, `docs-planning/gap-analysis.md` | +| Feature investigation | `CLAUDE.md`, `references/repo-map.md`, `docs-planning/gap-analysis.md`, `docs-planning/recommended-site-structure.md` | +| API mapping | `CLAUDE.md`, `references/repo-map.md` | +| Config extraction | `CLAUDE.md`, `references/repo-map.md` | +| Architecture mapping | `CLAUDE.md`, `references/repo-map.md`, `docs-planning/recommended-site-structure.md`, `docs-planning/glossary.md` | + +## Reference files + +| File | Purpose | +|---|---| +| `references/repo-map.md` | Structured map of all 20+ DIAL repos with URLs, languages, key files, related components, and known documentation gaps | +| `references/research-brief-template.md` | Output template for research briefs including naming convention, frontmatter, section structure, and index format | + +--- + +## Operating modes + +### Quick lookup + +For narrow, targeted questions: "what env vars does DIAL Core have?", "where is the auth middleware defined?", "what's the SDK base class for apps?" + +**Behavior:** +1. Identify the relevant repo(s) from the repo map +2. Fetch the specific files needed to answer the question +3. Answer inline — no file saved +4. If the question reveals a topic worth deeper investigation, suggest a full research brief + +### Full research brief + +For substantial investigations that will feed into documentation writing. Produces a saved brief at `.claude-workspace/research/--.md`. + +Five sub-modes: + +1. **Component deep-dive** — Research a specific DIAL component +2. **Feature investigation** — Cross-cutting research across multiple repos +3. **API mapping** — Extract API surface and compatibility information +4. **Config extraction** — Pull configuration schemas, env vars, defaults +5. **Architecture mapping** — Map component relationships and data flows + +**When to use which:** Use quick lookup for questions answerable by reading 1–3 files. Use full brief when the topic is broad, touches multiple repos, or will directly feed a documentation page. + +--- + +## Repo access strategy + +Use a tiered approach, starting with the lightest-weight method: + +### Tier 1: `gh api` (default) + +For reading specific files and listing repo structure. Fast, no disk usage. + +```bash +# List full file tree +gh api repos/epam/ai-dial-core/git/trees/main?recursive=1 -q '.tree[].path' + +# Filter for specific patterns +gh api repos/epam/ai-dial-core/git/trees/main?recursive=1 -q '.tree[].path' | grep -E '(Config|Setting|Properties)' + +# Read a specific file +gh api repos/epam/ai-dial-core/contents/README.md -q .content | base64 -d + +# Read from a subdirectory +gh api repos/epam/ai-dial-core/contents/src/main/resources -q '.[].name' +``` + +### Tier 2: `WebFetch` + +For rendered content, docs site pages, and external sources. + +``` +WebFetch: https://raw.githubusercontent.com/epam/ai-dial-core/main/README.md +WebFetch: https://docs.dialx.ai/platform/core/about-core +``` + +### Tier 3: GitHub code search + +When searching across many files is needed (e.g., "find all environment variables in DIAL Core"). Use the GitHub search API — no cloning required. + +```bash +# Search for env vars in a Java repo +gh api -X GET '/search/code?q=System.getenv+repo:epam/ai-dial-core+language:java' -q '.items[] | "\(.path):\(.name)"' + +# Search for env vars in a Python repo +gh api -X GET '/search/code?q=os.environ+repo:epam/ai-dial-sdk+language:python' -q '.items[] | "\(.path):\(.name)"' + +# Then fetch the specific files you need +gh api repos/epam/ai-dial-core/contents/src/main/java/com/epam/aidial/core/config/Config.java -q .content | base64 -d +``` + +If GitHub search rate limits are hit, fall back to listing the repo tree and fetching files individually: + +```bash +# List all files in a repo +gh api repos/epam/ai-dial-core/git/trees/main?recursive=1 -q '.tree[] | select(.path | test("Config|config|settings")) | .path' + +# Read a specific file +gh api repos/epam/ai-dial-core/contents/ -q .content | base64 -d +``` + +### Tier 4: `agent-browser` + +For complex browsing tasks: navigating the rendered docs site structure, reading API docs at `dialx.ai/dial_api`, or exploring GitHub UI when `gh api` output is hard to parse. + +--- + +## Research procedures + +### Mode 1: Component deep-dive + +The user names a specific DIAL component (e.g., "research DIAL Core", "deep dive into the SDK"). + +**Procedure:** + +1. Load `references/repo-map.md`. Identify the repo URL, language, build system, key files, and known documentation gaps. +2. Read the repo's README via `gh api`. +3. List the file tree via `gh api` to understand the codebase layout. +4. Identify and read key source files: + - Entry points and main classes + - Configuration loading (config classes, env var reads, default values) + - API route definitions (handlers, controllers, decorators) + - Test files (reveal expected behavior and edge cases) +5. Cross-reference with `docs-planning/gap-analysis.md` — which gaps does this component address? +6. Cross-reference with `docs-planning/recommended-site-structure.md` — which planned pages does this research feed? +7. Write the research brief using the template. +8. Update the research index. + +### Mode 2: Feature investigation + +The user names a cross-cutting feature (e.g., "how does access control work?", "how do Tool Sets work?"). + +**Procedure:** + +1. Determine which repos are involved using the repo map and architectural knowledge. Most features touch 2–4 repos. +2. For each relevant repo, do a targeted investigation focused on the feature: + - Search for relevant classes, functions, config keys + - Read the implementation to understand behavior +3. Trace the feature flow across components. Document the sequence: + - Who initiates? (client, admin, system) + - What passes through Core? + - What touches storage, auth, or external services? +4. Synthesize findings into a "Component interactions" section showing data flow. +5. Write the brief with a cross-component synthesis, not just per-repo findings. +6. Update the index. + +### Mode 3: API mapping + +Extract the API surface for a specific area (e.g., Unified API, Admin API, file management). + +**Procedure:** + +1. Identify where API routes are defined in the source code: + - DIAL Core (Java): handler/controller classes, route registrations + - SDK-based components (Python): route decorators, endpoint definitions +2. Read route definitions to extract: method, path, parameters, request/response bodies. +3. Cross-reference with the external API docs at `dialx.ai/dial_api` via WebFetch. +4. Document OpenAI compatibility: which endpoints match the OpenAI spec, which are DIAL extensions, which parameters are unsupported. +5. Write the brief with endpoint tables. +6. Update the index. + +### Mode 4: Config extraction + +Pull configuration details from source code for a specific component. + +**Procedure:** + +1. Identify the component and repo. +2. Shallow clone — config extraction almost always requires grep across many files. +3. Search for configuration patterns by language: + - **Java** (Core, Admin Backend): `System.getenv`, `@Value("${...}")`, `@ConfigProperty`, HOCON/YAML loading, `*Config*` classes + - **Python** (SDK, Adapters, RAG): `os.environ`, `os.getenv`, `pydantic.BaseSettings`, `.env` file loading, `*Settings*` classes + - **TypeScript** (Chat, Admin Frontend): `process.env`, `.env.*` files, config modules +4. For each config key found, extract: key name, env var, type, default value, description (from comments or variable names), source file. +5. Compile into a configuration table. +6. Clean up the clone. +7. Write the brief. +8. Update the index. + +### Mode 5: Architecture mapping + +Map component relationships, data flows, and extension points. + +**Procedure:** + +1. Read architecture-related source from multiple repos: + - Entry points that show startup and component wiring + - Client libraries that show inter-service communication + - Protocol definitions (SDK base classes, API contracts) +2. Trace the primary request flow: client → Core → interceptors → model/application → response. +3. Identify extension points — where custom code can be injected: + - Application (via SDK) + - Adapter (via SDK) + - Interceptor (via Interceptors SDK) + - Quick App / Code App (via App Builder) + - Tool Set (MCP server) + - Overlay, Visualizer, Theme (Chat extensions) +4. Use canonical terms from `docs-planning/glossary.md`. +5. Include a Mermaid diagram in the brief showing component relationships. +6. Write the brief. +7. Update the index. + +--- + +## Output format + +### Quick lookup + +Answer inline. Use code blocks, tables, or bullet lists as appropriate. End with a note if a full research brief would be valuable: + +> This covers the quick answer. For a comprehensive research brief on [topic], I can run a full [mode] investigation. + +### Full research brief + +Follow the template in `references/research-brief-template.md`. Key requirements: + +- **Naming**: `--.md` in `.claude-workspace/research/` +- **Frontmatter**: topic, mode, date, status, repos_consulted, gaps_addressed, target_pages +- **Sections**: Scope → Sources consulted → Findings → Code examples → Config details → Documentation implications → Open questions +- **Tables**: Use tables for structured data (config keys, API endpoints, sources) +- **Terminology**: Use canonical terms from `docs-planning/glossary.md` + +### Research index + +Maintain `.claude-workspace/research/_index.md` — a markdown table tracking all briefs. Update it every time a brief is created or modified. + +--- + +## Multi-session research + +Large topics may require multiple sessions. + +1. **Before starting**: Check `.claude-workspace/research/` for existing briefs on the same topic. Read them to avoid duplicate work. +2. **Partial briefs**: Set `status: partial` in frontmatter. Add an "Open questions / Remaining work" section listing what's been covered and what remains. +3. **Resuming**: When the user says "continue research on X", read the existing partial brief, pick up from the "Remaining work" list, and update the brief in place. +4. **Completion**: Change status to `complete` when all planned subtopics are covered and open questions are either resolved or explicitly deferred. + +--- + +## Integration with other skills + +| Skill | Integration | +|---|---| +| `docs-page-writer` | Writer reads research briefs as input context when creating documentation pages. The "Documentation implications" section maps directly to target pages. | +| `docs-auditor` | Research briefs use canonical terminology. The auditor can verify that pages written from briefs accurately reflect source material. | +| `docs-planning/` | Every brief links to specific gaps in `gap-analysis.md` and target pages in `recommended-site-structure.md`. | + +**Workflow**: gap analysis → **research** → page writing → audit → iteration diff --git a/.claude/skills/docs-researcher/references/repo-map.md b/.claude/skills/docs-researcher/references/repo-map.md new file mode 100644 index 000000000..1fef3c7fc --- /dev/null +++ b/.claude/skills/docs-researcher/references/repo-map.md @@ -0,0 +1,283 @@ +# DIAL Repository Map + +This is the structured reference of all DIAL repositories. Use it to determine where to look when researching a topic. + +## How to use this map + +1. Identify which component(s) are relevant to your research topic +2. Use the GitHub URL to access the repo via `gh api` +3. Start with the **Key files** to orient yourself in the codebase +4. Check **Related components** for cross-cutting features +5. Note **Documentation gaps** to understand what research is most needed + +## Core platform + +### DIAL Core + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-core` | +| URL | https://github.com/epam/ai-dial-core | +| Language | Java 21 | +| Build | Gradle | +| Purpose | Main component — Unified API server for models, apps, and adapters | +| Key files | `README.md`, `build.gradle`, `src/main/java/**/` (route handlers, config classes), `src/main/resources/` (default config), any `*Config*` or `*Settings*` or `*Properties*` classes, OpenAPI spec if present | +| Related | Auth Helper (auth), SDK (app protocol), Adapters (model routing), Helm (deployment) | +| Gaps | #2 (no API guide), #8 (config on GitHub), #10 (no OpenAI compat matrix), #15 (no dependency config), #16 (no error reference) | + +### DIAL Chat + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-chat` | +| URL | https://github.com/epam/ai-dial-chat | +| Language | TypeScript | +| Build | NX monorepo | +| Purpose | Default web UI — includes Overlay, Theming, Visualizer Connector | +| Key files | `README.md`, `apps/chat/` (main app), `libs/` (shared libraries), `libs/overlay/` (Overlay SDK), `libs/theming/`, environment config files | +| Related | Core (API client), Admin Frontend (admin UI) | +| Gaps | #3 (apps ecosystem — Code Apps, Quick Apps surface here), #11 (naming chaos — "Application builders" label used here) | + +### DIAL Admin Frontend + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-admin-frontend` | +| URL | https://github.com/epam/ai-dial-admin-frontend | +| Language | TypeScript | +| Build | NX / Next.js | +| Purpose | Admin Panel web application | +| Key files | `README.md`, `src/` or `apps/` (main app), page components, API client code | +| Related | Admin Backend (API), Core (managed entities) | +| Gaps | #11 (naming — "Builders" label used in Admin sidebar) | + +### DIAL Admin Backend + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-admin-backend` | +| URL | https://github.com/epam/ai-dial-admin-backend | +| Language | Java | +| Build | Gradle | +| Purpose | Admin Panel API for DIAL Core | +| Key files | `README.md`, API route definitions, entity models, config classes | +| Related | Admin Frontend (UI), Core (upstream API) | +| Gaps | #8 (config on GitHub) | + +### DIAL Auth Helper + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-auth-helper` | +| URL | https://github.com/epam/ai-dial-auth-helper | +| Language | Python / TypeScript (check repo) | +| Build | Check repo | +| Purpose | AuthProxy service implementing OpenID-compatible endpoints | +| Key files | `README.md`, auth flow implementation, IDP configuration examples, env var definitions | +| Related | Core (auth integration), Helm (deployment config) | +| Gaps | #14 (networking/firewall), auth configuration across IDPs | + +## SDKs and frameworks + +### DIAL SDK + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-sdk` | +| URL | https://github.com/epam/ai-dial-sdk | +| Language | Python >= 3.11 | +| Build | Poetry | +| Purpose | Framework for creating applications and model adapters | +| Key files | `README.md`, `pyproject.toml`, `aidial_sdk/` (main package), base classes for apps/adapters, example apps if any, tests | +| Related | Core (protocol), Interceptors SDK (sister SDK), App Builder (deployment) | +| Gaps | #1 (no tutorials), #3 (apps ecosystem), #21 (no testing guidance) | +| Note | Also a git submodule in this repo at `dial-sdk/` | + +### DIAL Interceptors SDK + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-interceptors-sdk` | +| URL | https://github.com/epam/ai-dial-interceptors-sdk | +| Language | Python | +| Build | Poetry | +| Purpose | Framework for creating interceptors for chat completion and embedding models | +| Key files | `README.md`, `pyproject.toml`, base interceptor classes, examples | +| Related | Core (interceptor chain), SDK (sister SDK) | +| Gaps | #3 (apps ecosystem — interceptors underdocumented) | + +## Model adapters + +### OpenAI Adapter + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-adapter-openai` | +| URL | https://github.com/epam/ai-dial-adapter-openai | +| Language | Python | +| Build | Poetry | +| Purpose | Adapter for Azure OpenAI, OpenAI, vLLM, and other OpenAI-compatible APIs | +| Key files | `README.md`, model mapping, supported parameters, env vars | +| Related | Core (adapter protocol), Helm (deployment) | +| Gaps | #10 (OpenAI compatibility matrix — which params supported) | + +### Bedrock Adapter + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-adapter-bedrock` | +| URL | https://github.com/epam/ai-dial-adapter-bedrock | +| Language | Python | +| Build | Poetry | +| Purpose | Adapter for AWS Bedrock models | +| Key files | `README.md`, supported models, env vars, parameter mapping | +| Related | Core, Helm | +| Gaps | #10 (compatibility — which Bedrock models/features) | + +### Vertex AI Adapter + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-adapter-vertexai` | +| URL | https://github.com/epam/ai-dial-adapter-vertexai | +| Language | Python | +| Build | Poetry | +| Purpose | Adapter for Google Vertex AI models | +| Key files | `README.md`, supported models, env vars, parameter mapping | +| Related | Core, Helm | +| Gaps | #10 (compatibility — which Vertex models/features) | + +## Deployment + +### DIAL Helm + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-helm` | +| URL | https://github.com/epam/ai-dial-helm | +| Language | Helm / YAML | +| Build | Helm charts | +| Purpose | Helm charts for Kubernetes deployment; stable assemblies published here | +| Key files | `README.md`, `charts/dial/` (main chart), `charts/dial/values.yaml` (all config), `charts/dial/examples/` (example configs) | +| Related | Core, Chat, Admin, Auth Helper, all adapters | +| Gaps | #8 (DevOps sidebar links here instead of docs site), #15 (dependency config), #17 (version compat matrix), #18 (production hardening) | + +## Tools and utilities + +### DIAL RAG + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-rag` | +| URL | https://github.com/epam/ai-dial-rag | +| Language | Python | +| Build | Poetry | +| Purpose | RAG project for retrieval-augmented generation | +| Key files | `README.md`, configuration, supported document types, chunking strategies | +| Related | Core (app protocol), SDK | +| Gaps | No RAG explanation on docs site (gap-analysis §4) | + +### DIAL RAG Eval + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-rag-eval` | +| URL | https://github.com/epam/ai-dial-rag-eval | +| Language | Python | +| Build | Poetry | +| Purpose | Library for RAG evaluation (retrieval and generation metrics) | +| Key files | `README.md`, metric definitions, evaluation pipelines | +| Related | RAG | +| Gaps | No evaluation documentation on docs site | + +### Analytics Realtime + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-analytics-realtime` | +| URL | https://github.com/epam/ai-dial-analytics-realtime | +| Language | Check repo | +| Build | Check repo | +| Purpose | Real-time usage analytics — transforms logs into InfluxDB metrics | +| Key files | `README.md`, config, metric definitions, InfluxDB schema | +| Related | Core (log source) | +| Gaps | #13 (observability stops at "use OTEL") | + +### Python Code Interpreter + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-code-interpreter` | +| URL | https://github.com/epam/ai-dial-code-interpreter | +| Language | Python | +| Build | Check repo | +| Purpose | Uses Jupyter Kernel to execute arbitrary Python code | +| Key files | `README.md`, security sandboxing, supported packages | +| Related | Core (app protocol) | +| Gaps | #3 (apps ecosystem — undocumented app type) | + +### DIAL-to-DIAL Adapter + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-adapter-dial` | +| URL | https://github.com/epam/ai-dial-adapter-dial | +| Language | Python | +| Build | Check repo | +| Purpose | Adapter for local development against a remote DIAL Core | +| Key files | `README.md`, env vars, usage examples | +| Related | Core | +| Gaps | Misleadingly placed as first page of "Apps Development" in current docs | + +### Log Parser + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-log-parser` | +| URL | https://github.com/epam/ai-dial-log-parser | +| Language | Python | +| Build | Check repo | +| Purpose | Tool to parse DIAL log files and repack to parquet dataset | +| Key files | `README.md`, input/output formats | +| Related | Core (log format), Analytics | +| Gaps | None specific | + +### App Builder + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-app-builder` | +| URL | https://github.com/epam/ai-dial-app-builder | +| Language | Python | +| Build | Check repo | +| Purpose | Downloads source from DIAL file storage and prepares container images | +| Key files | `README.md`, build pipeline, supported app types (Code Apps) | +| Related | Core, Chat (Code Apps UI surface) | +| Gaps | #3 (apps ecosystem — Code Apps build pipeline undocumented), #4 (Tool Sets) | + +## Infrastructure + +### DIAL CI + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-ci` | +| URL | https://github.com/epam/ai-dial-ci | +| Language | GitHub Actions YAML | +| Build | Reusable workflows | +| Purpose | Reusable GitHub Actions workflows for all DIAL repos | +| Key files | `.github/workflows/` | +| Related | All repos | +| Gaps | None specific | + +## Archived (do not use in new content) + +### DIAL Assistant + +| Field | Value | +|---|---| +| Repo | `epam/ai-dial-assistant` | +| URL | https://github.com/epam/ai-dial-assistant | +| Language | Python | +| Purpose | **Archived.** ChatGPT plugin protocol implementation (abandoned) | +| Note | Do not reference in new documentation. The "Assistant" and "Addon" concepts are deprecated. | diff --git a/.claude/skills/docs-researcher/references/research-brief-template.md b/.claude/skills/docs-researcher/references/research-brief-template.md new file mode 100644 index 000000000..8788b48f7 --- /dev/null +++ b/.claude/skills/docs-researcher/references/research-brief-template.md @@ -0,0 +1,109 @@ +# Research Brief Template + +Use this template for all research briefs saved to `.claude-workspace/research/`. + +## Naming convention + +``` +--.md +``` + +- **date**: ISO format, e.g., `2026-04-28` +- **mode**: one of `component`, `feature`, `api`, `config`, `architecture` +- **slug**: kebab-case topic identifier + +Examples: +- `2026-04-28-component-dial-core.md` +- `2026-04-28-feature-access-control.md` +- `2026-04-28-api-unified-api-endpoints.md` +- `2026-04-28-config-core-env-vars.md` +- `2026-04-28-architecture-app-lifecycle.md` + +## Template + +```markdown +--- +topic: "" +mode: +date: +status: +repos_consulted: + - +gaps_addressed: + - "#N: " +target_pages: + - "" +--- + +# Research: + +## Scope + +What this research covers and what it explicitly does not cover. + +## Sources consulted + +| Source | Type | Path / URL | Notes | +|---|---|---|---| +| epam/ai-dial-core | README | README.md | | +| epam/ai-dial-core | Source | src/.../ConfigHandler.java | Config loading logic | +| dialx.ai | API docs | https://dialx.ai/dial_api | Endpoint reference | + +## Findings + +### + +Structured findings. Use tables for structured data (config keys, API endpoints). +Use prose for architectural explanations and design decisions. + +### + +Continue as needed. + +## Code examples found + +Runnable examples discovered in source repos, READMEs, or test files. + +Source: `/` +``` + +``` + +## Configuration details + +| Key | Env var | Type | Default | Description | Source | +|---|---|---|---|---|---| +| | | | | | `` | + +(Omit this section if not applicable to the research mode.) + +## Documentation implications + +What pages need to be written or updated, mapped to the recommended site structure. + +| Target page | Diátaxis type | What this research provides | Priority | +|---|---|---|---| +| §X.Y Page name | reference | Complete config key listing | High | +| §X.Z Page name | explanation | Architecture diagram | Medium | + +## Open questions + +Things this research could not determine. Each should note where to look next. + +- [ ] Question 1 — look in `/` or ask `` +- [ ] Question 2 — requires running the service to verify +``` + +## Index file + +Maintain a research index at `.claude-workspace/research/_index.md`: + +```markdown +# Research Index + +| Date | Brief | Mode | Topic | Status | Gaps | Target pages | +|---|---|---|---|---|---|---| +| 2026-04-28 | [DIAL Core](2026-04-28-component-dial-core.md) | component | DIAL Core | complete | #2, #8, #15, #16 | §4.4, §4.8 | +``` + +Update this index every time a brief is created or updated. diff --git a/.github/workflows/deploy-development.yml b/.github/workflows/deploy-development.yml index ca4df88d2..afc6256a2 100644 --- a/.github/workflows/deploy-development.yml +++ b/.github/workflows/deploy-development.yml @@ -2,7 +2,7 @@ name: Deploy development on: push: - branches: [feature/doc_improvements_by_section] + branches: [feature/doc_improvements] jobs: gitlab-dev-deploy: diff --git a/.gitignore b/.gitignore index c4739b6e2..95506827a 100644 --- a/.gitignore +++ b/.gitignore @@ -14,6 +14,7 @@ .env.development.local .env.test.local .env.production.local +tmp/ npm-debug.log* yarn-debug.log* @@ -29,6 +30,14 @@ core-logs .ollama /.quarto/ +.vscode +.claude-workspace + +graphify-out +main.py +uv.lock +.python-version +pyproject.toml # Autogenerated files by Quarto docs/tutorials/1.developers/4.apps-development/3.multimodality/dial-cookbook/examples @@ -37,3 +46,4 @@ docs/tutorials/1.developers/4.apps-development/3.multimodality/dial-cookbook/exa ~* **/*.quarto_ipynb .vscode/spellright.dict +.claude/settings.local.json \ No newline at end of file diff --git a/BRANCH_GUIDE.md b/BRANCH_GUIDE.md new file mode 100644 index 000000000..a65934735 --- /dev/null +++ b/BRANCH_GUIDE.md @@ -0,0 +1,161 @@ +# Branch Guide — `feature/doc_improvements` + +A working map of this branch: how to control what the docs site shows, what the +`docs/` and `docs_v2/` trees are for, what each script in `scripts/` does (and which +ones were removed now that the migration is complete), and what lives in +`docs-planning/`. + +This is the **documentation/meta repository** for AI DIAL — the Docusaurus 3 site is +rooted at the repo root (`docusaurus.config.js`, `sidebars.js`, `sidebars-v2.js`, +`package.json`, `docs.config.js` are all top-level, **not** inside `docs/`). + +--- + +## 1. Configuring what the branch shows (OLD/NEW docs + sections) + +[`docs.config.js`](./docs.config.js) (repo root) is the **single source of truth** for +build-time visibility. It is read by both `docusaurus.config.js` (which doc sets get +built/served) and `sidebars-v2.js` (which NEW sections appear in the menu). + +There are two knobs. Each has a committed in-file default and a matching env var that +overrides it (**env always wins**): + +| Knob | In-file default | Env var | Values | +|---|---|---|---| +| Which doc sets | `DEFAULT_VARIANT = 'both'` | `DOCS_VARIANT` | `both` \| `old` \| `new` | +| Which NEW sections | `DEFAULT_SECTIONS = 'all'` | `DOCS_V2_SECTIONS` | `all` \| comma-separated section keys | + +**Committed default on this branch: `both` + `all`** — OLD and NEW are both served, and +every NEW section shows in the sidebar. + +### `DOCS_VARIANT` — which doc sets + +- `both` (default) — OLD served at `/`, NEW served at `/v2`. The navbar OLD/NEW switcher + appears only in this mode. +- `old` — only `docs/`, served at `/`. +- `new` — only `docs_v2/`, served at `/` (the sole shown set owns the root). + +The NEW instance has no page of its own at its root. An inline redirect plugin +(`docusaurus.config.js` → `src/components/RootRedirect.js`, target computed as +`NEW_ROOT_REDIRECT` in `docs.config.js`) sends the NEW root to the **first visible +section's landing page** (catalog order, per each section's `landing` in `V2_SECTIONS`). +So hiding the Home section just moves the landing to the next visible section. + +### `DOCS_V2_SECTIONS` — which NEW sections show in the sidebar + +Comma-separated **section keys** (catalog in `V2_SECTIONS` in `docs.config.js`): + +``` +home, understand-dial, building-with-dial, operating-dial, +administering-dial, chat-user-guide, reference, use-cases, demos +``` + +Hiding a section removes it from the **menu only** — its pages still build and stay +reachable, so cross-links don't break. A mistyped key prints a warning (it won't +silently drop a real section). + +> Do **not** trim the `V2_SECTIONS` catalog to hide sections — that's reference data. +> Use `DEFAULT_SECTIONS` / `DOCS_V2_SECTIONS` instead. + +### How to change it + +For a **committed** change affecting every build (including production), edit the two +`DEFAULT_*` constants near the top of `docs.config.js`. For **one-off / local** +experiments, set the env vars: + +```bash +# Defaults (both sets, all sections) — what this branch ships: +npm run start + +# NEW only, served at /: +DOCS_VARIANT=new npm run build + +# NEW sidebar shows just two sections: +DOCS_V2_SECTIONS="home,building-with-dial" npm run start +``` + +--- + +## 2. Role of `docs/` (OLD) and `docs_v2/` (NEW) + +OLD and NEW are **two separate Docusaurus instances** — physically and structurally +distinct: + +| | OLD (legacy, being phased out) | NEW (restructured) | +|---|---|---| +| Content folder | `docs/` | `docs_v2/` | +| Sidebar file | `sidebars.js` (`CustomSideBar`) | `sidebars-v2.js` (`v2Sidebar`) | +| Served at | `/` | `/v2` (in `both` mode) | +| Plugin | classic preset | second `@docusaurus/plugin-content-docs` instance (`id: v2`) | + +**All new documentation goes into `docs_v2/`. Never add content to `docs/`.** When +migrating an OLD page, move/rewrite it into the right place under `docs_v2/` per the +recommended site structure. Page ids in `sidebars-v2.js` are relative to `docs_v2/` +(e.g. `building-with-dial/apps/index` — no `docs/NEW/` prefix). + +**Link conventions** (`onBrokenLinks`, `onBrokenAnchors`, `onBrokenMarkdownLinks`, +`onBrokenMarkdownImages` are all `throw` — a broken link fails the build): + +- Internal doc-to-doc links use a **relative path ending in `.md`**, including the + numeric file prefix (e.g. `](../apps/0.index.md)`, `](./3.prompts.md#variables)`). + Never use an absolute root path (`/v2/...` or `/platform/...`). +- Keep the NEW tree **self-contained** — `docs_v2/` pages link to other `docs_v2/` + pages, not into OLD `docs/`. Cross-instance links can't be relative `.md` links. +- Index docs use a `0.index.md` filename (the numeric prefix prevents folder-collapsing). + +--- + +## 3. The `scripts/` folder + +Run via the npm scripts in `package.json` (all use `npx tsx`). + +### Active scripts + +| Script | npm script(s) | What it does | +|---|---|---| +| `build-changelog.ts` | `changelog:build` (also run by `start` / `build`) | Generates the on-site Changelog pages under `docs_v2/7.reference/changelog/` from `docs/releases/`. Idempotent, no network. | +| `check-references.ts` | `check:references`, `check:references:strict` | Scans `docs_v2/` for pinned PyPI/npm versions in sample code, crawls referenced GitHub sample repos, and writes `outdated_references.json`. `--strict` exits 1 on any outdated reference. | +| `references-shared.ts` | — (imported by the two reference scripts) | Shared types/helpers: resolves pinned versions against PyPI/npm and applies the freshness rule. | +| `render-outdated-references.ts` | `references:render-md` | Renders `outdated_references.json` into a human-readable `outdated_references.md`. Rendering only, no network. | +| `check-frontmatter.mts` | `lint:frontmatter`, `:strict`, `:fix` | Validates YAML frontmatter in every Markdown file under `docs/` and `docs_v2/` against the Style Guide. `--fix` inserts stub frontmatter. | + +### Removed: obsolete progress-tracking scripts + +Progress tracking existed to monitor the OLD → NEW migration. **All NEW docs are now +complete per the original plan, so progress tracking is obsolete** and the machinery was +removed. The build never depended on it (`build` / `start` only call `changelog:build`). + +Removed scripts: + +- `scripts/sync-tracking.ts` — generated/updated `tracking.json` + `progress.md` from the sidebar + structure doc +- `scripts/validate-tracking-records.ts` — validated `tracking.json` +- `scripts/render-progress-table.ts` — rendered `tracking.json` into `docs_v2/progress.md` +- `scripts/upsert-remove-tracking-record.ts` — upsert/remove a single tracking record +- `scripts/tracking-shared.ts` — shared types/helpers for the four above + +Removed npm scripts from `package.json`: `tracking:validate`, `tracking:sync`, +`tracking:progress`. + +Removed data/artifacts: `docs-planning/tracking.json`, `docs-planning/tracking.schema.json`, +and the auto-generated `docs_v2/progress.md` page (and its hidden sidebar stub in +`sidebars-v2.js`). + +--- + +## 4. Role of `docs-planning/` + +The non-shipped working documents that **govern** the docs-improvement initiative. These +are planning/reference material — they are not part of the published site. + +| File | Purpose | +|---|---| +| `gap-analysis.md` | 25 evidenced gaps in the current site, ranked by severity, with full URLs. | +| `improvement-roadmap.md` | Four-phase action plan (Foundation → Stop the bleeding → Depth and differentiation → Sustainability). | +| `recommended-site-structure.md` | Target information architecture with a page-by-page migration map from current state. | +| `style-guide.md` | Diátaxis-based writing rules, terminology, voice/tone per content type, review process. | +| `glossary.md` | Terminology glossary — use it for consistency. | +| `repositories.md` | Full list of DIAL component repositories. | +| `release-strategy-adr.md` | ADR on how/when to graduate the NEW docs to production. (References the old `tracking.json`/`progress.md` flow as historical record.) | + +> Note: the former `tracking.json` and `tracking.schema.json` in this folder were removed +> along with the progress-tracking scripts (see §3). diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 000000000..e0817c2c0 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,164 @@ +# CLAUDE.md + +## Rules + +- **Do not commit or push without explicit permission.** Only run `git commit` / `git push` (or any variant) when the user has explicitly authorized that specific action. Absent such authorization, the user handles version control manually. +- **Never upgrade or install packages without explicit permission.** Do not run `npm install` / `npm ci` / `npm update` (or any variant) to add, upgrade, or change dependencies, and do not edit `package.json` / `package-lock.json`, unless the user has explicitly authorized it. A build that fails on a dependency issue is a finding to report, not a license to change packages. +- **Never assume redirects are necessary after a rename or path change.** Renaming files/folders or changing routes does not automatically require adding redirects (e.g. `@docusaurus/plugin-client-redirects`). Only add redirects when the user explicitly asks for them, and if you do, wire the targets to real, existing routes and verify with a full build. + +## What this repo is + +This is the **documentation and meta repository** for AI DIAL — an open-source, enterprise-grade LLM orchestration platform by EPAM. It is NOT the application code. The platform code lives in 20+ sibling repositories. + +This repo contains: +- A **Docusaurus 3** documentation site **rooted at the repo root** (`docusaurus.config.js`, `sidebars.js`, `sidebars-v2.js`, `package.json` are all at the top level — NOT inside `docs/`), published to https://docs.dialx.ai/. It serves two doc instances: + - `docs/` — **OLD** (legacy) content, served at `/` + - `docs_v2/` — **NEW** (restructured) content, served at `/v2` +- `dial-docker-compose/` — minimal Docker Compose setups for quick start +- `dial-docker-compose-advanced/` — advanced Docker Compose configs (multiple providers, auth) +- `dial-cookbook/` — code examples and Jupyter notebooks +- `dial-samples/` — sample DIAL applications +- `dial-sdk` — git submodule pointing to https://github.com/epam/ai-dial-sdk + +## Current initiative: documentation improvement + +The docs site (https://docs.dialx.ai/) is undergoing a major restructuring. The current site has significant gaps: no in-depth tutorials, duplicate page titles pointing to different content, a mislabeled "Tutorials" section that contains no actual tutorials, canonical content redirected to GitHub READMEs, and the primary value surface (DIAL Apps, Toolsets) is largely undocumented. + +The improvement work is governed by four companion documents in `docs-planning/`: + +- **Gap Analysis** (`docs-planning/gap-analysis.md`) — 25 evidenced gaps ranked by severity, with full URLs to every referenced page +- **Improvement Roadmap** (`docs-planning/improvement-roadmap.md`) — four-phase action plan (Foundation → Stop the bleeding → Depth and differentiation → Sustainability) +- **Recommended Site Structure** (`docs-planning/recommended-site-structure.md`) — target information architecture with page-by-page migration map from current state +- **Style Guide** (`docs-planning/style-guide.md`) — Diátaxis-based writing rules, terminology glossary, voice/tone per content type, review process +- **Glossary** (`docs-planning/glossary.md`) — Terminology glossary, use it to be consistent + +Key structural changes in progress: +- Replacing the incoherent Platform/Tutorials split with journey-based sections: Understand DIAL, Building with DIAL, Operating DIAL, Administering DIAL +- Eliminating 7 duplicate-title pairs (Access Control ×2, About ×2, Deployment ×2, etc.) +- Flattening sidebar from 7 levels to max 4 +- Creating the first real tutorials (Getting started with the API, Custom App, Quick App) +- Shipping DIAL Apps documentation as a unified section (Custom Apps, Quick Apps, Code Apps, Mind Map Studio, Toolsets) +- Consolidating configuration reference on-site (currently redirects to 7+ GitHub READMEs) +- Splitting integrations by purpose: chatbot, productivity add-ins, workflow automation, orchestration patterns + +**OLD and NEW are two separate Docusaurus instances.** The legacy site and the restructured site are physically and structurally separated: + +| | OLD (legacy, being phased out) | NEW (restructured) | +|---|---|---| +| Content folder | `docs/` | `docs_v2/` | +| Sidebar file | `sidebars.js` (`CustomSideBar`) | `sidebars-v2.js` (`v2Sidebar`) | +| Served at | `/` | `/v2` | +| Plugin | classic preset | second `@docusaurus/plugin-content-docs` instance (`id: v2`) in `docusaurus.config.js` | + +**All new documentation goes into `docs_v2/`. Never add new content to `docs/` (OLD).** When migrating an existing OLD page, move or rewrite it into the appropriate place under `docs_v2/` per the Structure document. Page ids in `sidebars-v2.js` are relative to `docs_v2/` (e.g. `building-with-dial/apps/index`, no `docs/NEW/` prefix). + +When working on docs content, always follow the Style Guide conventions and place content according to the Structure document. + +## Docs site + +The Docusaurus 3 project lives at the **repo root**. Run all commands from there: + +```bash +npm install +npm run start # local dev server at http://localhost:3000 +npm run build # production build (also regenerates the changelog) +npm run serve # serve production build locally +``` + +NEW content lives in `docs_v2/` as Markdown files; OLD content in `docs/`. Sidebars: `sidebars.js` (OLD) and `sidebars-v2.js` (NEW). Site config: `docusaurus.config.js`. + +### Build-time visibility toggles + +`docs.config.js` (repo root) is the single source of truth for what gets built, read by both `docusaurus.config.js` and `sidebars-v2.js`. Override via env vars (defaults reproduce the full site): + +- **`DOCS_VARIANT`** = `both` (default) | `old` | `new` — which docs sets to build/serve. With `both`, OLD is at `/` and NEW at `/v2`. With a single set, that set owns `/`. The NEW instance root (`/v2`, or `/` in `new` mode) has no page of its own; an inline redirect plugin (`docusaurus.config.js` → `src/components/RootRedirect.js`, target from `NEW_ROOT_REDIRECT` in `docs.config.js`) sends it to the **first visible section's landing** (catalog order, per each section's `landing` in `V2_SECTIONS`). So hiding the Home section just moves the landing to the next visible section. The navbar OLD/NEW switcher appears only in `both` mode. +- **`DOCS_V2_SECTIONS`** = `all` (default) | comma-separated section keys — which NEW top-level sections show in the sidebar. Keys: `home, understand-dial, building-with-dial, operating-dial, administering-dial, chat-user-guide, reference, use-cases, demos`. Hiding a section removes it from the menu only; its pages still build and stay reachable (so cross-links don't break). + +```bash +DOCS_VARIANT=new npm run build # NEW only, served at / +DOCS_V2_SECTIONS="home,building-with-dial" npm run start # NEW sidebar shows two sections +``` + +### Links + +`onBrokenLinks`, `onBrokenAnchors`, `onBrokenMarkdownLinks`, and `onBrokenMarkdownImages` are all set to `throw` — a broken link fails the build. Conventions: +- Internal doc-to-doc links use a **relative path ending in `.md`**, including the numeric file prefix (e.g. `](../apps/0.index.md)`, `](./3.prompts.md#variables)`). **Never use an absolute root path** (`/v2/...` or `/platform/...`). +- Keep the NEW tree **self-contained** — `docs_v2/` pages should link to other `docs_v2/` pages, not into OLD `docs/`. Cross-instance links can't be relative `.md` links (Docusaurus won't resolve across instances); avoid creating them. +- Index docs use a `0.index.md` filename and route to `…/index` (the numeric prefix prevents folder-collapsing). + +### File naming and numbering (keep the tree in sync with the menu) + +**Invariant: a file's/folder's numeric `N.` prefix MUST equal its position in `sidebars-v2.js`.** Docusaurus strips the leading `N.` from every path segment (file *and* directory) when computing a route id, so the prefix is purely organizational — but the file tree must still read in menu order. Keep them aligned: + +- **Every content file and folder under `docs_v2/` carries a numeric prefix that matches its order in the menu**, at every level: top-level sections (`1.home/`, `2.understand-dial/`, … `9.demos/` — in `V2_SECTIONS` catalog order), sub-folders, and files. Numbering is **per parent directory**: files and sub-folders that share a parent draw from one sequence in their first menu appearance order. +- **`0.index.md`** is the section/category landing (route `…/index`). It is always `0.`. +- When you **add** a page: give it the prefix of its menu slot and **bump the prefixes of every sibling that comes after it** (files *and* sub-folders in that directory). When you **reorder** the menu, renumber the affected files to match. When you **remove** a page, prefer closing the numeric gap. +- **Renaming a file's prefix does NOT change its route id** (the prefix is stripped), so `sidebars-v2.js` needs no edit for a pure renumber. But it **does** change any relative `.md`/image link that spells out the old prefixed name — fix those (see Links above) and let the build verify. + +**Things that are NOT numbered** (leave them alone): +- `img/` asset folders and the images inside them. +- The generator-owned `7.reference/changelog/` folder — its files (`index.md`, `release-notes-*.md`, `upgrade-to-*.md`) are emitted by `scripts/build-changelog.ts` with **no** prefix and are overwritten on every build. Link to them by their real names (e.g. `](changelog/index.md)`), never `changelog/0.index.md`. If the `reference` section's prefix ever changes, update `OUT_DIR` in `scripts/build-changelog.ts` **and** the `autogenerated` `dirName` in `sidebars-v2.js` to match — `dirName` is a filesystem path, not a route id, so it is **not** prefix-stripped. +- Extension-less links are Docusaurus **route** paths (already prefix-stripped); never add a numeric prefix to them. + +**Before finishing any structural change, run `npm run build`** — `onBroken*` are all `throw`, so it is the authoritative check that the tree, the menu, and every link still agree. + +## Key conventions + +- **Always use "DIAL" in all-caps** when referring to the platform. Not "Dial" or "dial." +- Component names are capitalized: **DIAL Core**, **DIAL Chat**, **DIAL Admin**, **DIAL SDK**. +- The API is called the **Unified API** (not "DIAL API" which is too vague). +- **Application**, **Adapter**, **Interceptor** — these are distinct DIAL concepts with specific meanings. Don't conflate them. +- **Toolset** / **Toolsets** — one word, not "Tool Set". This matches the name used in DIAL Chat and DIAL Admin. Use it consistently (code identifiers such as `tool_sets`, `client_toolset` are unaffected). +- Deprecated concepts: **Assistant** (repo archived), **Addon** (ChatGPT plugin protocol, abandoned). Don't use in new content. + +## All related repositories + +See [`docs-planning/repositories.md`](docs-planning/repositories.md) for the full list of DIAL component repositories. + +## Docker Compose quick start + +```bash +cd dial-docker-compose/application +docker compose up +``` + +Opens DIAL Chat with an echo application at http://localhost:3000. + +## Contributing + +See `CONTRIBUTING.md`. Key points: +- DIAL is API-first. All components except Core are optional. +- Each component has its own release cadence and owner. +- Stable assemblies are published via the Helm chart. +- Biweekly releases (Wednesdays). Semantic versioning. +- Branch: `main`. PRs reviewed Thursdays. + +## Gotchas + +- The `dial-sdk` directory is a **git submodule**, not a regular folder. Run `git submodule update --init` after cloning. +- The docs site references pages in sibling repos. Broken links in docs often mean the sibling repo's README changed — check there. +- URL path `/video demos/` has a space. This is a known issue. +- Some docs pages link to GitHub as the authoritative source for configuration — this is being migrated to on-site docs (see Documentation Improvement Plan). + +## Available tools + +### Shell utilities +- `tree` — directory structure visualization. Use for exploring repo layouts. +- `git` — full git access. Use for: log, blame, diff, checking file history, + finding when a page was last modified, listing contributors. + +### Agent browser +[agent-browser](https://github.com/vercel-labs/agent-browser) is available +for headless web browsing. Use for: +- Fetching and parsing docs.dialx.ai pages +- Browse other github repos, if it's more efficient +- Verifying external links +- Crawling site structure +- Comparing rendered docs against source Markdown + +Check capabilities: `agent-browser -h` + +* * * + +## When working with Claude Code Skills +- Always refer to https://code.claude.com/docs/en/skills \ No newline at end of file diff --git a/dial-docker-compose/common.yml b/dial-docker-compose/common.yml index cfc93d472..a6127012e 100644 --- a/dial-docker-compose/common.yml +++ b/dial-docker-compose/common.yml @@ -1,6 +1,7 @@ services: themes: image: epam/ai-dial-chat-themes:0.9.1 + platform: linux/amd64 ports: - "3001:8080" @@ -8,6 +9,7 @@ services: ports: - "3000:3000" image: epam/ai-dial-chat:0.26.0 + platform: linux/amd64 depends_on: - themes - core @@ -38,6 +40,7 @@ services: ports: - "8080:8080" image: epam/ai-dial-core:0.25.1 + platform: linux/amd64 environment: 'AIDIAL_SETTINGS': '/opt/settings/settings.json' 'JAVA_OPTS': '-Dgflog.config=/opt/settings/gflog.xml' diff --git a/dial-docker-compose/model/docker-compose.yml b/dial-docker-compose/model/docker-compose.yml index 59b949a28..97cdc28c7 100644 --- a/dial-docker-compose/model/docker-compose.yml +++ b/dial-docker-compose/model/docker-compose.yml @@ -5,5 +5,6 @@ include: services: adapter-openai: image: epam/ai-dial-adapter-openai:0.22.0 + platform: linux/amd64 environment: WEB_CONCURRENCY: "3" \ No newline at end of file diff --git a/dial-docker-compose/ollama/docker-compose.yml b/dial-docker-compose/ollama/docker-compose.yml index 31bad0b82..b570553e3 100644 --- a/dial-docker-compose/ollama/docker-compose.yml +++ b/dial-docker-compose/ollama/docker-compose.yml @@ -23,6 +23,7 @@ services: adapter-openai: image: epam/ai-dial-adapter-openai:0.22.0 + platform: linux/amd64 environment: WEB_CONCURRENCY: "3" DIAL_URL: "http://core:8080" diff --git a/docs-planning/gap-analysis.md b/docs-planning/gap-analysis.md new file mode 100644 index 000000000..8a85d7238 --- /dev/null +++ b/docs-planning/gap-analysis.md @@ -0,0 +1,290 @@ +# DIAL Documentation - Gap Analysis + +> **Status:** Draft + +* * * + + +## 1. Navigation and menu structure + +### 1.1 Duplicate titles, different content + +The same label appears in multiple sidebar locations, pointing to different pages with overlapping but non-identical content. + +| Duplicated title | Location 1 | Location 2 | Problem | Duplicated title | Location 1 | Location 2 | Problem | +| :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | +| Access Control | "Introduction to Access Control in DIAL" — Menu: Platform > Architecture & Concepts > Access Control Overview — URL: https://docs.dialx.ai/platform/architecture-and-concepts/access-control | "Access Control in DIAL" — Menu: Platform > Core > Access Control — URL: https://docs.dialx.ai/platform/core/access-control-intro | Both cover access control. The first is a high-level conceptual overview (subjects, objects, roles). The second is a detailed technical treatment (authorization rules, config file vs API, per-request keys). Neither links to the other cleanly. A reader who finds one doesn't know the other exists. | | | | | +| About | "DIAL Core" — Menu: Platform > Core > About — URL: https://docs.dialx.ai/platform/core/about-core | "DIAL Chat" — Menu: Platform > Chat > About — URL: https://docs.dialx.ai/platform/chat/about-chat | Both pages use the generic sidebar label "About" for two unrelated component overviews. A reader scanning the sidebar sees "About" twice and can't distinguish them without clicking. | | | | | +| Deployment | "DIAL Deployment Highlights" — Menu: Platform > Deployment — URL: https://docs.dialx.ai/platform/deployment-intro | Menu: Tutorials > DevOps > Deployment — links directly to GitHub: https://github.com/epam/ai-dial-helm/tree/main/charts/dial/examples/generic/simple | Two separate "Deployment" entries: one is a conceptual overview on the docs site, the other exits to GitHub with no docs-site landing page. | | | | | +| Observability | "Observability and Monitoring" — Menu: Platform > Observability — URL: https://docs.dialx.ai/platform/observability-intro | Menu: Tutorials > DevOps > Observability — URL: https://docs.dialx.ai/tutorials/devops/observability-config | Same topic in two locations. The first is conceptual (what OTEL is, what Prometheus does). The second is a configuration how-to. No cross-reference. | | | | | +| Analytics | "Analytics" — Menu: Platform > Analytics — URL: https://docs.dialx.ai/platform/realtime-analytics-intro | "Analytics Realtime Configuration" — Menu: Tutorials > DevOps > Configuration > Analytics Realtime Configuration — URL: https://docs.dialx.ai/tutorials/devops/configuration/realtime-analytics-config | Same split as Observability: overview vs setup, two locations. | | | | | +| Admin Panel | "About DIAL Admin" — Menu: Platform > Admin Panel — URL: https://docs.dialx.ai/platform/admin-panel | "Introduction to DIAL Admin Panel" — Menu: Tutorials > Admins > Admin Panel User Guide > Introduction — URL: https://docs.dialx.ai/tutorials/admin/home | Both are introductions to the Admin Panel. One is a feature overview, the other is the start of the user guide. A reader searching "Admin Panel" gets two results. | | | | | +| Interceptors | "Interceptors" — Menu: Platform > Core > Interceptors — URL: https://docs.dialx.ai/platform/core/interceptors | Interceptors section within "DIAL Architecture" — Menu: Platform > Architecture & Concepts > Architecture Highlights — URL: https://docs.dialx.ai/platform/architecture-and-concepts/architecture#interceptors | Two separate treatments of interceptors at different depths, no cross-linking between them. | | | | | + + + +### 1.2 The Platform vs Tutorials split is incoherent + +The site implies: **Platform** = "what DIAL is" (explanation), **Tutorials** = "how to do things." In practice, the boundary is arbitrary: + +* "Access Control in DIAL" (URL: [https://docs.dialx.ai/platform/core/access-control-intro](https://docs.dialx.ai/platform/core/access-control-intro), Menu: `Platform > Core > Access Control`) is a **detailed technical treatment** covering authorization rules, object types, configuration files, and API endpoints — not an "explanation." By [Diátaxis](https://diataxis.fr/) standards, it is a reference with how-to elements. +* "DIAL Deployment Highlights" (URL: [https://docs.dialx.ai/platform/deployment-intro](https://docs.dialx.ai/platform/deployment-intro), Menu: `Platform > Deployment`) is a **lightweight conceptual overview** that immediately links to the Tutorials > DevOps section for actual instructions. +* "Authentication" (URL: [https://docs.dialx.ai/platform/core/auth-intro](https://docs.dialx.ai/platform/core/auth-intro), Menu: `Platform > Architecture & Concepts > Authentication`) lives at a `/platform/core/` URL but is displayed in the Architecture & Concepts sidebar group — the URL contradicts the menu placement. +* The DevOps sidebar entry (Menu: `Tutorials > DevOps`) links directly to GitHub ([https://github.com/epam/ai-dial-helm/tree/main/charts/dial/examples/generic/simple](https://github.com/epam/ai-dial-helm/tree/main/charts/dial/examples/generic/simple)) with no docs-site landing page. + +### 1.3 What the "Tutorials" section actually contains + +The top-level "Tutorials" section (Menu: `Tutorials`, URL: [https://docs.dialx.ai/tutorials/user-guide](https://docs.dialx.ai/tutorials/user-guide)) contains no in-depth, learning-oriented tutorials that teach a developer to build something from scratch with explanation, verification, and learning outcomes. Here is what each page within it actually is: + +**User guides (product manuals):** + +| Page title | Menu path | URL | Actual content type | Page title | Menu path | URL | Actual content type | +| :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | +| "Chat User Guide" | Tutorials > Chat User Guide | https://docs.dialx.ai/tutorials/user-guide | Comprehensive end-user manual for DIAL Chat UI — describes conversations, prompts, agents, marketplace, workspace, files, settings. Not a guided learning experience. | | | | | +| "Mind Map Studio User Guide" | Tutorials > Mind Map Studio | https://docs.dialx.ai/tutorials/mind-map | End-user guide for Mind Map Studio — covers creating mind maps, sources, content editing, and customization. Has brief procedural steps but no learning outcomes or verification. | | | | | +| "Introduction to DIAL Admin Panel" | Tutorials > Admins > Admin Panel User Guide > Introduction | https://docs.dialx.ai/tutorials/admin/home | Admin Panel user guide with sub-pages for Entities, Builders, Assets, Deployments, Access Management, Approvals, and Audit. Comprehensive manual, not a tutorial. | | | | | + + + +**Quickstart pages (minimal step-by-step, no depth):** + +| Page title | Menu path | URL | Actual content type | Page title | Menu path | URL | Actual content type | +| :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | +| "Launch DIAL Chat with a Sample Application" | Tutorials > Developers > Run DIAL Locally > Chat with Application | https://docs.dialx.ai/tutorials/developers/local-run/quick-start-with-application | Quickstart: Prerequisites → download folder from GitHub → docker compose up → "it works." Has numbered steps but: no explanation of what happens during setup, no verification beyond "select Echo Application and type," no "what you learned," no code to write. | | | | | +| "Launch DIAL Chat with an Azure model" | Tutorials > Developers > Run DIAL Locally > Chat with OpenAI Model | https://docs.dialx.ai/tutorials/developers/local-run/quick-start-model | Same pattern as above. | | | | | +| "Launch DIAL Chat with Ollama" | Tutorials > Developers > Run DIAL Locally > Chat with a Self-Hosted Model (Ollama) | https://docs.dialx.ai/tutorials/developers/local-run/quick-start-with-self-hosted-model-ollama | Same pattern. | | | | | +| "Launch DIAL Chat with vLLM" | Tutorials > Developers > Run DIAL Locally > Chat with a Self-Hosted Model (vLLM) | https://docs.dialx.ai/tutorials/developers/local-run/quick-start-with-self-hosted-model-vllm | Same pattern. | | | | | + + + +**Configuration references and how-tos (mislabeled as tutorials):** + +| Page title | Menu path | URL | Actual content type | Page title | Menu path | URL | Actual content type | +| :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | +| "Quick App Configuration Guide" | Tutorials > Developers > Apps Development > How to Configure Quick App | https://docs.dialx.ai/tutorials/developers/apps-development/quick-app-configuration | JSON schema reference: documents fields, types, required flags, and example JSON for Quick App configuration. No guided steps. | | | | | +| "Custom Buttons in Apps" | Tutorials > Developers > Apps Development > Custom Buttons in Apps | https://docs.dialx.ai/tutorials/developers/apps-development/custom-buttons | Reference for button types (Starter, Populate, Action, Checkbox) and their JSON schema. | | | | | +| "How to Enable Apps in DIAL" | Tutorials > Developers > Apps Development > Enable Apps | https://docs.dialx.ai/tutorials/developers/apps-development/enable-app | How-to: explains the process for registering applications in DIAL Core via API or config files. | | | | | +| "DIAL-to-DIAL Adapter" | Tutorials > Developers > Apps Development > Local Development | https://docs.dialx.ai/tutorials/developers/apps-development/adapter-dial | Configuration guide for the adapter (environment variables, config generation script, docker compose). Misleadingly placed as the first page of "Apps Development." | | | | | +| "Custom Content in Chat" | Tutorials > Developers > Chat > Custom Content in Chat | https://docs.dialx.ai/tutorials/developers/chat/chat-objects | API reference for the custom_content field: attachments, stages, markdown, visualizers, plotly. | | | | | +| "Managing Authorization in Complex Application Ecosystems" | Tutorials > Developers > Apps Development > Auth Matrix | https://docs.dialx.ai/tutorials/developers/apps-development/auth-matrix | Explanation of authorization flows in multi-app scenarios. | | | | | +| "Prompt Caching" | Tutorials > Developers > Prompt Caching | https://docs.dialx.ai/tutorials/developers/prompt-caching | Conceptual explanation of how prompt caching works in LLMs and in DIAL. | | | | | + + + +**API how-tos:** + +| Page title | Menu path | URL | Actual content type | Page title | Menu path | URL | Actual content type | +| :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | +| "Publications" | Tutorials > Developers > Working with Resources > Publications | https://docs.dialx.ai/tutorials/developers/work-with-resources/work-with-publications | How-to with API reference: publication workflow, API endpoints, request examples. | | | | | +| "Sharing" | Tutorials > Developers > Working with Resources > Sharing | https://docs.dialx.ai/tutorials/developers/work-with-resources/sharing | How-to for sharing resources via API. | | | | | +| "Notifications" | Tutorials > Developers > Working with Resources > Notifications | https://docs.dialx.ai/tutorials/developers/work-with-resources/notifications | How-to for notification configuration. | | | | | + + + +**Integration guides:** + +| Page title | Menu path | URL | Actual content type | Page title | Menu path | URL | Actual content type | +| :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | +| "Integration between DIAL and Microsoft Copilot" | Tutorials > Developers > Examples of Integrations > Integration with MS Copilot | https://docs.dialx.ai/tutorials/developers/integrations/copilot-to-dial | Integration how-to. | | | | | +| "Integration with MS Excel" | Tutorials > Developers > Examples of Integrations > Integration with MS Excel | https://docs.dialx.ai/tutorials/developers/integrations/ms-excel-addin | Integration how-to. | | | | | +| "Integration with MS Teams" | Tutorials > Developers > Examples of Integrations > Integration with MS Teams | https://docs.dialx.ai/tutorials/developers/integrations/msteams-bot | Integration how-to. | | | | | +| "Integration with n8n" | Tutorials > Developers > Examples of Integrations > Integration with n8n | https://docs.dialx.ai/tutorials/developers/integrations/n8n-integration | Integration how-to. | | | | | + + + +**External link (no docs-site content):** + +| Menu label | Menu path | Destination | Menu label | Menu path | Destination | +| :-- | :-- | :-- | :-- | :-- | :-- | +| "DevOps" | Tutorials > DevOps | Links directly to GitHub: https://github.com/epam/ai-dial-helm/tree/main/charts/dial/examples/generic/simple | | | | + + + +**Summary:** The "Tutorials" section contains 3 user guides, 4 quickstarts, 7 configuration references/how-tos, 3 API how-tos, 4 integration guides, 1 explanation, 1 external link, and 7 admin guide sub-pages. The quickstart pages are the closest thing to tutorials — they have numbered steps leading to "DIAL is running" — but they lack explanation of what's happening, meaningful verification, and learning outcomes. There are no in-depth, guided tutorials that teach a developer to build an application, create an adapter, write an interceptor, or use the API programmatically. + +### 1.4 Excessive sidebar depth + + + +| Path | Depth | URL | Path | Depth | URL | +| :-- | :-- | :-- | :-- | :-- | :-- | +| Tutorials > Developers > Apps Development > Multimodality > DIAL Cookbook > Examples > [page] | 7 levels | https://docs.dialx.ai/tutorials/developers/apps-development/multimodality/dial-cookbook/examples/how_to_call_text_to_text_applications | | | | +| Tutorials > DevOps > Auth & Access Control > Configure IDPs > [specific IDP] | 5 levels | e.g., https://docs.dialx.ai/tutorials/devops/auth-and-access-control/configure-idps/cognito | | | | + +### 1.5 Misleading section labels + + + +| Menu label | Menu path | Expected content | Actual first page | URL | Menu label | Menu path | Expected content | Actual first page | URL | +| :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | +| "Apps Development" | Tutorials > Developers > Apps Development | How to build DIAL apps | "DIAL-to-DIAL Adapter" — a local dev proxy utility, not app development | https://docs.dialx.ai/tutorials/developers/apps-development/adapter-dial | | | | | | +| "Chat" | Tutorials > Developers > Chat | Chat-related tutorials | "Custom Content in Chat" — API reference for custom_content features (attachments, stages, visualizers) | https://docs.dialx.ai/tutorials/developers/chat/chat-objects | | | | | | +| "Working with Resources" | Tutorials > Developers > Working with Resources | Resource management overview | "Publications" — API reference for one specific sub-feature | https://docs.dialx.ai/tutorials/developers/work-with-resources/work-with-publications | | | | | | +| "DevOps" | Tutorials > DevOps | DevOps landing page on docs site | Direct link to GitHub Helm repo | https://github.com/epam/ai-dial-helm/tree/main/charts/dial/examples/generic/simple | | | | | | + +### 1.6 Inconsistent treatment of app types + + + +| App type | Treatment in sidebar | Details | App type | Treatment in sidebar | Details | +| :-- | :-- | :-- | :-- | :-- | :-- | +| Mind Map Studio | Top-level entry under Tutorials | Menu: Tutorials > Mind Map Studio — URL: https://docs.dialx.ai/tutorials/mind-map — same sidebar level as "Developers" and "DevOps" | | | | +| Quick Apps | Single buried page | Menu: Tutorials > Developers > Apps Development > How to Configure Quick App — URL: https://docs.dialx.ai/tutorials/developers/apps-development/quick-app-configuration — this is a JSON schema reference, not a guide for building Quick Apps | | | | +| Code Apps | No section anywhere | Mentioned in the Architecture page (URL: https://docs.dialx.ai/platform/architecture-and-concepts/architecture#agent-builders) and in the Chat User Guide, but has no dedicated developer documentation, no tutorial, and no menu entry | | | | + +### 1.7 No "what's next" navigation between pages + +Pages end without linking to logical next steps: + +| Page | URL | What a reader would logically do next | What the page provides | Page | URL | What a reader would logically do next | What the page provides | +| :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | +| Quick Start | https://docs.dialx.ai/quick-start | Try a real model, or build an app | Links to previous/next page in sidebar order. No "next steps" section. | | | | | +| "Launch DIAL Chat with a Sample Application" | https://docs.dialx.ai/tutorials/developers/local-run/quick-start-with-application | Call the API, build a custom app | No "next steps" section. Previous/Next links go to "Mind Map Studio" and "Chat with OpenAI Model." | | | | | +| "How to Enable Apps in DIAL" | https://docs.dialx.ai/tutorials/developers/apps-development/enable-app | Build an app to register | Links to "Local Development" and "How to Configure Quick App." No tutorial link. | | | | | + + + +### 1.8 Quick Start appears in two places + + + +| Entry | Menu path | URL | Scope | Entry | Menu path | URL | Scope | +| :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | +| Top-level Quick Start | Quick Start (nav bar) | https://docs.dialx.ai/quick-start | Docker Compose echo application only | | | | | +| Run DIAL Locally | Tutorials > Developers > Run DIAL Locally | https://docs.dialx.ai/tutorials/developers/local-run/quick-start-with-application | Four variations (application, Azure model, Ollama, vLLM) | | | | | + +The Quick Start page does not link to the "Run DIAL Locally" section. A reader who finishes Quick Start has no path to the more detailed local setups. + +### 1.9 URL paths vs menu paths don't match + + + +| URL | Menu placement | Problem | URL | Menu placement | Problem | +| :-- | :-- | :-- | :-- | :-- | :-- | +| https://docs.dialx.ai/platform/core/auth-intro | Platform > Architecture & Concepts > Authentication | URL says core/, menu says Architecture & Concepts | | | | +| https://docs.dialx.ai/legal-and-compliance | Platform > Compliance and Legal Q&A | URL has no platform/ prefix but page appears under Platform | | | | +| https://docs.dialx.ai/video%20demos/dial-product-overview | Demos | URL contains a space (video demos), a web anti-pattern | | | | +| https://docs.dialx.ai/tutorials/developers/apps-development/multimodality/dial-cookbook/examples/how_to_call_text_to_text_applications | Tutorials > Developers > Apps Development > Examples | URL contains multimodality/dial-cookbook/ which does not appear in the menu | | | | + +* * * + +## 2\. Structural gaps (site-wide) + + + +| Gap | Evidence | Impact | Gap | Evidence | Impact | +| :-- | :-- | :-- | :-- | :-- | :-- | +| Docs site redirects to GitHub READMEs as authoritative source | Configuration guide (URL: https://docs.dialx.ai/tutorials/devops/configuration/configuration-guide) says "Refer to the AI DIAL Core" with link to GitHub for every component; Quick Start asks users to download from GitHub; DevOps sidebar entry is a GitHub link | Fragmented UX, broken search, invisible staleness | | | | +| API Reference hosted off-site | API docs at https://dialx.ai/dial_api — not part of docs.dialx.ai navigation or search | Reader loses context | | | | +| No versioned documentation | DIAL ships biweekly with semver; docs have no version selector | Readers can't find docs matching their version | | | | +| No version compatibility matrix | Core, SDK, adapters, Chat, Admin, Helm chart version independently; no table shows which work together | Non-Helm deployments are guesswork | | | | +| No changelog on docs site | Release notes only on GitHub (e.g., https://github.com/epam/ai-dial-core/releases) | Product velocity invisible to docs readers | | | | +| Orphaned pages | "Handling High Loads" (URL: https://docs.dialx.ai/platform/high-load-performance), "DIAL Evolution" (URL: https://docs.dialx.ai/platform/history), "Compliance and Legal Q&A" (URL: https://docs.dialx.ai/legal-and-compliance, menu mismatch) — none connected to a narrative or learning path | Feel like blog posts, not documentation | | | | +| Demos conflate showcases with tutorials | Demos section (Menu: Demos, URL: https://docs.dialx.ai/video%20demos/dial-product-overview) mixes short capability flyovers with longer coding walkthroughs | Neither audience finds what they expect | | | | +| No community surface | No contribution guide on-site (only on GitHub: https://github.com/epam/ai-dial/blob/main/CONTRIBUTING.md), no community extension list, no forum links, no marketplace submission process | Open-source project looks proprietary | | | | + +* * * + +## 3\. Onboarding and positioning + + + +| Gap | Evidence | Impact | Gap | Evidence | Impact | +| :-- | :-- | :-- | :-- | :-- | :-- | +| Landing page is a flat list of 9 sections | Home page (URL: https://docs.dialx.ai/) lists Quick Start, Architecture, DIAL Admin, Run DIAL Locally, Helm Deployment, Configuration, User Manual, and Repositories in a flat layout | No persona routing, no "start here" | | | | +| "What is DIAL" opens with the acronym | "What is DIAL" page (URL: https://docs.dialx.ai/platform/architecture-and-concepts/vision) starts with "DIAL is an acronym for Deterministic Integrator of Applications and Language Models" | Readers don't learn what problem DIAL solves | | | | +| No comparison pages | Absent from site | Evaluators can't answer "DIAL vs X" | | | | +| No Quick Start per persona | Single Docker Compose path (URL: https://docs.dialx.ai/quick-start) targets the App Developer persona only | DevOps, Admin, Evaluator, and End user personas have no entry point | | | | +| No environment prerequisites | Quick Start (URL: https://docs.dialx.ai/quick-start) says "Docker engine (Docker Compose Version 2.20.0 +) installed on your machine" and nothing else. "Launch DIAL Chat with a Sample Application" (URL: https://docs.dialx.ai/tutorials/developers/local-run/quick-start-with-application) additionally requires Python 3.8+ and pip but doesn't address WSL2, Rosetta for ARM, Docker Desktop resource allocation, or tested OS distributions | First-time users fail silently and leave | | | | + +* * * + +## 4\. Content depth + + + +| Gap | Evidence | Impact | Gap | Evidence | Impact | +| :-- | :-- | :-- | :-- | :-- | :-- | +| No in-depth tutorials exist | As documented in §1.3 above: the "Tutorials" section contains user guides, quickstarts, references, how-tos, and external links. The quickstart pages (e.g., URL: https://docs.dialx.ai/tutorials/developers/local-run/quick-start-with-application) have numbered steps but no explanation of what happens, no meaningful verification, and no learning outcomes. There are no tutorials that teach a developer to build an application, create an adapter, write an interceptor, or use the API programmatically. | A developer who wants to learn DIAL beyond "run docker compose" has nowhere to go. This is the #1 content gap. | | | | +| No "getting started with the API" guide | After the Quick Start, the next developer task is calling the API. There is no bridge between "DIAL is running" and "I can programmatically use it." The API reference is off-site (https://dialx.ai/dial_api) with no on-site usage guide covering chat completions, streaming, file upload, or conversation management. | The most important developer on-ramp doesn't exist | | | | +| DIAL Apps ecosystem under-documented | Custom Apps: only the quickstart (URL: https://docs.dialx.ai/tutorials/developers/local-run/quick-start-with-application) shows a minimal echo app. Quick Apps: only a JSON schema reference exists (URL: https://docs.dialx.ai/tutorials/developers/apps-development/quick-app-configuration). Code Apps: mentioned on the Architecture page (URL: https://docs.dialx.ai/platform/architecture-and-concepts/architecture#agent-builders) and Chat User Guide but has no developer section. Mind Map Studio: end-user guide only (URL: https://docs.dialx.ai/tutorials/mind-map). No unified story of what each app type is, when to use which, or how they relate. | Primary value surface invisible to new users | | | | +| Tool Sets undocumented | Tool Sets are referenced in the Architecture page (URL: https://docs.dialx.ai/platform/architecture-and-concepts/architecture#mcp-servers) and in the Quick App Configuration Guide (URL: https://docs.dialx.ai/tutorials/developers/apps-development/quick-app-configuration) under the tools section. No standalone reference, authoring guide, or examples exist. | Agent-building unreachable | | | | +| Videos not self-sufficient | Videos are referenced from pages like the Quick Start (URL: https://docs.dialx.ai/quick-start) and "Launch DIAL Chat with a Sample Application" (URL: https://docs.dialx.ai/tutorials/developers/local-run/quick-start-with-application). These screencasts show code and configuration on-screen but the associated pages don't include the shown code, sample data, or config files. | Can't reproduce what you see | | | | +| Naming chaos across app creation surfaces | Architecture page (URL: https://docs.dialx.ai/platform/architecture-and-concepts/architecture#agent-builders) says "Agent Builders (technical name application runners)." Chat page architecture section uses "Application builders." Admin sidebar (URL: https://docs.dialx.ai/tutorials/admin/builders-application-runners) uses "Builders." SDK documentation (https://github.com/epam/ai-dial-sdk) uses neither term consistently. | Four labels for one concept; readers believe these are different things | | | | +| No OpenAI API compatibility matrix | "What is DIAL" page (URL: https://docs.dialx.ai/platform/architecture-and-concepts/vision) and Architecture page both describe the API as "OpenAI-compatible" but never specify which OpenAI API version, which endpoints are supported, what extensions DIAL adds (custom_content, stages, attachments), or what is unsupported. | Developers migrating from OpenAI discover incompatibilities at runtime | | | | +| SDK documentation absent from docs site | SDK only documented in GitHub README (https://github.com/epam/ai-dial-sdk). No on-site reference, tutorial, or getting-started guide. | Developers must leave the site | | | | +| No production hardening guide | "Handling High Loads" (URL: https://docs.dialx.ai/platform/high-load-performance) is an isolated page under Platform, not connected to HA, secrets, backup, upgrade, or cost control. No linear "how to go to production" narrative. | Can't answer "is this production-ready?" | | | | +| Observability stops at "use OTEL" | "Observability and Monitoring" (URL: https://docs.dialx.ai/platform/observability-intro) explains what OTEL and Prometheus are but provides no setup instructions, no tracing configuration, no provider-specific guides (Datadog, CloudWatch, etc.), and no alerting rules. | DevOps must figure out the last mile alone | | | | +| Networking/firewall docs promised but undelivered | "Launch DIAL Chat with a Sample Application" (URL: https://docs.dialx.ai/tutorials/developers/local-run/quick-start-with-application) explicitly states: "Deploying and distributing these applications for production purposes will require additional configurations that guarantee secure access to the application endpoints through the implementation of firewalls and other network security settings to prevent unauthorized intrusion." This documentation is never provided anywhere on the site. | Gap acknowledged in existing docs and left unfilled | | | | +| No dependency configuration | Redis and blob storage are required dependencies mentioned in the Architecture page (URL: https://docs.dialx.ai/platform/architecture-and-concepts/architecture) and Deployment page (URL: https://docs.dialx.ai/platform/deployment-intro). No guidance exists for: Redis version, cluster vs standalone, Sentinel, memory sizing; blob storage provider selection (S3 vs GCS vs Azure Blob), IAM configuration, bucket structure. | Day-one deployment decisions undocumented | | | | +| No error reference | DIAL Core returns HTTP error codes. No documentation explains what specific errors mean or how to fix them. No error code catalog exists on the site. | Developers read source code to debug | | | | +| No testing guidance | DIAL SDK (https://github.com/epam/ai-dial-sdk) has testing utilities, but docs site has no content on unit testing custom apps, integration testing against DIAL Core, or mocking patterns. | Developers ship untested apps | | | | +| Cookbook buried and stale | Located at 7-level depth (URL: https://docs.dialx.ai/tutorials/developers/apps-development/multimodality/dial-cookbook/examples/how_to_call_text_to_text_applications). The page header warns about using openai-python-sdk — the code examples pin dependencies over 2 years old. | Signals abandonware | | | | +| No reference architectures | No diagrams or guides for common patterns: enterprise RAG, eval-driven dev, multi-tenant deployment, hybrid cloud. | Architects design from scratch | | | | +| No troubleshooting hub | No centralized FAQ or troubleshooting page. Error resolution scattered across component READMEs on GitHub. | Support load | | | | +| No RAG explanation | DIAL RAG is a major repo (https://github.com/epam/ai-dial-rag). RAG is mentioned throughout the Architecture page. No page explains DIAL's approach to RAG vs building it with raw frameworks. | Developers can't evaluate the RAG offering | | | | +| No file management how-to | Architecture page (URL: https://docs.dialx.ai/platform/architecture-and-concepts/architecture#other-apis) references a File Storage Management API. No developer guide covers upload, list, or retrieve — the most common first task after API access. | Developers reverse-engineer from API spec | | | | + +* * * + +## 5\. Terminology and concepts + + + +| Gap | Evidence | Impact | Gap | Evidence | Impact | +| :-- | :-- | :-- | :-- | :-- | :-- | +| No glossary | No single page on the site defines the core terms: Application, Adapter, Tool Set, Quick App, Code App, Agent Builder, Interceptor, Overlay, Deployment, MCP Server, Publication | Readers conflate concepts; different pages use different terms for the same thing | | | | +| Four labels for one concept | "Agent Builders" at https://docs.dialx.ai/platform/architecture-and-concepts/architecture#agent-builders; "application runners" in parenthetical on the same page; "Application builders" on https://docs.dialx.ai/platform/chat/about-chat; "Builders" in Admin sidebar at https://docs.dialx.ai/tutorials/admin/builders-application-runners | Readers believe these are different things | | | | +| Deprecated concepts surfaced without marking | Assistant repo (https://github.com/epam/ai-dial-assistant) is archived with a notice, but docs site doesn't flag it. Addon protocol is legacy. | Dead ends | | | | +| Inconsistent capitalization | "DIAL Core" vs "the backend" vs "dial core" appear across different pages | Breaks search, signals inattention | | | | + +* * * + +## 6\. Sustainability + + + +| Gap | Evidence | Impact | Gap | Evidence | Impact | +| :-- | :-- | :-- | :-- | :-- | :-- | +| No freshness enforcement | Cookbook examples (URL: https://docs.dialx.ai/tutorials/developers/apps-development/multimodality/dial-cookbook/examples/how_to_call_text_to_text_applications) use 2+ year old dependencies | Erodes trust | | | | +| No page ownership | No owner field in page frontmatter | No accountability for decay | | | | +| No CI quality gates | Broken links and outdated terminology merge without checks | Quality regresses between releases | | | | +| No video/article pairing enforced | Some pages reference videos as primary learning material without providing equivalent written content with code and data | Can't reproduce content | | | | + +* * * + +## 7\. Gap severity ranking + + + +| Rank | Gap | Category | Key evidence URL | Rank | Gap | Category | Key evidence URL | +| :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | +| 1 | No in-depth tutorials exist (quickstarts are not tutorials) | Content | https://docs.dialx.ai/tutorials/developers/local-run/quick-start-with-application | | | | | +| 2 | No "getting started with the API" guide | Content | https://dialx.ai/dial_api (off-site, no on-site bridge) | | | | | +| 3 | DIAL Apps ecosystem under-documented / asymmetric | Content | https://docs.dialx.ai/tutorials/developers/apps-development/quick-app-configuration | | | | | +| 4 | Tool Sets undocumented | Content | https://docs.dialx.ai/platform/architecture-and-concepts/architecture#mcp-servers | | | | | +| 5 | Seven duplicate-title pairs in navigation | Navigation | §1.1 table above | | | | | +| 6 | Platform/Tutorials split incoherent | Navigation | §1.2 above | | | | | +| 7 | "Tutorials" section entirely mislabeled | Navigation | §1.3 above | | | | | +| 8 | GitHub redirects as authoritative source | Structural | https://docs.dialx.ai/tutorials/devops/configuration/configuration-guide | | | | | +| 9 | Videos not self-sufficient | Content | https://docs.dialx.ai/quick-start | | | | | +| 10 | No OpenAI API compatibility matrix | Content | https://docs.dialx.ai/platform/architecture-and-concepts/vision | | | | | +| 11 | Four labels for "Agent Builder" concept | Terminology | https://docs.dialx.ai/platform/architecture-and-concepts/architecture#agent-builders | | | | | +| 12 | No glossary | Terminology | (absent) | | | | | +| 13 | Observability stops at "use OTEL" | Content | https://docs.dialx.ai/platform/observability-intro | | | | | +| 14 | Networking/firewall promised, not delivered | Content | https://docs.dialx.ai/tutorials/developers/local-run/quick-start-with-application | | | | | +| 15 | No dependency config (Redis, blob storage) | Content | https://docs.dialx.ai/platform/deployment-intro | | | | | +| 16 | No error reference | Content | (absent) | | | | | +| 17 | No version compatibility matrix | Structural | (absent) | | | | | +| 18 | No production hardening guide | Content | https://docs.dialx.ai/platform/high-load-performance | | | | | +| 19 | Cookbook buried and stale | Content + Nav | https://docs.dialx.ai/tutorials/developers/apps-development/multimodality/dial-cookbook/examples/how_to_call_text_to_text_applications | | | | | +| 20 | No community surface | Structural | https://github.com/epam/ai-dial/blob/main/CONTRIBUTING.md (GitHub only) | | | | | +| 21 | No testing guidance for custom apps | Content | https://github.com/epam/ai-dial-sdk (GitHub only) | | | | | +| 22 | No environment prerequisites | Content | https://docs.dialx.ai/quick-start | | | | | +| 23 | No "what's next" navigation | Navigation | https://docs.dialx.ai/quick-start | | | | | +| 24 | Excessive sidebar depth (up to 7 levels) | Navigation | https://docs.dialx.ai/tutorials/developers/apps-development/multimodality/dial-cookbook/examples/how_to_call_text_to_text_applications | | | | | +| 25 | No changelog on docs site | Structural | (absent — only on GitHub) | | | | | + +[Filter table data](#)[Create a pivot table](#)[Create a chart from data series](#) + +[Configure buttons visibility](/users/tfac-settings.action) \ No newline at end of file diff --git a/docs-planning/glossary.md b/docs-planning/glossary.md new file mode 100644 index 000000000..88e3212cb --- /dev/null +++ b/docs-planning/glossary.md @@ -0,0 +1,445 @@ +--- +title: "Glossary" +type: reference +persona: all +component: platform +last_verified: 2026-04-27 +owner: "@dial-docs-team" +--- + +# Glossary + +This glossary defines the canonical terms used throughout DIAL documentation. Each term has one definition. When a term appears for the first time on a page, it should link here. + +If you encounter a synonym or variant not listed below, use the canonical term instead. See also the [concept map](#concept-map) at the end of this page for a visual overview of how terms relate. + +--- + +## A + +### Adapter + +A DIAL application that translates an external AI provider's API into the DIAL [Unified API](#unified-api). Adapters allow DIAL to communicate with models from providers such as Azure OpenAI, AWS Bedrock, and Google Vertex AI through a single, consistent interface. Each adapter runs in its own container. + +DIAL ships three adapters: the [OpenAI Adapter](https://github.com/epam/ai-dial-adapter-openai), [Bedrock Adapter](https://github.com/epam/ai-dial-adapter-bedrock), and [Vertex AI Adapter](https://github.com/epam/ai-dial-adapter-vertexai). Organizations can build custom adapters using the [DIAL SDK](#dial-sdk). + +Not to be confused with: [Application](#application) (generic), [Interceptor](#interceptor). + +### Addon + +**Deprecated.** A legacy extension mechanism based on the ChatGPT plugin protocol. The Addon concept has been abandoned and the [DIAL Assistant](#assistant-deprecated) repository that implemented it is archived. Do not use in new documentation. Modern equivalents are [Application](#application), [Interceptor](#interceptor), and [Toolset](#toolset). + +### Agent + +An umbrella term for any conversational entity a user can interact with in DIAL — including language models and [applications](#application). Agents are listed in the [Marketplace](#marketplace) and can be used as building blocks in multi-agent workflows. + +See also: [Agent Builder](#agent-builder). + +### Agent Builder + +The canonical name for an application runner — a component that enables end users to create customized AI applications from predefined templates without writing code. Agent builders process parameters of a specific [application type](#application-type) and expose a UI wizard for configuration. + +Standard agent builders included in DIAL: [Quick Apps](#quick-app), [Code Apps](#code-app), and [Mind Map Studio](#mind-map-studio). + +Also known as: "application runner" (technical name in source code), "application builder" (UI label in DIAL Chat), "builder" (DIAL Admin sidebar label). These all refer to the same concept. Use **agent builder** in documentation. + +### Analytics Realtime + +A DIAL component that processes chat completion logs to extract usage insights and operational metrics without storing sensitive user information. It applies embedding algorithms, clustering, and lightweight models to analyze conversation patterns in real time. Results are stored in time-series databases such as InfluxDB and visualized through platforms such as Grafana. + +Repository: [ai-dial-analytics-realtime](https://github.com/epam/ai-dial-analytics-realtime). + +### App Builder + +A Python-based utility that downloads source code from DIAL file storage and prepares files to build container images for [Code Apps](#code-app). Not to be confused with [agent builder](#agent-builder), which is the no-code UI concept. + +Repository: [ai-dial-app-builder](https://github.com/epam/ai-dial-app-builder). + +### Application + +A first-class extension in DIAL that exposes chat completion or embedding endpoints via the [DIAL SDK](#dial-sdk) and conforms to the [Unified API](#unified-api). Applications can be invoked directly by users or composed as building blocks in multi-agent workflows. + +Applications come in two flavors: [schema-rich applications](#schema-rich-application) (defined by a JSON schema and configurable via API or UI) and applications without schemas (logic embedded in application code and container). + +Not to be confused with: [Adapter](#adapter) (translates external provider APIs), [Interceptor](#interceptor) (middleware that modifies requests/responses), or the deprecated [Addon](#addon). + +### Application Type + +A schema-rich template that defines the structure, properties, endpoints, and optional UI wizard for a category of [applications](#application). Application types allow end users to create instances of a specific kind of application without writing code. + +Standard application types included in DIAL: [Quick App](#quick-app), [Code App](#code-app), and [Mind Map](#mind-map-studio). + +Organizations can define custom application types and register them in [DIAL Core](#dial-core). + +### Assistant (deprecated) + +**Deprecated.** The DIAL Assistant was an implementation of the ChatGPT plugin protocol. The repository ([ai-dial-assistant](https://github.com/epam/ai-dial-assistant)) is archived. Do not use in new documentation. The modern replacement is [Application](#application). + +### Attachment + +A file included in or produced by a chat completion request or response. Applications built with the [DIAL SDK](#dial-sdk) can accept and return attachments of various types. Attachments are stored in the DIAL [file storage](#file-storage). + +## C + +### Code App + +An [application type](#application-type) that allows users to develop, deploy, and run Python applications directly in the [DIAL Chat](#dial-chat) UI. Code Apps are useful for rapid prototyping and proof-of-concept work. They run in a secure, isolated environment managed by the DIAL platform and do not have internet access. + +See also: [Quick App](#quick-app), [Mind Map Studio](#mind-map-studio). + +### Conversation + +A dialogue between a user and an [agent](#agent) in DIAL. Each conversation maintains its own context — messages in one conversation are not shared with another. Conversations are stored server-side and accessible from any device. Conversations can be [shared](#sharing), [published](#publication), [replayed](#replay), or [played back](#playback). + +### Custom UI + +A fully custom user interface that replaces the standard [DIAL Chat](#dial-chat) conversational UI during interactions with a specific [application](#application). Custom UIs can implement non-conversational interfaces. For example, [Mind Map Studio](#mind-map-studio) uses a custom UI to render interactive knowledge graphs. + +Custom UIs are defined through the `applicationTypeViewerUrl` property in an [application type](#application-type) schema. + +## D + +### Deployment (configuration sense) + +A named endpoint in [DIAL Core](#dial-core) configuration that exposes a model or [application](#application) to clients. A deployment defines the connection between a logical name (used in API requests) and the actual service endpoint, along with associated [upstreams](#upstream), [interceptors](#interceptor), and access control settings. + +Not to be confused with: Helm deployment or Kubernetes deployment (infrastructure sense). When the context is ambiguous, qualify as "DIAL deployment" or "Kubernetes deployment." + +### DIAL + +**D**eterministic **I**ntegrator of **A**pplications and **L**anguage Models. An open-source, enterprise-grade AI platform by EPAM that provides a unified gateway to language models, application orchestration, access control, observability, and collaboration features. Always written in all-caps: **DIAL**, never "Dial" or "dial." + +Licensed under [Apache License, Version 2.0](https://github.com/epam/ai-dial/blob/main/LICENSE). + +### DIAL Admin + +The administration interface for the DIAL platform, consisting of a frontend ([ai-dial-admin-frontend](https://github.com/epam/ai-dial-admin-frontend)) and backend ([ai-dial-admin-backend](https://github.com/epam/ai-dial-admin-backend)). DIAL Admin provides system administrators with a UI to manage models, applications, deployments, interceptors, MCP servers, access control, publications, and system monitoring. + +### DIAL Chat + +The default web-based user interface for the DIAL platform. DIAL Chat provides a conversational interface, the [Marketplace](#marketplace), no-code [agent builders](#agent-builder), [collaboration](#sharing) features, and extensibility through [custom UIs](#custom-ui) and [visualizers](#visualizer). + +Repository: [ai-dial-chat](https://github.com/epam/ai-dial-chat). Not to be confused with: [DIAL Overlay](#dial-overlay) (an embeddable subset of DIAL Chat). + +### DIAL Core + +The central Java service and **only mandatory component** of the DIAL platform. DIAL Core exposes the [Unified API](#unified-api) and provides the LLM gateway, [load balancing](#load-balancer), authentication, access control, [interceptor](#interceptor) orchestration, file storage, cost management, and observability. + +DIAL Core is headless — it functions without a UI. All other DIAL components are optional. + +Repository: [ai-dial-core](https://github.com/epam/ai-dial-core). Not to be confused with: "the backend" (too vague — use **DIAL Core**). + +### DIAL Helm + +The Helm chart repository for deploying DIAL on Kubernetes. Stable assemblies that combine compatible versions of all DIAL components are published here. + +Repository: [ai-dial-helm](https://github.com/epam/ai-dial-helm). + +### DIAL Overlay + +A library that allows embedding [DIAL Chat](#dial-chat) in external web applications via an iframe event-based protocol. Overlay enables third-party applications to integrate DIAL conversational capabilities without building a custom UI from scratch. + +Documentation: [DIAL Chat Overlay](https://github.com/epam/ai-dial-chat/blob/development/libs/overlay/README.md). + +### DIAL RAG + +A retrieval-augmented generation component for the DIAL platform. DIAL RAG provides document ingestion, chunking, embedding, and retrieval capabilities that can be used as building blocks in RAG applications. + +Repository: [ai-dial-rag](https://github.com/epam/ai-dial-rag). See also: [DIAL RAG Eval](#dial-rag-eval). + +### DIAL RAG Eval + +A library for evaluating RAG pipelines, providing both retrieval metrics (precision, recall, NDCG) and generation metrics (faithfulness, relevance). + +Repository: [ai-dial-rag-eval](https://github.com/epam/ai-dial-rag-eval). + +### DIAL SDK + +A Python framework (Python ≥3.11) for creating [applications](#application) and [adapters](#adapter) for DIAL. Applications and adapters built with the SDK are fully compatible with the [Unified API](#unified-api). + +Repository: [ai-dial-sdk](https://github.com/epam/ai-dial-sdk). See also: [Interceptors SDK](#interceptors-sdk). + +### DIAL-to-DIAL Adapter + +A development utility that allows a local DIAL instance to proxy requests to a remote [DIAL Core](#dial-core). Useful during application development when you want to test against a remote environment without deploying locally. + +Repository: [ai-dial-adapter-dial](https://github.com/epam/ai-dial-adapter-dial). + +### Dynamic Settings + +DIAL Core configuration files that can be reloaded at runtime without restarting the service. Dynamic settings define [deployments](#deployment-configuration-sense), [interceptors](#interceptor), [roles](#role), [routes](#route), API keys, and other operational parameters. + +Administrators can trigger a reload via the DIAL Core API or through [DIAL Admin](#dial-admin). + +## E + +### Embedding API + +A DIAL API endpoint that provides unified access to embedding models from any supported provider. The Embedding API normalizes provider-specific embedding interfaces into a single protocol, supporting asymmetric models, instruct models, and multi-modal embeddings. + +## F + +### File Storage + +The persistent storage layer for DIAL, used to store conversations, prompts, applications, user files, and other [resources](#resource). DIAL supports cloud blob storage (AWS S3, Google Cloud Storage, Azure Blob Storage) or a local file system. Redis is deployed on top as an in-memory cache. + +DIAL does not require a centralized database — file storage and Redis are the only persistence dependencies. + +## I + +### Interceptor + +Middleware that modifies requests and/or responses flowing through [DIAL Core](#dial-core). Interceptors are inserted into [deployments](#deployment-configuration-sense) and execute before or after chat completion requests reach the target model or application. + +Interceptors fall into three categories: **pre-interceptors** (modify the incoming request), **post-interceptors** (modify the outgoing response), and **generic interceptors** (modify both). Common use cases include PII detection and redaction, content filtering, prompt injection detection, and compliance enforcement. + +When multiple interceptors are configured, they execute in sequence: global interceptors first, then application-type interceptors, then local interceptors. The response travels back through the chain in reverse order. + +Not to be confused with: [Adapter](#adapter) (translates external APIs), [Application](#application) (business logic). + +### Interceptors SDK + +A Python framework for creating [interceptors](#interceptor) for DIAL chat completion and embedding models. + +Repository: [ai-dial-interceptors-sdk](https://github.com/epam/ai-dial-interceptors-sdk). Not to be confused with: [DIAL SDK](#dial-sdk) (for applications and adapters). + +## L + +### Load Balancer + +A feature of [DIAL Core](#dial-core) that distributes requests across model [upstreams](#upstream). The load balancer supports weighted distribution across deployments, regions, and cloud subscriptions, and can prioritize provisioned throughput unit (PTU) deployments over pay-per-token options. + +Configuration is defined through `upstream` parameters in [dynamic settings](#dynamic-settings) and can be adjusted without redeployment. + +## M + +### Marketplace + +The single-entry point in [DIAL Chat](#dial-chat) for browsing and accessing all available [agents](#agent), [applications](#application), models, and [toolsets](#toolset). The Marketplace respects [role](#role)-based access control, so each user sees only the resources available to their permissions. Resources can be filtered by type, topic, and source. + +The Marketplace also serves as a collaboration hub where users can [share](#sharing) and [publish](#publication) their creations. + +### MCP Server + +A service conforming to the [Model Context Protocol](https://modelcontextprotocol.io/) (MCP) that extends AI application capabilities with external tools and data sources. DIAL supports two integration approaches: connecting to external MCP servers hosted outside the platform, and deploying custom MCP servers as Docker containers through [DIAL Admin](#dial-admin). + +MCP servers are used through [toolsets](#toolset) in [Quick Apps](#quick-app) and other applications. + +### Mind Map Studio + +An [application type](#application-type) that enables users to explore information through interactive knowledge graphs built from documents, URLs, and other data sources. Mind Map Studio uses a [custom UI](#custom-ui) that replaces the standard chat interface with a visual, interactive graph. + +End-user documentation belongs in the Chat User Guide. Developer-facing documentation (authoring workflows, export, extension points) belongs in Building with DIAL. + +### Multimodality + +The ability of DIAL applications and models to handle non-textual content alongside text. This includes image-to-text, text-to-image, text-to-video, image-to-video, and file transfers. Multimodality is supported through the [Unified API](#unified-api) and rendered in [DIAL Chat](#dial-chat) through [attachments](#attachment) and [visualizers](#visualizer). + +### My Workspace + +The personal area in [DIAL Chat](#dial-chat) where a user manages their bookmarked [agents](#agent), [toolsets](#toolset), and [agent builders](#agent-builder). Resources added from the [Marketplace](#marketplace) appear here. My Workspace also provides access to no-code application creation tools. + +## P + +### Parameterized Replay + +A variant of [replay](#replay) where specific parts of conversation messages are replaced with variables. When the conversation is replayed or shared, users are prompted to provide their own values for those variables, creating a personalized experience from a shared template. + +### Per-Request Key + +A short-lived API key generated by [DIAL Core](#dial-core) for the duration of a single request. Per-request keys manage file access permissions for applications, enable distributed tracing, and attribute costs to the originating user or application. They are automatically invalidated when the request completes. + +### Playback + +A mode in [DIAL Chat](#dial-chat) that reproduces a conversation exactly as it occurred, without re-submitting prompts to any model. Playback simulates the conversation like a recording. Not to be confused with [replay](#replay), which re-submits prompts and may produce different results. + +### Private Space + +The logical storage area in DIAL where [resources](#resource) are accessible only to their owner and users with whom they have been explicitly [shared](#sharing). By default, all user-created resources reside in private space. + +See also: [Public Space](#public-space). + +### Prompt + +A reusable text template stored in DIAL that users can invoke in conversations. Prompts can contain [variables](#parameterized-replay) (e.g., `{{country|Japan}}`) with optional default values. Prompts can be [shared](#sharing) or [published](#publication). + +### Publication + +The workflow through which a user submits [resources](#resource) from their [private space](#private-space) to the [public space](#public-space), making them available to all authenticated users or a restricted audience. All publication requests require administrator approval. Published [applications](#application) and [toolsets](#toolset) appear in the [Marketplace](#marketplace). + +Not to be confused with: [Sharing](#sharing) (user-to-user, no approval required). + +### Public Space + +The logical storage area in DIAL where [resources](#resource) are accessible to all authenticated users by default. Subfolders within public space can have access rules that restrict visibility to specific [roles](#role). Resources enter public space through [publication](#publication) or direct configuration by administrators. + +See also: [Private Space](#private-space). + +## Q + +### Quick App + +An [application type](#application-type) and no-code orchestrator, conceptually similar to OpenAI's GPTs, that simplifies the creation of multi-agent workflows. Quick Apps can use [agents](#agent), [toolsets](#toolset), REST APIs, language models, and other DIAL resources as building blocks. Configuration is defined through a JSON schema and can be managed via API or the [DIAL Chat](#dial-chat) UI wizard. + +Common use cases: RAG-like applications with predefined sources, applications that call external APIs via [MCP toolsets](#toolset), and multi-step workflows combining multiple agents. + +See also: [Quick App 2.0](#quick-app-20), [Code App](#code-app), [Mind Map Studio](#mind-map-studio). + +### Quick App 2.0 + +An evolution of [Quick Apps](#quick-app) that provides a composer for building applications from reusable components. Quick App 2.0 enables task-oriented workflows combining DIAL [agents](#agent) and external integrations ([MCP servers](#mcp-server)), with any AI model that supports tool use acting as an orchestrator. + +## R + +### Replay + +A feature in [DIAL Chat](#dial-chat) that reproduces a conversation by re-submitting the original prompts, optionally with different settings such as a different model or temperature. The replayed conversation appears as a new conversation tagged `[Replay]`. Useful for comparing model responses to identical inputs. + +Not to be confused with: [Playback](#playback) (exact reproduction without model calls). + +### Resource + +Any object managed by [DIAL Core](#dial-core): applications, conversations, prompts, files, and toolsets. Resources can reside in [private space](#private-space) or [public space](#public-space) and are subject to access control rules. Resources can be [shared](#sharing) and [published](#publication). + +### Role + +A named set of permissions and limits assigned to JWTs or API keys in [DIAL Core](#dial-core). Roles determine which [resources](#resource) a subject can access, enforce usage limits (token rate limits, cost limits), control access to system features (e.g., the admin console), and configure sharing limits. + +Roles are defined in [dynamic settings](#dynamic-settings) or through [DIAL Admin](#dial-admin). + +### Route + +A configuration entry in [DIAL Core](#dial-core) that maps incoming request patterns to specific [deployments](#deployment-configuration-sense). Routes provide flexible request routing and can be used to implement patterns such as A/B testing, failover, or cost-capped routing. + +## S + +### Schema-Rich Application + +An [application](#application) whose structure is defined by a JSON schema conforming to the DIAL Core [meta schema](https://github.com/epam/ai-dial-core/blob/development/config/src/main/resources/custom-application-schemas/schema.json). Schema-rich applications can be created and configured through the [DIAL Core API](https://dialx.ai/dial_api) or UI wizards, and their properties can be modified without redeploying containers. + +Contrast with: applications without schemas, where business logic properties are embedded in application code and cannot be changed via API. + +### Sharing + +A collaboration feature that allows a resource owner to grant access to specific users via a sharing link, without [publishing](#publication) the resource to the entire organization. Shared resources remain in the owner's [private space](#private-space). The owner retains control and can revoke access at any time. Sharing links can optionally grant editing and re-sharing permissions. + +Not to be confused with: [Publication](#publication) (organization-wide, requires admin approval). + +### Stage + +A structured step in an AI-generated response that shows the intermediate reasoning or actions an [agent](#agent) took to produce its final output. Stages are rendered in [DIAL Chat](#dial-chat) as expandable sections within the conversation, providing transparency into multi-step or agentic workflows. + +Stages are returned by applications through the `stages` field in the [Unified API](#unified-api) response. + +### Starter Button + +A predefined prompt button displayed at the beginning of a conversation in [DIAL Chat](#dial-chat). Starter buttons let users begin a conversation with a single click instead of typing. Configured through the `starters` property in a [Quick App](#quick-app) or [application](#application) schema. + +## T + +### Tool + +A capability that an AI model can invoke during a conversation to perform a specific action — such as calling an external API, querying a database, or executing a computation. Tools are declared as part of a chat completion request and conform to the OpenAI function calling convention. + +In DIAL, tools can be provided by [MCP servers](#mcp-server), other [applications](#application), or models deployed in DIAL Core. See also: [Toolset](#toolset). + +### Toolset + +A named collection of [tools](#tool) exposed through an [MCP server](#mcp-server) connection. Toolsets serve as connectors between [Quick Apps](#quick-app) (and other applications) and external services. Users can browse available toolsets in the [Marketplace](#marketplace), add them to [My Workspace](#my-workspace), and use them in their applications. + +Toolsets support authentication (user credentials or organization-wide credentials) and can be [shared](#sharing) or [published](#publication). + +Also written as: "toolset" (single word, used in API field names and some UI labels). In documentation, use **toolset** (two words) in prose. + +## U + +### Unified API + +The DIAL-defined, OpenAI-compatible API exposed by [DIAL Core](#dial-core) for accessing all language models, embedding models, and [applications](#application) through a single interface. The Unified API extends the OpenAI Chat Completions API with DIAL-specific features including [attachments](#attachment), [stages](#stage), [custom content rendering](#visualizer), state management, and interactive controls. + +The Unified API is the unification layer that makes all models and applications interchangeable within the platform. + +Not to be confused with: "DIAL API" (too vague — use **Unified API** when referring to the chat/embeddings protocol). + +### Upstream + +A backend endpoint that serves requests for a specific [deployment](#deployment-configuration-sense). Each deployment can have multiple upstreams with different weights and tiers. The [load balancer](#load-balancer) distributes requests across upstreams based on the configured strategy. + +Upstream parameters include `endpoint`, `key`, `tier`, and `weight`. + +## V + +### Visualizer + +A special-purpose [application](#application) used to render specific content types within the [DIAL Chat](#dial-chat) UI. Visualizers extend the default rendering capabilities beyond built-in Markdown and Plotly support. Custom visualizers are built using the [DIAL Chat Visualizer Connector](https://github.com/epam/ai-dial-chat/blob/development/libs/chat-visualizer-connector/README.md) library. + +Examples: rendering 3D protein structures, financial charts, or custom data formats in the chat interface. + +--- + +## Concept Map + +The following diagram shows how key DIAL concepts relate to each other: + +``` +DIAL Core (required) +├── Unified API +│ ├── Chat Completion API +│ └── Embedding API +├── Deployments +│ ├── Models (via Adapters) +│ ├── Applications +│ │ ├── Schema-Rich Applications (via Application Types) +│ │ │ ├── Quick App / Quick App 2.0 +│ │ │ ├── Code App +│ │ │ └── Mind Map Studio +│ │ └── Applications without Schemas +│ └── Interceptors +├── Toolsets (via MCP Servers) +├── Resources +│ ├── Conversations +│ ├── Prompts +│ ├── Files (in File Storage) +│ └── Applications +├── Access Control +│ ├── Roles +│ ├── Private Space / Public Space +│ ├── Sharing (user-to-user) +│ └── Publication (org-wide, admin-approved) +├── Load Balancer (distributes across Upstreams) +├── Per-Request Keys +└── Dynamic Settings + +DIAL Chat (optional) +├── Marketplace +├── My Workspace +├── Agent Builders (UI wizards for Application Types) +├── Conversations (Replay, Playback, Parameterized Replay) +├── Visualizers +└── Custom UIs + +DIAL Admin (optional) +├── Deployment management +├── Access control management +├── Publication approval +└── Monitoring + +DIAL Overlay (optional) +└── Embeds DIAL Chat via iframe + +SDKs +├── DIAL SDK (applications, adapters) +└── Interceptors SDK (interceptors) +``` + +--- + +## Deprecated Terms + +| Term | Status | Replacement | +|---|---|---| +| Addon | Archived. ChatGPT plugin protocol, abandoned. | [Application](#application), [Interceptor](#interceptor), [Toolset](#toolset) | +| Assistant | Archived. Repository [ai-dial-assistant](https://github.com/epam/ai-dial-assistant). | [Application](#application) | +| "the backend" | Informal. Too vague. | [DIAL Core](#dial-core) | +| "the frontend" | Informal. Too vague. | [DIAL Chat](#dial-chat) | +| "DIAL API" | Ambiguous. Could refer to any DIAL endpoint. | [Unified API](#unified-api) (for the chat/embeddings protocol) | \ No newline at end of file diff --git a/docs-planning/improvement-roadmap.md b/docs-planning/improvement-roadmap.md new file mode 100644 index 000000000..d63090de3 --- /dev/null +++ b/docs-planning/improvement-roadmap.md @@ -0,0 +1,280 @@ +# DIAL Documentation - Improvement Roadmap + +> **Status:** Draft + + +## Executive summary + +The current `[docs.dialx.ai](http://docs.dialx.ai)` is best described as **an index of GitHub READMEs** rather than a documentation product. The site's structure, onboarding flow, and canonical reference pages systematically redirect readers to component repositories, which fragments the experience, breaks search, makes content stale invisibly, and leaves no single place to answer "what is DIAL and how do I use it?" + +Beyond the content gaps, the **information architecture itself is a barrier to adoption**: duplicate page titles with different content, the same topic split across unrelated sidebar locations, seven-level-deep menu paths, and a Platform/Tutorials split that follows no consistent principle. + +Most critically: **the site contains no in-depth tutorials**. The top-level "Tutorials" section is largely mislabeled — it contains user guides, configuration references, how-tos, and external links. The closest thing to tutorials are four minimal quickstart pages that run `docker compose up` without explaining what happens, verifying the result, or teaching any concept. A developer who wants to learn DIAL beyond "download and run" has nowhere to go. + +This roadmap proposes a **four-phase program** to turn the docs into a first-class product surface: + +| Phase | Exit outcome | Phase | Exit outcome | +| :-- | :-- | :-- | :-- | +| Phase 0 — Foundation | Evidence-based plan and CI quality gates in place | | | +| Phase 1 — Stop the bleeding | Self-contained docs for the top user journeys, including DIAL Apps and Tool Sets; navigation coherent; first tutorials exist | | | +| Phase 2 — Depth and differentiation | A developer can evaluate, pilot, and ship DIAL without leaving the site | | | +| Phase 3 — Sustainability | Quality does not decay between releases | | | + + + +The guiding principle: **treat documentation as product**. Every assumption we make about code quality — tested, reviewed, versioned, observable — applies to docs. + +* * * + +## 1\. Approach and methodology + +Before writing a single page, four things must be true: + +### 1.1 Commit to an information architecture + +Adopt **[Diátaxis](https://diataxis.fr/)** (tutorials / how-to / reference / explanation). Every page is classified as exactly one type; each type has its own voice and structure defined in the Style Guide. + +The target site structure is defined in the **DIAL Recommended Doc Structure** document. It replaces the current Platform/Tutorials split with journey-based sections, eliminates all duplicate titles, and enforces a maximum sidebar depth of 4 levels. + +### 1.2 Define personas and their first-hour journey + +Six personas, each with a named entry point on the landing page: + +| Persona | First-hour goal | Persona | First-hour goal | +| :-- | :-- | :-- | :-- | +| End user | Learn to use DIAL Chat: conversations, prompts, marketplace, apps, files, sharing | | | +| App Developer | Run DIAL locally, hit the Unified API, build a minimal application with the SDK | | | +| Platform / DevOps | Deploy DIAL to a chosen cloud with observability and auth wired in | | | +| Admin / Governance | Configure roles, rate limits, publications, and compliance settings | | | +| Evaluator / PoC lead | Understand what DIAL is, what it replaces, and run a demo | | | +| Solution Architect | Read reference architectures, scaling patterns, and integration trade-offs | | | + + + +### 1.3 Inventory before authoring + +Content audit spreadsheet: every URL classified by Diátaxis type, persona, last-updated date, quality score (1–5), owner, and action (keep / merge / rewrite / delete / redirect). The migration map in the Structure document provides the target destination for every existing page. + +### 1.4 Instrument for outcomes + +Baseline the metrics in §4 before work begins. Ship improvements against measurable targets, not vibes. + +* * * + +## 2\. Gap summary + +The **DIAL Documentation Gap Analysis** identifies 25 gaps across six categories. The top findings, ranked by severity: + +| Rank | Gap | Category | Rank | Gap | Category | +| :-- | :-- | :-- | :-- | :-- | :-- | +| 1 | No in-depth tutorials exist — quickstarts run docker compose up but teach nothing; no tutorial builds an app, adapter, or interceptor | Content | | | | +| 2 | No "getting started with the API" guide — no bridge from Quick Start to API usage | Content | | | | +| 3 | DIAL Apps ecosystem under-documented — Custom, Quick, Code, Mind Map scattered and asymmetric | Content | | | | +| 4 | Tool Sets undocumented — no reference, no authoring guide, no examples | Content | | | | +| 5 | Seven duplicate-title pairs in navigation | Navigation | | | | +| 6 | Platform/Tutorials split incoherent — no consistent principle | Navigation | | | | +| 7 | "Tutorials" section largely mislabeled — contains user guides, config references, how-tos, and external links | Navigation | | | | +| 8 | GitHub redirects as authoritative source for canonical content | Structural | | | | +| 9 | Videos not self-sufficient — no code, data, or config to reproduce | Content | | | | +| 10 | No OpenAI API compatibility matrix — "compatible" unspecified | Content | | | | + + + +Additional high-impact gaps include: naming chaos ("Agent Builders" × 4 labels), no glossary, observability stops at "use OTEL," networking/firewall docs promised but never delivered, no dependency configuration guidance (Redis, blob storage), no error reference, no version compatibility matrix, no production hardening guide, no community surface, no testing guidance, no environment prerequisites, and no "what's next" navigation between pages. + +See the full Gap Analysis document for detailed evidence, URLs, and impact assessment for all 25 gaps. + +* * * + +## 3\. The roadmap + +### Phase 0 — Foundation + +**Goal:** Plan with evidence, not opinion. Put quality gates in place before they're needed. + +**Workstreams** + +1. **Content inventory** — every page classified by Diátaxis type, persona, quality, last-updated, owner, action. Use the migration map in the Structure document. Flag every mislabeled page (especially "tutorials" that aren't tutorials). +2. **Style guide** finalized, reviewed by stakeholders, and committed to the docs repo as the governing standard for all new and modified content. Includes the **"what's next" rule** (every page must end with 2–3 links to logical follow-up pages). Machine-enforceable rules (terminology, frontmatter) are wired into CI. Existing pages are **not** retroactively updated — they come into compliance when touched during Phase 1 migration. +3. **Terminology glossary** stubbed per the Structure document's concept map. +4. **CI quality gates**: + * Internal + external link checker. + * Vale lint pack for terminology and forbidden phrases. + * Dependency-freshness check. + * Frontmatter completeness check. +5. **Analytics baseline** captured. + +**Exit criteria** + +* Inventory complete and reviewed. +* At least three CI gates enforcing on `main`. +* Baseline metrics dashboard exists. + +* * * + +### Phase 1 — Stop the bleeding + +**Goal:** The top user journeys work end-to-end on `[docs.dialx.ai](http://docs.dialx.ai)` without redirecting to GitHub, without requiring a video to fill in the blanks, and without forcing the reader to guess which of two duplicate pages to visit. **The first real tutorials exist.** + +**Workstreams** + +#### 1.1 Implement the target site structure + +Migrate to the structure in the **Structure document**: + +* Collapse Platform/Tutorials into journey-based sections. +* Eliminate all seven duplicate-title pairs. +* Flatten sidebar to ≤ 4 levels. +* Apply sidebar grouping labels. +* Fix URL ↔ menu alignment with redirect rules. +* Replace the DevOps GitHub link with a docs-site landing page. +* Add "what's next" links to every page touched during migration. + +#### 1.2 Ship the first tutorials — starting with the API + +The most critical new content is the **"Getting started with the DIAL API" tutorial** (Structure document §4.0) — the bridge between "DIAL is running" and "I can use it." This is the first real tutorial on the site and sets the quality bar for all subsequent ones. + +Also ship in Phase 1: + +* **Environment prerequisites** page (OS-specific notes, Docker resource allocation). +* At least one tutorial per app type: one Custom App tutorial and one Quick App tutorial, each with a paired sample repo, sample data, and configuration. + +#### 1.3 DIAL Apps and Tool Sets — top priority + +Ship the DIAL Apps subsections per the Structure document: Custom Apps, Quick Apps (including Tool Sets and MCP), **Code Apps**, and Mind Map Studio. + +#### 1.4 Fix the video content model + +Split Demos section: capability showcases stay (no written companion required); tutorial videos migrate to parent pages (written companion required). + +#### 1.5 Kill the GitHub-redirect habit + +Canonical home on docs site for Configuration, Quick Starts, Helm walkthroughs, and SDK overview. + +#### 1.6 Rewrite the landing page + +Persona-routed entry for all six personas per the Structure document. + +#### 1.7 Ship the Glossary, concept map, and OpenAI compatibility page + +Per the Structure document §3. The glossary resolves naming chaos. The OpenAI compatibility page tells migrating developers what works and what doesn't. + +#### 1.8 Fix the highest-impact stale examples + +Prioritized by analytics. + +#### 1.9 Launch the Changelog and version compatibility matrix + +Per the Structure document §7. + +**Exit criteria** + +* Site structure matches the Structure document. Zero duplicate titles. Max sidebar depth ≤ 4. +* **At least three real tutorials exist** (API getting started, one Custom App, one Quick App), each following the Diátaxis tutorial pattern with sample repo, data, and verification steps. +* Every page touched has "what's next" links. +* DIAL Apps and Tool Sets section live (including Code Apps). +* Demo/tutorial video split implemented. +* Zero GitHub redirects on canonical pages. +* Landing page, glossary, OpenAI compatibility, changelog, and version matrix live. +* Environment prerequisites page live. + +* * * + +### Phase 2 — Depth and differentiation + +**Goal:** A developer can evaluate, pilot, and ship DIAL without ever leaving the docs. + +**Workstreams** + +#### 2.1 Complete the tutorial library + +Ship the remaining flagship tutorials per the Structure document (RAG app, translator app rewrite, Code App, PII interceptor, custom adapter, Overlay UI, eval-driven development, multi-provider routing). Each with sample repo, data, CI. + +#### 2.2 SDK reference + +Per Structure document §4.8. + +#### 2.3 Comparison pages + +Four pages per Structure document §3. + +#### 2.4 Production readiness guide + +Per Structure document §5.7. Includes **networking and firewall documentation** (the content explicitly promised by the current Quick Start and never delivered). + +#### 2.5 Observability depth + +Per Structure document §5.6: tracing (OTEL), provider-specific guides (Datadog, Grafana+Prometheus, ELK, Azure Monitor, AWS CloudWatch), alerting. + +#### 2.6 Operational reference content + +* **Dependency configuration** (Redis, blob storage, InfluxDB) — per Structure document §5.4. +* **Error code reference** — per Structure document §5.8. +* **Troubleshooting guide** — per Structure document §5.8. + +#### 2.7 Developer experience content + +* **Testing guidance for custom apps** — per Structure document §4.9. +* **RAG explanation** — per Structure document §3. +* **File management how-to** — per Structure document §4.14. + +#### 2.8 Reference architectures and use cases + +Per Structure document §8. + +#### 2.9 Community section + +Per Structure document §9: contribution guide, community extension list, repository map, discussion/support links. + +**Exit criteria** + +* Nine flagship tutorials pass CI on a regular cadence. +* SDK reference live. +* Four comparison pages published. +* Production readiness guide published (including networking/firewall). +* Observability section includes tracing and at least three provider-specific guides. +* Dependency config, error reference, and troubleshooting live. +* Testing guidance live. +* Community section live with contribution guide and extension list. +* Use-case index browsable in both directions. + +* * * + +### Phase 3 — Sustainability + +**Goal:** Quality does not decay between releases. + +**Workstreams** + +* **Versioned documentation (optional, evaluate after Phase 2).** DIAL components release independently — there is no single release cadence to snapshot against. Instead, the primary mechanism is the **version compatibility matrix** (shipped in Phase 1) combined with **inline version annotations** on pages where behavior is version-dependent (e.g., "since Core 0.42"). Full site-wide docs versioning may be added later if user demand justifies the maintenance cost. +* **Per-page ownership** enforced via frontmatter; un-owned pages block merge. +* **Stale-content audit on a regular cadence:** `last_verified` beyond threshold generates a review task for the owner. +* **Unified changelog on docs site.** Aggregate release notes from all DIAL component repositories into a single chronological view on the docs site, so users can see what changed across the platform without visiting 15+ GitHub repos individually. +* **Docs-impact check in component release process.** Each component's release checklist includes a "docs impact" step: if a change affects user-facing behavior, a corresponding docs PR is opened and linked before the release is published. Enforced by process and PR template, not by cross-repo tooling. +* **Per-page feedback widget** → triaged, fed into the docs backlog. +* **Contributor onboarding path:** short guide for engineers submitting their first doc PR. +* **Community content pipeline:** accept community tutorials into a curated `community/` section with a lower review bar but the same freshness CI. + +**Exit criteria (continuous)** + +* Share of pages with fresh `last_verified` stays above target threshold. +* Zero pages without an owner. +* Mean time from feature ship to doc update stays below target threshold. + +* * * + +## Appendix: Phase 1 showcase page recommendation + +Before broad rollout, rewrite one page end-to-end as a proof of concept. Three candidates: + +* **The "Getting started with the DIAL API" tutorial** — demonstrates the Diátaxis tutorial pattern on net-new content. Proves that a real tutorial can exist on this site. Highest developer impact. +* **The Configuration page** — demonstrates the reference pattern on migrated content. Proves that GitHub redirects can be eliminated. +* **A DIAL Apps overview page** — demonstrates the explanation pattern on the most strategically important topic. + +Pick these before scaling the approach. A good rewrite will sell the rest of the roadmap internally better than any slide deck. + +  + +[Filter table data](#)[Create a pivot table](#)[Create a chart from data series](#) + +[Configure buttons visibility](/users/tfac-settings.action) \ No newline at end of file diff --git a/docs-planning/pr_555_todo.md b/docs-planning/pr_555_todo.md new file mode 100644 index 000000000..6c2e3b238 --- /dev/null +++ b/docs-planning/pr_555_todo.md @@ -0,0 +1,332 @@ +# PR #555 — Review Comment TODO / Triage + +**PR:** [epam/ai-dial#555 — "All changes in feature/doc_improvements branch"](https://github.com/epam/ai-dial/pull/555) +**Branch:** `feature/doc_improvements` → `main` · **State:** OPEN +**Scope of this document:** every review comment on PR #555, triaged for **clarity** (do we understand what is being asked), **impact** (what/how much has to change), **caveats** (risks, dependencies, things to verify), and **other important details**. + +## How to read this document + +537 inline review comments were left, but they are heavily duplicated (parallel `CC @sr-remsha` acknowledgments, repeated "screenshot outdated" notes, and one theme — Redis→Valkey — repeated across ~30 files). To keep this actionable, comments are grouped: + +- **Part A — Cross-cutting themes.** One decision/action resolves many comments at once. Do these first. +- **Part B — Section-by-section substantive comments.** Unique, content-bearing feedback that needs a per-item decision. +- **Part C — Noise / already-resolved.** Comments that need no independent action (CC pings, "screenshot committed" replies). +- **Part D — Prioritized action plan.** + +### Reviewer map (line-comment counts) + +| Reviewer | Count | Area of focus | +|---|---:|---| +| VolhaBazhkova | 179 | Chat User Guide text + screenshot accuracy | +| PolinaGurinovich97 | 87 | `CC @sr-remsha` pings (noise) | +| siarhei-fedziukovich | 73 | Administering DIAL (Admin Panel accuracy) | +| sr-remsha | 41 | Structure + "screenshot committed" replies | +| kamiakou-epam | 37 | Cloud/production deployment (Redis→Valkey, ingress) | +| andrii-novikov | 37 | Quick Apps 2.0 schema/config correctness | +| adubovik | 25 | Interceptors, Adapters, SDK reference scope | +| Mmefko | 19 | Observability (OTEL vs Prometheus) | +| YuriyIvon | 15 | Quick starts, positioning/comparisons, architecture | +| Pasichniuk | 11 | Deployments (images, container management) | +| serguei-gorokhov | 6 | Security & governance (authN/authZ) | +| alexey-ban | 5 | Production readiness | +| valerydluski | 2 | Mind Map Studio | +| olegmikhnovich | 1 | n8n integration | +| sdryapko | 3 (issue-level) | Evaluations, Code Apps | + +> Note: the only PR-level review body is a stray `"t"` from siarhei-fedziukovich — ignore. + +--- + +## Applied changes log — session 2026-08-10 (trivial batch) + +The "truly-trivial, zero-dependency" batch was applied and **committed** to `feature/doc_improvements` as **`9676ba30`** (the repo-rule change is a separate commit, **`0fc82d0f`**). Build passes (`npm run build`, exit 0). **Threaded replies referencing `9676ba30` were posted to every related PR #555 comment thread.** Status key: ✅ applied · ⏭️ deferred · ⏸️ already present. + +| Item | File | Change | Status | +|---|---|---|---| +| ~~B3.1~~ | ~~`.../quick-app-2/1.create-via-ui.md:62`~~ | ~~Knowledge-base wording (andrii-novikov `suggestion`)~~ | ⏸️ already present verbatim | +| ~~B3.3~~ | ~~`.../quick-app-2/2.create-via-api.md:19`~~ | ~~Core-URL clarification (andrii-novikov `suggestion`)~~ | ⏸️ already present verbatim | +| ~~B3.4~~ | ~~`.../9.tool-sets/2.mcp-server-integration.md:188,196`~~ | ~~`audience` → `aud` (JSON key + prose)~~ | ✅ applied | +| ~~B5.8~~ | ~~`5.administering-dial/5.deployments/1.images.md:101`~~ | ~~dialog title `Save new version` → `Save as new version`~~ | ✅ applied | +| ~~B6.8~~ | ~~`6.chat-user-guide/5.files.md:16,27`~~ | ~~`My files` → `My Files` (label, link text, heading; anchor `#my-files` unchanged)~~ | ✅ applied | +| ~~B6.6~~ | ~~`6.chat-user-guide/3.marketplace-and-apps.md:41`~~ | ~~filter label `Source` → `Sources`~~ | ✅ applied | +| ~~B2.10~~ | ~~`2.understand-dial/5.foundations/2.dial-evolution.md:46`~~ | ~~"roadmap is not publicly available" → link to https://dialx.ai/roadmap~~ | ✅ applied | +| ~~B4.5~~ | ~~`.../sdk-reference/1.dial-app.md:61`~~ | ~~`during streaming responses` → `during SSE streaming responses`~~ | ✅ applied | +| B5.13 | `5.administering-dial/img/100.png` | rename to meaningful name | ⏭️ deferred — **image is orphaned** (no reference in `docs_v2/`); nothing to re-point, and a meaningful name needs visual inspection. Replied on the thread asking for a **delete vs keep+wire-in** decision instead of a blind rename. | + +> All 10 related threads have been answered on GitHub (6 fixed → `9676ba30`; 2 already-correct suggestions confirmed; 1 orphaned-image awaiting your delete/embed decision). + +--- + +## Part A — Cross-cutting themes (do these first) + +### ~~A1. Redis → Valkey replacement~~ ✅ DONE (~30 comments, kamiakou-epam + YuriyIvon + alexey-ban) + +> ✅ **Done (2026-08-10)** — commits **`da868bc9`** (cloud-deployment + production-readiness) and **`931a26cd`** (architecture/glossary/what-is-dial). Verified against `ai-dial-helm@main` (dial-core 6.0.0 → valkey subchart; values `redis:` → `valkey:`). Build green; 30 threads (kamiakou ×28, alexey-ban Bitnami, YuriyIvon dial-stack) answered with commit ids. +> +> **Contextual, not a blanket rename:** the bundled component → **Valkey (Redis-compatible)**; **kept** the Redis-protocol config (`aidial.redis.*`, `redis://`, `settings.json` `redis`), the **local docker-compose** Redis image, and **managed-service names** (ElastiCache / Azure Cache for Redis / Memorystore). **Deferred:** the `persistant-layer.svg` diagram still labels the cache "Redis" (regenerate separately); non-Valkey B8 items (ingress, Deployment Manager Backend, scaling limits) remain open. + +- **Files:** all of `4.operating-dial/2.cloud-deployment/*` (aws, azure, gcp, generic-kubernetes), `4.operating-dial/7.production-readiness/*` (0.index, 1.high-availability, 3.secrets-management, 4.backup-and-restore, 5.upgrade-procedure), `2.understand-dial/2.architecture/2.dial-stack.md`, `4.operating-dial/2.cloud-deployment/0.index.md:29`. +- **What's asked:** Redis was replaced by **Valkey** in [dial-core 6.0.0](https://github.com/epam/ai-dial-helm/releases/tag/dial-core-6.0.0) and [dial 7.0.0](https://github.com/epam/ai-dial-helm/releases/tag/dial-7.0.0) Helm charts. Every mention of Redis in deployment/HA/backup context must be reviewed and updated. Separately, YuriyIvon (dial-stack.md:33) asks to note that any Redis-compatible managed PaaS (e.g. Azure Cache for Redis) can also be used; alexey-ban (high-availability.md:67) notes "we no longer use Bitnami Redis." +- **Clear?** Yes — unambiguous and evidenced with release links. +- **Impact:** High volume, low complexity. Mechanical find-and-replace-plus-verify across ~10 files, but each occurrence must be checked in context (config keys, chart values, port numbers may differ, not just the name). +- **Caveats:** + - Confirm the exact cutover: is Redis still *supported* (compatibility mode) or fully replaced? Valkey is Redis-protocol-compatible, so "Redis-compatible PaaS still works" (YuriyIvon) and "we use Valkey now" (kamiakou) are both true — the docs should say Valkey is the default/bundled store while any Redis-protocol-compatible service remains valid. + - Do NOT blanket-rename in conceptual pages where "Redis" describes the *protocol/ecosystem* rather than the bundled component. + - Requires a `last_verified` bump on every touched page. +- **Owner hint:** kamiakou-epam (raised all deployment instances). + +### A2. Chat User Guide screenshots outdated (~120 image comments, VolhaBazhkova; partly resolved by sr-remsha) + +- **Files:** `6.chat-user-guide/img/*.png` (app-builder, app-link, chat, compare*, conv-menu, delete-*, deploy-code-app*, dial-marketplace*, edit-*, home, logs-code-app, marketplace-home-select, math-prompt*, mindmap2, model-link, move_prompt, publish-*, quick-app*, register-*, remove-*, replay*, share-*, system_prompt, temperature, toolset-*, unpublish-*, versioning*, workspace-talk-model, regenerate, like, and more). +- **What's asked:** Update screenshots to match current UI. Recurring specific changes: **new DIAL logo**; new **three-dots (`...`) context-menu** design (tracked in [ai-dial-chat#8073](https://github.com/epam/ai-dial-chat/issues/8073)); **radio-button fix**; new menu items (**Info**, **Use**, **View**, **Connect**, **Redeploy**); auto-populated publication fields; new features (Token limits, Process files, File tools, filter counts). +- **Clear?** Yes, individually. Each comment names the exact delta. +- **Impact:** Very high effort — requires authenticated DIAL Chat access to re-capture ~60+ screenshots (see memory: *screenshots require login, not anonymous*). Several depend on **ai-dial-chat#8073** landing first (bookmark + three-dots icon redesign) — VolhaBazhkova explicitly says "better to update the screenshot after the fix" for those. +- **Partial status:** sr-remsha has already committed replacements for a batch: `create-pt`, `app-wizards`, `compare-3/4`, `compare`, `conversation-menu`, `dial-marketplace`, `dial-marketplace5`, `home`, `isolated_view_mode`, `marketplace-home-select`, `mindmap2`, `prompt-menu`, `quick-app-builder`, `replay1`, `Replay_as_is`, `response-format`, `system_prompt`, `temperature`, `toolset-editor`, `workspace-talk-model`. `app-link` — sr-remsha asked VolhaBazhkova to re-confirm it may already be current. +- **Caveats:** + - **Gate on #8073.** For every "three dots"/bookmark comment, capturing now guarantees rework. Decide: block those screenshots until the ui-kit fix ships, or accept a second pass. + - Text must be re-validated alongside screenshots — several comments (e.g. `quick-app2-starters`, `deploy-code-app2`) note new *features* in the shot, not just cosmetics, which means the surrounding prose is also stale. + - Track remaining vs. done in a checklist to avoid re-capturing already-fixed ones. +- **Recommended:** maintain a per-image status table; split into "cosmetic (logo/radio)" (do now) vs "blocked on #8073" vs "feature-changed (needs text edit too)". + +### ~~A3. "Tool Set" → "Toolset" terminology~~ ✅ DONE (~30 comments, VolhaBazhkova; related: siarhei-fedziukovich, andrii-novikov) + +> ✅ **Done (2026-08-10)** — canonical form **Toolset / Toolsets** (user decision per VolhaBazhkova). Commits **`db36decf`** (route renames `tool-sets` → `toolsets`) and **`43ac88e4`** (project-wide prose/heading/label/anchor sweep across all of docs_v2 + `docs-planning/glossary.md` + CLAUDE.md convention; client-redirects plugin for old URLs). Build green; 28 VolhaBazhkova Toolset threads answered with commit ids. Code identifiers (`tool_sets`, `client_toolset`, …) and the verbatim upstream schema snapshot `7.reference/changelog/quickapp2-schema.json` left unchanged. +> +> **Not part of this pass (tracked under B6):** her non-terminology sub-points — specific `#…` anchor links, "sharing a Toolset is not possible / publish-only", and a dedicated Toolsets section in the Marketplace page. **Deferred bundled step:** the B6.5 sharing/publishing behavior fixes were intentionally kept out of A3 to avoid a half-done B6.5. + +- **Files:** `6.chat-user-guide/4.tool-sets.md` (throughout), `6.chat-user-guide/6.sharing-and-publishing.md` (many lines), `6.chat-user-guide/3.marketplace-and-apps.md`, and anywhere "Tool Set(s)" appears. +- **What's asked:** DIAL Chat UI and Admin app use **Toolset** (one word). Standardize. VolhaBazhkova (4.tool-sets.md:2) explicitly says: pick one form and apply globally via search, and flags that the **Admin user guide uses "Tool set"** (lowercase second word) — so there is an existing inconsistency to resolve. +- **Clear?** Yes, but requires a **terminology decision** first. +- **Impact:** Global find-and-replace once the canonical form is chosen — but this **conflicts with the project glossary/style guide**, which currently standardizes on "Tool Set(s)" (see project CLAUDE.md and `docs-planning/glossary.md`). +- **Caveats — IMPORTANT:** + - This is a **project-wide terminology conflict**, not a local fix. The CLAUDE.md conventions and glossary must be updated in lockstep, or the docs and the style guide will disagree. + - Decide the single canonical form: **`Toolset`** (matches Chat + Admin UI, reviewer preference) vs `Tool Set` (current glossary). Recommendation: adopt **Toolset** to match the product UI, and amend the glossary + style guide accordingly. + - After deciding, sweep filenames/anchors too — `#publish-a-tool-set` anchors are referenced in other comments; changing display text may or may not change slugs. +- **Blocker:** get sign-off from docs lead before the global sweep. + +### ~~A4. Quick Apps 1.0 removal + Quick Apps 2.0 schema correctness~~ ✅ DONE (~30 comments, andrii-novikov + others) + +> ✅ **Done (2026-08-10)** — commits `1d62515e` (remove 1.0), `497aa105` (schema/config + open items), `049e445a` (API routes + protocol). Build green; 34 andrii-novikov threads answered with commit ids. Verified against `ai-dial-quickapps-backend@development` (generated schema, applications.json) and `ai-dial-core@development` (applications API in `docs/open_api_core.yaml`). +> +> **Still open (tracked elsewhere):** glossary "keep both QuickApps" reconcile (B2.11); screenshot re-captures for `1.create-via-ui.md` and the Notion image in `5.tutorial-agent-loop-ui.md` (A2 / B3.5); runtime execution of every example (A5); installation-guide rebuild is **B8.4** (separate). The `{{variable}}` claim removal was applied **per reviewer request** even though the shipped schema still exposes `system_prompt.variables` — flag for re-confirmation with the QuickApps team. + +- **Files:** `3.building-with-dial/1.apps/2.quick-apps/0.index.md`, `.../coverage-status.md`, `.../1.quick-app-2/1.create-via-ui.md`, `2.create-via-api.md`, `3.create-via-config.md`, `4.working-with-tools-and-agents.md`, `5.tutorial-agent-loop-ui.md`, `6.tutorial-agent-loop-api.md`, `7.tutorial-agent-loop-config.md`, `8.examples.md`, `9.tool-sets/*`; glossary QuickApps entry. +- **What's asked (two strands):** + 1. ~~**Remove Quick Apps 1.0.**~~ ✅ **done in `1d62515e`** (2.quick-app-original/ deleted, sidebar + index reframed). YuriyIvon/glossary:280 "keep both QuickApps" reconcile still open — see B2.11. + 2. ~~**Fix the 2.0 schema/config throughout**~~ ✅ **done in `497aa105` + `049e445a`** — every item below was corrected and source-verified: + - `name` is **deprecated → use `deployment_id`** (0.index.md:99, 8.examples.md:24) + - `starters` **deprecated → `conversation_starters`** (8.examples.md:35) + - `applicationTypeSchemaId` is a **top-level property**, not nested under `reference` (3.create-via-config.md:115 & 263, 7.tutorial-agent-loop-config.md:75) + - missing `dial:applicationTypeSchemaEndpoint` (7.tutorial-agent-loop-config.md:43) + - tool query field is **`query`, not `text`** (4.working-with-tools-and-agents.md:99, 6.tutorial-agent-loop-api.md:144, 7...:66) + - **no `{{variable}}` support** (0.index.md:64, 6.tutorial-agent-loop-api.md:123) + - **no built-in RAG** in Quick Apps (0.index.md:147, coverage-status.md:35, examples need to connect a RAG tool — 8.examples.md:45) + - properties table stale — "Starters deprecated, new properties added" (0.index.md:86); "there is no such feature" (coverage-status.md:38); `conversation_mode` will replace a soon-deprecated field (coverage-status.md:39) + - **does the create-via-API path even work?** (2.create-via-api.md:1) — reviewer "was not able to create and could not find the endpoint"; "other endpoints outdated as well" + - "10 iterations ≠ 10 tool calls" (6.tutorial-agent-loop-api.md:196); "core routes are incorrect" (6...:1); no `max_iterations` param (9.tool-sets/5.examples.md:305); "retest all examples, most configs outdated/not working" (9.tool-sets/5.examples.md:1); reference outdated vs fresh schema (9.tool-sets/4.reference.md:1) +- **Clear?** Yes — very specific, code-level corrections. +- **Impact:** **High and blocking.** This is the platform's primary value surface (per project brief). Multiple documented configs and API calls are **factually wrong and won't work**. This isn't editorial — it's correctness. +- **Caveats:** + - **Verify against the live schema/source**, not against reviewer memory alone — reviewer flags several as "at least X is broken, but maybe not only." Use `docs-researcher` against [ai-dial-quickapps-backend](https://github.com/epam/ai-dial-quickapps-backend) + the current `applicationTypeSchema`. + - Every config/API example should be **executed** before re-publishing (ties to A5 — CI-test the snippets). + - The Quick Apps 1.0 removal decision affects sidebar structure, redirects, and the installation guide (see B-Operating: `7.quick-apps-installation.md` is also stale). + - The `1.create-via-ui.md` UI is described as "outdated" AND sr-remsha has uploaded new screenshots but says "text must be validated and updated to reflect new features" — so screenshots and prose are out of sync here. +- **Owner:** andrii-novikov for schema truth; coordinate with QA-backend team. + +### A5. Code snippets / tutorials must be CI-tested (adubovik) + +- **Files:** `3.building-with-dial/2.interceptors/1.tutorial-pii-interceptor.md:10`, `3.adapters/1.tutorial-custom-adapter.md:12`. (Applies by extension to A4 Quick App examples.) +- **What's asked:** Integrate tutorial code into project CI as tests, the same way [dial-cookbook](https://github.com/epam/ai-dial/tree/main/dial-cookbook) examples are tested — otherwise there's no guarantee the snippets run. +- **Clear?** Yes. +- **Impact:** Medium-high, and it's **infrastructure/process work**, not doc editing — needs a test harness that extracts and runs the tutorial code. Broad payoff (prevents the exact class of breakage A4 is full of). +- **Caveats:** Requires runnable environment (DIAL Core + a model or echo). Decide scope: which tutorials become CI-gated. This is a roadmap/"sustainability" item, likely a separate follow-up issue rather than part of this PR. + +### A6. SDK / low-level reference belongs with the source code, not the docs site (adubovik) + +- **Files:** `3.building-with-dial/2.interceptors/2.sdk-reference.md` (10, 24, 28), `2.interceptors/4.examples.md:10`, `5.developer-tools/1.sdk-reference/*` (0.index.md:34, 1.dial-app.md:21/61/125, 4.exceptions.md:12, 5.telemetry.md:12). +- **What's asked:** Don't reproduce SDK method signatures, module tables, exception lists, and telemetry internals in the high-level docs — they are fragile (version-coupled to source) and belong in the SDK repo's own docs. There's a dedicated issue for telemetry: [ai-dial-sdk#249](https://github.com/epam/ai-dial-sdk/issues/249). Guiding principle he states: *"Do I help the reader answer their question, or overload them with redundant info?"* Prefer **usage-by-use-case** over type signatures; document only stable integration points (e.g. which HTTP clients support propagation, that only **SSE** streaming responses are affected — not "every streaming response"). +- **Clear?** Yes, and philosophically consistent across all his SDK comments. +- **Impact:** **Structural/scope decision.** Potentially deletes or heavily trims several pages (`sdk-reference/*`, interceptor `sdk-reference.md`, `examples.md`). Affects the recommended site structure. +- **Caveats:** + - This partially **contradicts the improvement roadmap's goal** of "consolidating configuration/reference on-site (stop redirecting to GitHub READMEs)." Reconcile: the roadmap wants *config reference* on-site; adubovik wants *SDK API reference* to stay with source. These aren't the same thing — config/behavior reference on-site is fine; mirroring SDK class signatures is what he objects to. Draw the line explicitly in the style guide. + - Get docs-lead alignment before deleting pages that were deliberately created. + - `4.exceptions.md` / `5.telemetry.md` — coordinate with SDK repo owners so the content lands there (link out, don't just delete). + +### A7. Prefer Mermaid / visual diagrams (YuriyIvon, adubovik) + +- **Files:** `1.home/5.architect-overview.md:22` ("make a visual diagram"), `3.adapters/1.tutorial-custom-adapter.md:25` ("Mermaid sequence diagram, supported by GitHub"). +- **Clear?** Yes. +- **Impact:** Low-medium. Replace ASCII/prose flows with Mermaid. Docusaurus + GitHub both render Mermaid. +- **Caveats:** Confirm Mermaid is enabled in the Docusaurus theme config; keep diagrams theme-aware (light/dark). Architect overview currently uses a ```text block — good candidate. + +--- + +## Part B — Section-by-section substantive comments + +### B1. Home / Quick starts (`1.home/`) + +| # | Comment (author, file:line) | Clear? | Impact | Caveats / notes | +|---|---|---|---|---| +| B1.1 | Reshape quick starts on the **full Docker Compose incl. Keycloak + Admin**, not the minimal "developer-only" compose (YuriyIvon — developer-quick-start:25, devops-quick-start:26, admin-quick-start:16) | Yes | High — rewrites 3 quick starts | Strategic direction change. Admin quick start currently links DevOps quick start as prereq but **DevOps QS doesn't include Admin** (his admin-quick-start:16 note) — the whole prereq chain is broken. Decide the canonical compose bundle first. | +| B1.2 | **Merge DevOps QS into Developer QS** — "no significant difference" (adubovik — devops-quick-start:10) | Yes | Medium — deletes/absorbs a page | Overlaps with B1.1. If quick starts are reshaped around one full compose, this consolidation likely happens naturally. Check for missing details before deleting. | +| B1.3 | Architect overview needs a **visual diagram** (YuriyIvon:22) | Yes | Low | See A7. | +| B1.4 | Admin QS: `Deployments` section is optional (needs Deployment Manager); use **Assets** instead; **Entities** materialize to Core config while Assets store apps/toolsets in marketplace/resource buckets; add an **Assets** section; "can see at least one entity" is wrong (empty env is valid); published resource should appear in Assets (siarhei-fedziukovich — admin-quick-start:24/28/33/42) | Mostly | Medium | Requires Admin Panel domain knowledge. Reconcile with B-Admin comments on the same concepts (entities vs assets vs deployments). | +| B1.5 | End-user guide: page "has almost no meaning" — it's really a chat quick start; drop "SaaS" wording; "model selector" → **agent selector** (can be non-model); "select an agent"/"enter your prompt"; the model list isn't limited to "document analysis/summarization/knowledge exploration" — it shows whatever the instance exposes; mention configurable auth providers + prior access (sr-remsha — end-user-guide:2/16/23/27/28/34/21) | Yes | Medium | Rename/repurpose the page; terminology (**agent** vs model) recurs across Chat guide. "SaaS" wording objection also raised by sdryapko (see B6) — decide globally. | +| B1.6 | Home index: "make sure this definition of DIAL meets expectations"; add a bullet per role in the roles table (sr-remsha — 0.index.md:12/29) | Partly | Low-Medium | "Definition meets expectations" needs a stakeholder review of the DIAL one-liner — get canonical wording from product. | +| B1.7 | Release-notes videos don't display on client-side nav (refresh fixes) — applies to **all** release-note videos (PolinaGurinovich97 — release-notes-1.38.md:17) | Yes | Medium (bug, not content) | This is a **Docusaurus/site bug**, not a content fix — likely a `