Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions site/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,16 @@ The browser adds section-linked guide search, copy buttons, and a full-size scre

- The landing page (`src/Landing.tsx`) is the user guide in one page: install, how to read the treemap, the views, workflows, the command line, a reference, and a short "How I built it".
- The reference tables (`src/Reference.tsx`) read `src/data/catalog.json`, Srcly's own metric catalog. Do not retype metric definitions.
- The longer guides are Markdown in `src/docs/pages/`. Add a new page to the navigation in `src/App.tsx`.
- The longer guides are Markdown in `src/docs/pages/`. Add a new page to the navigation in `src/App.tsx`. Also add it to the `order` list in `scripts/prerender.mjs`, which writes the agent files.
- The build writes files for agents: `llms.txt` (an index), `llms-full.txt` (every guide plus the metric reference), `metrics.md`, and an `index.md` beside each guide.
- The landing header search (`src/SiteSearch.tsx`) indexes the guide sections, every docs heading, every metric and every command. `/` focuses it.
- The hero treemap recolors by complexity, commits or findings and reads out the tile under the pointer. `src/data/tree.json` holds those three values per file.
- Keep command examples in sync with `server/app/run.py`.

## Screenshots and data

`node scripts/capture.mjs [name,name…]` captures everything from a real Srcly run on Srcly's own source.
It clones this checkout's HEAD to a temporary `srcly` folder, starts the API and client on free ports with a temporary cache, and writes:
`node scripts/capture.mjs [name,name… | data] [--ref <git ref>]` captures everything from a real Srcly run on Srcly's own source (`data` skips the screenshots).
It clones this checkout at `origin/main` (or `--ref`) to a temporary `srcly` folder, starts the API and client on free ports with a temporary cache, and writes:

- `public/screenshots/*.jpg`: the app at 1440 × 900 CSS pixels, 1.5x.
- `src/data/tree.json`: folder and file LOC and complexity, for the treemaps drawn on the page.
Expand Down
18 changes: 11 additions & 7 deletions site/scripts/capture.mjs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// Captures the site's screenshots and data from a real Srcly run on Srcly's own source.
//
// cd site && node scripts/capture.mjs [name,name…] [--ref <git ref>]
// cd site && node scripts/capture.mjs [name,name… | data] [--ref <git ref>] (data: no screenshots)
//
// Needs the server venv (`cd server && uv sync`), client node_modules, and Google Chrome.
// It clones this checkout at `origin/main` (or --ref) to a temporary `srcly` folder (so the shots show the
Expand All @@ -16,7 +16,7 @@
// The data-flow and symbols shots use the checked-in engine/examples/symbol-graph fixture:
// a small app with a route, because a trace over the whole repository takes minutes.
import { execFileSync, spawn } from "node:child_process";
import { mkdtemp, mkdir, realpath, rm, writeFile } from "node:fs/promises";
import { mkdtemp, mkdir, readFile, realpath, rm, writeFile } from "node:fs/promises";
import { createServer } from "node:net";
import { tmpdir } from "node:os";
import { join } from "node:path";
Expand Down Expand Up @@ -82,8 +82,11 @@ const full = JSON.parse(srcly("scan", clone, "--out", "-"));
const fileTypes = new Set(["file"]);
const slim = (node) => {
const out = { n: node.name, l: node.metrics?.loc ?? 0 };
const ccn = node.metrics?.complexity;
if (ccn !== null && ccn !== undefined) out.c = Math.round(ccn);
// Short keys keep the file small: c = max complexity, m = commits (90 days), f = findings. Absent is N/A.
for (const [key, id] of [["c", "complexity"], ["m", "commit_count"], ["f", "finding_count"]]) {
const value = node.metrics?.[id];
if (value !== null && value !== undefined) out[key] = Math.round(value);
}
if (node.type === "folder") out.k = (node.children ?? []).filter((child) => child.type === "folder" || fileTypes.has(child.type)).map(slim).filter((child) => child.l > 0);
return out;
};
Expand All @@ -92,8 +95,8 @@ await writeFile(join(data, "tree.json"), JSON.stringify({ ...slim(full), n: "src
const catalog = await (await fetch(`http://127.0.0.1:${api}/api/metrics/catalog`)).json();
await writeFile(join(data, "catalog.json"), JSON.stringify({
groups: catalog.groups,
metrics: catalog.metrics.map(({ id, label, short_label, unit, group, direction, aggregation, languages, description, witness }) =>
({ id, label, short_label, unit, group, direction, aggregation, languages, description, witness })),
metrics: catalog.metrics.map(({ id, label, short_label, unit, group, direction, aggregation, languages, description, witness, color_domain, color_per_file }) =>
({ id, label, short_label, unit, group, direction, aggregation, languages, description, witness, color_domain, color_per_file })),
windows: catalog.activity_windows,
presets: catalog.quadrant_presets.map(({ id, label, description, x, y, cohorts, quadrants }) =>
({ id, label, description, x: x.label, y: y.label, cohorts, quadrants: quadrants.map(({ name, description: text }) => ({ name, text })) })),
Expand All @@ -108,7 +111,8 @@ await writeFile(join(data, "cli.json"), JSON.stringify(Object.fromEntries(Object

const m = full.metrics;
await writeFile(join(data, "meta.json"), JSON.stringify({
sha, date, loc: m.loc, files: m.file_count, functions: m.function_count,
sha, date, version: (await readFile(join(clone, "server/pyproject.toml"), "utf8")).match(/^version = "(.+)"/m)?.[1],
loc: m.loc, files: m.file_count, functions: m.function_count,
topCcn: Math.round(m.complexity), topCcnName: m.complexity_witness?.name,
topCcnPath: m.complexity_witness?.path.split("::")[0].replace(`${clone}/`, ""),
findings: m.finding_count,
Expand Down
67 changes: 66 additions & 1 deletion site/scripts/prerender.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ function html(page) {
const metadata = [
`<meta name="description" content="${escape(page.description)}" />`,
`<link rel="canonical" href="${canonical}" />`,
page.route.startsWith("docs/") ? `<link rel="alternate" type="text/markdown" href="${canonical}index.md" />` : `<link rel="alternate" type="text/plain" title="llms.txt" href="${origin}${base}llms.txt" />`,
`<meta property="og:type" content="website" />`,
`<meta property="og:site_name" content="Srcly" />`,
`<meta property="og:title" content="${escape(page.title)}" />`,
Expand Down Expand Up @@ -69,4 +70,68 @@ await writeFile("dist/404.html", html({ route: "404/", title: "Page not found ·
await writeFile("dist/.nojekyll", "");
await writeFile("dist/robots.txt", `User-agent: *\nAllow: /\nSitemap: ${origin}${base}sitemap.xml\n`);
await writeFile("dist/sitemap.xml", `<?xml version="1.0" encoding="UTF-8"?>\n<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">${pages.map((page) => `<url><loc>${origin}${base}${page.route}</loc></url>`).join("")}</urlset>\n`);
console.log(`Prerendered ${pages.length} pages, docs entry point, and 404 page.`);
// For agents: each guide as Markdown beside its page, llms.txt as the index, llms-full.txt as everything.
const absolute = (markdown) => markdown
.replace(/<figure>.*?<img src="([^"]+)" alt="([^"]+)".*?<\/figure>/gs, (_m, src, alt) => `![${alt}](${origin}${src})`)
.replace(/\]\(\.\.\/([a-z0-9-]+)\/?(#[^)]+)?\)/gi, (_m, slug, hash = "") => `](${origin}${base}docs/${slug}/${hash})`)
.replace(/\]\(\/srcly\//g, `](${origin}${base}`);
const guides = [];
const order = ["overview", "tour", "design", "get-started", "explore", "metrics", "findings", "data-flow", "reports", "cli", "configuration"];
const slugs = (await readdir("src/docs/pages")).filter((name) => name.endsWith(".md")).map((name) => name.slice(0, -3));
assert.deepEqual([...slugs].sort(), [...order].sort(), "Add new guides to the llms.txt order");
for (const slug of order) {
const filename = `${slug}.md`;
const markdown = absolute(await readFile(`src/docs/pages/${filename}`, "utf8"));
const page = pages.find((item) => item.route === (slug === "overview" ? "docs/" : `docs/${slug}/`));
await writeFile(`dist/docs/${slug}/index.md`, markdown);
if (slug === "overview") await writeFile("dist/docs/index.md", markdown);
guides.push({ slug, title: markdown.match(/^# (.+)$/m)[1], description: page.description.split(/(?<=\.)\s/)[0], markdown });
}
const catalog = JSON.parse(await readFile("src/data/catalog.json", "utf8"));
const meta = JSON.parse(await readFile("src/data/meta.json", "utf8"));
const cell = (text) => String(text ?? "").replace(/\|/g, "\\|").replace(/\n/g, " ");
const metrics = [
"# Srcly metrics",
"",
`Every metric Srcly measures, from its metric catalog (version ${meta.version}). Rollup is how a folder or file combines its children; "max, named" also names the function that sets it. N/A means the metric does not apply or its tool did not run. It is never 0.`,
...catalog.groups.flatMap((group) => [
"", `## ${group.id}`, "", group.description, "",
"| Metric | ID | Rollup | Languages | What it measures |", "| --- | --- | --- | --- | --- |",
...catalog.metrics.filter((metric) => metric.group === group.id).map((metric) =>
`| ${cell(metric.label)} | \`${metric.id}\` | ${metric.aggregation === "max" && metric.witness ? "max, named" : metric.aggregation} | ${cell((metric.languages ?? ["any file"]).join(", "))} | ${cell(metric.description)} |`),
]),
].join("\n") + "\n";
await writeFile("dist/metrics.md", metrics);
const site = `${origin}${base}`;
await writeFile("dist/llms.txt", `# Srcly

> Srcly is a local codebase map for developers and coding agents. It scans a repository into folders, files, functions and nested scopes, measures complexity, size, comment noise, React and Solid components, Git activity and lint findings, and shows them as a treemap, graphs and a 2x2 in the browser. Headless commands write the same analysis as Markdown and JSON.

Run it with \`uvx srcly\` (needs uv; Python 3.13+). Version ${meta.version}. Full analysis for TypeScript, JavaScript and Python; size and complexity for the other languages Lizard reads. A metric that does not apply is N/A, never 0.

For an agent, the usual loop is:

1. \`uvx srcly report . --out .srcly\`, then read \`.srcly/report.md\`.
2. \`uvx srcly explain . --at <folder | file | file::symbol>\` to drill in. Each answer ends with a \`Next:\` command.
3. \`uvx srcly compare --json\` to see what a change did, and \`uvx srcly check . --ratchet\` to stop big files from growing.

## Guides

${guides.map((guide) => `- [${guide.title}](${site}docs/${guide.slug}/index.md): ${guide.description}`).join("\n")}

## Reference

- [Metrics](${site}metrics.md): every metric with its ID, rollup, languages and definition.
- [Agent skill](https://raw.githubusercontent.com/byronwall/srcly/main/skills/srcly-code-quality/SKILL.md): the report and explain workflow as a skill file.
- [Source](https://github.com/byronwall/srcly): the repository.

## Optional

- [Everything in one file](${site}llms-full.txt): every guide and the metric reference.
`);
await writeFile("dist/llms-full.txt", [
`# Srcly documentation\n\nAll guides and the metric reference, from ${site}. Version ${meta.version}.`,
...guides.map((guide) => `\n---\n\nSource: ${site}docs/${guide.slug}/\n\n${guide.markdown.trim()}`),
`\n---\n\nSource: ${site}metrics.md\n\n${metrics.trim()}`,
].join("\n") + "\n");
console.log(`Prerendered ${pages.length} pages, docs entry point, and 404 page. Wrote llms.txt, llms-full.txt, metrics.md and ${guides.length} Markdown guides.`);
5 changes: 4 additions & 1 deletion site/src/App.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,10 @@ function DocsPage(props: { page: (typeof pages)[number]; base: string }) {
</div>
<div class="article-head">
<div><h1>{props.page.title}</h1><p class="article-summary">{props.page.description}</p></div>
<a class="edit-link" href={`https://github.com/byronwall/srcly/blob/main/site/src/docs/pages/${props.page.slug}.md`} target="_blank" rel="noreferrer">Edit page <ExternalLink size={13}/></a>
<div class="article-links">
<a class="edit-link" href={`${props.base}docs/${props.page.slug}/index.md`}>Markdown</a>
<a class="edit-link" href={`https://github.com/byronwall/srcly/blob/main/site/src/docs/pages/${props.page.slug}.md`} target="_blank" rel="noreferrer">Edit page <ExternalLink size={13}/></a>
</div>
</div>
<div class="article-grid">
<article class="markdown" innerHTML={contentHtml} ref={el => {
Expand Down
Loading
Loading