Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 6 additions & 22 deletions astro.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -22,29 +22,13 @@ export default defineConfig({
react(),
tailwind(),
sitemap({
filter: (page) => !page.includes('/admin'),
changefreq: 'weekly',
priority: 0.7,
filter: (page) =>
!page.includes('/admin') &&
!page.includes('/og/') &&
!page.includes('/og-image/') &&
!page.includes('/study/review') &&
!page.includes('/study/settings'),
lastmod: new Date(),
serialize(item) {
// Boost priority/lastmod for hub pages and canonical references
if (item.url === 'https://gemmology.dev/' ||
item.url === 'https://gemmology.dev/gallery/' ||
item.url === 'https://gemmology.dev/learn/') {
item.priority = 1.0;
item.changefreq = 'weekly';
} else if (/\/minerals\/[^/]+\/$/.test(item.url)) {
item.priority = 0.8;
item.changefreq = 'monthly';
} else if (/\/learn\//.test(item.url)) {
item.priority = 0.7;
item.changefreq = 'monthly';
} else if (/\/docs\//.test(item.url)) {
item.priority = 0.5;
item.changefreq = 'monthly';
}
return item;
},
}),
],
vite: {
Expand Down
87 changes: 87 additions & 0 deletions audits/seo-2026-05/v2-validation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# SEO v2 Validation Report

**Branch:** `feature/seo-v2`
**Built:** 2026-05-11 (commit chain after `c9171a4`)
**Validator:** automated JSON parse + required-field check (`/tmp/extract-jsonld.cjs`)

## Build assertions (W9)

| Assertion | Expected | Observed | Status |
|-----------|----------|----------|--------|
| Sitemap `<url>` count | ≈466 | 466 | PASS |
| Sitemap `/og/` URLs | 0 | 0 | PASS |
| Sitemap `/og-image` URLs | 0 | 0 | PASS |
| Sitemap `/study/` URLs | 0 | 0 | PASS |
| Sitemap `/admin` URLs | 0 | 0 | PASS |
| `/tools/` body words | ≥200 | 468 | PASS |
| `/tools/measurement/` body words | ≥300 | 1149 | PASS |
| `/tools/optical/` body words | ≥300 | 945 | PASS |
| `/tools/lab/` body words | ≥300 | 1213 | PASS |
| `/tools/identification/` body words | ≥300 | 902 | PASS |
| `/tools/advanced/` body words | ≥300 | 993 | PASS |
| `/tools/conversions/` body words | ≥200 | 600 | PASS |
| `/quiz/` body words | ≥250 | 429 | PASS |
| `/learn/` body words | rich | 3246 | PASS |
| `/about/` body words | ≥350 | 402 | PASS |

## JSON-LD inventory (W10)

Every block parses as valid JSON, contains `@context` and `@type`, and satisfies the required-field heuristics in our validator.

| Page | Blocks | Types | All pass? |
|------|--------|-------|-----------|
| `/` | 2 | `WebSite`, `Organization` | yes |
| `/about/` | 3 | `BreadcrumbList`, `WebPage`, `Person` | yes |
| `/learn/` | 3 | `Course`, `CollectionPage`, `BreadcrumbList` | yes |
| `/learn/fundamentals/chemical-properties/` | 2 | `LearningResource+Article`, `BreadcrumbList` | yes |
| `/learn/fundamentals/crystal-systems/` | 2 | `LearningResource+Article` (with `hasPart: Quiz`), `BreadcrumbList` | yes |
| `/quiz/` | 2 | `Course` (8 `hasCourseInstance`), `Quiz` | yes |
| `/tools/` | 1 | `SoftwareApplication` | yes |
| `/tools/measurement/` | 1 | `SoftwareApplication` | yes |
| `/tools/optical/` | 1 | `SoftwareApplication` | yes |
| `/tools/lab/` | 1 | `SoftwareApplication` | yes |
| `/tools/identification/` | 1 | `SoftwareApplication` | yes |
| `/tools/advanced/` | 1 | `SoftwareApplication` | yes |
| `/tools/conversions/` | 1 | `SoftwareApplication` | yes |
| `/minerals/diamond/` | 2 | `Thing`, `BreadcrumbList` | yes |
| `/minerals/ruby/` | 2 | `Thing`, `BreadcrumbList` | yes |

## Key fixes verified

- **Breadcrumb fragment URL bug (F-04, prior audit):** `/learn/fundamentals/chemical-properties/` now emits `"item": "https://gemmology.dev/learn"` (no `#fundamentals` fragment). Confirmed via direct grep.
- **Quiz `hasPart`:** present on `/learn/fundamentals/crystal-systems/` and `/learn/fundamentals/physical-properties/` (and any other article with a pretest).
- **Person schema:** `/about/` carries `Person` JSON-LD with `name: "Bissbert"`, `sameAs: ["https://github.com/Bissbert"]`, `knowsAbout` array of 4 gemmology topics, and `worksFor: Organization gemmology.dev`. No fabricated credentials.

## Heuristics applied by validator

- `Course` must have `provider`
- `LearningResource` must have `learningResourceType` or `educationalLevel`
- `BreadcrumbList` must have non-empty `itemListElement`
- `Person` must have `name`
- `Quiz` must have at least one of `about`, `hasPart`, `educationalLevel`
- `SoftwareApplication` must have `applicationCategory`

All 31 blocks across 15 representative pages satisfy these checks.

## External validation (manual, post-deploy)

After production deploy, run each template URL through Google's Rich Results Test
(https://search.google.com/test/rich-results) and Schema.org's structured-data linter
(https://validator.schema.org/) and append the screenshots / pass-fail markers here.

Templates to test:

- `https://gemmology.dev/`
- `https://gemmology.dev/about/`
- `https://gemmology.dev/learn/`
- `https://gemmology.dev/learn/fundamentals/crystal-systems/` (article with pretest)
- `https://gemmology.dev/quiz/`
- `https://gemmology.dev/tools/`
- `https://gemmology.dev/tools/measurement/`
- `https://gemmology.dev/minerals/diamond/`

## Outcome

Wave 2 verification **PASSES**. Branch is ready for PR and deploy. Background workstreams WB2/WB3
(llms.txt + robots.txt) and WB1 (learn intro expansion in `gemmology-knowledge`) are tracked
separately.
55 changes: 52 additions & 3 deletions public/robots.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,69 @@

User-agent: *
Allow: /
Disallow: /api/
Disallow: /admin/

# Sitemap location
Sitemap: https://gemmology.dev/sitemap-index.xml

# LLM crawler index (https://llmstxt.org/)
LLM-Content: https://gemmology.dev/llms.txt

# Disallow API endpoints (served by separate worker)
# Named AI-crawler stanzas.
# Several AI agents (notably PerplexityBot) require an explicit Allow per
# named User-agent in addition to the wildcard rule above before content
# is eligible for answer-engine surfacing.

# OpenAI ChatGPT / GPTBot — training crawler
User-agent: GPTBot
Allow: /
Disallow: /api/
Disallow: /admin/

# OpenAI on-demand search crawler (ChatGPT browsing / SearchGPT)
User-agent: OAI-SearchBot
Allow: /
Disallow: /api/
Disallow: /admin/

# Anthropic Claude — training/answer crawler
User-agent: ClaudeBot
Allow: /
Disallow: /api/
Disallow: /admin/

# Disallow admin surface
# Anthropic on-demand fetcher (claude.ai citations)
User-agent: Claude-Web
Allow: /
Disallow: /api/
Disallow: /admin/

# Perplexity AI answer engine — requires named Allow per their docs
User-agent: PerplexityBot
Allow: /
Disallow: /api/
Disallow: /admin/

# Google AI Overviews / Gemini training opt-in
User-agent: Google-Extended
Allow: /
Disallow: /api/
Disallow: /admin/

# Common Crawl — feeds many AI training corpora
User-agent: CCBot
Allow: /
Disallow: /api/
Disallow: /admin/

# Apple Intelligence on-demand fetcher
User-agent: Applebot-Extended
Allow: /
Disallow: /api/
Disallow: /admin/

# Allow all crawlers to access static assets
# Static assets are explicitly allowed for all crawlers above.
Allow: /crystals/
Allow: /og-images/
Allow: /favicon.svg
57 changes: 48 additions & 9 deletions src/components/seo/LearnSchema.astro
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@
* Improves SEO and enables rich results in search engines
*/

interface PretestQuestion {
questionText: string;
correctAnswer: string;
}

interface Props {
title: string;
description: string;
Expand All @@ -15,9 +20,25 @@ interface Props {
datePublished?: string;
/** ISO 8601 datetime — when the article was last meaningfully updated. */
dateModified?: string;
/** Number of pretest questions available for this article. */
pretestQuestionCount?: number;
/** Pretest question data for schema serialisation. */
pretestQuestions?: PretestQuestion[];
}

const { title, description, category, subcategory, difficulty, slug, datePublished, dateModified } = Astro.props;
const {
title,
description,
category,
subcategory,
difficulty,
slug,
datePublished,
dateModified,
pretestQuestionCount = 0,
pretestQuestions = [],
} = Astro.props;

const url = `https://gemmology.dev/learn/${slug}`;

// Helper to format category names
Expand All @@ -28,6 +49,22 @@ function formatName(name: string): string {
.join(' ');
}

// Build hasPart Quiz node when pretest questions are available
const quizNode = pretestQuestionCount > 0
? {
"@type": "Quiz",
"name": `Pretest: ${title}`,
"educationalLevel": "Intermediate",
"numberOfQuestions": pretestQuestionCount,
"about": { "@type": "Thing", "name": title },
"hasPart": pretestQuestions.map(q => ({
"@type": "Question",
"name": q.questionText,
"acceptedAnswer": { "@type": "Answer", "text": q.correctAnswer },
})),
}
: null;

// Use combined LearningResource + Article so Google can pick either rich-result type
const articleSchema = {
"@context": "https://schema.org",
Expand Down Expand Up @@ -64,34 +101,36 @@ const articleSchema = {
},
...(datePublished ? { "datePublished": datePublished } : {}),
...(dateModified ? { "dateModified": dateModified } : {}),
...(quizNode ? { "hasPart": quizNode } : {}),
};

// Build breadcrumb from category/subcategory
// Build BreadcrumbList — no fragment URLs (Google rejects them).
// Chain: Home → Learn → [subcategory overview if present →] Article
const breadcrumbItems = [
{ "@type": "ListItem", "position": 1, "name": "Home", "item": "https://gemmology.dev" },
{ "@type": "ListItem", "position": 2, "name": "Learn", "item": "https://gemmology.dev/learn" },
{ "@type": "ListItem", "position": 3, "name": formatName(category), "item": `https://gemmology.dev/learn#${category}` },
];

if (subcategory) {
// Position 3: article page (subcategory is visible-only; no clean URL to use here)
breadcrumbItems.push({
"@type": "ListItem",
"position": 4,
"position": 3,
"name": formatName(subcategory),
"item": url
"item": url,
});
breadcrumbItems.push({
"@type": "ListItem",
"position": 5,
"position": 4,
"name": title,
"item": url
"item": url,
});
} else {
breadcrumbItems.push({
"@type": "ListItem",
"position": 4,
"position": 3,
"name": title,
"item": url
"item": url,
});
}

Expand Down
48 changes: 46 additions & 2 deletions src/components/seo/StructuredData.astro
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
/**
* StructuredData - emit a BreadcrumbList plus optional WebPage JSON-LD.
* StructuredData - emit a BreadcrumbList, optional WebPage, and optional Person JSON-LD.
* Use for non-mineral / non-learn pages that still benefit from breadcrumbs.
*/

Expand All @@ -9,16 +9,27 @@ interface Crumb {
url: string;
}

interface PersonData {
name: string;
jobTitle?: string;
credentials?: string[];
sameAs?: string[];
}

interface Props {
breadcrumbs?: Crumb[];
webPage?: {
name: string;
description?: string;
url: string;
};
/**
* Optional Person JSON-LD — used on /about to signal E-E-A-T authorship.
*/
person?: PersonData;
}

const { breadcrumbs = [], webPage } = Astro.props;
const { breadcrumbs = [], webPage, person } = Astro.props;

const breadcrumbSchema = breadcrumbs.length > 0
? {
Expand Down Expand Up @@ -48,7 +59,40 @@ const webPageSchema = webPage
},
}
: null;

const personSchema = person
? {
"@context": "https://schema.org",
"@type": "Person",
name: person.name,
...(person.jobTitle ? { jobTitle: person.jobTitle } : {}),
knowsAbout: [
"Gemmology",
"Mineralogy",
"Crystallography",
"Gemstone identification",
],
...(person.credentials && person.credentials.length > 0
? {
hasCredential: person.credentials.map((c) => ({
"@type": "EducationalOccupationalCredential",
credentialCategory: "certificate",
name: c,
})),
}
: {}),
...(person.sameAs && person.sameAs.length > 0
? { sameAs: person.sameAs }
: {}),
worksFor: {
"@type": "Organization",
name: "gemmology.dev",
url: "https://gemmology.dev",
},
}
: null;
---

{breadcrumbSchema && <script type="application/ld+json" set:html={JSON.stringify(breadcrumbSchema)} />}
{webPageSchema && <script type="application/ld+json" set:html={JSON.stringify(webPageSchema)} />}
{personSchema && <script type="application/ld+json" set:html={JSON.stringify(personSchema)} />}
Loading
Loading