Skip to content

ipynb exporter: refactor markdown flavor handling (transform + serializer config) #48

Description

@mmcky

Context

The fork's myst-to-ipynb has a commonmark mode that down-converts MyST AST nodes to plain mdast (packages/myst-to-ipynb/src/commonmark.ts) and then serializes with writeMd from myst-to-md. This works well, but a design discussion surfaced several improvements worth capturing.

Key insight: we do not need to build standalone CommonMark/GFM renderers. myst-to-md is already built on mdast-util-to-markdown (the unified ecosystem's CommonMark serializer), which is fully pluggable along three axes:

  • handlers — per-node-type serialization functions. This is how myst-to-md adds MyST syntax today (directiveHandlers, roleHandlers, …).
  • extensions — bundles of handlers plus escaping/join rules, shipped per flavor feature: mdast-util-gfm-table, mdast-util-gfm-footnote, mdast-util-gfm-strikethrough (or the mdast-util-gfm aggregate), and notably mdast-util-math.
  • Style optionsfences, bullet, rule, etc.

So a "renderer for flavor X" is not a package, it's a configuration object:

  • CommonMark = default handlers only
  • GFM = defaults + gfm extensions
  • MyST = all of that + the MyST handlers

Markdown flavors differ mostly in what the AST is allowed to contain, not in how serialization works. A flavor therefore decomposes into:

  1. a serializer configuration — which toMarkdown extensions/handlers are active, covering everything the flavor can express (tables, footnotes, math, code, images), and
  2. a down-transform that eliminates only the nodes the flavor has no syntax for (admonitions, tabs, cards, proofs, exercises) — where "what should this become" is a content decision, not a serialization one.

The transform-then-serialize pipeline we already have is the right architecture; the recommendation below refines it rather than rebuilds it.

Recommendation

1. Name the target flavor honestly: it's GFM + math, not CommonMark

Jupyter's markdown renderers (marked in classic Notebook, markdown-it in JupyterLab, plus MathJax) target roughly GFM + $-math, not strict CommonMark. Our "commonmark" mode already emits GFM tables (myst-to-md's table extension is always on) and $$ math via raw nodes.

  • Rename/alias the option so flavors are myst | gfm (or gfm-jupyter) rather than myst | commonmark.
  • Being explicit clarifies that GFM features (footnotes, strikethrough, tables) are fair game in exported notebooks.

2. Let the serializer do what it can; shrink the down-transform

The current transform works around the serializer in places where it could configure it instead:

  • MathtransformMathBlock / transformInlineMath smuggle LaTeX through fake html nodes to dodge underscore escaping. mdast-util-math's mathToMarkdown() serializes math/inlineMath nodes natively to $$…$$ / $…$ with correct escaping — exactly Jupyter's dialect. Adding it to the flavor's extension set deletes both hacks. (Caveat: myst-to-md pins mdast-util-to-markdown ^1.5.0, so we need mdast-util-math 2.x, not 3.x.)
  • Code and image strippingtransformCodeBlock / transformImage exist only to stop the MyST-flavored handlers from emitting {code-block} / {image} directives. If the flavor config simply omits the MyST handlers, the default CommonMark handlers serialize these nodes correctly and those transforms disappear too.

After this, the down-transform handles only nodes with no target syntax — admonitions, tabs, cards, proofs, exercises.

3. Extract the down-transform out of myst-to-ipynb

"Flatten MyST AST to flavor-X-expressible mdast" is useful beyond notebooks — e.g. exporting plain .md that renders well on GitHub would want exactly the same logic.

  • Move transformToCommonMark into myst-transforms (or a small myst-downgrade module).
  • Parameterize it by a flavor capability profile, e.g. { tables: true, math: 'dollar', footnotes: true }. The profile selects both the extension set passed to toMarkdown and which down-transform rules fire.
  • myst-to-ipynb then becomes a thin consumer: cell-splitting + flavor transform + writeMd.
  • This shape is also the most plausible candidate for upstreaming to jupyter-book/mystmd, versus new renderer packages.

4. Expose flavor on writeMd / the md exporter

  • Add writeMd(file, tree, frontmatter, { flavor }) which selects the transform + extension set internally.
  • This gives users something like myst build --md --flavor gfm for free, with the ipynb writer calling the same code path.

Non-goals

  • Building standalone myst-to-commonmark / myst-to-gfm renderer packages — flavors as separate packages would duplicate the handler machinery of mdast-util-to-markdown and drift apart; as configuration objects inside myst-to-md they share all machinery by construction.

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions