Skip to content

About

polytypo — normalize typography for Python (quotes, dashes, ellipses, no-break spaces). Issues: github.com/polytypo/polytypo/issues

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

59 Commits

Folders and files

Repository files navigation

polytypo

polytypo

PyPI version CI PyPI downloads License: MIT

Locale-correct quotes, dashes, ellipses, apostrophes, symbols and no-break spaces —
one portable spec, designed for byte-identical output across runtimes.

Try it live, no install: polytypo.dev

This is the Python implementation. The full spec — all locales, all rules, worked examples in each — lives in polytypo/polytypo. This runtime supports the text, html and yaml modes fully, and markdown for the commonmark dialect only — mdx raises POLYTYPO_INVALID_DIALECT (no MDX/JSX parser is available for Python; see Supported dialects).

Install

pip install polytypo

Usage

from polytypo import transform

transform("She said, \"it's fine\" -- but I wasn't sure...", locale="en-US")
# She said, “it’s fine”—but I wasn’t sure…

Same input, one locale changed — quotes, dash spacing and all follow the target locale, not a single hardcoded style:

transform('Sie sagte: "Alles gut" -- aber ich war mir nicht sicher...', locale="de-DE")
# Sie sagte: „Alles gut“ – aber ich war mir nicht sicher…

HTML and Markdown are first-class modes, not an afterthought — tags, attributes and fenced code are left alone; only text content is touched:

from polytypo.html import transform

transform('<a title="test... wait">Wait... she said "go on."</a>', locale="en-US")
# <a title="test... wait">Wait… she said “go on.”</a>

html mode leaves ten elements alone, with everything inside them: code, pre, kbd, samp, var, script, style, textarea, svg and math. If you walk a document with a parser of your own, read the list instead of copying it: polytypo.html.SKIPPED_ELEMENTS is a tuple.

from polytypo.html import SKIPPED_ELEMENTS

polytypo.text, polytypo.html, polytypo.markdown and polytypo.yaml each exclude the parser dependency the other modes don't need (importing polytypo.text never imports html.parser-based span extraction or tree-sitter; polytypo.yaml needs no parser at all). The aggregate polytypo module supports every mode via a mode keyword, defaulting to "text". locale has no default anywhere and must always be passed explicitly — there is no silent fallback to English.

import polytypo

polytypo.transform("...", locale="fr", mode="markdown", dialect="commonmark")

yaml mode is the one that asks something of you, and it asks for a reason. YAML is a data format with prose in some of it, so you name the keys whose values are prose; there is no default and no guess:

from polytypo.yaml import transform

transform(
    "summary: Rates -- all of them...\nrun: git diff -- a--b\n", locale="en-US", keys=["summary"]
)
# summary: Rates—all of them…
# run: git diff -- a--b

Nothing in YAML's syntax separates a sentence from a shell script: description holds one and run holds the other, spelled identically. Quoting, indentation, anchors and a block scalar's chomping indicator are never decoded and rewritten — the file is located, not re-emitted — so the trailing newlines of a |+ block come back exactly as you wrote them. An empty keys list is legal and processes nothing.

In markdown mode frontmatter is skipped whole by default, delimiters included — it is a machine-read block, and fr would put a narrow no-break space in front of the colon of every field in it. On a site whose frontmatter carries the headline that leaves the most visible string on the page untouched, so name the keys you want processed:

from polytypo.markdown import transform

transform(
    '---\ntitle: He said "hello" once\nslug: "he-said-hello"\n---\n\nThe body was typeset all along.\n',
    locale="en-US",
    dialect="commonmark",
    frontmatter_keys=["title"],
)
# ---
# title: He said “hello” once
# slug: "he-said-hello"
# ---
#
# The body was typeset all along.

Without the option nothing in the block changes, and a key you do not name never does. The block is read with the same scanner yaml mode uses, so it refuses the same constructs — a single-quoted scalar containing '' among them, which is how an apostrophe is written inside single quotes, so title: 'It''s a test' comes back untouched. Double-quoted and plain scalars have no such limit. The block is also processed separately from the body, which shows up in exactly one place: an unbalanced quotation mark in title cannot pair with a mark in your first paragraph.

protect is the option for text that has holes no mode knows about: a template literal with ${…}, an ICU message, a Django or Handlebars template. It is text mode only. You pass the ranges to leave alone, as (start, end) pairs of code-point offsets into the string you pass, half-open, ascending and non-overlapping; touching ranges are one. The rules see each range as opaque content, so the text around it keeps its context, and the range itself comes back byte for byte:

from polytypo.text import transform

transform("${x} - Rules", locale="en-US", protect=[(0, 4)])  # '${x}—Rules'
transform("${x} - Rules", locale="en-GB", protect=[(0, 4)])  # '${x} – Rules'

None and [] are the same as leaving it out. A malformed range, one past the end of the input (counted in code points, not UTF-16 units or bytes), or any non-empty protect in another mode raises POLYTYPO_INVALID_OPTION. A text with protect converts less than the same characters without it, never more: a conversion that would add code points at the edge of a hole is not made. The offsets refer to the input as passed, so to run the output through transform again you move the ranges yourself. analyze() takes the same option, with the same validation. Full contract: spec/rules/modes.md 3.9.

analyze() runs the same pipeline and reports what it would do instead of doing it — one record per edit, each with the rule that made it and code-point offsets into the input you passed (into the document, in html, markdown and yaml mode, not into a span):

import polytypo

polytypo.analyze('Wait... "really"?', locale="en-US")
# [Change(rule_id='ellipsis', start=4, end=7, before='...', after='…'),
#  Change(rule_id='quotes', start=8, end=9, before='"', after='“'), …]

It is a report, not a patch. The list is empty exactly when transform would return the input unchanged, and every rule_id is a rule that was enabled for that call — but two rules may touch the same original range (French spaces deletes the space before : and nbsp puts a no-break one back), so replaying the list is not guaranteed to reproduce the output. Call transform for the text. Full contract: spec/rules/analyze.md.

Supported dialects

markdown mode requires a dialect keyword, exactly as the spec requires (no default, detection is forbidden). This runtime supports dialect="commonmark" (CommonMark plus GFM — tables, strikethrough, task lists, autolink literals). dialect="mdx" is a real dialect the spec names, but this runtime has no MDX/JSX parser for it and raises POLYTYPO_INVALID_DIALECT immediately rather than silently mishandling it — a narrower, honest conformance claim, not a port defect.

Licence

MIT. See LICENSE.

About

polytypo — normalize typography for Python (quotes, dashes, ellipses, no-break spaces). Issues: github.com/polytypo/polytypo/issues

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages