Granite Document Intelligence

Granite Document Intelligence turns multi-page PDFs into structured output on your Mac. It runs IBM's Granite Docling and Granite Vision models locally with MLX. It is a Python library with a command-line tool, gdi, and a local web UI, gdi ui (see The UI). For each PDF it writes:

• the whole document as Markdown and as lossless DoclingDocument JSON
• charts as CSV data, a one-paragraph summary and the cropped image
• tables as CSV and HTML
• key-values filled in against a JSON Schema you supply (optional)
• RAG-ready chunks that follow the headings and carry page numbers, as JSON Lines

Watch the demo: a 75-second walkthrough of gdi convert and gdi ui on GAO-07-631, the report the examples below use, or the 40-second Short.

Version 1.0.0 is the PDF pipeline's first release. Releases before it, up to v0.6.0, are the earlier Streamlit text-analysis app, which this project replaces.

Requirements

• An Apple Silicon (M-series) Mac with 16 GB of unified memory. Everything was measured on a 16 GB MacBook Pro M2 Pro. The run peaks at about 6.7 GB of model memory and is designed to stay under 10 GB.
• uv, which installs Python 3.12 and every dependency.
• About 16 GB of free disk for setup and about 7.5 GB afterwards (see Disk space).

The models run on MLX, but the project is not MLX-only. Docling imports PyTorch, and its picture classifier runs on PyTorch, so uv sync installs it.

Setup

uv sync
uv run gdi convert-model

gdi convert-model is a one-time step, and it took 3.3 minutes on a fast connection when it was measured on 2026-10-29, including the ~9 GB download. It downloads three models at pinned revisions: granite-docling, Docling's figure classifier and the Granite Vision 4.1 source. IBM publishes no MLX build of Granite Vision 4.1, so the step converts it to 8-bit MLX on your Mac and removes the Python code that ships in the model repository. Everything goes into ~/.cache/granite-document-intelligence/models/; set GDI_CACHE_DIR to put it somewhere else. Afterwards, the 8 GB bf16 source can be deleted (see Disk space).

gdi convert refuses to start until this step has completed. It never falls back to downloading models.

The models are public, so a Hugging Face token is optional. Without one, the Hub applies lower rate limits and logs an "unauthenticated requests" warning. To use one, copy .env.example to .env and set HF_TOKEN to a token with read scope (create one).

Usage

This example converts the PDF the project's quality evaluation runs on. It is GAO report GAO-07-631, 42 pages with nine charts, and it is public domain:

curl -L -o gao-07-631.pdf https://www.govinfo.gov/content/pkg/GAOREPORTS-GAO-07-631/pdf/GAOREPORTS-GAO-07-631.pdf
uv run gdi convert gao-07-631.pdf -o out

It writes out/gao-07-631/ and took 4.5 minutes when it was measured. Its run.json warns that page 24 hit the token cap, a known loss on this PDF. Progress goes to stderr, and a summary of what was done, skipped and failed comes last.

To pull named fields out of a page, pass a JSON Schema. This one reads the report's cover (page 1, the default for --pages):

cat > cover.json <<'EOF'
{
  "type": "object",
  "properties": {
    "report_number": {"type": "string", "description": "The GAO report number, e.g. GAO-00-000"},
    "date": {"type": "string", "description": "The month and year printed on the cover"},
    "title": {"type": "string", "description": "The report's title"}
  },
  "required": ["report_number", "date", "title"]
}
EOF
uv run gdi convert gao-07-631.pdf -o out --schema cover.json --force

--force is needed here because the PDF was already converted. Without it, a PDF whose run.json records success for the same file and options is skipped. Changing an option such as --schema also converts it again.

gdi convert INPUT [INPUT ...] -o OUT_DIR

Option	What it does
INPUT	A PDF, or a directory of PDFs. Each PDF goes to OUT_DIR/<its path relative to the directory>/.
--recursive	Also convert PDFs in subdirectories.
--schema FIELDS_JSON	Fill the schema's fields from the selected pages into kvp.json.
--pages 1,3-4|all	The pages to extract key-values from (default 1). Needs --schema.
--text-source pdf	Take text from the PDF's own text layer instead of the model. Use it only for born-digital PDFs you trust.
--no-charts, --no-chunks	Skip chart extraction, or chunks.jsonl.
--chunk-tokens N	Chunk size in tokens, 64–8192 (default 512).
--chart-code	Also write each chart as Python. This is untrusted model output, so never run it.
--save-page-images	Also write a PNG of every page.
--force	Convert PDFs again even when they are already done.
--max-pages N, --max-file-mb MB	Refuse larger PDFs (defaults 500 pages and 200 MB).
--quiet	Print only warnings and the summary.
--no-wait	Exit instead of waiting when another gdi run is in progress.
--events PATH	Also append progress to PATH as JSON Lines, for a program to follow (see Progress events).

gdi convert exits with 0 when every PDF was converted or skipped, 1 when any PDF failed or the run could not start, 2 on a usage error and 130 on Ctrl-C. A failed PDF records its error in its run.json, and the batch continues.

Only one gdi run works at a time on a Mac, because two runs would each load the models and push 16 GB into swap. A second run waits for the first to finish. With --no-wait, it exits with 1 instead.

Resume is per PDF. A batch interrupted by Ctrl-C, a closed lid or a reboot keeps every PDF it finished and redoes the one it was working on, from its first page. Page-level resume is not possible through Docling's public API yet.

Progress events

--events PATH appends one JSON object per line to PATH as the run goes, for a program that drives gdi to follow. The progress on stderr is unchanged. Each line is flushed before the run moves on, so a reader can follow the file while it grows. A batch of one PDF with a chart looks like this:

{"v": 1, "event": "batch", "total": 1}
{"v": 1, "event": "pdf", "position": 1, "total": 1, "pdf": "report.pdf", "out_dir": "out/report"}
{"v": 1, "event": "page", "completed": 1, "total": 2}
{"v": 1, "event": "page", "completed": 2, "total": 2}
{"v": 1, "event": "chart", "position": 1, "total": 1, "page_no": 2}
{"v": 1, "event": "result", "pdf": "report.pdf", "status": "done"}
{"v": 1, "event": "summary", "done": 1, "skipped": 0, "failed": 0}

Event	When	Fields
waiting	Another gdi run holds the lock, and this one waits for it.	holder_pid, or null when unknown
batch	This run holds the lock and starts.	total: the number of PDFs
pdf	A PDF starts.	position, total, pdf, out_dir
page	granite-docling finished a page.	completed, total
chart	Granite Vision starts on a chart.	position, total, page_no
kvp	Key-value extraction starts on a page.	position, total, page_no
result	A PDF finished.	pdf, status: done, skipped or failed
error	The run could not start, or stopped as a whole.	kind: usage (exit 2), models or locked (exit 1); message
interrupted	Ctrl-C (exit 130).	none
summary	Every PDF was done, skipped or failed (exit 0 or 1).	done, skipped, failed

Reading the file:

• Every line carries "v": 1, the format's version. Skip any line you cannot parse or whose v you do not know.
• A run ends with one summary, interrupted or error. A file that ends without one belongs to a run that is still going or that crashed: a crash or a kill -9 writes nothing. A pdf with no result after it is the PDF the run was working on.
• A failed result has no message, because an error's text can quote the PDF. The error is in that PDF's run.json. The events hold counts and paths, never text from a PDF or a model.
• The file is appended to, never replaced, so give each run its own path.
• If the file cannot be written, gdi prints one warning and carries on without it. A conversion never fails because of its events.
• A command line that gdi rejects while parsing it, such as an unknown flag or --max-pages 0, exits with 2 before the file is opened and writes nothing to it.

Output

out/<pdf-stem>/
  document.md
  document.json              # DoclingDocument, lossless; images referenced, not embedded
  tables/table-001.csv
  tables/table-001.html
  charts/chart-001.png       # the crop Granite Vision read
  charts/chart-001.csv       # absent when the model wrote no table; then:
  charts/chart-001.raw.txt   #   what it wrote instead
  charts/chart-001.summary.txt
  charts/chart-001.py        # only with --chart-code
  pictures/picture-001.png   # pictures that are not charts
  pages/page-001.png         # only with --save-page-images
  kvp.json                   # only with --schema
  chunks.jsonl
  run.json

run.json records how the output was made: the model revisions, package versions, timings for each phase and page, peak memory, the output's size and any warnings. It is written last, and it is what marks a PDF as done. Each PDF's folder is built beside the output and swapped into place in one step. So a re-run never leaves files from an earlier run behind, and an interrupted run leaves the previous output intact. gdi only ever replaces a folder it wrote itself. If a folder already sits where a PDF's output would go and holds anything else (your own files, or a note you added to an output folder), that PDF fails with ForeignFolderError and the folder is left untouched. Versions before 1.0.1 replaced it.

Warnings worth reading:

• A page that hit the model's 8192-token limit, or that repeated itself, is listed by page number. Such a page is usually lost or garbled.
• A chart whose summary pairs a number with the wrong category is flagged against the chart's own CSV.

kvp.json holds:

• the values read from each page, and the values merged across pages
• any conflicts between pages
• whether the result validates against your schema, and the validation errors if it does not
• the model's raw responses

Python API

The CLI wraps a library, which can be used directly:

from pathlib import Path

from granite_document_intelligence import ConvertOptions, convert

opts = ConvertOptions(schema=Path("cover.json"), pages="1")
result = convert(Path("gao-07-631.pdf"), options=opts)

for chart in result.charts:
    print(chart.page_no, chart.summary)
print(result.kvp.merged if result.kvp else None)
print(len(result.chunks), "chunks;", result.run.peak_memory_bytes)

result.save(Path("out/gao-07-631"))  # writes the tree above

convert() returns data and writes nothing; save() is the only call that writes. The result holds:

Field	Contents
document	The DoclingDocument
charts	Each chart's page, CSV, raw CSV, summary and image
pictures	The pictures that are not charts
tables	Each table as a pandas DataFrame and as HTML
kvp	The key-value result, or None without a schema
chunks	The chunks
run	What run.json records

convert_many(inputs, out_dir, options) is the batch form that gdi convert uses. It is a generator, so it does nothing until you iterate it, and it yields one BatchItem per PDF whose status is done, skipped or failed. It also handles resume and saving.

Both functions take the run lock described above; pass wait=False to raise RunLockedError instead of waiting. Before any model loads, both raise ModelsNotReadyError if gdi convert-model has not completed.

The UI

gdi ui does what gdi convert does, in a browser on your Mac: pick PDFs and settings, convert them with progress and cancel, and browse the results of this run or any earlier one.

uv run gdi ui

It serves the UI on this Mac only, at http://localhost:8501/, and opens it in your browser. Ctrl-C in the terminal stops it. --port N serves on another port, and a port in use is an error rather than moved. --no-browser prints the address without opening a browser. The UI is themed to follow your system's light or dark setting.

uv sync installs Streamlit, which the UI needs. In an install without it, gdi ui prints the command that adds it (the ui extra) and exits with 1. The library and gdi convert never need Streamlit.

Converting.

• Give a PDF or a folder of PDFs by its path, or upload PDFs. A folder's subfolders are not converted. Uploads stop at 200,000,000 bytes, because the browser counts a megabyte as 1,000,000 bytes. gdi convert counts 1,048,576 to the megabyte, so its 200 MB limit is about 9.7 MB higher, and a PDF in that gap has to be given by its path.
• Results go to the Output folder, ~/gdi-out by default. It is not under ~/Documents, because macOS asks for permission before an app writes there.
• Each sidebar setting is one gdi convert option, so a conversion from the UI is the same as one from the terminal. The Run view shows the exact command it ran. Convert again is --force.
• Resume works as in the terminal: a PDF already converted with the same settings is skipped. Resume compares the schema's path as well as its content, so the same schema at another path converts again.
• Cancel asks once more before it stops the conversion. Like Ctrl-C, it keeps every PDF that finished and leaves the previous results of the current PDF untouched.
• Closing the tab, or stopping the server, does not stop a conversion. Open the UI again to follow it.
• One conversion runs at a time. A gdi convert started in a terminal makes the UI's conversion wait, and the UI says so.

Browsing. The PDF list holds every folder under the output folder that has a run.json, newest first. Six views show the selected PDF: Document, Tables, Charts (each chart's crop, CSV, summary and warnings), Key-values, Chunks and Run (timings, peak memory, warnings, model revisions and the command line). A failed PDF opens on the Run view with its error.

What it keeps. Each conversion's progress, logs and command line go to ~/.cache/granite-document-intelligence/ui/runs/, and uploaded files to ui/inputs/ beside it. The UI deletes both after 30 days. It never deletes anything in the output folder.

The UI runs each conversion as a separate gdi convert process, so the models' memory goes back when a conversion ends and a crash cannot take the UI down. With GAO-07-631's results open, the UI's server used 0.14 GiB of memory, and at most 0.5 GiB while it held a 200 MB upload. Measured on 2026-10-30 beside a conversion of the same PDF, the conversion's peak stayed the same, nothing went to swap and nothing slowed down.

Performance

These figures are from a 16 GB M2 Pro on AC power, measured on 2026-10-27 with docling 2.143.1, mlx 0.32.4 and mlx-vlm 0.7.10. They are the medians of three runs of the project's evaluation corpus.

Reading pages	~5.7 s per page, about 10 pages a minute sustained
Each chart (CSV and summary)	~8.3 s
Each key-value page	~6.0 s
Loading the models	~7 s for the first PDF of a run
Peak model memory	6.68 GB

A page where the model falls into a repetition loop costs 30–40 s, and its text is usually lost. Such a page shows up in run.json's warnings. These figures are for born-digital PDFs. A scanned page reads as fast as a born-digital one when it converts cleanly. On the 15-page scan in the evaluation corpus (300 DPI, grayscale), reading averaged 7.0 s per page on 2026-10-29, because two of its pages looped.

Disk space

What	Where	Size
Python environment (includes PyTorch)	.venv/	1.9 GB
granite-docling and the figure classifier	~/.cache/granite-document-intelligence/models/	0.67 GB
Granite Vision, 8-bit	the same folder	4.94 GB
Granite Vision bf16 source	the Hugging Face cache	8.0 GB, needed only to convert
Chunking tokenizer	the Hugging Face cache	small

Output is small next to the models: roughly 25–60 KB per page by default, and 160–190 KB per page with --save-page-images.

gdi convert-model ends by printing the size of the bf16 source and the command that removes it. Delete it afterwards with:

uv run hf cache rm model/ibm-granite/granite-vision-4.1-4b

Alternatively, run uv run gdi convert-model --delete-source, which deletes the source once the 8-bit model has loaded. Keep the source if you expect to convert again, since downloading it again takes 8 GB.

Untrusted documents

Treat every PDF as untrusted, and everything derived from one, because a PDF can be malformed, hostile or simply huge.

• Limits. PDFs over 500 pages or 200 MB are refused before conversion (--max-pages, --max-file-mb). A file is processed only if it starts with %PDF-, whatever its extension.
• No timeout. A PDF that hangs a parser hangs the run, and only Ctrl-C stops it. There is no per-PDF timeout or process isolation in this version.
• Parsers run in-process. PDFs are parsed by pdfium and docling-parse, which are C and C++ code. They run with your privileges and without a sandbox. Keep the dependencies current, and do not use this tool on strangers' PDFs at volume.
• Text in a page can try to steer the model. That includes text you cannot see, such as white text on white. The pipeline treats model output as data, never as instructions, and schema validation rejects off-schema output. Still, review key-values from an untrusted document before anything acts on them.
• Generated chart code is never run. --chart-code writes model output that starts with an "Untrusted: review before running" header. Nothing in this package runs it, and you should read it before you do.
• Outputs are files, not web pages. table-NNN.html and document.md contain text from the PDF. Escape them before rendering them in a browser or UI of your own.
• The UI fetches nothing a PDF names. Images and links in document.md and in chart summaries are reduced to text, except images inside the PDF's own output folder. Every other string from a PDF or a model is shown as plain text. Generated chart code is shown, never run, and the UI has no way to run it.
• The UI listens on this Mac only (127.0.0.1). Exposing it to a network is unsupported. Other web apps on localhost can reach it, and other web pages can frame it; SECURITY.md lists these limits.
• Network. The only network access is to the Hugging Face Hub, and only to download models. To run fully offline once the models are cached, set HF_HUB_OFFLINE=1.

Output that a model extracts from a copyrighted PDF is still derived from that PDF. Being able to extract it does not make it free to reuse.

To report a vulnerability, see SECURITY.md.

Development

uv run ruff check .    # lint
uv run ruff format .   # format
uv run ty check        # typecheck
uv run pytest          # test

All four run in CI on every push to main and every pull request, on an Apple Silicon (macOS) runner. They also run as scripts/gate.sh, which the project's Claude Code Stop hook invokes. The unit tests mock every model, so they need no download. The quality evaluation (eval/score.py) runs the real models on a corpus of public PDFs and is run by hand, never in CI.

Contributions are welcome — open an issue or a pull request. Run the four checks above before submitting.

Releases

CI cuts releases; nobody tags by hand. Bump version in pyproject.toml, run uv lock, and commit both files to main. Once the checks pass, CI tags v<version> and publishes a release with generated notes. A push that does not change the version does nothing. A commit carrying pyproject.toml without uv.lock fails CI at uv sync --locked, so it cuts no release.

License

The project's code is licensed under the Apache License 2.0. The key-value extraction prompt is copied from Docling under the MIT License; THIRD_PARTY_NOTICES.md carries that notice.

The models are downloaded at setup and are not part of this repository. Each is under its own license:

• Apache-2.0: granite-docling-258M, Granite Vision 4.1 and granite-embedding-english-r2.
• MIT: Docling's DocumentFigureClassifier.

The 8-bit Granite Vision that setup builds is a modified version of Apache-2.0 weights. It stays on your Mac. Anyone who publishes it must include the license and a notice that it was quantized.

The evaluation corpus keeps its own licenses. They are listed in eval/corpus/SOURCES.md, which arrives with the corpus itself.