Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,18 @@

gtfparse
========
Parsing tools for GTF (gene transfer format) files.

Read GTF (gene transfer format) annotations into a pandas DataFrame, with one
row per feature and attribute values such as gene and transcript IDs expanded
into columns.

## Install

```sh
python -m pip install gtfparse
```

Python 3.9 or later is required. Version 3 uses pandas and PyArrow; Polars
output is optional and requires `gtfparse[polars]`.
Python 3.9 or later is required. For optional Polars output, install
`gtfparse[polars]`.

## Read an annotation and find genes

Expand Down Expand Up @@ -62,5 +63,6 @@ it is not the gene ID. The ninth GTF field supplies attributes such as
- [Version 3 migration and optional Polars output](docs/guides/migration.md)
- [Complete API reference](docs/reference.md)

Build with `python -m pip install -r requirements-docs.txt` and `./docs.sh`.
Run `python scripts/check_docs_examples.py` to verify the printed examples.
To build the documentation locally, run
`python -m pip install -r requirements-docs.txt` and `./docs.sh`.
`python scripts/check_docs_examples.py` checks the printed example output.
43 changes: 23 additions & 20 deletions docs/guides/progress.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,10 @@
# Report parsing progress

Pass `progress_callback(stage, completed, total)` to `read_gtf` to connect
parsing to your own progress UI. The callback runs synchronously; exceptions
from it stop parsing and propagate to the caller. Without a callback, gtfparse
does not display a progress bar or add a progress-library dependency.
parsing to your own progress UI. Without a callback, gtfparse does not display
a progress bar or add a progress-library dependency.

| Stage | Reports |
| --- | --- |
| `read` | `(0, None)` before loading/filtering the file; `(rows, rows)` after it succeeds |
| `attributes` | `(0, rows)`, then every 10,000 rows, then `(rows, rows)` |
| `convert` | `(0, None)` before output conversion/transforms; `(rows, rows)` after success |

Counts refer to rows retained after `features` filtering, not bytes or overall
percentages. `None` means the total is unknown: loading and conversion expose
only stage boundaries, so use an indeterminate indicator during those steps.
Attribute expansion is omitted when `expand_attribute_column=False`. A stage
that fails does not emit completion. An empty filtered result is supported;
its attributes stage emits a single `(0, 0)` event. Empty input files raise `gtfparse.ParsingError`.

For example, with the optional `tqdm` package installed:
This example requires the optional `tqdm` package and your own GTF file:

```python
from gtfparse import read_gtf
Expand All @@ -37,9 +23,26 @@ with tqdm(unit="rows") as bar:
df = read_gtf("gene_annotations.gtf", progress_callback=show_progress)
```

## Stages and counts

| Stage | Reports |
| --- | --- |
| `read` | `(0, None)` before loading/filtering the file; `(rows, rows)` after it succeeds |
| `attributes` | `(0, rows)`, then every 10,000 rows, then `(rows, rows)` |
| `convert` | `(0, None)` before output conversion/transforms; `(rows, rows)` after success |

Counts refer to rows retained after `features` filtering, not bytes or overall
percentages. `None` means the total is unknown: loading and conversion expose
only stage boundaries, so use an indeterminate indicator during those steps.
Attribute expansion is omitted when `expand_attribute_column=False`.

The callback runs synchronously; exceptions from it stop parsing and propagate
to the caller. A stage that fails does not emit completion. An empty filtered
result is supported; its attributes stage emits a single `(0, 0)` event. Empty
input files raise `gtfparse.ParsingError`.

## Lower-level functions

`parse_gtf` reports the `read` stage, `parse_gtf_and_expand_attributes` reports
`read` and `attributes`, `parse_gtf_pandas` reports `read` and `convert`, and
`expand_attribute_strings` reports only `attributes` using the same callback.

Quoted attribute values preserve spaces, semicolons, and apostrophes. Raw
attributes (`expand_attribute_column=False`) retain their original quote marks.
6 changes: 4 additions & 2 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ attribute values such as gene and transcript IDs expanded into columns.
python -m pip install gtfparse
```

Python 3.9 or later is required. Version 3 uses pandas and PyArrow; Polars
output is optional and requires `gtfparse[polars]`.
Python 3.9 or later is required. For optional Polars output, install
`gtfparse[polars]`.

## Read an annotation and find genes

Expand Down Expand Up @@ -59,6 +59,8 @@ features = read_gtf("annotations.gtf.gz", features={"gene", "transcript"})
The `features` option filters row types while reading. Compressed inputs are
detected from their contents. Attribute expansion does not turn every row into
a gene or automatically supply missing parent features.
Quoted attribute values preserve spaces, semicolons, and apostrophes. Raw
attributes (`expand_attribute_column=False`) retain their original quote marks.

Continue with [filter and write annotations](guides/annotations.md), then
[progress callbacks](guides/progress.md) or [version 3 migration](guides/migration.md).
Expand Down
2 changes: 1 addition & 1 deletion gtfparse/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
)
from .write_gtf import write_gtf

__version__ = "3.0.3"
__version__ = "3.0.4"

__all__ = [
"GENCODE_BIOTYPE_ALIASES",
Expand Down
6 changes: 4 additions & 2 deletions tasks/todo.md
Original file line number Diff line number Diff line change
Expand Up @@ -512,10 +512,12 @@ after the modern stack is verified. Update PR branches without publishing.
- [x] Inspect PR heads, current master and version requirements.
- [x] Preserve current code while resolving the version conflict.
- [x] Run lint, tests and documentation checks on #93.
- [ ] Incorporate #93 in #95 and verify its separate patch bump.
- [ ] Update the PR branches and close superseded #74.
- [x] Incorporate #93 in #95 and verify its separate patch bump.
- [x] Update both PR branches. Closing superseded #74 is the final remote step.

## Review

PR #93: lint passed, all 213 tests passed, all three executable documentation
examples passed, and the strict MkDocs build passed.
PR #95: merged the refreshed parent; version 3.0.4, lint, 213 tests, and the
strict MkDocs build passed.
Loading