PyRanges.intersect - "invert=True" kwarg behavior

[vincentczhou]

From Pyranges.intersect
https://github.com/pyranges/pyranges/blob/a379f33cf043b1219cd139177807d6a58ca9b490/pyranges/pyranges_main.py#L2213-L2216
From Pyranges.overlap
https://github.com/pyranges/pyranges/blob/a379f33cf043b1219cd139177807d6a58ca9b490/pyranges/pyranges_main.py#L3624-L3627

Please correct me if I'm wrong, but the behavior of Pyranges.intersect and Pyranges.overlap with the invert=True kwarg present behave identically the same.
If two ranges (intervals) have intersections, they must also overlap; therefore, returning only the intervals that do not overlap will be exactly the same for both functions.

This is stated in the identical docstrings for both Pyranges.intersect and Pyranges.overlap! However, if the intersect function returns all intersecting subintervals, it makes more sense to return all non-intersecting subintervals with the invert=True kwarg present.

Please see the details dropdown (below) for an example:

Details

Example below generated by setting a pdb trace at line 49 here with an input file present containing the following:
(modified version of Day 5 - Advent of Code 2023 example input)

seeds: 81 14 57 13

fertilizer-to-water map:
49 53 8
0 11 42
42 0 7
57 7 4

water-to-light map:
88 18 7
18 25 70

light-to-temperature map:
45 77 23
81 45 19
68 64 13

temperature-to-humidity map:
0 69 1
1 0 69

humidity-to-location map:
60 56 37
56 93 4

(Pdb) pr_seed
+-----------+-----------+--------------+-----------+
|     Start |       End | Chromosome   |    __ix__ |
|   (int64) |   (int64) | (category)   |   (int64) |
|-----------+-----------+--------------+-----------|
|        81 |        95 | aoc          |         0 |
|        57 |        70 | aoc          |         1 |
+-----------+-----------+--------------+-----------+
Unstranded PyRanges object has 2 rows and 4 columns from 1 chromosomes.
For printing, the PyRanges was sorted on Chromosome.
(Pdb) pr_map
+-----------+-----------+-----------+-----------+--------------+
|      dest |     Start |     range |       End | Chromosome   |
|   (int64) |   (int64) |   (int64) |   (int64) | (category)   |
|-----------+-----------+-----------+-----------+--------------|
|        49 |        53 |         8 |        61 | aoc          |
|         0 |        11 |        42 |        53 | aoc          |
|        42 |         0 |         7 |         7 | aoc          |
|        57 |         7 |         4 |        11 | aoc          |
+-----------+-----------+-----------+-----------+--------------+
Unstranded PyRanges object has 4 rows and 5 columns from 1 chromosomes.
For printing, the PyRanges was sorted on Chromosome.
(Pdb) pr_seed.overlap(pr_map)
+-----------+-----------+--------------+-----------+
|     Start |       End | Chromosome   |    __ix__ |
|   (int64) |   (int64) | (category)   |   (int64) |
|-----------+-----------+--------------+-----------|
|        57 |        70 | aoc          |         1 |
+-----------+-----------+--------------+-----------+
Unstranded PyRanges object has 1 rows and 4 columns from 1 chromosomes.
For printing, the PyRanges was sorted on Chromosome.
(Pdb) pr_seed.overlap(pr_map, invert=True)
+-----------+-----------+--------------+
|     Start |       End | Chromosome   |
|   (int64) |   (int64) | (category)   |
|-----------+-----------+--------------|
|        81 |        95 | aoc          |
+-----------+-----------+--------------+
Unstranded PyRanges object has 1 rows and 3 columns from 1 chromosomes.
For printing, the PyRanges was sorted on Chromosome.
(Pdb) pr_seed.intersect(pr_map)
+-----------+-----------+--------------+-----------+
|     Start |       End | Chromosome   |    __ix__ |
|   (int64) |   (int64) | (category)   |   (int64) |
|-----------+-----------+--------------+-----------|
|        57 |        61 | aoc          |         1 |
+-----------+-----------+--------------+-----------+
Unstranded PyRanges object has 1 rows and 4 columns from 1 chromosomes.
For printing, the PyRanges was sorted on Chromosome.
(Pdb) pr_seed.intersect(pr_map, invert=True)
+-----------+-----------+--------------+
|     Start |       End | Chromosome   |
|   (int64) |   (int64) | (category)   |
|-----------+-----------+--------------|
|        81 |        95 | aoc          |
+-----------+-----------+--------------+
Unstranded PyRanges object has 1 rows and 3 columns from 1 chromosomes.
For printing, the PyRanges was sorted on Chromosome.

Returning all non-intersecting subintervals in the call of pr_seed.intersect(pr_map, invert=True), we would expect the following:

+-----------+-----------+--------------+-----------+
|     Start |       End | Chromosome   |    __ix__ |
|   (int64) |   (int64) | (category)   |   (int64) |
|-----------+-----------+--------------+-----------|
|        81 |        95 | aoc          |         0 |
|        61 |        70 | aoc          |         1 |
+-----------+-----------+--------------+-----------+
Unstranded PyRanges object has 2 rows and 4 columns from 1 chromosomes.
For printing, the PyRanges was sorted on Chromosome.

This issue is inspired by Day 5 - Advent of Code 2023, and is something I encountered and documented in my solution here.

There's a working alternative using Pyranges.subtract, perhaps that logic can be integrated to return all non-intersecting subintervals with the invert=True kwarg present.

Thanks!

[endrebak]

Thanks for giving me something to think about :) I'll have a look. @marco-mariotti