How much does the background influence what an image classifier "sees"? This project segments the foreground object out of an image, systematically replaces its background (removed, solid contrast colours, or entirely new scenery), and measures how the predictions of two pre-trained classifiers — ResNet50 and ConvNeXt — change as a result.
It is the code behind my master's thesis, "Badanie wpływu tła na klasyfikację zwierząt na obrazach" (Investigating the impact of background on the classification of animals in images), and operates on 10 animal classes from ImageNet.
The full pipeline runs as a sequence of bgimpact sub-commands:
| Stage | Command | What it does |
|---|---|---|
| 1. Segment | bgimpact segment |
DeepLabV3 produces a mask + overlay per image. |
| 2. Remove background | bgimpact remove-bg |
Keep the object, paint the background black. |
| 3. Remove foreground | bgimpact remove-fg |
Keep only the background. |
| 4. Contrast colours | bgimpact colors |
KMeans finds per-class dominant colours → low/high contrast. |
| 5. Contrast backgrounds | bgimpact apply-contrast |
Composite the object onto solid contrast colours. |
| 6. Scenic backgrounds | bgimpact apply-scenic |
Composite the object onto 8 scenes (city, jungle, desert, …). |
| 7. Classify | bgimpact classify --model {resnet50,convnext} |
Score originals vs. every modification, save top-5 confidences to CSV. |
This project uses uv for dependency management.
git clone https://github.com/paulpel/background-impact-on-classification.git
cd background-impact-on-classification
uv sync # core dependencies
uv sync --extra notebooks # + matplotlib/seaborn/opencv for the notebooks# See all stages and options
uv run bgimpact --help
# Run the whole pipeline end to end
uv run bgimpact segment
uv run bgimpact remove-bg
uv run bgimpact colors
uv run bgimpact apply-contrast
uv run bgimpact apply-scenic
uv run bgimpact classify --model resnet50
uv run bgimpact classify --model convnextEvery command accepts --help and exposes its input/output directories as
options, so individual stages can be re-run in isolation.
The study uses 10 animal classes from ImageNet-1k (junco, German shepherd,
Egyptian cat, bighorn sheep, Old English sheepdog, hummingbird, American robin,
ram, Greater Swiss Mountain dog, Persian cat — see analysis/config.py).
ImageNet is access-gated, so the images are not included here. Arrange them as:
data/
├── train/<synset>/*.JPEG # original images, one folder per class
└── scenarios/*.jpg # 8 scenic background images
Across both models, background changes measurably degrade classification — the models clearly lean on background context, not just the object itself.
- ConvNeXt is more robust than ResNet. On background-modified images ConvNeXt reaches ~79% accuracy versus ~69% for ResNet — a ~10 percentage-point gap — while, interestingly, being less confident in its predictions.
- Confidence drops after modification for both models, with a sharper drop for ConvNeXt; correct predictions stay more confident than incorrect ones.
- Small objects suffer most. When the object is small relative to the frame, accuracy collapses under background changes (ResNet: 0.88 → 0.55), whereas large objects stay comparatively stable.
- Replacing the background occasionally helps — for a few classes that are routinely confused with a visually similar class, moving the object to a new scene helped the model tell them apart.
The full top-5 confidence scores for every image/modification are produced as
image_confidence_scores_{resnet,convnext}.csv and are published, together with
the compiled thesis, as release assets.
The full master's thesis (in Polish) is available as a PDF in the
latest release;
its LaTeX sources live in praca/.
- Title: Badanie wpływu tła na klasyfikację zwierząt na obrazach
- Author: Paweł Pelar
- Supervisor: dr hab. inż. Henryk Maciejewski
- Field: Trusted Systems of Artificial Intelligence
analysis/
├── config.py # dataset layout, class list, shared constants
├── segmentation.py # DeepLabV3 mask/overlay generation
├── masks.py # per-class foreground grayscale analysis
├── backgrounds.py # background removal / contrast / scenic replacement
├── colors.py # dominant-colour & contrast analysis
├── classify.py # ResNet50 / ConvNeXt inference and comparison
├── pipeline.py # mask-generation orchestration
└── cli.py # the `bgimpact` command-line interface
praca/ # LaTeX sources of the thesis
tests/ # unit tests for the pure helpers
PyTorch · torchvision (DeepLabV3, ResNet50, ConvNeXt) · scikit-learn · NumPy · Pillow · pandas · Typer · uv · Ruff · pytest