feat: add icon download processor

[Rabenherz112]

• adds a new processor download_icons.py
• updates documentation and tests

---

This adds the discussed icon downloader from #157. It currently:

• Walks <source_directory>/software/*.yml, reads each entry's icon_url, downloads it, and writes a square WebP to <output_directory>/<yaml-name>.webp
• Downloads have several safeguards: a HEAD preflight check (which is allowed to fail since some sites block HEAD requests and which turns out that is not uncommon), a hard byte cap enforced during the download, and a content validation pass after the download completes
• Resizes within a target box (default 128×128) preserving aspect ratio, then pads to a square canvas (transparent for RGBA sources, white for opaque) so all output files have the same dimensions
• Encoding is split into two paths, a lossless WebP for alpha sources and {PNG, ICO, GIF, BMP} (which is what most icons actually are), and lossy WebP at quality 85 for opaque JPEG/WEBP (the lossy branch only exists because when I tested it, encoding a JPEG icon losslessly actually doubled the output size for some reason)
• Picks the largest frame for multi-resolution .ico sources
• Preserves source ICC profiles when present
• Skips entries whose .webp already exists by default (skip_when_icon_present: True)
• Adds an additional check in awesome_lint that flags .webp files in icons/ with no matching software YAML (dangling-icon detection)

One deviation from what was originally discussed: instead of the warn-mostly behavior mentioned in the issue, this errors out on failures instead. That probably makes more sense for a pipeline and matches the existing behavior of the other processors much better anyway.

---

I also did some testing around size and format. I had AI generate a benchmark to check whether PNG or WebP made more sense in practice. The data comes from an import of dashboard-icons where I added the WebP URL to every project matched, so the benchmark ran across 467 icons. Unless otherwise noted, all values below are means.

At 128×128	WebP	PNG
Lossless	7.3 KiB	10.3 KiB (+29%)
Lossy / quantized	3.9 KiB (q85)	4.7 KiB (256-color) (+17%)

Size	WebP bytes	vs 128
64	3.0 KiB	41%
96	5.1 KiB	70%
128	7.3 KiB	baseline
192	12.6 KiB	173%
256	17.8 KiB	244%

Details

Run metadata

• Source directory: tests/awesome-selfhosted-data
• Entries discovered: 1309
• Entries successfully encoded: 467
• Entries rejected: 842
• Sizes benchmarked: 64, 96, 128, 192, 256
• Variants benchmarked: webp-current, webp-lossy-q85, webp-lossy-q90, png-current, png-lossless, png-quantized
• Wall-clock time: 2798.6s

At-a-glance: mean bytes per icon (and sum across encoded entries)

Variant	64px	96px	128px	192px	256px
webp-current	3.0 KiB (sum 1.4 MiB)	5.1 KiB (sum 2.3 MiB)	7.3 KiB (sum 3.3 MiB)	12.6 KiB (sum 5.7 MiB)	17.8 KiB (sum 8.1 MiB)
webp-lossy-q85	1.8 KiB (sum 854.8 KiB)	2.9 KiB (sum 1.3 MiB)	3.9 KiB (sum 1.8 MiB)	6.0 KiB (sum 2.8 MiB)	8.2 KiB (sum 3.7 MiB)
webp-lossy-q90	2.0 KiB (sum 942.5 KiB)	3.2 KiB (sum 1.4 MiB)	4.3 KiB (sum 2.0 MiB)	6.7 KiB (sum 3.1 MiB)	9.2 KiB (sum 4.2 MiB)
png-current	4.2 KiB (sum 1.9 MiB)	7.1 KiB (sum 3.2 MiB)	10.3 KiB (sum 4.7 MiB)	17.5 KiB (sum 8.0 MiB)	25.3 KiB (sum 11.5 MiB)
png-lossless	4.3 KiB (sum 1.9 MiB)	7.2 KiB (sum 3.3 MiB)	10.4 KiB (sum 4.8 MiB)	17.8 KiB (sum 8.1 MiB)	25.8 KiB (sum 11.8 MiB)
png-quantized	2.3 KiB (sum 1.0 MiB)	3.4 KiB (sum 1.6 MiB)	4.7 KiB (sum 2.1 MiB)	7.7 KiB (sum 3.5 MiB)	10.9 KiB (sum 5.0 MiB)

Per-size detail

64x64

Variant	mean	median	p95	p99	max	sum across encoded
webp-current	3.0 KiB	2.7 KiB	5.9 KiB	7.0 KiB	7.9 KiB	1.4 MiB
webp-lossy-q85	1.8 KiB	1.8 KiB	3.1 KiB	3.7 KiB	4.7 KiB	854.8 KiB
webp-lossy-q90	2.0 KiB	2.0 KiB	3.4 KiB	4.0 KiB	5.0 KiB	942.5 KiB
png-current	4.2 KiB	4.0 KiB	7.4 KiB	8.5 KiB	10.2 KiB	1.9 MiB
png-lossless	4.3 KiB	4.1 KiB	7.4 KiB	8.5 KiB	10.2 KiB	1.9 MiB
png-quantized	2.3 KiB	2.3 KiB	3.5 KiB	4.1 KiB	4.6 KiB	1.0 MiB

96x96

Variant	mean	median	p95	p99	max	sum across encoded
webp-current	5.1 KiB	4.5 KiB	10.7 KiB	13.1 KiB	14.6 KiB	2.3 MiB
webp-lossy-q85	2.9 KiB	2.8 KiB	5.0 KiB	6.1 KiB	8.3 KiB	1.3 MiB
webp-lossy-q90	3.2 KiB	3.1 KiB	5.6 KiB	6.7 KiB	9.0 KiB	1.4 MiB
png-current	7.1 KiB	6.6 KiB	13.7 KiB	16.3 KiB	19.3 KiB	3.2 MiB
png-lossless	7.2 KiB	6.7 KiB	14.0 KiB	16.3 KiB	19.3 KiB	3.3 MiB
png-quantized	3.4 KiB	3.4 KiB	5.9 KiB	7.0 KiB	8.6 KiB	1.6 MiB

128x128

Variant	mean	median	p95	p99	max	sum across encoded
webp-current	7.3 KiB	6.2 KiB	15.9 KiB	20.4 KiB	22.3 KiB	3.3 MiB
webp-lossy-q85	3.9 KiB	3.7 KiB	7.1 KiB	9.2 KiB	12.3 KiB	1.8 MiB
webp-lossy-q90	4.3 KiB	4.1 KiB	7.8 KiB	10.1 KiB	13.4 KiB	2.0 MiB
png-current	10.3 KiB	9.3 KiB	20.9 KiB	26.0 KiB	29.3 KiB	4.7 MiB
png-lossless	10.4 KiB	9.4 KiB	21.3 KiB	26.0 KiB	29.3 KiB	4.8 MiB
png-quantized	4.7 KiB	4.5 KiB	8.8 KiB	10.8 KiB	13.3 KiB	2.1 MiB

192x192

Variant	mean	median	p95	p99	max	sum across encoded
webp-current	12.6 KiB	10.6 KiB	29.3 KiB	35.7 KiB	40.7 KiB	5.7 MiB
webp-lossy-q85	6.0 KiB	5.7 KiB	11.1 KiB	14.5 KiB	19.7 KiB	2.8 MiB
webp-lossy-q90	6.7 KiB	6.5 KiB	12.5 KiB	16.0 KiB	21.7 KiB	3.1 MiB
png-current	17.5 KiB	15.3 KiB	38.0 KiB	48.3 KiB	53.5 KiB	8.0 MiB
png-lossless	17.8 KiB	15.3 KiB	39.7 KiB	48.3 KiB	53.5 KiB	8.1 MiB
png-quantized	7.7 KiB	7.1 KiB	16.0 KiB	19.9 KiB	27.8 KiB	3.5 MiB

256x256

Variant	mean	median	p95	p99	max	sum across encoded
webp-current	17.8 KiB	14.7 KiB	42.5 KiB	55.1 KiB	62.5 KiB	8.1 MiB
webp-lossy-q85	8.2 KiB	7.8 KiB	15.2 KiB	22.2 KiB	27.9 KiB	3.7 MiB
webp-lossy-q90	9.2 KiB	8.7 KiB	17.0 KiB	24.5 KiB	30.4 KiB	4.2 MiB
png-current	25.3 KiB	21.5 KiB	55.6 KiB	73.4 KiB	85.3 KiB	11.5 MiB
png-lossless	25.8 KiB	21.6 KiB	59.7 KiB	73.4 KiB	85.3 KiB	11.8 MiB
png-quantized	10.9 KiB	9.8 KiB	23.4 KiB	30.9 KiB	47.7 KiB	5.0 MiB

Source-size distribution (pre-processing)

Use this to pick a sensible max_download_bytes cap — the p99 row tells you the smallest cap that would accept ~99% of real icons.

stat	bytes
count	467
mean	36.1 KiB
median	27.0 KiB
p95	96.2 KiB
p99	170.1 KiB
max	267.2 KiB

Source format breakdown

source format	count	share
WEBP	467	100.0%

has alpha channel	count	share
yes	453	97.0%
no	14	3.0%

Rejection breakdown

reason	count
missing_icon_url	842

---

To test it, you can use the current awesome-selfhosted-repo, run my dashboard-icons import script to populate icon_url fields, then run hecat against it.