Zikithezikit · Nov 5, 2026 · Nov 5, 2026 · Nov 5, 2026 · Nov 5, 2026 · Nov 5, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -5,11 +5,25 @@ All notable changes are listed here. Format based on [Keep a Changelog](https://
 ## Unreleased
 
 ### Added
+- `BidsSource` and one small class per city for 12 municipalities that share the `/bids/` page layout (Akko, Arad,
+  Beit Shemesh, Dimona, Kfar Saba, Kiryat Motzkin, Nesher, Netivot, Nof HaGalil, Savyon, Tamra, Yavne). They are
+  part of the default sources.
+- `michraz.sources.BUILT_IN` and `default_sources()`: one registry of every built-in source. `SourceName` lists all
+  of them, and the CLI `--source` choices and the default search both follow the registry.
 - `GovSource` (`gov`): central-government tenders from mr.gov.il, read from the portal's HTML search. It is part of
   the default sources.
 - Tests use a fixed clock, so open/closed results no longer depend on the date the suite runs.
 - `HttpClient.get_text` for sources that serve HTML.
 
+### Changed
+- `BaseSource` is now a template. A source implements `records()` and `parse(record)`, and the shared `fetch()`
+  skips and reports bad records and drops repeated tenders. `RmiSource.fetch()` is now lazy like the others.
+- `Tender.kind` (`TenderKind.TENDER` or `VACANCY`) and a `kind` filter on `Michraz.search` and the CLI
+  (`--kind`). Staff vacancies that some municipal sites list beside their tenders are hidden by default. Pass
+  `kind=None` for everything. The CSV has a new `kind` column.
+- `Michraz.search` no longer fails when only some sources are down. Their errors go to `last_errors`, and it raises
+  only if every searched source failed.
+
 ## 0.1.0 (2026-11-05)
 
 ### Added

diff --git a/Makefile b/Makefile
@@ -39,5 +39,6 @@ clean:  ## Remove build and cache files
 	rm -rf dist build *.egg-info src/*.egg-info .pytest_cache .mypy_cache .ruff_cache .coverage
 	find . -name __pycache__ -not -path './.venv/*' -prune -exec rm -rf {} +
 
-diagrams:  ## Re-render docs/images from docs/diagrams (needs Graphviz and the Inter font)
-	for f in docs/diagrams/*.dot; do dot -Tpng -Gdpi=200 $$f -o docs/images/$$(basename $$f .dot).png; done
+diagrams:  ## Re-render docs/images from docs/diagrams (needs draw.io desktop and Graphviz)
+	for f in docs/diagrams/*.drawio; do drawio -x -f png -s 2 -b 16 --no-sandbox -o docs/images/$$(basename $$f .drawio).png $$f; done
+	dot -Tpng -Gdpi=200 docs/diagrams/cli-output.dot -o docs/images/cli-output.png
diff --git a/README.md b/README.md
@@ -7,14 +7,17 @@
 
 Israeli public tenders (מכרזים) in Python. One `Tender` object, whichever site it came from.
 
-Sources today: Israel Land Authority land tenders, Beer Sheva municipal tenders, and central-government tenders
-from mr.gov.il. The first search reads about 25 pages from mr.gov.il, so it takes around 20 seconds. After that the
+Sources today: Israel Land Authority land tenders, central-government tenders (mr.gov.il), Beer Sheva, and 12 more
+municipalities (Akko, Arad, Beit Shemesh, Dimona, Kfar Saba, Kiryat Motzkin, Nesher, Netivot, Nof HaGalil, Savyon,
+Tamra, Yavne). The first search takes around 20 seconds because mr.gov.il is read page by page. After that the
 responses are cached for an hour.
 
 > Unofficial client. It can lag or miss tenders, and parsing can break when a source changes. Confirm deadlines on
 > the publisher's page before you act. Not affiliated with the Israel Land Authority or any municipality.
 
-![What michraz does](https://raw.githubusercontent.com/Zikithezikit/michraz/main/docs/images/overview.png)
+![michraz architecture: how one search reaches the data and comes back](https://raw.githubusercontent.com/Zikithezikit/michraz/main/docs/images/architecture.png)
+
+[Edit in draw.io](https://app.diagrams.net/#Uhttps%3A%2F%2Fraw.githubusercontent.com%2FZikithezikit%2Fmichraz%2Fmain%2Fdocs%2Fdiagrams%2Farchitecture.drawio)
 
 ## Install
 
@@ -40,31 +43,59 @@ to_csv(tenders, "tenders.csv")
 Results are sorted by deadline. Filter by `source`, `closing_before` and `closing_after` (dates or aware
 datetimes) too. Each tender has an `attribution` string. Keep it when you republish the data.
 
+Some city sites list staff vacancies next to their tenders. They are hidden by default. Pass `kind=None` to see both,
+or `--kind all` on the command line.
+
 Or from the terminal:
 
 ```bash
 michraz search --open --source rmi
 michraz search --keyword גינון --csv gardening.csv
 ```
 
-![Terminal output](https://raw.githubusercontent.com/Zikithezikit/michraz/main/docs/images/cli.png)
+![michraz search in the terminal](https://raw.githubusercontent.com/Zikithezikit/michraz/main/docs/images/cli-output.png)
 
-A record that fails to parse is skipped and listed in `m.last_errors`. An unreachable source raises
-`SourceUnavailableError`. More scripts are in [examples](https://github.com/Zikithezikit/michraz/tree/main/examples).
+A record that fails to parse is skipped and listed in `m.last_errors`. If one source is down, the others still
+answer and its error is listed there too. `SourceUnavailableError` is raised only when every searched source fails.
+
+More scripts are in [examples](https://github.com/Zikithezikit/michraz/tree/main/examples).
 
 ## Add a source
 
-Write a class with `name`, `description`, `attribution`, `errors` and a `fetch()` that yields `Tender`s, then pass
-it to `Michraz([MySource()])`.
+Subclass `BaseSource`. Say how to read the records and how to turn one into a `Tender`. Skipping bad records,
+reporting them in `errors` and dropping repeats is shared.
+
+```python
+from michraz.sources import BaseSource
+
+
+class MySource(BaseSource[dict]):
+    name = "my-city"
+    description = "My city's tenders"
+    attribution = "My City, my-city.example"
 
-![Classes](https://raw.githubusercontent.com/Zikithezikit/michraz/main/docs/images/classes.png)
+    def records(self):
+        yield from self._client().get_json(self.name, "https://my-city.example/api/tenders")
+
+    def parse(self, record):
+        return Tender(source=self.name, source_id=record["id"], title=record["title"], attribution=self.attribution)
+```
+
+Then `Michraz([MySource()])`. A city that uses the shared `/bids/` page needs only a name, a publisher and a domain
+(see `michraz/sources/municipal.py`).
+
+![Class diagram: sources and the Tender model](https://raw.githubusercontent.com/Zikithezikit/michraz/main/docs/images/class-diagram.png)
+
+[Edit in draw.io](https://app.diagrams.net/#Uhttps%3A%2F%2Fraw.githubusercontent.com%2FZikithezikit%2Fmichraz%2Fmain%2Fdocs%2Fdiagrams%2Fclass-diagram.drawio)
 
 Fields are documented in [docs/model.md](https://github.com/Zikithezikit/michraz/blob/main/docs/model.md), and how
 each source was checked in [docs/sources.md](https://github.com/Zikithezikit/michraz/blob/main/docs/sources.md).
 
 ## Polite by default
 
-![Request flow](https://raw.githubusercontent.com/Zikithezikit/michraz/main/docs/images/request_flow.png)
+![Flowchart: what happens during one search](https://raw.githubusercontent.com/Zikithezikit/michraz/main/docs/images/search-flowchart.png)
+
+[Edit in draw.io](https://app.diagrams.net/#Uhttps%3A%2F%2Fraw.githubusercontent.com%2FZikithezikit%2Fmichraz%2Fmain%2Fdocs%2Fdiagrams%2Fsearch-flowchart.drawio)
 
 Honest User-Agent, one request per second per host, one-hour disk cache, at most two retries. Tune it with
 `Michraz(cache_dir=None, cache_ttl=..., min_interval=...)`.
@@ -79,9 +110,9 @@ Honest User-Agent, one request per second per host, one-hour disk cache, at most
 
 ```bash
 make venv && make check
+make diagrams   # re-export the PNGs from docs/diagrams (needs draw.io desktop and Graphviz)
 ```
 
-Tests run against recorded responses and never touch the network. See
-[CONTRIBUTING.md](https://github.com/Zikithezikit/michraz/blob/main/CONTRIBUTING.md).
+See [CONTRIBUTING.md](https://github.com/Zikithezikit/michraz/blob/main/CONTRIBUTING.md).
 
 MIT licensed. The data belongs to its publishers and carries their licenses.
diff --git a/docs/diagrams/architecture.drawio b/docs/diagrams/architecture.drawio
@@ -0,0 +1,15 @@
+<mxfile host="app.diagrams.net"><diagram name="Architecture"><mxGraphModel dx="1120" dy="420" grid="0" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="0" pageScale="1" pageWidth="1120" pageHeight="420" math="0" shadow="0"><root><mxCell id="0"/><mxCell id="1" parent="0"/>
+<mxCell id="c2" value="&lt;b style='font-size:16px'&gt;michraz architecture&lt;/b&gt;&lt;br&gt;&lt;font color='#666666'&gt;how one search reaches the data and comes back&lt;/font&gt;" style="text;html=1;strokeColor=none;fillColor=none;whiteSpace=wrap;fontFamily=Arial;fontSize=12;fontColor=#000000;align=center;verticalAlign=middle;" vertex="1" parent="1"><mxGeometry x="0" y="0" width="1060" height="44" as="geometry"/></mxCell>
+<mxCell id="c3" value="&lt;b&gt;You&lt;/b&gt;&lt;br&gt;&lt;font color='#666666'&gt;Python or terminal&lt;/font&gt;" style="rounded=1;arcSize=12;whiteSpace=wrap;html=1;fillColor=#efefef;strokeColor=#333333;strokeWidth=2;fontFamily=Arial;fontSize=12;fontColor=#000000;" vertex="1" parent="1"><mxGeometry x="20" y="90" width="160" height="60" as="geometry"/></mxCell>
+<mxCell id="c4" value="&lt;b&gt;Michraz.search()&lt;/b&gt;&lt;br&gt;&lt;font color='#666666'&gt;filter, sort, collect errors&lt;/font&gt;" style="rounded=1;arcSize=12;whiteSpace=wrap;html=1;fillColor=#cfe2f3;strokeColor=#333333;strokeWidth=2;fontFamily=Arial;fontSize=12;fontColor=#000000;" vertex="1" parent="1"><mxGeometry x="270" y="90" width="190" height="60" as="geometry"/></mxCell>
+<mxCell id="c5" value="&lt;b&gt;Polite HTTP client&lt;/b&gt;&lt;br&gt;&lt;font color='#666666'&gt;rate limit, disk cache, retries&lt;/font&gt;" style="rounded=1;arcSize=12;whiteSpace=wrap;html=1;fillColor=#cfe2f3;strokeColor=#333333;strokeWidth=2;fontFamily=Arial;fontSize=12;fontColor=#000000;" vertex="1" parent="1"><mxGeometry x="560" y="90" width="210" height="60" as="geometry"/></mxCell>
+<mxCell id="c6" value="&lt;b&gt;Sources&lt;/b&gt;&lt;br&gt;&lt;div style='text-align:left'&gt;Israel Land Authority &lt;font color='#666666'&gt;(JSON API)&lt;/font&gt;&lt;br&gt;Government portal &lt;font color='#666666'&gt;(mr.gov.il)&lt;/font&gt;&lt;br&gt;Beer Sheva &lt;font color='#666666'&gt;(data.gov.il)&lt;/font&gt;&lt;br&gt;12 city sites &lt;font color='#666666'&gt;(shared /bids/ page)&lt;/font&gt;&lt;br&gt;&lt;i&gt;Your source&lt;/i&gt; &lt;font color='#666666'&gt;(one small class)&lt;/font&gt;&lt;/div&gt;" style="rounded=1;arcSize=12;whiteSpace=wrap;html=1;fillColor=#d9ead3;strokeColor=#333333;strokeWidth=2;fontFamily=Arial;fontSize=12;fontColor=#000000;align=left;spacingLeft=10;verticalAlign=top;spacingTop=6;" vertex="1" parent="1"><mxGeometry x="860" y="55" width="240" height="130" as="geometry"/></mxCell>
+<mxCell id="c7" value="&lt;b&gt;list of Tender&lt;/b&gt;&lt;br&gt;&lt;font color='#666666'&gt;one model for every source&lt;/font&gt;" style="rounded=1;arcSize=12;whiteSpace=wrap;html=1;fillColor=#cfe2f3;strokeColor=#333333;strokeWidth=2;fontFamily=Arial;fontSize=12;fontColor=#000000;" vertex="1" parent="1"><mxGeometry x="270" y="290" width="190" height="60" as="geometry"/></mxCell>
+<mxCell id="c8" value="&lt;b&gt;Output&lt;/b&gt;&lt;br&gt;&lt;div style='text-align:left'&gt;Table in the terminal&lt;br&gt;CSV &lt;font color='#666666'&gt;(opens in Excel)&lt;/font&gt;&lt;br&gt;JSON&lt;br&gt;pandas DataFrame&lt;/div&gt;" style="rounded=1;arcSize=12;whiteSpace=wrap;html=1;fillColor=#fff2cc;strokeColor=#333333;strokeWidth=2;fontFamily=Arial;fontSize=12;fontColor=#000000;align=left;spacingLeft=10;verticalAlign=top;spacingTop=6;" vertex="1" parent="1"><mxGeometry x="20" y="260" width="160" height="120" as="geometry"/></mxCell>
+<mxCell id="c9" value="calls" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;html=1;jettySize=auto;strokeColor=#000000;strokeWidth=2;fontSize=11;fontFamily=Arial;labelBackgroundColor=#ffffff;endArrow=block;endFill=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="c3" target="c4"><mxGeometry relative="1" as="geometry"></mxGeometry></mxCell>
+<mxCell id="c10" value="asks" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;html=1;jettySize=auto;strokeColor=#000000;strokeWidth=2;fontSize=11;fontFamily=Arial;labelBackgroundColor=#ffffff;endArrow=block;endFill=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="c4" target="c5"><mxGeometry relative="1" as="geometry"></mxGeometry></mxCell>
+<mxCell id="c11" value="GET" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;html=1;jettySize=auto;strokeColor=#000000;strokeWidth=2;fontSize=11;fontFamily=Arial;labelBackgroundColor=#ffffff;endArrow=block;endFill=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="c5" target="c6"><mxGeometry relative="1" as="geometry"></mxGeometry></mxCell>
+<mxCell id="c12" value="parsed into" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;html=1;jettySize=auto;strokeColor=#000000;strokeWidth=2;fontSize=11;fontFamily=Arial;labelBackgroundColor=#ffffff;endArrow=block;endFill=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=1;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="c6" target="c7"><mxGeometry relative="1" as="geometry"></mxGeometry></mxCell>
+<mxCell id="c13" value="returned to" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;html=1;jettySize=auto;strokeColor=#000000;strokeWidth=2;fontSize=11;fontFamily=Arial;labelBackgroundColor=#ffffff;endArrow=block;endFill=1;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;" edge="1" parent="1" source="c7" target="c4"><mxGeometry relative="1" as="geometry"></mxGeometry></mxCell>
+<mxCell id="c14" value="you export" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;html=1;jettySize=auto;strokeColor=#000000;strokeWidth=2;fontSize=11;fontFamily=Arial;labelBackgroundColor=#ffffff;endArrow=block;endFill=1;exitX=0;exitY=0.5;exitDx=0;exitDy=0;entryX=1;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="c7" target="c8"><mxGeometry relative="1" as="geometry"></mxGeometry></mxCell>
+</root></mxGraphModel></diagram></mxfile>