Skip to content

feat: add bids source for 12 municipalities - #5

Merged
Zikithezikit merged 6 commits into
mainfrom
more-sources-2
Nov 5, 2026
Merged

Zikithezikit merged 6 commits into
mainfrom
more-sources-2

Conversation

@Zikithezikit

@Zikithezikit Zikithezikit commented Nov 5, 2026 •

Copy link
Copy Markdown
Owner

This adds tenders from 12 more cities: Akko, Arad, Beit Shemesh, Dimona, Kfar Saba, Kiryat Motzkin, Nesher, Netivot, Nof HaGalil, Savyon, Tamra and Yavne. They publish on the same kind of page, so one parser reads them all, and each city is a short class with just a name, a publisher and a domain. To keep every source uniform I also reworked the base: a source now only says how to read its records and how to turn one into a tender, and the shared fetch() skips and reports bad records and drops repeats. RMI, Beer Sheva, the government portal and the cities all work this way, they are listed in one registry (BUILT_IN), and a normal search, the CLI choices and SourceName all follow it. Four of these cities list staff vacancies in the same tables as their tenders, so every listing now has a kind (tender or vacancy). Vacancies are hidden by default, and you can ask for them with kind=TenderKind.VACANCY, for both with kind=None, or with --kind tender|vacancy|all on the command line. That split is a guess based on the table caption and title, so an odd one can slip through. If one source is down the search keeps going and notes the error in last_errors; it only fails when every source fails. I ran it against the live sites today and got 678 open tenders from 15 sources with no errors. Dates on these pages have no hour, so a deadline is treated as the end of that day, and Azor is left out because its robots.txt blocks everything.

tests

live data

all sources

class diagram

architecture

search flowchart

@Zikithezikit
Zikithezikit merged commit 20ea5c4 into main Nov 5, 2026
6 checks passed
@Zikithezikit
Zikithezikit deleted the more-sources-2 branch November 5, 2026 07:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant