We need to add an integration test suite where we run the scraper on (at least) one representative and small configuration and ensure everything goes as expected. It is important the configuration is small enough to have a reactive CI (ideally do not run for more than 5 minutes, less is even better)
These tests should run daily and in "Tests.yaml".
maps scraper is an example of where this has been implemented as intended:
We need to add an integration test suite where we run the scraper on (at least) one representative and small configuration and ensure everything goes as expected. It is important the configuration is small enough to have a reactive CI (ideally do not run for more than 5 minutes, less is even better)
These tests should run daily and in "Tests.yaml".
mapsscraper is an example of where this has been implemented as intended:run-integration-testsjob): https://github.com/openzim/maps/blob/main/.github/workflows/Tests.yml