[surrealwolf]

Description

Implements rate limiting in the scraper to prevent hitting the Hell Divers 2 community API's 5 requests per 10 seconds limit, which was causing 429 (Too Many Requests) errors during data collection cycles.

Changes

Implementation

• Added _rate_limit() method to HellDivers2Scraper class
• Enforces 2-second delay between requests (optimized for 5 requests per 10 seconds)
• Tracks last_request_time and calculates necessary sleep duration
• Applied rate limiting to all 10 scraper methods:
  • get_war_status()
  • get_planets()
  • get_statistics()
  • get_factions()
  • get_biomes()
  • get_campaign_info()
  • get_assignments()
  • get_dispatches()
  • get_planet_events()
  • get_planet_status()

Documentation

• Updated .github/copilot-instructions.md with rate limiting patterns
• Added debug logging when rate limiting delays are applied
• Documented graceful degradation for 429 errors (logs and continues)

Testing

• All 11 unit tests passing (no server dependency due to mocking)
• Linting: ruff all checks passed
• Type checking: mypy success with no issues

Related Issues

Fixes 429 errors observed during data collection cycles when hitting API rate limits.

Checklist

• All tests passing
• Linting passes (ruff + mypy)
• Documentation updated
• No breaking changes

[Copilot]

Pull Request Overview

This PR implements rate limiting in the Hell Divers 2 API scraper to prevent 429 (Too Many Requests) errors by enforcing a 2-second delay between consecutive API requests, staying within the community API's limit of 5 requests per 10 seconds.

Key changes:

• Added _rate_limit() method to track and enforce delays between requests
• Applied rate limiting to all 10 scraper methods before HTTP calls
• Updated documentation to reflect rate limiting patterns and API details

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
src/scraper.py	Implements rate limiting with _rate_limit() method and adds delay tracking to all API request methods
.github/copilot-instructions.md	Documents rate limiting implementation details, API specifications, and updates component descriptions with accurate line counts

[Copilot]

Pull Request Overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.