feat: add Redis-backed SMTP rate limiting, email caching, and provide…

[revanth127]

closes #12

• Migrate rate limiting from in-memory to Redis (atomic INCR + EXPIRE)
• Add per-domain rate limit: 3 verification attempts per hour
• Implement Redis-backed result cache with 24h TTL
• Skip caching transient 'unverified' states (network failures)
• Add SMTP blocklist for known providers (Gmail, Outlook, Yahoo, iCloud, etc.)
• Allow blocklist extension via VERIFIER_SMTP_BLOCKLIST_EXTENDS env var
• Graceful degradation when Redis is unavailable (fail open, log warnings)
• Works across multiple workers/processes

[vercel]

@revanth127 is attempting to deploy a commit to the Vaibhav Sharma's projects Team on Vercel.

A member of the Team first needs to authorize it.

[sharmavaibhav31]

Thanks for the PR — the Redis-backed direction is good overall, especially the move away from process-local rate limiting.

However, while reviewing verifier.py, I found several structural and correctness issues that need cleanup before merge:

• _resolve_mx() currently appears malformed/mixed with verification flow logic
• email is referenced inside _resolve_mx() without being defined in scope
• rate-limit rollback using DECR after limit exceed is race-prone under concurrency
• Redis/cache concerns and SMTP verification flow are too tightly coupled right now
• some fallback/error-handling paths need hardening
• input normalization/sanitization for cache + rate-limit keys should be stricter
• async execution/timeouts around SMTP verification need another pass
• there are a few maintainability concerns where large sections feel over-abstracted compared to the current codebase style

Please do a focused cleanup/refactor pass on verifier.py before further review.

Priority should be:

1. correctness
2. concurrency safety
3. Redis failure handling
4. clean separation of responsibilities
5. consistency with existing project architecture/style

Also please test:

• Redis unavailable scenarios
• concurrent verification attempts
• cache hit/miss behavior
• rate-limit expiration/reset behavior
• malformed email input handling

Once those are cleaned up, I’ll continue with a deeper review.

[sharmavaibhav31]

Fine, this works.

[revanth127]

Thanks for the PR — the Redis-backed direction is good overall, especially the move away from process-local rate limiting.

However, while reviewing verifier.py, I found several structural and correctness issues that need cleanup before merge:

• _resolve_mx() currently appears malformed/mixed with verification flow logic
• email is referenced inside _resolve_mx() without being defined in scope
• rate-limit rollback using DECR after limit exceed is race-prone under concurrency
• Redis/cache concerns and SMTP verification flow are too tightly coupled right now
• some fallback/error-handling paths need hardening
• input normalization/sanitization for cache + rate-limit keys should be stricter
• async execution/timeouts around SMTP verification need another pass
• there are a few maintainability concerns where large sections feel over-abstracted compared to the current codebase style

Please do a focused cleanup/refactor pass on verifier.py before further review.

Priority should be:

1. correctness
2. concurrency safety
3. Redis failure handling
4. clean separation of responsibilities
5. consistency with existing project architecture/style

Also please test:

• Redis unavailable scenarios
• concurrent verification attempts
• cache hit/miss behavior
• rate-limit expiration/reset behavior
• malformed email input handling

Once those are cleaned up, I’ll continue with a deeper review.

Thanks for the PR — the Redis-backed direction is good overall, especially the move away from process-local rate limiting.

However, while reviewing verifier.py, I found several structural and correctness issues that need cleanup before merge:

• _resolve_mx() currently appears malformed/mixed with verification flow logic
• email is referenced inside _resolve_mx() without being defined in scope
• rate-limit rollback using DECR after limit exceed is race-prone under concurrency
• Redis/cache concerns and SMTP verification flow are too tightly coupled right now
• some fallback/error-handling paths need hardening
• input normalization/sanitization for cache + rate-limit keys should be stricter
• async execution/timeouts around SMTP verification need another pass
• there are a few maintainability concerns where large sections feel over-abstracted compared to the current codebase style

Please do a focused cleanup/refactor pass on verifier.py before further review.

Priority should be:

1. correctness
2. concurrency safety
3. Redis failure handling
4. clean separation of responsibilities
5. consistency with existing project architecture/style

Also please test:

• Redis unavailable scenarios
• concurrent verification attempts
• cache hit/miss behavior
• rate-limit expiration/reset behavior
• malformed email input handling

Once those are cleaned up, I’ll continue with a deeper review.

Thanks for the detailed review. I’ll go through the issues you pointed out, research the best approach for the concurrency and Redis handling parts, and do my best to clean up/refactor verifier.py accordingly before the next review.

[revanth127]

Quick update — after spending more time on the verifier refactor and digging deeper into the issues from the review, I realized the async concurrency, Redis coordination, and SMTP reliability parts are currently beyond my experience level to complete confidently to the standard the project needs.

Rather than keep the PR hanging or continue with unreliable changes, I think it’s better for me to step back from this issue for now. I really appreciate the detailed review and guidance though — I learned a lot while attempting it.

Sorry for the inconvenience, and thanks again for the feedback.

[sharmavaibhav31]

Thanks for the honest update, Revanth. Stepping back was definitely the right call unless you wanted to be personally responsible for Redis exploding and rate limits completely losing their minds across our workers.

On paper, your blueprint was solid:

• Moving away from process-local limiting was smart.

• Redis coordination and caching are exactly what we need.

• Blocklisting providers is a great touch.

However, executing it safely means rewriting the whole thing to survive:

• Concurrency issues that will absolutely haunt us.
• Messy Redis interactions and unpredictable async timeouts.
• Schema and maintainability consistency (gotta keep it clean).

Keeping the PR open for now as a draft/reference, but we aren't merging this into main until it gets a heavy architectural cleanup. Thanks for doing the heavy lifting on the research though!