diff --git a/docs/design/starter-credits-bridge/README.md b/docs/design/starter-credits-bridge/README.md index 18f5bb85b34..20c522790d2 100644 --- a/docs/design/starter-credits-bridge/README.md +++ b/docs/design/starter-credits-bridge/README.md @@ -79,6 +79,7 @@ here. Those arrive in the policy payload. See | File | Answers | | --- | --- | -| [design.md](design.md) | Every decision, the options it was chosen against, and why | +| [design.md](design.md) | Every decision, the options it was chosen against, and why. Includes the plus-tag eligibility rule and what a refused signup sees | +| [anti-abuse-research.md](anti-abuse-research.md) | Libraries and hosted services that can own disposable-email and mailbox-canonicalization instead of homemade rules | | [write-only-secrets.md](write-only-secrets.md) | The vault contract the seeded row depends on: write-only values and managed rows | | [proxy-and-deployment.md](proxy-and-deployment.md) | What the proxy must look like, how it is routed, and what that constrains | diff --git a/docs/design/starter-credits-bridge/anti-abuse-research.md b/docs/design/starter-credits-bridge/anti-abuse-research.md new file mode 100644 index 00000000000..84e5e21b25b --- /dev/null +++ b/docs/design/starter-credits-bridge/anti-abuse-research.md @@ -0,0 +1,206 @@ +# Signup-email anti-abuse: libraries and services + +The plus-tag rule in [design.md](design.md) is a stopgap. This page is the survey of +things that can own the rest of the job: disposable domains, mailbox canonicalization +(Gmail dots, plus tags, `googlemail.com`), and hosted signup-fraud products. + +The constraint that every option has to meet: account creation stays open. A failed check +only skips the credit grant. A vendor outage must fail open to "no grant" or fall back to +the local list. It must never fail signup. + +No single library owns the whole problem. Split it: + +| Job | What it is | Can a list do it? | +| --- | --- | --- | +| Disposable / throwaway domain | mailinator, guerrilla, rotating MX farms | Yes, if the list is fresh | +| Alias / plus / Gmail dots | same inbox, many strings | Local normalizer, not a blocklist | +| Freemail vs work | gmail vs acme.com | Separate list or API flag | +| Fake account / bot / free-tier farm | device, IP, velocity, graph | Hosted fraud product | + +## Disposable-domain lists + +### disposable-email-domains + +- Source: [disposable-email-domains/disposable-email-domains](https://github.com/disposable-email-domains/disposable-email-domains), Python package [disposable-email-domains](https://pypi.org/project/disposable-email-domains/). +- License: list is CC0. PyPI wrapper is MIT. Maintained by Dustin Ingram (`@di`, PyPI). +- Shape: `from disposable_email_domains import blocklist`. In-process set. +- Freshness: PyPI 0.0.243 on 24 Aug 2026. Releases almost daily. 5.4k GitHub stars. +- Detects: known disposable / temporary domains. Maintainers require a screenshot of a page that generates an address on that domain. Second-level domains only; callers must match public-suffix parents (`foo.mailinator.com` to `mailinator.com`). +- Privacy: the email never leaves the process. +- False positives: lowest among public lists. It will miss brand-new rotating domains for days. It does not treat Proton, SimpleLogin, or Gmail as disposable. +- Signup fit: yes. Fail open on import error. Skip the grant only on a hit. +- Used in the wild: PyPI itself blocks throwaway domains with this list ([PyPI blog, Jun 2024](https://blog.pypi.org/posts/2024-06-16-prohibiting-msn-emails/)). + +### MailChecker + +- Source: [FGRibreau/mailchecker](https://github.com/FGRibreau/mailchecker), PyPI `mailchecker`. +- License: MIT. Python, JS, Go, Rust, PHP, Ruby. +- Freshness: PyPI 6.0.21 on 19 Jul 2026. Claims 55,000+ domains. +- Detects: regex format plus a large disposable-domain set. +- False positives: higher than disposable-email-domains. Bigger list, weaker admission bar. +- Signup fit: same fail-open pattern. Prefer the smaller list unless obscure throwaways become the problem. + +### Castle top-1,000 abuse list + +- Source: [castle/disposable-email-domains](https://github.com/castle/disposable-email-domains), announced [11 May 2026](https://blog.castle.io/inside-the-infrastructure-behind-fake-signups-our-open-source-disposable-email-domain-list/). +- Shape: one domain per line, ranked by observed abuse. +- Detects: the 1,000 disposable domains Castle actually sees in fake-signup traffic. They do not import other public lists. They exclude privacy relays (SimpleLogin, Addy). +- Signup fit: a second set unioned with disposable-email-domains. Not enough alone. Check the repo LICENSE before vendoring. + +### Lists that are a bad first choice + +| Name | Why not | +| --- | --- | +| [disposable/disposable](https://github.com/disposable/disposable) | ~100k domains, daily scrape of other lists. High false positives, no admission bar. | +| [7c/fakefilter](https://github.com/7c/fakefilter) | Decent middle ground. Smaller community, no Python package. | +| [wesbos/burner-email-providers](https://github.com/wesbos/burner-email-providers) | Slow updates, documented false-positive PRs. | +| [willwhite/freemail](https://github.com/willwhite/freemail) | Looks stale. Do not vendor. | +| WhoisXML disposable DB | Paid daily dump. Vendor lock. Overkill. | + +## Mailbox canonicalization + +A blocklist cannot do plus tags, Gmail dots, or `googlemail.com`. A one-line +`if "+" in local` also cannot do Gmail dots, Workspace-vs-consumer Gmail, Fastmail +subdomains, or `me.com` to `icloud.com`. + +### email-normalize + +- Source: [gmr/email-normalize](https://github.com/gmr/email-normalize), PyPI [email-normalize 3.2.0](https://pypi.org/project/email-normalize/). +- License: BSD-3-Clause. Python 3.11+. Sync `email_normalize.normalize(...)` and an async `Normalizer`. Optional MX lookup via `aiodns` + `tldextract`. +- Freshness: 3.2.0 on 29 Jul 2026. Active in 2026. +- Rewrites: + - Plus tags: Google, Microsoft, Apple, Fastmail, Proton, Rackspace, Yandex, Zoho. + - Strip dots: consumer Gmail / googlemail only. Google Workspace custom domains keep dots (Workspace treats dots as real). + - Domain folding: `googlemail.com` to `gmail.com`; `me.com` / `mac.com` to `icloud.com`. Does not fold `outlook.com` into `hotmail.com` (those are different mailboxes). + - Fastmail "local part as hostname" on custom domains. + - Yahoo: no plus-strip (Yahoo plus is not a real alias in the same way). +- Privacy: local. `skip_dns=True` uses a static domain map. Default mode does MX lookups so `user+x@custom-google-workspace.com` still normalizes. MX talks to DNS, not a vendor, and does not send the local part. +- False positives: low if the result is a **dedupe / velocity key**, not the stored login email. Send mail to the original address. +- Signup fit: yes. Normalize, then key Redis velocity on the canonical mailbox. Do not rewrite the auth-stored email. + +### Weaker options + +| Name | Why not | +| --- | --- | +| [CorentinTh/email-normalizer](https://github.com/CorentinTh/email-normalizer) | TypeScript, 10 stars, Gmail/Hotmail/Live/Outlook only. | +| [JoshData/python-email-validator](https://github.com/JoshData/python-email-validator) | Syntax + DNS. Does not strip plus or Gmail dots. | +| Homemade `local.split("+")[0]` | Misses Gmail dots, `googlemail.com`, the Workspace exception, Fastmail, Apple folding. Fine as the 1-hour stopgap that [design.md](design.md) ships. Not the long-term owner. | + +MaxMind publishes a [minFraud email-normalization recipe](https://dev.maxmind.com/minfraud/normalizing-email-addresses-for-minfraud/) (trim, lowercase, NFC, then provider rules, then hash). Useful as a spec if a hash ever goes to a fraud vendor. Not a library. + +## Hosted email APIs + +These send the address (or the domain) to a vendor. Most of this market is deliverability +(will this bounce?), not signup fraud. Read the flags, not the homepage. + +### UserCheck + +- Site: [usercheck.com](https://www.usercheck.com/), [docs](https://www.usercheck.com/docs). +- Shape: HTTP. `GET /email/{email}` or `GET /domain/{domain}`. Fine from FastAPI. +- Detects: + - `disposable` at domain or address level, including throwaways on Gmail/Outlook after normalization. + - `normalized_email` (plus tags, provider aliases, domain folding). Compare `email` vs `normalized_email`. Send mail to the original. + - `public_domain` (freemail: Gmail, Yahoo, Outlook). This can replace the 74-domain list. + - `relay_domain` (SimpleLogin, Apple Hide My Email, DuckDuckGo). + - `role_account`, `spam`, `free_subdomain`, MX, typo suggestion. + - Domain-only endpoint: same disposable / public / relay signals without sending the local part. +- Cost: 1,000 credits/month free, 1 req/s. Paid plans are usage-priced. +- Privacy: the email endpoint sends the full address. The domain endpoint does not. That is the cleanest hosted privacy option in this survey. EU hosted, GDPR, DPA on paid plans. +- False positives: they only mark `disposable: true` when the service is confirmed. Relays are a separate flag, so Proton/SimpleLogin stay allowed unless we choose otherwise. +- Signup fit: yes. Short timeout. On 5xx / timeout, keep the local list + velocity. Never block account creation. + +### Kickbox, ZeroBounce, NeverBounce, Mailgun, Abstract, IPQS + +| Vendor | Useful flag | Cost (public) | Fit | +| --- | --- | --- | --- | +| [Kickbox](https://docs.kickbox.com/docs/single-verification-api) | `disposable`, `free`. "Normalized" is lowercase only. | 100 free, then about $0.008–$0.01/check | Deliverability-first. Official Python SDK. Overkill if we only need a boolean. | +| [ZeroBounce](https://www.zerobounce.net/docs/email-validation-api-quickstart/v2-status-codes) | `disposable`, `free_email`. Also `abuse` / `do_not_mail`. | About $8 / 1,000 | Mailing-list hygiene. `abuse` will over-flag people we still want as users. | +| [NeverBounce](https://www.neverbounce.com/email-verifier) | disposable / accept-all | About $0.008/check | Same class as Kickbox. | +| [Mailgun Validate](https://www.mailgun.com/features/email-validation-api/) | MX, disposable, role, risk | Bundled with Mailgun | Fine if we already pay Mailgun. Not worth adding an ESP for this. | +| [Abstract](https://www.abstractapi.com/api/email-verification-validation-api) | disposable, free, `is_subaddress` (plus tags) | 100 free/month, then $17/month for 5,000 | Decent flags. Sends full email. 3 req/s cap. | +| [IPQualityScore](https://www.ipqualityscore.com/documentation/email-validation-api/overview) | `disposable`, `fraud_score`, `recent_abuse` | About $5 / 1,000 at the low end | Fraud-intel vendor. Higher false-positive risk. Higher tiers attach identity. Cannot justify that for a credit-grant skip. | + +## Broader signup-fraud products + +Realistic only if email rules stop being enough (device farms, residential proxies, stolen +Gmail). + +### Trueguard + +- Site: [trueguard.io](https://trueguard.io/). +- What it is: free-tier abuse product. JS snippet + backend API. Device ID, disposable-email intel, IP/VPN/proxy, bot score, rules engine. +- Cost: 1,000 events/month free. Standard $49.99/month for 10,000. +- Privacy: email + IP + device fingerprint leave the browser. DPA available. +- Signup fit: the closest hosted product to "people farm starter credits." Heavier than an email-only first step. Requires a frontend snippet. + +### Cloudflare Account Abuse Protection + +- Docs: [Account Abuse Protection](https://developers.cloudflare.com/bots/account-abuse-protection/) (Early Access, updated 1 Jul 2026). +- What it is: Bot Management Enterprise add-on. Fields: `cf.fraud_detection.disposable_email` / `disposable_domain`, `cf.fraud.email_risk`. Hashed per-zone User ID. Works with Turnstile ephemeral IDs. +- Cost: no extra charge for a limited period until GA. Requires Bot Management Enterprise. Not on the free/pro plan. +- Privacy: they say they do not store email addresses. User IDs are hashed per zone. +- False positives: `email_risk=high` on a random local part will hit some real people. Use it to skip credits, not to challenge signup. +- Signup fit: excellent if traffic already sits on Cloudflare with Bot Management. Useless as a standalone library. Complementary to Turnstile, not a replacement for a Python package. + +### Too heavy for this problem + +| Product | Why skip now | +| --- | --- | +| [Castle](https://castle.io) | Full device + behavior + graph. The open list is useful. The product is a trust-and-safety platform. | +| [Sift](https://sift.com) | Enterprise score, sales-led. Overkill for skipping a small grant. | +| [Sumsub](https://sumsub.com/account-fraud-prevention/) | KYC / identity. Wrong layer. | +| [Amazon Fraud Detector](https://aws.amazon.com/blogs/machine-learning/prevent-fake-account-sign-ups-in-real-time-with-ai-using-amazon-fraud-detector/) | Needs labeled history. We do not have it. | +| Arkose, Ping Identity | Bot/ATO suites. Pair with Turnstile later, not with the email policy. | + +## Do not pick + +- Abandoned or stale lists (`willwhite/freemail`, old `django-email-blacklist`). Disposable providers rotate weekly. +- Giant aggregated lists (`disposable/disposable` at ~100k). They block real users and still miss tomorrow's domain. +- Sending the full email to a deliverability vendor we do not already use (Kickbox, ZeroBounce, NeverBounce, Mailgun Validate) just to read a `disposable` boolean. That is personal data for a problem a local set solves. +- Enrichment APIs that attach names, phones, or "who is this person." +- SMTP "does this mailbox exist" probes. Extra latency. Some providers tarpit or lie. Different question than "will this person farm credits." +- Cloudflare AAP without Bot Management Enterprise. The fields will not exist. +- GPL infection is not a live risk here. MailChecker, disposable-email-domains, and email-normalize are MIT / CC0 / BSD-3. + +## Shortlist + +### 1. disposable-email-domains + email-normalize + +Use this if we want to stop writing rules and never send signup emails to a vendor. + +- `pip install disposable-email-domains email-normalize` +- On mint: if the domain is in `blocklist`, skip the grant. Normalize with `skip_dns=True` (or async MX if Workspace plus-tags matter) and key Redis velocity on the canonical mailbox. +- Keep the free-mail tuple for a while, or replace it later with a maintained public-domain set / UserCheck `public_domain`. +- Keep Redis caps. They still do work no list can do. +- Drop the homemade plus ban once normalization is live, or keep it as a belt on top. + +Privacy: none leaves the process. Cost: none. Latency: microseconds. Maintenance: Dependabot on two small packages. + +### 2. UserCheck (domain endpoint first) + +Use this if we want a vendor to own disposable + freemail + relay + canonical form, with a +privacy switch. + +- Domain-only call: no local part leaves Agenta. Replaces the disposable list and can replace the 74-domain free-mail list (`public_domain`). +- Email call: adds `normalized_email` and address-level disposable-on-Gmail. +- Timeout 200–400 ms. On failure, fall back to the local list. Skip the grant only. +- 1,000 free checks/month is enough to trial on cloud. + +### 3. Trueguard + +Use this if credit farming is already a device/IP problem, not just an email-string +problem. $50/month at 10k events. Requires a JS snippet and a DPA conversation. Do not +start here. + +### 4. Cloudflare Account Abuse Protection + +Use this if signup traffic already terminates on Cloudflare Bot Management Enterprise. +Pair with option 1 for OSS / self-hosted, where Cloudflare is not in the path. + +## Recommended sequence + +1. Ship the hardcoded plus-tag rule in [design.md](design.md). One check. Closes the hole we can see today. +2. Adopt option 1 (`disposable-email-domains` + `email-normalize`) in the mint path. That retires the homemade plus rule as the owner and closes Mailinator and Gmail-dot twins. +3. Spike UserCheck domain-only on a shadow log for two weeks. Compare hits against option 1. Decide if `public_domain` can replace `DEFAULT_FREEMAIL_DOMAINS`. +4. Do not buy Kickbox, ZeroBounce, or IPQS for this. Do not stand up Sift, Castle, or Sumsub for this. +5. Revisit Trueguard or Cloudflare AAP only after residual grant abuse is not an email-string problem. diff --git a/docs/design/starter-credits-bridge/design.md b/docs/design/starter-credits-bridge/design.md index 0c6c56788e2..14a9b981bd2 100644 --- a/docs/design/starter-credits-bridge/design.md +++ b/docs/design/starter-credits-bridge/design.md @@ -125,7 +125,8 @@ connection is Agenta's") beats a per-field verdict that always says no. The mint policy is nine values: the grant, three per-key limits (concurrency, requests per minute, tokens per minute), three velocity caps, the free-mail domain list, and one -eligibility rule. None of them lives in this repository. They arrive as the payload of a +eligibility rule (digit locals). The plus-tag rule is hardcoded, like the free-mail list, +and is not in the payload. None of the money values lives in this repository. They arrive as the payload of a PostHog feature flag whose name is the only configurable part (`AGENTA_STARTER_CREDITS_BRIDGE_POLICY_FLAG`). @@ -195,18 +196,75 @@ Velocity caps then bound the population that does qualify. They are counted in R against expiring keys: a global daily count, a global hourly count, and a per-domain daily count that applies only to domains that are not free mail. A cap on the world's most common mailbox provider would treat every personal address as one company and would start -refusing real people immediately. One eligibility rule rides alongside them: on a -non-free-mail domain, a digit in the local part of the address is refused, which is the -throwaway-address pattern. On free mail, digits are ordinary and stay allowed. +refusing real people immediately. + +Two eligibility rules ride alongside the caps. On a non-free-mail domain, a digit in the +local part of the address is refused (`john99@acme.com`). That is the throwaway-address +pattern on a company mailbox. On free mail, digits are ordinary and stay allowed. A plus +sign in the local part is refused on every domain (`jane+1@gmail.com`, +`jane+trial@acme.com`), except `agenta.ai`. That is the mailbox-alias pattern: one inbox, +many strings, one grant each. The plus check is hardcoded, like the free-mail list. It is +not a PostHog field. Adding a required field to `MintPolicy` would stop every mint until +the live payload was edited, and plus-tagging is an industry abuse pattern, not an +operator money knob. Internal testers who need a plus tag use `@agenta.ai`. + +The plus check applies to free mail as well as work mail. Plus-farming is a Gmail habit. +The digit rule stays work-only, because `john99@gmail.com` is a normal personal address. Two properties of the counting are worth stating. If Redis is unreachable, seeding is skipped, because a mint nobody can count is a mint nobody can bound. And when an attempt consumes a counter slot but funds no key, the slot is handed back, so a proxy outage does -not quietly eat the day's allowance. - -The known gap: there is no per-address cap, because the signup hook never sees the -caller's network address. The domain rules and whatever bot resistance fronts signup carry -that load. +not quietly eat the day's allowance. Eligibility refusals (digit, plus) run before the +counters, so a refused address never consumes a slot. + +### What a refused signup sees + +A failed eligibility rule does not fail signup. The account is created. No virtual key is +minted. No vault row is written. The organization meets the same connect-your-key wall +every organization met before the bridge existed. A warning is logged with `rule` and +`domain` only. The local part is never logged. There is no operator alert. Alerts are for +outages (a missing policy, an unverifiable team, a seed that raised), not for a person +who did not qualify. + +That silence is deliberate. The grant is a gift, not a signup gate. Telling the person +"your plus tag blocked the free model" teaches the farm how the filter works and still +leaves them without a key they can use. + +### Mailbox tricks the plus rule does not catch + +A plus tag is the cheapest farm. It is not the only one. + +| Trick | Example | What we do | +| --- | --- | --- | +| Plus tags | `jane+1@gmail.com` | Refuse, except `agenta.ai` | +| Gmail dots | `j.ane@gmail.com` is the same inbox as `jane@gmail.com` | Not yet. Next homemade rule if we stay homemade | +| `googlemail.com` | same inbox as `gmail.com` | Already classified as free mail. Not folded into one mailbox | +| Disposable domains | mailinator, guerrilla, rotating MX farms | Not in the free-mail list. A maintained blocklist owns this better than we do | +| Yahoo hyphen tags | `jane-tag@yahoo.com` | Not yet. Real hyphens exist, so a blunt ban has more false positives | +| Catch-all company domains | `bot1@acme.com` | Bounded by `work_domain_daily` and the digit rule | +| Many orgs from one account | loop `POST /organizations/` | Already blocked. Only the signup path seeds | +| Unicode lookalikes | homoglyphs in the local part | Not yet. Rare, more false positives | + +The known gap that none of those close: there is no per-address cap, because the signup +hook never sees the caller's network address. The domain rules and whatever bot +resistance fronts signup carry that load. + +A homemade plus ban is a stopgap. It punishes a real work habit +(`billing+agenta@acme.com`) and misses Gmail-dot twins. The longer-lived owner is a +normalizer plus a disposable-domain list, or a vendor that already maintains both. That +survey is in [anti-abuse-research.md](anti-abuse-research.md). The plus rule still ships +first: it is one check, it closes the hole we can see today, and it does not send signup +addresses to anyone. + +### How the plus rule is implemented + +The check lives in `_mint_policy_allows`, next to the digit rule, before Redis counters. +The allowlist is a constant (`agenta.ai`, compared case-insensitively on the domain). +Tests cover: refuse `a+b@gmail.com`, refuse `a+b@acme.com`, allow `a@gmail.com`, allow +`a+b@agenta.ai`, no counter bump on refuse, log `rule=plus_local_part` and the domain +only. No PostHog payload change. The development policy does not need a new field. + +This is a follow-up on `release/v0.114.0`. It does not reopen the seeding PR. ## The always-on ceiling: the program team