Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

amazon-asin-scraper-python

Extract, validate and check Amazon ASINs — including the ones that most ASIN regexes silently throw away.

pip install git+https://github.com/thunderbit-operations/amazon-asin-scraper-python

amzn-asin extract page.html          # pull every ASIN out of any text or HTML
amzn-asin check B0CR7N9MD9 --marketplace in   # does it exist on that storefront?
amzn-asin resolve https://amzn.to/3xYzAbC     # follow a short link

Built on amazon-product-scraper-python. Extraction and validation are pure functions with no network access; only check and resolve make requests.


Two things almost every ASIN regex gets wrong

1. B[0-9A-Z]{9} throws away every book on the site

The regex you will find in most tutorials and repos requires a B prefix. But for books with a 10-digit ISBN, the ASIN is the ISBN-10. Which means it can be entirely numeric, and it can end in the check character X:

0143448463    a real, live product page: /dp/0143448463
155404295X    ISBN-10 with an X check character

Both are valid ASINs. Both are rejected by B[0-9A-Z]{9}. The failure is silent and total for the entire books catalogue — you do not get an error, you get an empty result that looks like the page had no products.

The correct validation is ^[A-Z0-9]{10}$ after upper-casing, with no prefix requirement and no checksum: Amazon documents no check-digit scheme for the B0… namespace, so a validator that invents one rejects valid ASINs.

2. An ASIN is not a key on its own

ASINs are marketplace-scoped. Verified live:

ASIN amazon.in amazon.com amazon.co.uk
B0CR7N9MD9 200 404
B08N5WRWNW 200 404 404

The same ten characters resolve to a product on one storefront and to nothing on another. Any store, join or deduplication keyed on a bare ASIN will silently merge or lose rows across marketplaces.

The primary key is (asin, marketplace). This package treats it that way throughout, and check reports per marketplace.


Usage

Extract from anything

amzn-asin extract page.html
amzn-asin extract - < scraped.txt          # stdin
amzn-asin extract urls.txt --json

Handles every URL form seen in the wild — and, importantly, ASINs sitting at an attribute boundary:

/dp/B0CR7N9MD9                     bare path
/dp/B0CR7N9MD9/ref=sr_1_3          with a ref suffix
/Some-Product-Name/dp/B0CR7N9MD9   slug form
/-/en/dp/B0CR7N9MD9                locale-prefixed
/gp/product/B0CR7N9MD9             legacy path
/gp/aw/d/B0CR7N9MD9                mobile
?asin=B0CR7N9MD9                   query parameter
<a href="/dp/B0CR7N9MD9">          ← the one that breaks naive patterns

That last case matters more than it looks. A pattern anchored with (?:[/?#]|$) matches the first six and misses the seventh — which is the single most common form on a real listing page. The result is an empty list from a page full of products, with no error raised.

Results come back in document order, deduplicated. Order is not cosmetic here: if you are reading a search results page, position is the data, and an implementation that iterates URL patterns in the outer loop returns results grouped by which pattern matched rather than by where they appeared.

Check existence

amzn-asin check B0CR7N9MD9 --marketplace in
amzn-asin check asins.txt --marketplace us --out results.csv

Distinguishes a real 404 from an anti-bot challenge — which matters because Amazon's three block types all return 2xx status codes. A checker that reads resp.status_code will report a blocked page as "exists" and a challenge as success. Here a blocked check is reported as blocked_*, never as a verdict about the ASIN.

A genuine 404 is also never retried against another proxy: it is a catalogue fact, not an IP problem, and hammering it burns reputation for nothing.

Resolve short links

amzn-asin resolve https://amzn.to/3xYzAbC

amzn.to, amzn.eu, a.co and amzn.asia carry no ASIN — they need a redirect-following request. This is a separate command rather than something extract does implicitly, because it is a network call with its own failure and blocking modes, and it should be a visible decision.

As a library

from amazon_asin_scraper import extract_all, is_valid, normalise

extract_all(html)          # ['B0CR7N9MD9', '0143448463', ...] in document order
is_valid("0143448463")     # True  -- a book, no B prefix
normalise(" b0cr7n9md9 ")  # 'B0CR7N9MD9'

import asyncio
from amazon_asin_scraper import check

async def main():
    result = await check("B0CR7N9MD9", marketplace="in")
    print(result.exists, result.status, result.title)

asyncio.run(main())

Legal and responsible use

Extraction and validation are offline and unrestricted. The check and resolve commands make requests, and they are rate limited (~1 request/second per IP, jittered) and robots.txt-aware by default.

This package ships no CAPTCHA solving, no proof-of-work computation and no rate-limit evasion, and pull requests adding them will be declined. See the full legal section in the core project before deploying. Nothing here is legal advice.

This project is not affiliated with, authorised or endorsed by Amazon.

Who maintains this

Built and maintained by the team behind Thunderbit, an AI web scraper, alongside the core library this package is a thin layer over.

Extraction and validation are pure functions with no network access at all. No account, API key or quota is involved anywhere — nothing here calls a Thunderbit service.

Licence

Apache-2.0 — free for commercial use. See LICENSE.

About

Extract, validate and check Amazon ASINs from Python — including the book ASINs (ISBN-10) that prefix-requiring regexes silently discard.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages