Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,15 @@ minor versions can still change behaviour.

### Added

- **The free OpenStreetMap source knows tourism and hospitality.** Hotels,
motels, guesthouses and pensions, vacation rentals, campsites, restaurants,
cafés, bars, dive and surf centres, boat and car rentals, travel agencies and
excursions, estate agents, wedding and event planners, spas, in English and
French (`hôtel`, `pension`, `plongée`, `agence immobilière`, `mariage`…).
Each maps to the OSM tags that carry those premises, and a found business
keeps its `tourism`/`amenity` category. Before, every one of these terms
raised "OpenStreetMap has no category" and the run exited non-zero. A term
OSM has no category for still says so: no catch-all.
- **`writer.require_tip: true` skips prospects with nothing to build a tip
on.** A first email is meant to name one concrete thing the sender would
change; with no finding about the company, the draft could only say it had no
Expand Down
2 changes: 1 addition & 1 deletion docs/signals-and-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ sellbird discover # go

| Source | Cost | Free tier | Best for |
|---|---|---|---|
| **OpenStreetMap** *(default)* | free | unlimited, no account | Trying the whole pipeline before paying anyone. Coverage is thin for businesses without a storefront: OSM maps premises, so whole trades are barely present (under 2,000 roofers and under 2,300 plumbers in the entire US). Sellbird says so in the UI rather than quietly under-delivering. |
| **OpenStreetMap** *(default)* | free | unlimited, no account | Trying the whole pipeline before paying anyone, and a real list for premises-based businesses: hotels, guesthouses, vacation rentals, restaurants, cafés, dive centres, boat and car rentals, travel agencies, estate agents, wedding and event planners, spas (English and French terms). Coverage is thin for businesses without a storefront: OSM maps premises, so whole trades are barely present (under 2,000 roofers and under 2,300 plumbers in the entire US). Sellbird says so in the UI rather than quietly under-delivering. |
| **TinyFish** | free | 30 searches a minute, 500 an hour, no card | Finding businesses by phrase (including the startup and hiring presets) at no cost. Records no search rank: its ordering isn't Google's and it takes no location, so use Serper or DataForSEO SERP for rank signals. |
| **DataForSEO Business Listings** | $0.372 / 1,000 | $1 credit | Local trades, clinics, contractors. Phone, domain, rating, claimed status — and it can filter for businesses with **no website at all**. |
| **DataForSEO SERP** | $0.006 / search at depth 30 (Live, $0.002 per 10 results) | $1 credit + free sandbox | Organic rank as the buying signal, plus who holds the local 3-pack, searched from the city centre. |
Expand Down
2 changes: 1 addition & 1 deletion docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

**Codex calls fail** — Read the log line; every Codex error says what to do. "Not logged in" or "expired or is due for renewal" → use Codex once or run `codex login` with ChatGPT. "Hasn't been verified with Sellbird" → install the supported Codex version named in the message. "Isolation could not be verified" → a Codex config layer outside your `~/.codex` (for example a managed or `/etc/codex` config) forces a tool, hook, MCP server, instructions or custom endpoint back on; Sellbird refuses to send prompts until it's gone. Model catalog errors → choose an available model ID from the catalog shown in the message, or leave `brain.model` blank to select a listed model. Sellbird never switches to Claude on its own.

**Discovery finds nothing** — Confirm signals first (`sellbird signals --confirm free`); nothing is collected until you do. If you're on the free OpenStreetMap source, it only covers mapped trades and is thin for service-area businesses — `sellbird discover --providers` shows the alternatives.
**Discovery finds nothing** — Confirm signals first (`sellbird signals --confirm free`); nothing is collected until you do. If you're on the free OpenStreetMap source, it only covers mapped categories (home-services trades, and tourism and hospitality: hotels, guesthouses, restaurants, dive centres, rentals, estate agents, event planners) and is thin for service-area businesses — `sellbird discover --providers` shows the alternatives.

**Overpass is throttling** — It's free volunteer infrastructure. Sellbird walks three mirrors before giving up. Wait a few minutes, or use a paid provider for bulk work.

Expand Down
57 changes: 56 additions & 1 deletion sellbird/collectors/discover.py
Original file line number Diff line number Diff line change
Expand Up @@ -1115,6 +1115,59 @@ class OpenStreetMap(DiscoveryProvider):
"panader": ['shop=bakery'],
"clinica": ['amenity=clinic'],
"clínica": ['amenity=clinic'],
# Tourism and hospitality, English and French. OSM maps these well:
# a hotel or a restaurant is a premises by definition.
"hotel": ['tourism=hotel'],
"hôtel": ['tourism=hotel'],
"motel": ['tourism=motel'],
"resort": ['tourism=hotel', 'leisure=resort'],
"guesthouse": ['tourism=guest_house'],
"guest house": ['tourism=guest_house'],
"pension": ['tourism=guest_house'],
"chambre d'h": ['tourism=guest_house'], # chambre d'hôte(s)
"chambres d'h": ['tourism=guest_house'],
"bed and breakfast": ['tourism=guest_house'],
"b&b": ['tourism=guest_house'],
"hébergement": ['tourism=hotel', 'tourism=guest_house', 'tourism=chalet'],
"hebergement": ['tourism=hotel', 'tourism=guest_house', 'tourism=chalet'],
"vacation rental": ['tourism=chalet', 'tourism=apartment'],
"location de vacances": ['tourism=chalet', 'tourism=apartment'],
"location saisonni": ['tourism=chalet', 'tourism=apartment'],
"bungalow": ['tourism=chalet'],
"camping": ['tourism=camp_site'],
"restaurant": ['amenity=restaurant'],
"café": ['amenity=cafe'],
"cafe": ['amenity=cafe'],
"snack": ['amenity=fast_food'],
"plongée": ['shop=scuba_diving', 'sport=scuba_diving'],
"plongee": ['shop=scuba_diving', 'sport=scuba_diving'],
"diving": ['shop=scuba_diving', 'sport=scuba_diving'],
"dive center": ['shop=scuba_diving', 'sport=scuba_diving'],
"dive shop": ['shop=scuba_diving', 'sport=scuba_diving'],
"scuba": ['shop=scuba_diving', 'sport=scuba_diving'],
# Short stems like "bar" or "spa" are left out on purpose: they would
# match "barber" and "spare parts".
"surf": ['shop=surf', 'sport=surfing'],
"boat rental": ['amenity=boat_rental', 'shop=boat'],
"location de bateau": ['amenity=boat_rental', 'shop=boat'],
"boat tour": ['tourism=attraction', 'amenity=boat_rental'],
"excursion": ['shop=travel_agency', 'office=travel_agent', 'amenity=boat_rental'],
"tour operator": ['shop=travel_agency', 'office=travel_agent'],
"agence de voyage": ['shop=travel_agency', 'office=travel_agent'],
"travel agency": ['shop=travel_agency', 'office=travel_agent'],
"car rental": ['amenity=car_rental'],
"location de voiture": ['amenity=car_rental'],
"location de véhicule": ['amenity=car_rental'],
"real estate": ['office=estate_agent'],
"estate agent": ['office=estate_agent'],
"realtor": ['office=estate_agent'],
"immobili": ['office=estate_agent'], # immobilier, immobilière
"wedding": ['shop=wedding', 'office=event_management'],
"mariage": ['shop=wedding', 'office=event_management'],
"event planner": ['office=event_management'],
"événementiel": ['office=event_management'],
"evenementiel": ['office=event_management'],
"massage": ['shop=massage'],
}

def _tags(self, term: str) -> list[str]:
Expand Down Expand Up @@ -1201,7 +1254,9 @@ async def fetch(self, client, env, query):
address=" ".join(x for x in (tags.get("addr:housenumber"),
tags.get("addr:street")) if x),
location=city or query.location,
category=tags.get("craft") or tags.get("shop") or tags.get("office") or "",
category=(tags.get("craft") or tags.get("shop") or tags.get("office")
or tags.get("tourism") or tags.get("amenity")
or tags.get("leisure") or tags.get("sport") or ""),
has_website=bool(site),
external_id=f"osm:{el.get('type')}/{el.get('id')}",
source_url=f"https://www.openstreetmap.org/{el.get('type')}/{el.get('id')}",
Expand Down
49 changes: 49 additions & 0 deletions tests/test_discover.py
Original file line number Diff line number Diff line change
Expand Up @@ -925,6 +925,55 @@ def handler(request):
assert seen == list(D.OpenStreetMap.ENDPOINTS), "every mirror should be tried"


@pytest.mark.parametrize("term, expected", [
("hôtel Moorea", ["tourism=hotel"]),
("Hotel Papeete", ["tourism=hotel"]),
("pension de famille Moorea", ["tourism=guest_house"]),
("restaurant Papeete", ["amenity=restaurant"]),
("centre de plongée Rangiroa", ["shop=scuba_diving", "sport=scuba_diving"]),
("location de voiture Tahiti", ["amenity=car_rental"]),
("agence immobilière Punaauia", ["office=estate_agent"]),
("real estate agency Denver", ["office=estate_agent"]),
("organisateur de mariage Bora Bora", ["shop=wedding", "office=event_management"]),
("wedding planner Austin", ["shop=wedding", "office=event_management"]),
])
def test_osm_knows_tourism_and_hospitality_categories(term, expected):
"""Hotels, guesthouses, restaurants and the rest of the tourism trade are
premises, which is exactly what OpenStreetMap maps. Before this they all
raised "no category", in English and in French alike."""
assert D.OpenStreetMap()._tags(term) == expected


def test_osm_still_refuses_a_term_it_has_no_category_for():
"""No catch-all: a trade OSM doesn't carry says so instead of answering
with every registered office in town."""
assert D.OpenStreetMap()._tags("quantum consultancy Denver") == []


@pytest.mark.asyncio
async def test_osm_keeps_a_hotel_category_and_website(monkeypatch):
monkeypatch.setattr(D.OpenStreetMap, "BACKOFF", 0.0)

def handler(request):
body = request.content.decode()
assert "tourism=hotel" in body
return httpx.Response(200, json={"elements": [{
"type": "way", "id": 7, "center": {"lat": -17.5, "lon": -149.8},
"tags": {"name": "Lagoon View Lodge", "tourism": "hotel",
"website": "https://lagoon-view.example",
"addr:city": "Moorea"},
}]})

provider = D.OpenStreetMap()
async with _client(handler) as client:
out = await provider.fetch(client, {}, D.DiscoveryQuery(
term="hôtel", coordinate="-17.5,-149.8,20"))
assert len(out) == 1
assert out[0].category == "hotel"
assert out[0].domain == "lagoon-view.example"
assert out[0].external_id == "osm:way/7"


@pytest.mark.asyncio
async def test_overpass_falls_over_to_the_next_mirror(monkeypatch):
monkeypatch.setattr(D.OpenStreetMap, "BACKOFF", 0.0)
Expand Down