Photo Pinpointer

Guess where a photo was taken: a local, privacy-first photo geolocation tool, plus the pipeline used to fine-tune it on street-level imagery.

Built as an add-on to the open-source God's Eye View globe. The photo never leaves your PC: the only things sent to free public APIs are map bounding boxes and text queries.

How it works

1. Global guess: a local GeoCLIP service predicts the top-k likely coordinates. It can optionally fuse in StreetCLIP and a locally trained classifier head.
2. Text clues: a local Ollama vision model (default qwen2.5vl:7b) reads the signs, shop names, phone numbers and domains it can see. These are then resolved through OpenStreetMap Overpass name search, NANP area codes, country calling codes and ccTLDs.
3. Visual match: it fetches nearby Mapillary street-level images over a grid around each candidate and compares them to the photo with CLIP embeddings. The best match is returned as a proof image.

Repo layout

service/
  geoclip_service.py   local HTTP inference service (POST /predict), binds 127.0.0.1, unloads when idle
  photoGeo.js          Node server routes: /osint/photo/ocr, /textclues, /match, /mapillary/thumb
training/
  download.py          builds a geographically balanced street-level photo set from Mapillary
  osv.py               pulls additional samples from the OSV-5M dataset
  extract.py           caches StreetCLIP and GeoCLIP image features (resumable)
  round3.py            trains a cell/country classifier head on the features and evaluates it on a held-out split
  finetune.py          optional partial fine-tune of the CLIP vision tower's last blocks
  evalr.py             evaluation against the running service (median km error, % within 750 km)
  driver.py            orchestrator: retrains each time enough new photos arrive
  status_writer.py     writes a JSON status file for the Mission Control dashboard
  autostart.ps1        guarded launcher that starts each process hidden only if it isn't already running
  gevpaths.py          path configuration via environment variables

Training is GPU-temperature-aware. It pauses when the GPU runs hot and supports a duty-cycle throttle file.

Tech stack

Python, PyTorch, Hugging Face Transformers, GeoCLIP, StreetCLIP, NumPy, Node.js, Ollama, the Mapillary Graph API, OpenStreetMap Overpass and OSV-5M.

Setup

python -m venv .venv; .venv\Scripts\activate
pip install torch torchvision transformers geoclip pillow numpy
copy .env.example .env      # add your free Mapillary token
python service\geoclip_service.py      # http://127.0.0.1:8765/predict

For training, set GEV_DATA (dataset and feature cache, which needs plenty of disk space) or use the defaults under ./data. Then run training\autostart.ps1, or run the steps one at a time.

Model weights, datasets and feature caches are not included.