ShopRadar

ShopRadar is a competitor-density dashboard for small-business site selection. It uses Taiwan's government open dataset, 全國營業(稅籍)登記資料集, to compare business density by region, district, and industry.

Requirements

• Python 3.10+
• pip

Install dependencies:

pip install -r requirements.txt

Real-Data Mode

Real-data mode is the final project workflow. It downloads and verifies the official government ZIP, keeps the first 100,000 normalized records for detail browsing, and processes the full CSV into district-industry aggregates.

Data source:

• Dataset page: https://data.gov.tw/dataset/9400
• Direct download: https://eip.fia.gov.tw/data/BGMOPEN1.zip

Run:

python scripts/02_download_data.py
python scripts/03_clean_data.py
python scripts/04_build_database.py
streamlit run app/streamlit_app.py
pytest

Generated local artifacts:

• data/raw/BGMOPEN1.zip
• data/raw/BGMOPEN1_extracted/BGMOPEN1.csv
• data/processed/shops_cleaned.csv
• data/processed/district_industry_summary.csv
• data/processed/shopradar.db

Large raw files and the generated SQLite database are excluded from GitHub. Run the commands above to recreate them.

Sample Mode

Sample mode is only for a quick offline demo. It uses data/sample/shops_sample_1000.csv, a lightweight 1,000-row extract derived from the verified government-data pipeline.

python scripts/03_clean_data.py --sample
python scripts/04_build_database.py
streamlit run app/streamlit_app.py

Differences:

• Real-data mode processes the full official CSV for aggregate competition analysis.
• Sample mode processes only 1,000 included rows and is not suitable for real market conclusions.
• The final project demonstration and reported results use real-data mode.

Processing Strategy

scripts/02_download_data.py downloads, extracts, and verifies the source. It records the file name, file size, encoding, row count, columns, update note, and first five rows in:

• docs/data_inventory.md
• outputs/real_data_verification.md

scripts/03_clean_data.py uses pandas chunks of 50,000 rows:

• It stores at most 100,000 normalized detail rows in data/processed/shops_cleaned.csv.
• It reads the complete official CSV to create data/processed/district_industry_summary.csv.
• The aggregate contains region, district, industry_name, competitor_count, total_capital, and avg_capital.

scripts/04_build_database.py creates:

• shops
• district_industry_summary
• metadata

Dashboard

The dashboard prioritizes the full aggregate summary for competition analysis and uses detail rows for record previews and capital distribution.

Features:

• total record, region, district, and industry metrics
• region, district, and industry-keyword filters
• competitor count by district
• top industries
• business count by region
• capital distribution
• competition score from 0 to 100
• risk level and explanation
• downloadable site-selection Markdown report

Cleaned Detail Fields

• record_id
• business_name
• region
• district
• industry_code
• industry_name
• capital
• established_date
• organization_type
• address_partial
• keywords

Limitations

• Address parsing is rule-based.
• Industry matching uses text keywords rather than semantic classification.
• Government tax-registration data may lag real-world openings or closures.
• Sample mode exists for convenience only and must not be presented as full-data analysis.