FlatScraper

FlatScraper is a lightweight Java 17 Telegram bot that monitors apartment rental listings on OLX.pl and Otodom.pl, filters them using one JSON configuration, and sends new matches to every chat that has activated /start.

Why this project exists

This project was written for Poland and is tailored to Polish rental listings, including PLN prices, czynsz, OLX.pl, and Otodom.pl.

I wrote FlatScraper when I urgently needed an apartment. Because of my other responsibilities, I could not monitor rental websites every half hour. The bot does that work in the background and sends matching listings to Telegram as they appear.

Features

• city, room-count, rent, total-price, and area ranges;
• at least one required word or phrase;
• excluded words and phrases;
• one message per listing;
• photos, publication/update date, floor, district, area, price, additional czynsz, total price, advertiser type, and link;
• /start, /stop, and /status commands;
• separate delivered-listing history for every chat;
• continues with the second source when the first one is temporarily unavailable;
• rotation of HTTP, SOCKS4, and SOCKS5 proxies from a text file;
• lightweight polling of the first page after the current result set is exhausted;
• rate limiting for HTTP and Telegram requests, plus a listing-details cache.

Running

1. Create a bot through @BotFather and obtain its token.
2. Build and run the application for the first time:

.\gradlew.bat clean build
java -jar build\libs\flatscraper.jar

3. If config.json does not exist, the bot creates it with all default keys and exits. Set the token and your search criteria in the new file.
4. Run java -jar build\libs\flatscraper.jar again.

You can pass a different configuration file as the first argument:

java -jar build\libs\flatscraper.jar C:\configs\flatscraper.json

After startup, each user must send /start to the bot once.

Configuration

config.json is created automatically on the first launch and is excluded from Git so that the bot token is not committed to the repository.

• search.requiredPhrases: at least one phrase must match.
• search.requiredPhrasesMaxRooms: when set, required phrases are checked only for apartments with no more than this many rooms. For example, 2 allows apartments with 3+ rooms to pass without a phrase, but requires one for two-room apartments. null checks phrases for every room count.
• search.excludedPhrases: a match for any phrase rejects the listing.
• Phrase matching is case-insensitive and uses the title together with the full description.
• City, base price, area, and room count are passed directly to OLX/Otodom search URLs to avoid downloading obviously unsuitable listings. The bot checks these constraints again locally after downloading a listing.
• Set any unused range boundary to null.
• minPricePln and maxPricePln apply to rent without additional czynsz.
• maxTotalPricePln limits pricePln + additionalRentPln. If czynsz is not found, the total price equals the base rent.
• If a listing does not contain a property for which a range is configured, it is not sent. The bot does not treat an unchecked listing as a match.
• scheduler.intervalSeconds is the delay after a cycle that sent new listings.
• scheduler.noNewListingsIntervalSeconds controls polling after the current result set is exhausted. Only the first page of each source is loaded in this mode.
• scraping.maxPagesPerSource and maxListingsPerSource limit one cycle's size.
• scraping.requestTimeoutSeconds accepts values from 1 to 300 seconds.
• scraping.maxResponseSizeBytes limits one downloaded HTML response. The default is 10 MB and the maximum is 50 MB.
• scraping.detailsCacheMinutes controls cache lifetime. Set it to 0 to disable caching.
• scraping.detailsCacheMaxEntries limits the number of cached listing details. Set it to 0 to disable the cache.
• State is stored in data/state.json, with up to 2,000 delivered IDs per chat. Do not delete this file unless you want old listings to be sent again.

Proxies

To enable proxies for OLX/Otodom requests, set:

"proxyFile": "proxies.txt"

The path is resolved relative to the directory containing config.json. null disables proxies. Empty lines and lines starting with # are ignored. The file format is:

socks5://206.123.156.200:11011
http://135.87.39.23:443
socks4://147.45.60.241:1082

Proxies are tried in a round-robin order; each HTTP request makes up to three attempts. The Telegram API uses a direct connection.

For voivodeship capitals, the Otodom path is generated automatically. For another city, copy the part of the URL after /pl/wyniki/wynajem/mieszkanie/ into sources.otodomLocationPath, for example:

"otodomLocationPath": "mazowieckie/piaseczynski/piaseczno/piaseczno"

If a website changes its URL scheme, you can set sources.olxSearchUrl or sources.otodomSearchUrl directly. Custom URLs take precedence over generated ones.

Project structure

• source — independent OLX/Otodom adapters and parsers;
• service — filtering, caching, and notification cycles;
• storage — atomically persisted JSON state;
• telegram — commands, sending, and message formatting;
• config and model — Gson configuration and domain models.

OLX and Otodom may change their markup or enable anti-bot protection. The parsers use semantic attributes and JSON-LD, but a page change may require updating OlxParser or OtodomParser. Follow each website's rules and acceptable request frequency; the configuration includes a delay between requests for this purpose. If a website or Telegram cannot accept photos from the original URLs, the bot automatically sends a text-only notification.

License

FlatScraper is licensed under the PolyForm Noncommercial License 1.0.0. You may copy, modify, and distribute the project for permitted noncommercial purposes. Commercial use is not permitted under this license. See LICENSE for the complete terms.