Architecture

Architecture

Overview Diagram

Functional Architecture

Proposed Solution Architecture

Components

Scrapers

• Scrape information from Genossenschafts websites.
• Implemented as series of Azure Functions.
• One function dedicated to each of the 10 predefined providers.
• Triggered periodically, e.g. every 5 minutes, by Azure Function Timer Triggers.

CosmosDB

• Primary DB for storing scraped GeWo data and user (notification) preferences.
• Usage of change feed to trigger notifications when new GeWos are added.

Notification Generation Function

• An Azure Function that is triggered by CosmosDB's change feed.
• Checks user preferences and queries users that are interested in the new GeWo.
• Generate notifications for matching new listings. (could be put in another function)

Azure Communication Service

• Sends email notifications to users about new GeWos that match their preferences.
• Triggered by the Notification Generation Function.
• Alternative/Extension: Use Azure Notification Service for mobile notifications.

Frontend (Azure Static Web App)

• Host of the JS Web App that users interact with.
• Allows users to
  • Browse flats
  • Manage notification settings
• Technology Stack: Vue, PrimeVue, Typescript

Azure Static Web App Backend

• Set of Azure Functions that form the backend for the frontend application.
• Handling of business logic, e.g. querying/filtering/sorting of flat data, processing user preferences, etc.

Data flow

1. Scrapers to DB

• Scraper functions extract data from their respective GeWo websites periodically and insert/update the listings in CosmosDB.

2. Database to Notification Generation

• CosmosDB triggers the Notification Generation Function based on new/updated listings.

3. Notification Service

• Usage of Azure Communication Service to send out emails to interested users.

4. Frontend to Backend Interaction

• Frontend Static Web App communicates with the backend through integrated Azure Functions.

5. User Auth Flow

• Azure Active Directory will be used for authenticating users and providing password security to our customers.

Cost estimation

Assumptions

• One Genossenschaft releases 10 listings per month on average; We support 10 Genossenschaften --> approx. 100 new listings per month
• We expect 1000 monthly active users, each will be interested in ~10% of listings --> 100 notifications per new listing on average

Cost by Service

To estimate the cost of this architecture, we used the Azure Pricing Calculator and made a few assumptions, listed below:

• Azure Cosmos DB for PostgreSQL: €27.64 monthly
  • Region: West Europe
  • Tier: Single Node
  • Computing Power 1: vCore, 2 GiB RAM
  • Storage: 128 GiB
• Azure Functions: €0.00
  • Region: West Europe
  • Tier: Consumption
  • Memory size: 128 MiB
  • Assumed avg. execution Time: 5000ms
  • Executions per month: 400,000
    • 87,600: 10 scrapers every 5 minutes
    • 322,300: ~32 calls per user
    • 100: ~100 users notified per invocation
• Azure API Management: €0.00
  • Free until 1M API Calls
• Azure Static Web Apps: €0.00
• Azure Communication Services: €2.30
  • Number of emails per month: 10,000
• Azure Active Directory External Identities: €0.00
  • Region: West Europe
  • Assumed monthly active users: 1,000
  • Free until 50,000 MAUs

Based on assumptions regarding the number of users, calls to the API, scraping runs per month, etc., the monthly cost sums up to around €29.95.

The assumed number of calls to the Azure Functions is based on the number of scrapers and our plan to run each scraper every 5 minutes and the number of calls to the API from the website every month. For the database we assumed around 500 hours of it being active every month, since it won't be running 24/7, but development will be very active and require a lot of uptime of the database.