Implement monitoring

[stevesajeev1]

Description

Add monitoring to the backend. Use Alloy + Loki to ingest logs, and use Grafana as a dashboard. Grafana Alerts + Discord Webhooks are then used for notifications.

Every five minutes, error logs are checked for in the core-api service. These errors are then aggregated and sent via a Discord webhook (currently using default template, but can be modified if necessary).

To use our existing auth flow with Grafana, we use our API as an auth proxy. Going to routes matching /grafana/* checks that the user has the admin role, then proxies to Grafana.

I don't think this should be documented in our OpenAPI schema, since this route should only be accessed via separate subdomain (ie grafana.swamphacks.com?)

I made a new folder monitoring/ which stores the configuration files. Inside monitoring/ there is a .env file with the secret DISCORD_WEBHOOK= which specifies the Discord Webhook URL. I can reorganize this if necessary.

I don't know if this is the ideal structure, since most of the config will be shared with production. We'd have to reference it using ../monitoring

This PR is marked as draft since production deployment still needs to be configured, but this should work for local development (can test auth flow going to http://localhost:8080/grafana).

TODO:

• Create route in reverse proxy to access Grafana dashboard (grafana.swamphacks.com?)
• Implement authentication in Grafana (ensure only admins have access)
  • There are many ways of configuring authentication for Grafana. From what I understand, we'll probably have to use Auth Proxy if we want to use our existing auth flow.

Linked GitHub Issue

#317

Type of Change

• New feature (non-breaking change)

Checklist

• My code follows the project's style guidelines
• I have commented on complex parts of the code
• I have updated documentation if necessary
• I have updated or added tests to cover my changes
• I have updated the OpenAPI YAML or other API schema files if applicable

Additional Notes

To allow for more robust parsing of logs, I changed the output of zerolog to print structured JSON instead of using console writer.