Skip to content

[INCIDENT] middleware — memory_leak #22

Description

@vipulpadwal11

Summary

The middleware service is experiencing a high error rate and latency due to resource exhaustion caused by high CPU and memory usage, resulting in a high number of 504 Gateway Timeout errors. This issue is likely caused by a systemic problem with resource utilization, potentially related to a memory leak or optimization opportunity.

Root Cause

The probable cause of the incident is resource exhaustion due to high CPU and memory usage, with a high confidence level. The error logs indicate consistent 504 Gateway Timeout errors, and the metrics show high CPU and memory usage across multiple log entries, suggesting a systemic issue with resource utilization.

Recommended Action

  • Implement horizontal scaling to increase the number of instances and distribute the workload
  • Investigate potential memory leaks or optimization opportunities to address the root cause of the issue

Priority

P1

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions