Skip to content

[RORDEV-1229] Make docker-compose resource limits opt-in and host-size aware - #101

Merged
coutoPL merged 8 commits into
masterfrom
change/RORDEV-1229_resource_limits
Jul 30, 2026
Merged

[RORDEV-1229] Make docker-compose resource limits opt-in and host-size aware#101
coutoPL merged 8 commits into
masterfrom
change/RORDEV-1229_resource_limits

Conversation

@coutoPL

@coutoPL coutoPL commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Container memory limits were hardcoded in base.docker-compose.yml and apm.docker-compose.yml, so every environment got them whether it needed them or not. They are sized for small CI agents and are tight: es-ror capped at 2g against a 1g heap sits at 97–99% of its cgroup for a whole suite run, and one GC spike OOM-kills it. There is no restart policy, so the container stays dead and every remaining spec fails.

This moves mem_limit/memswap_limit out into base.limits.docker-compose.yml and apm.limits.docker-compose.yml, applied as compose overlays only when asked for.

APPLY_RESOURCE_LIMITS accepts true, false or auto:

  • false (default) — safe for the ROR KBN Docker-in-Docker flow, where a threaded /sys/fs/cgroup/docker cannot enable the memory controller and a mem_limit prevents containers from starting at all.
  • auto — applies the limits only when the host has less than 12 GB RAM, matching the threshold ci/e2e-tests-lib.sh uses in the ROR ES repo. Both workflows set this, so GitHub-hosted runners run unconstrained and small self-hosted agents still get the protection.
  • true — always apply.

The ES heap is now pinned explicitly (-Xms1g -Xmx1g) so dropping the cgroup limit does not let it grow unbounded.

Also raises the kbn-ror healthcheck start_period from 60s to 180s: two Kibana replicas boot alongside ES and APM, and on CPU-small agents a replica needs more than 60s to answer /api/status, which was causing docker compose up --wait to tear the stack down before the tests started.

No change for eck environments — they do not use docker-compose overlays.

@coderabbitai

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as resolved.

@coutoPL
coutoPL force-pushed the change/RORDEV-1229_resource_limits branch from 7411198 to c2a4e09 Compare July 29, 2026 15:35
@coutoPL coutoPL changed the title [RORDEV-1229] resource limits [RORDEV-1229] Make docker-compose resource limits opt-in and host-size aware Jul 29, 2026
@coutoPL
coutoPL requested a review from Dzuming July 29, 2026 16:28

@Dzuming Dzuming left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@coutoPL
coutoPL merged commit e9058d3 into master Jul 30, 2026
83 checks passed
@coutoPL
coutoPL deleted the change/RORDEV-1229_resource_limits branch July 30, 2026 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants