[MISC] Improve dev experience by adding a compose debug override #1765
Conversation
WalkthroughAdds a debug-focused Docker Compose override and a revamped development compose: introduces compose.debug.yaml with debugpy-enabled services and per-service debug ports, expands sample compose with new services, multiple V2 workers, and updated dev runtime/watch configurations; updates README debugging instructions. Changes
Sequence Diagram(s)mermaid Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes 🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
Tip 🧪 Unit Test Generation v2 is now available!We have significantly improved our unit test generation capabilities. To enable: Add this to your reviews:
finishing_touches:
unit_tests:
enabled: trueTry it out by using the Have feedback? Share your thoughts on our Discord thread! Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
chandrasekharan-zipstack
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@docker/compose.debug.yaml`:
- Around line 52-67: Remove the duplicate port mapping "3001:3001" from the
platform-service ports list in the docker compose debug file so only "5680:5680"
remains here (the "3001:3001" binding is already declared in the other Compose
override), i.e., edit the platform-service ports section to delete the
"3001:3001" entry and leave the "5680:5680" mapping to avoid confusion and keep
port definitions consolidated.
In `@docker/sample.compose.override.yaml`:
- Around line 42-79: The backend service's hot-reload isn't enabled: update the
backend service command (the entrypoint/command that runs gunicorn in the
backend service) to include the --reload gunicorn flag so code synced by the
develop.watch sync actions takes effect (i.e., add "--reload" to the existing
gunicorn options in the command string); alternatively, if you prefer
restart-based reloads, change the develop.watch action entries under the backend
service from "action: sync" to "action: sync+restart" so the container restarts
after file syncs.
652a715 to
f5020c6
Compare
chandrasekharan-zipstack
requested review from
hari-kuriakose,
muhammad-ali-e and
ritwik-g
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Fix all issues with AI agents
In `@docker/README.md`:
- Around line 115-124: The README debug ports table is out of sync with
compose.debug.yaml: update the table in docker/README.md to exactly match the
port assignments defined in compose.debug.yaml (ensure backend remains 5678,
change platform-service to 5680, prompt-service to 5681, runner to 5679), remove
or mark x2text-service as not configured if compose.debug.yaml has no debug
entry for it, and add the V2 worker debug port range (5682–5688) reflecting the
compose.debug.yaml entries; verify all service names match the keys in
compose.debug.yaml and update any captions or notes to reflect these corrected
ports.
In `@docker/sample.compose.override.yaml`:
- Around line 301-304: The service worker-log-history-scheduler-v2 is missing
the standard develop.watch configuration used by other V2 workers; update the
service block for worker-log-history-scheduler-v2 to include the same
develop.watch settings (e.g., bind mounts, polling/ignore patterns, commands
used by other V2 workers) so it matches the other V2 workers, or if omission is
intentional add a short inline comment in the worker-log-history-scheduler-v2
service explaining why live development/watch is not needed.
- Around line 23-24: Add a note documenting the Docker Compose version
requirement for the !override directive (used where !override appears for
services like reverse-proxy) by adding a brief header comment at the top of this
compose override file and/or updating the main README to state "Docker Compose
2.24.4+ required" (or equivalent), so developers know that the !override syntax
requires Compose 2.24.4 or newer.
| Debug ports per service: | ||
|
|
||
| | Service | Port | | ||
| |---------|------| | ||
| | backend | 5678 | | ||
| | platform-service | 5679 | | ||
| | prompt-service | 5680 | | ||
| | runner | 5681 | | ||
| | x2text-service | 5682 | | ||
|
|
There was a problem hiding this comment.
Critical: Debug port table doesn't match compose.debug.yaml.
The port assignments here contradict the actual configuration in compose.debug.yaml:
| Service | README says | compose.debug.yaml says |
|---|---|---|
| backend | 5678 | 5678 ✓ |
| platform-service | 5679 | 5680 |
| prompt-service | 5680 | 5681 |
| runner | 5681 | 5679 |
| x2text-service | 5682 | (not defined) |
Additionally, x2text-service has no debug configuration in compose.debug.yaml, but is listed here. The V2 workers (ports 5682-5688) are also missing from this table.
📝 Suggested fix to align with compose.debug.yaml
Debug ports per service:
| Service | Port |
|---------|------|
| backend | 5678 |
-| platform-service | 5679 |
-| prompt-service | 5680 |
-| runner | 5681 |
-| x2text-service | 5682 |
+| runner | 5679 |
+| platform-service | 5680 |
+| prompt-service | 5681 |
+| worker-file-processing-v2 | 5682 |
+| worker-callback-v2 | 5683 |
+| worker-api-deployment-v2 | 5684 |
+| worker-general-v2 | 5685 |
+| worker-notification-v2 | 5686 |
+| worker-log-consumer-v2 | 5687 |
+| worker-scheduler-v2 | 5688 |🧰 Tools
🪛 markdownlint-cli2 (0.20.0)
118-118: Table column style
Table pipe is missing space to the right for style "compact"
(MD060, table-column-style)
118-118: Table column style
Table pipe is missing space to the left for style "compact"
(MD060, table-column-style)
118-118: Table column style
Table pipe is missing space to the right for style "compact"
(MD060, table-column-style)
118-118: Table column style
Table pipe is missing space to the left for style "compact"
(MD060, table-column-style)
🤖 Prompt for AI Agents
In `@docker/README.md` around lines 115 - 124, The README debug ports table is out
of sync with compose.debug.yaml: update the table in docker/README.md to exactly
match the port assignments defined in compose.debug.yaml (ensure backend remains
5678, change platform-service to 5680, prompt-service to 5681, runner to 5679),
remove or mark x2text-service as not configured if compose.debug.yaml has no
debug entry for it, and add the V2 worker debug port range (5682–5688)
reflecting the compose.debug.yaml entries; verify all service names match the
keys in compose.debug.yaml and update any captions or notes to reflect these
corrected ports.
| depends_on: !override | ||
| - reverse-proxy |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, let's examine the actual file to see the context and any existing documentation
fd -t f "sample.compose.override.yaml" --exec cat -n {} \;Repository: Zipstack/unstract
Length of output: 13696
🏁 Script executed:
# Check if there's a README that documents Docker Compose version requirements
fd -t f "README*" --exec head -50 {} \;Repository: Zipstack/unstract
Length of output: 45972
🌐 Web query:
Docker Compose !override directive version requirement documentation
💡 Result:
The !override YAML tag (used to fully replace a value during multi-file Compose merges, instead of applying the normal merge rules) requires Docker Compose v2.24.4 or later (the docker compose v2 plugin). [1]
This is a Compose implementation (CLI/plugin) requirement, not a version: field requirement in your compose.yaml. The top-level version property is obsolete / informational only, and Compose validates using the most recent schema regardless of version:. [2]
Docs:
- Docker Docs (merge reference, includes the requirement): [1]
- Compose Specification (describes
!overridemerge behavior): [3]
Sources: [1] [2] [3]
Add Docker Compose version requirement note to file header and/or README.
The !override directive used at lines 23-24 and 49-54 requires Docker Compose 2.24.4+. Document this version requirement in the file header comment or main README to prevent confusion for developers using older versions.
🤖 Prompt for AI Agents
In `@docker/sample.compose.override.yaml` around lines 23 - 24, Add a note
documenting the Docker Compose version requirement for the !override directive
(used where !override appears for services like reverse-proxy) by adding a brief
header comment at the top of this compose override file and/or updating the main
README to state "Docker Compose 2.24.4+ required" (or equivalent), so developers
know that the !override syntax requires Compose 2.24.4 or newer.
| worker-log-history-scheduler-v2: | ||
| build: | ||
| dockerfile: docker/dockerfiles/worker-unified.Dockerfile | ||
| context: .. |
There was a problem hiding this comment.
worker-log-history-scheduler-v2 missing develop.watch configuration.
Unlike all other V2 workers, this service has no watch configuration. If this is intentional (e.g., this worker doesn't need live development), consider adding a comment explaining why. Otherwise, add the standard watch block for consistency.
🔧 Add watch config for consistency
worker-log-history-scheduler-v2:
build:
dockerfile: docker/dockerfiles/worker-unified.Dockerfile
context: ..
+ develop:
+ watch:
+ - action: sync+restart
+ path: ../workers/
+ target: /app
+ ignore: [.venv/, __pycache__/, "*.pyc", .pytest_cache/, .mypy_cache/]
+ - action: sync+restart
+ path: ../unstract/
+ target: /unstract
+ ignore: [.venv/, __pycache__/, "*.pyc", .pytest_cache/, .mypy_cache/]
+ - action: rebuild
+ path: ../workers/uv.lock📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| worker-log-history-scheduler-v2: | |
| build: | |
| dockerfile: docker/dockerfiles/worker-unified.Dockerfile | |
| context: .. | |
| worker-log-history-scheduler-v2: | |
| build: | |
| dockerfile: docker/dockerfiles/worker-unified.Dockerfile | |
| context: .. | |
| develop: | |
| watch: | |
| - action: sync+restart | |
| path: ../workers/ | |
| target: /app | |
| ignore: [.venv/, __pycache__/, "*.pyc", .pytest_cache/, .mypy_cache/] | |
| - action: sync+restart | |
| path: ../unstract/ | |
| target: /unstract | |
| ignore: [.venv/, __pycache__/, "*.pyc", .pytest_cache/, .mypy_cache/] | |
| - action: rebuild | |
| path: ../workers/uv.lock |
🤖 Prompt for AI Agents
In `@docker/sample.compose.override.yaml` around lines 301 - 304, The service
worker-log-history-scheduler-v2 is missing the standard develop.watch
configuration used by other V2 workers; update the service block for
worker-log-history-scheduler-v2 to include the same develop.watch settings
(e.g., bind mounts, polling/ignore patterns, commands used by other V2 workers)
so it matches the other V2 workers, or if omission is intentional add a short
inline comment in the worker-log-history-scheduler-v2 service explaining why
live development/watch is not needed.
…ry, add V2 workers support - Move debugpy to optional compose.debug.yaml for cleaner default dev setup - Update compose.override.yaml with memory-optimized settings (1 worker, 2 threads) - Add V2 workers configuration with build definitions - Move V1 workers to optional workers-v1 profile - Use modern uv run python -Xfrozen_modules=off pattern for services - Updated README with compose.debug.yaml usage instructions Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
chandrasekharan-zipstack
force-pushed
the
misc/docker-dev-experience-improvements
branch
from
f5020c6 to
74c9991
Compare
|
chandrasekharan-zipstack
changed the title
chandrasekharan-zipstack
changed the title
| - workers-v1 | ||
|
|
||
| worker-file-processing: | ||
| command: "-A backend.workers.file_processing worker --loglevel=info -Q file_processing,api_file_processing --autoscale=${WORKER_FILE_PROCESSING_AUTOSCALE}" |
There was a problem hiding this comment.
I don't think it in backend.workers ? is it in workers directory?
What
compose.debug.yamlfor debugpy supportsample.compose.override.yamlwith memory-optimized settings (1 worker, 2 threads)workers-v1profileWhy
How
compose.debug.yamlwith debugpy configuration for all Python services and V2 workers (ports 5678-5688)sample.compose.override.yamlwith new version usinguv run python -Xfrozen_modules=offpatternCan this PR break any existing features. If yes, please list possible items. If no, please explain why.
No breaking changes. Changes are additive and optional:
compose.debug.yamlsample.compose.override.yamlis backward compatibleworkers-v1profileworkers-v2profileDatabase Migrations
None
Env Config
None
Relevant Docs
/docker/README.md/docker/docker-compose.yamlRelated Issues or PRs
None
Dependencies Versions
None
Notes on Testing
Tested locally:
VERSION=test docker compose -f docker-compose.yaml -f compose.override.yaml watch- worksVERSION=test docker compose -f docker-compose.yaml -f compose.override.yaml -f compose.debug.yaml watch- debugpy ports accessibleScreenshots
N/A
Checklist