Skip to content

Jerome ng/2072 cdm creation fails in manual test#2736

Open
jerome-ng wants to merge 10 commits into
developfrom
jerome-ng/2072-cdm-creation-fails-in-manual-test
Open

Jerome ng/2072 cdm creation fails in manual test#2736
jerome-ng wants to merge 10 commits into
developfrom
jerome-ng/2072-cdm-creation-fails-in-manual-test

Conversation

@jerome-ng

@jerome-ng jerome-ng commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fixes several CDM config creation/editing failures found during manual testing. The changes focus on making CDW/Trex metadata calls safer, cleaning up opened connections, and avoiding cold-start attach races during config suggestions.

What Changed

  • Close analytics/Trex connections after CDM config metadata operations such as validation, autosave, activate, getColumns, and CDW suggestion services.
  • Avoid opening an analytics connection for attributeType_service, which only needs config metadata.
  • Update blank config defaults to use CDW service credentials instead of the old analytics service-tag lookup.
  • Preserve suggestion endpoint response shape by returning { "data": [] } for column/table suggestion failures.
  • Add a per-destination attach guard for Trex connections so the first cold getConnection() is serialized, then later requests use the normal fast path.

Why

Manual testing exposed multiple related failure modes:

  • Repeated metadata calls could leave Trex connections open until Trex became unstable.
  • configDefaults could fail with Found 0 matching services when looking up the old analytics tag.
  • Some UI flows passed dataset/config context differently, especially after changing the dataset selector.
  • Cold parallel suggestion requests could race while attaching DuckDB validation schemas and fail with validation_schema already exists.

Validation

  • Reproduced and diagnosed the CDW suggestion attach race.
  • Reproduced the unclosed-connection failure with cleanup reverted: sequential requests timed out around request 815.
  • Verified the cleanup fix with 5000 column_suggestion_service requests at concurrency 10 plus a successful follow-up request.
  • Compared no-lock, always-mutex, and fast-path guard approaches. The fast-path guard completed 5000 / 5000 requests and stayed effectively at baseline latency, while the always-mutex version was materially slower.
  • Confirmed no validation_schema already exists, Failed to attach database, UncaughtException, event loop error, or request timeout errors during the final benchmark run.
  • Successfully completed the CDM creation manual test locally.


const sendCDWServiceError = (res, action) => {
if (
action === "column_suggestion_service" ||

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preserving existing behaviour

isTestEnvironment,
);

if (req.query.action === "attributeType_service") {

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Early return as this action does not need analytics service

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses multiple CDM config creation/editing failures found during manual testing by improving connection lifecycle management around Trex/analytics metadata operations, hardening blank-config defaults, and stabilizing CDW suggestion behavior under cold-start concurrency.

Changes:

  • Close analytics/Trex connections after metadata-oriented operations (e.g., validation/autosave/activate/getColumns and CDW service calls).
  • Update blank config defaults to derive credentials from the CDW-tagged service in VCAP_SERVICES (instead of the old analytics tag lookup).
  • Preserve suggestion endpoint response shape by returning { data: [] } on column/table suggestion failures, and add a per-destination attach guard for Trex connections.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
plugins/functions/cdw-svc/src/utils/trexConnection.ts Adds a per-destination locking/ready mechanism intended to serialize cold-start Trex attach/setup.
plugins/functions/cdw-svc/src/qe/settings/SettingsFacade.ts Ensures analytics connection cleanup after getColumns metadata calls.
plugins/functions/cdw-svc/src/qe/config/configSuggestion.ts Introduces getCdwServiceCredentials() and uses it to seed blank config defaults.
plugins/functions/cdw-svc/src/qe/config/ConfigFacade.ts Centralizes analytics connection cleanup for several actions that open analytics connections.
plugins/functions/cdw-svc/src/main.ts Closes analytics connections after CDW service requests; returns { data: [] } for suggestion-service failures; avoids analytics connection for attributeType_service.
plugins/functions/cdw-svc/spec/qe/config/suggestion_test.ts Adds unit coverage for selecting the first CDW-tagged service when multiple exist.

Comment thread plugins/functions/cdw-svc/src/utils/trexConnection.ts Outdated
Comment thread plugins/functions/cdw-svc/src/utils/trexConnection.ts Outdated
Comment thread plugins/functions/cdw-svc/src/utils/trexConnection.ts Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CDM Creation fails in manual test

2 participants