Skip to content

KMS-703: Move metadata-correction auditing from RDF4J to Amazon DocumentDB Serverless - #135

Merged
cgokey merged 17 commits into
mainfrom
KMS-703
Sep 16, 2026
Merged

cgokey merged 17 commits into
mainfrom
KMS-703

Conversation

@cgokey

@cgokey cgokey commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Overview

What is the feature?

Move metadata-correction auditing from RDF4J to Amazon DocumentDB Serverless. Each collection
correction is stored as one audit document with its current status, history, correction details,
and CMR revision IDs. Existing RDF4J audit history will not be migrated.

What is the Solution?

  • Provision a shared DocumentDB Serverless database with credentials in Secrets Manager and
    deployment-managed indexes.
  • Update one audit document as a correction moves through checked, pending, and applied or
    failed states.
  • Provide searchable, paginated audit summary and detail responses in JSON or HTML.
  • Include a readable native metadata diff in the detailed audit view.
  • Add configurable CMR writeback timeout and optional request pacing to help prevent timeouts.

What areas of the application does this impact?

  • Metadata-correction auditing and CMR writeback
  • Audit API and browser views
  • CDK, Bamboo, and local development configuration

Testing

Reproduction steps

  1. Deploy the branch and publish a safe keyword update or run a manual collection correction.
  2. Open /metadata_correction_audit?format=html and confirm the audit log shows the collection ID,
    status, and expected old-to-new keyword path.
  3. Select the run ID and confirm the detail view shows the status history, prior and resulting CMR
    revision IDs, and a readable native metadata diff.
  4. Confirm the applied revision in CMR contains the expected metadata update.
  5. Verify basic audit filtering and perform basic metadata-correction regression testing.

Checklist

  • I have added automated tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings

@coderabbitai

coderabbitai Bot commented Sep 8, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 789b5598-37f6-4578-9e43-52e819bcafda


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov-commenter

codecov-commenter commented Sep 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.65%. Comparing base (bab8a4d) to head (55e6172).

Additional details and impacted files
@@           Coverage Diff            @@
##             main     #135    +/-   ##
========================================
  Coverage   99.64%   99.65%            
========================================
  Files         236      239     +3     
  Lines        6512     6918   +406     
  Branches     1909     2082   +173     
========================================
+ Hits         6489     6894   +405     
- Misses         21       22     +1     
  Partials        2        2            

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread cdk/app/lib/helper/MetadataCorrectionAuditDatabaseSetup.ts
Comment thread serverless/src/shared/buildNativeMetadataDiff.js
Comment thread serverless/certs/us-east-1-bundle.pem Outdated
Comment thread cdk/app/lib/helper/MetadataCorrectionAuditDatabaseSetup.ts Outdated
Comment thread cdk/app/lib/helper/MetadataCorrectionSetup.ts Outdated
*
* @returns {Promise<void>} Resolves after matching local audit documents are deleted.
*/
const clearAuditRowsForCollection = async () => {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we make this at a higher scope its getting remade in various of the local tests

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. I moved this into a shared local smoke-test helper and updated the callers.

await closeDocumentDbClient()
await getDocumentDbClient()

expect(close).toHaveBeenCalledOnce()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this one only gets called once since the conn didn't exist in the first call?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. The initial close is intentionally a no-op because no connection exists yet. The test then verifies that closing an existing connection resets it, allowing the next call to create a new client.

test('returns the expected ISO19115 payload shape when corrections are provided', async () => {
const mockPayload = `
<gmi:MI_Metadata xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gco="http://www.isotc211.org/2005/gco">
<gmi:MI_Metadata xmlns:gmi="http://www.isotc211.org/2005/gmi" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gco="http://www.isotc211.org/2005/gco">

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is for that error we'd seen on ISO where the longname was getting appended on a shortname update right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is separate from the ShortName/LongName issue. The fixture already used gmi:MI_Metadata but did not declare the gmi namespace, so the XML was invalid; the @xmldom/xmldom upgrade exposed it, and this adds the missing namespace declaration.

Comment thread scripts/local/run_platform_keyword_event_mapping_smoke.mjs
Comment thread serverless/src/shared/getMetadataCorrectionAuditLog.js
@cgokey
cgokey merged commit 2bcac33 into main Sep 16, 2026
7 checks passed
@cgokey
cgokey deleted the KMS-703 branch September 16, 2026 14:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants