Write permissions for sagemaker execution on 2605 coralnet bucket by gnieuwenhuis · Pull Request #692 · data-mermaid/mermaid-api

gnieuwenhuis · 2026-05-26T16:01:47Z

In order to write new feature vectors to the 2605-coralnet-public-sources bucket using processing jobs, we need to give write permission.

Summary by CodeRabbit

Infrastructure
- Updated SageMaker permissions to enable seamless data flow between feature extraction and model training stages for shared feature vector processing.

Signed-off-by: Greg Nieuwenhuis <26285069+gnieuwenhuis@users.noreply.github.com>

coderabbitai · 2026-05-26T16:02:06Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 0c6af77b-1378-4dd6-bc62-3e2bd2dc82f1

📥 Commits

Reviewing files that changed from the base of the PR and between c4aaf50 and c94e40e.

📒 Files selected for processing (1)

iac/stacks/sagemaker.py

📝 Walkthrough

Walkthrough

The PR modifies the SageMaker stack infrastructure code to grant the SageMaker execution role read/write access to the coralnet_feature_vectors S3 bucket. Previously, this role had read-only access. The inline comment is updated to document the bucket's dual role: storing feature-extraction outputs and serving as the source for training pipeline reads.

Changes

SageMaker S3 Feature Vector Permissions

Layer / File(s)	Summary
Feature vector bucket write access `iac/stacks/sagemaker.py`	SageMaker execution role permission on `coralnet_feature_vectors` S3 bucket (ARN `arn:aws:s3:::2605-coralnet-public-sources`) is changed from read-only to read/write. The comment is updated to describe the bucket's role as shared storage between feature-extraction (writes `.fv` and `annotations.csv`) and training (reads).

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

data-mermaid/mermaid-api#607: Both PRs modify iac/stacks/sagemaker.py to adjust IAM permission grants on S3/SageMaker-related resources (main PR changes coralnet_feature_vectors from read-only to read/write; retrieved PR adds/extends SageMaker execution-role policies/permissions including MLflow).

Suggested labels

enhancement

Suggested reviewers

gridcell
saanobhaai

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The pull request title accurately describes the main change: adding write permissions for SageMaker execution on the 2605 coralnet bucket.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch sagemaker_bucket_write

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions · 2026-05-26T16:03:11Z

cdk-nag report

✅ No unsuppressed errors.

See iac/nag_suppressions.py to add suppressions for accepted risks.

github-actions · 2026-05-26T16:04:25Z

`cdk diff` ✅ Success

Show Output

start: Building GithubAccess Template
success: Built GithubAccess Template
start: Publishing GithubAccess Template (554812291621-us-east-1-403885e2)
success: Published GithubAccess Template (554812291621-us-east-1-403885e2)
Hold on while we create a read-only change set to get a diff with accurate replacement information (use --no-change-set to use a less accurate but faster template-only diff)

Stack GithubAccess
There were no differences

start: Building mermaid-api-infra-common Template
success: Built mermaid-api-infra-common Template
start: Publishing mermaid-api-infra-common Template (554812291621-us-east-1-37993ba8)
success: Published mermaid-api-infra-common Template (554812291621-us-east-1-37993ba8)
Hold on while we create a read-only change set to get a diff with accurate replacement information (use --no-change-set to use a less accurate but faster template-only diff)

Stack mermaid-api-infra-common
There were no differences

start: Building dev-mermaid-static-site Template
success: Built dev-mermaid-static-site Template
start: Publishing dev-mermaid-static-site Template (554812291621-us-east-1-705b66ee)
success: Published dev-mermaid-static-site Template (554812291621-us-east-1-705b66ee)
Hold on while we create a read-only change set to get a diff with accurate replacement information (use --no-change-set to use a less accurate but faster template-only diff)

Stack dev-mermaid-static-site
There were no differences

start: Building dev-mermaid-api-django Template
success: Built dev-mermaid-api-django Template
start: Publishing dev-mermaid-api-django Template (554812291621-us-east-1-488ebee7)
success: Published dev-mermaid-api-django Template (554812291621-us-east-1-488ebee7)
Hold on while we create a read-only change set to get a diff with accurate replacement information (use --no-change-set to use a less accurate but faster template-only diff)

Stack dev-mermaid-api-django
Resources
[~] AWS::ECS::TaskDefinition ScheduledBackupTaskDef ScheduledBackupTaskDef48789D5A replace
 └─ [~] ContainerDefinitions (requires replacement)
     └─ @@ -147,7 +147,7 @@
        [ ] ],
        [ ] "Essential": true,
        [ ] "Image": {
        [-]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:d89f2553dce37326866a51cb2aa60c81ffe7008a9df01edef3f1654a2cd9534a"
        [+]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:75fd2bb0ad5c2c6545380e54a84afdaccfd106693931b09f6c0feda693ffc3a3"
        [ ] },
        [ ] "LogConfiguration": {
        [ ]   "LogDriver": "awslogs",
[~] AWS::ECS::TaskDefinition SummaryCacheTaskDef SummaryCacheTaskDefFAAC683D replace
 └─ [~] ContainerDefinitions (requires replacement)
     └─ @@ -151,7 +151,7 @@
        [ ] ],
        [ ] "Essential": true,
        [ ] "Image": {
        [-]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:d89f2553dce37326866a51cb2aa60c81ffe7008a9df01edef3f1654a2cd9534a"
        [+]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:75fd2bb0ad5c2c6545380e54a84afdaccfd106693931b09f6c0feda693ffc3a3"
        [ ] },
        [ ] "LogConfiguration": {
        [ ]   "LogDriver": "awslogs",
[~] AWS::ECS::TaskDefinition ApiTaskDefinition ApiTaskDefinition51EA709E replace
 └─ [~] ContainerDefinitions (requires replacement)
     └─ @@ -145,7 +145,7 @@
        [ ] ],
        [ ] "Essential": true,
        [ ] "Image": {
        [-]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:d89f2553dce37326866a51cb2aa60c81ffe7008a9df01edef3f1654a2cd9534a"
        [+]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:75fd2bb0ad5c2c6545380e54a84afdaccfd106693931b09f6c0feda693ffc3a3"
        [ ] },
        [ ] "LogConfiguration": {
        [ ]   "LogDriver": "awslogs",
[~] AWS::ECS::TaskDefinition General/Worker/QueueProcessingTaskDef GeneralWorkerQueueProcessingTaskDef1C2A1522 replace
 └─ [~] ContainerDefinitions (requires replacement)
     └─ @@ -158,7 +158,7 @@
        [ ] ],
        [ ] "Essential": true,
        [ ] "Image": {
        [-]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:d89f2553dce37326866a51cb2aa60c81ffe7008a9df01edef3f1654a2cd9534a"
        [+]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:75fd2bb0ad5c2c6545380e54a84afdaccfd106693931b09f6c0feda693ffc3a3"
        [ ] },
        [ ] "LogConfiguration": {
        [ ]   "LogDriver": "awslogs",
[~] AWS::ECS::TaskDefinition ImageProcess/Worker/QueueProcessingTaskDef ImageProcessWorkerQueueProcessingTaskDefACA5B138 replace
 └─ [~] ContainerDefinitions (requires replacement)
     └─ @@ -158,7 +158,7 @@
        [ ] ],
        [ ] "Essential": true,
        [ ] "Image": {
        [-]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:d89f2553dce37326866a51cb2aa60c81ffe7008a9df01edef3f1654a2cd9534a"
        [+]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:75fd2bb0ad5c2c6545380e54a84afdaccfd106693931b09f6c0feda693ffc3a3"
        [ ] },
        [ ] "LogConfiguration": {
        [ ]   "LogDriver": "awslogs",


start: Building dev-mermaid-sagemaker Template
success: Built dev-mermaid-sagemaker Template
start: Publishing dev-mermaid-sagemaker Template (554812291621-us-east-1-32848cce)
success: Published dev-mermaid-sagemaker Template (554812291621-us-east-1-32848cce)
Hold on while we create a read-only change set to get a diff with accurate replacement information (use --no-change-set to use a less accurate but faster template-only diff)

Stack dev-mermaid-sagemaker
IAM Statement Changes
┌───┬─────────────────────────────────────────────┬────────┬────────────────────────────┬──────────────────────────────────┬───────────┐
│   │ Resource                                    │ Effect │ Action                     │ Principal                        │ Condition │
├───┼─────────────────────────────────────────────┼────────┼────────────────────────────┼──────────────────────────────────┼───────────┤
│ + │ arn:aws:s3:::2605-coralnet-public-sources   │ Allow  │ s3:Abort*                  │ AWS:${devSagemakerExecutionRole} │           │
│   │ arn:aws:s3:::2605-coralnet-public-sources/* │        │ s3:DeleteObject*           │                                  │           │
│   │                                             │        │ s3:GetBucket*              │                                  │           │
│   │                                             │        │ s3:GetObject*              │                                  │           │
│   │                                             │        │ s3:List*                   │                                  │           │
│   │                                             │        │ s3:PutObject               │                                  │           │
│   │                                             │        │ s3:PutObjectLegalHold      │                                  │           │
│   │                                             │        │ s3:PutObjectRetention      │                                  │           │
│   │                                             │        │ s3:PutObjectTagging        │                                  │           │
│   │                                             │        │ s3:PutObjectVersionTagging │                                  │           │
│ - │ arn:aws:s3:::2605-coralnet-public-sources   │ Allow  │ s3:GetBucket*              │ AWS:${devSagemakerExecutionRole} │           │
│   │ arn:aws:s3:::2605-coralnet-public-sources/* │        │ s3:GetObject*              │                                  │           │
│   │                                             │        │ s3:List*                   │                                  │           │
└───┴─────────────────────────────────────────────┴────────┴────────────────────────────┴──────────────────────────────────┴───────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)

Resources
[~] AWS::IAM::Policy devSagemakerExecutionRole/DefaultPolicy devSagemakerExecutionRoleDefaultPolicyBA02F2A0
 └─ [~] PolicyDocument
     └─ [~] .Statement:
         └─ @@ -120,7 +120,14 @@
            [ ] "Action": [
            [ ]   "s3:GetObject*",
            [ ]   "s3:GetBucket*",
            [-]   "s3:List*"
            [+]   "s3:List*",
            [+]   "s3:DeleteObject*",
            [+]   "s3:PutObject",
            [+]   "s3:PutObjectLegalHold",
            [+]   "s3:PutObjectRetention",
            [+]   "s3:PutObjectTagging",
            [+]   "s3:PutObjectVersionTagging",
            [+]   "s3:Abort*"
            [ ] ],
            [ ] "Effect": "Allow",
            [ ] "Resource": [


start: Building prod-mermaid-static-site Template
success: Built prod-mermaid-static-site Template
start: Publishing prod-mermaid-static-site Template (554812291621-us-east-1-b2d960b0)
success: Published prod-mermaid-static-site Template (554812291621-us-east-1-b2d960b0)
Hold on while we create a read-only change set to get a diff with accurate replacement information (use --no-change-set to use a less accurate but faster template-only diff)

Stack prod-mermaid-static-site
There were no differences

start: Building prod-mermaid-api-django Template
success: Built prod-mermaid-api-django Template
start: Publishing prod-mermaid-api-django Template (554812291621-us-east-1-b7f76b74)
success: Published prod-mermaid-api-django Template (554812291621-us-east-1-b7f76b74)
Hold on while we create a read-only change set to get a diff with accurate replacement information (use --no-change-set to use a less accurate but faster template-only diff)

Stack prod-mermaid-api-django
Resources
[~] AWS::ECS::TaskDefinition ScheduledBackupTaskDef ScheduledBackupTaskDef48789D5A replace
 └─ [~] ContainerDefinitions (requires replacement)
     └─ @@ -147,7 +147,7 @@
        [ ] ],
        [ ] "Essential": true,
        [ ] "Image": {
        [-]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:38b11ecd7a34356ccac91b8c61f892e5a4b598bfeed13f480a5dfde2538bcce5"
        [+]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:75fd2bb0ad5c2c6545380e54a84afdaccfd106693931b09f6c0feda693ffc3a3"
        [ ] },
        [ ] "LogConfiguration": {
        [ ]   "LogDriver": "awslogs",
[~] AWS::ECS::TaskDefinition SummaryCacheTaskDef SummaryCacheTaskDefFAAC683D replace
 └─ [~] ContainerDefinitions (requires replacement)
     └─ @@ -151,7 +151,7 @@
        [ ] ],
        [ ] "Essential": true,
        [ ] "Image": {
        [-]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:38b11ecd7a34356ccac91b8c61f892e5a4b598bfeed13f480a5dfde2538bcce5"
        [+]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:75fd2bb0ad5c2c6545380e54a84afdaccfd106693931b09f6c0feda693ffc3a3"
        [ ] },
        [ ] "LogConfiguration": {
        [ ]   "LogDriver": "awslogs",
[~] AWS::ECS::TaskDefinition ApiTaskDefinition ApiTaskDefinition51EA709E replace
 └─ [~] ContainerDefinitions (requires replacement)
     └─ @@ -145,7 +145,7 @@
        [ ] ],
        [ ] "Essential": true,
        [ ] "Image": {
        [-]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:38b11ecd7a34356ccac91b8c61f892e5a4b598bfeed13f480a5dfde2538bcce5"
        [+]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:75fd2bb0ad5c2c6545380e54a84afdaccfd106693931b09f6c0feda693ffc3a3"
        [ ] },
        [ ] "LogConfiguration": {
        [ ]   "LogDriver": "awslogs",
[~] AWS::ECS::TaskDefinition General/Worker/QueueProcessingTaskDef GeneralWorkerQueueProcessingTaskDef1C2A1522 replace
 └─ [~] ContainerDefinitions (requires replacement)
     └─ @@ -158,7 +158,7 @@
        [ ] ],
        [ ] "Essential": true,
        [ ] "Image": {
        [-]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:38b11ecd7a34356ccac91b8c61f892e5a4b598bfeed13f480a5dfde2538bcce5"
        [+]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:75fd2bb0ad5c2c6545380e54a84afdaccfd106693931b09f6c0feda693ffc3a3"
        [ ] },
        [ ] "LogConfiguration": {
        [ ]   "LogDriver": "awslogs",
[~] AWS::ECS::TaskDefinition ImageProcess/Worker/QueueProcessingTaskDef ImageProcessWorkerQueueProcessingTaskDefACA5B138 replace
 └─ [~] ContainerDefinitions (requires replacement)
     └─ @@ -158,7 +158,7 @@
        [ ] ],
        [ ] "Essential": true,
        [ ] "Image": {
        [-]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:38b11ecd7a34356ccac91b8c61f892e5a4b598bfeed13f480a5dfde2538bcce5"
        [+]   "Fn::Sub": "554812291621.dkr.ecr.us-east-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-554812291621-us-east-1:75fd2bb0ad5c2c6545380e54a84afdaccfd106693931b09f6c0feda693ffc3a3"
        [ ] },
        [ ] "LogConfiguration": {
        [ ]   "LogDriver": "awslogs",



✨  Number of stacks with differences: 3

Workflow: pr

Write permissions for sagemaker execution on 2605 coralnet bucket

c94e40e

Signed-off-by: Greg Nieuwenhuis <26285069+gnieuwenhuis@users.noreply.github.com>

gnieuwenhuis requested a review from ms280690 May 26, 2026 16:01

Merge branch 'dev' into sagemaker_bucket_write

db907d6

ms280690 approved these changes May 27, 2026

View reviewed changes

gnieuwenhuis merged commit b547a02 into dev May 28, 2026
4 checks passed

gnieuwenhuis deleted the sagemaker_bucket_write branch May 28, 2026 23:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Write permissions for sagemaker execution on 2605 coralnet bucket#692

Write permissions for sagemaker execution on 2605 coralnet bucket#692
gnieuwenhuis merged 2 commits into
devfrom
sagemaker_bucket_write

gnieuwenhuis commented May 26, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented May 26, 2026 •

edited

Loading

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Suggested labels

Suggested reviewers

❌ Failed checks (1 warning)

Uh oh!

github-actions Bot commented May 26, 2026

Uh oh!

github-actions Bot commented May 26, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

gnieuwenhuis commented May 26, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Suggested labels

Suggested reviewers

❌ Failed checks (1 warning)

Uh oh!

github-actions Bot commented May 26, 2026

cdk-nag report

Uh oh!

github-actions Bot commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

cdk diff ✅ Success

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

gnieuwenhuis commented May 26, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented May 26, 2026 •

edited

Loading

github-actions Bot commented May 26, 2026 •

edited

Loading

`cdk diff` ✅ Success