Skip to content

feat(aws): expand CLI filters from 8 to 25 subcommands#885

Open
jbronssin wants to merge 4 commits intortk-ai:developfrom
jbronssin:feat/aws-expand-cli-filters
Open

feat(aws): expand CLI filters from 8 to 25 subcommands#885
jbronssin wants to merge 4 commits intortk-ai:developfrom
jbronssin:feat/aws-expand-cli-filters

Conversation

@jbronssin
Copy link
Copy Markdown
Contributor

@jbronssin jbronssin commented Mar 27, 2026

Summary

  • Add 17 specialized AWS filters covering: CloudWatch Logs (get-log-events, filter-log-events, get-query-results), CloudFormation events, Lambda (list + get), IAM (roles + users), DynamoDB (scan, query, get-item), ECS tasks, EC2 security groups, S3API objects, S3 sync/cp, EKS cluster, SQS messages, Secrets Manager
  • Extract run_aws_filtered() shared runner, replacing 25-line boilerplate per handler
  • Add force_tee_hint() — truncated output always saves full data to a file with a recovery hint, so LLMs don't burn tokens working around missing data
  • DynamoDB type unwrapping ({"S":"foo"}"foo") with i64-first precision
  • Security: Lambda Environment and Code.Location are never emitted

Inspired by #644 for DynamoDB get-item, S3 transfer, Secrets Manager, query-results, and N-type precision.

Test plan

  • 65 unit tests (19 existing + 46 new)
  • Token savings verified per filter
  • Security: Lambda tests assert secrets absent from output
  • Edge cases: invalid JSON, missing fields, empty arrays, truncation with tee recovery
  • cargo fmt --all --check && cargo clippy --all-targets && cargo test --all

Add 12 specialized filters for high-token-waste AWS operations:
- CloudWatch Logs (get-log-events, filter-log-events)
- CloudFormation describe-stack-events (failures-first)
- Lambda list-functions, get-function (strips Environment secrets)
- IAM list-roles, list-users (strips AssumeRolePolicyDocument)
- DynamoDB scan/query (recursive type unwrapping)
- ECS describe-tasks
- EC2 describe-security-groups (compact rule notation)
- S3API list-objects-v2
- EKS describe-cluster
- SQS receive-message (strips ReceiptHandle)

Refactor: extract run_aws_filtered() shared runner, eliminating
25-line boilerplate per handler. Add force_tee_hint() so truncated
output always has a recovery path (no dead ends for LLMs).

Security: Lambda Environment and Code.Location are never emitted.
@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Jb seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@jbronssin
Copy link
Copy Markdown
Contributor Author

CLA assistant check Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

Jb seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

already signed it in the original message/issue

@jbronssin
Copy link
Copy Markdown
Contributor Author

Live output examples

Tested against a real AWS account. Personal data redacted.

rtk aws sts get-caller-identity

AWS: 123456789012 arn:aws:iam::123456789012:user/dev

(raw: 150 chars → filtered: 55 chars)

rtk aws lambda list-functions

queue-discovery python3.12 128MB 30s active
consumer python3.12 1024MB 120s active
sync python3.12 512MB 300s active
migration python3.12 1024MB 900s active
dbcheck python3.12 128MB 120s active

(raw: 8.5KB → filtered: 224 chars — 97% reduction. Environment variables with secrets are stripped.)

rtk aws iam list-roles (38 roles, truncated at 20)

consumer-lambda 2026-02-25
queue-discovery 2026-02-27
AmazonBedrockExecutionRoleForKnowledgeBase_xxx 2025-09-24 [Bedrock Knowledge Base access]
AWSServiceRoleForECS 2024-12-15 [Policy to enable Amazon ECS...]
AWSServiceRoleForRDS 2025-01-07 [Allows Amazon RDS to manage AWS resources...]
... +18 more roles
[full output: ~/.local/share/rtk/tee/1774636622_aws_iam_list-roles.log]

(Truncated output includes tee recovery path — LLM can cat the file for full data. AssumeRolePolicyDocument stripped from each role.)

rtk aws ec2 describe-instances

EC2: 9 instances
  i-0f2cf06a0ecb running t3a.micro 10.0.3.228 (web-proxy)
  i-0d7487bba7d9 stopped t3.xlarge 10.0.1.183 (github-runner)
  i-0bcfc1103962 running t3a.micro 10.0.1.202 (swarm-node)
  i-03c59e09215  running t3.medium 10.0.1.242 (assets-prod)
  i-01a6a741e5c  running t3.medium 10.0.1.67 (biz-plan)
  i-0dca24ffb1c  running g4dn.xlarge 10.0.1.79 (stt-worker)
  i-034a6143845  terminated g4dn.xlarge - (stt-worker)
  i-0e21b30e34f  running t3a.micro 10.1.30.152 (monitoring)
  i-02f5cdab183  terminated g4dn.xlarge - (stt-worker)

(raw: ~25KB with BlockDeviceMappings, NetworkInterfaces, SecurityGroups, Tags, etc. → filtered: 9 lines)

rtk aws iam list-users

client created:2026-02-20
dean created:2025-05-20
deployment created:2025-11-13
eric created:2026-02-09
florent created:2025-12-10
github-actions created:2026-03-06
jb created:2025-08-06

(Strips UserId, Arn, Path, PasswordLastUsed — keeps what matters)

Key design choices visible in output

  1. Tee recovery on truncation: When roles are truncated (... +18 more), the full raw output is saved and the hint [full output: ...] is printed. LLMs can cat that file instead of burning tokens on workarounds.
  2. Security stripping: Lambda Environment variables (which often contain API keys, DB passwords) are never emitted.
  3. Smart formatting: Each command uses the format that makes sense — one-line summary for STS, tabular for EC2/Lambda, list for IAM.

…filters

Inspired by rtk-ai#644 — cherry-picks the best ideas and integrates them
into the shared runner architecture:

- DynamoDB get-item: single-item unwrapping with ConsumedCapacity
- DynamoDB scan/query: now shows ConsumedCapacity (RCU) and pagination status
- DynamoDB N-type: try i64 first, then f64 (better precision)
- CloudWatch Logs get-query-results: field=value format, strips @ptr
- S3 sync/cp: text-based transfer summary (upload/download/delete counts)
- Secrets Manager get-secret-value: extracts Name + SecretString only

Total: 25 specialized AWS filters + generic fallback.
@jbronssin
Copy link
Copy Markdown
Contributor Author

Update: Added a second commit with 5 more filters inspired by #644 (great work by @JamieCressey):

  • DynamoDB get-item: single-item type unwrapping with ConsumedCapacity display
  • CloudWatch Logs get-query-results: field=value compact format, strips @ptr
  • S3 sync/cp: text-based transfer summary (counts uploads/downloads/deletes, preserves error lines)
  • Secrets Manager get-secret-value: extracts Name + SecretString, strips ARN/VersionId noise
  • DynamoDB N-type precision: now tries i64 before f64 (from feat(aws): add serverless filters for DynamoDB, CloudWatch, S3 transfer, Secrets Manager #644's approach)
  • DynamoDB scan/query: now shows ConsumedCapacity (RCU) and pagination status

Total is now 25 specialized AWS filters + generic fallback. 65 unit tests.

@jbronssin jbronssin changed the title feat(aws): expand CLI filters from 8 to 20 subcommands feat(aws): expand CLI filters from 8 to 25 subcommands Mar 28, 2026
@aeppling aeppling self-assigned this Mar 28, 2026
@aeppling
Copy link
Copy Markdown
Contributor

Hello jbronssin !

Thanks for contributing to RTK, this PR will close #644 as expected, but a few thing to be addressed before :

Bug: filter_logs_query_results: redundant fallback

let field_value = field["value"]
    .as_str()
    .unwrap_or_else(|| field["value"].as_str().unwrap_or("?"));

The closure calls the exact same .as_str() again. Non-string values (numbers, booleans) always render as "?" instead of their serialized form.


Bug: filter_cfn_events underreports failure count

The summary uses failed.len() which is capped at MAX_ITEMS (20). If there are 50 failures, the summary says "20 failed" instead of "50 failed".


Missing: insta snapshot tests

CONTRIBUTING.md requires assert_snapshot!() for every new filter. None of the 17 new filters have snapshot tests. At minimum the major filters (logs, dynamodb, lambda, cfn events, security groups) need them.


Missing: force_tee_hint() has no unit tests

New public function in tee.rs with no tests. At minimum: verify it respects RTK_TEE=0 and enabled=false.


Missing: empty collection edge cases

No test covers empty arrays for any new filter ({"Functions": []}, {"Roles": []}, {"Items": [], "Count": 0, "ScannedCount": 0}, {"tasks": []}, {"SecurityGroups": []}, {"Messages": []}, {"events": []}, {"results": []}, etc). These can expose panics or unexpected output.

@aeppling
Copy link
Copy Markdown
Contributor

Thanks to @JamieCressey as well

Jb added 2 commits March 28, 2026 14:07
- Fix redundant .as_str() fallback in filter_logs_query_results
- Fix CFN events underreporting failure count (used capped vec len
  instead of separate counter)
- Add force_tee_hint() unit tests (env disable + write path)
- Add snapshot-style tests for major filters (logs, lambda, dynamodb,
  security groups, cfn events)
- Add empty collection edge case tests for all filters
- Add test verifying CFN failure count > MAX_ITEMS reports correctly
- Update src/cmds/cloud/README.md with expanded AWS filter details
- Add AWS section to README.md command list + rewrite mapping table
- Add CHANGELOG.md entries under [Unreleased]
- Document force_tee_hint() in src/core/README.md
@jbronssin
Copy link
Copy Markdown
Contributor Author

@aeppling thanks for the thorough review — all 5 points addressed in the last two commits:

Bug: redundant fallback in filter_logs_query_results — Fixed, now just .unwrap_or("?"). Good catch on the non-string values too.

Bug: filter_cfn_events underreports failure count — Added a separate failed_count counter that increments for every failure, independent of the failed vec which is capped at MAX_ITEMS for display. Test added to verify 30 failures reports "30 failed" not "20 failed".

Missing: snapshot tests — Added 5 snapshot-style assert_eq! tests on full formatted output for logs, lambda, dynamodb, security groups, and cfn events. (Note: insta isn't a dependency in the project — no module uses assert_snapshot!() currently — so these use exact assert_eq! on the full output string, which serves the same purpose.)

Missing: force_tee_hint() tests — Added 2 tests: env disable (RTK_TEE=0) and write+format verification.

Missing: empty collection edge cases — Added 11 tests covering empty arrays for Functions, Roles, Users, Items, tasks, SecurityGroups, Contents (absent key), Messages (absent key), events, Reservations, and StackEvents.

Total is now 82 AWS tests + 2 tee tests.

@aeppling
Copy link
Copy Markdown
Contributor

CLA already signed , will bypass this check
image

@aeppling
Copy link
Copy Markdown
Contributor

1. filter_logs_query_results: non-string values still render

The redundant fallback is gone (good), but non-string values (numbers, booleans) from CloudWatch Insights still become "?".

2. Aggresive filters (new feedback, forgot to mention this before:

RTK filter transparently commands output, so we cannot strip important information that could cost token from the LLM searching this information and having a hard time understanding why it has not.

Please review the following to ensure those are not too aggressive :

  1. EC2 instances — Strips SecurityGroups, PublicIP, SubnetId, VpcId, NetworkInterfaces. These are the fields LLMs need
    most for connectivity debugging. No tee recovery for field-level stripping.
  2. RDS instances — Strips the Endpoint (host:port). This is the single most commonly requested field from
    describe-db-instances. An LLM literally cannot answer "what's my database endpoint?" from this output.
  3. IAM roles — Strips AssumeRolePolicyDocument entirely. This is critical for debugging permission issues ("which
    service can assume this role?"). No recovery path for small result sets.
  4. ECS tasks — Strips stoppedReason and exitCode. When debugging why a task is STOPPED, the reason is THE essential
    field. Without it, the LLM sees "STOPPED" but can't diagnose anything.

Some can be debatable but those like EC2 instance stripping are too aggressive for example

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants