feat(aws): add serverless filters for DynamoDB, CloudWatch, S3 transfer, Secrets Manager

[JamieCressey]

Summary

Adds specialized RTK filters for high-frequency AWS serverless commands, plus scoped pipe-safety to prevent breaking downstream consumers.

New filters

• DynamoDB scan/query/get-item: Recursive type flattening strips {S/N/BOOL/NULL/L/M/SS/NS/BS} wrappers. Preserves LastEvaluatedKey (pagination token) and ConsumedCapacity (RCU cost) so LLMs know when results are truncated (~40%+ savings)
• CloudWatch Logs filter-log-events: Timestamps include date (MM-DD HH:MM:SS), logStreamName shown per event, consecutive duplicates collapsed with [xN] counts, metadata stripped (~40%+ savings)
• CloudWatch Logs get-query-results: Compact field=value format per row, internal @ptr field filtered out
• S3 sync/cp: Summarize upload/download/delete/copy counts from text output, preserve error and warning lines verbatim. Pass through short output (<10 lines) unchanged (~60%+ savings)
• Secrets Manager get-secret-value: Extract Name + SecretString only, compact-print JSON secrets, strip ARN, VersionId, VersionStages, CreatedDate (~60%+ savings)

Removed

• Lambda invoke: Removed — no meaningful token savings possible without stripping essential debugging info (LogResult contains base64-encoded execution logs, memory/duration stats). Falls through to the generic JSON schema compressor instead.

Pipe safety

• Scoped pipe rewriting via PIPE_UNSAFE_PREFIXES: commands like aws dynamodb scan | python are not rewritten (RTK's compressed output would break json.load()), while text-based commands like git log | grep are still rewritten for savings
• This replaces the previous global "skip all pipe rewrites" approach with targeted per-command scoping

All new filters are added as match arms in aws_cmd::run(), following the existing pattern for STS/S3/EC2/ECS/RDS/CloudFormation. Unmatched subcommands continue to fall through to the generic JSON schema compressor.

Test plan

• 46 unit tests for AWS filters (type flattening, filter output, edge cases, empty results, token savings)
• 158 registry tests including pipe-safety assertions for AWS
• Token savings assertions verified per filter
• cargo fmt --all --check passes
• cargo clippy --all-targets — 0 errors
• cargo test --all — 963 tests pass, 0 failures
• Manual test: rtk aws dynamodb scan --table-name <table>
• Manual test: rtk aws logs filter-log-events --log-group-name <group>
• Manual test: rtk aws s3 sync <src> <dst>
• Manual test: aws dynamodb scan ... | python passes raw JSON through (pipe-safe)

[aeppling]

Hello,

Thanks for contributing, new commands filters are always welcome !

There is things to be solved for this to be merged;

Main concern

This PR does pipe rewrite changes

The second commit changes pipe rewriting behavior in discover/registry.rs for all commands globally, not just AWS. This is not a AWS change but a core modification. Scope this in a new PR this could be a regression. Per CONTRIBUTING.md single-focus rule, this trade-off deserves its own PR with targeted handling. Same goes for the test of this feature which are located in registry.rs but aws focused

Filters review

DynamoDB scan/query

The filter drops LastEvaluatedKey. This is the pagination token. An LLM using this output has no way to know the scan was truncated at the DynamoDB page boundary. It could assume it has all the data.

Also drops ConsumedCapacity, which is minor but useful for cost debugging,

CloudWatch filter-log-events

Timestamps are truncated to HH:MM:SS, losing the date entirely. If searching across multiple days (common when debugging production incidents), you can't tell which day a log entry belongs to. At minimum keep the date: 01-15 10:30:00 or ISO short.

Also drops logStreamName, which matters when querying across multiple streams — you can't tell which Lambda invocation or container produced a given log line.

Lambda invoke

Only outputs Lambda: 200. Strips LogResult which contains the base64-encoded execution logs — START/END/REPORT lines, memory usage, billed duration, and all console.log/print output from the function. This is the primary debugging information when invoking a Lambda.

A developer running aws lambda invoke and getting back just a status code has lost the most useful part of the response. At minimum, decode and include the LogResult content (which is where the real token savings should come from — filter the decoded logs, not discard them).

Thanks again for contributing to RTK !

[aeppling]

I'm discussing with maintainers about the filters i mentioned in the review, if we should keep those or not .

We are thinking about different level of filtering options, for now i need to review with maintainers.

[aeppling]

This PR has many unrelated commits, can you please solve this and i'm ok with this feat ?

You could just cherry pick your commits in another PR , in this case, please tag the new PR here, i'll close this one and accept the other if it correctly introduce those cmd, as well as your PIPE_UNSAFE_PREFIXES once tested with others cmd

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ pszymkowiak
❌ JamieCressey
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[aeppling]

Hey

We are cleaning up the codebase and improving the project structure for better onboarding. As part of this effort, PR #826 reorganizes src/ from a flat layout into subfolders.

No logic changes — only file moves and import path updates.

What you need to do

Rebase your branch on develop when receiving this comment:

git fetch origin && git rebase origin/develop

Git detects renames automatically. If you get import conflicts, update the paths:

use crate::git;        // now: use crate::cmds::git::git;
use crate::tracking;   // now: use crate::core::tracking;
use crate::config;     // now: use crate::core::config;
use crate::init;       // now: use crate::hooks::init;
use crate::gain;       // now: use crate::analytics::gain;

Need help rebasing? Tag @aeppling

[jbronssin]

Hey @JamieCressey — really solid work here. The DynamoDB type flattening, the S3 transfer summarization, and especially the pipe safety logic are all well thought out.

I opened #885 which covers a broader AWS expansion (25 filters total), and I cherry-picked several of your ideas into it:

• The DynamoDB i64-first-then-f64 parsing for N types — much better than just f64
• DynamoDB get-item as a separate filter
• S3 sync/cp text summarization
• Secrets Manager get-secret-value
• CloudWatch get-query-results
• ConsumedCapacity and LastEvaluatedKey display in scan/query

The main architectural difference is that #885 uses a shared runner (run_aws_filtered()) with tee-on-truncation (so truncated lists always have a [full output: ...] recovery path), whereas this PR uses the original per-handler boilerplate.

I think #885 supersedes this one given the overlap, but your pipe safety feature (PIPE_UNSAFE_PREFIXES) is something we don't have yet — that's a great idea that should be a follow-up PR on its own since it touches the registry, not just aws_cmd.

Thanks for the inspiration on this — credit where it's due.

[aeppling]

Hello,

We are going to use this PR : #885

Which is up to date and implement more filters

Thanks for you contribution @JamieCressey !