Fix assetlibrary history batching

[TonySherman]

Description

As @aaronatbissell mentioned in #87 - Large amounts of assetlibrary changes can cause hitting lambda concurrency limits. This proposed change utilizes a Kinesis Stream to batch updates to batches of 100 or 30 second window (These can be configured differently if needed)

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Refactor (existing code being refactored)
• This change includes a documentation update

Submission Checklist

• Build Verified
• Bundle Verified
• Lint passing
• Unit tests passing
• Integration tests passing
• Change logs generated

Additional Notes:

[aaronatbissell]

Any questions or feedback on this PR that I can help to answer?

[aaronatbissell]

This PR has been sitting for quite a while. Any chance this is even on the radar to get merged in?

@hassankhokhar @boardthatpowder @rrangnekar

[aaronatbissell]

I see there's some movement on this repo again! Is there an appetite to merge in this Pull request (or any other community pull requests)?

@ts-amz @canavandl @BenjiTheC

[aaronatbissell]

One last effort here to get any kind of feedback from anyone on the team, otherwise we will just fork this and manage it on our own. Is this even worth me going through and fixing all this stuff so that we can get this in?

[ts-amz]

@aaronatbissell, as with the other PR, we have resumed reviewing PRs and will be happy to review this once you've updated the conflicts. I am sorry for the delay in response, as we were undergoing a team change.

[ts-amz]

Hello @aaronatbissell, are you planning to work on this PR? We plan to close this on 9/8/23 for tracking purposes if it's not still active (in that case, please re-open it if/when you have update it). Thank you

[aaronatbissell]

Yes - I do plan on working this. I just don't have the time to do it right now. It will probably be in a month or so

[aaronatbissell]

@ts-amz - rebased on main

[canavandl]

@aaronatbissell do you know how we can test this before merging?

[TonySherman]

@canavandl This is going back a bit for me and I no longer use cdf but I believe @aaronatbissell and I were able to deploy these changes to a lower environment and make sure that asset library history table was being updated with changes. It might be more difficult to test large batches. Maybe set the BatchSize and MaximumBatchingWindowInSeconds to a very low number to force updates to get batched?

[aaronatbissell]

@ts-amz @canavandl - FYI, just made a couple more updates to this. Unit tests are included with #193. We've pushed this out to our system to ensure it's working. I think this is ready for an in-depth review and merge.

[aaronatbissell]

After running for a few days in our production environment, this asset library history batching has reduced our total lambda duration by about 60% and reduced invocations by 99%.