Skip to content

Enable index worker SQS extended-client consumption#4

Open
dorr91 wants to merge 1 commit into
mainfrom
david/index-sqs-extended-client
Open

Enable index worker SQS extended-client consumption#4
dorr91 wants to merge 1 commit into
mainfrom
david/index-sqs-extended-client

Conversation

@dorr91

@dorr91 dorr91 commented Mar 3, 2026

Copy link
Copy Markdown

Hubspot ingest jobs are processing tons of records and then trying to send an index message that's too large for the index SQS queue. On the ingest queue side we solved this by using sqs extended client, let's do that here, too.

Note that we'll need to expand permissions for the index and ingest worker roles to allow using s3:

  • ingest: write to specified s3 bucket
  • index: read + delete from specified s3 bucket

Note

Cursor Bugbot is generating a summary for commit e726ea7. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant