There's currently no simple/automated process to allow the builder-api to scale in/out dependant on load. Ideally we'd be able to do this based on the number of outstanding jobs in the work queue.
One of the considerations for separating the search and builder elements to 2 different services is that they can scale independently, as mentioned on text-services.md
There is currently no automated way to determine the number of outstanding tasks. This ticket is to look at how we can achieve that. Initial thoughts:
- Have custom metrics emitted
- By Hangfire? Or an alternative service?
- Can Hangfire natively handle scaling out, do we need some sort of broker?
- At what point would Hangfire start to overload the postgres instance? (ie is there a ceiling we can scale to?)
- Switch from Hangfire to something that is more observable? i.e. a queueing service?
- SQS is the general service that we will use but we don't want to tie this to AWS.
There's currently no simple/automated process to allow the builder-api to scale in/out dependant on load. Ideally we'd be able to do this based on the number of outstanding jobs in the work queue.
One of the considerations for separating the search and builder elements to 2 different services is that they can scale independently, as mentioned on text-services.md
There is currently no automated way to determine the number of outstanding tasks. This ticket is to look at how we can achieve that. Initial thoughts: