Skip to content

Use 24h queues for data drip#3

Draft
arnauvp-factorial wants to merge 2 commits intomainfrom
FCT-47101-use-slo-queues
Draft

Use 24h queues for data drip#3
arnauvp-factorial wants to merge 2 commits intomainfrom
FCT-47101-use-slo-queues

Conversation

@arnauvp-factorial
Copy link

🚪 Why?

We want to simplify the queue system, so that later we can decouple redis instances per domain.

🔑 What?

Instead of dedicated data_drip queues, we use the 24h ones.

🏡 Context

backfill_run.amount_of_elements
) if backfill_run.amount_of_elements.present? &&
backfill_run.amount_of_elements > 0
if backfill_run.amount_of_elements.present? &&
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess some formatting rules?

module DataDrip
class Dripper < ActiveJob::Base
queue_as :data_drip
queue_as :within_24_hours
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sizief do we need to port anything else from the base job in the monorepo?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make this value configurable at lib/datadrip.rb? Outside factorial this queue name doesn't make a lot of sense

@arnauvp-factorial
Copy link
Author

@oriolgual this CI is needed to compile and publish the gem? Or it is just for validation? cc @Avalonvdhorst

@oriolgual
Copy link
Contributor

@oriolgual this CI is needed to compile and publish the gem? Or it is just for validation? cc @Avalonvdhorst

It's just for validation, the gem is not published yet

@arnauvp-factorial arnauvp-factorial marked this pull request as draft January 29, 2026 08:37
@arnauvp-factorial
Copy link
Author

Oriol mentioned since this is an open source project, and the SLO system is internal for factorial, that we need to make the queue name configurable and override that in our deployments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants