Record submission counts#2147
Conversation
| private | ||
|
|
||
| def submission_counts_by_form_id | ||
| Submission.where(mode: "form").group(:form_id).count |
There was a problem hiding this comment.
Currently I think the submission counts collected by Anne exclude submissions to test forms by our end to end tests and smoke tests. Ought we do the same here?
There was a problem hiding this comment.
That's okay - submissions are grouped by form id, so we can filter them out after.
There was a problem hiding this comment.
The end-to-end tests create a form, submit it and then delete it. I don't know if we'll easily have the form ids to exclude and including known form ids would be tricky too, i think?
There was a problem hiding this comment.
Mmm I agree with what Tom says, it feels like it will be easier to filter out before sending to CloudWatch, we can use the same criteria Anne uses currently in Splunk:
form_name!=capybara* form_name!=*smoke* form_name!=s3*
There was a problem hiding this comment.
I've updated this PR to now use OTel, and subsequently label these forms as mode "test" - so they can easily be filtered out.
theseanything
force-pushed
the
theseanything/submission-number-metrics
branch
from
82471a3 to
70c1c91
Compare
theseanything
force-pushed
the
theseanything/submission-number-metrics
branch
from
70c1c91 to
f5c3ea3
Compare
theseanything
changed the title
theseanything
force-pushed
the
theseanything/submission-number-metrics
branch
3 times, most recently
from
1a4b8c2 to
26b26d0
Compare
theseanything
changed the title
Record counts with an incrementing counter at submission creation time, exported via PeriodicMetricReader, instead of periodically aggregating submission counts from the database.
Submissions to forms used for automated platform testing (capybara, smoke test and s3 test forms) were counted with their real mode, mixing them in with genuine user submissions. Label them as "test" so dashboards can exclude them without name-based filters.
The submission count metric was incremented when the submission record was created, before the delivery job was enqueued. If enqueueing failed the submission was destroyed but the counter had already been bumped, inflating the KPI. Move the metric call to after the enqueue block so it only counts submissions that were successfully queued for delivery.
Use OpenTelemetry semantic conventions for the counter name, unit and attribute keys so the metric is consistent with other instrumented services and standard dashboards.
The deployment.environment.name attribute is upserted by the OTel collector, so we don't need to manually add an environment label to every metric we record.
theseanything
force-pushed
the
theseanything/submission-number-metrics
branch
from
ace0f57 to
760424f
Compare
|
🎉 A review copy of this PR has been deployed! You can reach it at: https://pr-2147.submit.review.forms.service.gov.uk/ It may take 5 minutes or so for the application to be fully deployed and working. If it still isn't ready For the sign in details and more information, see the review apps wiki page. |
Records a SubmissionCount metric using an OpenTelemetry counter, incremented as each submission is created and exported through the OTel collector (metrics support enabled in govuk-forms/forms-deploy#2182).
This allows as to track platform usage over time without manual reporting and is one of our key KPIs used to drive product development. We get stats exported every minute.
We are using Otel instead of standard CloudWatch Metrics as it's significantly cheaper. Allows us to have high cardinality and get granular submission counts per form.
Submissions to automated test forms (capybara, smoke test and s3 test forms) are labelled with mode test, so dashboards can exclude them without repeating name-based filters in every query.
Trello card: https://trello.com/c/b3cNvzu5/3136-record-kpis-in-cloudwatch
tested in dev.