Monitor listen_to connection process in Notifications.Listener#306
Open
satom99 wants to merge 1 commit intocommanded:masterfrom
Open
Monitor listen_to connection process in Notifications.Listener#306satom99 wants to merge 1 commit intocommanded:masterfrom
satom99 wants to merge 1 commit intocommanded:masterfrom
Conversation
Contributor
|
We believe we've seen this in our staging environment. Logs showed some unexplained dbconnection failures and after that point all event handlers/projectors/etc stopped receiving events. If we killed a projector, it would restart and handle all the pending events that it hadn't yet seen but, again, no new events were handled. |
Contributor
|
I think I like this one, would it be easy to add a test for this? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
listen_toparameter is the registered name of thePostgrex.Notificationsprocess that is used to listen for event notifications. That process is not monitored fromNotifications.Listener. If thePostgrex.Notificationsprocess goes down for any reason, theNotifications.Listenermodule does not call the private functionlisten_for_events/1again. This results in a broken state, where we are no longer listening for event notifications.The proposed changes monitor the
Postgrex.Notificationsprocess and react to a DOWN message and stopping theNotifications.Listenerprocess with the same reason. TheNotifications.Listenerprocess is started from aMonitoredServerprocess, and so it is guaranteed to be restarted after stopping.Note: the started
Postgrex.Notificationsprocess is configured withauto_connect: trueand so it should (in theory) never go down. But processes should not be trusted like this.