fix(processors/github_metadata): handle ChunkedEncodingError and other network exceptions#149
Closed
Rabenherz112 wants to merge 1 commit into
Closed
Conversation
…r network exceptions
Contributor
Author
|
Is now integrated as well in #152 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes #148
Okay, so correct me if I'm wrong, but as far as I understand it from looking at the requests library docs,
ChunkedEncodingErroris a server-side issue that happens when the server says it sends chunks, but doesn't.Interestingly, when I test this on my own system, I don't get
ChunkedEncodingError, like never. Instead I get a ton of 502 errors, which the code already handles properly with retries (since I had those issues when I built it). But apparently in production/CI environments (or maybe just under different network conditions or regions? not really sure how you / and the CI encounters them but I don't), the server problem shows up asChunkedEncodingErrorinstead of a 502 status code (but this seems also rather rarely? I only saw this happen in the awesome-selfhosted CI once when you manually executed it).As far as I can tell, I can't really tweak the code to prevent
ChunkedEncodingError, since it's a server-side issue (i.e. GitHub doing GitHub things with the API), so all we can do is handle it gracefully. The best way to fix this, which I can think of, is to treatChunkedEncodingError(and similar network exceptions likeConnectionErrorandTimeout) the same way we already treat HTTP502/503/504/429errors, since it's a similar issue:max_retries, error out and log itI tried to reproduce the issue, by running the metadata update process manually with different sleep times multiple times, on multiple days, but I never got the
ChunkedEncodingError. For that reason it would be great if you could give it a try, and see if it works for you. Again, when I run it, I just get this kind of output (I really get a lot of 502's):Output when running the metadata update process manually
Edit: I'm not even sure we want to catch this. This appears to be a one-off error. So not sure whether it's worth adding 100 lines of code for something that happened until now only once and seems also to have a really low chance.