Skip to content

[fix][broker] Fix potential OOM due to client use sasl authentication - #20713

Merged
315157973 merged 7 commits into
apache:masterfrom
315157973:oom
Jul 23, 2023
Merged

[fix][broker] Fix potential OOM due to client use sasl authentication#20713
315157973 merged 7 commits into
apache:masterfrom
315157973:oom

Conversation

@315157973

@315157973 315157973 commented Jul 4, 2023

Copy link
Copy Markdown
Contributor

Motivation

Sasl authentication has two phases. In the first phase, the broker will temporarily cache the authentication info, and clean the cache at the end of the second phase.
When the client crashes after phase 1, or someone deliberately attacks, the auth info will always remain in the broker's memory.
If the client keeps reconnecting like this, the broker will soon OOM.

Modifications

Change ConcurrentHashMap to caffeine cache , and force clean cache after a certain period of time.

Verifying this change

  • Make sure that the change passes the CI checks.

Documentation

  • doc-not-needed

Matching PR in forked repository

315157973#10

@315157973

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

@codecov-commenter

codecov-commenter commented Jul 5, 2023

Copy link
Copy Markdown

Codecov Report

Merging #20713 (d56f491) into master (1687244) will increase coverage by 0.44%.
The diff coverage is 100.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #20713      +/-   ##
============================================
+ Coverage     72.75%   73.19%   +0.44%     
- Complexity    32235    32293      +58     
============================================
  Files          1856     1868      +12     
  Lines        138999   139313     +314     
  Branches      15302    15339      +37     
============================================
+ Hits         101128   101976     +848     
+ Misses        29860    29304     -556     
- Partials       8011     8033      +22     
Flag Coverage Δ
inttests 24.21% <44.44%> (-0.07%) ⬇️
systests 25.18% <44.44%> (?)
unittests 72.49% <100.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ker/authentication/AuthenticationProviderSasl.java 59.84% <100.00%> (+1.44%) ⬆️
...org/apache/pulsar/broker/ServiceConfiguration.java 99.38% <100.00%> (+<0.01%) ⬆️

... and 160 files with indirect coverage changes

@lhotari
lhotari requested a review from michaeljmarshall July 12, 2023 14:08
Comment on lines +116 to +117
this.authStates = Caffeine.newBuilder()
.expireAfterWrite(config.getInflightSaslContextExpiryMs(), TimeUnit.MILLISECONDS).build();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This kind of caching would still be vulnerable to a high frequency of calls, similar to a TCP SYN flood. Should we also put an upper limit on the number of in flight contexts so that old contexts are dropped?

@315157973 315157973 closed this Jul 21, 2023
@315157973 315157973 reopened this Jul 21, 2023
@315157973
315157973 merged commit e7ae9e2 into apache:master Jul 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-not-needed Your PR changes do not impact docs ready-to-test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants