Skip to content

Use a dedicated header for hybrid-query DLS instead of a marker value - #6450

Closed
cwperks wants to merge 1 commit into
opensearch-project:mainfrom
cwperks:hybrid-dls-dedicated-header
Closed

Use a dedicated header for hybrid-query DLS instead of a marker value#6450
cwperks wants to merge 1 commit into
opensearch-project:mainfrom
cwperks:hybrid-dls-dedicated-header

Conversation

@cwperks

@cwperks cwperks commented Sep 1, 2026

Copy link
Copy Markdown
Member

Description

  • Category: Refactoring

Follow-up to #6416. That PR is correct on a fully-upgraded cluster, but it signals its state through a sentinel value stored in an existing header, and that choice is what forces the Version.V_3_9_0 gate to exist. This replaces the sentinel with a dedicated header so the mechanism is fail-safe on its own, and removes the version gate.

No behavior change on a cluster where every node has the fix.

Why the sentinel value needs a version gate

#6416 wrote the marker as a value of OPENDISTRO_SECURITY_FILTER_LEVEL_DLS_DONE and redefined isDlsDoneOnFilterLevel() to exclude it:

public boolean isDlsDoneOnFilterLevel() {
    return threadContext.getHeader(...FILTER_LEVEL_DLS_DONE) != null && !isDlsQueryFilterApplied();
}

That header is transport-propagated to data nodes. On a node that predates the sentinel, the original definition applies — any non-null value means "filter-level DLS handled this request" — and the consumer that matters is SecurityFlsDlsIndexSearcherWrapper:

if (!this.dlsFlsBaseContext.isDlsDoneOnFilterLevel()) {
    dlsRestriction = config.getDocumentPrivileges().getRestriction(privilegesEvaluationContext, index.getName());
} else {
    dlsRestriction = DlsRestriction.NONE;
}

So that shard takes DlsRestriction.NONE, builds no dlsQuery, and applies no reader-level DLS. Hits are still protected by the DLS filter pushed into the hybrid subqueries, but a global aggregation is not, because a global aggregation ignores the top-level query by definition — and retaining reader-level DLS for exactly that case is the stated purpose of the marker.

The design fails open, which is why it needs the version gate to be correct.

What this changes

Use a dedicated OPENDISTRO_SECURITY_HYBRID_QUERY_DLS_APPLIED header instead of overloading the existing one. A node that does not know the header sees absence, and absence means "do the normal thing" — compute the restriction, build the DLS query, apply reader-level DLS. The mechanism fails closed by construction, so there is nothing left to version-gate.

  • isDlsDoneOnFilterLevel() returns to a plain null check. Its four callers keep their original meaning, including the untouched SecurityFlsDlsIndexSearcherWrapper.
  • HYBRID_QUERY_DLS_FILTER_SUPPORTED_SINCE and isHybridQueryDlsFilterSupported are removed, dropping one gate from shouldApplyDlsFilterToHybridQueryInAdaptiveMode and the Version import from DlsFlsValveImpl.
  • New isDlsAlreadyHandled() backs the two coordinator-side re-entry guards, which now consult both markers. These stop the child request dispatched through nodeClient from being reprocessed and run only on the coordinating node, so they carry no compatibility exposure.
  • SecurityInterceptor copies the new header to local destinations and strips it for destinations outside the local cluster. The strip is now an unconditional remove of our own key rather than a conditional edit of a shared header's value.
  • The tri-state string becomes two independent, mutually exclusive booleans.

The runtime verification in applyFilterLevelDls is unchanged. It is still what actually guarantees the Neural Search hybrid-query contract, and it needs no version information.

Why this matters for backports

A Version comparison cannot express "has the fix" across several release lines at once:

  1. The constant differs per branch, so DlsFlsValveImpl diverges and conflicts on every later backport touching the file.
  2. Patch granularity. A 3.5.0 GA would not contain the fix, so V_3_5_0 is too permissive and reopens the gap, while V_3_5_4 requires a constant that OpenSearch core may not have declared yet.
  3. It defeats its own backport. On main, minNodeVersion.onOrAfter(V_3_9_0) is false for a 3.7-with-fix + 3.9 cluster, disabling the fix on exactly the mixed-version clusters a backport is meant to serve.

Without a version constant, the identical patch applies to every release line.

Mixed-version behavior

Shards on nodes with the fix return correct results. Shards on nodes without it wrap the parsed query as they do today and fail the hybrid-query shape, so the request surfaces a shard failure — the same class of failure as the bug #6416 fixed, and never a silent loss of reader-level DLS. This is a documentable rolling-upgrade limitation rather than a security regression, which the sentinel-value design could not offer.

Issues Resolved

Follow-up to #6416. No new issue; this hardens and simplifies the mechanism introduced there and unblocks backporting it.

Testing

./gradlew test for the five affected suites: 87 tests, 0 failures, 0 skipped.

./gradlew test \
  --tests org.opensearch.security.configuration.DlsFlsFilterLeafReaderTest \
  --tests org.opensearch.security.configuration.DlsFilterLevelActionHandlerTest \
  --tests org.opensearch.security.configuration.DlsFlsValveImplTest \
  --tests org.opensearch.security.privileges.dlsfls.DlsFlsBaseContextTest \
  --tests org.opensearch.security.transport.SecurityInterceptorTests

Test changes:

  • Removed hybridQueryDlsFilterRequiresOpenSearchThreeNineOnEveryNode and doesNotApplyHybridQueryFilterWhenClusterContainsOlderNode, which covered the deleted gate.
  • The valve helper now asserts on the marker header and that exactly one of the two markers is set while the child search runs.
  • Re-entry tests parameterize on the header key rather than the header value.
  • SecurityInterceptorTests asserts the new header is copied to local nodes and stripped for an unrecognized destination.

./gradlew precommit passes except for two failures that reproduce identically on unmodified main (verified by stashing this change and re-running) and involve no file touched here:

  • :licenseHeaders — 38 pre-existing unapproved headers under auth/** and ssl/**.
  • :opensearch-sample-resource-plugin:forbiddenApisMainURL#openStream() in CreateResourceTransportAction:77 and CreateResourceGroupTransportAction:70.

Cross-plugin verification

Published this branch to Maven Local and ran opensearch-project/neural-search#1957's HybridQueryDlsIT on a three-node secured cluster, which is what actually exercises the header crossing the transport layer:

cd security && ./gradlew publishPluginZipPublicationToMavenLocal
cd neural-search && ./gradlew :integTest \
  --tests org.opensearch.neuralsearch.query.HybridQueryDlsIT \
  -x spotlessApply -Dsecurity.enabled=true -PnumNodes=3 --refresh-dependencies
Test Result
testDlsFiltersHybridHitsAndAggregations PASSED
testDlsSupportsSuggestionBearingHybridRequest PASSED
testDlsCombinesWithExistingHybridFilter PASSED
testDlsSupportsSingleClauseHybridQuery PASSED
testDlsFiltersHybridKnnQuery Fails on this branch alone; passes with #6428 stacked — see below
testDlsFiltersHybridNeuralQuery Blocked by #6433 (ML model registration), both with and without this change

testDlsFiltersHybridHitsAndAggregations is the important one: it covers the DLS-protected global aggregation, which is precisely the behavior the version gate existed to protect, and it passes across three nodes with the dedicated header. That confirms the header propagates through the SecurityInterceptor copy allowlist and that reader-level DLS stays active on non-coordinating shards.

testDlsFiltersHybridKnnQuery fails on this branch with Hybrid query did not apply the DLS filter to every subquery. I re-ran it against an unmodified main build and got a byte-identical failure reason, so it is pre-existing and depends on the neural/k-NN filter traversal in #6428, not on this refactor.

Interaction with #6428 (verified, not assumed)

I merged #6428 into this branch on a scratch branch and ran the full suite, rather than just eyeballing the overlap.

The merge produced exactly two conflicts, both trivial:

DlsFilterLevelActionHandler itself — #6428's substantive +120 lines — auto-merged with no conflict, because its neural/k-NN filter discovery does not care how the marker is stored.

Stacked results:

That 5-of-6 result matches what #1957 reports for #6428 on its own, so this refactor neither improves nor regresses #6428's coverage. Whichever of the two merges first, the second rebases with a two-line resolution.

Check List

  • New functionality includes testing
  • New functionality has been documented — the compatibility rationale is documented on the new constant and in the code comments; no user-facing setting or API changes
  • New Roles/Permissions have a corresponding security dashboards plugin PR — N/A
  • API changes companion pull request created — N/A
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

PR opensearch-project#6416 signalled "DLS was applied to the top-level hybrid query" by
storing a sentinel value in the existing
OPENDISTRO_SECURITY_FILTER_LEVEL_DLS_DONE header, and redefined
isDlsDoneOnFilterLevel() to exclude that value.

That marker is transport-propagated to data nodes. A node that predates
the sentinel cannot distinguish it from "true", so isDlsDoneOnFilterLevel()
returns true, SecurityFlsDlsIndexSearcherWrapper takes DlsRestriction.NONE
and builds no DLS query, and reader-level DLS is skipped on that shard.
Hits are still protected by the injected hybrid filter, but global
aggregations are not, because a global aggregation ignores the top-level
query by definition. The design therefore fails open and needs the
Version.V_3_9_0 gate to be correct at all.

Use a dedicated OPENDISTRO_SECURITY_HYBRID_QUERY_DLS_APPLIED header
instead. A node that does not know the header sees no marker and runs its
normal reader-level DLS path, so the change fails closed by construction
and the version gate is no longer required:

- isDlsDoneOnFilterLevel() returns to a plain null check, so its four
  callers keep their original meaning.
- Remove HYBRID_QUERY_DLS_FILTER_SUPPORTED_SINCE and
  isHybridQueryDlsFilterSupported, dropping one gate from
  shouldApplyDlsFilterToHybridQueryInAdaptiveMode.
- Add isDlsAlreadyHandled() for the two coordinator-side re-entry guards,
  which now consult both markers. These run only on the coordinating node.
- Copy the new header to local destinations and strip it for destinations
  outside the local cluster.

Without a version constant the identical patch applies to every release
line, which matters for backports: a Version comparison cannot express
"has the fix" across 3.5.x, 3.7.x, and 3.9 at once, and would disable the
fix on exactly the mixed-version clusters a backport is meant to serve.

On a mixed-version cluster the hybrid request now fails on shards hosted
by nodes without the fix, as it does today, rather than silently dropping
reader-level DLS.

The runtime verification in applyFilterLevelDls is unchanged and remains
what actually guarantees the Neural Search hybrid-query contract.

Signed-off-by: Craig Perkins <cwperx@amazon.com>
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ Recommended focus areas for review

Mixed-version rolling upgrade risk

The PR premise is that on a node without this fix, absence of the new OPENDISTRO_SECURITY_HYBRID_QUERY_DLS_APPLIED header causes fail-safe behavior. However, the new header is coordinator-set and transport-propagated to data nodes via the allowlist in sendRequestDecorate. If a newer coordinator sends a search to an older data node (rolling upgrade), the older node will not recognize the new header, will not know that DLS was pushed into the top-level hybrid filter, and will apply its normal reader-level DLS — which is safe for hits and aggregations but may double-apply DLS or interact unexpectedly with the hybrid query wrapping. Confirm this "double DLS" path is functionally correct (e.g. hybrid subqueries already filtered plus reader-level DLS on top) and covered by an upgrade/BWC test; the removal of the HYBRID_QUERY_DLS_FILTER_SUPPORTED_SINCE version gate makes this the only remaining safety net.

if (isDestinationOutsideLocalCluster) {
    // The top-level query filter is only valid within the coordinating cluster. Strip its marker from every
    // action sent to an unrecognized destination, even if RemoteClusterService does not report CCS as enabled.
    headerMap.remove(ConfigConstants.OPENDISTRO_SECURITY_HYBRID_QUERY_DLS_APPLIED);
}

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
General
Verify mixed-version behavior of new header

In a mixed-version cluster during rolling upgrade, an older coordinator node may not
recognize OPENDISTRO_SECURITY_HYBRID_QUERY_DLS_APPLIED and could re-enter DLS
handling if the child request is dispatched locally to a newer node (or vice versa).
Since this header is a new coordinator-side re-entry guard, consider whether
transport-layer stripping to older peers is needed, or document that the marker is
only meaningful within a homogeneously-versioned cluster to prevent DLS being
applied twice or bypassed.

src/main/java/org/opensearch/security/configuration/DlsFilterLevelActionHandler.java [185-189]

+if (applyDlsFilterToHybridQuery) {
+    threadContext.putHeader(ConfigConstants.OPENDISTRO_SECURITY_HYBRID_QUERY_DLS_APPLIED, "true");
+} else {
+    threadContext.putHeader(ConfigConstants.OPENDISTRO_SECURITY_FILTER_LEVEL_DLS_DONE, "true");
+}
 
-
Suggestion importance[1-10]: 4

__

Why: The suggestion raises a legitimate concern about mixed-version behavior during rolling upgrades, but the improved_code is identical to existing_code and only asks to verify/document, limiting its actionable impact.

Low
Avoid unconditional header removal on hot path

The previous code only mutated headerMap when the hybrid marker was present. The new
code unconditionally calls remove for every outbound request to a destination
outside the local cluster, which is wasted work on the hot path. Guard the removal
on the header actually being present in headerMap to avoid unnecessary map mutations
on every remote transport hop.

src/main/java/org/opensearch/security/transport/SecurityInterceptor.java [226-230]

-if (isDestinationOutsideLocalCluster) {
+if (isDestinationOutsideLocalCluster && headerMap.containsKey(ConfigConstants.OPENDISTRO_SECURITY_HYBRID_QUERY_DLS_APPLIED)) {
     // The top-level query filter is only valid within the coordinating cluster. Strip its marker from every
     // action sent to an unrecognized destination, even if RemoteClusterService does not report CCS as enabled.
     headerMap.remove(ConfigConstants.OPENDISTRO_SECURITY_HYBRID_QUERY_DLS_APPLIED);
 }
Suggestion importance[1-10]: 2

__

Why: A Map.remove on a missing key is essentially a no-op with negligible overhead; adding a containsKey guard is a marginal micro-optimization with no meaningful impact.

Low

@codecov

codecov Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.63%. Comparing base (b8c667f) to head (113c546).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #6450      +/-   ##
==========================================
- Coverage   75.66%   75.63%   -0.03%     
==========================================
  Files         456      456              
  Lines       30414    30411       -3     
  Branches     4605     4606       +1     
==========================================
- Hits        23012    23001      -11     
- Misses       5281     5290       +9     
+ Partials     2121     2120       -1     
Files with missing lines Coverage Δ
...ity/configuration/DlsFilterLevelActionHandler.java 72.72% <100.00%> (+0.25%) ⬆️
...security/configuration/DlsFlsFilterLeafReader.java 64.39% <100.00%> (-0.14%) ⬇️
...search/security/configuration/DlsFlsValveImpl.java 72.67% <100.00%> (-0.23%) ⬇️
.../security/privileges/dlsfls/DlsFlsBaseContext.java 100.00% <100.00%> (ø)
...g/opensearch/security/support/ConfigConstants.java 96.55% <ø> (ø)
...search/security/transport/SecurityInterceptor.java 80.31% <100.00%> (-0.11%) ⬇️

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cwperks cwperks closed this Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant