Checksum enable fallback fixes for Spark 4.2 [databricks]

[abellina]

Fixes #14731

Description

The assertion in the issue #14731 is correct:

Investigation focus: Confirm on Spark 4.2 whether SparkEnv.get.conf.getBoolean for these keys matches effective SQLConf defaults everywhere RAPIDS shuffle initializes; if not, the plugin may not fall back when Spark SQL enables checksums by default (see GpuShuffleEnv vs ShuffleExchangeExec).

I find that in spark 4.2 that SparkConf does not get the SqlConf default, so we do need to check both, starting with the sql conf one. So we have to check. I added some helpers for the 4.1+ shim to deal with this and tests to show we fallback.

Checklists

Documentation

• Updated for new or modified user-facing features or behaviors
• No user-facing change

Testing

• Added or modified tests to cover new code paths
• Covered by existing tests
  (Please provide the names of the existing tests in the PR description.)
• Not required

Performance

• Tests ran and results are added in the PR description
• Issue filed with a link in the PR description
• Not required

[abellina]

build

[abellina]

I think I need to add handing for this case:

1. user starts with checksums disabled (we don't fallback)
2. user enabled checksums and now we have to fallback.

The current code does this only once on startup.

[nvauto]

NOTE: release/26.08 has been created from main. Please retarget your PR to release/26.08 if it should be included in the release.

[abellina]

I think I need to add handing for this case:

1. user starts with checksums disabled (we don't fallback)
2. user enabled checksums and now we have to fallback.

The current code does this only once on startup.

I added special handling for the user flipping the configs at runtime. I tested this manually in a spark shell where I had either false defaults, true defaults for either config, and flipped them on and off and made sure the correct writer and reader were instantiated.

[greptile-apps]

Greptile Summary

The PR fixes Spark 4.2 row-based checksum fallback behavior.

• Resolves checksum settings from SQLConf first, then SparkConf, and finally Spark’s registered defaults.
• Records the fallback decision when each shuffle is registered so its writers and readers consistently use the same implementation.
• Adds Spark 4.2 tests for configuration precedence, defaults, and dynamic per-shuffle decisions.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RowBasedShuffleChecksumConf.scala	Adds centralized resolution of row-based checksum settings with SQLConf precedence and Spark-default handling.
sql-plugin/src/main/scala/org/apache/spark/sql/rapids/GpuShuffleDependency.scala	Stores the registration-time checksum fallback decision on the shuffle dependency.
sql-plugin/src/main/scala/org/apache/spark/sql/rapids/GpuShuffleEnv.scala	Uses the centralized helper when determining whether row-based checksums are enabled.
sql-plugin/src/main/scala/org/apache/spark/sql/rapids/RapidsShuffleInternalManagerBase.scala	Selects fallback per shuffle and consistently honors the recorded decision in writer and reader paths.
sql-plugin/src/test/spark420/scala/com/nvidia/spark/rapids/shims/spark420/SparkShimsSuite.scala	Tests Spark 4.2 checksum defaults and SQLConf-over-SparkConf precedence.
sql-plugin/src/test/spark420/scala/org/apache/spark/sql/rapids/RapidsShuffleManagerChecksumSuite.scala	Tests registration-time fallback selection across checksum sources and changing SQLConf state.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Register GPU shuffle] --> B[Resolve checksum settings]
    B --> C{Checksum enabled?}
    C -->|Yes| D[Store checksumFallback = true]
    C -->|No| E[Store checksumFallback = false]
    D --> F[Spark writer and reader]
    E --> G[RAPIDS writer and reader]

Loading

_{Reviews (8): Last reviewed commit: "Fix the reader which was still using a d..." | Re-trigger Greptile}

[abellina]

build

[abellina]

build

[abellina]

build

[abellina]

build

[gerashegalov]

[P1] Persist the fallback choice for multithreaded shuffles. registerShuffle deliberately leaves these on an unmarked BaseShuffleHandle, so this writer and the reader at line 2012 independently evaluate the task-local SQLConf. If a shuffle is reused after the configuration changes, or is otherwise observed under a different SQLConf, one side can choose the threaded RAPIDS implementation and the other Spark shuffle, producing missing output or fetch failures. Please record the registration-time choice per shuffle and use it for every writer and reader. The new suite fixes useMultiThreadedShuffle = false, so it does not cover this branch.

[gerashegalov]

[P2] This reset is global to the shuffle manager, not scoped to a session or query. With concurrent sessions, any task with checksums disabled resets the flag, and the next task from an enabled session logs again; alternating tasks can therefore emit this warning repeatedly. Please either make this a once-per-manager warning without resetting or scope the state to a shuffle/query.

[abellina]

choosing the simpler: warn once for the lifetime of the manager. It seems unlikely users will flip between these configs at runtime.

[gerashegalov]

[P3] The version claim here is inaccurate: upstream Spark 4.1.0 and 4.1.1 default both checksum settings to false; they become true in Spark 4.2. Please change this to "Spark 4.2+". Because that default makes Spark 4.2 fall back to CPU shuffle unless both settings are disabled, please also document the performance impact and opt-out in user-facing docs or release notes.

[abellina]

build

[greptile-apps]

Want your agent to iterate on Greptile's feedback? Try greploops.

[abellina]

build

[abellina]

@gerashegalov if you do have questions just let me know and I'll get another PR going.