Skip to content

Reduce timezone integration tests in precommit [fast-ut] [reduced-it] [databricks] - #15419

Draft
res-life wants to merge 2 commits into
NVIDIA:mainfrom
res-life:fix/precommit-timezone-marker
Draft

Reduce timezone integration tests in precommit [fast-ut] [reduced-it] [databricks]#15419
res-life wants to merge 2 commits into
NVIDIA:mainfrom
res-life:fix/precommit-timezone-marker

Conversation

@res-life

@res-life res-life commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Fixes #15345.

Description

Problem

Premerge currently reuses tz_sensitive_test, the marker intended for comprehensive nightly coverage. Its parameterized tests collect 1,542 cases for each additional timezone, so the two premerge runs take approximately 10 minutes in total.

Using reduced-IT selection alone would still retain 318 cases per timezone. The premerge path needs a smaller, coverage-oriented subset instead of the complete nightly matrix.

Change

Register tz_sensitive_test_for_precommit and apply it to nine representative cases per timezone:

  • cast and date-time cases that exercise the core GpuTimeZoneDB conversion paths
  • ORC coverage for fixed-offset and recurring-DST timezone pairs
  • ORC V1 and V2 readers with one representative reader configuration

Update spark-premerge-build.sh to select the new marker. Existing tz_sensitive_test annotations and nightly selection remain unchanged.

Impact

Premerge collection decreases from 1,542 to 9 cases per timezone, or from 3,084 to 18 cases across both timezone runs—a 99.4% reduction. Nightly CI continues to run the complete timezone-sensitive matrix. There is no user-facing behavior change.

Validation

  • Collected 1,542 cases with -m tz_sensitive_test
  • Collected 9 cases with -m tz_sensitive_test_for_precommit
  • Passed all 9 selected cases with Asia/Shanghai
  • Passed all 9 selected cases with America/New_York
  • Completed a Spark 3.5.6 clean build with JDK 17

Checklists

Documentation

  • Updated for new or modified user-facing features or behaviors
  • No user-facing change

Testing

  • Added or modified tests to cover new code paths
  • Covered by existing tests
    (Please provide the names of the existing tests in the PR description.)
  • Not required

Performance

  • Tests ran and results are added in the PR description
  • Issue filed with a link in the PR description
  • Not required

Signed-off-by: Chong Gao <chongg@nvidia.com>
@res-life res-life changed the title Reduce timezone integration tests in precommit Reduce timezone integration tests in precommit [fast-ut] [reduced-it] Jul 29, 2026
@res-life

Copy link
Copy Markdown
Collaborator Author

build

Signed-off-by: Chong Gao <chongg@nvidia.com>
@res-life res-life closed this Jul 29, 2026
@res-life res-life reopened this Jul 29, 2026
@res-life

Copy link
Copy Markdown
Collaborator Author

build

@res-life res-life changed the title Reduce timezone integration tests in precommit [fast-ut] [reduced-it] Reduce timezone integration tests in precommit [fast-ut] [reduced-it] [databricks] Jul 30, 2026
@res-life

Copy link
Copy Markdown
Collaborator Author

build

1 similar comment
@res-life

Copy link
Copy Markdown
Collaborator Author

build

@res-life
res-life requested a review from a team July 30, 2026 07:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[test] reduce the timezone ITs in premerge

2 participants