Skip to content

[BUG] Canonicalize noncanonical BOOL8 values in murmurhash3_x86_32 #24039

Description

@PointKernel

Describe the bug

Follow-up from #23265 and review discussion #23265 (comment).

cuDF documents BOOL8 as 0 == false, else true, but cudf::hashing::murmurhash3_x86_32 can hash noncanonical stored bytes such as 2 or 255 as their raw values instead of canonicalizing them to 1. Logically equivalent true values can therefore produce different hashes.

Steps/Code to reproduce bug

  1. Construct a BOOL8 column directly from raw bytes containing {0, 1, 2, 255}. A raw column is required because cudf::test::fixed_width_column_wrapper<bool> canonicalizes values during construction.
  2. Hash the column with cudf::hashing::murmurhash3_x86_32.
  3. Compare the results for stored bytes 1, 2, and 255.

The existing boolean specialization converts the stored value directly to uint32_t, allowing noncanonical representations to remain distinct.

Expected behavior

Every nonzero BOOL8 value represents true and should be canonicalized to 1 before hashing. Stored bytes 1, 2, and 255 should produce identical hashes for the same seed.

Add explicit canonicalization (key ? 1 : 0) and regression coverage constructed from raw BOOL8 bytes.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinglibcudfAffects libcudf (C++/CUDA) code.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions