Repository navigation
fix: replace MD5 with SHA256 for FIPS compatibility - #3409
Conversation
🦋 Changeset detectedLatest commit: 96d1e0e The changes in this PR will be included in the next version bump. This PR includes changesets to release 5 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
@hercynium is attempting to deploy a commit to the HyperDX Team on Vercel. A member of the Team first needs to authorize it. |
|
Hi @hercynium, thanks for the pull request! Before we review code from a first-time contributor, a maintainer vouches for them, and you're not on our list yet. There's nothing else to file and this PR stays open — a maintainer will look at it, usually within a day or two, and vouch for you right here if it's a genuine attempt at the problem. To make that quick, check the PR description has:
More detail in our contributing guide. |
PR Review
4 finding(s): 🔴 0 critical · 🟠 0 major · 🔵 4 minor 1 posted as inline comment(s) on the changed lines. 3 listed below. Findings outside the changed lines3 minor
Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides. |
[Medium impact] The PR appears safe to merge; no new actionable issues were found. SummaryReplaces MD5 with SHA256 in app and CLI row lookups for FIPS-enabled ClickHouse.
No new actionable issues found. Tests were not run. Reviews (22) · Last reviewed commit: "Merge branch 'main' into bugfix/replace-..." · Reviewed by Greptile |
Deep Review✅ No critical issues found. This PR is a focused, low-risk swap of MD5 → SHA256 in client-side row-lookup 🟡 P2 -- recommended
🔵 P3 nitpicks (3)
Reviewers (7): security, performance, agent-native, previous-comments, project-standards, learnings-researcher, kieran-typescript. Testing gaps:
No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond |
|
TBH... I've kind of vibe-coded this and I don't like what I see since the initial commit. I'm going to go back and re-work some things manually. Difficulty is that I don't know TypeScript, but it doesn't seem all that different from other langs I do know. |
201f1e3 to
0eeb80a
Compare
| let prefix = ''; | ||
| let n = 0; | ||
| for (const ch of value) { | ||
| if (n++ === limit) break; |
There was a problem hiding this comment.
🔵 minor — New shared helper leftUtf8Prefix has no unit test of its own in common-utils
leftUtf8Prefix is exported for any caller but is only tested indirectly through the app and CLI row-WHERE hash snapshots. Add direct tests in common-utils __tests__ for: limit larger than the string, exact limit, surrogate pairs (e.g. '😀'.repeat(3) with limit 2 gives 2 emoji, not 1), and limit 0.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
There was a problem hiding this comment.
I will add these tests if deemed necessary, but I feel this PR is already touching more than initially planned, all to satisfy the bot.
| whereKind: Sha256WhereKind, | ||
| hash: string, | ||
| ): string { | ||
| if (whereKind === 'json') { |
There was a problem hiding this comment.
🔵 minor — JSON test cases say they match ClickHouse, but they hash raw literals instead of what toString(<JSON column>) actually returns
The expected hashes come from SHA256('<literal>'), so the tests only show that crypto-js and ClickHouse hash the same bytes the same way. The real risk is a serialization mismatch. toString() on a JSON column emits compact JSON. Per the comment at packages/app/src/components/DBRowTable.tsx:1848, it also escapes / as \/. So {"key": "value"} (with a space) and the 'actual log line' (with an unescaped github.com/...) would not match a stored row. Fix: build the input the way DBRowTable does (JSON.stringify(obj).replace(/\//g, '\\/')), and take the expected hash from lower(hex(SHA256(toString(Body)))) on a row inserted into a JSON column. Otherwise, drop the 'matches ClickHouse' claim and the clickHouse field.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
There was a problem hiding this comment.
I'm pretty sure the tests are now more robust than they were before. This minor finding is fixable, but seems like it should go into another PR.
This patch is to resolve Issue ClickHouse#3390 - when running HyperDX against ClickHouse in FIPS mode the MD5 function isn't available, so viewing some logs fails, because MD5 is not FIPS-compliant. This patch was generated using Cursor, running Composer 2.5 Fast. All relevant tests still pass. Co-authored-by: Cursor <cursoragent@cursor.com>
Hoist the ClickHouse leftUTF8-matching helper so app and CLI row lookups stay in sync and cannot drift silently. Co-authored-by: Cursor <cursoragent@cursor.com>
Match the app row-WHERE builder so null JSON columns are not hashed as an empty string. Co-authored-by: Cursor <cursoragent@cursor.com>
Cover the uniform null guard for a non-JSON column type alongside JSON. Co-authored-by: Cursor <cursoragent@cursor.com>
Use SHA256_TEST_CASES with it.each in app and CLI suites so ClickHouse golden hashes stay aligned and duplicated constants are removed. Co-authored-by: Cursor <cursoragent@cursor.com>
dfb4b59 to
dd18229
Compare
|
/vouch @hercynium FIPS fix for #3390 |
Triggered by [comment](#3409 (comment)) from @MikeShi42. Vouch: @hercynium
|
Hello! I'm hoping to get eyes on this PR, and the linked issue. I've been successfully running a patched version of HyperDX against a FIPS build of ClickHouse for the last two weeks, and I'd like to work my PR towards getting merged before something new conflicts requiring re-work of the fix. ☺ I'm open to all feedback. Thanks in advance! |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
@hercynium PR looks good. Could you fix the lint errors then we can merge the PR? Thanks |
|
Thanks, @wrn14897! I fixed the linter issues in the |
nvm... I found one running the linter locally. Fixed. ☺ |
| }, | ||
| '600 emoji code points': { | ||
| clickHouse: "lower(hex(SHA256(leftUTF8(repeat('😀', 600), 1000))))", | ||
| columnMeta: [{ name: 'description', type: 'String' }], |
There was a problem hiding this comment.
🔵 minor — The SHA256 test fixtures are copied word-for-word between the app and CLI tests
Sha256WhereKind, Sha256TestCase, SHA256_TEST_CASES and sha256RowWhere are restated in packages/cli/src/__tests__/rowWhere.test.ts:7-63, so the verified hashes now have to be kept in sync by hand. Move the fixture table into a shared module next to leftUtf8Prefix in common-utils and import it from both tests. Also build the expected SQL with LEFT_UTF8_HASH_CODE_POINTS instead of the hardcoded 1000.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
There was a problem hiding this comment.
As mentioned before, this change already touches more files than I'd like so addressing this can be a different PR.
Summary
This patch is to resolve Issue #3390 - when running HyperDX against ClickHouse in FIPS mode the MD5 function isn't available, so viewing some logs fails, because MD5 is not FIPS-compliant.
All uses of MD5 were replaced with equivalent uses of SHA256. This includes in the tests and in comments.
This patch was generated using Cursor, running Composer 2.5 Fast. All relevant tests still pass.
Screenshots or video
NA - no UI changes.
How to reproduce
Starting point:
On
main:On this branch:
References