Introduce SIMD implementation for x86_64 by Kleidukos · Pull Request #38 · emilypi/Base16

Kleidukos · 2026-02-26T20:51:02Z

This PR adds a SIMD implementation very generously gifted by @Granahir2 for x86_64.

The base16.c interface provides a way to easily dispatch between scalar (Haskell), SSE 4.1 and AVX2.

HTML report is attached, feast your eyes:
base16.html

No bindings yet, both SSE and AVX variants were roughly tested manually.

Also make symbols to vectorized functions default to nullptr whenever no suitable implementation is found.

Kleidukos · 2026-02-26T20:51:29Z

cc @kozross I'll be tweaking the benchmark code to reflect the scalar & SIMD implementations but this looks promising so far.

This reverts commit 67cf73f.

Kleidukos · 2026-02-27T10:44:07Z

Warning: The package list for 'hackage.haskell.org' is -2147483648 days old.

I think I'm going to deactivate CI for 32-bit, this is becoming ridiculous.

Kleidukos · 2026-02-27T15:37:57Z

Looking at the benchmarks, it would appear that we seem to pay around 100ns overhead for using FFI, which can be observed for 25 elements:

benchmarking encode/25/base16 (scalar)
time                 48.18 ns   (47.95 ns .. 48.48 ns)

benchmarking encode/25/base16 (simd)
time                 182.3 ns   (171.7 ns .. 195.4 ns)

We break even at 100 elements, and after that we are undeniably faster:

benchmarking encode/100k/base16 (scalar)
time                 170.1 μs   (169.6 μs .. 170.7 μs)

benchmarking encode/100k/base16 (simd)
time                 35.80 μs   (35.44 μs .. 36.12 μs)

Similar behaviour is observed for decode.

vaibhavsagar · 2026-02-28T09:57:20Z

Drive-by comment: IIUC foreign import capi creates a C wrapper function with some associated overhead, and in this case I think using ccall would also work and be faster. As per the last paragraph of https://www.haskell.org/ghc/blog/20210709-capi-usage.html:

Note, however, that capi does incur a small (arguably negligible) runtime cost due to the to the C stub. It is justifiable to use ccall to avoid this runtime cost in cases where the foreign function is shipped with a package’s cbits, where the calling convention is clear.

hasufell · 2026-02-28T10:01:47Z

+    cbits/base16.c
+    cbits/avx_base16.c
+    cbits/sse_base16.c
+  cc-options: -O3 -std=c2x


-O3? Why? It's more aggressive and unsafe than O2. Is there evidence that there's actually a performance gain? If so, I'd rather figure out which optimization flag exactly causes performance improvements over O2.

Here, -O3 only applies to the vector implementations in cbits. They're simple processing loops without memory management. So the drawbacks of -O3 (aggressive inlining making profiling/debugging difficult, memory allocations being skipped if the optimizer thinks it's ok, code size growth due to unrolling...) are not very important IMO and -O3 might squeeze some more performance due to unrolling.

Though to be fair, -O2 probably would not make much of a difference since we're heavily guiding the compiler here anyway.

Kleidukos · 2026-02-28T12:05:31Z

Benchmarks with ccall instead of capi base16.html. We indeed gain some time.

hasufell · 2026-02-28T13:12:12Z

Benchmarks with ccall instead of capi base16.html. We indeed gain some time.

ccall is a dangerous footgun that can cause UB and memory bugs. We had those cases in unix on mac, because it doesn't follow header indirection like capi does.

I'd strongly advise against it.

Edit: if it's just our own cbit functions, then I guess it's alright.

The first vector iteration is made to overlap with the second one, in such a way that from the second interation onwards, the input pointer is suitably aligned.

emilypi

I really appreciate these contributions @Granahir2 and @Kleidukos! Just add tests that explicitly exercise the new codepaths and I'm happy.

emilypi · 2026-04-18T16:52:44Z

+#include "base16.h"
+
+
+__attribute__((target("avx2")))


glad we're starting here and not avx512 - thanks.

Kleidukos · 2026-04-18T17:05:16Z

@emilypi Ok! Maybe a review from @kozross would also be appreciated since he has valuable experience in this domain. :)

emilypi · 2026-04-18T18:17:04Z

+bool isSIMDAvailable() {
+
+#ifdef __x86_64__
+	if(__builtin_cpu_supports("sse4.1")) {
+		return true;
+	}
+#endif
+
+	return false;
+}


Does this function need to exist? it will not inline nicely in the Base16.hs calls and creates a branch at the call site. I'd rather see this at the callsite as a CPP directive than calling it direclty like this and branching on it. GCC and Clang support __SSE4_1__ directives and associated intrinsics.

@emilypi For clarification, do you mean that the Haskell code should check whether or not SIMD is available through CPP? We felt that a run-time check would increase the portability of distributed binaries instead, instead of having to distribute cpu-specific bindists.

instead of

encodeBase16' :: ByteString -> Base16 ByteString encodeBase16' = if c_isSIMDAvailable then encodeBase16SIMD else assertBase16 . encodeBase16_ {-# INLINE encodeBase16' #-}

why not write

encodeBase16' :: ByteString -> Base16 ByteString encodeBase16' = #ifdef __SSE4_1__ encodeBase16SIMD #else assertBase16 . encodeBase16_ #endif {-# INLINE encodeBase16' #-}

I know it's brittle, but the call isn't so interesting that i'd want to kick that to a ccall. You're already making a CPU-specific branch. The question is whether to determine that branch at compile time vs runtime. I don't see why i'd want to pay the runtime cost.

The disadvantage of this approach is that a binary compiled on a machine that doesn't have SSE4.1 won't ever execute the SIMD version, even if it runs on a machine that does have SIMD support.

With isSIMDAvailable, we sidestep this and can use one binary per architecture. Currently, the specific encodeBase16SIMD variant is picked at object load time (and the symbol is NULL if no variant is available). Note that __builtin_cpu_supports results are cached (according to GCC docs) so isSIMDAvailable is not probing the CPU at every call.

Another way to do it would be to package a scalar C version of the code as a fallback, then every call could go through encodeBase16SIMD (which would have to be renamed) and the branch can be eliminated. That way there's virtually no runtime dispatch cost.

I don't much Haskell; would there be a way to keep the branch, but cache its result somehow, knowing that it'll always resolve to the same value? Or maybe do something similar to the C side, where the specific implementation of encodeBase16' is chosen at load time? I guess it also comes down to how important portability should be here.

Note that __builtin_cpu_supports results are cached (according to GCC docs) so isSIMDAvailable is not probing the CPU at every call.

Noted, and I do agree that the ccall is a more general approach. My line of questioning here is more to figure out what kind of operation we need to focus on, because yes, while a more general approach would be good for more architectures, if the general shape of programs that we're looking at are largely 1-off hex encodings, it just adds extra unnecessary overhead. However, for programs that expect to hex encode a lot of things, successively, or along running program that expects to encode or decode things over a long time span, it would make sense. We also need to consider the size of inputs in all cases.

The benchmarks are a little bit of a lie due to the way GHC optimizes, and because we don't really know what the distribution of values is that are being hex encoded. I fuck them up with randomness for this reason, but also, we run the gamut from 24chars up through 1mb ish worth of chars, but realistically, I haven't seen anything in the wild >100 chars that needs to be hexed, so the utility is mixed to me (unlike, e.g., SIMD on base64, which is routinely used to encode gigabytes). On average, GHC will optimize for a hotpath if there are a lot of successive calls to a particular branch within some time span due to the way it applies its heuristics (see: -fcmm-static-pred), so it will "cache" informally. So that leaves us wtih two options really, for how we should model the user: long running programs vs. one-offs, given some distribution of inputs of different sizes. I think we can assume correct values in most cases for encodeBase16 and decodeBase16.

Do we have any indication of how the library is going to be used? Maybe this is a quesion for @Kleidukos - what's your take?

Another way to do it would be to package a scalar C version of the code as a fallback, then every call could go through encodeBase16SIMD (which would have to be renamed) and the branch can be eliminated. That way there's virtually no runtime dispatch cost.

This is how i've seen it done by others (e.g. Dan Lemire's simdjson, A Klomp's base64). It looks tedious to maintain tho. If anyone has a strong opinion I'll fold - i just want this in everyone's heads. If it's reasonable to run SIMD in every case and drop the generic support within some time frame, I'm on board with that too.

Do we have any indication of how the library is going to be used? Maybe this is a quesion for @Kleidukos - what's your take?

I can put out a call to users of base16 so that they can tell us a bit about their usages. And I can also check the list of users here: https://flora.pm/packages/@hackage/base16/dependents?page=1

Geez alot more people are using than the last time I checked in 😅

@emilypi here's my conclusion: The SIMD code paths should be guarded by cabal flags so that the vast majority of current users are not impacted by the overhead. However we still give end-users of the library control over the code path so that they may adapt the strategy to their workloads. How does that sound to you?

chreekat · 2026-05-01T10:43:19Z

-isValidBase16 (BS ptr len) =
-    accursedUnutterablePerformIO $ do
+isValidBase16 bs@(BS ptr len) =
+#if !defined(PURE_HASKELL) && defined(SIMD)


I believe in the text example you are copying, SIMD was pre-existing, and chose between different implementations inside the cbits. PURE_HASKELL removed the cbits altogether. The flags were making semantically different (although overlapping) choices.

In this patch, I don't see any case where the semantics are different -- they're just opposite sign of each other. SIMD == !PURE_HASKELL. Either you're using pure haskell, or you're using simd instructions. Maybe one flag suffices?

Here's my reasoning: I tried to abide by existing flags in the ecosystem (simd and pure-haskell) and they will also mean different things at some point in the future:

simd will mean using generated code that uses SIMD instructions, be it

Through C (like today) or

Through native support for SIMD (See the meta-issue)

pure-haskell means no using C dependencies, but not necessarily no SIMD (especially for Wasm).

Typically, WASM will have SIMD but Pure Haskell, whereas JavaScript will have No SIMD and Pure Haskell.

Now I understand that this makes this tri-state logic harder to reason about. I'm just trying to find a future-proof model for those flags so that they don't have to be reworked later on.

Maybe those are bad flags (not granular enough) and we could have:

c-simd: SIMD is done through FFI to C

ghc-simd SIMD is done through GHC primops

pure-haskell No C, compatible with ghc-simd but not c-simd

What do you think @chreekat?

Ah! Well, I don't understand the usage patterns well enough to have a useful opinion. I don't know the trade-offs of using simd, either. Having fewer knobs and good defaults would be the principles I would use to decide in this case. Which would probably result in the patch as-is.

In our specific case of base16, the "good default" is not to enable it, because the majority of users have small data to encode/decode. However we want to leave the possibility to enable SIMD to (transitive) users of this library when they know they have big data sets to process.

Co-authored-by: Bryan Richter <b@chreekat.net>

Sélène Corbineau and others added 5 commits February 19, 2026 15:56

Introduced vectorized helper functions for x86

7c4e3d5

No bindings yet, both SSE and AVX variants were roughly tested manually.

Fix accidental alignment requirement

190e155

Don't reference x86 symbols on all archs

d2992d1

Also make symbols to vectorized functions default to nullptr whenever no suitable implementation is found.

Add isSIMDAvailable() probing function

de32308

Write bindings

465d643

Kleidukos added 2 commits February 27, 2026 10:11

Disable GHC 9.4 on CI

67cf73f

Revert "Disable GHC 9.4 on CI"

5448a63

This reverts commit 67cf73f.

Kleidukos changed the title ~~Introduce SIMD implementation~~ Introduce SIMD implementation for x86_64 Feb 27, 2026

Kleidukos force-pushed the simd-c branch from 416f573 to 3a8b32c Compare February 27, 2026 09:39

Use GCC 15 as C compiler

6573bc1

Kleidukos force-pushed the simd-c branch from 3a8b32c to 6573bc1 Compare February 27, 2026 09:40

Kleidukos added 3 commits February 27, 2026 10:41

Use haskell-actions

3f235ca

Use c2x for the C compiler

e88b258

Use GHC 9.12 in CI too

00b7cce

Deactivate i386 job

435a148

Kleidukos force-pushed the simd-c branch from 46a0286 to 435a148 Compare February 27, 2026 10:44

Selene Corbineau and others added 2 commits February 27, 2026 11:50

Ensure __builtin_cpu_init only appears on x86_64

40a9a1f

Separate scalar & simd implementations in the benchmarks

3c51895

Dispatch isValidBase16 to SIMD implementation too

75e26b2

Kleidukos marked this pull request as ready for review February 27, 2026 15:48

hasufell reviewed Feb 28, 2026

View reviewed changes

Comment thread .github/workflows/haskell-ci.yml

Use ccall

37304b5

Correctly use scalar versions

a83b88c

Handle misaligned inputs better on decode/isValid

048bd83

The first vector iteration is made to overlap with the second one, in such a way that from the second interation onwards, the input pointer is suitably aligned.

emilypi reviewed Apr 18, 2026

View reviewed changes

Guard SIMD with cabal / CPP flags

6f7afb4

chreekat reviewed May 1, 2026

View reviewed changes

Update base16.cabal

f934dbb

Co-authored-by: Bryan Richter <b@chreekat.net>

Kleidukos force-pushed the simd-c branch from 6918c69 to 1c34ffb Compare May 1, 2026 23:56

Enable benchmarks, tests and simd flag in their own command line

3b39104

Kleidukos force-pushed the simd-c branch from 1c34ffb to 3b39104 Compare May 2, 2026 07:00

		#include "base16.h"


		__attribute__((target("avx2")))

Conversation

Kleidukos commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kleidukos commented Feb 26, 2026

Uh oh!

Kleidukos commented Feb 27, 2026

Uh oh!

Kleidukos commented Feb 27, 2026

Uh oh!

vaibhavsagar commented Feb 28, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Kleidukos commented Feb 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hasufell commented Feb 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

emilypi left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Kleidukos commented Apr 18, 2026

Uh oh!

emilypi Apr 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

emilypi Apr 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

emilypi Apr 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Kleidukos May 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Kleidukos commented Feb 26, 2026 •

edited

Loading

Kleidukos commented Feb 28, 2026 •

edited

Loading

hasufell commented Feb 28, 2026 •

edited

Loading

emilypi left a comment •

edited

Loading

emilypi Apr 18, 2026 •

edited

Loading

emilypi Apr 19, 2026 •

edited

Loading

emilypi Apr 19, 2026 •

edited

Loading

Kleidukos May 1, 2026 •

edited

Loading