feat: A/B testing framework for campaign variants

[joelpeace48-cell]

Background & current state

Operators can't experiment on campaign presentation/parameters; there's no A/B testing. Optimizing conversion at scale requires controlled experiments.

Goal

Add an A/B testing framework: sticky variant assignment, exposure logging, and a results readout with significance.

Technical design

• Experiment definition (variants + traffic split). Sticky assignment per user (hash of user+experiment) so a user always sees the same variant; log exposures + outcomes (register/claim) to the indexer/analytics.
• Results endpoint computing conversion per variant + a basic significance test.

Edge cases

• Variant rebalancing mid-experiment → keep existing assignments sticky.
• Multiple concurrent experiments → independent assignment.
• Low sample → mark "not significant".
• Server + client consistency of assignment.

Task breakdown

• Experiment config + sticky assignment (server + client).
• Exposure/outcome logging.
• Results endpoint + significance.
• Tests: assignment stability + stats.

Acceptance criteria

• Users get a stable variant; results (conversion + significance) are measurable per variant.

Testing & verification

• Assignment stability test; significance computation test.

Out of scope

• Visual experiment builder.

Dependencies / related

• Uses analytics (NEW-099/100); pairs with feature flags (NEW-102).

Difficulty: hard · Effort: M–L

[Williams-1604]

please assign to me will impement in 3 phases.

Sticky assignment → deterministic hash of user_id + experiment_key into buckets, so a user gets the same variant across sessions/devices with zero stored state.
Exposure logging → log on exposure, not assignment — you only count someone in the experiment when they actually saw the variant, or your denominator is wrong.
Significance → guard against early peeking; the readout shouldn't flash "significant" at n=20. Fixed-horizon test (or sequential) so operators don't ship on noise.
Backend bucketing + frontend variant render is squarely my lane. I'll build it so adding the next experiment is a config change, not a code change.

[joelpeace48-cell]

@Williams-1604 you are assigned, PR will be thoroughly reviewed and all CI pipeline must pass

[grantfox-oss]

🦊 GrantFox — @Williams-1604 has been assigned to this issue as part of the Official Campaign campaign!

Next steps:

1. Open a Pull Request referencing this issue (e.g., Closes #624)
2. Your PR will be reviewed by the FinesseStudioLab maintainers

Good luck! Track your progress on GrantFox.