Skip to content

Jax Transpiler path - #671

Merged
HenryNdubuaku merged 9 commits into
cactus-compute:v2from
Ayan9074:jaxtranspilerpath
May 29, 2026
Merged

HenryNdubuaku merged 9 commits into
cactus-compute:v2from
Ayan9074:jaxtranspilerpath

Conversation

@Ayan9074

Copy link
Copy Markdown
Contributor

Summary

Adds a generic JAX/Flax transpiler path that lets users provide their own JAX graph entrypoints and params, then emits Cactus graph bundles with mmap-bound FP16 weights.
This is intentionally a graph-bundle path accessible through the python api, the caller is in charge of tokenization, masks, sampling, prefill/decode boundaries, and any KV-cache frontend logic.

Changed:

  • Added JAX graph capture through jax.make_jaxpr.
  • Added support for capturing JAX params pytrees as named bound constants.
  • Added a generic JAX user graph bundle writer:
    • weights_manifest.json
    • components/manifest.json
    • components/<name>/graph.cactus
    • components/<name>/raw_ir.json
    • components/<name>/optimized_ir.json
  • Added JAX semantic rewrites for currently useful patterns, including RMSNorm, SiLU, inference BatchNorm, import cleanup, and dead-node cleanup.
  • Added focused tests for JAX capture, Flax-style modules, edge cases, bundle writing/loading, mmap weight binding, and runtime execution.
  • Documented the generic JAX bundle API in docs/cactus_transpiler.md

Not Included

  • This path writes FP16 mmap weights only. Quantized/CQ weight output is left for a follow-up or should be manually converted then included as the weights
  • This does not integrate with any cactus commands such as cactus transpile (is only available through python api), have tokenizer loading, or chat formatting.
  • This does not automatically infer model-specific graph boundaries. Users provide graph specs such as encoder, decoder_prefill, or decoder_step.
  • Cache-aware decode can be represented by exposing a separate decoder_step graph with cache tensors in its input/output contract, but the generic JAX path does not yet automatically create or manage KV caches.
  • Some larger fusions, especially attention, are left decomposed for now because JAX often emits attention tensors in BTHD layout, while the existing Cactus/PyTorch fused attention path expects BHSD layout, so supporting both formats cleanly needs more layout-aware fusion work. Attention fusion needed for automatic internal kv-cache lowering.

JAX/Flax are optional runtime dependencies for this path. They are imported lazily, and tests skip when unavailable, so this PR does not change package dependencies.

@HenryNdubuaku
HenryNdubuaku merged commit c9b08fb into cactus-compute:v2 May 29, 2026
0 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants