Skip to content

T5gemma2 model support - #13

Open
timkaye11 wants to merge 9 commits into
antflydb:mainfrom
timkaye11:t5gemma2_model
Open

T5gemma2 model support#13
timkaye11 wants to merge 9 commits into
antflydb:mainfrom
timkaye11:t5gemma2_model

Conversation

@timkaye11

Copy link
Copy Markdown
Contributor

Adds T5Gemma-2 Model Support for embedding and text generation

relies on ajroetker/hugot#4

Embeddings (pkg/termite/lib/embeddings/):

  • t5gemma2.go (+606 lines) - T5Gemma2 embedder with text & image embedding support
  • t5gemma2_stub.go (+97 lines) - Non-ONNX build stub
  • hidden_states.go (+50 lines) - Hidden states extraction for advanced use cases

Generation/Rewriting (pkg/termite/lib/seq2seq/):

  • t5gemma2.go (+588 lines) - T5Gemma2 generator with multimodal seq2seq support
  • t5gemma2_stub.go (+143 lines) - Non-ONNX build stub

Export Scripts

  • export_t5gemma2.py (+1,143 lines)
    • Self-contained ONNX export script with:
    • Encoder, decoder-init, decoder, and vision encoder export
  • export_model_to_registry.py (+211 lines) - Enhanced model registry export

Documentation & Tests**

  • e2e/t5gemma2_test.go (+380 lines) - E2E tests for embedder and generator

Merge main branch changes into t5gemma2 feature branch:
- Accept lazy model loading approach from main for e2e tests
- Keep T5Gemma-2 detection and loading in seq2seq_registry
- Maintain hugot fork replace directive
@dovinmu

dovinmu commented Feb 26, 2026

Copy link
Copy Markdown
Contributor

Claude's assessment of this branch for VLM (scoping for v2 of Honeycomb pipeline):

The branch is incomplete. The vision encoder works, but image embeddings aren't injected into the decoder — so image→caption doesn't actually work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants