Repository navigation
Conversation
Signed-off-by: Noah Cylich <noahcylich@gmail.com>
Signed-off-by: Noah Cylich <noahcylich@gmail.com>
Signed-off-by: Noah Cylich <noahcylich@gmail.com>
Signed-off-by: Noah Cylich <noahcylich@gmail.com>
# Conflicts: # .gitignore # cactus-engine/src/engine.h # cactus-engine/src/model.cpp # python/cactus/cli/run.py # python/cactus/cli/transpile.py
Replace the Pixel Cactus LFM and Qwen full-core c128 prefill rows with fresh rerun measurements, including the focused Qwen 4096 retry row. Keep the corrected Mac, Samsung, and Pixel full-core rows in the main aggregate CSVs, and remove redundant handoff/per-run artifacts from the tracked branch after verifying the Samsung per-run rows are already present in matrix_galaxy_s24_or_s25_full_core.csv. Add the Android cactus_llm_bench target wiring needed to build the standalone benchmark runner, including the mbedTLS archive links and expected output path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes the native transpiled LLM runtime path and optimizes chunked prefill for Cactus component bundles. It keeps
cactus runon the raw C++/FFI-backed chat binary, adds token-id benchmark support, and makes c128 chunked prefill work across Gemma, LFM, and Qwen-style component bundles.Key Changes
cactus_benchmark_tokensFFI entry point and wirescactus run/chatto support--input-ids,--max-new-tokens, and--result-json.lm_encoder_text_chunkanddecoder_prefill_chunk.position_idsandattention_mask..gitignoreto keep local benchmark outputs, generated Android test artifacts, and local third-party runtime drops out of status.Validation
python -m py_compilefor the touched CLI and transpiler modules.cactus runsmoke tests with 1024-token fixtures for Gemma, LFM, and Qwen.Risks / Notes
cactus runbehavior to require a local transpiled component bundle rather than falling back through older download/native-loader paths.max_new_tokens=0, so those timings should not be compared directly to TTFT-style rows that include decode handoff.