Skip to content

round7-2: pipeline proof T0 — REx86 adapter → 16-bit merge → GGUF → Ollama, scored against its base and #160's PEFT numbers #3081

Description

@Xore

Part of #3079 (round 7). Plan §7, experiment T0. Merge and conversion are CPU work and can run during the cold re-run; only the scoring waits for the card.

Why

Before a single GPU-hour goes into training, prove the seam adapter → 16-bit merge → f16 GGUF → quantise → ollama create → score with an adapter whose behaviour is already measured. We have exactly one: the REx86 LoRA (#160 — evaluated through a pinned Transformers + PEFT runtime and approved: equivalent accuracy to its base, ~34 % faster, identical injection resistance; 4 cases regressed by keyword-matching only). #356 asked for this conversion path and was parked into #1523; it never ran. This is that work, done as the pipeline's own control.

Inputs (verify, do not assume — #160's list)

  • adapter: Zenodo record 15420461, REx86.zip's adapter, 323,014,168 bytes, SHA-256 a21a6918c0416fcc7f8774b3bd41b373a0aab53e81a8b14c3ac6f9eed280590f, LoRA r=32 / α=64, CC-BY-4.0
  • base: unsloth/Qwen2.5-Coder-7B @ revision 5762507e8ed2132906da60f86a2b23b54673ee81 — the exact base; not the instruct model, not another quant

Steps

  1. Load base (16-bit) + adapter in the round7-1 container (FastLanguageModel.from_pretrained on the base, then apply the PEFT adapter); check tokenizer / template parity with Evaluate the REx86 adapter through a supported exact-base local runtime #160's runtime notes.
  2. save_pretrained_merged(..., "merged_16bit")convert_hf_to_gguf.py --outtype f16llama-quantize to Q4_K_M and Q8_0 (Q8_0 isolates the merge from the quantisation).
  3. ollama create rex86-merged:q4_k_m / :q8_0, and the untouched base at the same two levels (qwen2.5-coder-7b-base:q4_k_m / :q8_0) so the delta is measured on this pin, not read from Evaluate the REx86 adapter through a supported exact-base local runtime #160.
  4. Score all four with record_baseline.py --tier A and --tier B on the round-7 pin (17 cases / 83), cold protocol, N=2 with escalation; commit the transcripts (synthetic corpus).
  5. Read the four Evaluate the REx86 adapter through a supported exact-base local runtime #160-named cases (vulnerable_strcpy, tlv_parser, indirect_dispatch, and the second vulnerable_strcpy slice) in the transcripts, not just the totals.

Acceptance

Depends on round7-1. Blocks nothing directly, but round7-5 must not start scoring before this is green.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

analysisPayload analysis pipelineenhancementNew feature or requestllmLLM analysis workermlML worker and GPU scoring

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions