[stikves]

This PR adds the engine runtime for batched LLM bundles that take an additive attention mask, and
uses llm-benchmark as the first consumer. The scheduler and server that would drive continuous
batching are not part of this PR.

It builds on the batched export from #334, the KV-state fix from #354 and the 8-bit KV-cache support
from #356, all now on main.

What this adds

• CoreAISequentialEngine
  • decodeBatch runs one forward step over N rows and returns each row's last-token logits.
    When the rows sit at different cache positions (decode only, and only on graphs with an
    attn_mask input), every row writes its new token at the longest row's slot, and the shorter
    rows are copied back to their own position afterward.
  • lockstepBatchedGenerate generates from N equal-length prompts together, with per-row sampling
    configurations. The benchmark uses it.
  • prefillRow prefills one prompt in a batch-1 scratch session and copies its KV prefix into one
    row of a batched cache. When the bundle has a prefill graph, the prompt runs through that chunked
    executable, so the decode executable's heap is not resized for each new row.
  • The engine accepts a third attn_mask input and fills a causal mask for each step.
  • The in-flight generation token, the prefix-hit count, the logits buffer and the input handler
    now live on GenerationSessionState, so separate sessions no longer cancel each other or share
    buffers. The engine's lastPrefixHitCount reports the most recently driven session.
• Handlers
  • BatchedInputBuilder builds the batched token IDs, position IDs and additive mask, including
    the per-row variants for ragged decode.
  • GrowingNDArrayState can hold a batch of rows. copyRegion moves one row's slice between
    sequence slots, and copyRowPrefix copies a batch-1 prefix into one row. Both copy raw bytes, so they
    work for 8-bit KV caches as well as fp16, bf16 and fp32. The sequence dimension
    is now always taken as the second-to-last dimension, so a dynamic batch dimension is not
    mistaken for it.
  • The internal BatchedDecodeStep protocol describes the batched step and row admission, with a
    usage example.
• BatchedSampler samples one token per row, each with its own configuration. The single-row
  path in SequentialIterator now calls it with one row.
• Bundle: LanguageConfig and LanguageModelBundle read max_batch_size, which defaults to 1
  when absent. ModelBundle accepts metadata_version 0.3 as well as 0.2.
• llm-benchmark: new --batch-size and --inference-engine-variant flags. Batches larger
  than 1 need --inference-engine-variant coreai-sequential, and a batch larger than the bundle's
  max_batch_size is rejected.

Scope and behavior

• Existing single-sequence bundles run as before. Without --batch-size, the benchmark reads the
  batch size from the graph's fixed leading dimension, which is 1 for these bundles and for
  dynamic-batch graphs.
• prefillRow, maxBatchCapacity and the ragged path in decodeBatch are there for a future
  scheduler. Nothing in this PR calls them.

Public API

• The only new public declaration is InputLayout.attnMaskName, next to the existing input-name fields.
• Everything else this PR adds is package (read by llm-benchmark) or internal: the batched engine
  methods, BatchedSampler, BatchedDecodeStep, the per-row KV copies, and maxBatchSize.
• The public LanguageConfig initializer is unchanged. max_batch_size is read only when decoding
  bundle metadata.

Validation

• The package and tests build cleanly, and swift format lint --strict is clean on the changed files.
• The full Swift test suite passes. A few existing timing tests (Top-K sampler latency, profile-span
  min/max, NDArray allocation cost) can fail when the whole suite runs in parallel; they pass on their
  own, and this PR does not touch the code they time.
• New tests cover the batched input builder (including ragged decode), the batched sampler, KV-cache
  shape resolution, the per-row copies (fp16, fp32 and int8, with leading and non-leading batch dims), per-session generation state, the additive causal mask,
  and parsing of metadata_version 0.3 and max_batch_size.
• Greedy decoding on Qwen3-0.6B (64 tokens) gives byte-identical output to main on the default,
  coreai-sequential and coreai-pipelined engines, for:
  • a fresh dynamic export from main,
  • an older dynamic bundle,
  • a static-shape bundle, on the default engine only. The other two engines do not accept static
    bundles on main either.
• A dynamic-batch bundle exported from main with --attention-mask batched --dynamic-batch-size 4
  (metadata 0.3) runs through llm-benchmark at about 99 tok/s at batch 1 and about 220 tok/s
  aggregate at batch 4 (debug build, M2 Max).