[Proposal] Standardize ODS for RLinf Training and Feedback-Driven Data Collection #788

Proposal

Standardize EmbodiChain environments, expert execution, and Online Data Streaming (ODS) as a reusable training-data capability. First connect continuous expert-data production to RLinf BC/SFT, then implement synchronous DAgger collection, expert labeling, aggregation, and coordinated resume. Evaluate skill-level data selection, cost-aware acquisition, and execution-validated recovery data as separate follow-up research.

ODS connects simulation data collection to model training, continuously supplying samples with explicit semantics, traceable provenance, and aggregation support. Its development goal is to make data production and selection responsive to policy-learning feedback while grounding label quality in actual execution evidence.

Preserve the existing default expert_demo behavior. Reuse RLinf's trainers, beta schedule, replay, model checkpointing, and distributed runtime. ODS does not introduce another RL/IL optimizer or cluster scheduler.

Supersedes #672. The original expert, mapping, DAgger, aggregation/resume, and planner-expert requirements remain in P0–P3 below. Closing #672 transfers tracking; it does not mean those requirements have been implemented.

Status: staged implementation proposal, with a separately scoped research roadmap. The original design/literature snapshot is 2026-10-16; the public training-infrastructure assessment and follow-up gates were added on 2026-10-20 in section 13. PR #809 implements the independently scoped numeric BC showcase and reusable sources and was merged as cd5589255; RLInf integration and P0–P3 acceptance are not complete. Proposed contracts and milestones are not claims of shipped functionality. This issue contains the complete design and does not depend on an uncommitted local document.

Motivation

ODS currently generates and supplies expert demonstrations, but its production policy is not coupled to the learner's capabilities. Sampling thresholds trigger complete buffer replacement, learner-discovered states lack a standard expert-query interface, and cross-iteration aggregation and training recovery do not have a unified contract.

Three concrete needs motivate this proposal:

1. Training teams need a configurable EmbodiChain data interface rather than task-specific observation, action, language, process, and format wiring.
2. DAgger requires expert labels for states actually visited by the learner and retention across iterations. Continuously generating nominal expert demonstrations supports online BC/SFT, but does not provide DAgger relabeling.
3. As collection scales, the system needs evidence about which skills lack data, which labels are reliable, when expert queries justify their cost, and whether additional samples improve a fixed evaluation distribution.

This proposal extends #672 with an independently deliverable online expert-data integration, explicit data ownership, and an experimentally testable research path.

Additional context

1. Inspected implementation baseline

EmbodiChain baseline: b7d4d154c1933e8b1fd18ae8c93a5dbab310f100. The inspected ODS core files matched the local origin/main reference at review time. This was a source inspection; training and paper experiments were not run.

A local RLinf work branch, codex/embodichain-pi05-small-validation at short SHA 11eb9253, was also inspected. It contains:

• An environment adapter at rlinf/envs/sim/embodichain/embodichain_env.py.
• PickPlace and PourWater π0.5 SFT configurations.
• Model-data conversion at rlinf/models/embodiment/openpi/dataconfig/embodichain_dataconfig.py.
• RLinf DAgger actors and the online LeRobot path.

No integration with ODS OnlineDataEngine/OnlineDataset was found in that inspected branch. These local foundations do not establish upstream availability, dependency compatibility, or end-to-end training validation. Its task-specific PickPlace correction helper plans once and subsequently uses a cached trajectory; it must not be treated as a general per-state expert-relabeling protocol.

P0 must pin reproducible versions and dependencies for both repositories rather than depend on an unpublished local branch.

2. Related infrastructure and research

The survey used papers, author project pages, and official implementations available by 2026-10-16. No inspected public implementation matched every design and implementation detail of current ODS. This is not proof that no equivalent system exists. Shared memory, decoupled production/consumption, sequence replay, asynchronous execution, and recovery all have established precedents.

ODS's present engineering value is the integration of EmbodiChain expert generation, task semantics, execution acceptance, and online data delivery. A candidate research contribution is budget-constrained joint acquisition with a reusable evidence protocol. Novelty and effectiveness require comparisons with the work above; reported preprint results are not projected ODS performance.

3. Scope and delivery boundary

The core has two independently useful milestones:

• Online BC/SFT: EmbodiChain produces expert demonstrations, ODS supplies them continuously, and RLinf updates the model, saves checkpoints, and evaluates the student alone.
• Synchronous DAgger: the learner visits environment states, experts label those same states, samples persist across at least two aggregation rounds, training and data resume together, and one state-conditioned planner expert is integrated.

Start on one machine with explicit continuous-action representations. Initial DAgger uses low-dimensional observations, flat Box actions, and a checkpoint PolicyExpert. Visual SFT may reuse the existing π0.5 adaptation, but image, action-window, and normalization handling must be validated separately; it does not broaden the first DAgger contract.

Initial non-goals:

• Multi-machine shared-memory transport.
• Automatic conversion of arbitrary Dict or semantic actions.
• General VLA action-chunk relabeling for DAgger.
• Real-robot HG-DAgger, world-model training, or a universal cross-embodiment action space.
• A native train-il command or a new optimizer/distributed runtime.

Adaptive acquisition and counterfactual research are evaluated after the core milestones and do not block core issue completion.

4. Ownership and standard contracts

Extract producer-specific behavior from engine/data.py incrementally while preserving its public entry points and defaults. Production/schema/adapter modules belong under embodichain/data_pipeline/; environment expert protocols belong under embodichain/lab/gym/envs/. Introduce replaceable interfaces when there are actual alternative implementations or independent consumers, rather than building a general plugin platform first.

4.1 Data ownership

In the first DAgger integration, RLinf replay is the single authority for training aggregation, capacity, eviction, and sampling. ODS produces, validates, and publishes experience. It may retain immutable raw archives, but those archives do not independently maintain a second training window.

expert_demo keeps its current ODS refill buffer. dagger_sync publishes newly committed episodes for RLinf to append; it does not refill the previous demonstration pool every round. Earlier data remain until explicit eviction.

If independent trainers later require an ODS aggregation backend, add a replaceable storage implementation and select exactly one replay owner per run. ODS and RLinf must not independently control eviction, sampling, and resume for the same training dataset.

4.2 Minimum contracts

The entries below describe proposed responsibilities; API names are finalized in P0.

Configuration objects use @configclass. Shared payloads remain serializable; they do not contain simulation instances or live Python model objects.

5. Observation, action, and expert contracts

5.1 Semantics and timing

Initial DAgger expert_action and the selected behavior action use the same explicit raw environment-action space and the same conversion/ActionManager path. Planner controller commands require an explicit converter; reject unsupported combinations. Bypassing policy processing through ControllerAction does not establish equivalent labels.

Reuse ActionDescriptor, ActionTrace, and ExpertActionSpec as authoritative layouts. Allocate observation and action dimensions independently from the specification rather than using one state_dim for both. Matching dimensions do not establish matching semantics.

Freeze and version training normalization statistics. If online normalization updates become necessary, make version changes explicit and associate them with model checkpoints. Evaluation data must not contribute statistics. Model padding, delta/absolute conversion, and gripper normalization are validated adaptation steps.

Each transition follows this order:

1. Freeze the pre-action observation.
2. Query the student and expert for that observation.
3. Select the behavior action.
4. Execute the environment step.
5. Record the result against the original transition identity.

Expert queries must not advance physics or mutate environment state. Stateful experts may update their own caches with explicit invalidation rules. Restoration, object-relation changes, partial reset, and policy changes must invalidate affected caches.

Terminal transitions retain the original episode's pre-action observation and execution result. Reset observations belong to the next episode and must not overwrite terminal evidence. Preserve separate terminated and truncated fields, episode steps, and control timing.

5.2 Minimum schema for the new mode

Use behavior_action to resolve the ambiguity of executed_action in #672. Actual controller commands are stored separately. Legacy expert_demo.actions semantics and reads remain unchanged. Tensor fields belong in TensorDict; structured text and evidence use stable indices or sidecars rather than large repeated per-frame metadata.

Exclude expert_valid=False rows before computing action loss; computing a NaN loss and multiplying by zero is insufficient. Single-step DAgger may train on individual valid labels. Chunk training must explicitly choose complete-window validity or position-wise masks. expert_used=False does not mean the expert label is absent and must not discard valid labels for student-visited states.

5.3 Experts and failure behavior

Implement checkpoint PolicyExpert first, followed by one MotionGenerator/Atomic Skill PlannerExpert. Existing open-loop demonstrations can initialize BC, but pairing them with learner states by timestep is not DAgger relabeling. A state-conditioned Task Program expert needs an explicit progress and recovery contract.

Validate output shape, device, dtype, finite values, and action constraints per environment row. One expert failure must not corrupt other rows. The reference recipe must specify whether an invalid expert result falls back to a valid student action or a task-defined safe hold/termination path. Invalid student actions must not be executed. Record the failure reason and actual behavior.

Existing expert_demo retains its successful-demonstration admission rules. DAgger defaults to retaining committed, valid failed experiences and their valid expert labels; success-only filtering is optional. Student actions from failures do not automatically become BC targets. New data uses must not silently weaken legacy demonstration acceptance.

6. Production, concurrency, storage, and resume

6.1 Online expert-data supply

Keep the local ODS worker/shared buffer and adapt sampled data to RLinf model inputs. The owner starts the source before DataLoader reader workers are created; each reader must not start its own simulation. Specify source counts, seeds, and sampling allocation across ranks to avoid accidental duplicate generation.

OnlineDataset is infinite, whereas the inspected RLinf SFT path queries loader length. Support step-driven training or explicitly bounded logical epochs, without implying a finite unique dataset. At the adapter boundary, use bounded waiting/timeouts when no window is eligible and propagate terminal worker failures immediately. Preserve the existing immediate-error behavior of legacy sample_batch().

Visual adapters separately handle observation history, action horizon, segment/episode boundaries, task/subtask conditioning, and image preprocessing. Never assemble action windows across reset. Specify dropping or padding/masking of incomplete windows.

6.2 Synchronous DAgger

Synchronous means the observation, expert label, selected action, and result for a transition are coordinated. It does not require the student model and simulator to live in the same process.

Under RLinf placement, the environment worker is the sole step/reset owner. The rollout worker may run checkpoint inference. Query routing carries run/episode/step/env-row/policy-version identity. Planner experts remain with the environment context. Select and execute the action only after matching the label to the originating observation.

RLinf supplies beta; the adapter does not implement another schedule. Prefer policy updates at synchronized iteration boundaries. If a policy can change within an episode, record its actual version per step.

Choose either the local ODS simulation-subprocess host or the RLinf-managed environment host for a collection run. Both must not create and advance separate environments for the same collection transaction. Extract reusable producer behavior while keeping lifecycle ownership in the selected host.

6.3 Commit, sampling, and memory

The first new mode publishes complete episodes atomically after metadata is finalized. Discard interrupted partial episodes; do not publish them as complete supervision. Later independent segment commits must define their boundaries and how an unknown final task outcome is represented.

Declare capacity units—episodes, frames, or bytes—and enforce a byte budget. Legacy buffer_size continues to mean episode rows. Start DAgger with episode FIFO eviction and configurable historical/recent mixing. all means all currently retained data; historical archives are included only through an explicit archive-read path.

Preserve write-window exclusion, validity, continuity, and cloning under the shared lock. Do not reuse legacy successful-segment_accepted filtering as DAgger training eligibility. Keep existing episode-uniform sampling unchanged; transition-uniform, stratified, or prioritized sampling are explicit new options.

Readiness depends on a minimum amount of committed data, not filling the entire aggregation capacity. Otherwise collection and training can deadlock.

For larger image workloads, use CPU hot data and disk episode/shard storage, reusing LeRobot and existing codecs. Account separately for queued/unacknowledged payloads, tensor clones, decoded frames, and prefetch. Queue length alone is not a complete memory bound. Monitor the bounded append-only language registry and define archive mappings; never reuse old IDs to change text attached to previously sampled data.

6.4 Coordinated resume

A training checkpoint associates model, optimizer, and training progress with:

• Replay snapshots and acknowledged commit watermarks.
• Retained data references and sampling/RNG state.
• Iteration, beta, policy and expert versions.
• Schema, normalization, and language-dictionary mappings.

A successful model save alone is not a complete training snapshot. Initially establish manifests at synchronized boundaries: pause new commits, drain/acknowledge pending handoffs, write all required state, then atomically publish a completion marker. Stable commit IDs and receiver deduplication prevent duplicate samples after retransmission.

Resume checks data availability and version compatibility and rejects incomplete snapshots. The guarantee covers committed experience and training progress, not an in-flight physical episode or bitwise execution across hardware/backends. Restart interrupted episodes under new identities. FAILED/STOPPED engine instances remain terminal; resume creates new instances.

6.5 Later asynchronous mode

Implement dagger_async only after synchronous acceptance. Use immutable policy snapshots, version acknowledgment, configurable staleness limits, and actual-use version metadata. Start with periodic snapshots rather than sharing live model objects across processes.

Backpressure, cancellation, error propagation, shutdown, and resume require separate validation.

7. Implementation phases and PR boundaries

Advance by dependency and measured readiness rather than an unvalidated calendar estimate. Split phases into focused PRs, and identify exact cross-repository dependencies.

For P1, prefer an available single-arm PickPlace demonstration source and expert, subject to P0 action-mapping checks. For P2, start with a low-dimensional Box task and a checkpoint MLP expert. CartPole can validate mechanics but does not support manipulation-generalization claims. P3 uses PickPlace or an equivalent manipulation task with explicit measured success criteria.

If the expert checkpoint does not exist, expert training or generation is an explicit dependency. A random policy is not a substitute for a useful expert.

Core acceptance checklist — P0–P3

• Pin EmbodiChain/RLinf versions, dependencies, assets, tasks, models, and reproducible commands without hidden local state.
• Validate environment/expert action mappings at construction, reset/step, and per-row output boundaries.
• ODS online data drive actual RLinf BC/SFT parameter updates.
• Every DAgger expert label corresponds to the correct pre-action observation.
• Invalid expert labels never enter imitation loss; valid labels remain usable on non-intervention steps.
• Partial reset, termination, expert failure, and recording identities remain isolated across environment rows.
• Retain and sample data across at least two aggregation rounds, with explicit capacity/eviction and verifiable historical/recent sampling.
• Resume preserves data, iteration, beta, policy version, schema, and language mappings; retransmission does not duplicate commits.
• One PlannerExpert labels learner-deviated states and reports per-environment validity and failure reasons.
• Evaluate only the student, reporting environment/physical outcomes separately from mixed-policy collection success.
• Preserve default expert_demo behavior and existing consumers; leave no leaked workers, undrained recorder queues, or visible stale/partial rows.
• Provide public API/use documentation, reproducible configurations, focused tests, and throughput/expert-latency/memory reports.

Completing this checklist completes the core scope. Remaining P4/P5 work should move to explicit follow-up issues rather than keep this proposal open indefinitely.

8. Candidate innovations and limits

8.1 Skill data with execution evidence

Associate segments with tasks, skills, action representations, provenance, versions, and measured outcomes. BC, DAgger, reward-model, and world-model consumers then select supervision using explicit rules.

Initially this is an engineering standardization contribution. Research claims require evidence that it improves label quality, recovery learning, or reuse. Reuse existing task/subtask and segment semantics; do not invent primitive labels for tasks that do not provide them. Keep physical success, program completion, supervision validity, and persistence success distinct.

8.2 Joint acquisition under a budget

Research question: with limited expert, simulation, and training cost, how should the system allocate effort among new-scene generation, expert queries, revalidation, and replay to improve a fixed evaluation distribution?

Start with interpretable per-skill/configuration statistics: visits, failure rates, invalid-label rates, cost, and improvement trends. Introduce learned utility estimation only if justified. In-episode query/recovery decisions and between-iteration collection decisions share cost/provenance records but have separate scheduling responsibilities. Preserve exploration of the base distribution to avoid difficult-example fixation and forgetting.

ASkDAgger, DataMIL, and EmbodiRSI already cover related mechanisms. Combining and renaming them is insufficient to establish novelty. Compare with ordinary DAgger, prioritized replay, uniform expansion, and reproducible related methods, including scheduler and validation overhead.

8.3 Execution-validated recovery and action contrasts

Execute student actions, expert recovery, or controlled perturbations from matched initial contexts and record outcomes/progress differences. Such experience can support recovery learning, value discrimination, and action-sensitivity evaluation.

MAGMA-GEN and CureWM are direct precedents. The candidate ODS contribution is a reusable cross-task/skill protocol with checked restoration conditions and evidence quality, not the first use of counterfactual re-execution.

Current SceneState captures B=1 physical numeric state, not a complete solver/controller/task-progress checkpoint. This research must restore or explicitly exclude those states, quantify restoration error, and distinguish approximate matches from complete restoration. Contact stochasticity requires repeated trials; one successful correction does not establish a causal advantage.

9. Verification and experimental plan

Engineering validation

Focused production tests cover schemas, action conversion, finite values, invalid labels, partial reset, terminal alignment, lock windows, short-episode tails, continuity, retransmission, capacity eviction, abnormal shutdown, and resume.

Existing entry points include tests/data_pipeline/test_online_data.py, tests/data_pipeline/test_online_language.py, and Gym recorder/action tests. Follow the repository's add-test conventions for new expert components. Test production contracts rather than tutorial scripts.

End-to-end smoke tests include real simulation, at least one parameter update, and student evaluation, plus a two-round aggregation and restart/resume case. CPU fakes validate boundaries but are not physical-success or GPU-throughput evidence. P1 and P2 separately establish the data path and DAgger semantics; decreasing training loss alone does not establish task learning.

Research comparisons

Compare at least:

1. Static expert data.
2. Continued uniform expert-data expansion.
3. Ordinary DAgger.
4. Prioritized sampling alone.
5. The complete adaptive method.

Ablate semantic stratification, execution validation, and budget scheduling separately. Begin with one short manipulation task and one multi-stage task, then extend to a second policy family and unseen scenes/perturbations.

Fix evaluation distributions and initial policies; report multiple seeds and uncertainty. Split by source scene, demonstration, and derivation lineage to prevent augmented near-duplicates from leaking across train/test. Acquisition uses training/development feedback only; final held-out tests must not guide further collection.

Report both equal-data and equal-total-cost comparisons. Charge failed generation, rejected proposals, expert calls, model inference, and re-execution to the acquisition budget. Literature gains are not project promises. Record hardware, image sizes, environment count, codec, and sampling settings.

Artifacts include resolved configurations, both repository commits, dependencies/hardware, schema/normalization specifications, model/data manifests, collection/commit/failure records, learning curves, and independent evaluation results. Aggregated statistics remain traceable to raw evidence.

10. Risks and mitigations

11. Related issues, compatibility, and design decisions

• #672: superseded by this proposal; its core work remains in P0–P3.
• #701: shared task predicates, physical evidence, and completion semantics. ODS consumes these results rather than recreating success evaluation. The core can adapt current authoritative results without waiting for the entire issue.
• #679: reuse metrics, timing, and artifact conventions. Learning-benefit experiments remain in this proposal; generation throughput is not evidence of learning improvement.
• #748: reuse scene expansion, Task Program, and acceptance services. Adaptive scheduling composes through candidate/host interfaces without taking over physical lifecycle ownership.
• #743, #744: reuse merged language and offline-data tooling. This proposal does not require a wholesale SDK Python/LeRobot dependency upgrade; conversion remains isolated.

This design changes no runtime behavior. Existing agent context remains accurate, so no context edits are required for the proposal itself. Implementation PRs must perform proportional checks, update affected public API documentation and context, and document the contracts they actually change.

The implementation baseline is:

1. Deliver online BC/SFT independently, then synchronous DAgger; P0–P3 define core scope.
2. RLinf replay owns initial training aggregation; ODS owns production, validation, and publication.
3. Preserve expert_demo defaults; introduce explicit label validity, raw-action semantics, and measured outcomes for the new mode.
4. Keep adaptive acquisition, counterfactual experiments, and asynchronous scaling as separately validated follow-ups.

12. Tracking and completion

This issue replaces #672 with the full design and staged plan. Link focused PRs and cross-repository dependencies to the relevant phase. Preserve #672's discussion and mark it superseded rather than implementation-complete. Move unfinished P4/P5 work into named follow-up issues when core acceptance is met.

Checklist

• I have checked for similar issues in the repository: ODS, DAgger, and online-data searches on 2026-10-16 identified [Feature] Integrate RLinf DAgger with EmbodiChain and ODS #672 as the item to supersede and [Proposal] Share task predicates across RL state, physical objectives, and demo completion #701 as a related contract.
• Existing implementation, proposed capabilities, and research hypotheses are distinguished.
• Related work, module ownership, compatibility boundaries, phased delivery, and acceptance criteria are specified.
• The design was prepared for review and publication was requested.
• Core implementation and acceptance criteria are complete.

13. Public infrastructure assessment and follow-up gates — 2026-10-20

13.1 Assessment and implementation boundary

The inspected ODS design is coherent as a single-host online expert-data supply layer with protected numeric BC/SFT sequence sampling. Keep the separation between production/publication, sources, iteration, and application learning. Reusable runtime control, optional data-state recovery, and distributed delivery/ownership contracts are the main next integration requirements. This bounded static assessment found no confirmed new defect; it was not a complete PR, robotics, GPU, or security audit.

The assessment inspected PR #809 at 10a399bf9. That candidate provides independent action width, manual refill generations, causal contexts/masks, provenance, atomic source quotas, throughput planning, numeric live/resident/LeRobot sources, and the local ChunkBC demonstration. Shared-memory copying, core worker lifecycle and error propagation already existed and must not all be presented as new innovations.

ODS architecture and simplified runtime diagram document the implemented local BC path. The proposed RLInf and DAgger handoffs remain tracked by the follow-up gates below. The showcase does not complete RLInf integration, same-state expert labeling, cross-round DAgger aggregation, or coordinated data/learner recovery.

13.2 Public upstream evidence and reuse decisions

Only official public sources support this update. References are a 2026-10-20 snapshot; released APIs and development-main capabilities are distinguished. The unpublished local RLInf branch mentioned in section 1 is historical context, not evidence of public support.

No inspected system matched the whole ODS composition exactly, but this is not proof of absence. Async production, shared replay, causal windows/masks, sample-count caps and sample/insert control have established precedents. ODS's defensible engineering value is the integration of EmbodiChain expert generation, measured physical acceptance, explicit action semantics, trajectory identity and auditable supply. Novelty or learning-efficiency claims require experiments.

Dependency adoption is a separate compatibility decision. The current demo validation used Python 3.11/Torch 2.7/LeRobot 0.4.4. For example, TorchRL v0.14.1 documents TensorDict >=0.14.3,<0.15.0; independently choosing each project's latest release is not a compatible environment specification. Do not upgrade the simulation/training stack merely because an upstream example uses a newer API.

13.3 Actionable follow-ups and acceptance gates

These gates refine P0–P3 and define separately scoped P4/P5 extensions. They do not retroactively expand PR #809 into a complete training framework.

• Host-side flow controller — P1 integration. Extract the reusable owner-side pacing/refill/wait/health/shutdown responsibilities currently exercised by the BC example. Keep configure_ods static and SequenceDataset immediate: no hidden sleeps, retries, refill or inferred epochs. Drain enabled lifecycle events during production and waits. Test slow supply, changing rates, coalesced requests, quota/window exhaustion, terminal worker failures, deadlines and complete shutdown. Memory pressure or reduced supply must lower consumption, not silently raise existing source caps.
  • Implementation under review: PR #823 adds an opt-in owner-process/thread flow controller for a static measured plan, compact operational counters, bounded event-draining waits, quota-based manual refill scheduling and batch pacing, and migrates/simplifies the BC demo and direct LeRobot collector. Engine start/stop and optimizer/evaluation/checkpoint ownership remain with the application. Validation includes 571 focused tests, 15 paired application-regression scenarios and short real online/LeRobot simulation checks; these do not qualify learning-safe reuse or training-efficiency claims. This gate remains open pending review and the remaining acceptance work.
• Explicit source capabilities and operational reporting — P1. Expose supported sampling modes/filters, worker/thread/rank restrictions, available statistics and recovery level before launching. Keep optional lifecycle/state capabilities separate so a fixed resident source is not required to implement refill. Distinguish publication, delivery and completed optimizer counters; provide aligned intervals rather than deriving live throughput from mismatched counter boundaries.
• Reuse-profile qualification and configuration method — P1, with P5 learning evidence. Bind recommended/hard bounds to task, model, action representation, history/horizon and trajectory-length assumptions. Measure real concurrent generation/learner service, publication latency and memory; retain raw counts, windows and hardware placement. Publish provisional/qualified status, executable guidance and explicit infeasibility reasons. Revalidate learning after changes to the profile or sampling distribution.
• Recovery levels and optional source state — P1/P2. Preserve declared fresh-cache online resume as a valid initial mode. Before supporting protected offline resume, restore corpus identity, ledger and sampling RNG. Coordinated P2 recovery must include replay/source identity and retained payload references, quota, source progress, pending/acknowledged commits, RNG, learner state and an explicit in-flight-batch policy. Test interruption before/after admission and repeated resume. Loading a model or restarting from a seed alone is insufficient.
• One sampling/quota owner — P1 local baseline, P4 distributed extension. Begin with a single owner and num_workers=0 for protected source consumption. Live shared-engine consumers use that engine's ledger; independent resident/rank copies cannot mint new credits. Before scaling, define central ownership or disjoint allocation and verify aggregate counts across consumers/ranks, retries and restarts. Keep global sample/insert pacing and per-episode delivery caps as distinct controls.
• RLInf BC/SFT adapter — P0/P1. Pin compatible public revisions and one model input contract. Consume causal BC contexts directly, with explicit schema, action conversion, normalization and logical update/epoch limits. Establish batch/mask/loss/update parity, actual fresh-data consumption, independent student evaluation, restart semantics and cleanup before expanding topology. A matching action width or environment adapter is insufficient proof.
• DAgger episode publication — P2/P3. Add an expert interface for learner-visited pre-action states; specify query side effects, invalid labels and per-row failures. Keep behavior actions, expert labels, label/intervention masks, policy/expert versions and measured outcomes separate. Preserve valid supervision from learner failures. Publish committed episodes once through a bounded, acknowledged, deduplicated handoff into one selected RLInf aggregation path. Once RLInf owns training sampling, ODS must not maintain a second training replay or apply BC minibatch debits as if they described RLInf replay consumption.
• Compatible backend reuse and larger data — P4. Evaluate a TorchRL-backed source using Storage/Sampler/Sequence only where it preserves padding, episode weighting, continuity/segment/generation filters, owned batch lifetime and atomic quota semantics. Benchmark before replacement. Add out-of-core image/video decoding and bounded episode/staging/prefetch caches when the workload requires them; the current resident numeric LeRobot reader is not that capability. Include SDK/native allocations in total memory measurements.
• Performance and learning-efficiency qualification — P5, and prerequisite to claims. Qualify paired full-path instrumentation cost first. Freeze balanced multi-seed comparisons including the strong small-offline baseline, with fresh held-out scenes and immutable checkpoint selection. Report acquisition-inclusive and data-ready timing separately, useful-policy availability, evaluation certification time, full process cost and negative results. Use equal-data and equal-total-cost comparisons; include failed generation, retries, persistence/loading and waits. Do not use increased utilization or a bounded quota alone as evidence of useful learning.

For performance tuning, measure lock wait/hold time, copy bytes, transform/transfer cost and batch latency before adding workers. Existing selection/copy synchronization remains required. Consider prebatched consumption and publication-revision-indexed eligibility caches only after correctness tests and representative measurements; no speedup is claimed by this assessment.

13.4 Configuration guidance: what two throughput inputs can determine

Let g be successfully published episodes per active production second, including failed attempts, and t completed training chunks per active consumer second, including sampling, conversion, transfer and optimization. Then t/g has units of chunk selections per accepted episode. The current planner computes:

K_balance = t / g
K = clamp(floor(K_balance), K_min, K_max)
training_rate_target <= min(t, g*K, unlocked_quota / safe_publication_latency)

K_min, K_max and the hard ceiling come from a qualified or explicitly provisional reuse profile. B/P, first-publication latency and optional episode-storage allowance remain measured inputs; the function preserves B/P and does not optimize them. Profile qualification, diversity needs, required update count and convergence cannot be inferred from g and t alone.

The intended guidance is qualified reuse bounds + measured runtime metadata + static configuration + host-side pacing/monitoring. Measure concurrency jointly, distinguish active capacity from deliberately throttled whole-window rates, and remeasure after changing hardware placement or schedule. Adjust pace/refill/waits when supply changes within an assigned profile. Do not automatically increase a retained trajectory's allowance to maintain GPU utilization.

A delivered chunk, a supervised-frame appearance and a completed optimizer contribution are separate quantities. Report at least:

• Accepted trajectories/s and eligible anchors/s; first/full-publication latency p50/p95.
• Delivered and completed chunks/s, source/generation shares, source age and first-use delay.
• Per-source chunk draws, unique-anchor coverage, valid supervised-frame exposure p50/p95/max, cap violations and unused allowance retired.
• Lock/copy/transfer latency, data/quota waits, intentional pacing, in-flight/prefetch memory and total CPU/GPU memory.
• Student-only task success, lift/goal-error metrics, quality drawdown and retention at fixed cost; time/GPU-hours to a predeclared quality threshold.

13.5 Evidence limits retained for future comparisons

The historical four-arm pilot at 3cd6379a used one learner seed and reused 20 development scenes. Offline32's first retrospectively qualifying checkpoint was saved at 92.79 s, versus 131.28 s for ODS; latest development results were 20/20 and 19/20. Offline256's cap did not bind. These observations establish neither a universal ODS advantage nor precise quality-threshold crossing times. Pilot evidence

The earlier selected 99/100 checkpoint and later 65/100 refresh regression remain evidence of physical feasibility and unresolved refresh stability. Those exposed test scenes cannot serve as a fresh sealed final panel. Learning-safe reuse profiles remain provisional, and the historical full-path instrumentation-overhead gate is incomplete. Working showcase and regression

The integrated py311 smoke checks exercised online/offline/resume/cleanup but required the documented matching native-runtime overlay on that host. They do not establish unmodified-install portability, distributed correctness or new held-out learning performance. Preserve exact executed revisions and setup failures when reporting later validation.

The immediate sequence is: make owner-side control/capabilities reusable; pin and validate one RLInf BC/SFT adapter; implement synchronous same-state episode handoff and coordinated recovery; then scale or run adaptive-acquisition research as separately qualified work.