Skip to content

Add pouring tasks, fluid replay and DexUni DFSPH prototype - #790

Draft
kioxli wants to merge 9 commits into
codex/fluid-sim-integrationfrom
feat/dexuni-fluid-prototype
Draft

kioxli wants to merge 9 commits into
codex/fluid-sim-integrationfrom
feat/dexuni-fluid-prototype

Conversation

@kioxli

@kioxli kioxli commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Adapt the pouring task, diagnostic replay and the standalone DexUni robot demonstration to the current fluid foundation in #777. The task now uses DexUni with nested DFSPH configuration, follows the relocated task and robot packages, and records current camera frames at the configured control rate.

The fluid deployment uses a task-specific trajectory policy with a 160-sample minimum, retaining the planner's joint limits and grasp/settling waits while shortening the motion. The standalone DexUni demonstration uses the same minimum; the redundant standalone DFSPH-named demonstration has been removed.

A fluid scene without an explicit joint selector assigns the finalized joint topology to MuJoCo before solver construction. This allows the current DexUni/DFSPH prototype to advance the Franka and free cup together while preserving explicit selectors. The task sends both gripper finger targets, avoiding a missed grasp after the newer Gym action-selection changes.

The cup keeps separate rigid-contact hulls and a particle-only SDF boundary. On DexSim builds without NewtonCollisionDesc.contributes_to_mass, dynamic boundaries require explicit mass and inertia; the task authors the cup COM and inertia from its contact hulls. The FastRT liquid uses a compatible blue material. The Gym runner saves incomplete or rejected videos as diagnostics rather than qualifying them as accepted recordings.

Stack and dependencies

  • Base: PR #777, codex/fluid-sim-integration.
  • Qualified with DexSim 0.5.1+0b8422cc6f6717058bd15371e61bd35cdbbe3c0c, Newton 1.7.0, Warp 1.17.0 and CUDA on RTX 5090. Published 0.5.1rc1 does not include the required DexUni/DFSPH composition.
  • Fluid coupling is one-way: rigid motion affects the liquid, but liquid forces do not feed back into the cup or robot. The native prototype requires a pure-MuJoCo rigid path when bodies are present and cannot combine fluid with cloth or soft particles.

Validation

  • Full FastRT Task Program, one environment, 1.5 mm spacing and 76,607 particles: all four segments and physical qualification passed. The new 960×720, 40 fps recording is 28.475 s versus 43.75 s for the previous policy. Measured lift 120.4 mm, maximum tilt 63.4°, return error 2.2 mm and received volume 144.9 mL (previously 160.1 mL; acceptance requires at least 50 mL).
  • The standalone DexUni Atomic Action demo completed a full four-environment FastRT run in one World, at 1.5 mm particle spacing and 76,607 particles per environment. All four passed physical qualification: each received 139.7–140.1 mL, lifted 126.5 mm, tilted 62.6°, and returned within 4.4 mm. Each camera recorded 26.5 s of simulation at 40 fps; the measured end-to-end run took 512.8 s on the RTX 5090. The robot and fluid were solved online.
  • Serial GPU checks passed for DexUni fluid lifecycle, isolated multi-world reset, dynamic boundary motion and mass/inertia, robot mimic handling, Gym partial reset, diagnostic replay, and FastRT camera visibility. The online test was rerun after updating its DexUni configuration.
  • Static deployment inspection passed for the fluid, Newton and default Franka variants. Selected CPU suites passed 582 tests; an additional overlapping backend/manager selection passed 250 tests. The full suite was not run.
  • For the shortened policy, the fluid deployment inspector and 130 configuration, package-staging and task-runner tests passed; package staging resolves the fluid deployment and its new policy from installed paths.
  • black . ran with 26.3.1 and 26.10.1; API coverage is 3300/3300, agent context checks pass, and task discovery lists pour_cup.
  • The full Task Program was qualified for one environment. The separate Atomic Action demo was qualified for four environments, and two-environment isolation and reset were tested separately; a full multi-environment Task Program has not been qualified.

Type of change

  • New feature (non-breaking change which adds functionality)

Checklist

  • I ran black ..
  • I reviewed and updated affected agent context.
  • Public API documentation coverage passes.
  • Appropriate behavioral coverage and test necessity were reviewed.
  • The required DexSim development capabilities are identified above.

kioxli and others added 4 commits October 15, 2026 04:30
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Allow fluid objects when DexUni carries an sph_cfg, route fluid state
invalidation through the backend's SPH solver, and add a pouring demo that
swaps the SPH+MuJoCo composition for DexUni followed by DFSPH.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… only

Every joint, the cup's free joint included, is MuJoCo-owned, so DexUni runs
its pure-MuJoCo backend and VBD is skipped. DFSPH steps at 480 Hz on a
second CUDA stream while MuJoCo takes 15 substeps per DFSPH step (7,200 Hz).
A stiffer cup contact offsets the coarser MuJoCo step's frictional creep,
so the pour matches the 10,080 Hz MuJoCo result 1.4x faster.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yuecideng yuecideng changed the title feat(fluid): prototype DexUni + DFSPH online pouring (MuJoCo + DFSPH, no VBD) Add pouring tasks, fluid replay and DexUni DFSPH prototype Oct 22, 2026
@yuecideng yuecideng added data-pipeline Data collection, ODS, sampling, recording, persistence, conversion and analysis. dexsim DexSim SDK interfaces, version adaptation and integration. feature New user-facing capabilities. physics Physics backends, dynamics, contacts, fluids, deformables and physics solvers. task Concrete tasks, deployments, objectives and recipes, including configured Task Program tasks. task-program Task Program language, semantics, compiler, runtime and integration contracts. assets Robot and scene assets, meshes, materials, asset catalogs and downloads. docs Documentation, API references and example explanations. gym Environment lifecycle, managers, configuration composition and registration. labels Oct 25, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

assets Robot and scene assets, meshes, materials, asset catalogs and downloads. data-pipeline Data collection, ODS, sampling, recording, persistence, conversion and analysis. dexsim DexSim SDK interfaces, version adaptation and integration. docs Documentation, API references and example explanations. feature New user-facing capabilities. gym Environment lifecycle, managers, configuration composition and registration. physics Physics backends, dynamics, contacts, fluids, deformables and physics solvers. task Concrete tasks, deployments, objectives and recipes, including configured Task Program tasks. task-program Task Program language, semantics, compiler, runtime and integration contracts.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants