Skip to content

Add basic fluid simulation and standalone tutorial - #777

Open
yuecideng wants to merge 22 commits into
codex/adapt-latest-dexsimfrom
codex/fluid-sim-integration
Open

yuecideng wants to merge 22 commits into
codex/adapt-latest-dexsimfrom
codex/fluid-sim-integration

Conversation

@yuecideng

@yuecideng yuecideng commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Description

Add fixed-count fluid objects, procedural initial-fill sampling, world-frame particle state access/reset, particle/surface rendering, and a standalone multi-environment tutorial. Fluid solving now consistently enters through DexUni: scene contents activate DFSPH, and optional SPH settings are nested in DexUniSolverCfg.sph_cfg. Standalone solver_type: sph mappings report how to migrate.

Fluid density and spacing belong to the fluid declaration. The tutorial uses a reconstructed mesh surface and static SDF container walls; DFSPH boundary collisions remain enabled without running a rigid-body solver in this fixed-only scene. Fluid render publication orders CUDA work and drains prior staged writes when necessary so cameras see explicit particle-state edits.

Stack and dependencies

  • Base: PR #829, codex/adapt-latest-dexsim.
  • Followed by PR #790. Its pouring/replay code still uses the prior standalone SPH path and needs a separate migration.
  • The inherited dependency range is dexsim_engine>=0.5.1rc1,<0.5.2, but fluid use now requires a build exposing DexUniSolverCfg.sph_cfg. This change was qualified with DexSim 0.5.1+0b8422cc6f6717058bd15371e61bd35cdbbe3c0c, Newton 1.7.0, Warp 1.17.0 and Torch 2.7.1+cu128. Published rc1 does not provide the DexUni/DFSPH composition.
  • DexSim currently marks this composition as a prototype: rigid motion influences the fluid one-way, fluid forces do not feed back to rigid bodies, the solid path must resolve to pure MuJoCo when rigid bodies are present, and fluid cannot coexist with cloth/soft-body particles. DexUni's rigid/soft two-way setting does not change the fluid coupling direction.
  • Adapt current DexSim and consolidate DexUni two-way support #829 remains Draft and has conflicts with main; it must be integrated before this stack can land on main.

Validation

  • 659 focused CPU checks passed across fluid configuration, state, Spawn, manager, solver configuration, visualization and context. An additional final focused selection passed 73 tests after error-message and staging-capability changes.
  • 6 serial GPU checks passed on RTX 5090: default/explicit DexUni activation, isolated and overlapping worlds compared with a single-world reference, selected-world reset without modifying dynamic rigid state, fluid diagnostics, FastRT camera pixels with and without clipping, and clipping-cache stream use. Repeated write/publish/camera cycles, including a non-default CUDA stream, passed three additional runs for both clipping modes.
  • The static-container tutorial completed 120 NoRender and 30 FastRT mesh-surface steps with 4 environments, each with 1,728 particles. A separate four-camera FastRT preview was rendered. An interactive native window was not tested because no display was available.
  • black . with 26.3.1 and 26.10.1, API coverage 3,278/3,278, context checks including 42 context tests, and the focused render benchmark smoke check passed.

A synchronized NoRender comparison used a fixed-container scene on a shared RTX 5090, 1,728 float32 particles per environment, three runs of 120 steps after 10 warmup steps, and the same DFSPH settings. DexUni disabled rigid-rigid contacts for this fixed, kinematic-walled comparison; the original SPH baseline used a temporary compatibility bridge only for declaration/reset. Final particle position and velocity arrays matched exactly in both cases.

Environments Standalone SPH DexUni + DFSPH
1 1.76 ms/step 2.39 ms/step
4 2.12 ms/step 2.82 ms/step

The measured wall-time includes manager/physics stepping and device completion, excludes setup/reset/warmup/rendering, and is specific to this shared-GPU workload. Sampled process GPU memory was approximately 768 MiB and 774 MiB respectively; sampling may miss short peaks. An additional four-environment NoRender profile measured DexUni at 2.86 ms/step with kinematic walls and 2.04 ms/step with static walls (median of 10 and 5 samples, respectively; 120 steps after 30 warmup steps). Particle positions and velocities at the final snapshot were identical; both measurements used the shared GPU.

The existing fluid lifecycle and camera tests were extended for the new dispatch and staging behavior; no tutorial-import tests were added. Fluid, physics configuration and rendering context now describe the capability and synchronization boundaries.

Type of change

  • New feature (non-breaking change which adds functionality)

Checklist

  • I ran black ..
  • I reviewed affected documentation and agent context and updated it where needed.
  • Public API documentation coverage passes.
  • Appropriate behavioral coverage and test necessity were reviewed.
  • Dependencies were checked against the target DexSim build.

kioxli and others added 5 commits October 12, 2026 23:09
Add typed fluid scene ownership, observations and row-local refill; gate batching on native SPH capabilities. Generate and qualify the pouring Task Program in MuJoCo before replaying measured states in an independent SPH Gym. Package physical assets and keep source acceptance separate from fluid metrics. Requires DexSim world-isolated SPH support (d2d36fbcb) for batched scenes.
Use Hybrid by default with FastRT support, and route replay videos through the configured record_camera event. Add separate environment buffers, selected save/discard and configurable cadence to record_camera_data. Explicitly commit the final replay recording. Qualified two complete 744-frame Hybrid videos, both renderer smoke runs, row-local recording reset, 128 focused tests and the live Gym reset regression.
@yuecideng yuecideng added enhancement Improvements to existing capabilities; use feature for new capabilities. rendering Image generation, lighting, material rendering, native renderers and image quality. physics Physics backends, dynamics, contacts, fluids, deformables and physics solvers. gym Environment lifecycle, managers, configuration composition and registration. dexsim DexSim SDK interfaces, version adaptation and integration. labels Oct 14, 2026
Sample a declared capped revolved profile and fill level on a deterministic lattice. Keep solid clearance separate from the free surface and retain the local template in memory for reset. Remove the default water_fill particle asset, resample the same recipe for spacing overrides, and preserve explicit point imports. Qualify complete two-environment Hybrid pouring and per-environment recordings.
@yuecideng yuecideng changed the title Add fluid scene APIs and Task Program Gym replay Add fluid APIs and Task Program pouring workflow Oct 14, 2026
@yuecideng
yuecideng changed the base branch from codex/franka-dfsph-pour-demo to main October 14, 2026 10:43
@yuecideng
yuecideng force-pushed the codex/fluid-sim-integration branch from 2547154 to 0dd4d16 Compare October 14, 2026 15:47
@yuecideng yuecideng changed the title Add fluid APIs and Task Program pouring workflow Add fluid objects and online Task Program pouring Oct 14, 2026
@yuecideng

Copy link
Copy Markdown
Contributor Author

Standalone fluid performance: CUDA Graph and DFSPH

CUDA Graph is enabled and replayed successfully. The dominant measured cost is GPU DFSPH projection work.

This investigation uses the standalone create_fluid.py example in the local worktree, which has not yet been pushed to the PR branch. These measurements characterize that workload: one environment, 27,000 particles at 3 mm spacing, five static SDF tank boundaries, an RTX 5090, and the same isolated development JIT runtime used for this PR's qualification (Newton 1.6.0, Warp 1.17.0, Torch 2.10; DexSim Python source d7a0435d with C++ core 21bfad).

CUDA Graph verification

  • The Graph-enabled run performed 480 Graph launches in 480 public physics updates, with a live captured graph on every update.
  • The control run disabled Graph through configuration before constructing the simulation. Sampled physical states were exactly identical between the two runs.
  • Each run advanced 2 simulated seconds with resets disabled. After skipping 240 updates, the following 120 updates cover simulated time 1.0–1.5 s:
Execution mode Wall time for the selected 0.5 simulated seconds
Graph enabled 1.763 s
Graph disabled 1.754 s

The difference is below 1% in this single-run comparison and is not evidence of a statistically established speedup or regression.

A separate Graph timing probe recorded 1,763.46 ms of GPU Graph execution in a 1,763.89 ms measurement window. Python scene preparation and submission are therefore not the primary throughput limit in this window.

Why the workload is expensive

The configuration uses a 240 Hz outer physics loop with six SPH substeps per update: 1,440 SPH substeps per simulated second. Each substep takes approximately 2.45 ms of GPU Graph execution, while advancing only 0.694 ms of simulated time. This accounts for approximately 0.28× real-time throughput, consistent with the earlier complete rollout measurements.

Warp CUDA-event sampling in a separate Graph-disabled diagnostic run identifies these hotspots:

GPU activity Share of recorded GPU activity time
compute_dfsph_coefficient_kernel 56.8%
apply_dfsph_velocity_correction_kernel 21.5%
Neighbor search 8.1%
Iteration completion / convergence checks 4.4%
Volume-map boundary queries and penetration projection 2.0%

The coefficient and velocity-correction kernels account for approximately 78.3% combined. These diagnostic percentages include instrumentation effects and should not be interpreted as an exact per-kernel breakdown of Graph replay.

At the 1.5 s sample point, density projection reached its 20-iteration cap, with a 0.0478% projection residual against the 0.01% target. Divergence projection took one iteration. The implementation expands the configured maximum iteration counts and uses device-side active flags to return early after convergence; Graph replay preserves that structure.

The coefficient kernel performs per-particle atomic additions into one sample-count scalar and one error-sum scalar per world, while repeatedly accessing neighbor data. Atomic contention and neighbor-memory access are priority optimization candidates, but their individual contributions have not yet been isolated by ablation or hardware-counter profiling.

Rendering is a smaller contributor: separate complete 6-second, single-environment runs took 20.68 s with NoRender and 23.22 s with FastRT plus recording, approximately 12.3% additional wall time. These timings exclude scene initialization.

Suggested optimization order and limits

  1. Investigate block/warp aggregation before per-world error reduction, and improve neighbor-cache access patterns in the DexSim DFSPH kernels.
  2. Characterize timestep, pressure relaxation, density tolerance, and iteration limits for the standalone fluid scene. Validate density residuals, penetration, finite states, and fluid retention alongside throughput.
  3. Obtain a kernel-level Nsight trace when an NVTX-enabled native build is available. The current collector stopped at preflight because C++ NVTX is disabled; no .nsys-rep was produced. The GPU measurements above come from Warp CUDA events.

No solver or tutorial implementation was changed during this investigation. Raw timing, state-parity, CPU-profile, and GPU-activity records are retained as local qualification artifacts.

@yuecideng yuecideng changed the title Add fluid objects and online Task Program pouring Add basic fluid simulation and standalone tutorial Oct 22, 2026
@yuecideng
yuecideng marked this pull request as ready for review October 22, 2026 07:42
@greptile-apps

greptile-apps Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 4/5

[Medium impact] The PR adds a bounded fluid workflow to Newton simulations and their rendering path.

The stack needs PR #790's fluid constructor calls updated before merging.

Fix All in CodexFindings

  1. P1 Dependent fluid creation breaks ▶
Fix with agent prompt
### Issue 1
embodichain/lab/sim/objects/fluid.py:76-77
The new `FluidObject(cfg, device)` signature breaks callers retained in PR #790. At its supplied head, `embodichain/lab/sim/sim_manager.py:2856–2858` still passes `points`, `pose`, and `on_write`, as do two constructors in `tests/sim/objects/test_fluid_object.py`. These calls raise `TypeError` with this signature.

Update the dependent layer to use the new constructor and `_initialize_spawn_source()` before landing the stack.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds fixed-count fluids, procedural sampling, world-frame state reads and resets, and particle or surface rendering through DexUni/DFSPH.

  • Simulation scenes can declare fixed-count fluids from sampled or saved particles.
  • Fluid particles stay visible after physics steps and direct state changes.
  • A standalone tutorial fills a static box and steps the fluid.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Fluid configuration] --> B[Sample local particles]
  B --> C[Spawn declaration]
  C --> D[Prepare and bind fluid]
  D --> E[DexUni DFSPH step]
  E --> F[World-frame state]
  F --> G[Render publication]
  F --> H[Selected-world reset]
Loading

Reviews (9) · Last reviewed commit: "perf(tutorial): use static fluid contain..." · Reviewed by Greptile

Comment thread embodichain/lab/sim/material.py Outdated
Comment on lines +76 to +77
device: torch.device = torch.device("cpu"),
) -> None:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Dependent fluid creation breaks

The new FluidObject(cfg, device) signature breaks callers retained in PR #790. At its supplied head, embodichain/lab/sim/sim_manager.py:2856–2858 still passes points, pose, and on_write, as do two constructors in tests/sim/objects/test_fluid_object.py. These calls raise TypeError with this signature.

Update the dependent layer to use the new constructor and _initialize_spawn_source() before landing the stack.

Prompt To Fix With AI
This is a comment left during a code review.
Path: embodichain/lab/sim/objects/fluid.py
Line: 76-77

Comment:
**Dependent fluid creation breaks**

The new `FluidObject(cfg, device)` signature breaks callers retained in PR #790. At its supplied head, `embodichain/lab/sim/sim_manager.py:2856–2858` still passes `points`, `pose`, and `on_write`, as do two constructors in `tests/sim/objects/test_fluid_object.py`. These calls raise `TypeError` with this signature.

Update the dependent layer to use the new constructor and `_initialize_spawn_source()` before landing the stack.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Codex Fix in Claude Code

@yuecideng yuecideng added feature New user-facing capabilities. sim-object Simulation entity configuration, state, creation, destruction and lifecycle. compute Shared numerical algorithms and kernels independent of simulation objects. visualization Browser viewers, markers, trajectory previews and interactive debugging displays. and removed enhancement Improvements to existing capabilities; use feature for new capabilities. gym Environment lifecycle, managers, configuration composition and registration. labels Oct 25, 2026
@yuecideng
yuecideng changed the base branch from main to codex/adapt-latest-dexsim October 27, 2026 14:19
@greptile-apps

greptile-apps Bot commented Oct 27, 2026

Copy link
Copy Markdown

Comments Outside Diff

These findings could not be posted inline.

  • P2 Tests depend on tutorial internals tests/sim/test_open_drawer_tutorial.py:121 ▶

    The changed test loads scripts/tutorials/sim/open_drawer.py and checks its private _tutorial_physics_cfg() helper. AGENTS.md says not to create or expand tutorial-focused tests unless explicitly requested. Move these solver checks to the production configuration tests and use static checks for the tutorial. This repository requirement must be satisfied before merging.

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Comment thread embodichain/lab/sim/cfg/physics.py

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

compute Shared numerical algorithms and kernels independent of simulation objects. dexsim DexSim SDK interfaces, version adaptation and integration. feature New user-facing capabilities. physics Physics backends, dynamics, contacts, fluids, deformables and physics solvers. rendering Image generation, lighting, material rendering, native renderers and image quality. sim-object Simulation entity configuration, state, creation, destruction and lifecycle. visualization Browser viewers, markers, trajectory previews and interactive debugging displays.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants