Skip to content

Unify parameter-driven visual materials on DexSim 0.5.1rc1 - #841

Merged
yuecideng merged 11 commits into
mainfrom
codex/visual-material-consistency
Oct 23, 2026
Merged

yuecideng merged 11 commits into
mainfrom
codex/visual-material-consistency

Conversation

@yuecideng

@yuecideng yuecideng commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Description

Visual materials used separate initialization paths that could overwrite authored resource values, force a scattering model, or lose material assignments during reset/rebuild. This change uses VisualMaterialCfg for handwritten properties and default presets, selects scattering from optical parameters, and preserves native identity/state across runtime creation and Spawn on DexSim 0.5.1rc1.

cfg = VisualMaterialCfg.from_preset(
    "GlassClear", uid="glass", ior=1.6,
    parameters={"volume_attenuation_distance": 8.0},
)
material = sim.create_visual_material(cfg)
shape = SphereCfg(radius=0.25, visual_material=cfg)

ice = VisualMaterialCfg(
    base_color=[1, 1, 1, 1], roughness=0.2, ior=1.31,
    parameters={"transmission": 1.0},
)
  • Keep None as preset inheritance through copy/replace and Python/YAML/dictionary round trips. GlassClear inherits IOR 1.52 unless explicitly overridden; zero/false remain explicit values. Common native aliases canonicalize into flat fields, and duplicate properties fail validation.
  • Add 24 public presets: six parameter definitions included in the wheel and 18 downloadable resources, plus the explicit SoapBubbleInterior helper. DefaultMaterials uses pinned archive/member digests, source attribution, revision-specific locked caching and extraction repair. Built-ins resolve without constructing the texture dataset; PNGs stay outside the wheel.
  • Share one manifest/template resolver across runtime and Spawn for resources and advanced handwritten parameters. Stage JSON before binding, reference cached images, share immutable templates by normalized definition, and allocate independent UID instances. Basic handwritten materials retain lightweight native descriptor creation without an explicit scattering-type write.
  • Remove VisualMaterialCfg.material_type, VisualMaterial.RT_MATERIAL_TYPES, MAT_TYPE_MAPPING and explicit native scattering-type calls. Migrate ice to transmission parameters with an OfflineRT default. cuRobo diagnostic shells select scattering through transmission and no longer send unsupported legacy absorption property names.
  • Preserve equivalent UID sharing, conflict rejection, independent unnamed materials, full resource resets, selected-environment assignments and bounded native instance reuse across rebuild/cleanup. NoRender retains resource declarations without downloading or creating native materials.
  • Add embodichain preview-material for native-window previews, headless captures, overrides, source filtering, direct-native comparisons and galleries. Help/list use a lightweight catalog without renderer imports or network access. The example delegates to this command with a headless default.
  • Fix the CI deformable-declaration fixture to pass an explicit renderer with its minimal World stub. Production renderer queries remain unchanged.

Dependencies remain dexsim_engine==0.5.1rc1, Warp 1.17.0 and Newton 1.6.0. The branch includes main through 658377f65.

Compatibility and scope

  • The scattering selector removal is a breaking API change. Remove material_type from stored configurations, including old null values. from_dict() reports a migration error; legacy BRDF/BTDF/BSDF labels are not silently converted because their optical semantics differ. Use explicit transmission, IOR, roughness and required volume/thin-wall parameters.
  • Preset fields omitted or set to None inherit resource values. replace(ior=None) restores inheritance; use from_preset() to choose a new source. Handwritten scalar defaults remain available.
  • Fourteen presets support Hybrid/FastRT/OfflineRT. Eight advanced resource presets, two surface-glass built-ins and advanced handwritten/preset parameter variants require OfflineRT. Incompatible requests fail before asset/native loading; compatible-only batch export explicitly records exclusions. Renderers never switch automatically for a material request.
  • Existing identity/selection corrections remain: RGB native color reads, uid=None for unnamed configurations, explicit UID reservation for the World lifetime, and rejection of partial selections for shared randomization.
  • Bubble shells, optional segmented face geometry, lights and cameras belong to preview scenes. Suggested texture dimensions do not change UVs. Skin approximates diffuse transmission.
  • Pre-create variants and rebind instances. Cross-model hot shader mutation and NRD packaging fixes remain outside this change.

Type of change

  • New feature
  • Bug fix
  • Breaking change: scattering selector removal and the documented material identity/selection corrections
  • Documentation update

Validation

Qualification used the actual rc1 wheel on an NVIDIA RTX 5090 with Python 3.10, Warp 1.17.0 and Newton 1.6.0.

  • 1,076 focused CPU tests passed, with 81 resource-dependent cases skipped and two deselected. Coverage includes material configuration/lowering, Spawn, object reuse, randomization, deformable declarations, preview CLI, cuRobo visualization, data assets, API docs and project context. The final cuRobo cleanup additionally passed its five existing visualization cases.
  • 6 serial native tests passed across Default/Newton. Both handwritten optical parameters and presets exercise UID sharing/isolation, full reset, alias binding, selective restoration, rebuild/cleanup and 1,025 repeated instance creations. Native type selection is never explicitly written by the wrappers.
  • Handwritten parameter parity: seven OfflineRT definitions and four opaque definitions on each of Hybrid/FastRT match independent native JSON loading at 256×256, four frames and six render iterations; every center-crop RMSE was zero in this run. The ice configuration and parameter-driven cuRobo diagnostic shell were rendered. Each real-time process rejected all three OfflineRT-only handwritten requests. The final render run emits no removed-selector or unsupported legacy-absorption warnings.
  • Preset optics: all 24 OfflineRT presets were qualified against direct native loading at 640×640 with six frames, OptiX denoising and 12 render iterations, including segmented skin and double-shell bubbles. Maximum center-crop RMSE was 0.125 on the 0–255 scale. All fourteen compatible presets were qualified on Hybrid/FastRT through the resource and built-in checks. The production CLI also completed 24/24 OfflineRT and 14/14 Hybrid direct-native comparisons.
  • Distribution: empty-cache download/verification, cache reuse, extraction repair and installed-wheel probes passed. A freshly built wheel outside the checkout has no removed selector/constants, supports parameter config round trips, and lists all 24 presets without accessing the material archive. Installed CLI discovery was also checked with native imports/network forbidden.
  • Native window: GlassClear was opened on the desktop, visually inspected and kept running as requested; it later shut down normally on the user's request.
  • CI regression: the four reported deformable failures were reproduced before the fixture fix. The repaired fixture and related Spawn/material tests passed all 114 cases locally.
  • Docs/style: Black 26.3.1 checks all 1,532 Python files; API docs cover 3,042/3,042 exports. Context checks, routes and git diff --check pass. The targeted material/CLI guides build as HTML with three existing ReuseSegmentState warnings. The full Sphinx dummy build passes with the same 816 existing warnings, using a temporary process-local ast.TryStar definition for the existing Python 3.10 architecture-generator issue; no tooling workaround is committed.

Test necessity was reviewed against the full merge-base delta. Existing production material/native cases were extended for selector rejection, parameter-driven loading and reset/rebuild behavior. Existing cuRobo/deformable tests cover the migrated native API and CI boundary; no tutorial-specific tests were added. Context review revised the owning material contract; other routing/lifecycle guidance remains accurate.

The acceptance plan, material guide and CLI guide record behavior and reproduction commands.

Resource availability

The six built-in definitions require no texture-archive download. The pinned source archive is currently on the DexSim internal resource server; a public Hugging Face copy is not published here. EMBODICHAIN_DOWNLOAD_PREFIX selects a mirror providing the identical documented revisioned archive. Upstream manifests/README and provenance are retained; the archive has no separate asset license. Source attribution is pinned to DexVerse commit c31dccd3d4a8ef98367e1a95c92c842e0a773596.

Dev source was inspected; a separate dev native binary was not built. Long-duration VRAM qualification remains outside the completed checks. The rc1 NoRender entity-balance warning also occurs with an empty baseline World.

Screenshots

The guide contains all 24 preset names/parameters, individual images and grouped OfflineRT galleries. The 28 documentation WebPs total approximately 0.84 MB and are excluded from the wheel. Regenerate them with embodichain preview-material --all --headless --gallery.

Built-in OfflineRT materials

All 24 OfflineRT presets

All default OfflineRT materials

Checklist

  • Ran black . and the read-only public API docs gate.
  • Updated affected documentation and the owning project context.
  • Reviewed behavioral coverage and test necessity.
  • Preserved the existing DexSim dependency pin.

Share PBR lowering and definition identity between runtime creation and Spawn, preserve native instance state, and restore assignments across reset and topology rebuilds. Bound randomization resources and isolate selected environments, with rc1 CPU/native regression coverage and an acceptance plan.

BREAKING CHANGE: material color reads return RGB; unnamed materials no longer share default_mat; repeated instance creation preserves state; shared material randomization requires all environments.
@yuecideng yuecideng added bug Incorrect existing behavior or a fix restoring the intended behavior. rendering Image generation, lighting, material rendering, native renderers and image quality. breaking User-visible incompatible changes requiring migration. dexsim DexSim SDK interfaces, version adaptation and integration. labels Oct 22, 2026
@greptile-apps

greptile-apps Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 3/5

[Medium impact] The changes reach the shared visual-material workflow across runtime creation, Spawn, randomization, and preview tools.

The PR should not merge until the two previous material reuse and texture-reset issues are fixed.

Fix All in CodexFindings

  1. P1 Equivalent textured materials are rejected ▶
  2. P1 Fixed resets keep old textures ▶
Fix with agent prompt
### Issue 1
embodichain/lab/sim/sim_manager.py:4821-4825
After Spawn registers a material whose `base_color_texture` is a package address such as `asset.usdz[textures/map.png]`, the registry contains the extracted file path. Calling `create_visual_material()` with the same configuration compares that path with the original package address and raises `ValueError`, so callers cannot reuse the same material across both APIs.

Resolve texture paths consistently before comparing definitions, and keep the stored definition consistent.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

### Issue 2
embodichain/lab/gym/envs/managers/randomization/visual.py:106-107
The new fixed-material reset leaves a previously assigned texture attached when the configuration has no texture. `set_default_properties()` passes `None` to the texture setters, but those setters do nothing when all inputs are `None`. Assigning a texture to the fixed material instance and then running this reset restores the color and scalar values, but not the requested untextured appearance. The group reset has the same problem.

Clear absent texture maps explicitly and cover this case in the reset tests.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR unifies handwritten and preset materials around VisualMaterialCfg and parameter-driven scattering.

  • Runtime materials and Spawn declarations share one configuration and keep their assigned appearance through rebuilds.
  • The material catalog offers built-in and downloadable presets for common surfaces, glass, metal, skin, and soap bubbles.
  • Developers can browse presets and render individual materials or galleries with preview-material.
  • Material randomization respects selected environments and updates shared instances consistently.
  • The ice example and cuRobo diagnostic materials use optical settings instead of the removed scattering selector.

No new findings were accepted in this review.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  C[VisualMaterialCfg] --> V[Validate and normalize]
  V --> B[Basic handwritten properties]
  V --> M[Preset or advanced parameters]
  B --> D[Native descriptor]
  M --> P[Check renderer support]
  P --> T[Shared immutable template]
  T --> I[Separate instance for each UID]
  D --> A[Runtime or Spawn assignment]
  I --> A
  A --> R[Preserve assignments across rebuilds]
Loading

Reviews (9) · Last reviewed commit: "refactor(sim): select visual scattering ..." · Reviewed by Greptile

Comment on lines +4820 to +4824
descriptor = _material_desc_from_cfg(cfg, cfg.uid)
definitions = self._spawn_scene.builder.materials
registered = definitions.get(cfg.uid)
if registered is not None and registered != descriptor:
raise ValueError(f"Conflicting visual material definition for {cfg.uid!r}.")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Equivalent textured materials are rejected

After Spawn registers a material whose base_color_texture is a package address such as asset.usdz[textures/map.png], the registry contains the extracted file path. Calling create_visual_material() with the same configuration compares that path with the original package address and raises ValueError, so callers cannot reuse the same material across both APIs.

Resolve texture paths consistently before comparing definitions, and keep the stored definition consistent.

Prompt To Fix With AI
This is a comment left during a code review.
Path: embodichain/lab/sim/sim_manager.py
Line: 4820-4824

Comment:
**Equivalent textured materials are rejected**

After Spawn registers a material whose `base_color_texture` is a package address such as `asset.usdz[textures/map.png]`, the registry contains the extracted file path. Calling `create_visual_material()` with the same configuration compares that path with the original package address and raises `ValueError`, so callers cannot reuse the same material across both APIs.

Resolve texture paths consistently before comparing definitions, and keep the stored definition consistent.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Codex Fix in Claude Code

Comment on lines +106 to +107
for instance in obj.get_visual_material_inst(env_ids=env_ids):
mat.set_default_properties(instance, mat_cfg)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Fixed resets keep old textures

The new fixed-material reset leaves a previously assigned texture attached when the configuration has no texture. set_default_properties() passes None to the texture setters, but those setters do nothing when all inputs are None. Assigning a texture to the fixed material instance and then running this reset restores the color and scalar values, but not the requested untextured appearance. The group reset has the same problem.

Clear absent texture maps explicitly and cover this case in the reset tests.

Prompt To Fix With AI
This is a comment left during a code review.
Path: embodichain/lab/gym/envs/managers/randomization/visual.py
Line: 106-107

Comment:
**Fixed resets keep old textures**

The new fixed-material reset leaves a previously assigned texture attached when the configuration has no texture. `set_default_properties()` passes `None` to the texture setters, but those setters do nothing when all inputs are `None`. Assigning a texture to the fixed material instance and then running this reset restores the color and scalar values, but not the requested untextured appearance. The group reset has the same problem.

Clear absent texture maps explicitly and cover this case in the reset tests.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Codex Fix in Claude Code

@yuecideng yuecideng changed the title Fix visual material identity and lifecycle consistency Unify visual material presets and lifecycle on DexSim 0.5.1rc1 Oct 23, 2026
@yuecideng yuecideng added enhancement Improvements to existing capabilities; use feature for new capabilities. assets Robot and scene assets, meshes, materials, asset catalogs and downloads. labels Oct 23, 2026
@yuecideng yuecideng changed the title Unify visual material presets and lifecycle on DexSim 0.5.1rc1 Unify parameter-driven visual materials on DexSim 0.5.1rc1 Oct 23, 2026
@yuecideng
yuecideng merged commit bb2b65e into main Oct 23, 2026
12 checks passed
@yuecideng
yuecideng deleted the codex/visual-material-consistency branch October 23, 2026 17:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

assets Robot and scene assets, meshes, materials, asset catalogs and downloads. breaking User-visible incompatible changes requiring migration. bug Incorrect existing behavior or a fix restoring the intended behavior. dexsim DexSim SDK interfaces, version adaptation and integration. enhancement Improvements to existing capabilities; use feature for new capabilities. rendering Image generation, lighting, material rendering, native renderers and image quality.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant