Viewer: shared materials, x-ray by material swap, merged component bodies

CHANGELOG.md

@@ -5,6 +5,14 @@ glTF profile has its own version (`profileVersion`, [spec §11](spec/README.md#1
 
 ## [Unreleased]
 
+- **Faster x-ray and first frame on big boards (viewer):** meshes that render alike share their
+  materials, x-ray mode swaps materials prepared after loading instead of rebuilding every
+  shader, and the components' placeholder bodies are drawn as a few merged meshes instead of one
+  instanced mesh per package. On testcase1 (1,656 components) an x-ray toggle takes a few
+  milliseconds of main-thread JavaScript instead of 1.5–1.6 s, loading about 1.6 s instead of
+  3.4 s, and the first frame has 21 draw calls instead of 223; on the KiCad sample
+  royalblue54l-feather with its runtime models, a toggle takes a few milliseconds instead of
+  0.4–0.7 s.
 - **Licence: AGPL-3.0-only or a commercial licence** (was MIT). boardui stays open source;
   closed-source products, and web applications that don't publish their source, need a
   commercial licence ([LICENSING.md](LICENSING.md), [ADR 0015](docs/adr/0015-agpl-plus-commercial-licence.md)).

docs/architecture.md

@@ -107,14 +107,14 @@ Files kept on disk or served over the network can be large. A dense board approa
 Every element can be hovered, selected, recoloured or hidden, but layers stay merged meshes ([ADR 0008](adr/0008-hybrid-metadata.md), [ADR 0009](adr/0009-viewer-three-js-web-component.md)):
 
 1. **One texel per element.** A small RGBA8 *state texture* has one texel per feature, across all layers (each layer gets an offset). It holds a highlight colour, a blend factor and a hidden flag.
-2. **The shader reads it.** The shader reads `_FEATURE_ID_0` plus the layer offset, looks up the texel, then tints or discards the fragment.
+2. **The shader reads it.** The shader reads the element's texel index from the geometry (when merging a layer, the viewer stores `_FEATURE_ID_0` plus the layer offset), looks up the texel, then tints or discards the fragment. The graph holds nothing per mesh, so meshes that render alike share one material, and x-ray mode swaps each mesh's material for its x-ray variant instead of changing materials.
 3. **Hover.**
    - A BVH ray cast finds the triangle under the cursor.
    - The triangle's vertex gives the feature ID, and the metadata gives the element.
    - The viewer writes one texel; nothing is rebuilt.
 4. **Net highlight.** The viewer writes every texel of the net's features. The feature → net index is built once at load time.
 5. **Feature ranges.** Contiguous features (spec §8.1) let the loader compute each feature's vertex range and bounding box in one pass. These feed widget anchors and "zoom to element".
-6. **Components.** Component nodes that share a mesh are batched (`BatchedMesh` / `InstancedMesh`) per material, and the instance index maps back to the component row. Runtime models (below) are batched the same way, on the same texels, so a component behaves alike whatever body it shows.
+6. **Components.** The asset's component meshes are merged into one static mesh per material, each vertex carrying its component's texel, so the number of meshes doesn't grow with the number of packages; meshes that would cost too much memory as copies, and runtime models (below), are instanced, with the texel per instance. Either way a component uses the same texel, so it behaves alike whatever body it shows.
 
 ### Runtime models
 

packages/viewer/README.md

@@ -156,15 +156,26 @@ component) is switched off or the element is hidden with `hide`.
   (bottom view). The soldermask gets a polygon offset: its bottom face is coplanar with the
   dielectric's top and the copper's bottom (ADR 0006), which z-fought with the dielectric hidden.
 - **Element state.** One RGBA8 texel per feature and per component. A TSL node graph reads it per
-  vertex (feature ID + layer offset) and tints, or discards, the fragment. Hover, selection,
-  highlights and hiding are texel writes. Copper and drill layers get a second, overlay draw that
-  shows tinted features through the translucent soldermask; it is drawn only while the layer has a
-  tint.
-- **Components.** Component nodes that share geometry and material become one `InstancedMesh`; an
-  instanced attribute carries each instance's component row into the same state-texture lookup.
-  Runtime models are batched the same way (`src/bodies.ts`); the placeholder batches they replace
-  draw only their remaining instances.
-- **Picking.** `three-mesh-bvh` BVHs per layer mesh, three's instanced ray cast for components.
+  vertex and tints, or discards, the fragment; the texel's index comes from the geometry (merged
+  layer meshes store each vertex's feature ID plus the layer's offset instead of the feature ID).
+  Hover, selection, highlights and hiding are texel writes. Copper and drill layers get a second,
+  overlay draw that shows tinted features through the translucent soldermask; it is drawn only
+  while the layer has a tint.
+- **Materials.** Meshes that render alike share one node material for the whole board
+  (`src/materials.ts`; runtime models by the material's content, as their loaders make copies of a
+  small palette), so three builds each shader once instead of once per mesh. Each has a normal and
+  an x-ray variant: `setXray` swaps the meshes' materials and changes none, and the other mode is
+  compiled (`compileAsync`) in idle time after loading, after runtime models arrive and after a
+  layer is shown, so toggling x-ray rebuilds nothing.
+- **Components.** The asset's component meshes (placeholder bodies, pin-1 markers, embedded user
+  models) are merged into one static mesh per material, in board coordinates, with each vertex's
+  state texel, so a board draws its bodies with a few meshes however many packages it has; the
+  merged copies cost some memory (testcase1, 1,656 components: 67k vertices). Meshes whose copies
+  would add more than 2¹⁸ vertices stay one `InstancedMesh` each, with the texel per instance.
+  Runtime models are instanced the same way (`src/bodies.ts`); placeholders they replace drop out
+  of the merged index buffer.
+- **Picking.** `three-mesh-bvh` BVHs per layer mesh; for components, a box test per instance, then
+  its triangles (three's instanced ray cast for instanced batches).
   Meshes with 50k triangles or more get their BVH built in Web Workers (`src/bvh.worker.ts`, a pool
   of up to three) from copies of their positions and indices, so loading a dense board doesn't
   block the main thread; smaller ones are built in idle time. Until a layer's BVH arrives, hover

packages/viewer/src/board-model.ts

@@ -22,7 +22,7 @@ import {
   DRAWING_ROLES,
   readBoardExtension,
 } from './board-extension.js';
-import { ComponentBatch, ComponentBodies } from './bodies.js';
+import { type BodyGroup, batchBodies, type ComponentBatch, ComponentBodies } from './bodies.js';
 import { FeatureRanges } from './feature-ranges.js';
 import { type ElementKind, featureId, idKind, parseFeatureId } from './ids.js';
 import {
@@ -31,6 +31,7 @@ import {
   parsePropertyTables,
   type StructuralMetadataJson,
 } from './metadata.js';
+import { STATE_ATTRIBUTE } from './state.js';
 
 /** Row reference meaning "none" (spec §8.2). */
 const NONE = 4294967295;
@@ -66,7 +67,7 @@ export interface LayerModel {
   readonly color: string | undefined;
 }
 
-export { ComponentBatch } from './bodies.js';
+export type { ComponentBatch } from './bodies.js';
 
 /** An element resolved to its state texels and bounding box (spec §9). */
 export interface ResolvedElement {
@@ -229,7 +230,7 @@ export class BoardModel {
       const group = new Group();
       group.name = info.id;
       group.visible = kind === 'drill' || (info as BoardLayerJson).visible;
-      const meshes = mergeLayer(info.id, node, ranges);
+      const meshes = mergeLayer(info.id, node, ranges, stateOffset);
       if (meshes.length) group.add(...meshes);
       const layer = {
         id: info.id,
@@ -596,12 +597,12 @@ export class BoardModel {
   }
 
   /**
-   * Groups component meshes by geometry and material into instanced meshes (spec §6.8), and
-   * finds the components whose body is a placeholder: meshes on the node itself with the
+   * Groups component meshes by geometry and material and batches them (spec §6.8, `bodies.ts`),
+   * and finds the components whose body is a placeholder: meshes on the node itself with the
    * materials `boardui/body` and `boardui/pin1` only.
    */
   #batchComponents(nodes: ReadonlyMap<number, Object3D>): ComponentBodies {
-    const groups = new Map<string, { source: Mesh; matrices: Matrix4[]; rows: number[] }>();
+    const groups = new Map<string, BodyGroup & { matrices: Matrix4[]; rows: number[] }>();
     const count = this.components.count;
     const matrices: (Matrix4 | null)[] = new Array(count).fill(null);
     const replaceable = new Uint8Array(count);
@@ -620,24 +621,22 @@ export class BoardModel {
         const key = `${mesh.geometry.uuid}/${material.uuid}`;
         let group = groups.get(key);
         if (!group) {
-          group = { source: mesh, matrices: [], rows: [] };
+          group = { geometry: mesh.geometry, material, matrices: [], rows: [] };
           groups.set(key, group);
         }
         group.matrices.push(mesh.matrixWorld.clone());
         group.rows.push(row);
       });
       replaceable[row] = placeholder && meshes > 0 ? 1 : 0;
     }
-    const batches = [...groups.values()].map(
-      ({ source, matrices, rows }) =>
-        new ComponentBatch(
-          source.geometry,
-          source.material as Material,
-          Uint32Array.from(rows),
-          matrices,
-        ),
+    const batches = batchBodies([...groups.values()], this.componentOffset);
+    return new ComponentBodies(
+      this.componentGroup,
+      batches,
+      this.componentOffset,
+      matrices,
+      replaceable,
     );
-    return new ComponentBodies(this.componentGroup, batches, matrices, replaceable);
   }
 }
 
@@ -654,9 +653,15 @@ function sourceKey(instance: number | null, source: number): number {
 
 /**
  * Merges the primitives of a layer node into one mesh per material, with positions in board
- * coordinates and 32-bit indices, and records the feature ranges (spec §8.1).
+ * coordinates, 32-bit indices and each vertex's state texel (its feature ID plus `stateOffset`,
+ * {@link STATE_ATTRIBUTE}), and records the feature ranges (spec §8.1).
  */
-function mergeLayer(id: string, node: Object3D, ranges: FeatureRanges): Mesh[] {
+function mergeLayer(
+  id: string,
+  node: Object3D,
+  ranges: FeatureRanges,
+  stateOffset: number,
+): Mesh[] {
   const byMaterial = new Map<Material, Mesh[]>();
   node.traverse((object) => {
     const mesh = object as Mesh;
@@ -674,7 +679,7 @@ function mergeLayer(id: string, node: Object3D, ranges: FeatureRanges): Mesh[] {
       indexCount += part.geometry.index?.count ?? count;
     }
     const positions = new Float32Array(vertexCount * 3);
-    const featureIds = new Float32Array(vertexCount);
+    const texels = new Float32Array(vertexCount);
     const index = new Uint32Array(indexCount);
     let v = 0;
     let i = 0;
@@ -688,25 +693,26 @@ function mergeLayer(id: string, node: Object3D, ranges: FeatureRanges): Mesh[] {
         point.fromBufferAttribute(position, k);
         if (transform) point.applyMatrix4(part.matrixWorld);
         point.toArray(positions, (v + k) * 3);
-        featureIds[v + k] = ids.getX(k);
+        texels[v + k] = ids.getX(k) + stateOffset;
       }
       const source = geometry.index;
       const count = source?.count ?? position.count;
       for (let k = 0; k < count; k++) {
         index[i + k] = v + (source ? source.getX(k) : k);
       }
-      ranges.scan(featureIds, positions, index, {
-        vertexStart: v,
-        vertexEnd: v + position.count,
-        indexStart: i,
-        indexEnd: i + count,
-      });
+      ranges.scan(
+        texels,
+        positions,
+        index,
+        { vertexStart: v, vertexEnd: v + position.count, indexStart: i, indexEnd: i + count },
+        stateOffset,
+      );
       v += position.count;
       i += count;
     }
     const geometry = new BufferGeometry();
     geometry.setAttribute('position', new BufferAttribute(positions, 3));
-    geometry.setAttribute('_feature_id_0', new BufferAttribute(featureIds, 1));
+    geometry.setAttribute(STATE_ATTRIBUTE, new BufferAttribute(texels, 1));
     geometry.setIndex(new BufferAttribute(index, 1));
     geometry.computeBoundingBox();
     geometry.computeBoundingSphere();