fix issue of mismatch type for binary on network

AGENTS.md

@@ -174,8 +174,69 @@ enum index → `InvalidCastException` (`Slots<Enum,…>` vs `Slots<int,…>`), N
 on the 2nd distinct key, and any **Guid schema field** → `AmbiguousMatchException` (.NET 10 added a
 `Guid.ToByteArray(bool)` overload; the unqualified `GetMethod("ToByteArray")` in `Persist`/`Transformer` now
 passes `Type.EmptyTypes`). Known gap: multi-column composite **prefix** query values are not per-slot
-normalized (enum/Nullable only supported for the LEADING prefix field); Guid AS an index field still maps to
-`byte[]` (no sentinel → filtered scan).
+normalized (enum/Nullable only supported for the LEADING prefix field).
+
+**Guid / Guid? end-to-end (2026-07, remote-crash chain):** a Guid maps to `DataType.ByteArray` →
+portable CLR `byte[]`; `Guid?` → `Slots(ByteArray)` → wrapper `Slots<byte[]>`. Five stacked bugs fixed
+(each unmasked the next; covered by `RemoteIndexTests` Guid tests):
+1. `DataTransformer.CreateToMethod` pre-`Expression.New(_type2)`d the target — `byte[]` has no default
+   ctor. Now delegates allocation+conversion to `TransformerHelper.BuildBody` (mirrors `CreateFromMethod`).
+2. `BuildBody`'s `Nullable<T>` branch used raw `Expression.Convert` on the inner value (no byte[]↔Guid
+   coercion) — now recurses through `BuildBody` for the inner value.
+3. `TableIndexManager.GetMemberType` returned the `Slots<byte[]>` wrapper for a `Guid?` slot
+   (`TryGetPortableNullableType` only handles value-type inners) → remote filter literal decoded as the
+   wrapper → index-seek cast crash. Now flattens via `NormalizeStorageType` (encodings match:
+   `DataPersist` for Guid / Guid? / byte[] / single-slot Slots<byte[]> all emit the same
+   length-prefixed bytes).
+4. A null wrapper normalized to `default(byte[])` = NULL index key → `GetHashCodeEx` NRE on leaf apply.
+   `SlotAccessor.NormalizeStorageValue` now emits an EMPTY byte[]/"" for null reference-storage values
+   (`NullStorageDefault`) — the byte[] analogue of null→default(T).
+5. Equality seek on a byte[] field matched 0 rows: `Sentinels` had no byte[] entry (now Min=empty
+   array, no Max — like string); `BuildScanFromKeyBuilder`/`BuildPrefixSeekKeyBuilder` filled trailing
+   slots with `Expression.Default` (NULL for byte[]/string PKs, unorderable — now the min sentinel);
+   `BuildFieldEqualityComparer` used `EqualityComparer<byte[]>.Default` = REFERENCE equality (now
+   `BigEndianByteArrayEqualityComparer` content equality, scalar + per-slot). Remaining: byte[] has no
+   MAX sentinel → equality/upper bounds use the min-seek + stop-on-field-change fallback (still a seek).
+
+**Remote `GetIndex` was session-cache-only (2026-07):** `RemoteTableIndexManager.GetIndex` consulted
+only its local cache → a fresh connection saw `null` for an existing server-side index, so an
+"ensure" pattern (`GetIndex ?? CreateIndex`) re-created it → server "already exists" throw. Now
+refreshes via `ListIndexes()` on a cache miss. (`HasIndexes` is still local-cache-only.)
+
+## Automatic Schema Migration (2026-07)
+
+Reopening a table whose RECORD type gained/lost/reordered properties **migrates automatically**
+instead of throwing. `StorageEngine.Obtain` detects `recordDataType != locator.RecordDataType` and
+calls `MigrateRecordSchema` ([StorageEngine.Migration.cs](src/CatDb/Database/StorageEngine.Migration.cs)):
+
+- **Mapping is name-based** (the locator persists top-level name→slot member maps; new names come
+  from the typed record class or the remote client's member map): same name ⇒ copy (slot DataType
+  must be identical), new member ⇒ default value, removed member ⇒ dropped. Works for slot SHIFTS
+  (property inserted in the middle). Without names (untyped portable open) it falls back to
+  positional-prefix (append/truncate only). Values are converted per-slot via
+  `TransformerHelper.BuildBody` when CLR representations differ (POCO Guid ↔ portable byte[], enum…).
+- **Mechanics = table rewrite into a fresh locator** (same name, new id): read all rows in the old
+  layout, transform, `Clear` + soft-delete the old locator AND the table's `__idx_…` index tables
+  (their composite keys embed slots positionally — stale), reinsert. Rows are materialized in RAM
+  during the rewrite (v1); streaming rewrite is the follow-up for huge tables.
+- **NOT migratable (diagnostic `ArgumentException`):** key type changes (keys order the tree —
+  forbidden), primitive (non-Slots) record schemas, and a same-named member whose type changed.
+- **Caveats:** table handles obtained BEFORE the migration (same process) are stale — they point at
+  the soft-deleted locator; reopen to get the live table. A remote client holding the old locator id
+  from before another client's migration is likewise stale until it reopens.
+- Tests: `SchemaMigrationTests` (add-at-end, add-in-middle/slot-shift, remove+add, type-change
+  diagnostic, index rebuild, remote end-to-end).
+
+**Index lifecycle fixes that migration depends on (same change):**
+- `TableIndexManager.BuildEntry` now obtains its backing table via
+  `StorageEngine.ObtainInternalTable` which **REUSES the persisted `__idx_…` locator** when its
+  schema matches → **index data survives process restarts** (previously every `CreateIndex` made a
+  brand-new empty locator: after any restart, EnsureIndex-style code silently produced EMPTY
+  indexes — queries matched nothing). Stale-layout index tables are dropped+recreated.
+- Index tables are enrolled in the engine `_map` → **`Commit` now flushes their buffered ops**
+  (previously they bypassed the map and unflushed index writes could be lost).
+- `CreateIndex` **backfills** from existing main-table rows when the backing table is new
+  (`BackfillIfNeeded` → `RebuildEntry`); a reused persisted index table skips the rebuild.
 
 **Query-value `null` literal desynced the plan-cache parameter slots (2026-07, found by
 `TypedIndexStressService` stress churn):** `QueryCompiler.Signature` (the plan-cache key) is

Directory.Build.props

@@ -21,13 +21,13 @@
          The release workflow (.github/workflows/release.yml) requires the GitHub release tag to equal
          "v$(VersionPrefix)" (or "v$(VersionPrefix)-$(VersionSuffix)" when VersionSuffix is set) and fails
          the release otherwise — bump this FIRST, then tag/publish the matching release. -->
-    <VersionPrefix>2.0.2</VersionPrefix>
+    <VersionPrefix>2.0.3</VersionPrefix>
     <!-- Leave empty for a stable release. Set to e.g. "beta.1" for a prerelease
-         (produces NuGet/assembly version "2.0.2-beta.1"; tag the GitHub release "v2.0.2-beta.1"
+         (produces NuGet/assembly version "2.0.3-beta.1"; tag the GitHub release "v2.0.3-beta.1"
          and mark it as a prerelease). -->
     <VersionSuffix></VersionSuffix>
-    <AssemblyVersion>2.0.2</AssemblyVersion>
-    <FileVersion>2.0.2</FileVersion>
+    <AssemblyVersion>2.0.3</AssemblyVersion>
+    <FileVersion>2.0.3</FileVersion>
 
     <!-- NuGet package metadata shared by all publishable projects -->
     <Authors>Omid Mafakher</Authors>

examples/CatDb.StressTest/Services.cs

@@ -2163,6 +2163,12 @@ protected override async Task ExecuteAsync(CancellationToken ct)
             EnsureIndex("Status", () => _items.CreateIndex("Status", r => r.Status, IndexType.NonUnique));
             EnsureIndex("ExpiresAt", () => _items.CreateIndex("ExpiresAt", r => r.ExpiresAt, IndexType.NonUnique));
 
+            // Seed the shadow with rows persisted by previous runs: index tables are REUSED across
+            // restarts now (and CreateIndex backfills), so index counts cover the WHOLE table. The
+            // old behavior (fresh empty index each start) let an empty shadow pass by accident.
+            foreach (var kv in _items)
+                _shadow[kv.Key] = kv.Value;
+
             _nextId = _items.LastRow?.Key ?? 0; // continue ids past any persisted rows
             for (int i = 0; i < 500; i++)
                 Insert();

src/CatDb/Data/DataTransformer.cs

@@ -43,11 +43,12 @@ public Expression<Func<T, IData>> CreateToMethod()
         }
         else
         {
-            // Different types — allocate T2, copy fields, return as object
+            // Different types — build T2 from the value (BuildBody handles conversion
+            // pairs like Guid↔byte[]/enum↔int and allocates POCOs itself; do NOT
+            // pre-New _type2 — byte[] has no default ctor), return as object.
             var t2   = Expression.Variable(_type2);
             var list = new List<Expression>
             {
-                Expression.Assign(t2, Expression.New(_type2)),
                 TransformerHelper.BuildBody(t2, value, _membersOrder1, _membersOrder2),
                 Expression.Convert(t2, typeof(object))
             };

src/CatDb/Data/Transformer.cs

@@ -165,21 +165,22 @@ public static Expression BuildBody(Expression value1, Expression value2, Func<Ty
 
         if (type1.IsNullable())
         {
-            var data1Var = Expression.Variable(value1.Type);
-            var data2Var = Expression.Variable(value2.Type);
-
-            new List<Expression>();
-
-            var constructParam = Expression.PropertyOrField(data2Var, type2.IsNullable() ? "Value" : DataTypeUtils.GetPublicMembers(type2, membersOrder2).First().Name);
-
-            var block = Expression.Block(new[] { data1Var, data2Var },
+            var data2Var  = Expression.Variable(value2.Type);
+            var innerType = type1.GetGenericArguments()[0];
+            var innerVar  = Expression.Variable(innerType);
+
+            // Source of the wrapped value: Value for a Nullable, else the single slot.
+            var srcInner = Expression.PropertyOrField(data2Var,
+                type2.IsNullable() ? "Value" : DataTypeUtils.GetPublicMembers(type2, membersOrder2).First().Name);
+
+            // Convert the inner value through BuildBody so conversion pairs (Guid↔byte[],
+            // enum↔integral, numeric widening, nested objects) are honored — a raw
+            // Expression.Convert has no coercion between e.g. byte[] and Guid.
+            var block = Expression.Block(new[] { data2Var, innerVar },
                     Expression.Assign(data2Var, value2),
-                    Expression.Assign(data1Var, Expression.New(
-                        type1.GetConstructor(new[] { type1.GetGenericArguments()[0] })!,
-                            constructParam.GetType() == type1.GetGenericArguments()[0] ?
-                            constructParam :
-                            Expression.Convert(constructParam, type1.GetGenericArguments()[0]))),
-                    Expression.Assign(value1, data1Var)
+                    BuildBody(innerVar, srcInner, membersOrder1, membersOrder2),
+                    Expression.Assign(value1, Expression.New(
+                        type1.GetConstructor(new[] { innerType })!, innerVar))
                 );
 
             return Expression.IfThenElse(Expression.NotEqual(value2, Expression.Constant(null, type2)),

src/CatDb/Database/Indexing/Sentinels.cs

@@ -36,6 +36,8 @@ internal static class Sentinels
         [typeof(TimeSpan)] = (TimeSpan.MinValue, TimeSpan.MaxValue),
         // string: ordinal-minimum is "", but there is no maximum string.
         [typeof(string)] = ("", null),
+        // byte[] (Guid fields normalize to it): minimum is the empty array, no maximum.
+        [typeof(byte[])] = (Array.Empty<byte>(), null),
     };
 
     /// <summary>

src/CatDb/Database/Indexing/SlotAccessor.cs

@@ -172,7 +172,12 @@ internal static Func<IData, IData> BuildScanFromKeyBuilder(
             }
         }
 
-        ctorArgs[^1] = Expression.Default(primaryKeyType);
+        // Min sentinel, not Default: for a reference-type PK (byte[] ← Guid, string) default is
+        // NULL, which the composite-key comparer/hasher cannot order — the seek silently matched
+        // nothing. Fall back to Default only for types without a sentinel.
+        ctorArgs[^1] = Sentinels.TryGet(primaryKeyType, max: false, out var pkMin)
+            ? Expression.Constant(pkMin, primaryKeyType)
+            : Expression.Default(primaryKeyType);
 
         var compositeCtor = compositeKeyType.GetConstructors()
             .First(c => c.GetParameters().Length == totalSlots);
@@ -203,30 +208,46 @@ internal static Func<IData, IData, bool> BuildFieldEqualityComparer(Type fieldTy
             Expression? combined = null;
             foreach (var member in members)
             {
-                var memberType      = GetMemberType(member);
-                var accessA         = AccessMember(valueA, member);
-                var accessB         = AccessMember(valueB, member);
-                var slotComparerType = typeof(System.Collections.Generic.EqualityComparer<>).MakeGenericType(memberType);
-                var slotDefaultProp  = slotComparerType.GetProperty("Default", BindingFlags.Static | BindingFlags.Public)!;
-                var slotEqualsMethod = slotComparerType.GetMethod("Equals", [memberType, memberType])!;
-                var slotComparer     = Expression.Property(null, slotDefaultProp);
-                var slotEquals       = Expression.Call(slotComparer, slotEqualsMethod, accessA, accessB);
+                var memberType = GetMemberType(member);
+                var accessA    = AccessMember(valueA, member);
+                var accessB    = AccessMember(valueB, member);
+                var slotEquals = BuildEqualsCall(memberType, accessA, accessB);
                 combined = combined == null ? slotEquals : Expression.AndAlso(combined, slotEquals);
             }
             equalsExpr = combined ?? Expression.Constant(true);
         }
         else
         {
-            var comparerType  = typeof(System.Collections.Generic.EqualityComparer<>).MakeGenericType(fieldType);
-            var defaultProp   = comparerType.GetProperty("Default", BindingFlags.Static | BindingFlags.Public)!;
-            var equalsMethod  = comparerType.GetMethod("Equals", [fieldType, fieldType])!;
-            var comparer      = Expression.Property(null, defaultProp);
-            equalsExpr        = Expression.Call(comparer, equalsMethod, valueA, valueB);
+            equalsExpr = BuildEqualsCall(fieldType, valueA, valueB);
         }
 
         return Expression.Lambda<Func<IData, IData, bool>>(equalsExpr, aParam, bParam).Compile();
     }
 
+    /// <summary>
+    /// Equality call for one (slot) value pair. byte[] (a Guid field's storage type) needs CONTENT
+    /// equality — <c>EqualityComparer&lt;byte[]&gt;.Default</c> is reference equality, which made a
+    /// re-materialized query value never match a stored index key.
+    /// </summary>
+    private static Expression BuildEqualsCall(Type type, Expression a, Expression b)
+    {
+        if (type == typeof(byte[]))
+        {
+            var cmp = Expression.Field(null, typeof(General.Comparers.BigEndianByteArrayEqualityComparer)
+                .GetField(nameof(General.Comparers.BigEndianByteArrayEqualityComparer.Instance))!);
+            return Expression.Call(cmp,
+                typeof(General.Comparers.BigEndianByteArrayEqualityComparer)
+                    .GetMethod(nameof(General.Comparers.BigEndianByteArrayEqualityComparer.Equals), [typeof(byte[]), typeof(byte[])])!,
+                a, b);
+        }
+
+        var comparerType = typeof(System.Collections.Generic.EqualityComparer<>).MakeGenericType(type);
+        var defaultProp  = comparerType.GetProperty("Default", BindingFlags.Static | BindingFlags.Public)!;
+        var equalsMethod = comparerType.GetMethod("Equals", [type, type])!;
+        var comparer     = Expression.Property(null, defaultProp);
+        return Expression.Call(comparer, equalsMethod, a, b);
+    }
+
     internal static Type GetSlotType(Type compositeKeyType, int slotIndex)
         => GetMemberType(GetSlotMember(compositeKeyType, slotIndex));
 
@@ -255,7 +276,10 @@ internal static Func<IData, IData> BuildPrefixSeekKeyBuilder(Type compositeKeyTy
                 args[i] = AccessMember(prefixValue, GetSlotMember(prefixType, i));
         }
         for (var i = prefixLen; i < ps.Length; i++)
-            args[i] = Expression.Default(ps[i].ParameterType);
+            // Min sentinel over Default: default(byte[])/default(string) is NULL and unorderable.
+            args[i] = Sentinels.TryGet(ps[i].ParameterType, max: false, out var min)
+                ? Expression.Constant(min, ps[i].ParameterType)
+                : Expression.Default(ps[i].ParameterType);
 
         var newComposite = Expression.New(ctor, args);
         var castResult   = Expression.Convert(newComposite, typeof(object));
@@ -314,14 +338,29 @@ private static Expression NormalizeStorageValue(Expression access)
             if (!t.IsValueType)
                 return Expression.Condition(
                     Expression.ReferenceEqual(access, Expression.Constant(null, t)),
-                    Expression.Default(inner.Type),
+                    NullStorageDefault(inner.Type),
                     inner);
             return inner;
         }
 
         return access;
     }
 
+    /// <summary>
+    /// Storage stand-in for a null wrapper: <c>default(T)</c> for value types, but a NON-null empty
+    /// value for reference storage types — a null <c>byte[]</c>/<c>string</c> index key would NRE the
+    /// key hash/comparer on leaf apply (portable <c>Guid?</c> normalizes to <c>byte[]</c>, so a null
+    /// Guid must become an empty array, the byte[] analogue of default(T)).
+    /// </summary>
+    private static Expression NullStorageDefault(Type t)
+    {
+        if (t == typeof(byte[]))
+            return Expression.Constant(Array.Empty<byte>(), typeof(byte[]));
+        if (t == typeof(string))
+            return Expression.Constant(string.Empty, typeof(string));
+        return Expression.Default(t);
+    }
+
     /// <summary>
     /// True when <paramref name="t"/> is the portable representation of a <see cref="Nullable{T}"/>
     /// (or any single-field composite): a one-slot <c>ISlots</c>. Multi-slot composites (real composite

src/CatDb/Database/Indexing/TableIndexManager.Query.cs

@@ -102,7 +102,7 @@
         return ScanPrefixKeys(entry, prefixValue, prefixFieldCount, backward);
     }
 
     bool IQueryEngineContext.TryFetch(IData pk, out IData record) => _table.TryGet(pk, out record);
 
     IEnumerable<KeyValuePair<IData, IData>> IQueryEngineContext.ScanRows(
         IData? from, bool hasFrom, IData? to, bool hasTo, bool backward)
@@ -146,7 +146,16 @@
         // same CLR type the client encoded it with (typeof(TField)); otherwise `.PrimitiveType` throws
         // "The type (T) is not primitive" on the wrapper.
         var clr = DataTypeUtils.BuildType(slotDt);
-        return SlotAccessor.TryGetPortableNullableType(clr, out var nullable) ? nullable : clr;
+        if (SlotAccessor.TryGetPortableNullableType(clr, out var nullable))
+            return nullable;
+
+        // Reference-type inner (Guid? → Slots(ByteArray) → Slots<byte[]>): TryGetPortableNullableType
+        // only handles value-type inners, so flatten the wrapper to its inner storage type here.
+        // The client encodes the FLAT value (DataPersist(Guid)/(Guid?) and (byte[]) share the same
+        // length-prefixed layout), and every downstream consumer (value normalizer, seek bounds,
+        // residual/sort comparers) works on the flat storage type — decoding as the Slots wrapper
+        // instead produced a Slots<byte[]> instance that the index seek then failed to cast to byte[].
+        return SlotAccessor.NormalizeStorageType(clr);
     }
 
     /// <summary>CLR type of the primary key (for remote key-range decoding).</summary>