From 5f130abf5aab3151b4872e1b15626497b7f9f7c9 Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 04:28:37 +0300 Subject: [PATCH 01/22] fix(policies): a projected row keeps its members, and a denial beneath one is enforced DCMP's second change request (DW-12, DW-13, DW-15), and three fail-opens found while checking it. DW-12. With a select-denied field and no Selects, the projection synthesized for a guarded query kept scalars only, so every nested object and list of a row projected before ApplyPolicy came back null or empty. A synthesized projection now carries what an unguarded call would return, less what the policy withholds. A member holding a value is kept when allowed, and so is a collection of values (byte[], List). A member holding an object, or a list of them, is kept whole when nothing beneath it is denied, and narrowed as a caller naming it would have it narrowed when something is. That applies where the source carries the member: every member of a projected or in-memory row, and an entity's owned and complex members, read from the EF Core model. An entity's other navigations are left out, as before. What a source carries is read from the query once (RowShape), so the sanitizer stays pure. F1, both tiers. A denial only beneath a member synthesized nothing, so the whole row came back and a projected list, an object in memory, an included navigation or an owned member carried the denied value out. Everything beneath every object member is now gated, whether or not the source carries it. F2, Convenience. Naming a navigation whose element key is denied narrowed the key away, and the core's projection builder added it back. That narrowing is now refused in both tiers, as naming a sibling of the key already was. A synthesized projection leaves such a member out whole. F3, both tiers. The projection gate read collections through a narrower list than the attribute walker, so a member typed IReadOnlyList hid every denial beneath it. The gate now reads a type the way the walker does (AttributePolicyProvider.Peeled / NavigationTypeOf). A narrowing the core cannot validate is refused, and a synthesized projection leaves the member out whole. A member in memory or an EF Core complex property is never narrowed, since the core's narrowing needs EF Core and compares the member to null. DW-13. A projection hid the declared default order whatever it assigned. A Select that builds the type in an initializer and assigns every field the default names, at every level of a nested path, now takes the default. A Select, or a Filter carrying a projection, composed on the guarded handle keeps the chain unordered. A composed Filter whose orders were all dropped gets no default later in the chain, as a composed Order already did not. DW-15. Every async terminal, guarded and unguarded, has overloads taking a CancellationToken: Filter, dynamic Filter, Summary and Segment. They are overloads, not an optional parameter, so code compiled against 3.1 still binds. The token reaches the count and the read. A dynamic or grouped read on EF Core now runs through EF Core's asynchronous operators, reached by reflection (AsyncReads), instead of a synchronous read on a pool thread. The EF Core query-root finder moves from SegmentComposer to QueryRoot so both can read the model. No frozen file is touched. Each security fix was mutation-checked: removing it turns the new tests red. EF Core 8 leg: 2102 passed. EF Core 6.0.22 leg: 1409 passed. A DCMP-shaped probe on EF Core 9, Npgsql 9 and PostgreSQL 17 gives the answers DCMP's acceptance asks for. Co-Authored-By: Claude Opus 5 --- DynamicWhere.Tests/CancellationTests.cs | 273 +++++++ DynamicWhere.Tests/ComplexMemberTests.cs | 128 ++++ DynamicWhere.Tests/DefaultOrderTests.cs | 119 +++ DynamicWhere.Tests/DynamicWhere.Tests.csproj | 4 + .../Policies/ProjectedRowTests.cs | 686 ++++++++++++++++++ .../Resolution/AttributePolicyProvider.cs | 17 +- .../Policies/Source/DefaultOrder.cs | 97 ++- .../Policies/Source/FilterSanitizer.cs | 331 +++++++-- .../Policies/Source/PolicyQueryable.cs | 123 +++- DynamicWhere.ex/Policies/Source/RowShape.cs | 156 ++++ DynamicWhere.ex/Source/AsyncReads.cs | 77 ++ DynamicWhere.ex/Source/Extention.cs | 129 +++- DynamicWhere.ex/Source/QueryRoot.cs | 64 ++ DynamicWhere.ex/Source/SegmentComposer.cs | 50 +- 14 files changed, 2108 insertions(+), 146 deletions(-) create mode 100644 DynamicWhere.Tests/CancellationTests.cs create mode 100644 DynamicWhere.Tests/ComplexMemberTests.cs create mode 100644 DynamicWhere.Tests/Policies/ProjectedRowTests.cs create mode 100644 DynamicWhere.ex/Policies/Source/RowShape.cs create mode 100644 DynamicWhere.ex/Source/AsyncReads.cs create mode 100644 DynamicWhere.ex/Source/QueryRoot.cs diff --git a/DynamicWhere.Tests/CancellationTests.cs b/DynamicWhere.Tests/CancellationTests.cs new file mode 100644 index 0000000..fa4ebff --- /dev/null +++ b/DynamicWhere.Tests/CancellationTests.cs @@ -0,0 +1,273 @@ +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Classes.Core; +using DynamicWhere.ex.Classes.Result; +using DynamicWhere.ex.Enums; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using DynamicWhere.ex.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Diagnostics; +using System.Data.Common; + +namespace DynamicWhere.Tests; + +public class CancelledRow +{ + public int Id { get; set; } + + public string Team { get; set; } = string.Empty; + + public int Points { get; set; } +} + +public sealed class CancellationContext : DbContext +{ + private readonly SqliteConnection _connection; + private readonly IInterceptor[] _interceptors; + + public CancellationContext(SqliteConnection connection, params IInterceptor[] interceptors) + { + _connection = connection; + _interceptors = interceptors; + } + + public DbSet Rows => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => + options.UseSqlite(_connection).AddInterceptors(_interceptors); +} + +/// +/// Cancels a token just before the numbered command runs, and counts the commands that finished, so a +/// test can tell which read the token reached. +/// +public sealed class CancelBeforeCommand : DbCommandInterceptor +{ + private readonly CancellationTokenSource _source; + private readonly int _at; + private int _started; + + public CancelBeforeCommand(CancellationTokenSource source, int at) + { + _source = source; + _at = at; + } + + public int Finished { get; private set; } + + public override ValueTask> ReaderExecutingAsync( + DbCommand command, + CommandEventData eventData, + InterceptionResult result, + CancellationToken cancellationToken = default) + { + if (++_started == _at) + { + _source.Cancel(); + } + + return base.ReaderExecutingAsync(command, eventData, result, cancellationToken); + } + + public override ValueTask ReaderExecutedAsync( + DbCommand command, + CommandExecutedEventData eventData, + DbDataReader result, + CancellationToken cancellationToken = default) + { + Finished++; + + return base.ReaderExecutedAsync(command, eventData, result, cancellationToken); + } +} + +/// +/// Every asynchronous terminal takes a , guarded or not, and hands it to +/// the provider. +/// +/// +/// Before 3.2.0 no method took one, so a request its client had abandoned could not cancel its read. +/// The overloads sit beside the ones without a token rather than replacing them: a parameter added to +/// an existing method changes its signature, which a caller compiled against the old one cannot find. +/// +/// Runs on the EF Core 6 leg too, because a dynamic read reaches EF Core's own asynchronous operators by +/// reflection, and a method found by name is exactly what differs between versions. +/// +/// +public sealed class CancellationTests : IDisposable +{ + private readonly SqliteConnection _connection; + private readonly CancellationContext _db; + + public CancellationTests() + { + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + + _db = new CancellationContext(_connection); + _db.Database.EnsureCreated(); + + for (int i = 1; i <= 6; i++) + { + _db.Rows.Add(new CancelledRow { Id = i, Team = i % 2 == 0 ? "even" : "odd", Points = i * 10 }); + } + + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static CancellationToken Canceled() + { + using CancellationTokenSource source = new(); + + source.Cancel(); + + return source.Token; + } + + private static Filter Paged() => new() + { + Orders = new List { new() { Sort = 1, Field = "Id", Direction = Direction.Ascending } }, + Page = new PageBy { PageNumber = 1, PageSize = 4 } + }; + + private static Filter PagedDynamic() + { + Filter filter = Paged(); + + filter.Selects = new List { "Id", "Team" }; + + return filter; + } + + private static Summary ByTeam() => new() + { + GroupBy = new GroupBy + { + Fields = new List { "Team" }, + AggregateBy = new List + { + new() { Field = "Points", Alias = "Total", Aggregator = Aggregator.Sumation } + } + }, + Orders = new List { new() { Sort = 1, Field = "Team", Direction = Direction.Ascending } } + }; + + private static Segment OddOrFirst() => new() + { + ConditionSets = + { + new ConditionSet + { + Sort = 1, + ConditionGroup = new ConditionGroup + { + Conditions = { new Condition { Field = "Team", DataType = DataType.Text, Operator = Operator.Equal, Values = { "odd" } } } + } + }, + new ConditionSet + { + Sort = 2, + Intersection = Intersection.Union, + ConditionGroup = new ConditionGroup + { + Conditions = { new Condition { Field = "Id", DataType = DataType.Number, Operator = Operator.Equal, Values = { 2 } } } + } + } + } + }; + + private PolicyQueryable Guarded() => + _db.Rows.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = DwTier.Strict, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + [Fact] + public async Task A_canceled_token_stops_every_unguarded_async_terminal() + { + CancellationToken canceled = Canceled(); + + await Assert.ThrowsAnyAsync(() => _db.Rows.ToListAsync(Paged(), canceled)); + await Assert.ThrowsAnyAsync(() => _db.Rows.ToListAsync(Paged(), true, canceled)); + await Assert.ThrowsAnyAsync(() => _db.Rows.ToListAsyncDynamic(PagedDynamic(), canceled)); + await Assert.ThrowsAnyAsync(() => _db.Rows.ToListAsyncDynamic(PagedDynamic(), true, canceled)); + await Assert.ThrowsAnyAsync(() => _db.Rows.ToListAsync(ByTeam(), canceled)); + await Assert.ThrowsAnyAsync(() => _db.Rows.ToListAsync(ByTeam(), true, canceled)); + await Assert.ThrowsAnyAsync(() => _db.Rows.ToListAsync(OddOrFirst(), canceled)); + } + + [Fact] + public async Task A_canceled_token_stops_every_guarded_async_terminal() + { + CancellationToken canceled = Canceled(); + + await Assert.ThrowsAnyAsync(() => Guarded().ToListAsync(Paged(), canceled)); + await Assert.ThrowsAnyAsync(() => Guarded().ToListAsyncDynamic(PagedDynamic(), canceled)); + await Assert.ThrowsAnyAsync(() => Guarded().ToListAsync(ByTeam(), canceled)); + await Assert.ThrowsAnyAsync(() => Guarded().ToListAsync(OddOrFirst(), canceled)); + } + + /// + /// Each terminal counts, then reads. Canceling just before the first command shows the count took the + /// token; canceling just before the second shows the read did, after a count that finished. + /// + [Theory] + [InlineData(1)] + [InlineData(2)] + public async Task The_token_reaches_both_the_count_and_the_read(int at) + { + async Task Check(Func run) + { + using CancellationTokenSource source = new(); + CancelBeforeCommand interceptor = new(source, at); + using CancellationContext db = new(_connection, interceptor); + + await Assert.ThrowsAnyAsync(() => run(db, source.Token)); + Assert.Equal(at - 1, interceptor.Finished); + } + + await Check((db, token) => db.Rows.ToListAsync(Paged(), token)); + await Check((db, token) => db.Rows.ToListAsyncDynamic(PagedDynamic(), token)); + await Check((db, token) => db.Rows.ToListAsync(ByTeam(), token)); + await Check((db, token) => db.Rows.ToListAsync(OddOrFirst(), token)); + } + + /// A token that is never canceled changes nothing: each overload answers as the one without a token does. + [Fact] + public async Task A_live_token_returns_what_the_overload_without_one_returns() + { + using CancellationTokenSource live = new(); + + FilterResult plain = await _db.Rows.ToListAsync(Paged()); + FilterResult tokened = await _db.Rows.ToListAsync(Paged(), live.Token); + Assert.Equal(plain.Data.Select(row => row.Id), tokened.Data.Select(row => row.Id)); + Assert.Equal((plain.TotalCount, plain.PageCount), (tokened.TotalCount, tokened.PageCount)); + + FilterResult dynamicRows = await _db.Rows.ToListAsyncDynamic(PagedDynamic(), live.Token); + Assert.Equal(new[] { 1, 2, 3, 4 }, dynamicRows.Data.Select(row => (int)row.Id)); + Assert.Equal(6, dynamicRows.TotalCount); + + SummaryResult summary = await _db.Rows.ToListAsync(ByTeam(), live.Token); + Assert.Equal(2, summary.TotalCount); + Assert.Equal(new[] { 120, 90 }, summary.Data.Select(row => (int)row.Total)); + + SegmentResult segment = await _db.Rows.ToListAsync(OddOrFirst(), live.Token); + Assert.Equal(new[] { 1, 2, 3, 5 }, segment.Data!.Select(row => row.Id).OrderBy(id => id)); + + Assert.Equal(new[] { 1, 2, 3, 4 }, (await Guarded().ToListAsync(Paged(), live.Token)).Data.Select(row => row.Id)); + Assert.Equal(new[] { 1, 2, 3, 4 }, (await Guarded().ToListAsyncDynamic(PagedDynamic(), live.Token)).Data.Select(row => (int)row.Id)); + Assert.Equal(new[] { 120, 90 }, (await Guarded().ToListAsync(ByTeam(), live.Token)).Data.Select(row => (int)row.Total)); + Assert.Equal(4, (await Guarded().ToListAsync(OddOrFirst(), live.Token)).TotalCount); + } +} diff --git a/DynamicWhere.Tests/ComplexMemberTests.cs b/DynamicWhere.Tests/ComplexMemberTests.cs new file mode 100644 index 0000000..c5b1495 --- /dev/null +++ b/DynamicWhere.Tests/ComplexMemberTests.cs @@ -0,0 +1,128 @@ +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; + +namespace DynamicWhere.Tests; + +/// An entity with a denied scalar and a complex property with nothing denied beneath it. +public class CxCrate +{ + public int Id { get; set; } + + [DwDenied] + public string? Label { get; set; } + + public CxSize Size { get; set; } = new(); +} + +public class CxSize +{ + public int Width { get; set; } + + public int Height { get; set; } +} + +public class CxSecretSize +{ + public int Width { get; set; } + + [DwDenied] + public int Height { get; set; } +} + +/// An entity whose only denial sits inside its complex property. +public class CxSealedBox +{ + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public CxSecretSize Inner { get; set; } = new(); +} + +public sealed class ComplexMemberContext : DbContext +{ + private readonly SqliteConnection _connection; + + public ComplexMemberContext(SqliteConnection connection) => _connection = connection; + + public DbSet Crates => Set(); + + public DbSet Boxes => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + protected override void OnModelCreating(ModelBuilder model) + { + model.Entity().ComplexProperty(crate => crate.Size); + model.Entity().ComplexProperty(box => box.Inner); + } +} + +/// +/// An EF Core complex property, which EF Core 8 loads with the entity on every query. A synthesized +/// projection keeps it whole, and leaves it out whole when something beneath it is denied: the core +/// narrows a nested object behind a comparison with null, which EF Core refuses for a complex type. +/// +/// Not on the EF Core 6 leg, which has no complex properties. +public sealed class ComplexMemberTests : IDisposable +{ + private readonly SqliteConnection _connection; + private readonly ComplexMemberContext _db; + + public ComplexMemberTests() + { + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + + _db = new ComplexMemberContext(_connection); + _db.Database.EnsureCreated(); + + _db.Crates.Add(new CxCrate { Label = "label-secret", Size = new CxSize { Width = 4, Height = 5 } }); + _db.Boxes.Add(new CxSealedBox { Name = "B1", Inner = new CxSecretSize { Width = 2, Height = 9 } }); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static PolicyQueryable Guard(IQueryable source) where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = DwTier.Strict }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + [Fact] + public void A_complex_property_with_nothing_denied_beneath_is_kept_whole() + { + CxCrate crate = Guard(_db.Crates).ToList(new Filter()).Data.Single(); + + Assert.Null(crate.Label); + Assert.Equal((4, 5), (crate.Size.Width, crate.Size.Height)); + } + + [Fact] + public void A_complex_property_with_a_denial_beneath_is_left_out_whole() + { + PolicyQueryable guarded = Guard(_db.Boxes); + + CxSealedBox box = guarded.ToList(new Filter()).Data.Single(); + + Assert.Equal("B1", box.Name); + Assert.Equal((0, 0), (box.Inner.Width, box.Inner.Height)); + Assert.Contains( + guarded.LastTrace!.Decisions, + decision => decision.FieldPath == "Inner" + && decision.Reason == "left out whole: it is a complex property, which EF Core cannot narrow"); + } +} diff --git a/DynamicWhere.Tests/DefaultOrderTests.cs b/DynamicWhere.Tests/DefaultOrderTests.cs index 80469f3..e7a0709 100644 --- a/DynamicWhere.Tests/DefaultOrderTests.cs +++ b/DynamicWhere.Tests/DefaultOrderTests.cs @@ -108,6 +108,20 @@ public class AuditedTicket public int Rank { get; set; } } +/// A default reaching through a reference, for the projections that do and do not assign it. +[DwEntity(DefaultOrder = "Owner.Name desc, Id")] +public class OwnedTicket +{ + public int Id { get; set; } + + public TicketOwner? Owner { get; set; } +} + +public class TicketOwner +{ + public string Name { get; set; } = string.Empty; +} + /// A type that declares no default, and so is never ordered by one. public class PlainTicket { @@ -537,6 +551,111 @@ public void A_denial_a_rule_can_lift_is_a_warning_and_so_is_one_for_segments() unsegmented.Warnings); } + // ------------------------------------------------------------------- a projected source (3.2.0) + + /// + /// DCMP's A9. A projection that builds the type in an initializer and assigns every field the + /// default names hides nothing, so the default applies: EF Core translates the order through the + /// members the projection assigned. It used to be left unordered, like every projection. + /// + [Fact] + public async Task A_projection_that_assigns_every_default_field_takes_the_default() + { + IQueryable rows = _db.Tickets.Select(t => new Ticket { Id = t.Id, Priority = t.Priority, Title = t.Title }); + + PolicyQueryable guarded = rows.ApplyPolicy(Caller(), Options(), Resolver()); + + Assert.Equal(ByDefault, Ids(guarded.ToList(new Filter()).Data)); + Assert.Equal(ByDefault, Ids((await guarded.ToListAsync(new Filter())).Data)); + Assert.Equal(new[] { 2, 4 }, Ids(guarded.Page(FirstTwo()).AsUnguardedQueryable().ToList())); + Assert.Equal(new[] { 2, 4, 3, 5 }, Ids((await guarded.ToListAsync(UrgentOrFive())).Data!)); + } + + /// A projection that leaves out a field the default names keeps whatever order it had. + [Fact] + public void A_projection_that_leaves_a_default_field_out_keeps_its_own_order() + { + IQueryable rows = _db.Tickets.Select(t => new Ticket { Id = t.Id, Title = t.Title }); + + Assert.Equal(AsWritten, Ids(rows.ApplyPolicy(Caller(), Options(), Resolver()).ToList(new Filter()).Data)); + } + + /// DCMP's A10 and A11: a source ordered before its projection keeps that order, and a caller's order wins. + [Fact] + public void A_source_ordered_before_its_projection_keeps_that_order_and_the_callers_order_wins() + { + IQueryable rows = _db.Tickets.OrderBy(t => t.Title) + .Select(t => new Ticket { Id = t.Id, Priority = t.Priority, Title = t.Title }); + + Assert.Equal(AsWritten, Ids(rows.ApplyPolicy(Caller(), Options(), Resolver()).ToList(new Filter()).Data)); + Assert.Equal( + new[] { 5, 4, 3, 2, 1 }, + Ids(rows.ApplyPolicy(Caller(), Options(), Resolver()) + .ToList(new Filter { Orders = new List { By("Id", Direction.Descending) } }).Data)); + } + + /// + /// The default is for the rows the caller's source makes. A projection the caller composes on the + /// guarded handle afterwards stays in its own order, even when it keeps every field the default names. + /// + [Fact] + public void A_projection_composed_after_ApplyPolicy_stays_unordered_even_with_every_default_field() + { + PolicyQueryable projected = Guarded().Select(new List { "Id", "Priority" }); + + Assert.Equal(new[] { 1, 2 }, Ids(projected.Page(FirstTwo()).AsUnguardedQueryable().ToList())); + Assert.Equal(AsWritten, Ids(projected.ToList(new Filter()).Data)); + } + + /// + /// A composed Filter whose orders were all dropped sent orders, so nothing later in the chain adds a + /// default in their place, exactly as a composed Order whose fields were all dropped does. + /// + [Fact] + public void A_filter_composed_with_orders_that_were_all_dropped_gets_no_default_later() + { + PolicyQueryable guarded = Guarded( + new FakePolicyProvider().Add("Title", PolicyFeature.Order, PolicyEffect.Deny, PolicyLevel.DynamicGlobal)); + + PolicyQueryable filtered = guarded.Filter(new Filter { Orders = new List { By("Title") } }); + + Assert.Equal(new[] { 1, 2 }, Ids(filtered.Page(FirstTwo()).AsUnguardedQueryable().ToList())); + } + + /// + /// A nested default is assigned only when every level of its path is an initializer that assigns + /// the next member. Any other expression along the way, or a member left out, hides it. + /// + [Fact] + public void A_nested_default_is_assigned_only_through_initializers() + { + OwnedTicket[] source = + { + new() { Id = 1, Owner = new TicketOwner { Name = "b" } }, + new() { Id = 2, Owner = new TicketOwner { Name = "c" } }, + new() { Id = 3, Owner = new TicketOwner { Name = "a" } } + }; + + IQueryable initialized = source.AsQueryable() + .Select(t => new OwnedTicket { Id = t.Id, Owner = new TicketOwner { Name = t.Owner!.Name } }); + IQueryable assignedWhole = source.AsQueryable() + .Select(t => new OwnedTicket { Id = t.Id, Owner = t.Owner }); + IQueryable leftOut = source.AsQueryable() + .Select(t => new OwnedTicket { Id = t.Id }); + + Assert.False(DefaultOrder.HidesDefault(initialized.Expression, typeof(OwnedTicket))); + Assert.True(DefaultOrder.HidesDefault(assignedWhole.Expression, typeof(OwnedTicket))); + Assert.True(DefaultOrder.HidesDefault(leftOut.Expression, typeof(OwnedTicket))); + Assert.False(DefaultOrder.HidesDefault(source.AsQueryable().Expression, typeof(OwnedTicket))); + + Assert.Equal( + new[] { 2, 1, 3 }, + initialized.ApplyPolicy(Caller(), Options(), Resolver()).ToList(new Filter()).Data.Select(row => row.Id)); + Assert.Equal( + new[] { 1, 2, 3 }, + assignedWhole.ApplyPolicy(Caller(), Options(), Resolver()).ToList(new Filter()).Data.Select(row => row.Id)); + } + [Fact] public void An_audited_default_field_is_recorded_only_when_the_query_orders_by_it() { diff --git a/DynamicWhere.Tests/DynamicWhere.Tests.csproj b/DynamicWhere.Tests/DynamicWhere.Tests.csproj index 679f213..dd49667 100644 --- a/DynamicWhere.Tests/DynamicWhere.Tests.csproj +++ b/DynamicWhere.Tests/DynamicWhere.Tests.csproj @@ -79,6 +79,10 @@ primary key it reads from the EF Core model, both of which differ between 6 and 8. It carries its own SQLite model. --> + + diff --git a/DynamicWhere.Tests/Policies/ProjectedRowTests.cs b/DynamicWhere.Tests/Policies/ProjectedRowTests.cs new file mode 100644 index 0000000..0ec14ff --- /dev/null +++ b/DynamicWhere.Tests/Policies/ProjectedRowTests.cs @@ -0,0 +1,686 @@ +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Classes.Core; +using DynamicWhere.ex.Classes.Result; +using DynamicWhere.ex.Enums; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.DTOs; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using DynamicWhere.ex.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; + +namespace DynamicWhere.Tests.Policies; + +// ------------------------------------------------------------------------------------ the database + +/// A role, scoped to a tenant or, with none, a platform template. +public class PrRole +{ + public int Id { get; set; } + + public string Code { get; set; } = string.Empty; + + public string NameAr { get; set; } = string.Empty; + + public string NameEn { get; set; } = string.Empty; + + public int? TenantId { get; set; } +} + +/// A permission, with a value no row built from it may carry out. +public class PrPermission +{ + public int Id { get; set; } + + public string Code { get; set; } = string.Empty; + + public string Secret { get; set; } = string.Empty; +} + +/// A permission granted to a role. +public class PrGrant +{ + public int Id { get; set; } + + public int RoleId { get; set; } + + public int PermissionId { get; set; } + + public PrPermission Permission { get; set; } = null!; +} + +/// An entity with nothing denied at the top: every denial sits beneath a member. +public class PrShelf +{ + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + /// Owned, so EF Core loads it with every shelf. + public PrLocation? Location { get; set; } + + /// Owned as well, and a collection. + public List Tags { get; set; } = new(); + + /// A navigation, loaded only when something includes it. + public List Books { get; set; } = new(); +} + +public class PrLocation +{ + public string Aisle { get; set; } = string.Empty; + + [DwDenied] + public string? Code { get; set; } +} + +public class PrTag +{ + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + [DwDenied] + public string? Hidden { get; set; } +} + +public class PrBook +{ + public int Id { get; set; } + + public int PrShelfId { get; set; } + + public string Title { get; set; } = string.Empty; + + [DwDenied] + public string? Isbn { get; set; } +} + +/// An entity with a denied scalar, an owned member with nothing denied beneath it, and a blob. +public class PrBin +{ + public int Id { get; set; } + + [DwDenied] + public string? Label { get; set; } + + public PrPlace? Place { get; set; } + + public byte[] Photo { get; set; } = Array.Empty(); +} + +public class PrPlace +{ + public string Aisle { get; set; } = string.Empty; + + public int Row { get; set; } +} + +/// An entity whose children's key no caller may see. +public class PrKeyParent +{ + public int Id { get; set; } + + public List Kids { get; set; } = new(); +} + +public class PrKeyChild +{ + [DwDenied] + public int Id { get; set; } + + public int PrKeyParentId { get; set; } + + public string Name { get; set; } = string.Empty; +} + +public sealed class ProjectedRowContext : DbContext +{ + private readonly SqliteConnection _connection; + + public ProjectedRowContext(SqliteConnection connection) => _connection = connection; + + public DbSet Roles => Set(); + + public DbSet Permissions => Set(); + + public DbSet Grants => Set(); + + public DbSet Shelves => Set(); + + public DbSet Bins => Set(); + + public DbSet KeyParents => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + protected override void OnModelCreating(ModelBuilder model) + { + model.Entity().OwnsOne(shelf => shelf.Location); + model.Entity().OwnsMany(shelf => shelf.Tags, tag => tag.HasKey(t => t.Id)); + model.Entity().OwnsOne(bin => bin.Place); + } +} + +// ------------------------------------------------------------------------------------ the rows + +/// DCMP's role row: a denied, forced tenant beside a nested object, a list of objects and a list of values. +[DwEntity(RequirePolicy = true)] +public sealed class PrRoleRow +{ + public int Id { get; set; } + + public string Code { get; set; } = string.Empty; + + [DwDenied, DwForceWhere(Operator.Equal, ContextValue = "TenantId", AllowNull = true)] + public int? TenantId { get; set; } + + [DwNoWhere, DwNoOrder] + public PrName? Name { get; set; } + + [DwNoWhere, DwNoOrder] + public List Contents { get; set; } = new(); + + [DwNoWhere, DwNoOrder] + public List Codes { get; set; } = new(); +} + +public sealed class PrName +{ + public string Ar { get; set; } = string.Empty; + + public string En { get; set; } = string.Empty; +} + +public sealed class PrGrantRow +{ + public string Code { get; set; } = string.Empty; + + public string NameEn { get; set; } = string.Empty; +} + +/// A row with nothing denied at the top and a denial inside its list. +[DwEntity(RequirePolicy = true)] +public sealed class PrOpenRow +{ + public int Id { get; set; } + + public string Code { get; set; } = string.Empty; + + [DwNoWhere, DwNoOrder] + public List Contents { get; set; } = new(); +} + +public sealed class PrSecretGrantRow +{ + public string Code { get; set; } = string.Empty; + + [DwDenied] + public string? Secret { get; set; } +} + +/// A row whose list elements carry a key no caller may see, which the core adds to any narrowing. +[DwEntity(RequirePolicy = true)] +public sealed class PrKeyedRow +{ + public int Id { get; set; } + + [DwNoWhere, DwNoOrder] + public List Contents { get; set; } = new(); +} + +public sealed class PrKeyedGrantRow +{ + [DwDenied] + public int Id { get; set; } + + public string Code { get; set; } = string.Empty; +} + +/// A row whose list is typed as a collection the core does not unwrap. +[DwEntity(RequirePolicy = true)] +public sealed class PrReadOnlyRow +{ + public int Id { get; set; } + + [DwNoWhere, DwNoOrder] + public IReadOnlyList Contents { get; set; } = new List(); + + [DwNoWhere, DwNoOrder] + public IReadOnlyList Clean { get; set; } = new List(); +} + +/// +/// A guarded query over rows its source builds, and over an entity whose denials sit beneath its +/// members: what a caller who sends no projection receives, and what one who names a member does. +/// +/// +/// DCMP projects every read into its own row type before guarding it, and a row there carries a +/// select-denied tenant. The projection the library synthesized for that denial kept scalars only, so +/// every nested object and list came back empty. Checking it found that a denial only beneath a member +/// synthesized nothing at all, and the whole row, denied values included, came back. +/// +public sealed class ProjectedRowTests : IDisposable +{ + private readonly SqliteConnection _connection; + private readonly ProjectedRowContext _db; + + public ProjectedRowTests() + { + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + + _db = new ProjectedRowContext(_connection); + _db.Database.EnsureCreated(); + + PrPermission read = new() { Code = "read", Secret = "s-read" }; + PrPermission write = new() { Code = "write", Secret = "s-write" }; + PrRole ownRole = new() { Code = "R1", NameAr = "دور", NameEn = "Role one", TenantId = 1 }; + PrRole otherRole = new() { Code = "R2", NameAr = "دور", NameEn = "Role two", TenantId = 2 }; + PrRole template = new() { Code = "T0", NameAr = "قالب", NameEn = "Template", TenantId = null }; + + _db.AddRange(read, write, ownRole, otherRole, template); + _db.SaveChanges(); + + _db.Grants.AddRange( + new PrGrant { RoleId = ownRole.Id, PermissionId = read.Id }, + new PrGrant { RoleId = ownRole.Id, PermissionId = write.Id }, + new PrGrant { RoleId = otherRole.Id, PermissionId = read.Id }, + new PrGrant { RoleId = template.Id, PermissionId = read.Id }); + + _db.Shelves.Add(new PrShelf + { + Name = "S1", + Location = new PrLocation { Aisle = "A7", Code = "location-secret" }, + Tags = { new PrTag { Name = "t1", Hidden = "tag-secret" } }, + Books = { new PrBook { Title = "B1", Isbn = "isbn-secret" } } + }); + + _db.Bins.Add(new PrBin + { + Label = "label-secret", Place = new PrPlace { Aisle = "B2", Row = 3 }, Photo = new byte[] { 1, 2, 3 } + }); + + _db.KeyParents.Add(new PrKeyParent { Kids = { new PrKeyChild { Name = "k1" } } }); + + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static DwPolicyContext Caller() => + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1").WithValue("TenantId", 1); + + private static DwPolicyOptions Options(DwTier tier, bool dryRun = false) => + new() { Tier = tier, DryRun = dryRun }; + + private static PolicyResolver Resolver() => new(new IDwPolicyProvider[] { new AttributePolicyProvider() }); + + private static PolicyQueryable Guard(IQueryable source, DwTier tier, bool dryRun = false) where T : class => + source.ApplyPolicy(Caller(), Options(tier, dryRun), Resolver()); + + private static Filter Everything() => new(); + + private static Filter Selecting(params string[] fields) => new() { Selects = fields.ToList() }; + + private IQueryable RoleRows() => _db.Roles.AsNoTracking().Select(role => new PrRoleRow + { + Id = role.Id, + Code = role.Code, + TenantId = role.TenantId, + Name = new PrName { Ar = role.NameAr, En = role.NameEn }, + Contents = _db.Grants + .Where(grant => grant.RoleId == role.Id) + .OrderBy(grant => grant.Permission.Code) + .Select(grant => new PrGrantRow { Code = grant.Permission.Code, NameEn = grant.Permission.Code + "!" }) + .ToList(), + Codes = _db.Grants + .Where(grant => grant.RoleId == role.Id) + .Select(grant => grant.Permission.Code) + .OrderBy(code => code) + .ToList() + }); + + private IQueryable OpenRows() => _db.Roles.AsNoTracking().Select(role => new PrOpenRow + { + Id = role.Id, + Code = role.Code, + Contents = _db.Grants + .Where(grant => grant.RoleId == role.Id) + .OrderBy(grant => grant.Permission.Code) + .Select(grant => new PrSecretGrantRow { Code = grant.Permission.Code, Secret = grant.Permission.Secret }) + .ToList() + }); + + private IQueryable KeyedRows() => _db.Roles.AsNoTracking().Select(role => new PrKeyedRow + { + Id = role.Id, + Contents = _db.Grants + .Where(grant => grant.RoleId == role.Id) + .Select(grant => new PrKeyedGrantRow { Id = grant.Permission.Id, Code = grant.Permission.Code }) + .ToList() + }); + + private IQueryable ReadOnlyRows() => _db.Roles.AsNoTracking().Select(role => new PrReadOnlyRow + { + Id = role.Id, + Contents = _db.Grants + .Where(grant => grant.RoleId == role.Id) + .Select(grant => new PrSecretGrantRow { Code = grant.Permission.Code, Secret = grant.Permission.Secret }) + .ToList(), + Clean = _db.Grants + .Where(grant => grant.RoleId == role.Id) + .Select(grant => new PrGrantRow { Code = grant.Permission.Code, NameEn = grant.Permission.Code }) + .ToList() + }); + + private static string Describe(PrRoleRow row) => + $"{row.Code}|{row.TenantId?.ToString() ?? "-"}|{row.Name?.En ?? "-"}|" + + $"{string.Join(",", row.Contents.Select(grant => grant.Code + "/" + grant.NameEn))}|{string.Join(",", row.Codes)}"; + + private static string Describe(PrOpenRow row) => + $"{row.Code}:{string.Join(",", row.Contents.Select(grant => grant.Code + "/" + (grant.Secret ?? "-")))}"; + + // ------------------------------------------------------------------ DW-12: a projected row stays whole + + /// + /// DCMP's B1b, B1f and C1. The denied tenant is stripped and every other member arrives as the + /// source built it: the nested name, the list of grants and the list of codes. + /// + [Theory] + [InlineData(DwTier.Convenience)] + [InlineData(DwTier.Strict)] + public async Task A_projected_row_keeps_its_nested_objects_and_lists_when_a_scalar_is_denied(DwTier tier) + { + string[] expected = + { + "R1|-|Role one|read/read!,write/write!|read,write", + "T0|-|Template|read/read!|read" + }; + + Assert.Equal(expected, Guard(RoleRows(), tier).ToList(Everything()).Data.Select(Describe).OrderBy(x => x)); + Assert.Equal(expected, (await Guard(RoleRows(), tier).ToListAsync(Everything())).Data.Select(Describe).OrderBy(x => x)); + + FilterResult dynamicRows = await Guard(RoleRows(), tier).ToListAsyncDynamic(Everything()); + dynamic first = dynamicRows.Data.Single(row => (string)row.Code == "R1"); + Assert.Equal("Role one", (string)first.Name.En); + Assert.Equal(2, ((IEnumerable)first.Contents).Count()); + Assert.Equal(new[] { "read", "write" }, (IEnumerable)first.Codes); + Assert.Null(((object)first).GetType().GetProperty("TenantId")); + + SegmentResult segment = await Guard(RoleRows(), tier).ToListAsync(new Segment()); + Assert.Equal(expected, segment.Data!.Select(Describe).OrderBy(x => x)); + } + + /// The same row, read without the library, for comparison: nothing is lost but the tenant. + [Fact] + public void The_guarded_row_matches_what_plain_EF_Core_returns_less_the_denied_field() + { + List plain = RoleRows().Where(row => row.TenantId == 1 || row.TenantId == null).ToList() + .Select(row => { row.TenantId = null; return Describe(row); }) + .OrderBy(x => x) + .ToList(); + + Assert.Equal(plain, Guard(RoleRows(), DwTier.Strict).ToList(Everything()).Data.Select(Describe).OrderBy(x => x)); + } + + /// The tenant stays denied everywhere else: a condition on it is refused, and the scope still applies. + [Theory] + [InlineData(DwTier.Convenience)] + [InlineData(DwTier.Strict)] + public void The_denied_field_is_still_refused_as_a_condition(DwTier tier) + { + Filter byTenant = new() + { + ConditionGroup = new ConditionGroup + { + Conditions = { new Condition { Field = "TenantId", DataType = DataType.Number, Operator = Operator.Equal, Values = { 2 } } } + } + }; + + PolicyException refused = Assert.Throws(() => Guard(RoleRows(), tier).ToList(byTenant)); + + Assert.Equal(PolicyErrorCode.FieldDeniedForWhere, refused.ErrorCode); + } + + /// A member the caller names is carried whole when nothing beneath it is denied, as it was. + [Fact] + public void A_member_named_by_the_caller_is_carried_whole() + { + PrRoleRow row = Guard(RoleRows(), DwTier.Strict) + .ToList(Selecting("Code", "Name", "Contents", "Codes")).Data.Single(r => r.Code == "R1"); + + Assert.Equal("R1|-|Role one|read/read!,write/write!|read,write", Describe(row)); + } + + // ------------------------------------------------------------------ F1: a denial beneath a member + + /// + /// Nothing is denied at the top of this row, so nothing used to be synthesized and the whole list, + /// its denied values included, came back. The list is now narrowed around the denial. + /// + [Theory] + [InlineData(DwTier.Convenience)] + [InlineData(DwTier.Strict)] + public async Task A_denial_beneath_a_projected_member_narrows_it(DwTier tier) + { + string[] expected = { "R1:read/-,write/-", "R2:read/-", "T0:read/-" }; + + PolicyQueryable guarded = Guard(OpenRows(), tier); + + Assert.Equal(expected, guarded.ToList(Everything()).Data.Select(Describe).OrderBy(x => x)); + Assert.Contains( + guarded.LastTrace!.Decisions, + decision => decision is { FieldPath: "Contents.Secret", Feature: PolicyFeature.Select, Action: PolicyAction.Dropped }); + + Assert.Equal(expected, (await Guard(OpenRows(), tier).ToListAsync(Everything())).Data.Select(Describe).OrderBy(x => x)); + Assert.Equal(expected, (await Guard(OpenRows(), tier).ToListAsync(new Segment())).Data!.Select(Describe).OrderBy(x => x)); + + FilterResult dynamicRows = Guard(OpenRows(), tier).ToListDynamic(Everything()); + foreach (dynamic row in dynamicRows.Data) + { + foreach (object grant in (System.Collections.IEnumerable)row.Contents) + { + Assert.Null(grant.GetType().GetProperty("Secret")); + } + } + } + + /// A dry run enforces nothing, as it never has: the rows come back whole and the decision is recorded. + [Fact] + public void A_dry_run_records_the_denial_beneath_and_returns_the_row_whole() + { + PolicyQueryable guarded = Guard(OpenRows(), DwTier.Strict, dryRun: true); + + Assert.Contains("R1:read/s-read,write/s-write", guarded.ToList(Everything()).Data.Select(Describe)); + Assert.Contains(guarded.LastTrace!.Decisions, decision => decision.FieldPath == "Contents.Secret"); + } + + /// + /// An included navigation of an entity used to come back whole whenever nothing at the top was + /// denied. It is now left out, as a navigation always was once a projection was needed; the owned + /// members, which EF Core loads with every shelf, are narrowed and kept. + /// + [Theory] + [InlineData(DwTier.Convenience)] + [InlineData(DwTier.Strict)] + public async Task An_entity_with_a_denial_beneath_leaves_out_its_navigations_and_narrows_what_it_owns(DwTier tier) + { + PrShelf shelf = (await Guard(_db.Shelves.Include(s => s.Books), tier).ToListAsync(Everything())).Data.Single(); + + Assert.Equal("S1", shelf.Name); + Assert.Empty(shelf.Books); + Assert.Equal(("A7", (string?)null), (shelf.Location!.Aisle, shelf.Location.Code)); + Assert.Equal(("t1", (string?)null), (shelf.Tags.Single().Name, shelf.Tags.Single().Hidden)); + + dynamic row = Guard(_db.Shelves.Include(s => s.Books), tier).ToListDynamic(Everything()).Data.Single(); + Assert.Null(((object)row).GetType().GetProperty("Books")); + Assert.Null(((object)row.Location).GetType().GetProperty("Code")); + } + + /// + /// A top-level denial on an entity keeps what EF Core loads with it: the owned member whole and + /// the blob, which a projection of scalars alone used to empty. + /// + [Fact] + public void An_entity_keeps_its_owned_member_and_its_blob_beside_a_denied_scalar() + { + PrBin bin = Guard(_db.Bins, DwTier.Strict).ToList(Everything()).Data.Single(); + + Assert.Null(bin.Label); + Assert.Equal(("B2", 3), (bin.Place!.Aisle, bin.Place.Row)); + Assert.Equal(new byte[] { 1, 2, 3 }, bin.Photo); + } + + /// + /// Rows in memory keep a member whole when nothing beneath it is denied. One with a denial beneath + /// is left out: the core's narrowing of a reference reads it through EF Core. + /// + [Fact] + public void Rows_in_memory_keep_clean_members_whole_and_leave_out_the_rest() + { + PrRoleRow[] roles = + { + new() { Id = 1, Code = "R1", TenantId = 1, Name = new PrName { En = "One" }, Contents = { new PrGrantRow { Code = "read" } }, Codes = { "read" } } + }; + PrOpenRow[] open = { new() { Id = 1, Code = "R1", Contents = { new PrSecretGrantRow { Code = "read", Secret = "s-read" } } } }; + + PrRoleRow role = roles.AsQueryable().ApplyPolicy(Caller(), Options(DwTier.Strict), Resolver()).ToList(Everything()).Data.Single(); + PolicyQueryable guarded = open.AsQueryable().ApplyPolicy(Caller(), Options(DwTier.Strict), Resolver()); + PrOpenRow row = guarded.ToList(Everything()).Data.Single(); + + Assert.Equal("R1|-|One|read/|read", Describe(role)); + Assert.Empty(row.Contents); + Assert.Contains( + guarded.LastTrace!.Decisions, + decision => decision.FieldPath == "Contents" && decision.Reason!.StartsWith("left out whole", StringComparison.Ordinal)); + } + + // ------------------------------------------------------------------ F2: the key the builder adds + + /// A synthesized projection cannot narrow a list whose key is denied, so it leaves the list out. + [Theory] + [InlineData(DwTier.Convenience)] + [InlineData(DwTier.Strict)] + public void A_member_whose_key_is_denied_is_left_out_whole(DwTier tier) + { + PolicyQueryable guarded = Guard(KeyedRows(), tier); + + Assert.All(guarded.ToList(Everything()).Data, row => Assert.Empty(row.Contents)); + Assert.Contains( + guarded.LastTrace!.Decisions, + decision => decision.FieldPath == "Contents" && decision.Reason == "left out whole: the projection would add its key 'Contents.Id', which is denied"); + } + + /// + /// Naming a navigation whose key is denied used to drop the key from the list in the convenience + /// tier, and the builder added it straight back. It is refused in both tiers, as naming a sibling + /// of the key already was. + /// + [Theory] + [InlineData(DwTier.Convenience)] + [InlineData(DwTier.Strict)] + public void Naming_a_navigation_whose_key_is_denied_is_refused(DwTier tier) + { + PolicyException projected = Assert.Throws( + () => Guard(KeyedRows(), tier).ToList(Selecting("Id", "Contents"))); + PolicyException entity = Assert.Throws( + () => Guard(_db.KeyParents, tier).ToList(Selecting("Id", "Kids"))); + + Assert.Equal(PolicyErrorCode.FieldDeniedForSelect, projected.ErrorCode); + Assert.Equal(PolicyErrorCode.FieldDeniedForSelect, entity.ErrorCode); + Assert.Equal(tier == DwTier.Strict ? "*" : "Contents.Id", projected.FieldPath); + Assert.Equal(tier == DwTier.Strict ? "*" : "Kids.Id", entity.FieldPath); + } + + // ------------------------------------------------------------------ F3: a list typed IReadOnlyList + + /// + /// The projection gate read collections through a narrower list than the policy, so a list typed + /// IReadOnlyList<T> hid its denied field: naming it carried the value out in both tiers. + /// + [Theory] + [InlineData(DwTier.Convenience)] + [InlineData(DwTier.Strict)] + public void Naming_a_read_only_list_with_a_denial_beneath_is_refused(DwTier tier) + { + PolicyException typed = Assert.Throws( + () => Guard(ReadOnlyRows(), tier).ToList(Selecting("Id", "Contents"))); + PolicyException dynamicRows = Assert.Throws( + () => Guard(ReadOnlyRows(), tier).ToListDynamic(Selecting("Id", "Contents"))); + + Assert.Equal(PolicyErrorCode.FieldDeniedForSelect, typed.ErrorCode); + Assert.Equal(PolicyErrorCode.FieldDeniedForSelect, dynamicRows.ErrorCode); + Assert.Equal(tier == DwTier.Strict ? "*" : "Contents.Secret", typed.FieldPath); + } + + /// + /// With no projection sent, the read-only list with a denial beneath is left out, since the core + /// cannot project a path through it; the clean one beside it is carried whole. + /// + [Fact] + public void A_read_only_list_is_kept_whole_when_clean_and_left_out_when_it_cannot_be_narrowed() + { + PolicyQueryable guarded = Guard(ReadOnlyRows(), DwTier.Strict); + PrReadOnlyRow row = guarded.ToList(Everything()).Data.Single(r => r.Id == 1); + + Assert.Empty(row.Contents); + Assert.Equal(new[] { "read", "write" }, row.Clean.Select(grant => grant.Code).OrderBy(x => x)); + Assert.Contains( + guarded.LastTrace!.Decisions, + decision => decision.FieldPath == "Contents" && decision.Reason == "left out whole: the core cannot project 'Contents.Code'"); + Assert.Equal( + new[] { "read", "write" }, + Guard(ReadOnlyRows(), DwTier.Strict).ToList(Selecting("Id", "Clean")).Data + .Single(r => r.Id == 1).Clean.Select(grant => grant.Code).OrderBy(x => x)); + } + + // ------------------------------------------------------------------ the gate, without a database + + /// A type with nothing denied anywhere keeps its null projection, on every source. + [Fact] + public void Nothing_is_synthesized_when_nothing_is_denied_at_any_depth() + { + foreach (RowShape rows in new[] { RowShape.Entity, RowShape.Projected, RowShape.InMemory }) + { + Filter sanitized = FilterSanitizer.Sanitize( + new Filter(), Resolver(), Caller(), Options(DwTier.Strict), new PolicyTrace(DwTier.Strict, dryRun: false), rows: rows); + + Assert.Null(sanitized.Selects); + } + } + + /// What each source keeps of the same type, with one denial beneath a list. + [Fact] + public void Each_source_keeps_what_it_carries() + { + List? Synthesized(RowShape rows) => FilterSanitizer.Sanitize( + new Filter(), Resolver(), Caller(), Options(DwTier.Strict), new PolicyTrace(DwTier.Strict, dryRun: false), rows: rows).Selects; + + Assert.Equal(new[] { "Code", "Id" }, Synthesized(RowShape.Entity)!.OrderBy(x => x, StringComparer.Ordinal)); + Assert.Equal(new[] { "Code", "Contents.Code", "Id" }, Synthesized(RowShape.Projected)!.OrderBy(x => x, StringComparer.Ordinal)); + Assert.Equal(new[] { "Code", "Id" }, Synthesized(RowShape.InMemory)!.OrderBy(x => x, StringComparer.Ordinal)); + } +} + +/// The shelf's shape with no attribute anywhere. +public class PrShelfWithoutDenials +{ + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public PrPlace? Place { get; set; } + + public List Contents { get; set; } = new(); +} diff --git a/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs b/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs index 77a409d..682e993 100644 --- a/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs +++ b/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs @@ -217,7 +217,20 @@ private static void Walk( /// followed as well as classes, because these attributes are supported on DTOs, where an /// IContact reference or a record struct is ordinary. /// - internal static Type? NavigationTypeOf(Type propertyType) + internal static Type? NavigationTypeOf(Type propertyType) => AsNavigation(Peeled(propertyType)); + + /// + /// The type a property holds once every collection layer and is peeled + /// off: LineDto for IReadOnlyList<LineDto>, for + /// List<string>, and the property's own type when it is not a collection. + /// + /// + /// The one reading of a property's shape that the walker and the projection gate share. The gate + /// once read collections through a narrower list of its own, and a member typed + /// IReadOnlyList<T> hid every denial beneath it from the projection while the walker + /// had put a fragment on each of them. + /// + internal static Type Peeled(Type propertyType) { Type current = Nullable.GetUnderlyingType(propertyType) ?? propertyType; @@ -233,7 +246,7 @@ private static void Walk( current = Nullable.GetUnderlyingType(element) ?? element; } - return AsNavigation(current); + return current; } /// diff --git a/DynamicWhere.ex/Policies/Source/DefaultOrder.cs b/DynamicWhere.ex/Policies/Source/DefaultOrder.cs index f2061e6..1e0753e 100644 --- a/DynamicWhere.ex/Policies/Source/DefaultOrder.cs +++ b/DynamicWhere.ex/Policies/Source/DefaultOrder.cs @@ -89,12 +89,55 @@ or nameof(Queryable.ThenBy) or nameof(Queryable.ThenByDescending)) /// True when a query's rows are a projection made somewhere along the chain that produced it. /// /// + /// Any Select counts. This answers what the rows are, not whether a default can be applied + /// to them, which answers. + /// + internal static bool IsProjected(Expression expression) => OutermostSelect(expression) is not null; + + /// + /// True when a projection along the chain could have left out a field the type's default names. + /// + /// /// A default names fields of the type, and a projection keeps only the members it assigns, so a - /// default ordering it could reach for a member the projection left out and fail to translate: + /// default ordering one it could reach for a member the projection left out and fail to translate: /// Select(["Id", "Title"]).Page(...) on a type ordered by Priority worked before the - /// default existed. Such a query is left in whatever order it had. + /// default existed. + /// + /// Only the outermost projection is read, because it makes the rows the default orders. It hides + /// nothing when it builds the type itself in an initializer — new Row { Code = t.Code } — + /// and assigns every field the default names, at every level of a nested path. EF Core can then + /// translate the order, since each field is a member the projection assigned. Anything else, a + /// constructor with arguments, a member it does not assign or a nested path through something + /// other than an initializer, leaves the query in whatever order it had. + /// /// - internal static bool IsProjected(Expression expression) + internal static bool HidesDefault(Expression expression, Type type) + { + if (OutermostSelect(expression) is not { } select) + { + return false; + } + + if (StripQuotes(select.Arguments[1]) is not LambdaExpression selector + || StripConversions(selector.Body) is not MemberInitExpression initializer + || !type.IsAssignableFrom(initializer.Type)) + { + return true; + } + + foreach (Entry entry in For(type)) + { + if (!Assigns(initializer.Bindings, entry.Field.Split('.'), 0)) + { + return true; + } + } + + return false; + } + + /// The Select nearest the end of the chain, or null when nothing projects it. + private static MethodCallExpression? OutermostSelect(Expression expression) { for (Expression? node = expression; node is MethodCallExpression call; node = call.Arguments.Count > 0 ? call.Arguments[0] : null) @@ -102,18 +145,58 @@ internal static bool IsProjected(Expression expression) if ((call.Method.DeclaringType == typeof(Queryable) || call.Method.DeclaringType == typeof(Enumerable)) && call.Method.Name == nameof(Queryable.Select)) { - return true; + return call; } } - return false; + return null; + } + + /// True when the bindings assign the path, following nested initializers down it. + private static bool Assigns(IEnumerable bindings, string[] path, int depth) + { + MemberBinding? binding = bindings.FirstOrDefault( + candidate => string.Equals(candidate.Member.Name, path[depth], StringComparison.Ordinal)); + + if (binding is null) + { + return false; + } + + if (depth == path.Length - 1) + { + return true; + } + + return binding switch + { + MemberAssignment { Expression: var assigned } + when StripConversions(assigned) is MemberInitExpression nested => + Assigns(nested.Bindings, path, depth + 1), + MemberMemberBinding nested => Assigns(nested.Bindings, path, depth + 1), + _ => false + }; + } + + private static Expression StripQuotes(Expression expression) => + expression is UnaryExpression { NodeType: ExpressionType.Quote } quote ? quote.Operand : expression; + + private static Expression StripConversions(Expression expression) + { + while (expression is UnaryExpression { NodeType: ExpressionType.Convert or ExpressionType.TypeAs } conversion) + { + expression = conversion.Operand; + } + + return expression; } /// /// True when a guarded query over this source takes the type's default: nothing has ordered it and - /// nothing has projected it. + /// no projection hides a field the default names. /// - internal static bool Applies(Expression expression) => !IsOrdered(expression) && !IsProjected(expression); + internal static bool Applies(Expression expression, Type type) => + !IsOrdered(expression) && !HidesDefault(expression, type); /// /// Everything wrong with a type's declared default: entries that cannot be read, fields no query can diff --git a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs index 189e7d0..77d5bbb 100644 --- a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs +++ b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs @@ -1,4 +1,5 @@ -using System.Reflection; +using System.Collections.Concurrent; +using System.Reflection; using DynamicWhere.ex.Classes.Complex; using DynamicWhere.ex.Classes.Core; using DynamicWhere.ex.Enums; @@ -58,6 +59,10 @@ internal static class FilterSanitizer /// field this caller may not order by. False for a clause composed on its own, which orders /// nothing the caller did not ask it to. /// + /// + /// What the query's source puts in the members of its rows, which decides what a synthesized + /// projection keeps. Null for an entity query whose model is unknown. + /// /// A sanitized copy, safe to hand to the existing pipeline. /// Thrown when any argument is null. /// @@ -75,7 +80,8 @@ internal static Filter Sanitize( DwPolicyOptions options, PolicyTrace trace, bool synthesizeProjection = true, - bool applyDefaultOrder = true) + bool applyDefaultOrder = true, + RowShape? rows = null) where T : class { if (filter is null) @@ -141,7 +147,7 @@ internal static Filter Sanitize( working.Orders = defaults; } - GateSelects(working, gate, synthesizeProjection); + GateSelects(working, gate, synthesizeProjection, rows ?? RowShape.Entity); // The strict tier checks the bill once every field has passed its gate. It drops nothing, so a // request still here names no field the caller may not use, and a weighted field the caller may @@ -610,6 +616,10 @@ private static void GateSummaryOrders( /// When true and the caller sent no orders, the type's declared default order is added, less every /// field this caller may not order by. /// + /// + /// What the query's source puts in the members of its rows, which decides what a synthesized + /// projection keeps. Null for an entity query whose model is unknown. + /// /// /// Every condition set is gated independently, because each one becomes its own subquery and a /// field refused in one must not be reachable through another. @@ -626,7 +636,8 @@ internal static Segment Sanitize( DwPolicyContext context, DwPolicyOptions options, PolicyTrace trace, - bool applyDefaultOrder = true) + bool applyDefaultOrder = true, + RowShape? rows = null) where T : class { if (segment is null) @@ -720,7 +731,7 @@ internal static Segment Sanitize( working.Orders = defaults; } - GateSegmentSelects(working, gate); + GateSegmentSelects(working, gate, rows ?? RowShape.Entity); // The strict tier checks the bill once every field has passed its gate. It drops nothing, so a // request still here names no field the caller may not use, and a weighted field the caller may @@ -964,11 +975,11 @@ private static void GateSegmentOrders(Segment segment, Gate gate) /// materializes. /// /// - private static void GateSegmentSelects(Segment segment, Gate gate) + private static void GateSegmentSelects(Segment segment, Gate gate, RowShape rows) { if (segment.Selects is null || segment.Selects.Count == 0) { - if (SynthesizedProjection(gate) is List synthesized) + if (SynthesizedProjection(gate, rows) is List synthesized) { segment.Selects = synthesized; } @@ -1604,7 +1615,7 @@ private static List DefaultOrders(Gate gate, bool segment = false) w /// entity — turning the strictest possible policy into the widest possible result. /// /// - private static void GateSelects(Filter filter, Gate gate, bool synthesize) + private static void GateSelects(Filter filter, Gate gate, bool synthesize, RowShape rows) { if (filter.Selects is null || filter.Selects.Count == 0) { @@ -1613,7 +1624,7 @@ private static void GateSelects(Filter filter, Gate gate, bool synthesize) // refuse the call outright on a type whose every field is denied for select. if (synthesize) { - SynthesizeSelects(filter, gate); + SynthesizeSelects(filter, gate, rows); } return; @@ -1644,6 +1655,12 @@ private static void GateSelects(Filter filter, Gate gate, bool synthesize) /// expansion happens only where there is something to narrow, so a type the policy has no /// opinion about still generates the SQL the unguarded path would. /// + /// + /// A narrowing the core cannot build as written is refused rather than handed over. The builder + /// adds the key of every node it narrows, so dropping a denied key from the list put it straight + /// back; and a path through a collection the core does not unwrap, such as an + /// IReadOnlyList<T>, failed validation with an error about the caller's field names. + /// /// private static List GateProjection(IEnumerable selects, Gate gate) { @@ -1657,9 +1674,9 @@ void Keep(string path) } } - bool Allowed(string path) + bool Allowed(string path, out FieldPolicy policy) { - FieldPolicy policy = gate.PolicyFor(path, PolicyFeature.Select); + policy = gate.PolicyFor(path, PolicyFeature.Select); return policy.Allows(PolicyFeature.Select) || !gate.Refuse(path, PolicyFeature.Select, PolicyErrorCode.FieldDeniedForSelect, policy); @@ -1669,7 +1686,7 @@ bool Allowed(string path) { // The field's own decision first. A navigation that is itself denied is refused as it // stands; what it carries is only asked about once it has survived on its own account. - if (!Allowed(field)) + if (!Allowed(field, out _)) { continue; } @@ -1684,27 +1701,45 @@ bool Allowed(string path) IReadOnlyList carried = gate.ProjectionUnder(field); List survivors = new(carried.Count); - bool narrowed = false; + (string Path, FieldPolicy Policy)? cause = null; foreach (string path in carried) { - if (Allowed(path)) + if (Allowed(path, out FieldPolicy policy)) { survivors.Add(path); } else { - narrowed = true; + cause ??= (path, policy); } } - if (!narrowed) + if (cause is not { } narrowing) { Keep(field); continue; } + if (gate.DeniedKey(survivors) is { } key) + { + gate.RefuseNarrowing( + key.Path, key.Policy, + $"'{field}' was narrowed, and the projection builder adds this key to every node it narrows"); + + continue; + } + + if (gate.Unprojectable(survivors) is { } unbuilt) + { + gate.RefuseNarrowing( + narrowing.Path, narrowing.Policy, + $"'{field}' cannot be narrowed around it: the core cannot project '{unbuilt}'"); + + continue; + } + foreach (string path in survivors) { Keep(path); @@ -1760,7 +1795,7 @@ private static void KeepCarriedKeys(string path, Gate gate, Action keep) /// denied. /// /// - /// A caller who sends no projection gets the whole entity, denied columns included, because + /// A caller who sends no projection gets the whole row, denied columns included, because /// the pipeline only projects when Selects is non-null. Gating the list alone would /// therefore enforce deny-select against precisely the callers who volunteered one, and leave /// it bypassable by asking for less. @@ -1771,19 +1806,14 @@ private static void KeepCarriedKeys(string path, Gate gate, Action keep) /// one that rewrites every query in the application. /// /// - /// Scalars only. A navigation is not loaded by an unguarded call in the first place, and - /// projecting one whole would carry every field beneath it — reopening the same hole one level - /// down. Where a caller had eagerly loaded one, narrowing it away fails closed. - /// - /// /// This never throws in the strict tier for a denied field. Strict refuses what a caller asks /// for, and here the caller named nothing; throwing would fail every strict query against a /// type carrying any denied field at all. /// /// - private static void SynthesizeSelects(Filter filter, Gate gate) + private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) { - if (SynthesizedProjection(gate) is List allowed) + if (SynthesizedProjection(gate, rows) is List allowed) { filter.Selects = allowed; } @@ -1795,26 +1825,106 @@ private static void SynthesizeSelects(Filter filter, Gate gate) /// /// Shared by the filter and the segment paths, because they are closing the same hole and a /// second copy of this is a second place to forget. + /// + /// What an unguarded call would return, less what the policy withholds. A member that holds a + /// value is kept when it is allowed. A member that holds an object, or a list of them, is kept + /// whole when nothing beneath it is denied, and narrowed the way a caller naming it would have it + /// narrowed when something is — but only where the source carries it: every member of a + /// projected row or a row in memory, and the owned and complex members of an entity. An entity's + /// other navigations are left out, since an unguarded call loads them only when something includes + /// them, and projecting one would load it. + /// + /// + /// Everything beneath every object member is asked about, whether or not the source carries it. + /// A denial only beneath a member used to synthesize nothing, so the row came back whole, and an + /// included navigation, an owned member, a projected list or an object in memory carried the + /// denied value out with it. An entity query does not say what it loads: an include, an automatic + /// include and a lazy loader all fill a navigation the query text never names. + /// + /// + /// A member that cannot be narrowed as the projection would build it is left out whole: in + /// memory, where the core's narrowing of a reference reads it through EF Core; an EF Core complex + /// property, which the narrowing compares to null; where the builder would add a denied key back; + /// and where the core cannot project a path beneath it. + /// /// - private static List? SynthesizedProjection(Gate gate) + private static List? SynthesizedProjection(Gate gate, RowShape rows) { List allowed = new(); bool anyDenied = false; - foreach (string field in gate.ProjectableFields()) + foreach (PropertyInfo member in gate.Members()) { - FieldPolicy policy = gate.PolicyFor(field, PolicyFeature.None); + string name = member.Name; + FieldPolicy policy = gate.PolicyFor(name, PolicyFeature.None); + + if (!policy.Allows(PolicyFeature.Select)) + { + anyDenied = true; + + gate.Record(name, PolicyFeature.Select, PolicyAction.Dropped, policy); + + continue; + } - if (policy.Allows(PolicyFeature.Select)) + if (Gate.HoldsValue(member.PropertyType)) { - allowed.Add(field); + allowed.Add(name); continue; } - anyDenied = true; + List survivors = new(); + bool narrowed = false; + + foreach (string path in gate.ProjectionUnder(name)) + { + FieldPolicy beneath = gate.PolicyFor(path, PolicyFeature.None); - gate.Record(field, PolicyFeature.Select, PolicyAction.Dropped, policy); + if (beneath.Allows(PolicyFeature.Select)) + { + survivors.Add(path); + + continue; + } + + narrowed = true; + + gate.Record(path, PolicyFeature.Select, PolicyAction.Dropped, beneath); + } + + anyDenied |= narrowed; + + if (!rows.Carries(name)) + { + continue; + } + + if (!narrowed) + { + allowed.Add(name); + + continue; + } + + string? reason = !rows.Narrows(name) + ? rows.Kind == RowKind.InMemory + ? "left out whole: the rows are in memory, and narrowing it needs EF Core" + : "left out whole: it is a complex property, which EF Core cannot narrow" + : gate.DeniedKey(survivors) is { } key + ? $"left out whole: the projection would add its key '{key.Path}', which is denied" + : gate.Unprojectable(survivors) is { } unbuilt + ? $"left out whole: the core cannot project '{unbuilt}'" + : null; + + if (reason is not null) + { + gate.LeaveOut(name, reason); + + continue; + } + + allowed.AddRange(survivors); } if (!anyDenied || gate.IsDryRun) @@ -2347,6 +2457,10 @@ private sealed class Gate // where a denied field is, as a denied field is. private readonly HashSet _unknown = new(StringComparer.Ordinal); + // What a projection of each navigation carries, per root type and path. Reflection only, so it + // is the same for every caller and every query. + private static readonly ConcurrentDictionary<(Type Root, string Path), IReadOnlyList> Beneath = new(); + internal Gate( Type entityType, PolicyResolver resolver, @@ -2447,29 +2561,37 @@ internal string Unknown(string name) internal bool IsDryRun => _options.DryRun || _context.DryRun; /// - /// The scalar properties of the entity that a typed projection can actually assign. + /// The members of the entity that a typed projection can actually assign. /// /// /// Read through the same reflection cache the validator uses, so a synthesized projection /// cannot name a field the pipeline would then reject. Write-only and indexed members are /// excluded because a typed projection assigns into them. /// - internal IEnumerable ProjectableFields() + internal IEnumerable Members() { foreach (KeyValuePair entry in CacheReflection.GetTypeProperties(_entityType)) { PropertyInfo property = entry.Value; - if (property.CanRead - && property.CanWrite - && property.GetIndexParameters().Length == 0 - && CacheReflection.IsSimpleType(property.PropertyType)) + if (property.CanRead && property.CanWrite && property.GetIndexParameters().Length == 0) { - yield return property.Name; + yield return property; } } } + /// + /// True when a member of this type holds a value rather than an object: a scalar, or a + /// collection of scalars such as List<string> or byte[]. + /// + /// + /// A collection of scalars is a value the projection assigns whole. Nothing beneath it can + /// carry a policy, and on an entity it is a column the unguarded call loads. + /// + internal static bool HoldsValue(Type type) => + CacheReflection.IsSimpleType(AttributePolicyProvider.Peeled(type)); + /// /// True when a validated path names a navigation rather than a scalar. /// @@ -2499,16 +2621,22 @@ internal bool HasKey(string path) /// deny-select against whoever spelled the child out in full and leaves it bypassable by /// asking for its parent instead — the same shape as sending no projection at all. /// - /// The walk stops where 's does, and carries the same - /// cycle guard. Nothing below that depth holds a fragment, so there is no decision down - /// there to enforce and descending further would only enumerate paths the resolver cannot - /// speak about. + /// A type is read the way reads it, through + /// , so every path a fragment can sit on + /// is a path this walk reaches. It once read collections through a narrower list of its own, + /// and a member typed IReadOnlyList<T> hid every denial beneath it. The walk stops + /// at the same depth, and does not follow a type back into itself along one path. /// /// - internal IReadOnlyList ProjectionUnder(string path) + internal IReadOnlyList ProjectionUnder(string path) => + Beneath.GetOrAdd((_entityType, path), key => Enumerate(key.Root, key.Path)); + + private static IReadOnlyList Enumerate(Type root, string path) { List paths = new(); - Type? type = TypeAt(path); + Type? type = TypeAt(root, path) is { } declared + ? AttributePolicyProvider.NavigationTypeOf(declared) + : null; if (type is not null) { @@ -2540,20 +2668,9 @@ private static void Descend( string child = $"{prefix}.{property.Name}"; - if (CacheReflection.IsSimpleType(property.PropertyType)) - { - paths.Add(child); - - continue; - } - - Type nested = CacheReflection.GetCollectionElementType(property.PropertyType) - ?? property.PropertyType; - - // A collection of a primitive is a value the projection assigns whole, not a - // navigation into its element type. Descending into byte would enumerate nothing - // and drop the member from an expansion that is supposed to preserve it. - if (CacheReflection.IsSimpleType(nested)) + // A value, a collection of values, or a type the walker does not enter: projected + // whole, and nothing beneath it carries a policy. + if (AttributePolicyProvider.NavigationTypeOf(property.PropertyType) is not { } nested) { paths.Add(child); @@ -2571,12 +2688,78 @@ private static void Descend( seen.Remove(type); } + /// + /// The key the projection builder would add to a node that a list of narrowed paths passes + /// through, when this caller may not select it; null when every such key is allowed. + /// + /// + /// The builder adds the key of each nested node it builds whether the list names it or not, + /// so a narrowing that dropped a denied key would carry it anyway. + /// + internal (string Path, FieldPolicy Policy)? DeniedKey(IEnumerable paths) + { + HashSet nodes = new(StringComparer.Ordinal); + + foreach (string path in paths) + { + for (int cut = path.IndexOf(SegmentSeparator); cut >= 0; cut = path.IndexOf(SegmentSeparator, cut + 1)) + { + string node = path[..cut]; + + if (!nodes.Add(node) + || TypeAt(_entityType, node) is not { } type + || CacheReflection.FindProperty(type, "Id") is not { } key) + { + continue; + } + + string keyPath = $"{node}.{key.Name}"; + FieldPolicy policy = PolicyFor(keyPath, PolicyFeature.None); + + if (!policy.Allows(PolicyFeature.Select)) + { + return (keyPath, policy); + } + } + } + + return null; + } + + /// + /// The first of a list of narrowed paths that the core's own validation refuses, or null when + /// it accepts them all. + /// + /// + /// The walk above reads collections the way the policy does, and the core reads a narrower + /// set of them: a path through an IReadOnlyList<T> is one the policy can deny and + /// the core cannot project. + /// + internal string? Unprojectable(IEnumerable paths) + { + foreach (string path in paths) + { + try + { + CacheReflection.ValidatePropertyPath(_entityType, path); + } + catch (LogicException) + { + return path; + } + } + + return null; + } + /// The type a validated path names, or null when it names nothing. - private Type? TypeAt(string path) + private Type? TypeAt(string path) => TypeAt(_entityType, path); + + private static Type? TypeAt(Type root, string path) { try { - return CacheReflection.GetFieldType(_entityType, path); + return CacheReflection.GetFieldType(root, path); } catch (LogicException) { @@ -2907,6 +3090,34 @@ internal void DenySegmentInference(string fieldPath, FieldPolicy policy) throw Exception(fieldPath, PolicyFeature.Segment, PolicyErrorCode.FieldDeniedForSegment, policy); } + /// + /// Records that a synthesized projection left a member out whole, and why: something beneath it + /// is denied and the member cannot be narrowed. + /// + internal void LeaveOut(string fieldPath, string reason) => + _trace.Add(new PolicyDecision(fieldPath, PolicyFeature.Select, PolicyAction.Dropped, reason)); + + /// + /// Refuses, in either tier, a projection that would have to be narrowed around a denied field + /// and cannot be built that way. + /// + /// + /// Dropping is not an option here, for the reason it is not for a carried key: the projection + /// that would be built is not the one that was gated, so the only way to honour the denial is + /// to decline to build it. + /// + internal void RefuseNarrowing(string fieldPath, FieldPolicy policy, string reason) + { + _trace.Add(new PolicyDecision(fieldPath, PolicyFeature.Select, PolicyAction.Denied, reason)); + + if (IsDryRun) + { + return; + } + + throw Exception(fieldPath, PolicyFeature.Select, PolicyErrorCode.FieldDeniedForSelect, policy); + } + /// Records that a field of the type's default order was left out for this caller. internal void SkipDefaultOrder(string fieldPath, FieldPolicy policy) { diff --git a/DynamicWhere.ex/Policies/Source/PolicyQueryable.cs b/DynamicWhere.ex/Policies/Source/PolicyQueryable.cs index 9501871..68156be 100644 --- a/DynamicWhere.ex/Policies/Source/PolicyQueryable.cs +++ b/DynamicWhere.ex/Policies/Source/PolicyQueryable.cs @@ -42,6 +42,11 @@ public sealed class PolicyQueryable where T : class // gate. A caller whose orders were all dropped still sent orders, and gets no default in their place. private readonly bool _ordered; + // True once a composed Select, or a composed Filter carrying a projection, has run on this chain. + // Such a chain stays in whatever order it had: the default is for the rows the caller's source + // makes, and a projection the caller composes afterwards is theirs to order. + private readonly bool _projected; + private TypePolicy? _typePolicy; /// @@ -53,7 +58,8 @@ internal PolicyQueryable( PolicyResolver resolver, DwPolicyOptions options, PolicyTrace? carried = null, - bool ordered = false) + bool ordered = false, + bool projected = false) { _source = source; _context = context; @@ -61,6 +67,7 @@ internal PolicyQueryable( _options = options; _carried = carried; _ordered = ordered; + _projected = projected; } /// @@ -123,7 +130,27 @@ public FilterResult ToList(Filter filter, bool getQueryString = false) /// When true, includes the generated SQL in the result. /// Thrown when is null. /// Thrown when the policy refuses part of the filter. - public async Task> ToListAsync(Filter filter, bool getQueryString = false) + public Task> ToListAsync(Filter filter, bool getQueryString = false) => + ToListAsync(filter, getQueryString, CancellationToken.None); + + /// Applies a filter asynchronously and returns the matching page. + /// The caller's filter. Never modified. + /// Cancels the count and the read. + /// Thrown when is null. + /// Thrown when the policy refuses part of the filter. + /// Thrown when is canceled. + public Task> ToListAsync(Filter filter, CancellationToken cancellationToken) => + ToListAsync(filter, false, cancellationToken); + + /// Applies a filter asynchronously and returns the matching page. + /// The caller's filter. Never modified. + /// When true, includes the generated SQL in the result. + /// Cancels the count and the read. + /// Thrown when is null. + /// Thrown when the policy refuses part of the filter. + /// Thrown when is canceled. + public async Task> ToListAsync( + Filter filter, bool getQueryString, CancellationToken cancellationToken) { try { @@ -134,7 +161,7 @@ public async Task> ToListAsync(Filter filter, bool getQueryStrin using (PolicyScope.Enter(_context, LastTrace)) { - FilterResult result = await Guarded().ToListAsync(sanitized, getQueryString); + FilterResult result = await Guarded().ToListAsync(sanitized, getQueryString, cancellationToken); ResultTransformer.Rows( result.Data, TypePolicy, sanitized.Selects, _context, _options, trace); @@ -192,7 +219,27 @@ public FilterResult ToListDynamic(Filter filter, bool getQueryString = /// When true, includes the generated SQL in the result. /// Thrown when is null. /// Thrown when the policy refuses part of the filter. - public async Task> ToListAsyncDynamic(Filter filter, bool getQueryString = false) + public Task> ToListAsyncDynamic(Filter filter, bool getQueryString = false) => + ToListAsyncDynamic(filter, getQueryString, CancellationToken.None); + + /// Applies a filter asynchronously and returns the matching page as dynamic objects. + /// The caller's filter. Never modified. + /// Cancels the count and the read. + /// Thrown when is null. + /// Thrown when the policy refuses part of the filter. + /// Thrown when is canceled. + public Task> ToListAsyncDynamic(Filter filter, CancellationToken cancellationToken) => + ToListAsyncDynamic(filter, false, cancellationToken); + + /// Applies a filter asynchronously and returns the matching page as dynamic objects. + /// The caller's filter. Never modified. + /// When true, includes the generated SQL in the result. + /// Cancels the count and the read. + /// Thrown when is null. + /// Thrown when the policy refuses part of the filter. + /// Thrown when is canceled. + public async Task> ToListAsyncDynamic( + Filter filter, bool getQueryString, CancellationToken cancellationToken) { try { @@ -203,7 +250,7 @@ public async Task> ToListAsyncDynamic(Filter filter, bool using (PolicyScope.Enter(_context, LastTrace)) { - FilterResult result = await Guarded().ToListAsyncDynamic(sanitized, getQueryString); + FilterResult result = await Guarded().ToListAsyncDynamic(sanitized, getQueryString, cancellationToken); ResultTransformer.Rows( result.Data, TypePolicy, sanitized.Selects, _context, _options, trace); @@ -280,7 +327,27 @@ public SummaryResult ToList(Summary summary, bool getQueryString = false) /// When true, includes the generated SQL in the result. /// Thrown when is null. /// Thrown when the policy refuses part of the summary. - public async Task ToListAsync(Summary summary, bool getQueryString = false) + public Task ToListAsync(Summary summary, bool getQueryString = false) => + ToListAsync(summary, getQueryString, CancellationToken.None); + + /// Applies a summary asynchronously and returns the grouped page. + /// The caller's summary. Never modified. + /// Cancels the count and the read. + /// Thrown when is null. + /// Thrown when the policy refuses part of the summary. + /// Thrown when is canceled. + public Task ToListAsync(Summary summary, CancellationToken cancellationToken) => + ToListAsync(summary, false, cancellationToken); + + /// Applies a summary asynchronously and returns the grouped page. + /// The caller's summary. Never modified. + /// When true, includes the generated SQL in the result. + /// Cancels the count and the read. + /// Thrown when is null. + /// Thrown when the policy refuses part of the summary. + /// Thrown when is canceled. + public async Task ToListAsync( + Summary summary, bool getQueryString, CancellationToken cancellationToken) { try { @@ -291,7 +358,7 @@ public async Task ToListAsync(Summary summary, bool getQueryStrin using (PolicyScope.Enter(_context, LastTrace)) { - SummaryResult result = await Guarded().ToListAsync(sanitized, getQueryString); + SummaryResult result = await Guarded().ToListAsync(sanitized, getQueryString, cancellationToken); // First of the three, and the order matters. A group below the floor is one the caller // may not see at all, so nothing downstream should form an opinion about it: transformed @@ -330,7 +397,16 @@ public async Task ToListAsync(Summary summary, bool getQueryStrin /// The caller's segment. Never modified. /// Thrown when is null. /// Thrown when the policy refuses part of the segment. - public async Task> ToListAsync(Segment segment) + public Task> ToListAsync(Segment segment) => + ToListAsync(segment, CancellationToken.None); + + /// Applies a set operation asynchronously and returns the combined page. + /// The caller's segment. Never modified. + /// Cancels the count and the read. + /// Thrown when is null. + /// Thrown when the policy refuses part of the segment. + /// Thrown when is canceled. + public async Task> ToListAsync(Segment segment, CancellationToken cancellationToken) { try { @@ -338,13 +414,14 @@ public async Task> ToListAsync(Segment segment) Segment sanitized = FilterSanitizer.Sanitize( segment, _resolver, _context, _options, trace, - applyDefaultOrder: TakesDefaultOrder); + applyDefaultOrder: TakesDefaultOrder, + rows: RowShape.Of(_source)); LastTrace = trace; using (PolicyScope.Enter(_context, LastTrace)) { - SegmentResult result = await Guarded().ToListAsync(sanitized); + SegmentResult result = await Guarded().ToListAsync(sanitized, cancellationToken); ResultTransformer.Rows( result.Data, TypePolicy, sanitized.Selects, _context, _options, trace); @@ -373,7 +450,7 @@ public PolicyQueryable Select(List fields) using (PolicyScope.Enter(_context, LastTrace)) { - return Chain(Scoped(sanitized).Select(sanitized.Selects!)); + return Chain(Scoped(sanitized).Select(sanitized.Selects!), projected: true); } } catch (PolicyException refusal) when (Refused(refusal)) @@ -551,7 +628,12 @@ public PolicyQueryable Filter(Filter filter) using (PolicyScope.Enter(_context, LastTrace)) { - return Chain(Guarded().Filter(sanitized)); + // A caller who sent orders gets no default later in the chain, whether or not any of + // them survived, exactly as a composed Order does. + return Chain( + Guarded().Filter(sanitized), + ordered: filter.Orders is { Count: > 0 }, + projected: sanitized.Selects is not null); } } catch (PolicyException refusal) when (Refused(refusal)) @@ -674,15 +756,18 @@ static bool IsGroupSize(AggregateBy aggregate) => /// Wraps a composed query back into a handle, carrying the trace so far. /// The composed query. - /// True when the composing call was an Order. - private PolicyQueryable Chain(IQueryable composed, bool ordered = false) => - new(composed, _context, _resolver, _options, LastTrace, _ordered || ordered); + /// True when the composing call sent orders. + /// True when the composing call projected the rows. + private PolicyQueryable Chain(IQueryable composed, bool ordered = false, bool projected = false) => + new(composed, _context, _resolver, _options, LastTrace, _ordered || ordered, _projected || projected); /// /// True when a query over this handle takes the type's default order: the caller composed no - /// Order, and nothing ordered or projected the source. + /// Order and no projection, nothing ordered the source, and no projection in the source hides + /// a field the default names. /// - private bool TakesDefaultOrder => !_ordered && DefaultOrder.Applies(_source.Expression); + private bool TakesDefaultOrder => + !_ordered && !_projected && DefaultOrder.Applies(_source.Expression, typeof(T)); /// /// Refuses a method that hands back a query the caller materializes, when this type's values @@ -789,7 +874,9 @@ private Filter Sanitize(Filter filter, PolicyTrace trace) { // A source the caller ordered before guarding it keeps that order: a default would replace it. Filter sanitized = FilterSanitizer.Sanitize( - filter, _resolver, _context, _options, trace, applyDefaultOrder: TakesDefaultOrder); + filter, _resolver, _context, _options, trace, + applyDefaultOrder: TakesDefaultOrder, + rows: RowShape.Of(_source)); LastTrace = trace; diff --git a/DynamicWhere.ex/Policies/Source/RowShape.cs b/DynamicWhere.ex/Policies/Source/RowShape.cs new file mode 100644 index 0000000..722fd84 --- /dev/null +++ b/DynamicWhere.ex/Policies/Source/RowShape.cs @@ -0,0 +1,156 @@ +using System.Collections; +using System.Reflection; +using DynamicWhere.ex.Source; +using Microsoft.EntityFrameworkCore.Metadata; + +namespace DynamicWhere.ex.Policies.Source; + +/// +/// What a guarded query's source puts in the members of its rows, as far as a projection the library +/// synthesizes is concerned. +/// +/// +/// A caller who sends no projection is owed what an unguarded call would return, less what the policy +/// withholds. Which members that is depends on the source. An entity query loads a navigation only +/// when something includes it; a projection computes every member it assigns; rows already in memory +/// hold everything. A synthesized projection that left out every navigation was right for the first +/// and emptied every list and nested object of the other two. +/// +/// Read from the query itself, never from the rows, so the sanitizer stays pure: this is decided once, +/// before it runs. +/// +/// +internal sealed class RowShape +{ + /// An entity query whose model could not be read: no navigation is known to load. + internal static readonly RowShape Entity = new(RowKind.Entity, Empty(), Empty()); + + /// A projection the caller built, whose every member holds a value it computed. + internal static readonly RowShape Projected = new(RowKind.Projected, Empty(), Empty()); + + /// Rows held in memory, whose every member holds whatever the object holds. + internal static readonly RowShape InMemory = new(RowKind.InMemory, Empty(), Empty()); + + private readonly HashSet _alwaysLoaded; + private readonly HashSet _complex; + + private RowShape(RowKind kind, HashSet alwaysLoaded, HashSet complex) + { + Kind = kind; + _alwaysLoaded = alwaysLoaded; + _complex = complex; + } + + /// Where the rows come from. + internal RowKind Kind { get; } + + /// + /// True when rows from this source carry a value in the member whatever the caller includes: every + /// member of a projected or in-memory row, and an entity's owned and complex members, which EF Core + /// loads with the entity on every query. + /// + internal bool Carries(string member) => Kind != RowKind.Entity || _alwaysLoaded.Contains(member); + + /// + /// True when the core's projection can narrow the member to some of the fields beneath it. Not in + /// memory, where its narrowing of a reference reads it through EF Core, and not for an EF Core + /// complex property, which it compares to null and EF Core refuses to. + /// + internal bool Narrows(string member) => Kind != RowKind.InMemory && !_complex.Contains(member); + + /// Reads the shape of a guarded query's source. + /// + /// Rows in memory first, because a projection over them is still in memory. Then any Select + /// along the chain, which is how a caller builds its own row type out of entities. What is left is + /// an entity query, whose owned and complex members are read from the EF Core model; a navigation it + /// does not own loads only when something includes it, and a guarded query that has to narrow the + /// row leaves such a navigation out, as it always has. + /// + internal static RowShape Of(IQueryable source) where T : class + { + if (source.Provider is EnumerableQuery) + { + return InMemory; + } + + if (DefaultOrder.IsProjected(source.Expression)) + { + return Projected; + } + + IEntityType? entityType = QueryRoot.EntityType(source.Expression, typeof(T)); + + if (entityType is null) + { + return Entity; + } + + HashSet alwaysLoaded = Empty(); + HashSet complex = Empty(); + + foreach (INavigation navigation in entityType.GetNavigations()) + { + if (navigation.ForeignKey.IsOwnership && !navigation.IsOnDependent) + { + alwaysLoaded.Add(navigation.Name); + } + } + + foreach (string name in ComplexProperties(entityType)) + { + alwaysLoaded.Add(name); + complex.Add(name); + } + + return alwaysLoaded.Count == 0 ? Entity : new RowShape(RowKind.Entity, alwaysLoaded, complex); + } + + private static HashSet Empty() => new(StringComparer.Ordinal); + + /// + /// The names of an entity type's complex properties, which EF Core 8 introduced and loads with the + /// entity. None on an earlier version, where the method does not exist. + /// + /// + /// Read by reflection because the library compiles against EF Core 6. The method is declared on an + /// interface the entity type implements, so the interfaces are searched rather than the class. + /// + private static IEnumerable ComplexProperties(IEntityType entityType) + { + foreach (Type contract in entityType.GetType().GetInterfaces()) + { + MethodInfo? method = contract.GetMethod("GetComplexProperties", Type.EmptyTypes); + + if (method is null) + { + continue; + } + + if (method.Invoke(entityType, null) is IEnumerable properties) + { + foreach (object? property in properties) + { + if (property?.GetType().GetProperty("Name")?.GetValue(property) is string name) + { + yield return name; + } + } + } + + yield break; + } + } +} + +/// Where a guarded query's rows come from. +internal enum RowKind +{ + /// An entity query: a navigation holds a value only when something loads it. + Entity, + + /// A projection the caller built: every member holds a value the projection computed. + Projected, + + /// A sequence in memory: every member holds whatever the object holds. + InMemory +} diff --git a/DynamicWhere.ex/Source/AsyncReads.cs b/DynamicWhere.ex/Source/AsyncReads.cs new file mode 100644 index 0000000..f33255b --- /dev/null +++ b/DynamicWhere.ex/Source/AsyncReads.cs @@ -0,0 +1,77 @@ +using System.Collections; +using System.Linq.Dynamic.Core; +using System.Reflection; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Query; + +namespace DynamicWhere.ex.Source; + +/// +/// Asynchronous reads of a query whose element type is only known at run time: a dynamic projection or +/// a grouping. +/// +/// +/// EF Core's own asynchronous operators are generic in the element type, and a dynamic query's element +/// type is a class generated for its projection, so they are reached by reflection. Through them the +/// provider runs the command asynchronously and a cancellation reaches the database. System.Linq.Dynamic.Core's +/// ToDynamicListAsync, which these replace on an EF Core query, runs the synchronous read on a +/// thread-pool thread and checks the token only before it starts. +/// +/// Any other provider, rows in memory among them, keeps the synchronous read it had. +/// +/// +internal static class AsyncReads +{ + /// EntityFrameworkQueryableExtensions.ToListAsync<TSource>(source, cancellationToken). + private static readonly MethodInfo ToListAsyncMethod = typeof(EntityFrameworkQueryableExtensions) + .GetMethods(BindingFlags.Public | BindingFlags.Static) + .Single(method => method.Name == nameof(EntityFrameworkQueryableExtensions.ToListAsync) + && method.GetParameters().Length == 2); + + /// EntityFrameworkQueryableExtensions.CountAsync<TSource>(source, cancellationToken). + private static readonly MethodInfo CountAsyncMethod = typeof(EntityFrameworkQueryableExtensions) + .GetMethods(BindingFlags.Public | BindingFlags.Static) + .Single(method => method.Name == nameof(EntityFrameworkQueryableExtensions.CountAsync) + && method.GetParameters().Length == 2 + && method.GetParameters()[1].ParameterType == typeof(CancellationToken)); + + /// Reads every row of a query into a list of dynamic objects. + internal static async Task> ToDynamicListAsync(IQueryable query, CancellationToken cancellationToken) + { + if (query.Provider is not IAsyncQueryProvider) + { + return await query.ToDynamicListAsync(cancellationToken); + } + + Task read = (Task)ToListAsyncMethod + .MakeGenericMethod(query.ElementType) + .Invoke(null, new object[] { query, cancellationToken })!; + + await read; + + IList rows = (IList)read.GetType().GetProperty(nameof(Task.Result))!.GetValue(read)!; + List list = new(rows.Count); + + foreach (object? row in rows) + { + list.Add(row!); + } + + return list; + } + + /// Counts the rows of a query. + internal static async Task CountAsync(IQueryable query, CancellationToken cancellationToken) + { + if (query.Provider is not IAsyncQueryProvider) + { + cancellationToken.ThrowIfCancellationRequested(); + + return query.Count(); + } + + return await (Task)CountAsyncMethod + .MakeGenericMethod(query.ElementType) + .Invoke(null, new object[] { query, cancellationToken })!; + } +} diff --git a/DynamicWhere.ex/Source/Extention.cs b/DynamicWhere.ex/Source/Extention.cs index 05f3d45..9149c40 100644 --- a/DynamicWhere.ex/Source/Extention.cs +++ b/DynamicWhere.ex/Source/Extention.cs @@ -681,7 +681,36 @@ public static FilterResult ToListDynamic(this IEnumerable query, /// A containing entities that match the filter conditions in the with pagination information. /// Thrown if either or is null. /// Thrown when contains invalid data. - public static async Task> ToListAsync(this IQueryable query, Filter filter, bool getQueryString = false) where T : class + public static Task> ToListAsync(this IQueryable query, Filter filter, bool getQueryString = false) where T : class => + query.ToListAsync(filter, getQueryString, CancellationToken.None); + + /// + /// Asynchronously retrieves a list of entities from the with optional filtering based on a . + /// + /// The entity type. + /// The to retrieve entities from. + /// The containing filter conditions and optional pagination settings. + /// Cancels the count and the read. + /// A containing entities that match the filter conditions in the with pagination information. + /// Thrown if either or is null. + /// Thrown when contains invalid data. + /// Thrown when is canceled. + public static Task> ToListAsync(this IQueryable query, Filter filter, CancellationToken cancellationToken) where T : class => + query.ToListAsync(filter, false, cancellationToken); + + /// + /// Asynchronously retrieves a list of entities from the with optional filtering based on a . + /// + /// The entity type. + /// The to retrieve entities from. + /// The containing filter conditions and optional pagination settings. + /// If true, includes the generated query string in the result. + /// Cancels the count and the read. + /// A containing entities that match the filter conditions in the with pagination information. + /// Thrown if either or is null. + /// Thrown when contains invalid data. + /// Thrown when is canceled. + public static async Task> ToListAsync(this IQueryable query, Filter filter, bool getQueryString, CancellationToken cancellationToken) where T : class { // Refuse a type that requires a policy context when the call is not inside one. PolicyScope.Require(); @@ -730,7 +759,7 @@ public static async Task> ToListAsync(this IQueryable quer } // Calculate the total count of entities before pagination. - int totalCount = await query.CountAsync(); + int totalCount = await query.CountAsync(cancellationToken); // Calculate the total page count based on the page size. An unpaged query is one page of // everything: dividing by one reported as many pages as there were rows, which read as a @@ -748,7 +777,7 @@ public static async Task> ToListAsync(this IQueryable quer TotalCount = totalCount, // Execute the query to retrieve the data. - Data = await newQuery.ToListAsync(), + Data = await newQuery.ToListAsync(cancellationToken), QueryString = getQueryString ? newQuery.ToQueryString() : null }; } @@ -764,7 +793,38 @@ public static async Task> ToListAsync(this IQueryable quer /// A of dynamic containing dynamic objects that match the filter conditions with pagination information. /// Thrown if either or is null. /// Thrown when contains invalid data. - public static async Task> ToListAsyncDynamic(this IQueryable query, Filter filter, bool getQueryString = false) where T : class + public static Task> ToListAsyncDynamic(this IQueryable query, Filter filter, bool getQueryString = false) where T : class => + query.ToListAsyncDynamic(filter, getQueryString, CancellationToken.None); + + /// + /// Asynchronously retrieves a list of dynamic objects from the with optional filtering based on a , + /// using for the field projection. + /// + /// The entity type. + /// The to retrieve entities from. + /// The containing filter conditions and optional pagination settings. + /// Cancels the count and the read. + /// A of dynamic containing dynamic objects that match the filter conditions with pagination information. + /// Thrown if either or is null. + /// Thrown when contains invalid data. + /// Thrown when is canceled. + public static Task> ToListAsyncDynamic(this IQueryable query, Filter filter, CancellationToken cancellationToken) where T : class => + query.ToListAsyncDynamic(filter, false, cancellationToken); + + /// + /// Asynchronously retrieves a list of dynamic objects from the with optional filtering based on a , + /// using for the field projection. + /// + /// The entity type. + /// The to retrieve entities from. + /// The containing filter conditions and optional pagination settings. + /// If true, includes the generated query string in the result. + /// Cancels the count and the read. + /// A of dynamic containing dynamic objects that match the filter conditions with pagination information. + /// Thrown if either or is null. + /// Thrown when contains invalid data. + /// Thrown when is canceled. + public static async Task> ToListAsyncDynamic(this IQueryable query, Filter filter, bool getQueryString, CancellationToken cancellationToken) where T : class { // Refuse a type that requires a policy context when the call is not inside one. PolicyScope.Require(); @@ -786,7 +846,7 @@ public static async Task> ToListAsyncDynamic(this IQuer } // Calculate the total count of entities before pagination. - int totalCount = await query.CountAsync(); + int totalCount = await query.CountAsync(cancellationToken); // Create a new query to apply ordering and pagination. IQueryable newQuery = query; @@ -830,7 +890,7 @@ public static async Task> ToListAsyncDynamic(this IQuer TotalCount = totalCount, // Execute the query to retrieve the dynamic data asynchronously. - Data = await result.ToDynamicListAsync(), + Data = await AsyncReads.ToDynamicListAsync(result, cancellationToken), QueryString = getQueryString ? result.ToQueryString() : null }; } @@ -1038,7 +1098,36 @@ public static SummaryResult ToList(this IEnumerable query, Summary summary /// A containing grouped entities that match the summary criteria with pagination information. /// Thrown if either or is null. /// Thrown when contains invalid data. - public static async Task ToListAsync(this IQueryable query, Summary summary, bool getQueryString = false) where T : class + public static Task ToListAsync(this IQueryable query, Summary summary, bool getQueryString = false) where T : class => + query.ToListAsync(summary, getQueryString, CancellationToken.None); + + /// + /// Asynchronously retrieves a list of dynamic grouped entities from the with optional filtering based on a . + /// + /// The entity type. + /// The to retrieve grouped entities from. + /// The containing filter conditions, group-by criteria, and optional pagination settings. + /// Cancels the count and the read. + /// A containing grouped entities that match the summary criteria with pagination information. + /// Thrown if either or is null. + /// Thrown when contains invalid data. + /// Thrown when is canceled. + public static Task ToListAsync(this IQueryable query, Summary summary, CancellationToken cancellationToken) where T : class => + query.ToListAsync(summary, false, cancellationToken); + + /// + /// Asynchronously retrieves a list of dynamic grouped entities from the with optional filtering based on a . + /// + /// The entity type. + /// The to retrieve grouped entities from. + /// The containing filter conditions, group-by criteria, and optional pagination settings. + /// If true, includes the generated query string in the result. + /// Cancels the count and the read. + /// A containing grouped entities that match the summary criteria with pagination information. + /// Thrown if either or is null. + /// Thrown when contains invalid data. + /// Thrown when is canceled. + public static async Task ToListAsync(this IQueryable query, Summary summary, bool getQueryString, CancellationToken cancellationToken) where T : class { // Refuse a type that requires a policy context when the call is not inside one. PolicyScope.Require(); @@ -1077,7 +1166,7 @@ public static async Task ToListAsync(this IQueryable query, } // Calculate the total count of grouped entities before pagination. - int totalCount = result.Count(); + int totalCount = await AsyncReads.CountAsync(result, cancellationToken); // Create a new query to apply ordering and pagination. IQueryable newResult = result; @@ -1126,7 +1215,7 @@ public static async Task ToListAsync(this IQueryable query, TotalCount = totalCount, // Execute the query to retrieve the data asynchronously. - Data = await newResult.ToDynamicListAsync(), + Data = await AsyncReads.ToDynamicListAsync(newResult, cancellationToken), QueryString = getQueryString ? newResult.ToQueryString() : null }; } @@ -1147,7 +1236,25 @@ public static async Task ToListAsync(this IQueryable query, /// the total count then run exactly as they do for a , so only the requested page /// is read. /// - public static async Task> ToListAsync(this IQueryable query, Segment segment) where T : class + public static Task> ToListAsync(this IQueryable query, Segment segment) where T : class => + query.ToListAsync(segment, CancellationToken.None); + + /// + /// Asynchronously retrieves a list of entities from the with optional filtering based on a . + /// + /// The entity type. + /// The to retrieve entities from. + /// The containing filter conditions and optional pagination settings. + /// Cancels the count and the read. + /// A containing entities that match the filter conditions in the with pagination information. + /// Thrown if either or is null. + /// Thrown when contains invalid data. + /// Thrown when is canceled. + /// + /// The condition sets become one query, which the database answers, exactly as they do for the + /// overload without a token. + /// + public static async Task> ToListAsync(this IQueryable query, Segment segment, CancellationToken cancellationToken) where T : class { // Refuse a type that requires a policy context when the call is not inside one. PolicyScope.Require(); @@ -1178,7 +1285,7 @@ public static async Task> ToListAsync(this IQueryable que Page = segment.Page }; - FilterResult fresult = await combined.ToListAsync(filter); + FilterResult fresult = await combined.ToListAsync(filter, false, cancellationToken); // Return the results as a SegmentResult. return new() diff --git a/DynamicWhere.ex/Source/QueryRoot.cs b/DynamicWhere.ex/Source/QueryRoot.cs new file mode 100644 index 0000000..acfcf5e --- /dev/null +++ b/DynamicWhere.ex/Source/QueryRoot.cs @@ -0,0 +1,64 @@ +using System.Collections.Concurrent; +using System.Linq.Expressions; +using System.Reflection; +using Microsoft.EntityFrameworkCore.Metadata; + +namespace DynamicWhere.ex.Source; + +/// +/// The EF Core model behind a query, read from the query's root. +/// +/// +/// The root expression's type differs between EF Core versions — QueryRootExpression in 6, +/// EntityQueryRootExpression from 7 — and both carry the entity type in a property named +/// EntityType, so the property is found by name. Any root reaches the model, and the model is +/// asked for the type the caller names, which also answers for a derived type queried through +/// OfType. +/// +internal static class QueryRoot +{ + /// The EntityType property of each query-root expression type, or null for a type with none. + private static readonly ConcurrentDictionary RootEntityTypeProperties = new(); + + /// + /// The entity type the model maps for , or null when the query is not an + /// EF Core query or the model does not map that type. + /// + internal static IEntityType? EntityType(Expression expression, Type type) + { + RootFinder finder = new(); + + finder.Visit(expression); + + return finder.EntityType?.Model.FindEntityType(type); + } + + /// Finds the first query root that names an entity type. + private sealed class RootFinder : ExpressionVisitor + { + public IEntityType? EntityType { get; private set; } + + public override Expression? Visit(Expression? node) => EntityType is null ? base.Visit(node) : node; + + protected override Expression VisitExtension(Expression node) + { + // Enumerated rather than looked up by name, which throws when a derived root hides the + // property with a new one. + PropertyInfo? property = RootEntityTypeProperties.GetOrAdd( + node.GetType(), + type => type + .GetProperties(BindingFlags.Public | BindingFlags.Instance) + .FirstOrDefault(candidate => candidate.Name == "EntityType" + && candidate.GetIndexParameters().Length == 0)); + + if (property?.GetValue(node) is IEntityType entityType) + { + EntityType = entityType; + + return node; + } + + return base.VisitExtension(node); + } + } +} diff --git a/DynamicWhere.ex/Source/SegmentComposer.cs b/DynamicWhere.ex/Source/SegmentComposer.cs index f40b015..f3b69a4 100644 --- a/DynamicWhere.ex/Source/SegmentComposer.cs +++ b/DynamicWhere.ex/Source/SegmentComposer.cs @@ -1,4 +1,3 @@ -using System.Collections.Concurrent; using System.Linq.Expressions; using System.Reflection; using DynamicWhere.ex.Classes.Core; @@ -40,9 +39,6 @@ namespace DynamicWhere.ex.Source; /// internal static class SegmentComposer { - /// The EntityType property of each query-root expression type, or null for a type with none. - private static readonly ConcurrentDictionary RootEntityTypeProperties = new(); - /// /// Combines validated condition sets, already in Sort order, into one query. /// @@ -205,50 +201,8 @@ private static Expression SameValue(Expression left, Expression right) /// The primary key EF Core maps for , or null when the query is not an EF Core /// query or the type has none. /// - /// - /// Read from the model behind the query's root. The root expression's type differs between EF Core - /// versions — QueryRootExpression in 6, EntityQueryRootExpression from 7 — and both - /// carry the entity type in a property named EntityType, so the property is found by name. - /// Any root reaches the model, and the model is asked for itself, which - /// also answers for a derived type queried through OfType. - /// - private static IReadOnlyList? PrimaryKey(IQueryable query) - { - RootFinder finder = new(); - - finder.Visit(query.Expression); - - return finder.EntityType?.Model.FindEntityType(typeof(T))?.FindPrimaryKey()?.Properties; - } - - /// Finds the first query root that names an entity type. - private sealed class RootFinder : ExpressionVisitor - { - public IEntityType? EntityType { get; private set; } - - public override Expression? Visit(Expression? node) => EntityType is null ? base.Visit(node) : node; - - protected override Expression VisitExtension(Expression node) - { - // Enumerated rather than looked up by name, which throws when a derived root hides the - // property with a new one. - PropertyInfo? property = RootEntityTypeProperties.GetOrAdd( - node.GetType(), - type => type - .GetProperties(BindingFlags.Public | BindingFlags.Instance) - .FirstOrDefault(candidate => candidate.Name == "EntityType" - && candidate.GetIndexParameters().Length == 0)); - - if (property?.GetValue(node) is IEntityType entityType) - { - EntityType = entityType; - - return node; - } - - return base.VisitExtension(node); - } - } + private static IReadOnlyList? PrimaryKey(IQueryable query) => + QueryRoot.EntityType(query.Expression, typeof(T))?.FindPrimaryKey()?.Properties; /// Replaces one lambda parameter with another expression. private sealed class ParameterSwap : ExpressionVisitor From b45a47f0bc512c2d7795291176b8077c54e1381f Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 04:34:36 +0300 Subject: [PATCH 02/22] =?UTF-8?q?release:=203.2.0=20=E2=80=94=20the=20vers?= =?UTF-8?q?ion,=20the=20release=20notes=20and=20the=20agent=20reference?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All four packages move to 3.2.0 together. The core's release notes describe the three security fixes, the synthesized projection that keeps what the source carries, the default order on projected sources and the CancellationToken overloads. The companions say they changed nothing of their own, and the ASP.NET Core package says what /simulate now reports. llms.txt carries the new overloads and the method count (28), the ambiguity a default literal now causes, the enforcement table's new rows and the "allowed members" rule that replaces "allowed scalars", the default-order rule for projections, the trace's "left out whole" decisions, the simulator's entity reading, the 3.2.0 history entry, and two limits: an entity's navigations are left out once a projection is needed, and a type held in a dictionary is not policed through it. The trap that said nothing takes a CancellationToken is replaced. Co-Authored-By: Claude Opus 5 --- ...DynamicWhere.ex.Policies.AspNetCore.csproj | 6 +- ...ere.ex.Policies.EntityFrameworkCore.csproj | 6 +- .../DynamicWhere.ex.Policies.Redis.csproj | 6 +- DynamicWhere.ex/DOC.md | 4 +- DynamicWhere.ex/DynamicWhere.ex.csproj | 22 ++- OfficialWebsite/app/docs/ai/page.tsx | 2 +- .../app/docs/installation/page.tsx | 6 +- OfficialWebsite/app/docs/page.tsx | 4 +- .../app/docs/policies/admin/page.tsx | 2 +- .../app/docs/policies/providers/page.tsx | 4 +- OfficialWebsite/lib/nav.ts | 2 +- OfficialWebsite/package.json | 2 +- OfficialWebsite/public/llms.txt | 172 +++++++++++++++--- README.md | 4 +- 14 files changed, 188 insertions(+), 54 deletions(-) diff --git a/DynamicWhere.ex.Policies.AspNetCore/DynamicWhere.ex.Policies.AspNetCore.csproj b/DynamicWhere.ex.Policies.AspNetCore/DynamicWhere.ex.Policies.AspNetCore.csproj index b97432c..80e8f71 100644 --- a/DynamicWhere.ex.Policies.AspNetCore/DynamicWhere.ex.Policies.AspNetCore.csproj +++ b/DynamicWhere.ex.Policies.AspNetCore/DynamicWhere.ex.Policies.AspNetCore.csproj @@ -31,8 +31,10 @@ Sajjad H. Al-Khafaji Free Forever — Copyright © Sajjad H. Al-Khafaji MIT - 3.1.0 - v3.1.0 — Released in lockstep with DynamicWhere.ex 3.1.0, which it now requires. No API change in this package. The core release changes what its endpoints report, as it changes any guarded call: /simulate answers with 3.1.0's pipeline, the new caps, the strict tier's refusals and a type's DefaultOrder included, and /health's loadedAt and ageSeconds advance on a poll that confirms the version served, so an idle healthy store no longer reads as stale. One change needs action: DwClaimsAdapter.FromClaims returns an unprepared context, and ApplyPolicy(context) now refuses one with PolicyContextNotPrepared even when no store is configured. Use CreateContextAsync or ToPolicyContextAsync, or pass the context through DwPolicy.PrepareAsync before querying. In this package, the audit middleware's warning for events recorded with no IDwAuditSink registered now names both ways to stop recording them: remove [DwAudit] from the fields that produced them, or turn off DwPolicyOptions.AuditRefusals, the core's new switch that records refused guarded queries as audit events, which the middleware drains like any other. + 3.2.0 + v3.2.0 — Released in lockstep with DynamicWhere.ex 3.2.0, which it now requires. No API change in this package. The core release changes what /simulate reports for a request that sends no Selects. The projection it shows is now built whenever a field is denied at any depth, and keeps members holding a collection of values. A simulation has no source, so it reads the type as an entity query and leaves out members holding an object, which a guarded query over a projected or in-memory source keeps. + +v3.1.0 — Released in lockstep with DynamicWhere.ex 3.1.0, which it now requires. No API change in this package. The core release changes what its endpoints report, as it changes any guarded call: /simulate answers with 3.1.0's pipeline, the new caps, the strict tier's refusals and a type's DefaultOrder included, and /health's loadedAt and ageSeconds advance on a poll that confirms the version served, so an idle healthy store no longer reads as stale. One change needs action: DwClaimsAdapter.FromClaims returns an unprepared context, and ApplyPolicy(context) now refuses one with PolicyContextNotPrepared even when no store is configured. Use CreateContextAsync or ToPolicyContextAsync, or pass the context through DwPolicy.PrepareAsync before querying. In this package, the audit middleware's warning for events recorded with no IDwAuditSink registered now names both ways to stop recording them: remove [DwAudit] from the fields that produced them, or turn off DwPolicyOptions.AuditRefusals, the core's new switch that records refused guarded queries as audit events, which the middleware drains like any other. v3.0.0 — First release. The administrative surface for the DynamicWhere.ex field-level policy layer, introduced in DynamicWhere.ex 3.0.0. MapDwPolicyAdmin mounts schema discovery, rule management, explain, simulate and health, and refuses to map at all without a named authorization policy — there is deliberately no default, and it fails at startup rather than on the first request. diff --git a/DynamicWhere.ex.Policies.EntityFrameworkCore/DynamicWhere.ex.Policies.EntityFrameworkCore.csproj b/DynamicWhere.ex.Policies.EntityFrameworkCore/DynamicWhere.ex.Policies.EntityFrameworkCore.csproj index 9367e98..6837aa4 100644 --- a/DynamicWhere.ex.Policies.EntityFrameworkCore/DynamicWhere.ex.Policies.EntityFrameworkCore.csproj +++ b/DynamicWhere.ex.Policies.EntityFrameworkCore/DynamicWhere.ex.Policies.EntityFrameworkCore.csproj @@ -34,8 +34,10 @@ Free Forever — Copyright © Sajjad H. Al-Khafaji MIT - 3.1.0 - v3.1.0 — Released in lockstep with DynamicWhere.ex 3.1.0, which it now requires. No API or behaviour change in this package. A stored rule's forced predicate can now carry allowNull, which lets rows with no value through: the core's PolicyRuleDocument writes it into the Detail column's forced object as "allowNull": true, only when it is true, so no migration is needed and a rule without it is stored exactly as before. A row whose forced object pairs IsNull or IsNotNull with "allowNull": true, value or no value, or with a contextValue, is refused when it is read, like any row the core cannot read: a Global, Tenant, Role or Custom rule fails the load, and a User rule fails DwPolicy.PrepareAsync for that user. + 3.2.0 + v3.2.0 — Released in lockstep with DynamicWhere.ex 3.2.0, which it now requires. No API or behaviour change in this package. + +v3.1.0 — Released in lockstep with DynamicWhere.ex 3.1.0, which it now requires. No API or behaviour change in this package. A stored rule's forced predicate can now carry allowNull, which lets rows with no value through: the core's PolicyRuleDocument writes it into the Detail column's forced object as "allowNull": true, only when it is true, so no migration is needed and a rule without it is stored exactly as before. A row whose forced object pairs IsNull or IsNotNull with "allowNull": true, value or no value, or with a contextValue, is refused when it is read, like any row the core cannot read: a Global, Tenant, Role or Custom rule fails the load, and a User rule fails DwPolicy.PrepareAsync for that user. v3.0.0 — First release. A database-backed policy store for the DynamicWhere.ex field-level policy layer, introduced in DynamicWhere.ex 3.0.0. Works with any EF Core provider; ships DwPolicyDbContext, the rule and version records, and their EF configurations. Passes the same store conformance suite as the in-memory and Redis stores, verified against PostgreSQL. diff --git a/DynamicWhere.ex.Policies.Redis/DynamicWhere.ex.Policies.Redis.csproj b/DynamicWhere.ex.Policies.Redis/DynamicWhere.ex.Policies.Redis.csproj index 931b439..9b1ea88 100644 --- a/DynamicWhere.ex.Policies.Redis/DynamicWhere.ex.Policies.Redis.csproj +++ b/DynamicWhere.ex.Policies.Redis/DynamicWhere.ex.Policies.Redis.csproj @@ -34,8 +34,10 @@ - 3.1.0 - v3.1.0 — Released in lockstep with DynamicWhere.ex 3.1.0, which it now requires. No API or behaviour change in this package. A stored rule's forced predicate can now carry allowNull, which lets rows with no value through: the core's PolicyRuleDocument writes it into the rule document's forced object as "allowNull": true, only when it is true, so a rule without it is stored exactly as before. A document whose forced object pairs IsNull or IsNotNull with "allowNull": true, value or no value, or with a contextValue, is refused when it is read, like any document the core cannot read: a Global, Tenant, Role or Custom rule fails the load, and a User rule fails DwPolicy.PrepareAsync for that user. + 3.2.0 + v3.2.0 — Released in lockstep with DynamicWhere.ex 3.2.0, which it now requires. No API or behaviour change in this package. + +v3.1.0 — Released in lockstep with DynamicWhere.ex 3.1.0, which it now requires. No API or behaviour change in this package. A stored rule's forced predicate can now carry allowNull, which lets rows with no value through: the core's PolicyRuleDocument writes it into the rule document's forced object as "allowNull": true, only when it is true, so a rule without it is stored exactly as before. A document whose forced object pairs IsNull or IsNotNull with "allowNull": true, value or no value, or with a contextValue, is refused when it is read, like any document the core cannot read: a Global, Tenant, Role or Custom rule fails the load, and a User rule fails DwPolicy.PrepareAsync for that user. v3.0.0 — First release. A Redis-backed policy store for the DynamicWhere.ex field-level policy layer, introduced in DynamicWhere.ex 3.0.0. Holds runtime rules in Redis and invalidates through pub/sub with a poll behind it, because pub/sub is fire-and-forget and the poll is what bounds a dropped message. Passes the same store conformance suite as the in-memory and Entity Framework Core stores. diff --git a/DynamicWhere.ex/DOC.md b/DynamicWhere.ex/DOC.md index c12a4e5..6c3c9a9 100644 --- a/DynamicWhere.ex/DOC.md +++ b/DynamicWhere.ex/DOC.md @@ -1,6 +1,6 @@ # DynamicWhere.ex -**Version:** 3.1.0  |  **Target Framework:** .NET 6+  |  **License:** MIT (Free Forever) +**Version:** 3.2.0  |  **Target Framework:** .NET 6+  |  **License:** MIT (Free Forever) > A powerful and versatile library for dynamically creating complex filter, sort, paginate, group, aggregate, and set-operation expressions in Entity Framework Core applications — all driven by simple JSON objects from any front-end or API consumer. @@ -31,7 +31,7 @@ ## Installation ```bash -dotnet add package DynamicWhere.ex --version 3.1.0 +dotnet add package DynamicWhere.ex --version 3.2.0 ``` **Dependencies:** diff --git a/DynamicWhere.ex/DynamicWhere.ex.csproj b/DynamicWhere.ex/DynamicWhere.ex.csproj index 370045a..d0671b3 100644 --- a/DynamicWhere.ex/DynamicWhere.ex.csproj +++ b/DynamicWhere.ex/DynamicWhere.ex.csproj @@ -34,21 +34,37 @@ DynamicWhere.ex DynamicWhere.ex — JSON-driven queries for EF Core - JSON-driven queries for Entity Framework Core. A powerful, versatile library for dynamically composing complex filter, sort, paginate, group, aggregate, and set-operation (Union / Intersect / Except) expressions — all driven by simple JSON objects from any front-end or API consumer. Three composable shapes (Filter / Segment / Summary), twenty-one extension methods, nested navigation through references and collections with auto-wrapped .Any() lambdas, heterogeneous Condition.Values with type-safe coercion, and a thread-safe reflection cache with five tuned presets (FIFO / LRU / LFU). Since 3.0 it also carries an opt-in field-level policy layer: attributes and runtime rules decide what each caller may filter, sort, select, group, aggregate and see, with masking, tokenization, forced predicates and a k-anonymity floor. Targets .NET 6+. Free Forever. Full reference, JSON cookbook, and tuning guide at https://doc.dynamicwhere.com. + JSON-driven queries for Entity Framework Core. A powerful, versatile library for dynamically composing complex filter, sort, paginate, group, aggregate, and set-operation (Union / Intersect / Except) expressions — all driven by simple JSON objects from any front-end or API consumer. Three composable shapes (Filter / Segment / Summary), twenty-eight extension methods, nested navigation through references and collections with auto-wrapped .Any() lambdas, heterogeneous Condition.Values with type-safe coercion, and a thread-safe reflection cache with five tuned presets (FIFO / LRU / LFU). Since 3.0 it also carries an opt-in field-level policy layer: attributes and runtime rules decide what each caller may filter, sort, select, group, aggregate and see, with masking, tokenization, forced predicates and a k-anonymity floor. Targets .NET 6+. Free Forever. Full reference, JSON cookbook, and tuning guide at https://doc.dynamicwhere.com. Dynamic filters, sort, paginate, group, aggregate, and set operations for EF Core — driven by JSON. DynamicWhere DynamicWhere.ex EFCore EntityFrameworkCore EF-Core LINQ DynamicLinq Filter DynamicFilter JsonFilter Where OrderBy Paging Pagination GroupBy Aggregation Summary Segment Union Intersect Except SetOperations QueryBuilder Query Expression ExpressionTree Reflection Cache JSON Dynamic FieldLevelSecurity DataMasking Tokenization Anonymization KAnonymity Authorization RBAC ABAC Multitenancy DotNet NET6 NET7 NET8 NET9 NET10 CSharp Sajjad H. Al-Khafaji Sajjad H. Al-Khafaji Free Forever — Copyright © Sajjad H. Al-Khafaji MIT - 3.1.0 + 3.2.0 https://doc.dynamicwhere.com git master https://github.com/Sajadh92/DynamicWhere.ex icon.png README.md - v3.1.0 — Date comparisons that work on every date member, segments combined in the database, five new caps, a stable code where a sentence used to be, preparation enforced whether or not a store is configured, a policy bypass through members named Root, It or Parent closed, a long In list that ended the process fixed, the names the expression parser keeps refused by name, a strict tier that no longer discloses its trace or which fields exist, forced predicates that can let rows with no value through, a declared default order for guarded queries, and refused queries written to the audit. + v3.2.0 — A row projected before ApplyPolicy keeps its nested objects and lists, a denial beneath a member is enforced, a declared default order reaches projected rows, and every asynchronous terminal takes a CancellationToken. + +Security fix: a field denied for Select only beneath a member was not enforced when the caller sent no Selects. A guarded query synthesizes a projection for a denied field, and it did so only when a top-level field was denied. With every denial beneath a member, the whole row came back with the denied value in it: in a list or nested object of a row projected before ApplyPolicy, in a row held in memory, and in an entity's included, automatically included, lazily loaded or owned member, typed and dynamic, in both tiers, for a Filter and a Segment. A denial at any depth now synthesizes the projection. + +Security fix: under the convenience tier, Selects naming a navigation whose element key (Id) is denied was narrowed to the allowed fields beneath it, and the core's typed projection added the key back. Such a narrowing is refused with FieldDeniedForSelect in both tiers, as naming a sibling of the key already was. + +Security fix: Selects naming a member typed as a collection the core does not unwrap, such as IReadOnlyList<T>, returned every field beneath it, denied ones included, in both tiers. The projection gate read collections through a narrower list than the attribute walker that puts policy on the fields beneath. It now reads them the same way, and a narrowing the core cannot project is refused with FieldDeniedForSelect. + +Behaviour change: the projection synthesized for a guarded query that sends no Selects keeps what the source carries. It used to keep scalars only, so every nested object and list of a row projected before ApplyPolicy came back null or empty as soon as any field was denied. A member holding a value is kept when allowed, and that now includes a collection of values such as byte[] or List<string>. A member holding an object, or a list of them, is kept whole when nothing beneath it is denied, and narrowed to the allowed fields when something is. That applies to a projected row, a row in memory, and an entity's owned and complex members, which are read from the EF Core model. An entity's other navigations are left out, as they were: an included navigation is not returned once a denial anywhere in the type needs a projection, and the type needs a public parameterless constructor for it, as it already did for a top-level denial. A member that cannot be narrowed is left out whole and recorded as Dropped with a reason starting "left out whole": one in memory, an EF Core complex property, one whose key is denied, and one the core cannot project through. + +Behaviour change: [DwEntity(DefaultOrder)] applies to a projected source whose outermost Select builds the type in an object initializer and assigns every field the default names, at every level of a nested path. Any other projection still leaves the query in its own order. A Select, or a Filter with Selects, composed on the guarded handle keeps the rest of the chain unordered, and a composed Filter that sent orders gets no default later in the chain, as a composed Order already did not. + +New: every asynchronous terminal has overloads taking a CancellationToken, guarded and unguarded: ToListAsync and ToListAsyncDynamic with a Filter, ToListAsync with a Summary, and ToListAsync with a Segment. The token reaches both the count and the read. The overloads sit beside the 3.1 signatures, which are unchanged, so code compiled against 3.1 still binds. ToListAsync(filter, default) no longer compiles, because default fits both overloads. + +Change: ToListAsyncDynamic and the asynchronous Summary read through EF Core's ToListAsync, and the Summary counts through CountAsync. They used to read synchronously on a thread-pool thread, so on an EF Core query a canceled token now reaches the database. A provider that is not EF Core's keeps the synchronous read. + +v3.1.0 — Date comparisons that work on every date member, segments combined in the database, five new caps, a stable code where a sentence used to be, preparation enforced whether or not a store is configured, a policy bypass through members named Root, It or Parent closed, a long In list that ended the process fixed, the names the expression parser keeps refused by name, a strict tier that no longer discloses its trace or which fields exist, forced predicates that can let rows with no value through, a declared default order for guarded queries, and refused queries written to the audit. Security fix: a member named Root, It or Parent was read as a System.Linq.Dynamic.Core keyword. Root.Name and It.Name filtered, sorted, grouped, aggregated and projected the row's own Name, Parent threw, and an alias named root, it or parent failed in Having and Summary orders. Under ApplyPolicy the gate decided on the path the caller named while the query read the row's own column: a dynamic projection of Root.Name returned a [DwDenied] Name, a filter on it tested the denied column, and a [DwForceWhere] scope reached through such a navigation filtered the row's own column. Every expression is now parsed with a library-owned ParsingConfig with the context keywords off. ParsingConfig.Default is no longer read, so a host's changes to it no longer reach DynamicWhere queries. diff --git a/OfficialWebsite/app/docs/ai/page.tsx b/OfficialWebsite/app/docs/ai/page.tsx index ccb5646..e485a4b 100644 --- a/OfficialWebsite/app/docs/ai/page.tsx +++ b/OfficialWebsite/app/docs/ai/page.tsx @@ -136,7 +136,7 @@ DynamicWhere.ex code. It is the complete API surface.`} - The reference is generated against version 3.1.0 from the source, and + The reference is generated against version 3.2.0 from the source, and checked by running the library, so it states behaviour — including the parts that are deliberately blunt, such as neither hashing nor tokenization hiding equality. Where a page in these docs disagrees with it, diff --git a/OfficialWebsite/app/docs/installation/page.tsx b/OfficialWebsite/app/docs/installation/page.tsx index aafdb54..88a27ce 100644 --- a/OfficialWebsite/app/docs/installation/page.tsx +++ b/OfficialWebsite/app/docs/installation/page.tsx @@ -28,14 +28,14 @@ export default function Page() {

dotnet CLI

- {`dotnet add package DynamicWhere.ex --version 3.1.0`} + {`dotnet add package DynamicWhere.ex --version 3.2.0`}

Package Manager (Visual Studio)

- {`Install-Package DynamicWhere.ex -Version 3.1.0`} + {`Install-Package DynamicWhere.ex -Version 3.2.0`}

PackageReference (csproj)

{` - + `}

Dependencies

diff --git a/OfficialWebsite/app/docs/page.tsx b/OfficialWebsite/app/docs/page.tsx index d928124..8101e10 100644 --- a/OfficialWebsite/app/docs/page.tsx +++ b/OfficialWebsite/app/docs/page.tsx @@ -30,7 +30,7 @@ export default function Page() {

- Version 3.1.0. Target framework .NET 6+. + Version 3.2.0. Target framework .NET 6+. License MIT — free forever, for commercial and personal use.

@@ -83,7 +83,7 @@ export default function Page() {

30-second tour

Install the package:

- {`dotnet add package DynamicWhere.ex --version 3.1.0`} + {`dotnet add package DynamicWhere.ex --version 3.2.0`}

Build a filter from a JSON body and apply it to a DbSet:

{`using DynamicWhere.ex.Source; diff --git a/OfficialWebsite/app/docs/policies/admin/page.tsx b/OfficialWebsite/app/docs/policies/admin/page.tsx index e087a85..a468b60 100644 --- a/OfficialWebsite/app/docs/policies/admin/page.tsx +++ b/OfficialWebsite/app/docs/policies/admin/page.tsx @@ -15,7 +15,7 @@ export default function Page() { return (

Admin API

- {`dotnet add package DynamicWhere.ex.Policies.AspNetCore --version 3.1.0`} + {`dotnet add package DynamicWhere.ex.Policies.AspNetCore --version 3.2.0`} {`app.MapDwPolicyAdmin(options => { options.RoutePrefix = "/dw-policies"; // the default; mount it anywhere diff --git a/OfficialWebsite/app/docs/policies/providers/page.tsx b/OfficialWebsite/app/docs/policies/providers/page.tsx index ba2f1d9..1cdead2 100644 --- a/OfficialWebsite/app/docs/policies/providers/page.tsx +++ b/OfficialWebsite/app/docs/policies/providers/page.tsx @@ -22,7 +22,7 @@ export default function Page() {

Redis

- {`dotnet add package DynamicWhere.ex.Policies.Redis --version 3.1.0`} + {`dotnet add package DynamicWhere.ex.Policies.Redis --version 3.2.0`} {`var redis = await ConnectionMultiplexer.ConnectAsync(connectionString); var store = new RedisPolicyStore(redis); @@ -37,7 +37,7 @@ DwPolicy.Configure(options, provider);`}

Entity Framework Core

- {`dotnet add package DynamicWhere.ex.Policies.EntityFrameworkCore --version 3.1.0`} + {`dotnet add package DynamicWhere.ex.Policies.EntityFrameworkCore --version 3.2.0`} {`var policyDbOptions = new DbContextOptionsBuilder() .UseNpgsql(connection, sql => sql.MigrationsAssembly("YourProject")) .Options; diff --git a/OfficialWebsite/lib/nav.ts b/OfficialWebsite/lib/nav.ts index f1c8223..077ead3 100644 --- a/OfficialWebsite/lib/nav.ts +++ b/OfficialWebsite/lib/nav.ts @@ -8,7 +8,7 @@ export const SITE = { "DynamicWhere.ex is a free .NET library for building dynamic, JSON-driven LINQ queries on Entity Framework Core — filter, sort, paginate, project, group, aggregate, and run UNION / INTERSECT / EXCEPT set operations from your front-end. Works with ASP.NET Core on .NET 6, 7, 8, 9, and 10.", shortDescription: "Dynamic JSON filter, sort, paginate, group, aggregate, and set operations for EF Core. .NET 6/7/8/9/10.", - version: "3.1.0", + version: "3.2.0", domain: "doc.dynamicwhere.com", repo: "https://github.com/Sajadh92/DynamicWhere.ex", nuget: "https://www.nuget.org/packages/DynamicWhere.ex", diff --git a/OfficialWebsite/package.json b/OfficialWebsite/package.json index 9843f75..f17a1de 100644 --- a/OfficialWebsite/package.json +++ b/OfficialWebsite/package.json @@ -1,6 +1,6 @@ { "name": "dynamicwhere-docs", - "version": "3.1.0", + "version": "3.2.0", "private": true, "scripts": { "dev": "next dev", diff --git a/OfficialWebsite/public/llms.txt b/OfficialWebsite/public/llms.txt index a7f3129..8f74f93 100644 --- a/OfficialWebsite/public/llms.txt +++ b/OfficialWebsite/public/llms.txt @@ -4,7 +4,7 @@ > opt-in field-level policy layer that decides what each caller may filter, sort, select, group, > aggregate and see, and a reflection cache that needs no setup. > -> Version 3.1.0 · targets net6.0 · runs on .NET 6, 7, 8, 9, 10 · EF Core 6+ · MIT +> Version 3.2.0 · targets net6.0 · runs on .NET 6, 7, 8, 9, 10 · EF Core 6+ · MIT > Docs: https://doc.dynamicwhere.com · Source: https://github.com/Sajadh92/DynamicWhere.ex This file is the whole library in one pass: every public type and member of the four packages, the @@ -14,7 +14,7 @@ library. No other page is needed; where another page disagrees with this file, t Where a name is not in this file, it does not exist — do not invent members. ``` -dotnet add package DynamicWhere.ex --version 3.1.0 +dotnet add package DynamicWhere.ex --version 3.2.0 dotnet add package DynamicWhere.ex.Policies.Redis # optional, same version as the core dotnet add package DynamicWhere.ex.Policies.EntityFrameworkCore # optional, same version as the core dotnet add package DynamicWhere.ex.Policies.AspNetCore # optional, same version as the core @@ -619,7 +619,8 @@ Path Where predicate generated ## 6. Extension methods `public static class Extension`, namespace `DynamicWhere.ex.Source` (the source file is spelled `Extention.cs`; -the class is `Extension`). 21 public methods, all generic with `where T : class`. None takes a `CancellationToken`. +the class is `Extension`). 28 public methods, all generic with `where T : class`. Every asynchronous one also has +overloads taking a `CancellationToken` (3.2.0). ``` On IQueryable query — composable (validates and builds, executes nothing) @@ -644,6 +645,15 @@ On IQueryable query — terminal ToListAsync(Summary summary, bool getQueryString = false) -> Task ToListAsync(Segment segment) -> Task> +On IQueryable query — terminal, cancellable (3.2.0) + ToListAsync(Filter filter, CancellationToken cancellationToken) -> Task> + ToListAsync(Filter filter, bool getQueryString, CancellationToken cancellationToken) -> Task> + ToListAsyncDynamic(Filter filter, CancellationToken cancellationToken) -> Task> + ToListAsyncDynamic(Filter filter, bool getQueryString, CancellationToken cancellationToken) -> Task> + ToListAsync(Summary summary, CancellationToken cancellationToken) -> Task + ToListAsync(Summary summary, bool getQueryString, CancellationToken cancellationToken) -> Task + ToListAsync(Segment segment, CancellationToken cancellationToken) -> Task> + On IEnumerable query — terminal, in memory ToList(Filter filter, bool getQueryString = false) -> FilterResult ToListDynamic(Filter filter, bool getQueryString = false) -> FilterResult @@ -651,7 +661,9 @@ On IEnumerable query — terminal, in memory ``` These do not exist: a synchronous `ToList(Segment)`, `getQueryString` on Segment, any async or composable -method on `IEnumerable`, names such as `ToListFilter` / `ToListAsyncSegment`, and a `new()` constraint. +method on `IEnumerable`, names such as `ToListFilter` / `ToListAsyncSegment`, a `new()` constraint, and a +`CancellationToken` parameter on the 3.1 signatures: the token overloads are separate methods, so code compiled +against 3.1 still binds. ### Rules for every method @@ -791,10 +803,16 @@ ToListDynamic, ToListAsyncDynamic Where -> COUNT(where-only query) -> build Or - Every call runs two queries: a count of the filtered set (ignoring Page) and the data query. - In the dynamic pair an invalid `Orders`, `Page` or `Selects` throws after the count has already run. -- `ToListAsync` uses EF Core `CountAsync` / `ToListAsync`; `ToListAsyncDynamic` uses EF Core `CountAsync` and - Dynamic LINQ `ToDynamicListAsync`. Both need an EF Core async provider: on a plain `list.AsQueryable()` they throw - `InvalidOperationException` ("The provider for the source 'IQueryable' doesn't implement 'IAsyncQueryProvider'…"). - Use the synchronous methods in memory. +- `ToListAsync` uses EF Core `CountAsync` / `ToListAsync`. `ToListAsyncDynamic` uses EF Core `CountAsync`, then + EF Core's `ToListAsync` over the projection's generated type, so the read runs asynchronously (3.2.0); it used to + run Dynamic LINQ's `ToDynamicListAsync`, a synchronous read on a thread-pool thread. Both need an EF Core async + provider: on a plain `list.AsQueryable()` they throw `InvalidOperationException` ("The provider for the source + 'IQueryable' doesn't implement 'IAsyncQueryProvider'…"). Use the synchronous methods in memory. +- The overloads taking a `CancellationToken` (3.2.0) pass it to the count and to the read, so a canceled token stops + whichever is running and the call throws `OperationCanceledException` (EF Core's `TaskCanceledException` derives + from it). The overloads without a token pass `CancellationToken.None`. + - `ToListAsync(filter, default)` does not compile: `default` fits both `bool getQueryString` and + `CancellationToken`. Write `false`, `CancellationToken.None` or a named argument. - `ToListDynamic` rows are `DynamicClass` objects when `Selects` is set, and T instances when it is null. ### ToListAsync(Segment) @@ -828,7 +846,8 @@ then exactly as ToListAsync(Filter): Order(Orders) -> Page(Page) -> Select(Selec a set with a null `ConditionGroup` → `ArgumentNullException`. Every clause is validated before the database is queried. - Empty or null `ConditionSets` runs `ToListAsync(new Filter { Selects, Orders, Page })`: there is nothing to combine. -- No synchronous version, no `getQueryString`; needs an EF Core async provider. Only `Except` on a type with a +- No synchronous version, no `getQueryString`; needs an EF Core async provider. An overload takes a + `CancellationToken` (3.2.0) and passes it to the count and the read. Only `Except` on a type with a primary key needs the provider to translate a correlated `EXISTS`; `Union` and `Intersect` there are plain `OR` and `AND`. @@ -1316,10 +1335,11 @@ Each of these compiles, passes validation, and returns something other than what 10. **Summary column names collide silently.** Two group fields ending in the same segment (`Name`, `Category.Name`) throw `InvalidOperationException` at run time, and an alias equal to a dot-stripped group field (`CategoryName` beside `Category.Name`) silently drops a column. -11. **In-memory sources behave differently from EF Core.** The async terminals throw; typed `Select` through a +11. **In-memory sources behave differently from EF Core.** The async Filter and Segment terminals throw; typed `Select` through a reference navigation throws; a null navigation inside a path throws `NullReferenceException`; ordering by a navigation throws; `getQueryString` returns a placeholder sentence instead of SQL. -12. **No method takes a `CancellationToken`.** `ToListAsync(filter, ct)` does not compile, and nothing can cancel a query. +12. **`ToListAsync(filter, default)` is ambiguous (3.2.0).** `default` fits both `getQueryString` and the new + `CancellationToken` overload, and the call does not compile. Write `false`, a token, or a named argument. 13. **`Selects: []` throws `MustHasFields`.** Omit the property (null) to return whole entities. 14. **A null inside `Values` is the empty string, not NULL.** Test for NULL with `IsNull` / `IsNotNull` and no values. 15. **`Page` does not order.** Paging without `Orders` returns whatever order the database chooses; always send an @@ -1728,6 +1748,15 @@ Terminal: sanitize, run, transform, set result.Policy (null under Strict unless ToListAsync(Summary summary, bool getQueryString = false) -> Task ToListAsync(Segment segment) -> Task> +Terminal, cancellable (3.2.0): the same, with the token passed to the count and the read + ToListAsync(Filter filter, CancellationToken cancellationToken) -> Task> + ToListAsync(Filter filter, bool getQueryString, CancellationToken cancellationToken) -> Task> + ToListAsyncDynamic(Filter filter, CancellationToken cancellationToken) -> Task> + ToListAsyncDynamic(Filter filter, bool getQueryString, CancellationToken cancellationToken) -> Task> + ToListAsync(Summary summary, CancellationToken cancellationToken) -> Task + ToListAsync(Summary summary, bool getQueryString, CancellationToken cancellationToken) -> Task + ToListAsync(Segment segment, CancellationToken cancellationToken) -> Task> + Composable: sanitize one clause, apply the type's forced predicates, return a new handle Select(List fields) -> PolicyQueryable Where(Condition condition) -> PolicyQueryable @@ -1756,8 +1785,9 @@ How this differs from the unguarded surface: Rules: - Every guarded query runs over `AsNoTracking()`. Returned EF Core entities are detached, so a masked value is never - saved back. An in-memory source has no such copy: without `Selects`, the rows returned are the source objects - themselves, transformed in place. + saved back. An in-memory source has no such copy: without `Selects` and with nothing denied, the rows returned are + the source objects themselves, transformed in place. When a projection is synthesized the rows are new, but a + member kept whole is still the source's own object, and a transform beneath it changes it in place (3.2.0). - Chaining keeps decisions: each composable returns a new handle, and the terminal's trace includes the earlier links' decisions. - The terminal's trace is on `result.Policy` only when `DwPolicyOptions.IncludeTraceInResult` allows it: null follows the tier (off under `Strict`, on under `Convenience`), and `true` or `false` overrides it (3.1.0). `LastTrace` holds it whatever the setting. - `getQueryString: true` under `Strict` → `QueryStringDenied` (14). This is checked before sanitizing; dry run records it instead. @@ -1793,7 +1823,7 @@ public class Ticket { … } entry, such as the one a trailing comma leaves, is ignored, and a field named twice is used once. - Applied only under `ApplyPolicy`, when `Orders` is null or empty: `ToList`, `ToListAsync`, `ToListDynamic` and `ToListAsyncDynamic` with a `Filter`; `ToListAsync(Segment)`; the composable `Filter` and `FilterDynamic`; and the - composable `Page` on a source nothing has ordered or projected. + composable `Page` on a source nothing has ordered and whose projection hides no default field. - Never applied: - by an unguarded call. The core methods of section 6 on a plain `IQueryable` or `IEnumerable`, `Page` included, never read the attribute and order only as their caller asks, exactly as in 3.0; so does a DynamicWhere @@ -1801,12 +1831,21 @@ public class Ticket { … } - when the caller sends orders: the default is not appended as a tiebreak; - to a source already ordered, before it was guarded (`db.Tickets.OrderBy(t => t.Title).ApplyPolicy(ctx)`) or by a composed `Order` earlier in the chain — even one whose every order the policy dropped, because the caller - still sent orders. Only the query's expression is read, so a sequence sorted in memory before - `ApplyPolicy(IEnumerable)` does not count as ordered: send `Orders` for it; - - to a projected source: a `Select` anywhere in the query's chain, whether the guarded composable `Select` - or a projection before `ApplyPolicy`, leaves the query in its own order. A default applied after a projection could - name a field the projection left out, which EF Core cannot translate, so `guarded.Select(["Id", "Title"]).Page(page)` - pages as it did in 3.0.0, unordered; + still sent orders. A composed `Filter` that sent orders counts the same way (3.2.0). Only the query's expression + is read, so a sequence sorted in memory before `ApplyPolicy(IEnumerable)` does not count as ordered: send + `Orders` for it; + - to a source whose projection could hide a default field. Only the outermost `Select` of the chain counts, because + it makes the rows the default orders. Since 3.2.0 it hides nothing when it builds T itself in an object + initializer, `Select(t => new Row { Code = t.Code, … })`, and assigns every field the default names, at every + level of a nested path (`"Owner.Name"` needs `Owner = new OwnerRow { Name = … }`); the default then applies, + because EF Core translates an order through a member the projection assigned. A constructor with arguments, a + default field the initializer does not assign, or a nested path through anything but an initializer leaves the + query in its own order, as every projection did in 3.1.0: a default applied there could name a field the + projection left out, which EF Core cannot translate; + - after a projection composed on the handle: the guarded `Select`, or a guarded `Filter` whose `Selects` is set, + leaves the rest of the chain unordered even when it keeps every default field, so + `guarded.Select(["Id", "Title"]).Page(page)` pages as it did in 3.0.0, unordered. The default is for the rows the + caller's source makes; - to a `Summary`, or by the composable `Where`, `Select` and `Order`. - A type that declares no `DefaultOrder` is never ordered by the library, guarded or not; only a caller's own orders apply. End a default with a unique field, such as the key, or rows sharing the leading values can still @@ -1836,7 +1875,7 @@ public class Ticket { … } ### [DwEntity(RequirePolicy = true)] enforcement -- All 21 public methods of `DynamicWhere.ex.Source.Extension` (the `IQueryable` and `IEnumerable` overloads) run the guard first. +- All 28 public methods of `DynamicWhere.ex.Source.Extension` (the `IQueryable` and `IEnumerable` overloads) run the guard first. - They throw `PolicyException` with `ErrorCode = PolicyRequired` (10) when `T` requires a policy and the call is not running inside a `PolicyQueryable` method. - Composables such as `Select` and `Where` throw when called, not when enumerated. - The exception carries: @@ -2238,9 +2277,12 @@ HAVING through a key or alias of such a field or operator throw t ORDER BY a field denied for Order (also a summary key/alias) drop throw FieldDeniedForOrder SELECT a field denied for Select drop throw FieldDeniedForSelect SELECT a navigation with a denied field beneath it allowed leaves throw FieldDeniedForSelect +SELECT a navigation that cannot be narrowed around a denied + field: its key a.Id is denied, or a path through it is one + the core cannot project (3.2.0) throw throw FieldDeniedForSelect SELECT a.b when the key a.Id is denied throw throw FieldDeniedForSelect every requested SELECT dropped throw - AllSelectsDenied -no Selects while some field is denied for Select allowed scalars allowed scalars +no Selects while a field is denied for Select, at any depth allowed members allowed members GROUP BY a denied field throw throw FieldDeniedForGroup AGGREGATE a denied field, or a transformed field lacking AllowAggregate on a stage throw throw FieldDeniedForAggregate @@ -2263,14 +2305,36 @@ DefaultOrder field denied for Segment, in a Segment (3.1.0) left out l - "drop" removes the entry and records `Dropped`. A Strict throw records `Denied`. - The guarded composable `Order(...)` and `Select(...)` drop and throw the same way. -- "allowed leaves": when `Selects` names a navigation, it is replaced by the allowed leaf paths beneath it. -- "allowed scalars" applies when `Selects` is null or empty and some field is denied for Select. It runs - for a whole-`Filter` terminal and for a Segment, not for a single-clause composable call. - - The projection becomes every allowed property that is readable, writable and not an indexer, and whose - type is simple: primitive, enum, `string`, `decimal`, `DateTime`, `DateOnly`, `TimeOnly`, - `DateTimeOffset`, `TimeSpan` or `Guid`. Navigations are left out. +- "allowed leaves": when `Selects` names a navigation with a denied field beneath it, it is replaced by the allowed + leaf paths beneath it. A navigation with nothing denied beneath it is kept as written. + - The fields beneath are read the way the attribute walker reads them (3.2.0): through any collection type, so a + member typed `IReadOnlyList` or an application's own collection no longer hides its denied fields. + - A narrowing that cannot be built as gated is refused with `FieldDeniedForSelect`, in both tiers (3.2.0). The + core's typed projection adds the key (`Id`) of every nested node it builds, so a narrowing whose nodes carry a + denied key would return it; and a path through a collection the core does not unwrap fails its validation. +- "allowed members" (3.2.0; "allowed scalars" before) applies when `Selects` is null or empty and a field is denied + for Select at the top of T or anywhere beneath one of its members, to the depth the attribute walker reaches. It + runs for a whole-`Filter` terminal and for a Segment, not for a single-clause composable call. The projection is + what an unguarded call would return, less what the policy withholds: + - every allowed member that is readable, writable and not an indexer and holds a value: a simple type (primitive, + enum, `string`, `decimal`, `DateTime`, `DateOnly`, `TimeOnly`, `DateTimeOffset`, `TimeSpan`, `Guid`) or a + collection of one (`byte[]`, `string[]`, `List`); + - every allowed member holding an object or a collection of objects, where the source carries it: whole when + nothing beneath it is denied, and narrowed to the allowed leaves when something is, as a caller naming it would + get. The source carries every member of a row projected before `ApplyPolicy` (any `Select` in the chain) and of + a row in memory, and an entity's owned and complex members, which are read from the EF Core model; + - an entity's other navigations are left out: an unguarded call loads one only when something includes it, and + projecting it would load it. An included or automatically included navigation is therefore not returned once a + projection is needed. Name it in `Selects` to get it, narrowed; + - a member that cannot be narrowed is left out whole, recorded as `Dropped` on `Select` with a reason starting + `left out whole`: on rows in memory, where the core narrows a reference through `EF.Property`; on an EF Core + complex property, which the core compares to null and EF Core refuses to; where the core's projection would add a + denied key back; and where the core cannot project a path beneath it; + - a narrowed reference that is null in the source comes back as an empty object, as it does for a caller's own + dotted `Selects`; + - each denied field, at the top or beneath, is recorded as `Dropped` on `Select`. - It never throws for the denied field. It throws `AllSelectsDenied` only when no field is left. - - With nothing denied, `Selects` stays null. + - With nothing denied at any depth, `Selects` stays null and the query is the one an unguarded call runs. - A path that matches nothing on `T` (3.1.0): - Convenience, and dry run in either tier: validation throws `LogicException` `ConditionMustHasValidFieldName` before any policy decision, as it does unguarded. @@ -2695,6 +2759,8 @@ What is recorded Dropped a field removed from a Convenience request (Select, Order); one per field Dropped a DefaultOrder field left out for this caller (Order), both tiers, and in a Segment a field denied for Segment too; Reason starts "left out of the default order" (3.1.0) + Dropped a field a synthesized projection leaves out (Select), at the top or beneath a member, and a + member it leaves out whole because it cannot be narrowed; Reason starts "left out whole" (3.2.0) Denied a refusal: Strict denial, cap, cost, query string ("*"), required filter, segment inference, MaxAuditEvents; the throw follows unless dry run. Under Strict a name that matches nothing is Denied under that name, Reason "names nothing on " (3.1.0) @@ -3081,6 +3147,9 @@ if (!sim.WouldRun) logger.LogInformation("{Code}", sim.Refusal!.ErrorCode); - A `PolicyException` becomes `Refusal`. Any other exception propagates, unwrapped even from the runtime overload. - Under `Strict`, outside dry run, a path that matches nothing is therefore a `Refusal` with the clause's `FieldDeniedFor*` code and `FieldPath` `"*"`, not a `LogicException` (3.1.0). The `Trace` names it. - A simulated `Filter` or `Segment` that sends no orders gets the type's `DefaultOrder` in `Clause.Orders`, less the fields the caller may not order by (3.1.0). +- A simulation has no source, so it reads T as an entity query whose model it cannot see (3.2.0). With no `Selects`, + a synthesized `Clause.Selects` leaves out every member holding an object; the guarded query over a projected or + in-memory source keeps those members, whole or narrowed, and one over an entity keeps its owned members. - Runtime overload errors: - a value-type `entityType`, or a `TClause` other than `Filter`, `Summary` or `Segment` → `ArgumentException`; - null entity, context or options → `ArgumentNullException`. @@ -4580,6 +4649,43 @@ MemoryCalculationInput ### History ``` +3.2.0 A projected row keeps its members, a denial beneath a member is enforced, a default order that reaches + projected rows, and a CancellationToken on every async terminal. The bullets marked "Behaviour change" + change what code written for 3.1.0 does. + - Security fix and behaviour change. With no Selects, a field denied for Select only beneath a member, none + at the top of T, synthesized no projection, so the whole row came back with the denied value in it: in a + list or nested object of a row projected before ApplyPolicy, in a row in memory, and in an entity's + included, automatically included, lazily loaded or owned member. A denial at any depth now synthesizes the + projection, in both tiers, typed and dynamic, for a Filter and a Segment. On an entity that projection + leaves out every navigation EF Core does not own, so an included navigation is not returned once a denial + anywhere in T needs one; and T needs a public parameterless constructor for it, as it already did for a + top-level denial. + - Behaviour change. The synthesized projection keeps what the source carries. A row projected before + ApplyPolicy, or held in memory, keeps its nested objects and lists: whole when nothing beneath is denied, + narrowed around a denial. An entity keeps its owned and complex members, and every member holding a + collection of simple values (byte[], List). In 3.1.0 all of these came back null or empty whenever + a field was denied. A member that cannot be narrowed is left out whole, with a "left out whole" Dropped + decision. + - Security fix. Under Convenience, Selects naming a navigation whose element key (Id) is denied narrowed the + key away, and the core's typed projection added it back. Such a narrowing is refused with + FieldDeniedForSelect in both tiers, as naming a sibling of the key already was. + - Security fix. Selects naming a member typed as a collection the core does not unwrap (IReadOnlyList, + IReadOnlyCollection, Collection or an application's own) returned every field beneath it, denied ones + included, in both tiers: the projection gate read collections through a narrower list than the attribute + walker. The gate reads them as the walker does, and a narrowing the core cannot project is refused with + FieldDeniedForSelect. + - Behaviour change. [DwEntity(DefaultOrder)] applies to a projected source whose outermost Select builds T + in an object initializer assigning every field the default names; any other projection still leaves the + query in its own order. A Select, or a Filter with Selects, composed on the guarded handle keeps the rest + of the chain unordered. A composed Filter that sent orders gets no default later in the chain, as a + composed Order already did not. + - Every async terminal has overloads taking a CancellationToken, guarded and unguarded: ToListAsync and + ToListAsyncDynamic with a Filter, ToListAsync with a Summary, and ToListAsync with a Segment. The 3.1 + signatures are unchanged, so code compiled against 3.1 still binds. The token reaches the count and the + read. + - Behaviour change. ToListAsyncDynamic and the async Summary read through EF Core's ToListAsync, and the async + Summary counts through CountAsync, instead of reading synchronously; on an EF Core query a canceled token + now reaches the database. A provider that is not EF Core's keeps the synchronous read. 3.1.0 Dates rebuilt, segments combined in the database, five new caps, two new error codes, preparation enforced, a strict tier that discloses less, declared default orders, forced predicates that admit null, audited refusals, and long In lists that no longer end the process. The eleven bullets marked "Behaviour change" @@ -4733,7 +4839,13 @@ MemoryCalculationInput - Enum filtering works whether the column stores names or numbers: the parser converts the name to the enum value before EF Core translates it. `Contains` / `StartsWith` / `EndsWith` work only on string members. - Cache configuration changes are eventually consistent: calls already running finish with the options they read. -- Nothing accepts a `CancellationToken`. +- With no `Selects`, a guarded query over an entity leaves out every navigation EF Core does not own once a denial + anywhere in T needs a projection (section 17). Name the navigation in `Selects` to get it. +- A denial on a type held inside a dictionary or another `System` generic (`Dictionary`, + `KeyValuePair<,>`, `Tuple<>`) is not enforced through it. The attribute walker treats such a member as a value, so + nothing beneath it has a path, and naming it, or a synthesized projection keeping it, returns it whole. Hold such + values in a list of the policed type instead. +- A simulation reads T as an entity query (section 23). --- diff --git a/README.md b/README.md index 664106e..25e026e 100644 --- a/README.md +++ b/README.md @@ -47,13 +47,13 @@ Stop concatenating LINQ predicates by hand. Your front-end sends one JSON shape; ## Install ```bash -dotnet add package DynamicWhere.ex --version 3.1.0 +dotnet add package DynamicWhere.ex --version 3.2.0 ``` Or via Package Manager: ```powershell -Install-Package DynamicWhere.ex -Version 3.1.0 +Install-Package DynamicWhere.ex -Version 3.2.0 ``` Dependencies (restored automatically): From 2d3b5d6c86a1ff7358986736759734b9f1e861ea Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 04:37:42 +0300 Subject: [PATCH 03/22] fix(policies): a projection handing back entities is read as an entity query Found by my own review pass. RowShape counted any Select in the chain as a projection whose every member the source carries. A Select that hands back the entity itself, Select(o => o.Customer) or Select(s => s), builds no row: its navigations hold a value only when something includes them. Kept in a synthesized projection they were loaded, so the guarded query returned more than the unguarded one. Allowed data only, but a navigation the caller never asked for, and possibly a large one. A source now counts as projected only when the outermost Select constructs the row, in an initializer or with a constructor. Anything else is read as an entity query from the EF Core model. Removing the check turns the new test red. AsyncReads reached EF Core's CountAsync and ToListAsync through MethodInfo.Invoke, which wraps anything the operator throws before its task exists in a TargetInvocationException. The async Summary used to count synchronously, so a query EF Core could not translate failed with EF Core's own exception, and now it does again (BindingFlags.DoNotWrapExceptions). A provider that fails while building the query proves it; wrapping turns the test red. Co-Authored-By: Claude Opus 5 --- DynamicWhere.Tests/CancellationTests.cs | 43 +++++++++++++++++++ .../Policies/ProjectedRowTests.cs | 14 ++++++ .../Policies/Source/DefaultOrder.cs | 15 +++++-- DynamicWhere.ex/Policies/Source/RowShape.cs | 18 ++++---- DynamicWhere.ex/Source/AsyncReads.cs | 17 +++++--- OfficialWebsite/public/llms.txt | 6 ++- 6 files changed, 93 insertions(+), 20 deletions(-) diff --git a/DynamicWhere.Tests/CancellationTests.cs b/DynamicWhere.Tests/CancellationTests.cs index fa4ebff..d5d3af1 100644 --- a/DynamicWhere.Tests/CancellationTests.cs +++ b/DynamicWhere.Tests/CancellationTests.cs @@ -11,7 +11,10 @@ using Microsoft.Data.Sqlite; using Microsoft.EntityFrameworkCore; using Microsoft.EntityFrameworkCore.Diagnostics; +using Microsoft.EntityFrameworkCore.Query; +using System.Collections; using System.Data.Common; +using System.Linq.Expressions; namespace DynamicWhere.Tests; @@ -85,6 +88,33 @@ public override ValueTask ReaderExecutedAsync( } } +/// An async provider that fails as it builds the query, before any task exists. +public sealed class UntranslatableQuery : IQueryable, IAsyncQueryProvider +{ + public UntranslatableQuery() => Expression = Expression.Constant(this); + + public Type ElementType => typeof(T); + + public Expression Expression { get; } + + public IQueryProvider Provider => this; + + public IEnumerator GetEnumerator() => throw new InvalidOperationException("untranslatable"); + + IEnumerator IEnumerable.GetEnumerator() => GetEnumerator(); + + public IQueryable CreateQuery(Expression expression) => this; + + public IQueryable CreateQuery(Expression expression) => throw new InvalidOperationException("untranslatable"); + + public object Execute(Expression expression) => throw new InvalidOperationException("untranslatable"); + + public TResult Execute(Expression expression) => throw new InvalidOperationException("untranslatable"); + + public TResult ExecuteAsync(Expression expression, CancellationToken cancellationToken = default) => + throw new InvalidOperationException("untranslatable"); +} + /// /// Every asynchronous terminal takes a , guarded or not, and hands it to /// the provider. @@ -243,6 +273,19 @@ async Task Check(Func run) await Check((db, token) => db.Rows.ToListAsync(OddOrFirst(), token)); } + /// + /// A grouped count reaches EF Core's operator by reflection. A query it cannot build fails with its own + /// exception, as the synchronous count it replaced did, not one wrapped by the reflection call. + /// + [Fact] + public async Task A_provider_failure_leaves_the_grouped_count_as_itself() + { + InvalidOperationException failure = await Assert.ThrowsAsync( + () => AsyncReads.CountAsync(new UntranslatableQuery(), CancellationToken.None)); + + Assert.Equal("untranslatable", failure.Message); + } + /// A token that is never canceled changes nothing: each overload answers as the one without a token does. [Fact] public async Task A_live_token_returns_what_the_overload_without_one_returns() diff --git a/DynamicWhere.Tests/Policies/ProjectedRowTests.cs b/DynamicWhere.Tests/Policies/ProjectedRowTests.cs index 0ec14ff..05e71f5 100644 --- a/DynamicWhere.Tests/Policies/ProjectedRowTests.cs +++ b/DynamicWhere.Tests/Policies/ProjectedRowTests.cs @@ -528,6 +528,20 @@ public async Task An_entity_with_a_denial_beneath_leaves_out_its_navigations_and Assert.Null(((object)row.Location).GetType().GetProperty("Code")); } + /// + /// A projection that hands back the entity itself builds no rows: its navigations hold a value only + /// when something includes them, so they are left out as an entity query's are. Kept, they would be + /// loaded, and the guarded query would return more than the unguarded one. + /// + [Fact] + public void A_projection_returning_entities_is_read_as_an_entity_query() + { + PrShelf shelf = Guard(_db.Shelves.Select(s => s), DwTier.Strict).ToList(Everything()).Data.Single(); + + Assert.Empty(shelf.Books); + Assert.Equal(("A7", (string?)null), (shelf.Location!.Aisle, shelf.Location.Code)); + } + /// /// A top-level denial on an entity keeps what EF Core loads with it: the owned member whole and /// the blob, which a projection of scalars alone used to empty. diff --git a/DynamicWhere.ex/Policies/Source/DefaultOrder.cs b/DynamicWhere.ex/Policies/Source/DefaultOrder.cs index 1e0753e..a7e6111 100644 --- a/DynamicWhere.ex/Policies/Source/DefaultOrder.cs +++ b/DynamicWhere.ex/Policies/Source/DefaultOrder.cs @@ -86,13 +86,20 @@ or nameof(Queryable.ThenBy) or nameof(Queryable.ThenByDescending)) } /// - /// True when a query's rows are a projection made somewhere along the chain that produced it. + /// True when a query's rows are built by a projection: the outermost Select of the chain + /// constructs each row, in an initializer or with a constructor, so every member holds what the + /// projection gave it. /// /// - /// Any Select counts. This answers what the rows are, not whether a default can be applied - /// to them, which answers. + /// A Select that hands back an entity, Select(o => o.Customer), builds nothing: its + /// rows are entities as EF Core loads them, whose navigations hold a value only when something + /// includes them. This answers what the rows are, not whether a default can be applied to them, + /// which answers. /// - internal static bool IsProjected(Expression expression) => OutermostSelect(expression) is not null; + internal static bool BuildsRows(Expression expression) => + OutermostSelect(expression) is { } select + && StripQuotes(select.Arguments[1]) is LambdaExpression selector + && StripConversions(selector.Body) is MemberInitExpression or NewExpression; /// /// True when a projection along the chain could have left out a field the type's default names. diff --git a/DynamicWhere.ex/Policies/Source/RowShape.cs b/DynamicWhere.ex/Policies/Source/RowShape.cs index 722fd84..f5f6b6b 100644 --- a/DynamicWhere.ex/Policies/Source/RowShape.cs +++ b/DynamicWhere.ex/Policies/Source/RowShape.cs @@ -25,7 +25,7 @@ internal sealed class RowShape /// An entity query whose model could not be read: no navigation is known to load. internal static readonly RowShape Entity = new(RowKind.Entity, Empty(), Empty()); - /// A projection the caller built, whose every member holds a value it computed. + /// A projection that builds its rows, whose every member holds what the projection gave it. internal static readonly RowShape Projected = new(RowKind.Projected, Empty(), Empty()); /// Rows held in memory, whose every member holds whatever the object holds. @@ -60,11 +60,13 @@ private RowShape(RowKind kind, HashSet alwaysLoaded, HashSet com /// Reads the shape of a guarded query's source. /// - /// Rows in memory first, because a projection over them is still in memory. Then any Select - /// along the chain, which is how a caller builds its own row type out of entities. What is left is - /// an entity query, whose owned and complex members are read from the EF Core model; a navigation it - /// does not own loads only when something includes it, and a guarded query that has to narrow the - /// row leaves such a navigation out, as it always has. + /// Rows in memory first, because a projection over them is still in memory. Then a projection that + /// builds its rows, which is how a caller makes its own row type out of entities. What is left is an + /// entity query, or a projection handing back entities, Select(o => o.Customer). Their owned + /// and complex members are read from the EF Core model; a navigation the entity does not own loads + /// only when something includes it, and a guarded query that has to narrow the row leaves such a + /// navigation out, as it always has. Projecting one would load it, and return more than the + /// unguarded call does. /// internal static RowShape Of(IQueryable source) where T : class { @@ -73,7 +75,7 @@ internal static RowShape Of(IQueryable source) where T : class return InMemory; } - if (DefaultOrder.IsProjected(source.Expression)) + if (DefaultOrder.BuildsRows(source.Expression)) { return Projected; } @@ -148,7 +150,7 @@ internal enum RowKind /// An entity query: a navigation holds a value only when something loads it. Entity, - /// A projection the caller built: every member holds a value the projection computed. + /// A projection that builds its rows: every member holds what the projection gave it. Projected, /// A sequence in memory: every member holds whatever the object holds. diff --git a/DynamicWhere.ex/Source/AsyncReads.cs b/DynamicWhere.ex/Source/AsyncReads.cs index f33255b..92e1508 100644 --- a/DynamicWhere.ex/Source/AsyncReads.cs +++ b/DynamicWhere.ex/Source/AsyncReads.cs @@ -43,9 +43,7 @@ internal static async Task> ToDynamicListAsync(IQueryable query, C return await query.ToDynamicListAsync(cancellationToken); } - Task read = (Task)ToListAsyncMethod - .MakeGenericMethod(query.ElementType) - .Invoke(null, new object[] { query, cancellationToken })!; + Task read = (Task)Call(ToListAsyncMethod, query, cancellationToken); await read; @@ -70,8 +68,15 @@ internal static async Task CountAsync(IQueryable query, CancellationToken c return query.Count(); } - return await (Task)CountAsyncMethod - .MakeGenericMethod(query.ElementType) - .Invoke(null, new object[] { query, cancellationToken })!; + return await (Task)Call(CountAsyncMethod, query, cancellationToken); } + + /// + /// Calls one of EF Core's operators for the query's element type, letting whatever it throws leave as + /// itself: a query EF Core cannot translate fails with its own exception, not one wrapped by reflection. + /// + private static object Call(MethodInfo operatorMethod, IQueryable query, CancellationToken cancellationToken) => + operatorMethod + .MakeGenericMethod(query.ElementType) + .Invoke(null, BindingFlags.DoNotWrapExceptions, null, new object[] { query, cancellationToken }, null)!; } diff --git a/OfficialWebsite/public/llms.txt b/OfficialWebsite/public/llms.txt index 8f74f93..1176c28 100644 --- a/OfficialWebsite/public/llms.txt +++ b/OfficialWebsite/public/llms.txt @@ -2321,8 +2321,10 @@ DefaultOrder field denied for Segment, in a Segment (3.1.0) left out l collection of one (`byte[]`, `string[]`, `List`); - every allowed member holding an object or a collection of objects, where the source carries it: whole when nothing beneath it is denied, and narrowed to the allowed leaves when something is, as a caller naming it would - get. The source carries every member of a row projected before `ApplyPolicy` (any `Select` in the chain) and of - a row in memory, and an entity's owned and complex members, which are read from the EF Core model; + get. The source carries every member of a row a projection builds before `ApplyPolicy` (the outermost `Select` + constructs the row, in an initializer or with a constructor) and of a row in memory, and an entity's owned and + complex members, which are read from the EF Core model. A `Select` handing back an entity, + `Select(o => o.Customer)`, builds no row and is read as an entity query; - an entity's other navigations are left out: an unguarded call loads one only when something includes it, and projecting it would load it. An included or automatically included navigation is therefore not returned once a projection is needed. Name it in `Selects` to get it, narrowed; From 8a647c3861b67cacf2439c3b044164cab919bddd Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 05:23:25 +0300 Subject: [PATCH 04/22] fix(policies): what the 3.2.0 review found in the synthesized projection Three reviews of the first cut: security, correctness, and a sweep of the demo API against master. The first cut kept object members whole wherever the gate found nothing denied beneath them, and asked for a projection whenever a denial sat anywhere beneath a member. Both were wrong. What asks for a projection. Only a denial whose value can reach the result. On an entity that means one beneath a column, an owned or complex member, or a navigation something loads: an Include or ThenInclude on the chain (read from the expression, and any form it cannot read counts as loading everything), an automatic include, or a lazy loader (an ILazyLoader service property, which proxies add too). A denial beneath a navigation nothing loads never leaves the database. In a connected model such denials sit beneath almost every type, and the first cut projected nearly every entity query: abstract and table-per-hierarchy types, types with no public parameterless constructor, and converter-mapped members all broke. They are back to 3.1.0's shape. A lazy loader could also carry a denied value out after the query; a navigation it can fill now counts as loaded. What a member may be kept whole around. The gate asks the providers' fragments for every denied path beneath a member, not only the paths its walk produces. That finds a denial beneath a property with no setter, one reached around a cycle by a runtime rule, and one deeper than the walk. It also scans every type the member can hold, however deep and through the type arguments of framework generics such as Dictionary, for a field denied for Select and for a member typed object. A forced scope beneath a member cannot be applied to what it holds, so such a member is left out whole. So is one with a transform on a property that has no setter. What a projection holds. A projected row carries the members its initializer assigns: an unassigned one used to be narrowed through EF.Property, which EF Core cannot translate. A member is narrowed only where the core's narrowing translates: an object the initializer builds, a subquery list the core can bind (not an array or a set), or a navigation that is neither complex nor stored as JSON. An entity keeps every mapped column, converted and JSON ones included, whole, and no member EF Core does not map, which made EF Core read the denied column to compute it. Rows in memory keep their values and leave their objects out, as in 3.1.0, since a kept object is the caller's own and a transform would change it in place. A member named with a word the parser keeps is skipped. The walk now stops where the walker stops, four segments, so a narrowing never projects a field no policy can speak about. Found in passing, all present in 3.1.0: - The walker read a namespace starting with "System" as the framework's, so SystemsCorp.Payroll got no fragment beneath its types and a [DwDenied] field there was returned and filterable. - Naming Main.Lead kept it whole without gating Main's key, which the builder adds. - A denied member that is not a simple value (a blob, an owned object, a JSON column) never asked for a projection, so with nothing else denied it came back. The demo API read Employee's denied WorkSchedule this way. DW-13: a default whose field the projection computes with an application method is not applied; EF Core evaluates such a method on the client and cannot order by it. Every rule was mutation-checked. EF Core 8 leg: 2131 passed. EF Core 6.0.22 leg: 1435 passed. Co-Authored-By: Claude Opus 5 --- DynamicWhere.Tests/ComplexMemberTests.cs | 87 ++ .../Policies/ProjectedRowTests.cs | 20 +- .../Policies/ProjectionReviewTests.cs | 816 ++++++++++++++++++ .../Resolution/AttributePolicyProvider.cs | 15 +- .../Policies/Resolution/PolicyResolver.cs | 13 + .../Policies/Source/DefaultOrder.cs | 57 +- .../Policies/Source/FilterSanitizer.cs | 470 ++++++++-- DynamicWhere.ex/Policies/Source/RowShape.cs | 450 ++++++++-- 8 files changed, 1785 insertions(+), 143 deletions(-) create mode 100644 DynamicWhere.Tests/Policies/ProjectionReviewTests.cs diff --git a/DynamicWhere.Tests/ComplexMemberTests.cs b/DynamicWhere.Tests/ComplexMemberTests.cs index c5b1495..1a61f1d 100644 --- a/DynamicWhere.Tests/ComplexMemberTests.cs +++ b/DynamicWhere.Tests/ComplexMemberTests.cs @@ -46,6 +46,47 @@ public class CxSealedBox public CxSecretSize Inner { get; set; } = new(); } +/// A row projected from , carrying its complex value. +[DwEntity(RequirePolicy = true)] +public sealed class CxBoxRow +{ + public int Id { get; set; } + + [DwDenied] + public string? Tenant { get; set; } + + public CxSecretSize Inner { get; set; } = new(); +} + +/// An owned member stored as JSON, with a denial beneath it. +public class CxJsonShelf +{ + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public CxJsonMeta Meta { get; set; } = new(); +} + +public class CxJsonMeta +{ + public string Label { get; set; } = string.Empty; + + [DwDenied] + public string? Code { get; set; } +} + +/// A primitive collection named with a word the expression parser keeps. +public class CxCastEntity +{ + public int Id { get; set; } + + [DwDenied] + public string? Secret { get; set; } + + public List Cast { get; set; } = new(); +} + public sealed class ComplexMemberContext : DbContext { private readonly SqliteConnection _connection; @@ -56,12 +97,17 @@ public sealed class ComplexMemberContext : DbContext public DbSet Boxes => Set(); + public DbSet JsonShelves => Set(); + + public DbSet CastEntities => Set(); + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); protected override void OnModelCreating(ModelBuilder model) { model.Entity().ComplexProperty(crate => crate.Size); model.Entity().ComplexProperty(box => box.Inner); + model.Entity().OwnsOne(shelf => shelf.Meta, meta => meta.ToJson()); } } @@ -86,6 +132,8 @@ public ComplexMemberTests() _db.Crates.Add(new CxCrate { Label = "label-secret", Size = new CxSize { Width = 4, Height = 5 } }); _db.Boxes.Add(new CxSealedBox { Name = "B1", Inner = new CxSecretSize { Width = 2, Height = 9 } }); + _db.JsonShelves.Add(new CxJsonShelf { Name = "J1", Meta = new CxJsonMeta { Label = "L", Code = "json-secret" } }); + _db.CastEntities.Add(new CxCastEntity { Secret = "s", Cast = new List { "a" } }); _db.SaveChanges(); _db.ChangeTracker.Clear(); } @@ -111,6 +159,45 @@ public void A_complex_property_with_nothing_denied_beneath_is_kept_whole() Assert.Equal((4, 5), (crate.Size.Width, crate.Size.Height)); } + /// + /// A projected row carrying a complex value: the core's narrowing compares it to null, which EF Core + /// refuses, so with a denial beneath it the value is left out whole rather than failing the query. + /// + [Fact] + public void A_projected_complex_value_with_a_denial_beneath_is_left_out() + { + PolicyQueryable guarded = Guard(_db.Boxes.Select(b => new CxBoxRow { Id = b.Id, Tenant = b.Name, Inner = b.Inner })); + + CxBoxRow row = guarded.ToList(new Filter()).Data.Single(); + + Assert.Equal((0, 0), (row.Inner.Width, row.Inner.Height)); + Assert.Contains(guarded.LastTrace!.Decisions, decision => decision.FieldPath == "Inner" + && decision.Reason == "left out whole: the projection builds it in a way the core cannot narrow"); + } + + /// An owned member stored as JSON cannot be narrowed by EF Core, so it is left out whole. + [Fact] + public void An_owned_member_stored_as_json_with_a_denial_beneath_is_left_out() + { + PolicyQueryable guarded = Guard(_db.JsonShelves); + + CxJsonShelf shelf = guarded.ToList(new Filter()).Data.Single(); + + Assert.Equal("J1", shelf.Name); + Assert.Null(shelf.Meta.Code); + Assert.Contains(guarded.LastTrace!.Decisions, decision => decision.FieldPath == "Meta" + && decision.Reason == "left out whole: it is stored as JSON, which EF Core cannot narrow"); + } + + /// A primitive collection named with a parser word cannot be projected, so it is skipped. + [Fact] + public void A_primitive_collection_named_with_a_parser_word_is_skipped() + { + CxCastEntity entity = Guard(_db.CastEntities).ToList(new Filter()).Data.Single(); + + Assert.Null(entity.Secret); + } + [Fact] public void A_complex_property_with_a_denial_beneath_is_left_out_whole() { diff --git a/DynamicWhere.Tests/Policies/ProjectedRowTests.cs b/DynamicWhere.Tests/Policies/ProjectedRowTests.cs index 05e71f5..51a7a31 100644 --- a/DynamicWhere.Tests/Policies/ProjectedRowTests.cs +++ b/DynamicWhere.Tests/Policies/ProjectedRowTests.cs @@ -557,11 +557,12 @@ public void An_entity_keeps_its_owned_member_and_its_blob_beside_a_denied_scalar } /// - /// Rows in memory keep a member whole when nothing beneath it is denied. One with a denial beneath - /// is left out: the core's narrowing of a reference reads it through EF Core. + /// Rows in memory keep their values and leave every object member out once a projection is needed, + /// as 3.1.0 did. A member kept from them would be the caller's own object, which a transform would + /// then change in place, and a denial only beneath a member now needs that projection too. /// [Fact] - public void Rows_in_memory_keep_clean_members_whole_and_leave_out_the_rest() + public void Rows_in_memory_keep_their_values_and_leave_their_objects_out() { PrRoleRow[] roles = { @@ -573,11 +574,12 @@ public void Rows_in_memory_keep_clean_members_whole_and_leave_out_the_rest() PolicyQueryable guarded = open.AsQueryable().ApplyPolicy(Caller(), Options(DwTier.Strict), Resolver()); PrOpenRow row = guarded.ToList(Everything()).Data.Single(); - Assert.Equal("R1|-|One|read/|read", Describe(role)); - Assert.Empty(row.Contents); + Assert.Equal("R1|-|-||read", Describe(role)); + Assert.Equal(("R1", 0), (row.Code, row.Contents.Count)); + Assert.Equal("s-read", open[0].Contents[0].Secret); Assert.Contains( guarded.LastTrace!.Decisions, - decision => decision.FieldPath == "Contents" && decision.Reason!.StartsWith("left out whole", StringComparison.Ordinal)); + decision => decision is { FieldPath: "Contents.Secret", Action: PolicyAction.Dropped }); } // ------------------------------------------------------------------ F2: the key the builder adds @@ -665,7 +667,7 @@ public void A_read_only_list_is_kept_whole_when_clean_and_left_out_when_it_canno [Fact] public void Nothing_is_synthesized_when_nothing_is_denied_at_any_depth() { - foreach (RowShape rows in new[] { RowShape.Entity, RowShape.Projected, RowShape.InMemory }) + foreach (RowShape rows in new[] { RowShape.Unknown, RowShape.InMemory }) { Filter sanitized = FilterSanitizer.Sanitize( new Filter(), Resolver(), Caller(), Options(DwTier.Strict), new PolicyTrace(DwTier.Strict, dryRun: false), rows: rows); @@ -681,8 +683,8 @@ public void Each_source_keeps_what_it_carries() List? Synthesized(RowShape rows) => FilterSanitizer.Sanitize( new Filter(), Resolver(), Caller(), Options(DwTier.Strict), new PolicyTrace(DwTier.Strict, dryRun: false), rows: rows).Selects; - Assert.Equal(new[] { "Code", "Id" }, Synthesized(RowShape.Entity)!.OrderBy(x => x, StringComparer.Ordinal)); - Assert.Equal(new[] { "Code", "Contents.Code", "Id" }, Synthesized(RowShape.Projected)!.OrderBy(x => x, StringComparer.Ordinal)); + Assert.Equal(new[] { "Code", "Id" }, Synthesized(RowShape.Unknown)!.OrderBy(x => x, StringComparer.Ordinal)); + Assert.Equal(new[] { "Code", "Contents.Code", "Id" }, Synthesized(RowShape.Of(OpenRows()))!.OrderBy(x => x, StringComparer.Ordinal)); Assert.Equal(new[] { "Code", "Id" }, Synthesized(RowShape.InMemory)!.OrderBy(x => x, StringComparer.Ordinal)); } } diff --git a/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs b/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs new file mode 100644 index 0000000..5a344a2 --- /dev/null +++ b/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs @@ -0,0 +1,816 @@ +using System.ComponentModel.DataAnnotations.Schema; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Classes.Core; +using DynamicWhere.ex.Enums; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.DTOs; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Infrastructure; + +namespace SystemsCorp.Payroll +{ + /// An application type whose namespace only starts with the word System. + public sealed class RvPay + { + public string? Grade { get; set; } + + [DwDenied] + public string? Salary { get; set; } + } +} + +namespace DynamicWhere.Tests.Policies +{ + // --------------------------------------------------------------------------------- the database + + public class RvRole + { + public int Id { get; set; } + + public string Code { get; set; } = string.Empty; + + public int? KeeperId { get; set; } + + public RvKeeper? Keeper { get; set; } + } + + public class RvKeeper + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + [DwDenied] + public string? Ssn { get; set; } + } + + /// An entity with values EF Core maps through converters, and a denial only beneath a navigation. + public class RvAsset + { + public int Id { get; set; } + + public Uri? Home { get; set; } + + public RvMoney Price { get; set; } + + [NotMapped] + public List Tags { get; set; } = new(); + + public int? KeeperId { get; set; } + + public RvKeeper? Keeper { get; set; } + } + + public readonly record struct RvMoney(decimal Amount); + + /// The same shape with a denied column, so a projection is always needed. + public class RvSealedAsset + { + public int Id { get; set; } + + [DwDenied] + public string? Serial { get; set; } + + public Uri? Home { get; set; } + + public RvMoney Price { get; set; } + + [NotMapped] + public string Display => Serial ?? "none"; + + [NotMapped] + public List Tags { get; set; } = new(); + } + + /// A table-per-hierarchy base with a denial only beneath a navigation it does not load. + public abstract class RvVehicle + { + public int Id { get; set; } + + public string Plate { get; set; } = string.Empty; + + public int? KeeperId { get; set; } + + public RvKeeper? Keeper { get; set; } + } + + public class RvTruck : RvVehicle + { + public int Axles { get; set; } + } + + /// An entity whose owned member has a denied property with no setter. + public class RvShop + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public RvAddress Address { get; set; } = new(); + } + + public class RvAddress + { + public RvAddress() + { + } + + public RvAddress(string city, string? zip) + { + City = city; + Zip = zip; + } + + public string City { get; set; } = string.Empty; + + [DwDenied] + public string? Zip { get; } + } + + /// An entity whose denied members are not simple values: nothing else is denied. + public class RvVault + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + [DwDenied] + public byte[] Blob { get; set; } = Array.Empty(); + + [DwDenied] + public RvVaultPlace? Place { get; set; } + } + + public class RvVaultPlace + { + public string Aisle { get; set; } = string.Empty; + } + + /// An entity whose child's key no caller may see, reached through another navigation. + public class RvHouse + { + public int Id { get; set; } + + public int? MainId { get; set; } + + public RvGroup? Main { get; set; } + } + + public class RvGroup + { + [DwDenied] + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public int? LeadId { get; set; } + + public RvLead? Lead { get; set; } + } + + public class RvLead + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + } + + /// An entity whose children are scoped to a tenant: the scope filters parents, not children. + public class RvFamily + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public List Kids { get; set; } = new(); + } + + public class RvKid + { + public int Id { get; set; } + + public int RvFamilyId { get; set; } + + public string Name { get; set; } = string.Empty; + + [DwForceWhere(Operator.Equal, ContextValue = "TenantId")] + public int TenantId { get; set; } + } + + /// An entity whose posts arrive through an injected lazy loader. + public class RvBlog + { + private List? _posts; + + public RvBlog() + { + } + + private RvBlog(ILazyLoader lazyLoader) => LazyLoader = lazyLoader; + + private ILazyLoader? LazyLoader { get; set; } + + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public List Posts + { + get + { + if (LazyLoader is not null) + { + LazyLoader.Load(this, ref _posts); + } + + return _posts ??= new List(); + } + set => _posts = value; + } + } + + public class RvPost + { + public int Id { get; set; } + + public int RvBlogId { get; set; } + + public string Title { get; set; } = string.Empty; + + [DwDenied] + public string? Draft { get; set; } + } + + public sealed class ProjectionReviewContext : DbContext + { + private readonly SqliteConnection _connection; + + public ProjectionReviewContext(SqliteConnection connection) => _connection = connection; + + public DbSet Roles => Set(); + + public DbSet Assets => Set(); + + public DbSet SealedAssets => Set(); + + public DbSet Vehicles => Set(); + + public DbSet Shops => Set(); + + public DbSet Vaults => Set(); + + public DbSet Houses => Set(); + + public DbSet Families => Set(); + + public DbSet Blogs => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + protected override void OnModelCreating(ModelBuilder model) + { + model.Entity().Property(a => a.Price).HasConversion(m => m.Amount, a => new RvMoney(a)); + model.Entity().Property(a => a.Price).HasConversion(m => m.Amount, a => new RvMoney(a)); + model.Entity(); + model.Entity().OwnsOne(s => s.Address, a => a.Property(x => x.Zip)); + model.Entity().OwnsOne(v => v.Place); + model.Entity().Navigation(b => b.Posts).HasField("_posts"); + } + } + + // --------------------------------------------------------------------------------- the rows + + [DwEntity(RequirePolicy = true)] + public sealed class RvRoleRow + { + public int Id { get; set; } + + public string Code { get; set; } = string.Empty; + + [DwDenied] + public string? Tenant { get; set; } + + public RvKeeper? Keeper { get; set; } + } + + public sealed class RvContact + { + public RvContact() + { + } + + public RvContact(string email) => Email = email; + + public string Phone { get; set; } = string.Empty; + + [DwDenied] + public string? Email { get; } + } + + [DwEntity(RequirePolicy = true)] + public sealed class RvContactRow + { + public int Id { get; set; } + + [DwDenied] + public string? Tenant { get; set; } + + public RvContact? Contact { get; set; } + } + + public sealed class RvLine + { + public string Sku { get; set; } = string.Empty; + + [DwDenied] + public string? Cost { get; set; } + } + + /// Framework types holding a policed type: the walker does not enter them. + [DwEntity(RequirePolicy = true)] + public sealed class RvKeyedRow + { + public int Id { get; set; } + + public Dictionary BySku { get; set; } = new(); + + public object? Payload { get; set; } + } + + public sealed class RvLevel1 + { + public RvLevel2? Next { get; set; } + } + + public sealed class RvLevel2 + { + public RvLevel3? Next { get; set; } + } + + public sealed class RvLevel3 + { + public string Name { get; set; } = string.Empty; + + public RvLevel4? Next { get; set; } + } + + public sealed class RvLevel4 + { + [DwDenied] + public string? Secret { get; set; } + } + + /// A denial five segments down, deeper than the walker gives a fragment. + [DwEntity(RequirePolicy = true)] + public sealed class RvDeepRow + { + public int Id { get; set; } + + [DwDenied] + public string? Tenant { get; set; } + + public RvLevel1? First { get; set; } + } + + public sealed class RvStaffRow + { + public int Id { get; set; } + + public SystemsCorp.Payroll.RvPay? Pay { get; set; } + } + + [DwEntity(DefaultOrder = "Label desc")] + public sealed class RvLabelRow + { + public int Id { get; set; } + + public string Label { get; set; } = string.Empty; + } + + [DwEntity(RequirePolicy = true)] + public sealed class RvReservedRow + { + public int Id { get; set; } + + [DwDenied] + public string? Tenant { get; set; } + + public List Cast { get; set; } = new(); + } + + public sealed class RvItem + { + public string Name { get; set; } = string.Empty; + + [DwDenied] + public string? Hidden { get; set; } + } + + [DwEntity(RequirePolicy = true)] + public sealed class RvArrayRow + { + public int Id { get; set; } + + public RvItem[] Items { get; set; } = Array.Empty(); + } + + /// + /// What the 3.2.0 review found in the first cut of the synthesized projection, and the fixes for + /// it: denials the gate could not see, projections EF Core cannot translate, and members the first + /// cut lost that 3.1.0 returned. + /// + public sealed class ProjectionReviewTests : IDisposable + { + private readonly SqliteConnection _connection; + private readonly ProjectionReviewContext _db; + + public ProjectionReviewTests() + { + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + + _db = new ProjectionReviewContext(_connection); + _db.Database.EnsureCreated(); + + RvKeeper keeper = new() { Name = "K", Ssn = "ssn-secret" }; + + _db.Roles.Add(new RvRole { Code = "R1", Keeper = keeper }); + _db.Assets.Add(new RvAsset { Home = new Uri("https://example.test/"), Price = new RvMoney(12.5m), Keeper = keeper }); + _db.SealedAssets.Add(new RvSealedAsset { Serial = "serial-secret", Home = new Uri("https://example.test/"), Price = new RvMoney(7m) }); + _db.Vehicles.Add(new RvTruck { Plate = "T42", Axles = 3, Keeper = keeper }); + _db.Shops.Add(new RvShop { Name = "S1", Address = new RvAddress("Basra", "zip-secret") }); + _db.Vaults.Add(new RvVault { Name = "V1", Blob = new byte[] { 9 }, Place = new RvVaultPlace { Aisle = "A1" } }); + _db.Houses.Add(new RvHouse { Main = new RvGroup { Name = "G", Lead = new RvLead { Name = "L" } } }); + _db.Families.Add(new RvFamily + { + Name = "F1", + Kids = { new RvKid { Name = "own", TenantId = 1 }, new RvKid { Name = "other", TenantId = 2 } } + }); + _db.Blogs.Add(new RvBlog { Name = "B1", Posts = { new RvPost { Title = "P1", Draft = "draft-secret" } } }); + + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static DwPolicyContext Caller() => + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1").WithValue("TenantId", 1); + + private static PolicyResolver Resolver(params IDwPolicyProvider[] more) => + new(new IDwPolicyProvider[] { new AttributePolicyProvider() }.Concat(more).ToArray()); + + private static PolicyQueryable Guard(IQueryable source, DwTier tier = DwTier.Strict, params IDwPolicyProvider[] more) + where T : class => + source.ApplyPolicy(Caller(), new DwPolicyOptions { Tier = tier, Caps = { MinGroupSize = 1 } }, Resolver(more)); + + private static Filter Selecting(params string[] fields) => new() { Selects = fields.ToList() }; + + // ---------------------------------------------------------------- entity queries keep 3.1.0's shape + + /// + /// A denial beneath a navigation the query does not load reaches nothing, so it needs no + /// projection: the entity comes back as EF Core loads it, converted members and all. + /// + [Fact] + public void A_denial_beneath_a_navigation_nothing_loads_leaves_the_entity_as_loaded() + { + PolicyQueryable guarded = Guard(_db.Assets); + RvAsset asset = guarded.ToList(new Filter()).Data.Single(); + + Assert.Equal(("https://example.test/", 12.5m), (asset.Home!.ToString(), asset.Price.Amount)); + Assert.DoesNotContain(guarded.LastTrace!.Decisions, decision => decision.Action == PolicyAction.Dropped); + } + + /// A hierarchy keeps its runtime types and its abstract base, as it did in 3.1.0. + [Fact] + public void A_hierarchy_is_still_read_as_entities() + { + RvVehicle vehicle = Guard(_db.Vehicles).ToList(new Filter()).Data.Single(); + + Assert.Equal(3, Assert.IsType(vehicle).Axles); + } + + /// + /// When a projection is needed, an entity keeps every mapped column, converted ones included, and + /// leaves out a member EF Core does not map, which would make it read the denied column. + /// + [Fact] + public void A_needed_projection_keeps_converted_columns_and_skips_unmapped_members() + { + PolicyQueryable guarded = Guard(_db.SealedAssets, DwTier.Convenience); + FilterResultOf result = new(guarded.ToList(new Filter(), getQueryString: true)); + + Assert.Equal(("https://example.test/", 7m, "none"), (result.Row.Home!.ToString(), result.Row.Price.Amount, result.Row.Display)); + Assert.DoesNotContain("Serial", result.Sql, StringComparison.OrdinalIgnoreCase); + } + + /// + /// A denied member that holds no simple value, a blob or an owned object, used to be passed over, + /// so with nothing else denied the whole entity came back with it. + /// + [Fact] + public void A_denied_member_that_is_not_a_simple_value_is_withheld() + { + RvVault vault = Guard(_db.Vaults).ToList(new Filter()).Data.Single(); + + Assert.Equal("V1", vault.Name); + Assert.Empty(vault.Blob); + Assert.Null(vault.Place); + } + + /// A denied owned property with no setter is left out of the owned member, which is kept. + [Theory] + [InlineData(DwTier.Convenience)] + [InlineData(DwTier.Strict)] + public void A_denied_property_with_no_setter_beneath_an_owned_member_is_left_out(DwTier tier) + { + RvShop shop = Guard(_db.Shops, tier).ToList(new Filter()).Data.Single(); + + Assert.Equal(("Basra", (string?)null), (shop.Address.City, shop.Address.Zip)); + + if (tier == DwTier.Strict) + { + Assert.Throws(() => Guard(_db.Shops, tier).ToList(Selecting("Id", "Address"))); + } + else + { + RvAddress named = Guard(_db.Shops, tier).ToList(Selecting("Id", "Address")).Data.Single().Address; + + Assert.Equal(("Basra", (string?)null), (named.City, named.Zip)); + } + } + + /// + /// A navigation a lazy loader can fill carries its denied values out after the query, so its + /// entity needs the projection, whose rows have no loader. + /// + [Fact] + public void A_navigation_a_lazy_loader_can_fill_needs_the_projection() + { + PolicyQueryable guarded = Guard(_db.Blogs); + RvBlog blog = guarded.ToList(new Filter()).Data.Single(); + + Assert.Equal("B1", blog.Name); + Assert.Empty(blog.Posts); + Assert.Contains(guarded.LastTrace!.Decisions, decision => decision.FieldPath == "Posts.Draft"); + } + + /// + /// A forced scope on a list's elements filters the rows of the query, never the elements, so an + /// included list would carry every tenant's children. The entity is projected and the list left out. + /// + [Fact] + public void A_list_whose_elements_carry_a_forced_scope_is_not_returned_whole() + { + RvFamily family = Guard(_db.Families.Include(f => f.Kids)).ToList(new Filter()).Data.Single(); + + Assert.Equal("F1", family.Name); + Assert.Empty(family.Kids); + } + + /// + /// A navigation named through another keeps the key of each node it passes through, and a + /// denied one refuses the projection, as naming a sibling of the key always did. + /// + [Theory] + [InlineData(DwTier.Convenience)] + [InlineData(DwTier.Strict)] + public void A_navigation_named_through_one_whose_key_is_denied_is_refused(DwTier tier) + { + PolicyException refused = Assert.Throws( + () => Guard(_db.Houses, tier).ToList(Selecting("Id", "Main.Lead"))); + + Assert.Equal(PolicyErrorCode.FieldDeniedForSelect, refused.ErrorCode); + } + + // ---------------------------------------------------------------- projected rows + + /// + /// A member the projection never assigns holds its default and is not carried: narrowing it + /// would read it through EF.Property, which EF Core cannot translate for an unbound member. + /// + [Fact] + public void A_member_the_projection_does_not_assign_is_not_narrowed() + { + RvRoleRow row = Guard(_db.Roles.Select(r => new RvRoleRow { Id = r.Id, Code = r.Code })) + .ToList(new Filter()).Data.Single(); + + Assert.Equal(("R1", (RvKeeper?)null), (row.Code, row.Keeper)); + } + + /// A composed projection feeds the terminal only what it selected. + [Fact] + public void A_composed_projection_then_a_terminal_runs() + { + Assert.Equal("R1", Guard(_db.Roles).Filter(new Filter()).ToList(new Filter()).Data.Single().Code); + Assert.Equal("R1", Guard(_db.Roles).Select(new List { "Id", "Code" }).ToList(new Filter()).Data.Single().Code); + } + + /// + /// A property with no setter is not one the gate can narrow a member around, and a member built + /// by a constructor is not one the core can narrow: such a member is left out whole. + /// + [Fact] + public void A_member_built_by_a_constructor_with_a_hidden_denial_is_left_out() + { + PolicyQueryable guarded = Guard( + _db.Roles.Select(r => new RvContactRow { Id = r.Id, Contact = new RvContact(r.Code) })); + + Assert.Null(guarded.ToList(new Filter()).Data.Single().Contact); + Assert.Contains(guarded.LastTrace!.Decisions, decision => decision.FieldPath == "Contact" + && decision.Reason == "left out whole: the projection builds it in a way the core cannot narrow"); + } + + /// + /// A denial five segments down has no fragment, since the walker stops at four. A projection + /// carrying the member whole used to return it; the member is now narrowed to what the policy + /// can speak about. + /// + [Fact] + public void A_denial_deeper_than_the_walker_goes_is_not_carried() + { + RvDeepRow row = Guard(_db.Roles.Select(r => new RvDeepRow + { + Id = r.Id, + First = new RvLevel1 { Next = new RvLevel2 { Next = new RvLevel3 { Name = r.Code, Next = new RvLevel4 { Secret = "deep-secret" } } } } + })).ToList(new Filter()).Data.Single(); + + Assert.Equal("R1", row.First!.Next!.Next!.Name); + Assert.Null(row.First.Next.Next.Next?.Secret); + } + + /// + /// The same member named in Selects: the strict tier refuses a denial it cannot name, and the + /// convenience tier narrows it away. + /// + [Theory] + [InlineData(DwTier.Convenience)] + [InlineData(DwTier.Strict)] + public void Naming_a_member_with_a_denial_deeper_than_the_walker_goes(DwTier tier) + { + IQueryable rows = _db.Roles.Select(r => new RvDeepRow + { + Id = r.Id, + First = new RvLevel1 { Next = new RvLevel2 { Next = new RvLevel3 { Name = r.Code, Next = new RvLevel4 { Secret = "deep-secret" } } } } + }); + + if (tier == DwTier.Strict) + { + Assert.Throws(() => Guard(rows, tier).ToList(Selecting("Id", "First"))); + + return; + } + + RvDeepRow row = Guard(rows, tier).ToList(Selecting("Id", "First")).Data.Single(); + + Assert.Null(row.First!.Next!.Next!.Next?.Secret); + } + + /// A default order computed on the client cannot be translated, so it is not applied. + [Fact] + public void A_default_computed_by_an_application_method_is_not_applied() + { + IQueryable rows = _db.Roles.Select(r => new RvLabelRow { Id = r.Id, Label = Decorate(r.Code) }); + + Assert.Equal("[R1]", Guard(rows).ToList(new Filter()).Data.Single().Label); + } + + private static string Decorate(string code) => "[" + code + "]"; + + /// A member named with a word the parser keeps cannot be projected, so it is skipped. + [Fact] + public void A_member_named_with_a_parser_word_is_skipped() + { + RvReservedRow row = Guard(_db.Roles.Select(r => new RvReservedRow + { + Id = r.Id, + Tenant = r.Code, + Cast = _db.Roles.Where(x => x.Id == r.Id).Select(x => x.Code).ToList() + })) + .ToList(new Filter()).Data.Single(); + + Assert.Equal((1, (string?)null), (row.Id, row.Tenant)); + } + + /// An array needing narrowing is left out, since the core can only bind a list. + [Fact] + public void An_array_that_needs_narrowing_is_left_out() + { + PolicyQueryable guarded = Guard(_db.Roles.Select(r => new RvArrayRow + { + Id = r.Id, + Items = _db.Roles.Where(x => x.Id == r.Id).Select(x => new RvItem { Name = x.Code, Hidden = "hidden" }).ToArray() + })); + + Assert.Empty(guarded.ToList(new Filter()).Data.Single().Items); + Assert.Contains(guarded.LastTrace!.Decisions, decision => decision.FieldPath == "Items" + && decision.Reason!.StartsWith("left out whole", StringComparison.Ordinal)); + } + + // ---------------------------------------------------------------- what the policy cannot see + + /// + /// A framework collection of a policed type, and a member typed object, are not entered by the + /// walker. Rows in memory leave them out once a projection is needed, which a denial inside them + /// now asks for; naming one is refused. + /// + [Theory] + [InlineData(DwTier.Convenience)] + [InlineData(DwTier.Strict)] + public void A_framework_collection_of_a_policed_type_is_not_returned(DwTier tier) + { + RvKeyedRow[] rows = + { + new() { Id = 1, BySku = { ["a"] = new RvLine { Sku = "a", Cost = "cost-secret" } }, Payload = new RvLine { Cost = "payload-secret" } } + }; + + RvKeyedRow row = Guard(rows.AsQueryable(), tier).ToList(new Filter()).Data.Single(); + + Assert.Empty(row.BySku); + Assert.Null(row.Payload); + Assert.Throws(() => Guard(rows.AsQueryable(), tier).ToList(Selecting("Id", "BySku"))); + } + + /// + /// A namespace that only starts with the word System is the application's. The walker read it + /// as the framework's and put no fragment beneath its types, so a denied field there was + /// returned, filtered on and ordered by. + /// + [Theory] + [InlineData(DwTier.Convenience)] + [InlineData(DwTier.Strict)] + public void A_namespace_starting_with_System_is_policed(DwTier tier) + { + RvStaffRow[] rows = { new() { Id = 1, Pay = new SystemsCorp.Payroll.RvPay { Grade = "G1", Salary = "salary-secret" } } }; + + Filter where = new() + { + ConditionGroup = new ConditionGroup + { + Conditions = { new Condition { Field = "Pay.Salary", DataType = DataType.Text, Operator = Operator.Equal, Values = { "x" } } } + } + }; + + Assert.Throws(() => Guard(rows.AsQueryable(), tier).ToList(where)); + Assert.Contains( + new AttributePolicyProvider().GetFragments(typeof(RvStaffRow), Caller()), + fragment => fragment.FieldPath == "Pay.Salary"); + } + + /// A runtime rule on a path reached through a cycle is found beneath a member, as its fragment names it. + [Fact] + public void A_rule_on_a_path_through_a_cycle_is_seen_beneath_a_member() + { + RvLink[] rows = { new() { Id = 1, Head = new RvNode { Name = "n1", Next = new RvNode { Name = "n2-secret" } } } }; + FakePolicyProvider rule = new FakePolicyProvider().Add( + "Head.Next.Name", PolicyFeature.Select, PolicyEffect.Deny, PolicyLevel.DynamicGlobal); + + PolicyQueryable guarded = Guard(rows.AsQueryable(), DwTier.Convenience, rule); + + Assert.Null(guarded.ToList(new Filter()).Data.Single().Head); + Assert.Throws(() => Guard(rows.AsQueryable(), DwTier.Convenience, rule).ToList(Selecting("Id", "Head"))); + } + + /// Holds a reference to a copy of the result and its SQL, to keep a test to one statement per line. + private sealed class FilterResultOf + where T : class + { + internal FilterResultOf(DynamicWhere.ex.Classes.Result.FilterResult result) + { + Row = result.Data.Single(); + Sql = result.QueryString ?? string.Empty; + } + + internal T Row { get; } + + internal string Sql { get; } + } + } + + public sealed class RvNode + { + public string Name { get; set; } = string.Empty; + + public RvNode? Next { get; set; } + } + + [DwEntity(RequirePolicy = true)] + public sealed class RvLink + { + public int Id { get; set; } + + public RvNode? Head { get; set; } + } +} diff --git a/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs b/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs index 682e993..c75de5a 100644 --- a/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs +++ b/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs @@ -309,9 +309,22 @@ internal static Type Peeled(Type propertyType) return null; } - return type.Namespace?.StartsWith("System", StringComparison.Ordinal) == true ? null : type; + return IsFramework(type) ? null : type; } + /// + /// True for a type the framework declares: one in the System namespace or beneath it. + /// + /// + /// The namespace is compared as a whole segment. A prefix match took an application's own + /// SystemsCorp.Payroll or SystemX.Domain for the framework, so nothing beneath one of + /// its types got a fragment, and a [DwDenied] field there was returned and filtered on as + /// though it carried no policy. + /// + internal static bool IsFramework(Type type) => + type.Namespace is { } name + && (name == "System" || name.StartsWith("System.", StringComparison.Ordinal)); + /// /// Converts one attribute into a fragment at the level its Overridable flag implies. /// diff --git a/DynamicWhere.ex/Policies/Resolution/PolicyResolver.cs b/DynamicWhere.ex/Policies/Resolution/PolicyResolver.cs index 53fd442..c5be3b3 100644 --- a/DynamicWhere.ex/Policies/Resolution/PolicyResolver.cs +++ b/DynamicWhere.ex/Policies/Resolution/PolicyResolver.cs @@ -373,6 +373,19 @@ public TypePolicy ResolveType(Type entityType, DwPolicyContext context) return new TypePolicy(aliases, forced, required, transforms); } + /// + /// Every fragment every provider has for a type and caller, in one list. + /// + /// + /// For the projection gate, which asks whether anything beneath a member is denied. A fragment + /// matches a path exactly or matches every path, so the paths these fragments name, and the + /// wildcard, are every place a denial can land. Walking the type instead missed the paths a walk + /// does not produce: a property with no setter, a type reached again through a cycle, a rule on a + /// path deeper than the walk goes. + /// + internal IReadOnlyList Fragments(Type entityType, DwPolicyContext context) => + Sweep(entityType, context).ToList(); + /// /// Reads every fragment every provider has for a type, refusing a provider that misbehaves. /// diff --git a/DynamicWhere.ex/Policies/Source/DefaultOrder.cs b/DynamicWhere.ex/Policies/Source/DefaultOrder.cs index a7e6111..24e9be5 100644 --- a/DynamicWhere.ex/Policies/Source/DefaultOrder.cs +++ b/DynamicWhere.ex/Policies/Source/DefaultOrder.cs @@ -144,7 +144,7 @@ internal static bool HidesDefault(Expression expression, Type type) } /// The Select nearest the end of the chain, or null when nothing projects it. - private static MethodCallExpression? OutermostSelect(Expression expression) + internal static MethodCallExpression? OutermostSelect(Expression expression) { for (Expression? node = expression; node is MethodCallExpression call; node = call.Arguments.Count > 0 ? call.Arguments[0] : null) @@ -172,7 +172,7 @@ private static bool Assigns(IEnumerable bindings, string[] path, if (depth == path.Length - 1) { - return true; + return binding is not MemberAssignment { Expression: var assigned } || Translatable(assigned); } return binding switch @@ -185,10 +185,59 @@ when StripConversions(assigned) is MemberInitExpression nested => }; } - private static Expression StripQuotes(Expression expression) => + /// + /// True when EF Core can order by what an expression computes: nothing in it calls a method outside + /// the framework and EF Core, or invokes a delegate. + /// + /// + /// An assigned field is not enough on its own. Label = Decorate(r.Code) assigns the field + /// the default names, and EF Core evaluates the application's own method on the client, where it can + /// project the value but cannot order by it. Such a default is left out, as the projection left the + /// query unordered before. + /// + private static bool Translatable(Expression expression) + { + ClientCallFinder finder = new(); + + finder.Visit(expression); + + return !finder.Found; + } + + /// Finds a call EF Core would have to evaluate on the client. + private sealed class ClientCallFinder : ExpressionVisitor + { + internal bool Found { get; private set; } + + public override Expression? Visit(Expression? node) => Found ? node : base.Visit(node); + + protected override Expression VisitMethodCall(MethodCallExpression node) + { + string space = node.Method.DeclaringType?.Namespace ?? string.Empty; + + if (!(space == "System" || space.StartsWith("System.", StringComparison.Ordinal) + || space == "Microsoft.EntityFrameworkCore" || space.StartsWith("Microsoft.EntityFrameworkCore.", StringComparison.Ordinal))) + { + Found = true; + + return node; + } + + return base.VisitMethodCall(node); + } + + protected override Expression VisitInvocation(InvocationExpression node) + { + Found = true; + + return node; + } + } + + internal static Expression StripQuotes(Expression expression) => expression is UnaryExpression { NodeType: ExpressionType.Quote } quote ? quote.Operand : expression; - private static Expression StripConversions(Expression expression) + internal static Expression StripConversions(Expression expression) { while (expression is UnaryExpression { NodeType: ExpressionType.Convert or ExpressionType.TypeAs } conversion) { diff --git a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs index 77d5bbb..e3ebbd4 100644 --- a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs +++ b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs @@ -5,6 +5,7 @@ using DynamicWhere.ex.Enums; using DynamicWhere.ex.Exceptions; using DynamicWhere.ex.Optimization.Cache.Source; +using DynamicWhere.ex.Policies.Attributes; using DynamicWhere.ex.Policies.Audit; using DynamicWhere.ex.Policies.Config; using DynamicWhere.ex.Policies.Context; @@ -147,7 +148,7 @@ internal static Filter Sanitize( working.Orders = defaults; } - GateSelects(working, gate, synthesizeProjection, rows ?? RowShape.Entity); + GateSelects(working, gate, synthesizeProjection, rows ?? RowShape.Unknown); // The strict tier checks the bill once every field has passed its gate. It drops nothing, so a // request still here names no field the caller may not use, and a weighted field the caller may @@ -731,7 +732,7 @@ internal static Segment Sanitize( working.Orders = defaults; } - GateSegmentSelects(working, gate, rows ?? RowShape.Entity); + GateSegmentSelects(working, gate, rows ?? RowShape.Unknown); // The strict tier checks the bill once every field has passed its gate. It drops nothing, so a // request still here names no field the caller may not use, and a weighted field the caller may @@ -987,7 +988,7 @@ private static void GateSegmentSelects(Segment segment, Gate gate, RowShape rows return; } - List kept = GateProjection(segment.Selects, gate); + List kept = GateProjection(segment.Selects, gate, rows); if (kept.Count == 0) { @@ -1630,7 +1631,7 @@ private static void GateSelects(Filter filter, Gate gate, bool synthesize, RowSh return; } - List kept = GateProjection(filter.Selects, gate); + List kept = GateProjection(filter.Selects, gate, rows); if (kept.Count == 0) { @@ -1662,7 +1663,7 @@ private static void GateSelects(Filter filter, Gate gate, bool synthesize, RowSh /// IReadOnlyList<T>, failed validation with an error about the caller's field names. /// /// - private static List GateProjection(IEnumerable selects, Gate gate) + private static List GateProjection(IEnumerable selects, Gate gate, RowShape rows) { List kept = new(); @@ -1674,19 +1675,14 @@ void Keep(string path) } } - bool Allowed(string path, out FieldPolicy policy) - { - policy = gate.PolicyFor(path, PolicyFeature.Select); - - return policy.Allows(PolicyFeature.Select) - || !gate.Refuse(path, PolicyFeature.Select, PolicyErrorCode.FieldDeniedForSelect, policy); - } - foreach (string field in selects) { // The field's own decision first. A navigation that is itself denied is refused as it // stands; what it carries is only asked about once it has survived on its own account. - if (!Allowed(field, out _)) + FieldPolicy own = gate.PolicyFor(field, PolicyFeature.Select); + + if (!own.Allows(PolicyFeature.Select) + && gate.Refuse(field, PolicyFeature.Select, PolicyErrorCode.FieldDeniedForSelect, own)) { continue; } @@ -1699,30 +1695,66 @@ bool Allowed(string path, out FieldPolicy policy) continue; } - IReadOnlyList carried = gate.ProjectionUnder(field); - List survivors = new(carried.Count); + (List survivors, List<(string Path, FieldPolicy Policy)> denied) = + gate.Beneath(field, PolicyFeature.Select); + + // Each denied path, as a denial beneath a named navigation always was: refused under the + // strict tier, dropped from the narrowing otherwise, left in place in a dry run. (string Path, FieldPolicy Policy)? cause = null; - foreach (string path in carried) + foreach ((string path, FieldPolicy policy) in denied) { - if (Allowed(path, out FieldPolicy policy)) - { - survivors.Add(path); - } - else + if (gate.Refuse(path, PolicyFeature.Select, PolicyErrorCode.FieldDeniedForSelect, policy)) { cause ??= (path, policy); } } - if (cause is not { } narrowing) + // A member that carries everything its type holds can hold a denied field no path names: + // beyond the walker's depth, or inside a framework collection. An entity's navigation loads + // only its own entity, which the paths above already cover. + bool hidden = cause is null + && rows.LoadsWhole(field) + && gate.DeclaredType(field) is { } type + && Gate.Facts(type).DeniesSelect + && gate.RefuseHidden(field); + + if (cause is null && !hidden && !gate.ReadOnlyTransformBeneath(field)) { Keep(field); + // Kept whole, a navigation reached through others still passes through their nodes, and + // the builder adds each one's key. + KeepCarriedKeys(field, gate, Keep); + continue; } - if (gate.DeniedKey(survivors) is { } key) + (string Path, FieldPolicy? Policy) blame = cause is { } found ? (found.Path, found.Policy) : (field, null); + + if (!rows.Narrows(field.Split(SegmentSeparator)[0])) + { + gate.RefuseNarrowing(blame.Path, blame.Policy, $"'{field}' cannot be narrowed: {rows.WhyNotNarrowed(field)}"); + + continue; + } + + List narrowed = new(survivors.Count); + + foreach (string path in survivors) + { + // Projected whole by the core, a field holding a framework collection of a policed type + // carries the denied fields the policy cannot name inside it. + if (gate.DeclaredType(path) is { } leaf && !Gate.HoldsValue(leaf) && Gate.Facts(leaf).DeniesSelect + && gate.RefuseHidden(path)) + { + continue; + } + + narrowed.Add(path); + } + + if (gate.DeniedKey(narrowed) is { } key) { gate.RefuseNarrowing( key.Path, key.Policy, @@ -1731,16 +1763,16 @@ bool Allowed(string path, out FieldPolicy policy) continue; } - if (gate.Unprojectable(survivors) is { } unbuilt) + if (gate.Unprojectable(narrowed) is { } unbuilt) { gate.RefuseNarrowing( - narrowing.Path, narrowing.Policy, + blame.Path, blame.Policy, $"'{field}' cannot be narrowed around it: the core cannot project '{unbuilt}'"); continue; } - foreach (string path in survivors) + foreach (string path in narrowed) { Keep(path); } @@ -1867,64 +1899,63 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) continue; } + // A projection assigns only a member with a setter, and no path can name a member the + // expression parser keeps a word for; such a member is still asked about below. + bool assignable = member.CanWrite && !ReservedNames.Starts(name); + if (Gate.HoldsValue(member.PropertyType)) { - allowed.Add(name); + if (assignable && rows.CarriesValue(name)) + { + allowed.Add(name); + } continue; } - List survivors = new(); - bool narrowed = false; - - foreach (string path in gate.ProjectionUnder(name)) - { - FieldPolicy beneath = gate.PolicyFor(path, PolicyFeature.None); - - if (beneath.Allows(PolicyFeature.Select)) - { - survivors.Add(path); + (List survivors, List<(string Path, FieldPolicy Policy)> denied) = + gate.Beneath(name, PolicyFeature.None); - continue; - } + Gate.TypeFacts facts = Gate.Facts(member.PropertyType); + bool forced = gate.ForcedBeneath(name); - narrowed = true; + // Only a denial whose value can reach the result asks for a projection: one beneath a + // navigation nothing loads never leaves the database. + List<(string Path, FieldPolicy Policy)> reached = denied.Where(d => rows.Materializes(d.Path)).ToList(); + bool opens = rows.Materializes(name + SegmentSeparator + name); + foreach ((string path, FieldPolicy beneath) in reached) + { gate.Record(path, PolicyFeature.Select, PolicyAction.Dropped, beneath); } - anyDenied |= narrowed; + if (reached.Count > 0 || (opens && (facts.Opaque || forced))) + { + anyDenied = true; + } - if (!rows.Carries(name)) + if (!assignable || !rows.Carries(name)) { continue; } - if (!narrowed) + if (reached.Count == 0 && !facts.Opaque && !forced && !gate.ReadOnlyTransformBeneath(name)) { allowed.Add(name); continue; } - string? reason = !rows.Narrows(name) - ? rows.Kind == RowKind.InMemory - ? "left out whole: the rows are in memory, and narrowing it needs EF Core" - : "left out whole: it is a complex property, which EF Core cannot narrow" - : gate.DeniedKey(survivors) is { } key - ? $"left out whole: the projection would add its key '{key.Path}', which is denied" - : gate.Unprojectable(survivors) is { } unbuilt - ? $"left out whole: the core cannot project '{unbuilt}'" - : null; + string? reason = forced + ? "left out whole: a scope forced beneath it cannot be applied to what it holds" + : rows.Narrows(name) + ? Narrowed(name, survivors, gate, allowed) + : "left out whole: " + rows.WhyNotNarrowed(name); if (reason is not null) { gate.LeaveOut(name, reason); - - continue; } - - allowed.AddRange(survivors); } if (!anyDenied || gate.IsDryRun) @@ -1940,6 +1971,52 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) return allowed; } + /// + /// Narrows a member for a synthesized projection, adding what survives to , + /// or returns why the member is left out whole instead. + /// + /// + /// A field whose type can hold what the policy cannot name, a framework collection of a policed + /// type or a member typed , is left out of the narrowing: the core projects it + /// whole. The builder adds the key of every node it narrows, so a denied key leaves the member out, + /// and so does a path the core cannot project. + /// + private static string? Narrowed(string member, List survivors, Gate gate, List allowed) + { + List kept = new(survivors.Count); + + foreach (string path in survivors) + { + if (gate.DeclaredType(path) is { } type && !Gate.HoldsValue(type) && Gate.Facts(type).Opaque) + { + gate.LeaveOut(path, "left out whole: it can hold what the policy cannot name"); + + continue; + } + + kept.Add(path); + } + + if (gate.DeniedKey(kept) is { } key) + { + return $"left out whole: the projection would add its key '{key.Path}', which is denied"; + } + + if (gate.Unprojectable(kept) is { } unbuilt) + { + return $"left out whole: the core cannot project '{unbuilt}'"; + } + + if (kept.Count == 0) + { + return "left out whole: nothing beneath it may be selected"; + } + + allowed.AddRange(kept); + + return null; + } + /// /// Refuses a request that fails to supply a filter the policy requires. /// @@ -2459,7 +2536,7 @@ private sealed class Gate // What a projection of each navigation carries, per root type and path. Reflection only, so it // is the same for every caller and every query. - private static readonly ConcurrentDictionary<(Type Root, string Path), IReadOnlyList> Beneath = new(); + private static readonly ConcurrentDictionary<(Type Root, string Path), IReadOnlyList> CarriedPaths = new(); internal Gate( Type entityType, @@ -2561,12 +2638,13 @@ internal string Unknown(string name) internal bool IsDryRun => _options.DryRun || _context.DryRun; /// - /// The members of the entity that a typed projection can actually assign. + /// The members of the entity: every readable property that is not an indexer. /// /// /// Read through the same reflection cache the validator uses, so a synthesized projection - /// cannot name a field the pipeline would then reject. Write-only and indexed members are - /// excluded because a typed projection assigns into them. + /// cannot name a field the pipeline would then reject. A projection assigns only the writable + /// ones, but a read-only one is still asked about: a denial on it, or beneath it, still needs a + /// projection, or the row comes back holding it. /// internal IEnumerable Members() { @@ -2574,7 +2652,7 @@ internal IEnumerable Members() { PropertyInfo property = entry.Value; - if (property.CanRead && property.CanWrite && property.GetIndexParameters().Length == 0) + if (property.CanRead && property.GetIndexParameters().Length == 0) { yield return property; } @@ -2622,35 +2700,37 @@ internal bool HasKey(string path) /// asking for its parent instead — the same shape as sending no projection at all. /// /// A type is read the way reads it, through - /// , so every path a fragment can sit on - /// is a path this walk reaches. It once read collections through a narrower list of its own, - /// and a member typed IReadOnlyList<T> hid every denial beneath it. The walk stops - /// at the same depth, and does not follow a type back into itself along one path. + /// , and no path is longer than the + /// longest the walker puts a fragment on. It once read collections through a narrower list of + /// its own, so a member typed IReadOnlyList<T> hid every denial beneath it, and it + /// went a level deeper than the walker, so a narrowing projected fields no policy could speak + /// about. It does not follow a type back into itself along one path: a narrowing leaves such a + /// region out. /// /// internal IReadOnlyList ProjectionUnder(string path) => - Beneath.GetOrAdd((_entityType, path), key => Enumerate(key.Root, key.Path)); + CarriedPaths.GetOrAdd((_entityType, path), key => Enumerate(key.Root, key.Path)); private static IReadOnlyList Enumerate(Type root, string path) { List paths = new(); - Type? type = TypeAt(root, path) is { } declared - ? AttributePolicyProvider.NavigationTypeOf(declared) - : null; - if (type is not null) + if (DeclaredType(root, path) is { } declared + && AttributePolicyProvider.NavigationTypeOf(declared) is { } type) { - Descend(path, type, 1, paths, new HashSet()); + Descend(path, type, path.Split(SegmentSeparator).Length, paths, new HashSet()); } return paths; } + // depth is how many segments prefix has; a child has one more. private static void Descend( string prefix, Type type, int depth, List paths, HashSet seen) { - // A type reachable from itself would otherwise enumerate until the stack ran out. - if (!seen.Add(type)) + // A child would be longer than any path the walker gives a fragment, or the type is one this + // path already passed through and would otherwise enumerate until the stack ran out. + if (depth >= AttributePolicyProvider.MaxDepth || !seen.Add(type)) { return; } @@ -2669,7 +2749,7 @@ private static void Descend( string child = $"{prefix}.{property.Name}"; // A value, a collection of values, or a type the walker does not enter: projected - // whole, and nothing beneath it carries a policy. + // whole, and nothing beneath it has a path of its own. if (AttributePolicyProvider.NavigationTypeOf(property.PropertyType) is not { } nested) { paths.Add(child); @@ -2677,15 +2757,221 @@ private static void Descend( continue; } - if (depth >= AttributePolicyProvider.MaxDepth) + Descend(child, nested, depth + 1, paths, seen); + } + + seen.Remove(type); + } + + /// + /// What lies beneath a member, as the policy sees it: the fields a narrowing of it could keep, + /// and every path beneath it this caller may not select. + /// + /// The member's path. + /// + /// when the caller named the member, so an audited field a + /// narrowing would carry is recorded as used; otherwise. + /// + /// + /// The denials come from two places. The walk above finds every field a projection of the + /// member could hold. The fragments the providers hold find the rest: a fragment matches a path + /// exactly or matches every path, so a denied path the walk never produced, beneath a property + /// with no setter, around a cycle, or deeper than the walk goes, is still one of them. + /// + internal (List Survivors, List<(string Path, FieldPolicy Policy)> Denied) Beneath( + string member, PolicyFeature use) + { + List survivors = new(); + List<(string Path, FieldPolicy Policy)> denied = new(); + HashSet asked = new(StringComparer.OrdinalIgnoreCase); + + foreach (string path in ProjectionUnder(member)) + { + asked.Add(path); + + FieldPolicy policy = PolicyFor(path, use); + + if (policy.Allows(PolicyFeature.Select)) + { + survivors.Add(path); + } + else + { + denied.Add((path, policy)); + } + } + + string prefix = member + SegmentSeparator; + + foreach (PolicyFragment fragment in Fragments) + { + if (fragment.IsWildcard + || !fragment.FieldPath.StartsWith(prefix, StringComparison.OrdinalIgnoreCase) + || !asked.Add(fragment.FieldPath)) { continue; } - Descend(child, nested, depth + 1, paths, seen); + FieldPolicy policy = PolicyFor(fragment.FieldPath, PolicyFeature.None); + + if (!policy.Allows(PolicyFeature.Select)) + { + denied.Add((fragment.FieldPath, policy)); + } } - seen.Remove(type); + return (survivors, denied); + } + + /// Every fragment the providers hold for this type and caller, read once per query. + private IReadOnlyList Fragments => _fragments ??= _resolver.Fragments(_entityType, _context); + + private IReadOnlyList? _fragments; + + /// + /// True when a forced predicate reaches beneath the member. A projection cannot apply it to the + /// elements the member holds: the scope filters the rows of the query, and a list kept whole or + /// narrowed carries every element, those the scope excludes included. + /// + internal bool ForcedBeneath(string member) + { + string prefix = member + SegmentSeparator; + + return TypePolicy.Forced.Any( + forced => forced.FieldPath.StartsWith(prefix, StringComparison.OrdinalIgnoreCase)); + } + + /// + /// True when a transform beneath the member lands on a property with no setter, which the + /// outbound walk could not write back: a narrowing leaves such a property out. + /// + internal bool ReadOnlyTransformBeneath(string member) + { + string prefix = member + SegmentSeparator; + + foreach (string path in TypePolicy.Transforms.Keys) + { + if (path.StartsWith(prefix, StringComparison.OrdinalIgnoreCase) + && Property(_entityType, path) is { CanWrite: false }) + { + return true; + } + } + + return false; + } + + /// + /// What a type can hold that no path of the policy's reaches, read once per type. + /// + internal static TypeFacts Facts(Type type) => FactsByType.GetOrAdd(type, Scan); + + private static readonly ConcurrentDictionary FactsByType = new(); + + /// + /// Reads every type a value of this type can hold, however deep, for a field denied for Select + /// and for a member typed . + /// + /// + /// Wider than the walker on purpose. The walker stops four segments deep and does not enter a + /// framework type, so a field denied beneath either has no path, and resolves as allowed. This + /// follows every property, a read-only one included, every collection, and the type arguments of + /// a framework generic such as Dictionary<string, T>, and remembers the types it has + /// read rather than counting levels. A member typed can hold anything, so + /// it is reported rather than read. + /// + private static TypeFacts Scan(Type type) + { + bool denies = false; + bool holdsObject = false; + HashSet seen = new(); + Stack pending = new(); + + void Enqueue(Type candidate) + { + Type peeled = AttributePolicyProvider.Peeled(candidate); + + if (peeled == typeof(object)) + { + holdsObject = true; + + return; + } + + if (AttributePolicyProvider.NavigationTypeOf(candidate) is { } navigation && seen.Add(navigation)) + { + pending.Push(navigation); + } + + if (peeled.IsGenericType && AttributePolicyProvider.IsFramework(peeled)) + { + foreach (Type argument in peeled.GetGenericArguments()) + { + Enqueue(argument); + } + } + } + + Enqueue(type); + + while (pending.Count > 0 && !(denies && holdsObject)) + { + foreach (PropertyInfo property in pending.Pop().GetProperties(BindingFlags.Public | BindingFlags.Instance)) + { + if (property.GetIndexParameters().Length != 0) + { + continue; + } + + if (property.GetCustomAttributes(inherit: true) + .Any(attribute => (attribute.Features & PolicyFeature.Select) != 0)) + { + denies = true; + } + + Enqueue(property.PropertyType); + } + } + + return new TypeFacts(denies, holdsObject); + } + + /// + /// What a type can hold that the policy cannot name a path to. + /// + /// A field denied for Select somewhere in what it can hold. + /// A member typed somewhere in what it can hold. + internal readonly record struct TypeFacts(bool DeniesSelect, bool HoldsObject) + { + /// True when either fact holds, so the type cannot be shown to be free of policy. + internal bool Opaque => DeniesSelect || HoldsObject; + } + + /// + /// The declared type at a path, read the way the walker reads it: through any collection, one + /// segment at a time. Null when a segment names nothing. + /// + internal Type? DeclaredType(string path) => DeclaredType(_entityType, path); + + private static Type? DeclaredType(Type root, string path) => Property(root, path)?.PropertyType; + + private static PropertyInfo? Property(Type root, string path) + { + Type? current = root; + PropertyInfo? property = null; + + foreach (string segment in path.Split(SegmentSeparator)) + { + if (current is null || CacheReflection.FindProperty(current, segment) is not { } next) + { + return null; + } + + property = next; + current = AttributePolicyProvider.NavigationTypeOf(next.PropertyType); + } + + return property; } /// @@ -3106,7 +3392,7 @@ internal void LeaveOut(string fieldPath, string reason) => /// that would be built is not the one that was gated, so the only way to honour the denial is /// to decline to build it. /// - internal void RefuseNarrowing(string fieldPath, FieldPolicy policy, string reason) + internal void RefuseNarrowing(string fieldPath, FieldPolicy? policy, string reason) { _trace.Add(new PolicyDecision(fieldPath, PolicyFeature.Select, PolicyAction.Denied, reason)); @@ -3118,6 +3404,32 @@ internal void RefuseNarrowing(string fieldPath, FieldPolicy policy, string reaso throw Exception(fieldPath, PolicyFeature.Select, PolicyErrorCode.FieldDeniedForSelect, policy); } + /// + /// Applies a denial the policy cannot name a path to: a field denied for Select that a named + /// member holds beyond the walker's depth or inside a framework collection. Throws in the strict + /// tier, as a denied field beneath a named navigation does, and otherwise records the drop. + /// + /// True when the caller must narrow it away. False in a dry run. + internal bool RefuseHidden(string fieldPath) + { + const string origin = "it holds a field denied for Select where no path can name it"; + + _trace.Add(new PolicyDecision( + fieldPath, PolicyFeature.Select, IsStrict ? PolicyAction.Denied : PolicyAction.Dropped, origin)); + + if (IsDryRun) + { + return false; + } + + if (IsStrict) + { + throw Exception(fieldPath, PolicyFeature.Select, PolicyErrorCode.FieldDeniedForSelect, null); + } + + return true; + } + /// Records that a field of the type's default order was left out for this caller. internal void SkipDefaultOrder(string fieldPath, FieldPolicy policy) { diff --git a/DynamicWhere.ex/Policies/Source/RowShape.cs b/DynamicWhere.ex/Policies/Source/RowShape.cs index f5f6b6b..b471fcb 100644 --- a/DynamicWhere.ex/Policies/Source/RowShape.cs +++ b/DynamicWhere.ex/Policies/Source/RowShape.cs @@ -1,7 +1,12 @@ using System.Collections; +using System.Linq.Expressions; using System.Reflection; +using DynamicWhere.ex.Optimization.Cache.Source; using DynamicWhere.ex.Source; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Infrastructure; using Microsoft.EntityFrameworkCore.Metadata; +using Microsoft.EntityFrameworkCore.Query; namespace DynamicWhere.ex.Policies.Source; @@ -11,62 +16,233 @@ namespace DynamicWhere.ex.Policies.Source; /// /// /// A caller who sends no projection is owed what an unguarded call would return, less what the policy -/// withholds. Which members that is depends on the source. An entity query loads a navigation only -/// when something includes it; a projection computes every member it assigns; rows already in memory -/// hold everything. A synthesized projection that left out every navigation was right for the first -/// and emptied every list and nested object of the other two. -/// -/// Read from the query itself, never from the rows, so the sanitizer stays pure: this is decided once, -/// before it runs. -/// +/// withholds. Which members hold what depends on the source, and so does which denied value could +/// reach the result at all: +/// +/// +/// An entity query loads the entity's columns, its owned and complex members, and a navigation +/// only when something loads it: an Include, an automatic include or a lazy loader. A value +/// beneath a navigation nothing loads never reaches the result, so a denial there needs no +/// projection; and projecting the navigation would load it. +/// +/// +/// A projection holds exactly the members its initializer assigns. The others hold defaults, and +/// narrowing one reads it through EF.Property, which EF Core cannot translate for a member +/// the initializer never bound. +/// +/// +/// Rows in memory hold everything, and a member kept from them is the caller's own object, which +/// a transform would then change in place. +/// +/// +/// Read from the query itself, never from the rows, so the sanitizer stays pure: this is decided +/// once, before it runs. /// internal sealed class RowShape { - /// An entity query whose model could not be read: no navigation is known to load. - internal static readonly RowShape Entity = new(RowKind.Entity, Empty(), Empty()); + /// The key EF Core's relational layer gives an owned type stored in a JSON column. + private const string JsonContainer = "Relational:ContainerColumnName"; - /// A projection that builds its rows, whose every member holds what the projection gave it. - internal static readonly RowShape Projected = new(RowKind.Projected, Empty(), Empty()); + /// + /// A source this library cannot read: no model, not a projection it can see into, not rows in + /// memory. Every value beneath a member is taken to reach the result, and only members holding a + /// value are kept, as in 3.1.0. + /// + internal static readonly RowShape Unknown = new(RowKind.Entity); + + /// Rows in memory. + internal static readonly RowShape InMemory = new(RowKind.InMemory); - /// Rows held in memory, whose every member holds whatever the object holds. - internal static readonly RowShape InMemory = new(RowKind.InMemory, Empty(), Empty()); + private readonly HashSet? _assigned; + private readonly HashSet _narrowable = new(StringComparer.Ordinal); + private readonly HashSet _kept = new(StringComparer.Ordinal); + private readonly HashSet _includes = new(StringComparer.Ordinal); + private readonly Dictionary _loadedWhole = new(StringComparer.Ordinal); + private readonly IEntityType? _entity; + private readonly bool _includeUnknown; - private readonly HashSet _alwaysLoaded; - private readonly HashSet _complex; + private RowShape(RowKind kind) => Kind = kind; + + private RowShape(RowKind kind, HashSet? assigned, IEnumerable narrowable) + : this(kind) + { + _assigned = assigned; + _narrowable.UnionWith(narrowable); + } - private RowShape(RowKind kind, HashSet alwaysLoaded, HashSet complex) + private RowShape(IEntityType entity, HashSet includes, bool includeUnknown) + : this(RowKind.Entity) { - Kind = kind; - _alwaysLoaded = alwaysLoaded; - _complex = complex; + _entity = entity; + _includes = includes; + _includeUnknown = includeUnknown; + + foreach (IProperty property in entity.GetProperties()) + { + _kept.Add(property.Name); + _loadedWhole[property.Name] = "it is a column, which EF Core reads whole"; + } + + foreach (INavigation navigation in entity.GetNavigations()) + { + if (IsJson(navigation.TargetEntityType)) + { + _loadedWhole[navigation.Name] = "it is stored as JSON, which EF Core cannot narrow"; + } + else + { + // An owned member is kept and narrowed; any other navigation is only ever narrowed for a + // caller who names it, since a synthesized projection leaves it out. + _narrowable.Add(navigation.Name); + } + + if (IsOwned(navigation)) + { + _kept.Add(navigation.Name); + } + } + + foreach (ISkipNavigation navigation in entity.GetSkipNavigations()) + { + _narrowable.Add(navigation.Name); + } + + foreach (string complex in ComplexProperties(entity)) + { + _kept.Add(complex); + _loadedWhole[complex] = "it is a complex property, which EF Core cannot narrow"; + } } /// Where the rows come from. internal RowKind Kind { get; } /// - /// True when rows from this source carry a value in the member whatever the caller includes: every - /// member of a projected or in-memory row, and an entity's owned and complex members, which EF Core - /// loads with the entity on every query. + /// True when a synthesized projection may keep a member holding an object: one a projection + /// assigned, or an entity's column, owned or complex member. Never an entity's navigation, which + /// projecting would load, and never an object held by a row in memory. + /// + internal bool Carries(string member) => Kind switch + { + RowKind.Projected => _assigned is null || _assigned.Contains(member), + RowKind.Entity => _entity is not null && _kept.Contains(member), + _ => false + }; + + /// + /// True when a synthesized projection may keep a member that holds a value. An entity keeps only + /// its mapped ones: a value EF Core does not map makes it read the whole entity to compute it, the + /// columns the policy denies included, and holds only its initial value anyway. + /// + internal bool CarriesValue(string member) => Kind switch + { + RowKind.Projected => _assigned is null || _assigned.Contains(member), + RowKind.Entity => _entity is null || _kept.Contains(member), + _ => true + }; + + /// + /// True when the core's projection can narrow the member to some of the fields beneath it. Only on + /// EF Core: its narrowing of a reference reads it through EF.Property. Not a column or a + /// complex property, which EF Core loads whole, not an owned type stored as JSON, and not a member + /// a projection built in a way the narrowing cannot reach. A source this cannot read narrows as it + /// always did, and leaves the core to refuse what it cannot build. + /// + internal bool Narrows(string member) => + (Kind == RowKind.Entity && _entity is null) || _narrowable.Contains(member); + + /// Why the core cannot narrow a member this source carries, for the trace. + internal string WhyNotNarrowed(string member) => Kind switch + { + RowKind.InMemory => "the rows are in memory, and narrowing it needs EF Core", + RowKind.Projected => "the projection builds it in a way the core cannot narrow", + _ => _loadedWhole.TryGetValue(member, out string? reason) ? reason : "narrowing it needs EF Core" + }; + + /// + /// True when a member named whole can carry anything its type can hold. An entity's navigation + /// cannot: projecting it loads that entity and what the entity owns, never the navigations beneath + /// it. Its columns, owned and complex members, and every member of any other source, can. /// - internal bool Carries(string member) => Kind != RowKind.Entity || _alwaysLoaded.Contains(member); + internal bool LoadsWhole(string path) + { + string member = path.Split('.')[0]; + + return Kind != RowKind.Entity || _entity is null || _kept.Contains(member); + } /// - /// True when the core's projection can narrow the member to some of the fields beneath it. Not in - /// memory, where its narrowing of a reference reads it through EF Core, and not for an EF Core - /// complex property, which it compares to null and EF Core refuses to. + /// True when the value at a path beneath the root can reach the result: every one in memory or in + /// a projection's assigned member, and, on an entity, one every navigation on the way to is loaded. /// - internal bool Narrows(string member) => Kind != RowKind.InMemory && !_complex.Contains(member); + /// + /// A navigation is loaded when it is owned, included, automatically included, or reachable by a + /// lazy loader. Anything the model cannot answer for counts as loaded, which only ever asks for a + /// projection that turns out not to be needed. + /// + internal bool Materializes(string path) + { + string[] segments = path.Split('.'); + + if (Kind == RowKind.InMemory) + { + return true; + } + + if (Kind == RowKind.Projected) + { + return _assigned is null || _assigned.Contains(segments[0]); + } + + IEntityType? current = _entity; + string prefix = string.Empty; + + for (int i = 0; i < segments.Length - 1; i++) + { + prefix = i == 0 ? segments[0] : $"{prefix}.{segments[i]}"; + + if (current is null) + { + return true; + } + + // A column holding an object, or a complex property, is read whole with its row. + if (current.FindProperty(segments[i]) is not null || ComplexProperties(current).Contains(segments[i])) + { + return true; + } + + INavigationBase? navigation = + (INavigationBase?)current.FindNavigation(segments[i]) ?? current.FindSkipNavigation(segments[i]); + + if (navigation is null) + { + return true; + } + + bool loaded = (navigation is INavigation owned && IsOwned(owned)) + || navigation.IsEagerLoaded + || _includeUnknown + || _includes.Contains(prefix) + || LoadsLazily(current); + + if (!loaded) + { + return false; + } + + current = navigation.TargetEntityType; + } + + return true; + } /// Reads the shape of a guarded query's source. /// /// Rows in memory first, because a projection over them is still in memory. Then a projection that /// builds its rows, which is how a caller makes its own row type out of entities. What is left is an - /// entity query, or a projection handing back entities, Select(o => o.Customer). Their owned - /// and complex members are read from the EF Core model; a navigation the entity does not own loads - /// only when something includes it, and a guarded query that has to narrow the row leaves such a - /// navigation out, as it always has. Projecting one would load it, and return more than the - /// unguarded call does. + /// entity query, or a projection handing back entities, Select(o => o.Customer), both read + /// from the EF Core model; without one, the source is . /// internal static RowShape Of(IQueryable source) where T : class { @@ -77,37 +253,207 @@ internal static RowShape Of(IQueryable source) where T : class if (DefaultOrder.BuildsRows(source.Expression)) { - return Projected; + return Projected(source); } - IEntityType? entityType = QueryRoot.EntityType(source.Expression, typeof(T)); + if (QueryRoot.EntityType(source.Expression, typeof(T)) is not { } entity) + { + return Unknown; + } + + HashSet includes = new(StringComparer.Ordinal); + bool unknown = !ReadIncludes(source.Expression, includes); + + return new RowShape(entity, includes, unknown); + } - if (entityType is null) + /// + /// A projection that builds its rows: which members its outermost initializer assigns, and which of + /// those the core can narrow. + /// + private static RowShape Projected(IQueryable source) where T : class + { + MethodCallExpression select = DefaultOrder.OutermostSelect(source.Expression)!; + LambdaExpression selector = (LambdaExpression)DefaultOrder.StripQuotes(select.Arguments[1]); + Expression body = DefaultOrder.StripConversions(selector.Body); + + // A constructor with arguments says nothing about which member each argument sets. + if (body is not MemberInitExpression initializer) { - return Entity; + return new RowShape(RowKind.Projected, null, Array.Empty()); } - HashSet alwaysLoaded = Empty(); - HashSet complex = Empty(); + HashSet assigned = new(StringComparer.Ordinal); + List narrowable = new(); + + ParameterExpression row = selector.Parameters[0]; + bool efCore = source.Provider is IAsyncQueryProvider; + IEntityType? rowSource = efCore ? QueryRoot.EntityType(source.Expression, row.Type) : null; - foreach (INavigation navigation in entityType.GetNavigations()) + foreach (MemberBinding binding in initializer.Bindings) { - if (navigation.ForeignKey.IsOwnership && !navigation.IsOnDependent) + assigned.Add(binding.Member.Name); + + // The narrowing reads the member through EF.Property, so only EF Core can run it. + if (efCore && binding is MemberAssignment assignment && CanNarrow(assignment, row, rowSource)) { - alwaysLoaded.Add(navigation.Name); + narrowable.Add(binding.Member.Name); } } - foreach (string name in ComplexProperties(entityType)) + return new RowShape(RowKind.Projected, assigned, narrowable); + } + + /// + /// True when the core's narrowing of an assigned member translates: an object the initializer + /// builds, a list a subquery reads into one the core can bind, or a navigation of the entity the + /// projection reads that is neither complex nor stored as JSON. + /// + private static bool CanNarrow(MemberAssignment assignment, ParameterExpression row, IEntityType? rowSource) + { + Type memberType = assignment.Member is PropertyInfo property ? property.PropertyType : typeof(object); + + // The core builds a List for a narrowed collection and skips a member that cannot hold one. + if (CacheReflection.GetCollectionElementType(memberType) is { } element + && !memberType.IsAssignableFrom(typeof(List<>).MakeGenericType(element))) + { + return false; + } + + Expression value = DefaultOrder.StripConversions(assignment.Expression); + + if (value is MemberInitExpression) + { + return true; + } + + if (value is MethodCallExpression { Method.Name: "ToList" or "ToArray" or "ToHashSet" } read + && (read.Method.DeclaringType == typeof(Enumerable) || read.Method.DeclaringType == typeof(Queryable))) { - alwaysLoaded.Add(name); - complex.Add(name); + return true; } - return alwaysLoaded.Count == 0 ? Entity : new RowShape(RowKind.Entity, alwaysLoaded, complex); + if (value is MemberExpression { Member: PropertyInfo member, Expression: { } owner } + && DefaultOrder.StripConversions(owner) == row + && rowSource?.FindNavigation(member.Name) is { } navigation) + { + return !IsJson(navigation.TargetEntityType); + } + + return false; } - private static HashSet Empty() => new(StringComparer.Ordinal); + /// + /// Collects every navigation path an Include or ThenInclude on the chain loads, each + /// with its prefixes. False when one names its path in a form this cannot read, so every navigation + /// counts as loaded. + /// + private static bool ReadIncludes(Expression expression, HashSet includes) + { + List calls = new(); + + for (Expression? node = expression; node is MethodCallExpression call; + node = call.Arguments.Count > 0 ? call.Arguments[0] : null) + { + if (call.Method.DeclaringType == typeof(EntityFrameworkQueryableExtensions) + && call.Method.Name is nameof(EntityFrameworkQueryableExtensions.Include) + or nameof(EntityFrameworkQueryableExtensions.ThenInclude)) + { + calls.Add(call); + } + } + + bool readable = true; + string? current = null; + + for (int i = calls.Count - 1; i >= 0; i--) + { + MethodCallExpression call = calls[i]; + bool then = call.Method.Name == nameof(EntityFrameworkQueryableExtensions.ThenInclude); + string? path = null; + bool named = false; + + if (!then && call.Arguments[1] is ConstantExpression { Value: string text }) + { + path = string.Join('.', text.Split('.', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries)); + named = true; + } + else if (DefaultOrder.StripQuotes(call.Arguments[1]) is LambdaExpression lambda + && MemberChain(lambda.Body, lambda.Parameters[0]) is { } chain) + { + path = then ? (current is null ? null : $"{current}.{chain}") : chain; + } + + if (string.IsNullOrEmpty(path)) + { + readable = false; + current = null; + + continue; + } + + string[] segments = path.Split('.'); + + for (int j = 1; j <= segments.Length; j++) + { + includes.Add(string.Join('.', segments, 0, j)); + } + + // A ThenInclude continues a lambda Include; one named by a string starts nothing to continue. + current = named ? null : path; + } + + return readable; + } + + /// + /// The member path a navigation lambda reads, o => o.Lines or o => o.Customer.Address, + /// through casts and a filtered include's operators; null for any other shape. + /// + private static string? MemberChain(Expression body, ParameterExpression parameter) + { + Expression node = DefaultOrder.StripConversions(body); + + // A filtered include reads the navigation through Where, OrderBy, Skip, Take and their kin. + while (node is MethodCallExpression call + && (call.Method.DeclaringType == typeof(Enumerable) || call.Method.DeclaringType == typeof(Queryable)) + && call.Arguments.Count > 0) + { + node = DefaultOrder.StripConversions(call.Arguments[0]); + } + + List names = new(); + + while (node is MemberExpression { Member: PropertyInfo or FieldInfo, Expression: { } owner } member) + { + names.Add(member.Member.Name); + node = DefaultOrder.StripConversions(owner); + } + + if (node != parameter || names.Count == 0) + { + return null; + } + + names.Reverse(); + + return string.Join('.', names); + } + + /// True for a navigation to a type the entity owns. + private static bool IsOwned(INavigation navigation) => + navigation.ForeignKey.IsOwnership && !navigation.IsOnDependent; + + /// True for an owned type EF Core stores in a JSON column, which it cannot project into. + private static bool IsJson(IEntityType entity) => entity.FindAnnotation(JsonContainer)?.Value is not null; + + /// + /// True when a lazy loader can fill this entity's navigations after the query: EF Core's proxies and + /// an injected ILazyLoader both give it a service property. + /// + private static bool LoadsLazily(IEntityType entity) => + entity.GetServiceProperties().Any(service => + service.ClrType == typeof(ILazyLoader) || service.ClrType == typeof(Action)); /// /// The names of an entity type's complex properties, which EF Core 8 introduced and loads with the @@ -117,8 +463,10 @@ internal static RowShape Of(IQueryable source) where T : class /// Read by reflection because the library compiles against EF Core 6. The method is declared on an /// interface the entity type implements, so the interfaces are searched rather than the class. /// - private static IEnumerable ComplexProperties(IEntityType entityType) + private static HashSet ComplexProperties(IEntityType entityType) { + HashSet names = new(StringComparer.Ordinal); + foreach (Type contract in entityType.GetType().GetInterfaces()) { MethodInfo? method = contract.GetMethod("GetComplexProperties", Type.EmptyTypes); @@ -134,13 +482,15 @@ private static IEnumerable ComplexProperties(IEntityType entityType) { if (property?.GetType().GetProperty("Name")?.GetValue(property) is string name) { - yield return name; + names.Add(name); } } } - yield break; + break; } + + return names; } } From a4ee5cfb722570450715120d5f6385925e54b4f9 Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 05:25:15 +0300 Subject: [PATCH 05/22] fix(policies): a forced scope beneath a member asks for no projection on its own It filters the rows that hold the member, as it always has for a list returned whole, and asking would leave out every included list of a scoped child type in a multi-tenant model. When a projection is needed for another reason, such a member is still left out whole. Co-Authored-By: Claude Opus 5 --- .../Policies/ProjectionReviewTests.cs | 40 ++++++++++++++++--- .../Policies/Source/FilterSanitizer.cs | 5 ++- 2 files changed, 38 insertions(+), 7 deletions(-) diff --git a/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs b/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs index 5a344a2..d9a42c6 100644 --- a/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs +++ b/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs @@ -568,16 +568,25 @@ public void A_navigation_a_lazy_loader_can_fill_needs_the_projection() } /// - /// A forced scope on a list's elements filters the rows of the query, never the elements, so an - /// included list would carry every tenant's children. The entity is projected and the list left out. + /// A forced scope on a list's elements filters the rows that hold the list, never the elements, + /// so a list kept whole would carry every tenant's children. A projection that is needed anyway + /// leaves it out whole; the scope alone asks for none, as it never did. /// [Fact] - public void A_list_whose_elements_carry_a_forced_scope_is_not_returned_whole() + public void A_list_whose_elements_carry_a_forced_scope_is_left_out_of_a_needed_projection() { - RvFamily family = Guard(_db.Families.Include(f => f.Kids)).ToList(new Filter()).Data.Single(); + PolicyQueryable guarded = Guard(_db.Families.Select(f => new RvFamilyRow + { + Id = f.Id, + Tenant = f.Name, + Kids = f.Kids.Select(k => new RvKidRow { Name = k.Name, TenantId = k.TenantId }).ToList() + })); + + RvFamilyRow row = guarded.ToList(new Filter()).Data.Single(); - Assert.Equal("F1", family.Name); - Assert.Empty(family.Kids); + Assert.Empty(row.Kids); + Assert.Contains(guarded.LastTrace!.Decisions, decision => decision.FieldPath == "Kids" + && decision.Reason == "left out whole: a scope forced beneath it cannot be applied to what it holds"); } /// @@ -799,6 +808,25 @@ internal FilterResultOf(DynamicWhere.ex.Classes.Result.FilterResult result) } } + public sealed class RvKidRow + { + public string Name { get; set; } = string.Empty; + + [DwForceWhere(Operator.Equal, ContextValue = "TenantId")] + public int TenantId { get; set; } + } + + [DwEntity(RequirePolicy = true)] + public sealed class RvFamilyRow + { + public int Id { get; set; } + + [DwDenied] + public string? Tenant { get; set; } + + public List Kids { get; set; } = new(); + } + public sealed class RvNode { public string Name { get; set; } = string.Empty; diff --git a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs index e3ebbd4..06f0532 100644 --- a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs +++ b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs @@ -1929,7 +1929,10 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) gate.Record(path, PolicyFeature.Select, PolicyAction.Dropped, beneath); } - if (reached.Count > 0 || (opens && (facts.Opaque || forced))) + // A forced scope beneath a member does not ask for a projection on its own. It filters the + // rows that hold the member, which is what it has always done for a list returned whole, + // and asking would leave out every included list of a scoped child type. + if (reached.Count > 0 || (opens && facts.Opaque)) { anyDenied = true; } From 60947151b994714a6f2153bb8fcf6909e9ad887c Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 05:28:17 +0300 Subject: [PATCH 06/22] docs: the agent reference and the release notes say what the review changed llms.txt section 17 now states when a denial asks for a projection (only where its value can reach the result: an entity's column, owned or complex member, or a navigation the query loads; a projection's assigned members; anything in memory), what each source keeps, when a member is kept whole, narrowed or left out whole, and what caller-written Selects refuse. The 3.2.0 history entry lists the three fail-opens the review found in passing, and the limits add the forced scope on a list's element type and members the policy cannot see into. Two stale lines are corrected: the async Summary no longer counts synchronously, and trap 37 no longer says every projected query goes unordered. The core's release notes are rewritten to match, and the ASP.NET Core package's say what /simulate now reports. Co-Authored-By: Claude Opus 5 --- ...DynamicWhere.ex.Policies.AspNetCore.csproj | 2 +- DynamicWhere.ex/DynamicWhere.ex.csproj | 18 +- OfficialWebsite/public/llms.txt | 176 +++++++++++------- 3 files changed, 125 insertions(+), 71 deletions(-) diff --git a/DynamicWhere.ex.Policies.AspNetCore/DynamicWhere.ex.Policies.AspNetCore.csproj b/DynamicWhere.ex.Policies.AspNetCore/DynamicWhere.ex.Policies.AspNetCore.csproj index 80e8f71..78ce335 100644 --- a/DynamicWhere.ex.Policies.AspNetCore/DynamicWhere.ex.Policies.AspNetCore.csproj +++ b/DynamicWhere.ex.Policies.AspNetCore/DynamicWhere.ex.Policies.AspNetCore.csproj @@ -32,7 +32,7 @@ Free Forever — Copyright © Sajjad H. Al-Khafaji MIT 3.2.0 - v3.2.0 — Released in lockstep with DynamicWhere.ex 3.2.0, which it now requires. No API change in this package. The core release changes what /simulate reports for a request that sends no Selects. The projection it shows is now built whenever a field is denied at any depth, and keeps members holding a collection of values. A simulation has no source, so it reads the type as an entity query and leaves out members holding an object, which a guarded query over a projected or in-memory source keeps. + v3.2.0 — Released in lockstep with DynamicWhere.ex 3.2.0, which it now requires. No API change in this package. The core release changes what /simulate reports for a request that sends no Selects. A simulation has no source, so it reads the type as one it cannot see into: every denial beneath a member counts, and the projection it shows keeps only members holding a value, where a guarded query over a projected row, an entity or rows in memory keeps what that source carries. v3.1.0 — Released in lockstep with DynamicWhere.ex 3.1.0, which it now requires. No API change in this package. The core release changes what its endpoints report, as it changes any guarded call: /simulate answers with 3.1.0's pipeline, the new caps, the strict tier's refusals and a type's DefaultOrder included, and /health's loadedAt and ageSeconds advance on a poll that confirms the version served, so an idle healthy store no longer reads as stale. One change needs action: DwClaimsAdapter.FromClaims returns an unprepared context, and ApplyPolicy(context) now refuses one with PolicyContextNotPrepared even when no store is configured. Use CreateContextAsync or ToPolicyContextAsync, or pass the context through DwPolicy.PrepareAsync before querying. In this package, the audit middleware's warning for events recorded with no IDwAuditSink registered now names both ways to stop recording them: remove [DwAudit] from the fields that produced them, or turn off DwPolicyOptions.AuditRefusals, the core's new switch that records refused guarded queries as audit events, which the middleware drains like any other. diff --git a/DynamicWhere.ex/DynamicWhere.ex.csproj b/DynamicWhere.ex/DynamicWhere.ex.csproj index d0671b3..76437e8 100644 --- a/DynamicWhere.ex/DynamicWhere.ex.csproj +++ b/DynamicWhere.ex/DynamicWhere.ex.csproj @@ -48,19 +48,23 @@ https://github.com/Sajadh92/DynamicWhere.ex icon.png README.md - v3.2.0 — A row projected before ApplyPolicy keeps its nested objects and lists, a denial beneath a member is enforced, a declared default order reaches projected rows, and every asynchronous terminal takes a CancellationToken. + v3.2.0 — A row projected before ApplyPolicy keeps its nested objects and lists, denials the policy gate could not see are enforced, a declared default order reaches projected rows, and every asynchronous terminal takes a CancellationToken. -Security fix: a field denied for Select only beneath a member was not enforced when the caller sent no Selects. A guarded query synthesizes a projection for a denied field, and it did so only when a top-level field was denied. With every denial beneath a member, the whole row came back with the denied value in it: in a list or nested object of a row projected before ApplyPolicy, in a row held in memory, and in an entity's included, automatically included, lazily loaded or owned member, typed and dynamic, in both tiers, for a Filter and a Segment. A denial at any depth now synthesizes the projection. +Security fix: a field denied for Select only beneath a member was not enforced when the caller sent no Selects. A guarded query synthesizes a projection for a denied field, and it did so only when a simple field at the top of the type was denied. With every denial beneath a member, the whole row came back with the denied value in it: in a list or nested object of a row projected before ApplyPolicy, in a row held in memory, and in an entity's included, automatically included, lazily loaded or owned member, typed and dynamic, in both tiers, for a Filter and a Segment. Such a denial now synthesizes the projection whenever its value can reach the result. On an entity that means beneath a column, an owned or complex member, or a navigation the query loads through Include, an automatic include or a lazy loader; a denial beneath a navigation nothing loads never leaves the database, and the entity is read exactly as before. -Security fix: under the convenience tier, Selects naming a navigation whose element key (Id) is denied was narrowed to the allowed fields beneath it, and the core's typed projection added the key back. Such a narrowing is refused with FieldDeniedForSelect in both tiers, as naming a sibling of the key already was. +Security fix: a field denied at the top of a type whose own type is not a simple value, such as a byte array, a list, an owned object or a JSON column, synthesized no projection either, so with nothing else denied it came back. -Security fix: Selects naming a member typed as a collection the core does not unwrap, such as IReadOnlyList<T>, returned every field beneath it, denied ones included, in both tiers. The projection gate read collections through a narrower list than the attribute walker that puts policy on the fields beneath. It now reads them the same way, and a narrowing the core cannot project is refused with FieldDeniedForSelect. +Security fix: the attribute walker read any namespace starting with "System" as the framework's, so an application namespace such as SystemsCorp.Payroll got no policy beneath its types, and a [DwDenied] field there was returned, filterable and sortable. Only System and the namespaces beneath it are treated as the framework's now. -Behaviour change: the projection synthesized for a guarded query that sends no Selects keeps what the source carries. It used to keep scalars only, so every nested object and list of a row projected before ApplyPolicy came back null or empty as soon as any field was denied. A member holding a value is kept when allowed, and that now includes a collection of values such as byte[] or List<string>. A member holding an object, or a list of them, is kept whole when nothing beneath it is denied, and narrowed to the allowed fields when something is. That applies to a projected row, a row in memory, and an entity's owned and complex members, which are read from the EF Core model. An entity's other navigations are left out, as they were: an included navigation is not returned once a denial anywhere in the type needs a projection, and the type needs a public parameterless constructor for it, as it already did for a top-level denial. A member that cannot be narrowed is left out whole and recorded as Dropped with a reason starting "left out whole": one in memory, an EF Core complex property, one whose key is denied, and one the core cannot project through. +Security fix: under the convenience tier, Selects naming a navigation whose element key (Id) is denied was narrowed to the allowed fields beneath it, and the core's typed projection added the key back. Such a narrowing is refused with FieldDeniedForSelect in both tiers, as naming a sibling of the key already was. A navigation named through another, such as Main.Lead, now gates the key of Main, which the projection adds; it did not. -Behaviour change: [DwEntity(DefaultOrder)] applies to a projected source whose outermost Select builds the type in an object initializer and assigns every field the default names, at every level of a nested path. Any other projection still leaves the query in its own order. A Select, or a Filter with Selects, composed on the guarded handle keeps the rest of the chain unordered, and a composed Filter that sent orders gets no default later in the chain, as a composed Order already did not. +Security fix: Selects naming a member typed as a collection the core does not unwrap, such as IReadOnlyList<T>, returned every field beneath it, denied ones included, in both tiers. The projection gate read collections through a narrower list than the attribute walker that puts policy on the fields beneath; it now reads them the same way. Denials beneath a named member are also read from the policy's own rules, so a denied property with no setter and a rule on a path reached through a cycle are found, and a field denied deeper than the walker reaches or inside a framework collection such as Dictionary<string, T> is refused under the strict tier and narrowed away under the convenience tier. -New: every asynchronous terminal has overloads taking a CancellationToken, guarded and unguarded: ToListAsync and ToListAsyncDynamic with a Filter, ToListAsync with a Summary, and ToListAsync with a Segment. The token reaches both the count and the read. The overloads sit beside the 3.1 signatures, which are unchanged, so code compiled against 3.1 still binds. ToListAsync(filter, default) no longer compiles, because default fits both overloads. +Behaviour change: the projection synthesized for a guarded query that sends no Selects keeps what the source carries. It used to keep simple fields only, so every nested object and list of a row projected before ApplyPolicy came back null or empty as soon as any field was denied. A row a projection builds keeps the members its initializer assigns. An entity keeps its mapped columns, converted and JSON ones included, its owned and complex members, and every collection of simple values such as byte[] or List<string>. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, recorded as Dropped with a reason starting "left out whole". An entity's navigations, the objects of a row in memory, and a value EF Core does not map are left out, and the type needs a public parameterless constructor for the projection, as it already did. + +Behaviour change: [DwEntity(DefaultOrder)] applies to a projected source whose outermost Select builds the type in an object initializer and assigns every field the default names, at every level of a nested path, with nothing EF Core would have to compute on the client. Any other projection still leaves the query in its own order. A Select, or a Filter with Selects, composed on the guarded handle keeps the rest of the chain unordered, and a composed Filter that sent orders gets no default later in the chain, as a composed Order already did not. + +New: every asynchronous terminal has overloads taking a CancellationToken, guarded and unguarded: ToListAsync and ToListAsyncDynamic with a Filter, ToListAsync with a Summary, and ToListAsync with a Segment. The token reaches both the count and the read. The overloads sit beside the 3.1 signatures, which are unchanged, so code compiled against 3.1 still binds. ToListAsync(filter, default) no longer compiles, because default fits both overloads, and a reflection lookup of one of these methods by name alone now finds several. Change: ToListAsyncDynamic and the asynchronous Summary read through EF Core's ToListAsync, and the Summary counts through CountAsync. They used to read synchronously on a thread-pool thread, so on an EF Core query a canceled token now reaches the database. A provider that is not EF Core's keeps the synchronous read. diff --git a/OfficialWebsite/public/llms.txt b/OfficialWebsite/public/llms.txt index 1176c28..ea63545 100644 --- a/OfficialWebsite/public/llms.txt +++ b/OfficialWebsite/public/llms.txt @@ -884,7 +884,8 @@ GroupBy.Fields emitted key's own type (an enum key stays an enum), then one property per `AggregateBy.Alias`, in list order. Rows are `DynamicClass` objects; `SummaryResult.Data` is `List`. - A null group key is its own group (`{ "categoryName": null, … }`). -- `ToListAsync(Summary)` counts synchronously; only the data read is async. +- `ToListAsync(Summary)` counts and reads through EF Core's `CountAsync` and `ToListAsync` (3.2.0; it used to count + synchronously). On a provider that is not EF Core's, rows in memory among them, it counts and reads synchronously. ``` Aggregator Emitted per group Field accepted Result type @@ -1786,8 +1787,8 @@ How this differs from the unguarded surface: Rules: - Every guarded query runs over `AsNoTracking()`. Returned EF Core entities are detached, so a masked value is never saved back. An in-memory source has no such copy: without `Selects` and with nothing denied, the rows returned are - the source objects themselves, transformed in place. When a projection is synthesized the rows are new, but a - member kept whole is still the source's own object, and a transform beneath it changes it in place (3.2.0). + the source objects themselves, transformed in place. When a projection is synthesized the rows are new, and they + hold no member of an object type, so the source objects are left as they were. - Chaining keeps decisions: each composable returns a new handle, and the terminal's trace includes the earlier links' decisions. - The terminal's trace is on `result.Policy` only when `DwPolicyOptions.IncludeTraceInResult` allows it: null follows the tier (off under `Strict`, on under `Convenience`), and `true` or `false` overrides it (3.1.0). `LastTrace` holds it whatever the setting. - `getQueryString: true` under `Strict` → `QueryStringDenied` (14). This is checked before sanitizing; dry run records it instead. @@ -2308,35 +2309,69 @@ DefaultOrder field denied for Segment, in a Segment (3.1.0) left out l - "allowed leaves": when `Selects` names a navigation with a denied field beneath it, it is replaced by the allowed leaf paths beneath it. A navigation with nothing denied beneath it is kept as written. - The fields beneath are read the way the attribute walker reads them (3.2.0): through any collection type, so a - member typed `IReadOnlyList` or an application's own collection no longer hides its denied fields. + member typed `IReadOnlyList` or an application's own collection no longer hides its denied fields, and no + deeper than the walker's four segments. The providers' fragments are asked too, so a denied property with no + setter and a rule on a path reached through a cycle count (3.2.0). + - A named member that can carry anything its type holds, a member of a projected or in-memory row or an entity's + column, owned or complex member, and whose type holds a field denied for Select that no path names (deeper than + four segments, or inside a framework generic such as `Dictionary`): Strict refuses it with + `FieldDeniedForSelect`, Convenience narrows it away (3.2.0). An entity's navigation loads only its own entity, so + only the paths are asked about there. - A narrowing that cannot be built as gated is refused with `FieldDeniedForSelect`, in both tiers (3.2.0). The core's typed projection adds the key (`Id`) of every nested node it builds, so a narrowing whose nodes carry a - denied key would return it; and a path through a collection the core does not unwrap fails its validation. + denied key would return it; a path through a collection the core does not unwrap fails its validation; and a + column, complex property or JSON-stored member, a member of a row in memory, or one a projection builds some way + the core cannot narrow, cannot be narrowed at all. + - A navigation named through another, `Main.Lead`, gates the key of every node it passes through, which the + builder adds, as a dotted path to a value always did (3.2.0). A denied one refuses the projection. - "allowed members" (3.2.0; "allowed scalars" before) applies when `Selects` is null or empty and a field is denied - for Select at the top of T or anywhere beneath one of its members, to the depth the attribute walker reaches. It - runs for a whole-`Filter` terminal and for a Segment, not for a single-clause composable call. The projection is - what an unguarded call would return, less what the policy withholds: - - every allowed member that is readable, writable and not an indexer and holds a value: a simple type (primitive, - enum, `string`, `decimal`, `DateTime`, `DateOnly`, `TimeOnly`, `DateTimeOffset`, `TimeSpan`, `Guid`) or a - collection of one (`byte[]`, `string[]`, `List`); - - every allowed member holding an object or a collection of objects, where the source carries it: whole when - nothing beneath it is denied, and narrowed to the allowed leaves when something is, as a caller naming it would - get. The source carries every member of a row a projection builds before `ApplyPolicy` (the outermost `Select` - constructs the row, in an initializer or with a constructor) and of a row in memory, and an entity's owned and - complex members, which are read from the EF Core model. A `Select` handing back an entity, - `Select(o => o.Customer)`, builds no row and is read as an entity query; - - an entity's other navigations are left out: an unguarded call loads one only when something includes it, and - projecting it would load it. An included or automatically included navigation is therefore not returned once a - projection is needed. Name it in `Selects` to get it, narrowed; - - a member that cannot be narrowed is left out whole, recorded as `Dropped` on `Select` with a reason starting - `left out whole`: on rows in memory, where the core narrows a reference through `EF.Property`; on an EF Core - complex property, which the core compares to null and EF Core refuses to; where the core's projection would add a - denied key back; and where the core cannot project a path beneath it; - - a narrowed reference that is null in the source comes back as an empty object, as it does for a caller's own - dotted `Selects`; - - each denied field, at the top or beneath, is recorded as `Dropped` on `Select`. + for Select where its value can reach the result. It runs for a whole-`Filter` terminal and for a Segment, not for a + single-clause composable call. + - A denial at the top of T always counts, whatever the member holds: a scalar, a blob, a list, an owned object, a + JSON column (3.2.0; 3.1.0 asked only about simple members, so a denied `byte[]` or owned member came back). + - A denial beneath a member counts when its value can reach the result: + - on an entity, beneath a column, an owned or complex member, or a navigation something loads: an `Include` or + `ThenInclude` on the query (a form the library cannot read counts as loading every navigation), an automatic + include, or a lazy loader (proxies, or an injected `ILazyLoader`, whose navigations fill after the query). A + denial beneath a navigation nothing loads never leaves the database and needs no projection, so a connected + model is read as it was in 3.1.0; + - on a row a projection builds, beneath a member its initializer assigns; + - on a row in memory, beneath any member. + - So does a member whose type can hold a field denied for Select that no path names, deeper than the walker's four + segments or inside a framework generic such as `Dictionary`, and a member typed `object`. + - The denials beneath a member come from the providers' fragments as well as from walking the type, so a denied + property with no setter, a rule on a path reached through a cycle, and a rule deeper than the walk all count. + - A forced scope beneath a member asks for no projection on its own: it filters the rows that hold the member, as it + always has. When a projection is needed anyway, the member is left out whole (below). + - The projection is what an unguarded call would return, less what the policy withholds: + - every allowed member holding a value, a simple type (primitive, enum, `string`, `decimal`, `DateTime`, + `DateOnly`, `TimeOnly`, `DateTimeOffset`, `TimeSpan`, `Guid`) or a collection of one (`byte[]`, `string[]`, + `List`), that the source carries: every one a projection assigns, every one of a row in memory, and + every one EF Core maps on an entity. A value EF Core does not map is left out: computing it would make EF Core + read the whole entity, the denied columns included, and it holds only its initial value anyway; + - every allowed member holding an object or a list of them that the source carries: a member a projection's + initializer assigns, and an entity's columns (converted or JSON), owned and complex members. It is kept whole + when nothing beneath it is denied, nothing in its type is denied or typed `object`, no forced scope is beneath + it and no transform beneath it lands on a property with no setter; + - otherwise it is narrowed to the allowed leaves beneath it, to four segments, as a caller naming it would get, + when the core's narrowing translates: an object the projection's initializer builds, a list a subquery reads + into a type the core can bind (not an array or a set), a navigation that is neither complex nor stored as JSON, + or an entity's owned member not stored as JSON. A leaf that can hold what the policy cannot name is left out; + - otherwise it is left out whole, recorded as `Dropped` on `Select` with a reason starting `left out whole`: a + scope forced beneath it; a column, complex property or JSON-stored member, which EF Core reads whole; one the + projection builds some other way, by a constructor, a conditional or an unassigned member; a denied key the + core's projection would add back; a path the core cannot project; nothing beneath it left to select; + - Never kept: an entity's navigation, included or not, since projecting it would load it (name it in `Selects` to + get it, narrowed); an object held by a row in memory, since a kept object is the caller's own and a transform + would change it in place; a member with no setter; a member named with one of the parser's words. + - A `Select` handing back an entity, `Select(o => o.Customer)`, builds no row and is read as an entity query. + - A narrowed reference that is null in the source comes back as an empty object, as it does for a caller's own + dotted `Selects`. + - A narrowed member carries every allowed leaf beneath it. An entity reached beneath it has its own navigations + projected and so loaded, which the source may not have included, exactly as `Selects` naming the member does. + - Each denied field whose value can reach the result, at the top or beneath, is recorded as `Dropped` on `Select`. - It never throws for the denied field. It throws `AllSelectsDenied` only when no field is left. - - With nothing denied at any depth, `Selects` stays null and the query is the one an unguarded call runs. + - With nothing counted, `Selects` stays null and the query is the one an unguarded call runs. - A path that matches nothing on `T` (3.1.0): - Convenience, and dry run in either tier: validation throws `LogicException` `ConditionMustHasValidFieldName` before any policy decision, as it does unguarded. @@ -3149,9 +3184,10 @@ if (!sim.WouldRun) logger.LogInformation("{Code}", sim.Refusal!.ErrorCode); - A `PolicyException` becomes `Refusal`. Any other exception propagates, unwrapped even from the runtime overload. - Under `Strict`, outside dry run, a path that matches nothing is therefore a `Refusal` with the clause's `FieldDeniedFor*` code and `FieldPath` `"*"`, not a `LogicException` (3.1.0). The `Trace` names it. - A simulated `Filter` or `Segment` that sends no orders gets the type's `DefaultOrder` in `Clause.Orders`, less the fields the caller may not order by (3.1.0). -- A simulation has no source, so it reads T as an entity query whose model it cannot see (3.2.0). With no `Selects`, - a synthesized `Clause.Selects` leaves out every member holding an object; the guarded query over a projected or - in-memory source keeps those members, whole or narrowed, and one over an entity keeps its owned members. +- A simulation has no source, so it reads T as a source it cannot see into (3.2.0): every denial beneath a member + counts, and with no `Selects` a synthesized `Clause.Selects` keeps only members holding a value. The guarded query + over a projected row keeps its assigned objects, one over an entity keeps its columns, owned and complex members + and asks only about denials whose value it loads, and one in memory keeps values only. - Runtime overload errors: - a value-type `entityType`, or a `TClause` other than `Filter`, `Summary` or `Segment` → `ArgumentException`; - null entity, context or options → `ArgumentNullException`. @@ -4202,8 +4238,9 @@ Read before generating policy attributes. ignore it. A caller who sends any `Orders` gets exactly those, so rows tied on them can still move between pages, and an `IQueryable` ordered before `ApplyPolicy`, or by a composed `Order` before `Page`, keeps its own order; a list sorted in memory before `ApplyPolicy` is not seen as ordered and gets the default, so send the - order with the filter. A projected query — a `Select` before `ApplyPolicy` or the guarded `Select` — takes no - default at all. A default field the caller may not order by is left out without an error. End every + order with the filter. A projected query takes the default only when its outermost `Select` builds T in an + object initializer assigning every default field (3.2.0); the guarded `Select` composed afterwards never does. + A default field the caller may not order by is left out without an error. End every order meant for paging with a unique field, such as the key. 38. **A `[DwEntity]` on a derived type replaces its base type's.** The attribute allows one per type, and .NET inheritance hands a derived type its own when it declares one, so the base type's `RequirePolicy` and @@ -4651,40 +4688,50 @@ MemoryCalculationInput ### History ``` -3.2.0 A projected row keeps its members, a denial beneath a member is enforced, a default order that reaches - projected rows, and a CancellationToken on every async terminal. The bullets marked "Behaviour change" - change what code written for 3.1.0 does. - - Security fix and behaviour change. With no Selects, a field denied for Select only beneath a member, none - at the top of T, synthesized no projection, so the whole row came back with the denied value in it: in a - list or nested object of a row projected before ApplyPolicy, in a row in memory, and in an entity's - included, automatically included, lazily loaded or owned member. A denial at any depth now synthesizes the - projection, in both tiers, typed and dynamic, for a Filter and a Segment. On an entity that projection - leaves out every navigation EF Core does not own, so an included navigation is not returned once a denial - anywhere in T needs one; and T needs a public parameterless constructor for it, as it already did for a - top-level denial. - - Behaviour change. The synthesized projection keeps what the source carries. A row projected before - ApplyPolicy, or held in memory, keeps its nested objects and lists: whole when nothing beneath is denied, - narrowed around a denial. An entity keeps its owned and complex members, and every member holding a - collection of simple values (byte[], List). In 3.1.0 all of these came back null or empty whenever - a field was denied. A member that cannot be narrowed is left out whole, with a "left out whole" Dropped - decision. +3.2.0 A projected row keeps its members, denials the gate could not see are enforced, a default order that + reaches projected rows, and a CancellationToken on every async terminal. The bullets marked "Behaviour + change" change what code written for 3.1.0 does. + - Security fix and behaviour change. With no Selects, a field denied for Select only beneath a member, none at + the top of T, synthesized no projection, so the whole row came back with the denied value in it: in a list or + nested object of a row projected before ApplyPolicy, in a row in memory, and in an entity's included, + automatically included, lazily loaded or owned member. Such a denial now synthesizes the projection when its + value can reach the result, in both tiers, typed and dynamic, for a Filter and a Segment. On an entity that + means beneath a column, an owned or complex member, or a navigation the query loads; a denial beneath a + navigation nothing loads never leaves the database, and the entity is read as in 3.1.0. + - Security fix. A field denied at the top of T whose type is not a simple value, a blob, a list, an owned + object or a JSON column, synthesized no projection either, so with nothing else denied it came back. + - Security fix. The attribute walker read any namespace starting with "System" as the framework's, so an + application's SystemsCorp.Payroll got no fragment beneath its types, and a [DwDenied] field there was + returned, filterable and sortable. Only System and the namespaces beneath it are the framework's now. - Security fix. Under Convenience, Selects naming a navigation whose element key (Id) is denied narrowed the key away, and the core's typed projection added it back. Such a narrowing is refused with - FieldDeniedForSelect in both tiers, as naming a sibling of the key already was. + FieldDeniedForSelect in both tiers, as naming a sibling of the key already was. A navigation named through + another, Main.Lead, now gates Main's key, which the builder adds; it did not. - Security fix. Selects naming a member typed as a collection the core does not unwrap (IReadOnlyList, IReadOnlyCollection, Collection or an application's own) returned every field beneath it, denied ones included, in both tiers: the projection gate read collections through a narrower list than the attribute - walker. The gate reads them as the walker does, and a narrowing the core cannot project is refused with - FieldDeniedForSelect. + walker. It reads them as the walker does, and a narrowing the core cannot project is refused. + - Security fix. Selects naming a member of a projected or in-memory row whose type holds a field denied for + Select that no path names, deeper than four segments or inside a framework generic such as Dictionary, returned it. Strict refuses it; Convenience narrows it away. A denied property with no setter, and a rule on + a path reached through a cycle, are found beneath a named member too. + - Behaviour change. The synthesized projection keeps what the source carries. A row a projection builds keeps + its assigned nested objects and lists; an entity keeps its columns, converted and JSON ones included, and its + owned and complex members, and every member holding a collection of simple values (byte[], List). In + 3.1.0 all of these came back null or empty whenever a field was denied. A member is kept whole when nothing + it can hold is denied, narrowed around a denial where the core's narrowing translates, and otherwise left out + whole with a "left out whole" Dropped decision. An entity's navigations and the objects of a row in memory + are left out, as before; a value EF Core does not map is left out too. - Behaviour change. [DwEntity(DefaultOrder)] applies to a projected source whose outermost Select builds T - in an object initializer assigning every field the default names; any other projection still leaves the - query in its own order. A Select, or a Filter with Selects, composed on the guarded handle keeps the rest - of the chain unordered. A composed Filter that sent orders gets no default later in the chain, as a - composed Order already did not. + in an object initializer assigning every field the default names, with nothing EF Core would compute on the + client; any other projection still leaves the query in its own order. A Select, or a Filter with Selects, + composed on the guarded handle keeps the rest of the chain unordered. A composed Filter that sent orders + gets no default later in the chain, as a composed Order already did not. - Every async terminal has overloads taking a CancellationToken, guarded and unguarded: ToListAsync and ToListAsyncDynamic with a Filter, ToListAsync with a Summary, and ToListAsync with a Segment. The 3.1 signatures are unchanged, so code compiled against 3.1 still binds. The token reaches the count and the - read. + read. ToListAsync(filter, default) no longer compiles, and a reflection lookup of one of these methods by + name alone now finds several. - Behaviour change. ToListAsyncDynamic and the async Summary read through EF Core's ToListAsync, and the async Summary counts through CountAsync, instead of reading synchronously; on an EF Core query a canceled token now reaches the database. A provider that is not EF Core's keeps the synchronous read. @@ -4842,11 +4889,14 @@ MemoryCalculationInput before EF Core translates it. `Contains` / `StartsWith` / `EndsWith` work only on string members. - Cache configuration changes are eventually consistent: calls already running finish with the options they read. - With no `Selects`, a guarded query over an entity leaves out every navigation EF Core does not own once a denial - anywhere in T needs a projection (section 17). Name the navigation in `Selects` to get it. -- A denial on a type held inside a dictionary or another `System` generic (`Dictionary`, - `KeyValuePair<,>`, `Tuple<>`) is not enforced through it. The attribute walker treats such a member as a value, so - nothing beneath it has a path, and naming it, or a synthesized projection keeping it, returns it whole. Hold such - values in a list of the policed type instead. + needs a projection (section 17), an included one too. Name the navigation in `Selects` to get it, narrowed. +- A forced scope declared on a list's element type filters the rows that hold the list, never its elements. Selects + naming the list returns every element, those the scope excludes included, as in every release; a synthesized + projection leaves such a list out. Scope the elements where the row is built. +- A member typed `object` is opaque to the policy: a synthesized projection leaves it out, and naming it returns + whatever it holds. A framework generic holding a policed type (`Dictionary`) has no paths + beneath it: naming it is refused under Strict and narrowed away under Convenience, and a synthesized projection + leaves it out. - A simulation reads T as an entity query (section 23). --- From b0feed6952b46ad766ad988ac3dc0d8f731338ff Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 05:30:11 +0300 Subject: [PATCH 07/22] fix(policies): a stale rule beneath a member, and a transform-only narrowing From my own pass over the round-2 change. A rule on a path the type does not have, one written for a member since removed, counted as a denial beneath the member above it, so the strict tier refused naming that member. Such a rule holds nothing a projection could carry, and 3.1.0 never looked at it. It is skipped now. A member named in Selects is narrowed when a transform beneath it lands on a property with no setter. Where the source cannot be narrowed that way, it was refused as a denied field, though nothing is denied. It is kept whole instead, as in 3.1.0. Co-Authored-By: Claude Opus 5 --- .../Policies/ProjectionReviewTests.cs | 16 ++++++++++++++++ .../Policies/Source/FilterSanitizer.cs | 15 ++++++++++++++- 2 files changed, 30 insertions(+), 1 deletion(-) diff --git a/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs b/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs index d9a42c6..0dca9ab 100644 --- a/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs +++ b/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs @@ -792,6 +792,22 @@ public void A_rule_on_a_path_through_a_cycle_is_seen_beneath_a_member() Assert.Throws(() => Guard(rows.AsQueryable(), DwTier.Convenience, rule).ToList(Selecting("Id", "Head"))); } + /// + /// A rule on a path the type does not have, one left behind by a member since removed, holds nothing + /// a projection could carry, so it does not stop a caller naming the member above it. + /// + [Fact] + public void A_rule_on_a_path_that_does_not_exist_beneath_a_member_changes_nothing() + { + RvLink[] rows = { new() { Id = 1, Head = new RvNode { Name = "n1" } } }; + FakePolicyProvider stale = new FakePolicyProvider().Add( + "Head.Fax", PolicyFeature.Select, PolicyEffect.Deny, PolicyLevel.DynamicGlobal); + + RvLink row = Guard(rows.AsQueryable(), DwTier.Strict, stale).ToList(Selecting("Id", "Head")).Data.Single(); + + Assert.Equal("n1", row.Head!.Name); + } + /// Holds a reference to a copy of the result and its SQL, to keep a test to one statement per line. private sealed class FilterResultOf where T : class diff --git a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs index 06f0532..029f697 100644 --- a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs +++ b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs @@ -1734,6 +1734,16 @@ void Keep(string path) if (!rows.Narrows(field.Split(SegmentSeparator)[0])) { + // Narrowed only to keep a transform off a property it could not write: nothing is withheld + // by keeping it whole, as it always was kept. + if (cause is null && !hidden) + { + Keep(field); + KeepCarriedKeys(field, gate, Keep); + + continue; + } + gate.RefuseNarrowing(blame.Path, blame.Policy, $"'{field}' cannot be narrowed: {rows.WhyNotNarrowed(field)}"); continue; @@ -2808,9 +2818,12 @@ private static void Descend( foreach (PolicyFragment fragment in Fragments) { + // A rule on a path the type does not have, one written for a member since removed, holds + // nothing a projection could carry. if (fragment.IsWildcard || !fragment.FieldPath.StartsWith(prefix, StringComparison.OrdinalIgnoreCase) - || !asked.Add(fragment.FieldPath)) + || !asked.Add(fragment.FieldPath) + || Property(_entityType, fragment.FieldPath) is null) { continue; } From b71b8c4feec727b59649095b40c91bebf7484dd3 Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 05:33:23 +0300 Subject: [PATCH 08/22] fix(policies): an include anywhere in the query, and proof for lazy-loading proxies From my own pass. Include paths were read only from the query's own chain. An Include on a join's inner source loads a navigation of the rows the join returns, and EF Core applies it, so a denial beneath that navigation asked for no projection and came back. Every Include and ThenInclude in the tree is counted now; one off the chain the paths are read from means every navigation counts as loaded. Lazy-loading proxies were covered by reasoning only: they give each entity type an ILazyLoader service property, which is what the check reads. A test with UseLazyLoadingProxies proves it, and the test project references Microsoft.EntityFrameworkCore.Proxies at the leg's EF Core version. Turning either check off turns a test red. EF Core 8 leg: 2134 passed. EF Core 6.0.22 leg: 1438 passed. Co-Authored-By: Claude Opus 5 --- DynamicWhere.Tests/DynamicWhere.Tests.csproj | 3 + .../Policies/ProjectionReviewTests.cs | 74 +++++++++++++++++++ DynamicWhere.ex/Policies/Source/RowShape.cs | 33 ++++++++- 3 files changed, 106 insertions(+), 4 deletions(-) diff --git a/DynamicWhere.Tests/DynamicWhere.Tests.csproj b/DynamicWhere.Tests/DynamicWhere.Tests.csproj index dd49667..48e78ca 100644 --- a/DynamicWhere.Tests/DynamicWhere.Tests.csproj +++ b/DynamicWhere.Tests/DynamicWhere.Tests.csproj @@ -27,6 +27,9 @@ + + diff --git a/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs b/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs index 0dca9ab..d633612 100644 --- a/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs +++ b/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs @@ -808,6 +808,47 @@ public void A_rule_on_a_path_that_does_not_exist_beneath_a_member_changes_nothin Assert.Equal("n1", row.Head!.Name); } + /// + /// A navigation EF Core's lazy-loading proxies can fill counts as loaded as well, and the rows the + /// projection returns are plain objects, with no proxy left to load it. + /// + [Fact] + public void A_navigation_a_lazy_loading_proxy_can_fill_needs_the_projection() + { + using SqliteConnection connection = new("DataSource=:memory:"); + connection.Open(); + + using ProxyContext proxies = new(connection); + + proxies.Database.EnsureCreated(); + proxies.Blogs.Add(new PxBlog { Name = "X1", Posts = { new PxPost { Title = "T", Draft = "proxy-draft" } } }); + proxies.SaveChanges(); + proxies.ChangeTracker.Clear(); + + PolicyQueryable guarded = Guard(proxies.Blogs); + PxBlog blog = guarded.ToList(new Filter()).Data.Single(); + + Assert.Equal("X1", blog.Name); + Assert.Empty(blog.Posts); + Assert.Equal(typeof(PxBlog), blog.GetType()); + Assert.Contains(guarded.LastTrace!.Decisions, decision => decision.FieldPath == "Posts.Draft"); + } + + /// + /// An include on a join's inner source loads a navigation of the rows the join returns, and it is + /// not on the chain the paths are read from, so every navigation counts as loaded. + /// + [Fact] + public void An_include_anywhere_in_the_query_counts() + { + IQueryable rows = _db.Assets.Join(_db.Roles.Include(r => r.Keeper), a => a.Id, r => r.Id, (a, r) => r); + + PolicyQueryable guarded = Guard(rows); + + Assert.Null(guarded.ToList(new Filter()).Data.Single().Keeper); + Assert.Contains(guarded.LastTrace!.Decisions, decision => decision.FieldPath == "Keeper.Ssn"); + } + /// Holds a reference to a copy of the result and its SQL, to keep a test to one statement per line. private sealed class FilterResultOf where T : class @@ -843,6 +884,39 @@ public sealed class RvFamilyRow public List Kids { get; set; } = new(); } + public class PxBlog + { + public virtual int Id { get; set; } + + public virtual string Name { get; set; } = string.Empty; + + public virtual List Posts { get; set; } = new(); + } + + public class PxPost + { + public virtual int Id { get; set; } + + public virtual int PxBlogId { get; set; } + + public virtual string Title { get; set; } = string.Empty; + + [DwDenied] + public virtual string? Draft { get; set; } + } + + public sealed class ProxyContext : DbContext + { + private readonly SqliteConnection _connection; + + public ProxyContext(SqliteConnection connection) => _connection = connection; + + public DbSet Blogs => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => + options.UseSqlite(_connection).UseLazyLoadingProxies(); + } + public sealed class RvNode { public string Name { get; set; } = string.Empty; diff --git a/DynamicWhere.ex/Policies/Source/RowShape.cs b/DynamicWhere.ex/Policies/Source/RowShape.cs index b471fcb..a4c14b0 100644 --- a/DynamicWhere.ex/Policies/Source/RowShape.cs +++ b/DynamicWhere.ex/Policies/Source/RowShape.cs @@ -355,15 +355,19 @@ private static bool ReadIncludes(Expression expression, HashSet includes for (Expression? node = expression; node is MethodCallExpression call; node = call.Arguments.Count > 0 ? call.Arguments[0] : null) { - if (call.Method.DeclaringType == typeof(EntityFrameworkQueryableExtensions) - && call.Method.Name is nameof(EntityFrameworkQueryableExtensions.Include) - or nameof(EntityFrameworkQueryableExtensions.ThenInclude)) + if (IsInclude(call)) { calls.Add(call); } } - bool readable = true; + // An include somewhere else in the tree, in the other branch of a Concat or a Union, loads a + // navigation of rows this chain returns, and nothing here can say which. + IncludeCounter counter = new(); + + counter.Visit(expression); + + bool readable = counter.Count == calls.Count; string? current = null; for (int i = calls.Count - 1; i >= 0; i--) @@ -406,6 +410,27 @@ or nameof(EntityFrameworkQueryableExtensions.ThenInclude)) return readable; } + private static bool IsInclude(MethodCallExpression call) => + call.Method.DeclaringType == typeof(EntityFrameworkQueryableExtensions) + && call.Method.Name is nameof(EntityFrameworkQueryableExtensions.Include) + or nameof(EntityFrameworkQueryableExtensions.ThenInclude); + + /// Counts every Include and ThenInclude anywhere in a query. + private sealed class IncludeCounter : ExpressionVisitor + { + internal int Count { get; private set; } + + protected override Expression VisitMethodCall(MethodCallExpression node) + { + if (IsInclude(node)) + { + Count++; + } + + return base.VisitMethodCall(node); + } + } + /// /// The member path a navigation lambda reads, o => o.Lines or o => o.Customer.Address, /// through casts and a filtered include's operators; null for any other shape. From 4949dc54be4c9e3771f28c0161bdd733ea4a433e Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 05:34:06 +0300 Subject: [PATCH 09/22] docs: an include off the query's own chain counts as loading every navigation Co-Authored-By: Claude Opus 5 --- OfficialWebsite/public/llms.txt | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/OfficialWebsite/public/llms.txt b/OfficialWebsite/public/llms.txt index ea63545..dfe0299 100644 --- a/OfficialWebsite/public/llms.txt +++ b/OfficialWebsite/public/llms.txt @@ -2331,8 +2331,9 @@ DefaultOrder field denied for Segment, in a Segment (3.1.0) left out l JSON column (3.2.0; 3.1.0 asked only about simple members, so a denied `byte[]` or owned member came back). - A denial beneath a member counts when its value can reach the result: - on an entity, beneath a column, an owned or complex member, or a navigation something loads: an `Include` or - `ThenInclude` on the query (a form the library cannot read counts as loading every navigation), an automatic - include, or a lazy loader (proxies, or an injected `ILazyLoader`, whose navigations fill after the query). A + `ThenInclude` on the query (a form the library cannot read, or an include off the query's own chain such as on + a join's inner source, counts as loading every navigation), an automatic include, or a lazy loader (proxies, + or an injected `ILazyLoader`, whose navigations fill after the query). A denial beneath a navigation nothing loads never leaves the database and needs no projection, so a connected model is read as it was in 3.1.0; - on a row a projection builds, beneath a member its initializer assigns; From f7e1cc6020dff4780fed57846ac5fb8411c8b388 Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 05:43:02 +0300 Subject: [PATCH 10/22] docs: 3.2.0 on the site, in README and in DOC.md The four async method pages carry the CancellationToken overloads, the token reaching the count and the read, the default-literal ambiguity, and the dynamic and summary reads going through EF Core. The extensions index counts 28 methods; README and the landing page counted 17 and 21. The configuration page says when a request with no Selects needs a projection, what each source keeps, when a member is kept whole, narrowed or left out whole, and what a caller naming a member is refused. The security page lists the denials the gate could not see and the limits that remain. The breaking-changes page adds points 25 to 29. The attributes page gives the default-order rule for projections, and the admin page says how /simulate reads a type. README gains the 3.2.0 highlights; DOC.md carries the same sections as the site. llms.txt's table row and simulator limit now say what section 17 says. Co-Authored-By: Claude Opus 5 --- DynamicWhere.ex/DOC.md | 172 ++++++++- OfficialWebsite/app/docs/ai/page.tsx | 2 +- .../app/docs/breaking-changes/page.tsx | 335 +++++++++++++++++- OfficialWebsite/app/docs/errors/page.tsx | 6 +- OfficialWebsite/app/docs/extensions/page.tsx | 77 +++- .../to-list-async-dynamic-filter/page.tsx | 75 +++- .../extensions/to-list-async-filter/page.tsx | 72 +++- .../extensions/to-list-async-segment/page.tsx | 28 +- .../extensions/to-list-async-summary/page.tsx | 83 ++++- .../app/docs/policies/admin/page.tsx | 14 + .../app/docs/policies/attributes/page.tsx | 66 +++- .../app/docs/policies/configuration/page.tsx | 295 ++++++++++++++- OfficialWebsite/app/docs/policies/page.tsx | 39 ++ .../app/docs/policies/security/page.tsx | 105 +++++- OfficialWebsite/app/page.tsx | 4 +- OfficialWebsite/public/llms.txt | 5 +- README.md | 23 +- 17 files changed, 1339 insertions(+), 62 deletions(-) diff --git a/DynamicWhere.ex/DOC.md b/DynamicWhere.ex/DOC.md index 6c3c9a9..0177d43 100644 --- a/DynamicWhere.ex/DOC.md +++ b/DynamicWhere.ex/DOC.md @@ -445,7 +445,7 @@ Inherits all properties from `FilterResult`. Returned by segment operations. ## Extension Methods Reference -All extension methods live in `DynamicWhere.ex.Source.Extension` and operate on `IQueryable` (or `IEnumerable` for in-memory variants). +All extension methods live in `DynamicWhere.ex.Source.Extension` and operate on `IQueryable` (or `IEnumerable` for in-memory variants). There are 28 of them: the eighteen on `IQueryable` that 3.1 had, three in-memory variants on `IEnumerable`, and, since 3.2.0, seven more on `IQueryable` — overloads of the asynchronous terminals that take a `CancellationToken` (see [Cancellation](#cancellation)). ### `.Select(List fields)` @@ -632,6 +632,8 @@ In-memory variant — wraps the collection with `AsQueryable()` then delegates. Async version of `ToList(Filter)`. Uses `CountAsync()` and `ToListAsync()` for EF Core. +Since 3.2.0 two overloads take a `CancellationToken`, which reaches the count and the read: `.ToListAsync(Filter filter, CancellationToken cancellationToken)` and `.ToListAsync(Filter filter, bool getQueryString, CancellationToken cancellationToken)`. See [Cancellation](#cancellation). + **Returns:** `Task>` --- @@ -655,7 +657,9 @@ In-memory variant — wraps the collection with `AsQueryable()` then delegates t ### `.ToListAsyncDynamic(Filter filter, bool getQueryString = false)` -Async version of `ToListDynamic(Filter)`. Uses `CountAsync()` and `ToDynamicListAsync()` for EF Core. +Async version of `ToListDynamic(Filter)`. Counts with EF Core's `CountAsync()` and, since 3.2.0, reads with EF Core's own `ToListAsync()`, called for the query's element type: the class the projection generates, or `T` when `Selects` is null. It used to read with Dynamic LINQ's `ToDynamicListAsync()`, which reads synchronously on a thread-pool thread; the database command now runs asynchronously and a canceled token reaches it. The rows are the same. + +Since 3.2.0 two overloads take a `CancellationToken`, which reaches the count and the read: `.ToListAsyncDynamic(Filter filter, CancellationToken cancellationToken)` and `.ToListAsyncDynamic(Filter filter, bool getQueryString, CancellationToken cancellationToken)`. See [Cancellation](#cancellation). **Returns:** `Task>` @@ -687,7 +691,9 @@ In-memory variant for summary operations. ### `.ToListAsync(Summary summary, bool getQueryString = false)` -Async version of `ToList(Summary)`. +Async version of `ToList(Summary)`. On an EF Core query it counts the groups with EF Core's `CountAsync()` and reads them with EF Core's `ToListAsync()`. Until 3.2.0 the count ran synchronously and the read went through Dynamic LINQ's `ToDynamicListAsync()`, a synchronous read on a thread-pool thread; on an EF Core query a canceled token now reaches the database. The count and the rows are the same. A source whose provider is not EF Core's, such as rows in memory through `AsQueryable()`, keeps the synchronous count and read. + +Since 3.2.0 two overloads take a `CancellationToken`, which reaches the count and the read: `.ToListAsync(Summary summary, CancellationToken cancellationToken)` and `.ToListAsync(Summary summary, bool getQueryString, CancellationToken cancellationToken)`. See [Cancellation](#cancellation). **Returns:** `Task` @@ -697,10 +703,41 @@ Async version of `ToList(Summary)`. Async-only segment operation. Combines every `ConditionSet` with set operations (`Union` / `Intersect` / `Except`) into one query, then orders, pages, projects and counts it in the database exactly as `ToListAsync(Filter)` does. Only the requested page is read, and `Orders` apply before `Selects`. `Union` and `Intersect` combine the sets' conditions and `Except` matches rows by primary key, never by object reference, so on a type with a primary key, tracking, `AsNoTracking()` and `Selects` return the same rows. Ordering follows the database: text sorts by its collation. +Since 3.2.0 an overload takes a `CancellationToken`, `.ToListAsync(Segment segment, CancellationToken cancellationToken)`, and passes it to the count and the read. See [Cancellation](#cancellation). + **Returns:** `Task>` --- +### Cancellation + +*New in 3.2.0.* Every asynchronous terminal has overloads that take a `CancellationToken`, guarded and unguarded: + +| Terminal | Overloads with a token | +|---|---| +| `ToListAsync` with a `Filter` | `(Filter filter, CancellationToken cancellationToken)` · `(Filter filter, bool getQueryString, CancellationToken cancellationToken)` | +| `ToListAsyncDynamic` with a `Filter` | `(Filter filter, CancellationToken cancellationToken)` · `(Filter filter, bool getQueryString, CancellationToken cancellationToken)` | +| `ToListAsync` with a `Summary` | `(Summary summary, CancellationToken cancellationToken)` · `(Summary summary, bool getQueryString, CancellationToken cancellationToken)` | +| `ToListAsync` with a `Segment` | `(Segment segment, CancellationToken cancellationToken)` | + +- The token reaches the count and the read. On an EF Core query a canceled token stops whichever of the two is running, and the call throws `OperationCanceledException`. EF Core's `TaskCanceledException` derives from it. `ToListAsync(Summary)` on a provider that is not EF Core's, such as rows in memory, checks the token before its synchronous count and read. +- The overloads without a token pass `CancellationToken.None`. +- They are overloads, not an optional parameter added to the old signatures. The 3.1 signatures are unchanged, so code compiled against 3.1 still binds. A reflection lookup of `ToListAsyncDynamic` by name alone now finds three methods where it found one, on the extension class and on the guarded handle, so `Type.GetMethod` given only the name throws `AmbiguousMatchException`; pass the parameter types. +- `ToListAsync(filter, default)` does not compile: `default` fits both `bool getQueryString` and `CancellationToken`, so the call is ambiguous (CS0121). So are `ToListAsyncDynamic(filter, default)` and `ToListAsync(summary, default)`, on a query and on the guarded handle alike. Write `false`, a token, or a named argument. A `Segment` takes no `getQueryString`, so `ToListAsync(segment, default)` binds the token overload. +- The guarded handle, `PolicyQueryable`, has the same seven overloads (see [The shape](#the-shape)). + +```csharp +// A minimal API binds a CancellationToken parameter to HttpContext.RequestAborted, +// so a client that disconnects cancels the count or the read. +app.MapPost("/customers/search", async (Filter filter, AppDbContext db, CancellationToken cancellationToken) => +{ + var result = await db.Customers.ToListAsync(filter, cancellationToken); + return Results.Ok(result); +}); +``` + +--- + ## Validation Rules ### Condition Validation Rules @@ -1502,6 +1539,12 @@ var result = await db.Employees.ApplyPolicy(caller).ToListAsync(filter); A context carries the snapshot it was served, and the staleness ceiling measures how old that snapshot is — which is why it is built once per request rather than reused. Since 3.1.0 an unprepared context is **refused by `ApplyPolicy` itself**, with `PolicyContextNotPrepared`, whether or not a store is configured: before that only a store provider refused one, so an attributes-only deployment accepted the missing call and would have started refusing the day it gained a store. `DwPolicyContext.IsPrepared` reports it. The overload taking explicit options and a resolver does not check — that host composes its own configuration and owns preparation. +`ApplyPolicy` returns a `PolicyQueryable`, whose terminals mirror the core's: `ToList`, `ToListAsync`, `ToListDynamic` and `ToListAsyncDynamic` with a `Filter`, `ToList` and `ToListAsync` with a `Summary`, and `ToListAsync` with a `Segment`. Since 3.2.0 every asynchronous one also has the overloads that take a `CancellationToken` — `ToListAsync(Filter, CancellationToken)`, `ToListAsync(Filter, bool, CancellationToken)`, the same two for `ToListAsyncDynamic` and for `ToListAsync` with a `Summary`, and `ToListAsync(Segment, CancellationToken)`. The policy is applied first, so a refusal is thrown whatever the token says; the token then reaches the count and the read. See [Cancellation](#cancellation). + +```csharp +var result = await db.Employees.ApplyPolicy(caller).ToListAsync(filter, cancellationToken); +``` + ### Attribute reference | Attribute | Applies to | Effect | @@ -1580,14 +1623,15 @@ It applies only under `ApplyPolicy`, when the caller sends no orders (`Orders` n - `ToList`, `ToListAsync`, `ToListDynamic` and `ToListAsyncDynamic` with a `Filter`; - `ToListAsync` with a `Segment`; - the composable `Filter` and `FilterDynamic`; -- the composable `Page`, on a source nothing has ordered or projected. +- the composable `Page`, on a source nothing has ordered and whose projection hides no field the default names. It is never applied: - by an unguarded call. The core extension methods on a plain `IQueryable` or `IEnumerable`, and a query taken out through `AsUnguardedQueryable()`, ignore the attribute and behave exactly as in 3.0; - when the caller sends orders — the default is not appended to them as a tiebreak; -- to an `IQueryable` that is already ordered, whether before it was guarded (`db.Tickets.OrderBy(t => t.Title).ApplyPolicy(caller)`) or by a composed `Order` earlier in the chain — even one whose every order the policy dropped, because the caller still sent orders. An in-memory sequence sorted with LINQ to Objects before `ApplyPolicy` is not seen as ordered, because `AsQueryable()` hides the sort, so the default replaces that order; -- to a projected query. A `Select` anywhere in the chain — the guarded composable `Select`, or a projection made before `ApplyPolicy` — leaves the query in its own order, because a default applied after a projection can name a field the projection left out, which EF Core cannot translate, so `guarded.Select(["Id", "Title"]).Page(page)` on a type ordered by `Priority` pages unordered, as in 3.0.0; +- to an `IQueryable` that is already ordered, whether before it was guarded (`db.Tickets.OrderBy(t => t.Title).ApplyPolicy(caller)`) or by a composed `Order` earlier in the chain — even one whose every order the policy dropped, because the caller still sent orders. Since 3.2.0 a `Filter` composed on the handle that sent orders counts the same way. An in-memory sequence sorted with LINQ to Objects before `ApplyPolicy` is not seen as ordered, because `AsQueryable()` hides the sort, so the default replaces that order; +- to a source whose projection could hide a field the default names. Only the outermost `Select` of the chain counts, because it makes the rows the default orders. Since 3.2.0 it hides nothing when it builds `T` itself in an object initializer, `Select(t => new TicketRow { CreatedAt = t.CreatedAt, Id = t.Id, … })`, and assigns every field the default names, at every level of a nested path (`"Owner.Name"` needs `Owner = new OwnerRow { Name = … }`), with nothing EF Core would compute on the client; the default then applies, because EF Core translates an order through a member the projection assigned. A constructor with arguments, a default field the initializer does not assign, a nested path through anything but an initializer, or a default field computed by the application's own method (`Label = Decorate(r.Code)`, which EF Core evaluates on the client, where it can project the value but cannot order by it) leaves the query in its own order, as every projection did in 3.1.0: a default applied there could name a field EF Core cannot translate; +- after a projection composed on the handle. The guarded `Select`, or a guarded `Filter` whose `Selects` is set, leaves the rest of the chain unordered even when it keeps every default field, so `guarded.Select(["Id", "Title"]).Page(page)` on a type ordered by `Priority` pages unordered, as in 3.0.0. The default is for the rows the caller's source makes; - to a `Summary`, or by the composable `Where`, `Select`, `Order` or `Group`. Nothing is ordered that the type's own `[DwEntity]` did not declare, and a default is never a reason for the library to refuse a query. An entry naming a field the type does not have, one that is not a field optionally followed by a direction, one the core refuses to order by — a path ending on a collection of entities, such as `Tags` — or one whose name the expression parser keeps for itself, such as `Null`, is skipped. A path through a collection to a value, such as `Tags.Value`, is kept and sorted by its smallest value ascending or its largest descending. A field this caller may not order by is left out, in either tier, and never refused: the caller did not send it, and ordering by it would rank rows by a value they may not see. In a `Segment`, a field this caller may not use in a segment is left out as well, since a segment refuses it in any clause; a filter still orders by it. The trace records a `Dropped` decision for `Order` whose reason starts `left out of the default order`; a dry run keeps the field and still records the decision. A caller whose own orders were all dropped under `Convenience` sent orders, and gets no default in their place. The startup check reports every entry a query would skip or leave out. @@ -1624,6 +1668,89 @@ A dropped field leaves nothing behind in the data, so the trace is the only way The convenience tier is unchanged: an unknown name fails validation with `LogicException` `ConditionMustHasValidFieldName`, a refusal names the field as the caller wrote it, with `RuleId` and `SourceOrigin` where a single source decided, and `MaxQueryCost` is checked before any field is gated. A dry run refuses no field, so an unknown name fails validation there in either tier. +### A navigation named in Selects + +A `Selects` entry can name a navigation, such as `"Lines"`, rather than the fields beneath it. With nothing denied beneath it, the entry is kept as written. With a denied field beneath it, the `Convenience` tier replaces the entry with the allowed fields beneath it, and the `Strict` tier refuses it with `FieldDeniedForSelect`. + +| `Selects` names a navigation | `Convenience` | `Strict` | +|---|---|---| +| with nothing denied beneath it | Kept whole | Kept whole | +| with a denied field beneath it | Replaced by the allowed fields beneath it | `FieldDeniedForSelect` | +| whose key, `Lines.Id`, is denied (3.2.0) | `FieldDeniedForSelect` | `FieldDeniedForSelect` | +| with a denied field beneath it, where the narrowing cannot be built (3.2.0) | `FieldDeniedForSelect` | `FieldDeniedForSelect` | +| whose type holds a field denied for `Select` that no path names (3.2.0) | Narrowed away, where it can be narrowed | `FieldDeniedForSelect` | + +- The fields beneath a member are read the way the attribute walker reads them: through any collection type, and no deeper than its four segments. Since 3.2.0 a member typed `IReadOnlyList`, `IReadOnlyCollection`, `Collection` or an application's own collection no longer hides the denials beneath it. The providers' own rules are asked too, so a denied property with no setter and a rule on a path reached through a cycle count. +- A narrowing that cannot be built as it was gated is refused in both tiers (3.2.0). The core's typed projection adds the key, `Id`, of every nested node it builds, so a navigation narrowed around its own denied key would get the key back. The core reads a path only through an array, `List`, `IList`, `ICollection`, `IEnumerable`, `HashSet` or `ISet`, so a narrowing through any other collection fails its validation. And some members cannot be narrowed at all: a column, a complex property or a member stored as JSON, which EF Core reads whole; a member of a row in memory; and a member a projection builds some way the core cannot narrow. +- A member that carries everything its type holds — a member of a projected or in-memory row, or an entity's column, owned or complex member — can hold a field denied for `Select` that no path names: one deeper than four segments, or inside a framework generic such as `Dictionary`. The `Strict` tier refuses such a member, and the `Convenience` tier narrows it away (3.2.0); where it cannot be narrowed, as a member of a row in memory cannot, both tiers refuse it. An entity's navigation loads only its own entity, so only its paths are asked about. +- A navigation named through another, `Main.Lead`, gates the key of every node it passes through, which the core's projection adds, as a dotted path to a value always did (3.2.0). A denied key refuses the projection. Naming a field beside a denied key, `Lines.Name` when `Lines.Id` is denied, was already refused in both tiers. +- Under `Convenience` the refusal names the denied key, the first denied field beneath the member, or, for a denial no path names, the member itself. Under `Strict` its `FieldPath` is `"*"`, as on every field refusal. The trace records the reason either way. + +### A request that sends no Selects + +A request that sends no `Selects` returns whole rows, denied fields included, because the core projects only when `Selects` is set. So when a field denied for `Select` could reach the result, a guarded query synthesizes the projection itself. It does so in both tiers, typed and dynamic, for a whole `Filter` and for a `Segment`. A clause composed on its own, such as `Where`, `Order` or `Page`, synthesizes nothing. + +The projection keeps the **allowed members**: what an unguarded call would return, less what the policy withholds. Before 3.2.0 it kept the allowed scalars only, and only a simple field denied at the top of the type asked for it (breaking point 20). + +**When a projection is needed.** + +- A field denied at the top of `T` always asks for one, whatever it holds: a scalar, a blob, a list, an owned object or a JSON column. +- A field denied beneath a member asks for one when its value can reach the result. On an entity, that is beneath a column, an owned or complex member, or a navigation something loads: an `Include` or `ThenInclude` on the query, an automatic include, or a lazy loader — EF Core's proxies, or an injected `ILazyLoader` — which fills a navigation after the query. An `Include` written in a form the library cannot read counts as loading every navigation. On a row a projection builds, it is beneath a member the initializer assigns. On a row in memory, it is beneath any member. +- A denial beneath a navigation nothing loads never leaves the database, so it asks for no projection. An entity whose only denials sit beneath such navigations is read as it was in 3.1.0. +- A member whose type can hold a field denied for `Select` that no path names — deeper than the walker's four segments, or inside a framework generic such as `Dictionary` — asks for one too, and so does a member typed `object`, when what it holds can reach the result. +- The denials beneath a member come from the providers' rules as well as from walking the type, so a denied property with no setter, a rule on a path reached through a cycle, and a rule deeper than the walk all count. +- A forced scope beneath a member asks for no projection on its own. It filters the rows that hold the member, as it always has. When a projection is needed anyway, the member is left out whole. + +**What it keeps.** A member holding a value — a simple type, or a collection of one such as `byte[]`, `string[]` or `List` — is kept when it is allowed and the source carries it. A member holding an object, or a list of them, is kept whole, narrowed or left out whole, as below, and only where the source carries it: + +| Source | Values kept | Objects kept | +|---|---|---| +| A projection that builds its rows before `ApplyPolicy`: the outermost `Select` constructs the row, in an object initializer or with a constructor, as in `db.Roles.Select(r => new RoleRow { … })` | Every member the initializer assigns; every member when a constructor builds the row | The same | +| An entity query, or a `Select` that hands back an entity, as in `db.Orders.Select(o => o.Customer)` | Every member EF Core maps | Its columns, converted and JSON ones included, its owned members and, on EF Core 8 or later, its complex properties, read from the EF Core model | +| Rows in memory, as in `roles.ApplyPolicy(caller)` | Every member | None | + +A value EF Core does not map is left out: computing it would make EF Core read the whole entity, the denied columns included, and it holds only its initial value anyway. A source the library cannot read — no EF Core model, and neither a projection it can see into nor rows in memory — keeps values only, as in 3.1.0, and every denial beneath a member counts. + +**Whole, narrowed or left out whole.** A member holding an object that the source carries is: + +- **kept whole** when nothing beneath it is denied, nothing in its type is denied or typed `object`, no forced scope is beneath it, and no transform beneath it lands on a property with no setter; +- **narrowed** otherwise, to the allowed fields beneath it, four segments deep, as a caller naming it would get it, where the core's narrowing translates: an object the projection's initializer builds, a list a subquery reads into a type the core can bind (not an array or a set), a navigation that is neither complex nor stored as JSON, or an entity's owned member not stored as JSON. A field beneath it that can hold what the policy cannot name is left out; +- **left out whole** otherwise, and recorded as `Dropped` on `Select` with a reason that starts `left out whole`. + +```text +left out whole: a scope forced beneath it cannot be applied to what it holds +left out whole: it is a column, which EF Core reads whole +left out whole: it is a complex property, which EF Core cannot narrow +left out whole: it is stored as JSON, which EF Core cannot narrow +left out whole: the projection builds it in a way the core cannot narrow +left out whole: the projection would add its key 'Contents.Id', which is denied +left out whole: the core cannot project 'Contents.Code' +left out whole: nothing beneath it may be selected +left out whole: it can hold what the policy cannot name +``` + +The last one is recorded on the field beneath the member that the narrowing leaves out. + +**Never kept.** + +- An entity's navigation, included or not: projecting it would load it. So once a denial needs a projection, an included or automatically included navigation is not returned. Name it in `Selects` to get it, narrowed. +- An object held by a row in memory: a kept object is the caller's own, and a transform beneath it would change it in place. The projection's rows are new and hold no member of an object type, so the source objects are left as they were. +- A member with no setter, and a member named with one of the expression parser's own words. + +**Other rules.** + +- A narrowed reference that is null in the source comes back as an empty object, as it does for a caller's own dotted `Selects`. +- A narrowed member carries every allowed field beneath it. An entity reached beneath it therefore has its own navigations projected, and so loaded, whether or not the source included them, exactly as when `Selects` names the member. +- Each denied field whose value can reach the result, at the top or beneath, is recorded as `Dropped` on `Select`. +- It never throws for a denied field, in either tier. It throws `AllSelectsDenied` only when no field is left. When nothing asks for a projection, `Selects` stays null and the query is the one an unguarded call runs. +- A typed query projects into `T`, so `T` needs a public parameterless constructor, or the query fails with `SelectTypeMustHaveParameterlessConstructor` (breaking point 1). The dynamic terminals do not need one. +- A dry run synthesizes nothing. It records the denials and returns the rows whole. +- A simulation has no source, so it reads `T` as a source it cannot see into: every denial beneath a member counts, and the projection it shows keeps only members holding a value (see [Administration](#administration)). + +**What the policy cannot see into.** A member typed `object` is opaque to the policy: a synthesized projection leaves it out, and naming it returns whatever it holds. A framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: naming it is refused under `Strict` and narrowed away under `Convenience`, or refused there too where the member cannot be narrowed, and a synthesized projection leaves it out. Hold such values in a list of the policed type instead. + +**A forced scope on a list's element type filters rows, not elements.** A forced scope declared on a list's element type filters the rows that hold the list, never its elements. `Selects` naming the list returns every element, those the scope excludes included, as in every release; a synthesized projection leaves such a list out. Scope the elements where the row is built. + ### Results and the trace Every guarded query records a `PolicyTrace`: its tier, whether it ran dry, and a `PolicyDecision` — `FieldPath`, `Feature`, `Action`, `Reason` — for each thing the policy decided. `PolicyQueryable.LastTrace` holds it for the most recent call on the handle, the composable methods included. @@ -1802,6 +1929,8 @@ Options are frozen at startup. Every cap refuses a value below one, except two t | `POST` | `/simulate` | The sanitized clause, without executing or auditing | | `GET` | `/health` | Snapshot version, age, degraded state, last error | +A simulation, through `/simulate` or `PolicySimulator`, has no source, so it reads the type as a source it cannot see into. That shows in a clause that sends no `Selects`: every denial beneath a member counts, and the projection it shows keeps only the members that hold a value, a collection of values included. A guarded query keeps what its own source carries — over a projected row, the objects its initializer assigns; over an entity, its columns, owned and complex members, asking only about the denials whose value it loads; over rows in memory, values only. So the simulated clause can list fewer members than the query returns, and can show a projection an entity query does not need. See [A request that sends no Selects](#a-request-that-sends-no-selects). + ### Schema discovery `POST /dw-policies/schema` describes what one caller may do with one entity. It is a POST rather @@ -2103,7 +2232,7 @@ All validation errors throw `LogicException` (inherits `Exception`) with one of ### ⚠️ Breaking Points 1. **Parameterless Constructor Required for Select Projection** - `Select(fields)` requires `T` to have a parameterless (default) constructor. If `T` does not have one — a positional record, most often — a `LogicException` is thrown whose `Message` is the stable code `SelectTypeMustHaveParameterlessConstructor` and whose `Subject` carries `typeof(T).Name`. Before 3.1.0 that message was an English sentence with the type name inside it. Most EF Core entity classes have parameterless constructors by default. A guarded query reaches the same refusal when a member carries `[DwNoSelect]`, because deny-select projects. + `Select(fields)` requires `T` to have a parameterless (default) constructor. If `T` does not have one — a positional record, most often — a `LogicException` is thrown whose `Message` is the stable code `SelectTypeMustHaveParameterlessConstructor` and whose `Subject` carries `typeof(T).Name`. Before 3.1.0 that message was an English sentence with the type name inside it. Most EF Core entity classes have parameterless constructors by default. A guarded query reaches the same refusal when a member carries `[DwNoSelect]`, because deny-select projects — since 3.2.0 whatever the member holds, and beneath another member when its value can reach the result (point 20). 2. **Segment Operations are Async-Only** `ToListAsync(Segment)` is the only entry point for segment queries. There is no synchronous `ToList(Segment)` variant. The condition sets are combined into one query that the database orders and pages. `Union` and `Intersect` combine the sets' conditions; only `Except` on a type with a primary key needs a provider that translates a correlated `EXISTS`. Under `ApplyPolicy`, `DwCaps.MaxConditionSets` (default 10) bounds how many sets one request may carry. @@ -2177,6 +2306,35 @@ All validation errors throw `LogicException` (inherits `Exception`) with one of 19. **`MaxConditionValues` and `MaxAggregates` Refuse Guarded Requests 3.0 Ran** Two more caps new in 3.1.0. `DwCaps.MaxConditionValues` (default 1000) bounds the values one condition carries — the largest condition of the where clause, a summary's `Having` and every segment set is the one compared — because an `In` is one comparison per value and so could build a predicate of any size for the price of one condition and one field. `DwCaps.MaxAggregates` (default 50) bounds the `AggregateBy` entries of one summary, through the summary terminals and the composable `Group` and `Summary`; the group-size floor's own count is not counted. A guarded request over either is refused in both tiers with `PolicyException` `CapExceeded`, `FieldPath` `"*"` and `SourceOrigin` `"MaxConditionValues cap (1000), request had 1001"` or `"MaxAggregates cap (50), request had 51"`, unless the deployment raises the cap. Both refuse a value below 1, freeze with the posture, and bind from `Caps:MaxConditionValues` and `Caps:MaxAggregates`. An aggregate with no field, such as a `Count`, is now charged `DefaultFieldCost` toward `MaxQueryCost`, where it cost nothing, so a summary that sat just under its budget can be refused with `QueryCostExceeded`. Every count cap is now checked before any field name is resolved, so an oversized request that also names a field that does not exist is refused with `CapExceeded`, where 3.0.0 resolved names first and answered `ConditionMustHasValidFieldName`. Only `ApplyPolicy` enforces them: an unguarded query is not affected. +20. **A Guarded Query That Sends No `Selects` Keeps What the Source Carries** + A request with no `Selects` returns whole rows, denied fields included, so a guarded query synthesizes a projection when a denied field could reach the result. 3.2.0 changed when it does so and what it keeps; the rules are under [A request that sends no Selects](#a-request-that-sends-no-selects). + + Fixed (security): a field denied only beneath a member, none at the top of `T`, synthesized nothing, so the whole row came back with the denied value in it — in a list or nested object of a row projected before `ApplyPolicy`, in a row held in memory, and in an entity's included, automatically included, lazily loaded or owned member — typed and dynamic, in both tiers, for a `Filter` and a `Segment`. Such a denial now synthesizes the projection whenever its value can reach the result: on an entity, beneath a column, an owned or complex member, or a navigation the query loads through `Include`, an automatic include or a lazy loader; on a projected row, beneath a member the initializer assigns; in memory, beneath any member. A denial beneath a navigation nothing loads never leaves the database, and the entity is read exactly as in 3.1.0. + + Fixed (security): a field denied at the top of `T` whose own type is not a simple value — a byte array, a list, an owned object, a JSON column — synthesized no projection either, so with nothing else denied the whole row came back with it. + + Changed: the projection kept simple fields only, so as soon as any field was denied, every nested object and list of a row projected before `ApplyPolicy` came back null or empty, and so did an entity's columns holding an object, its owned and complex members and its collections of simple values. A row a projection builds now keeps the members its initializer assigns; an entity keeps its mapped columns, converted and JSON ones included, its owned and complex members, and every collection of simple values; rows in memory keep their values. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, with a `Dropped` decision whose reason starts `left out whole`. An entity's navigations, included ones too, the objects of a row in memory, and a value EF Core does not map are left out; name a navigation in `Selects` to get it, narrowed. A forced scope beneath a member asks for no projection on its own, and a projection needed for another reason leaves such a member out whole. A typed query needs `T` to have a public parameterless constructor for the projection, as it already did (point 1). + +21. **`Selects` Naming a Member Is Gated Against Every Denial Beneath It** + When `Selects` names a navigation with a denied field beneath it, the `Convenience` tier replaces the entry with the allowed fields beneath it, and the `Strict` tier refuses it. Since 3.2.0 the gate finds every denial beneath the member, and refuses, with `FieldDeniedForSelect`, a narrowing it cannot build as gated. See [A navigation named in Selects](#a-navigation-named-in-selects). + + Fixed (security): under the convenience tier, a navigation whose key (`Id`) is denied was narrowed to the allowed fields beneath it, and the core's typed projection, which adds the key of every nested node it builds, put the key back. Such a narrowing is refused in both tiers, as naming a sibling of the key already was. A navigation named through another, such as `Main.Lead`, now gates the key of `Main`, which the projection adds; it did not. + + Fixed (security): a member typed as a collection the core does not unwrap — `IReadOnlyList`, `IReadOnlyCollection`, `Collection` or an application's own — returned every field beneath it, denied ones included, in both tiers. The projection gate read collections through a narrower list than the attribute walker and found nothing beneath the member. It now reads them as the walker does, and a narrowing the core cannot project is refused. + + Fixed (security): a member of a projected or in-memory row whose type holds a field denied for `Select` that no path names — deeper than four segments, or inside a framework generic such as `Dictionary` — was returned whole. The strict tier refuses it now, and the convenience tier narrows it away; where it cannot be narrowed, both tiers refuse it. Denials beneath a named member are also read from the policy's own rules, so a denied property with no setter and a rule on a path reached through a cycle are found. A member that cannot be narrowed at all — a column, a complex property or a JSON-stored member, a member of a row in memory, or one a projection builds some way the core cannot narrow — is refused in both tiers when something beneath it is denied. A request that sends no `Selects` is not refused for such a member: its synthesized projection narrows it or leaves it out whole. + +22. **`DefaultOrder` Reaches a Projection That Builds `T`** + In 3.1.0 a `Select` anywhere in the chain kept a guarded query in its own order. Since 3.2.0 only the outermost `Select` counts, and when it builds `T` in an object initializer that assigns every field the default names, at every level of a nested path, with nothing EF Core would compute on the client, the default applies: `db.Tickets.Select(t => new TicketRow { Id = t.Id, CreatedAt = t.CreatedAt, Title = t.Title }).ApplyPolicy(caller)` on a `TicketRow` declaring `"CreatedAt desc, Id"` was unordered and is now ordered. A constructor with arguments, a default field the initializer does not assign, a nested path through anything but an initializer, or a default field computed by the application's own method, which EF Core evaluates on the client and cannot order by, still leaves the query in its own order. A `Select`, or a `Filter` with `Selects`, composed on the guarded handle keeps the rest of the chain unordered, and a composed `Filter` that sent orders gets no default later in the chain, even when the policy dropped every one of them, as a composed `Order` already did not. See [Default order](#default-order). + +23. **Every Async Terminal Takes a `CancellationToken`** + Since 3.2.0 `ToListAsync` and `ToListAsyncDynamic` with a `Filter`, `ToListAsync` with a `Summary`, and `ToListAsync` with a `Segment` have overloads that take a `CancellationToken`, guarded and unguarded, and the token reaches the count and the read. The 3.1 signatures are unchanged, so code compiled against 3.1 still binds, but `ToListAsync(filter, default)`, `ToListAsyncDynamic(filter, default)` and `ToListAsync(summary, default)` no longer compile: `default` fits both `getQueryString` and the token (CS0121). Write `false`, a token, or a named argument. A reflection lookup of `ToListAsyncDynamic` by name alone now finds three methods where it found one. See [Cancellation](#cancellation). + + Changed: `ToListAsyncDynamic` and the async `Summary` read through EF Core's `ToListAsync`, and the async `Summary` counts through `CountAsync`. They used to read synchronously on a thread-pool thread, and the summary counted synchronously, so on an EF Core query a canceled token now reaches the database. The rows and the counts are the same. A provider that is not EF Core's keeps the synchronous read. + +24. **A Type in a Namespace That Starts with `System` Is Policed** + The attribute walker does not descend into the framework's own types, which carry no policy attributes. Until 3.2.0 it took any namespace whose name started with `System` for the framework's, so an application namespace such as `SystemsCorp.Payroll` or `SystemX.Domain` got no policy beneath its types, and a `[DwDenied]` field on such a type, reached through a member, was returned, filterable and sortable. Fixed (security): only `System` and the namespaces beneath it are the framework's now, so a guarded request that filtered on, sorted by or selected such a field is refused or dropped, as for any denied field. + --- ## License diff --git a/OfficialWebsite/app/docs/ai/page.tsx b/OfficialWebsite/app/docs/ai/page.tsx index e485a4b..61eba2c 100644 --- a/OfficialWebsite/app/docs/ai/page.tsx +++ b/OfficialWebsite/app/docs/ai/page.tsx @@ -84,7 +84,7 @@ DynamicWhere.ex code. It is the complete API surface.`} m spelling of Sumation.
  • - All twenty-one extension methods with their real + All twenty-eight extension methods with their real signatures, what each one validates, and which have no synchronous or in-memory form. Plus the generated predicate for every operator, value coercion per DataType, and how field paths resolve. diff --git a/OfficialWebsite/app/docs/breaking-changes/page.tsx b/OfficialWebsite/app/docs/breaking-changes/page.tsx index 743efb8..5eeb120 100644 --- a/OfficialWebsite/app/docs/breaking-changes/page.tsx +++ b/OfficialWebsite/app/docs/breaking-changes/page.tsx @@ -17,10 +17,29 @@ export default function Page() {

    Breaking Changes & Known Limitations

    DynamicWhere.ex is intentionally opinionated about how queries are shaped. - The twenty-four points below cover constraints, surprises, and corner cases — + The twenty-nine points below cover constraints, surprises, and corner cases — read them before designing an API around the library so you can pick the right entry points and avoid runtime exceptions in production.

    + + Points 25 to 29 changed in 3.2.0, and each is + visible to code written for 3.1.0. A guarded query that sends no{" "} + Selects synthesizes its projection whenever a denied value + can reach the result, and the projection keeps what the source carries: + the assigned members of a projected row, and an entity's columns, + owned and complex members, while an entity's navigations and the + objects of a row in memory are left out (point 25).{" "} + Selects naming a member is gated against every denial + beneath it, and a narrowing that cannot be built is refused + (point 26). A declared default order reaches a projection that + builds T and assigns every field the default names + (point 27). Every async terminal gains overloads that take a{" "} + CancellationToken, so ToListAsync(filter, default){" "} + no longer compiles, and the async dynamic Filter and the + async Summary read through EF Core's asynchronous + operators (point 28). And a type in an application namespace that + starts with System is policed (point 29). + Eleven behaviours changed in 3.1.0. Each one fixes a defect, and each one is visible to a caller that depended on the old shape. Date @@ -594,10 +613,13 @@ export default function Page() { A member carrying [DwNoSelect] makes the policy layer - synthesize a projection for a query that sent none, so a typed guarded - query on a type with no parameterless constructor raises the same code — - even though the caller never asked for a Select. The dynamic - terminals project through SelectDynamic and are not affected. + synthesize a projection for a query that sent none — since 3.2.0 + whatever the member holds, and beneath another member when its value + can reach the result (point 25) — so a typed guarded query on a + type with no parameterless constructor raises + the same code, even though the caller never asked for a{" "} + Select. The dynamic terminals project through{" "} + SelectDynamic and are not affected.

    18. A Guarded Query Requires a Prepared Context

    @@ -1077,6 +1099,293 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say }`}     +

    25. A Guarded Query That Sends No Selects Keeps What the Source Carries

    +

    + A request with no Selects returns whole rows, denied fields + included, so a guarded query synthesizes a projection when a denied + field could reach the result. 3.2.0 changed when it + does so and what the projection keeps. The rules are on{" "} + Policy configuration. +

    +
      +
    • + When. A field denied at the top of T{" "} + asks for it whatever the field holds. A field denied beneath a member + asks for it when its value can reach the result: on an entity, beneath + a column, an owned or complex member, or a navigation the query loads + through an Include, an automatic include or a lazy + loader; on a projected row, beneath a member the initializer assigns; + in memory, beneath any member. It does so in both tiers, typed and + dynamic, for a Filter and a Segment. Until + 3.2.0 only a simple field denied at the top of T asked for + one. A denial beneath a navigation nothing loads never leaves the + database, so an entity whose only denials sit there is read exactly as + in 3.1.0. +
      • +
    • + What. The allowed members, which replace the allowed + scalars. A row a projection builds keeps the members its initializer + assigns. An entity keeps its mapped columns, converted and JSON ones + included, its owned and complex members, and every collection of + simple values such as byte[] or{" "} + List<string>. Rows in memory keep their values. A + member holding an object is kept whole when nothing it can hold is + denied, narrowed to the allowed fields where the core's narrowing + translates, and otherwise left out whole. +
      • +
    + + With every denied field beneath a member and none at the top of{" "} + T, nothing was synthesized, and the whole row came back + with the denied value in it: in a list or nested object of a row + projected before ApplyPolicy, in a row held in memory, and + in an entity's included, automatically included, lazily loaded or + owned member — typed and dynamic, in both tiers, for a{" "} + Filter and a Segment. + + + A field denied at the top of T whose own type is not a + simple value — a byte array, a list, an owned object, a JSON column — + synthesized no projection either, so with nothing else denied the whole + row came back with it. + + + In 3.1.0, as soon as any field was denied, every nested object and list + of a row projected before ApplyPolicy came back null or + empty, and so did an entity's columns holding an object, its owned + and complex members and its collections of simple values. They are + returned now, whole or narrowed. A member that cannot be narrowed is + left out whole, and the trace records a Dropped decision + whose reason starts left out whole. + + + Once a projection is needed it leaves out an entity's navigations, + included ones too, since projecting one would load it: name the + navigation in Selects to get it, narrowed. It leaves out + the objects a row in memory holds, since a kept object is the + caller's own and a transform would change it in place, and a value + EF Core does not map, which EF Core could compute only by reading the + whole entity, the denied columns included. A member with no setter and + a member named with one of the parser's words are left out too. A + typed query projects into T, so T needs a + public parameterless constructor for it, as it already did + (point 1). + + + A forced scope declared on a list's element type asks for no + projection on its own. It filters the rows that hold the list, never its + elements, so Selects naming the list returns every element, + those the scope excludes included, as in every release. A projection + needed for another reason leaves such a list out whole. Scope the + elements where the row is built. + + +

    26. Selects Naming a Member Is Gated Against Every Denial Beneath It

    +

    + When Selects names a navigation with a denied field beneath + it, the Convenience tier replaces the entry with the + allowed fields beneath it, and the Strict tier refuses it. + Since 3.2.0 the gate finds every denial beneath the + member, and refuses, with FieldDeniedForSelect, a narrowing + it cannot build as gated. See{" "} + A navigation named in Selects. +

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    Selects namesUntil 3.1.0Since 3.2.0
    A navigation whose key, Id, is denied + The convenience tier narrowed the key away, and the core's + typed projection, which adds the key of every nested node it + builds, put it back. + + Refused in both tiers, as naming a sibling of the key already + was. +
    A navigation named through another, Main.Lead, when Main.Id is deniedKept, and the projection added Main's key.Refused: the key of every node the path passes through is gated.
    + A member typed as a collection the core does not unwrap —{" "} + IReadOnlyList<T>,{" "} + IReadOnlyCollection<T>,{" "} + Collection<T> or an application's own — + with a denied field beneath it + + Every field beneath it came back, the denied ones included, in both + tiers: the projection gate read collections through a narrower list + than the attribute walker, and found nothing beneath the member. + + The gate reads collections the way the walker does. The strict + tier refuses the denied field, and the convenience tier's + narrowing, which the core cannot project, is refused too. +
    + A member of a projected or in-memory row whose type holds a field + denied where no path reaches it: deeper than four segments, or + inside a framework generic such as{" "} + Dictionary<string, T> + Returned, the denied field included. + Refused under Strict, narrowed away under{" "} + Convenience, and refused in both tiers where it + cannot be narrowed. +
    A member with a denied property that has no setter beneath it, or a rule on a path reached through a cycleNot found, so the member came back with it.Found: the gate reads the providers' rules as well as the walk.
    +

    + A member that cannot be narrowed at all — a column, a complex property + or a member stored as JSON, a member of a row in memory, or one a + projection builds some way the core cannot narrow — is refused in both + tiers when something beneath it is denied. +

    + + A request that ran on 3.1.0 can now be refused. Under the{" "} + Convenience tier the refusal names the denied key, the + first denied field beneath the member, or, for a denial no path names, + the member itself; under Strict its FieldPath{" "} + is "*". A request that sends no{" "} + Selects is not refused for such a member: its synthesized + projection narrows the member or leaves it out whole (point 25). + + + A member typed object is opaque to the policy: a + synthesized projection leaves it out, and naming it returns whatever it + holds. A framework generic holding a policed type, such as{" "} + Dictionary<string, LineDto>, has no paths beneath it: + naming it is refused under Strict and narrowed away under{" "} + Convenience, or refused there too where the member cannot + be narrowed, and a synthesized projection leaves it out. + Hold such values in a list of the policed type instead. + + +

    27. DefaultOrder Reaches a Projection That Builds T

    +

    + In 3.1.0 a Select anywhere in the chain kept a guarded query + in its own order, because a default applied after a projection could + name a field the projection left out, which EF Core cannot translate. + Since 3.2.0 only the outermost Select{" "} + counts, because it makes the rows the default orders. When it builds{" "} + T in an object initializer and assigns every field the{" "} + [DwEntity(DefaultOrder)]{" "} + names, at every level of a nested path, with nothing EF Core would + compute on the client, the default applies. A constructor with + arguments, a default field the initializer does not assign, a nested + path through anything but an initializer, or a default field computed + by the application's own method, which EF Core evaluates on the + client and cannot order by, still leaves the query in its own order. +

    + {`[DwEntity(DefaultOrder = "CreatedAt desc, Id")] +public class TicketRow +{ + public int Id { get; set; } + public DateTime CreatedAt { get; set; } + public string Title { get; set; } = string.Empty; +} + +// 3.1.0: unordered. 3.2.0: ordered by CreatedAt desc, Id. +var rows = db.Tickets + .Select(t => new TicketRow { Id = t.Id, CreatedAt = t.CreatedAt, Title = t.Title }) + .ApplyPolicy(caller) + .ToList(new Filter());`} +
      +
    • + A projection composed on the guarded handle — the guarded{" "} + Select, or a guarded Filter whose{" "} + Selects is set — keeps the rest of the chain unordered, + even when it keeps every default field. So{" "} + {`guarded.Select(["Id", "Title"]).Page(page)`} pages as it + did in 3.0.0, unordered. +
      • +
    • + A Filter composed on the handle that sent orders gets no + default later in the chain, even when the policy dropped every one of + them, as a composed Order already did not. +
      • +
    + + A guarded query over such a projection that sends no orders now comes + back in the declared order, where it used to come back in the + database's. Paging through it is stable if the default ends with a + unique field. + + +

    28. Every Async Terminal Takes a CancellationToken

    +

    + Since 3.2.0 every asynchronous terminal, guarded and + unguarded, has overloads that take a CancellationToken:{" "} + ToListAsync{" "} + and{" "} + ToListAsyncDynamic{" "} + with a Filter,{" "} + ToListAsync{" "} + with a Summary, and{" "} + ToListAsync{" "} + with a Segment. The token reaches the count and the read. The + overloads sit beside the 3.1 signatures, which are unchanged, so code + compiled against 3.1 still binds. That brings the extension methods to + 28. A reflection lookup by name alone now finds several methods where + it found one — ToListAsyncDynamic, on the extension class + and on the guarded handle — so Type.GetMethod given only + the name throws AmbiguousMatchException; pass the + parameter types. +

    + + default fits both bool getQueryString and the + new CancellationToken overload, so the call is ambiguous + (CS0121). So are ToListAsyncDynamic(filter, default) and{" "} + ToListAsync(summary, default), on a query and on the guarded + handle alike. Write false, a token, or a named argument. + {`await query.ToListAsync(filter, default); // CS0121 since 3.2.0 +await query.ToListAsync(filter, false); // as 3.1 read it +await query.ToListAsync(filter, cancellationToken); // the new overload`} + + + ToListAsyncDynamic and the async Summary read + through EF Core's ToListAsync, and the async{" "} + Summary counts through CountAsync. They used to + read synchronously on a thread-pool thread, and the summary counted + synchronously, so on an EF Core query a canceled token now reaches the + database. The rows and the counts are the same. A provider that is not + EF Core's keeps the synchronous read. + + +

    29. A Type in a Namespace That Starts with System Is Policed

    +

    + The attribute walker does not descend into the framework's own + types, which carry no policy attributes. Until 3.2.0{" "} + it took any namespace whose name started with System for + the framework's, so an application namespace such as{" "} + SystemsCorp.Payroll or SystemX.Domain got no + policy beneath its types. A [DwDenied] field on such a type, + reached through a member, was returned, filterable and sortable. Only{" "} + System and the namespaces beneath it are the + framework's now. +

    + + A guarded request that filtered on, sorted by or selected such a field + ran on 3.1.0. It is now refused or dropped, as for any denied field. + +

    See also

    • @@ -1110,12 +1419,22 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say
    • Policy configuration →{" "} - context for points 21, 22, 23 and 24: the caps, the trace on a result, - and what a strict refusal carries. + context for points 21 to 26: the caps, the trace on a result, what a + strict refusal carries, and what a denied field does to a projection.
    • Security & k-anonymity →{" "} - why the strict tier hides which fields exist (point 23). + why the strict tier hides which fields exist (point 23), and{" "} + the denials the gate could not see{" "} + (points 25, 26 and 29). +
      • +
    • + Default order →{" "} + context for point 27. +
      • +
    • + Materialization →{" "} + context for point 28: every async terminal and its overloads.
    diff --git a/OfficialWebsite/app/docs/errors/page.tsx b/OfficialWebsite/app/docs/errors/page.tsx index 11d0cd6..7ac4933 100644 --- a/OfficialWebsite/app/docs/errors/page.tsx +++ b/OfficialWebsite/app/docs/errors/page.tsx @@ -340,8 +340,10 @@ export default function Page() { T the projection cannot construct — a positional record, most often. The type's name is on{" "} Subject, not in the message. Also reached by a typed - guarded query whose policy denies a field for Select, - since the deny synthesizes a projection + guarded query whose policy denies a field for Select{" "} + — since 3.2.0 whatever the field holds, and beneath a member where + its value can reach the result — because the deny synthesizes a + projection diff --git a/OfficialWebsite/app/docs/extensions/page.tsx b/OfficialWebsite/app/docs/extensions/page.tsx index 813f08d..1b4c94e 100644 --- a/OfficialWebsite/app/docs/extensions/page.tsx +++ b/OfficialWebsite/app/docs/extensions/page.tsx @@ -165,13 +165,15 @@ export default function Page() {

    Materialization

    Execute the composed query and return a paginated result — typed, - dynamic, summary, or segment. The two Filter async - terminals count with EF Core's CountAsync and read - with ToListAsync / ToDynamicListAsync;{" "} - ToListAsync(Summary) counts synchronously and only awaits - the read, and ToListAsync(Segment) combines its sets into - one query and then counts and reads it exactly as a{" "} - Filter does. + dynamic, summary, or segment. The async terminals count with EF + Core's CountAsync and read with EF Core's{" "} + ToListAsync. Since 3.2.0 that includes the dynamic{" "} + Filter's read and the Summary's count + and read, which used to run synchronously.{" "} + ToListAsync(Summary) on a provider that is not EF + Core's keeps its synchronous count and read, and{" "} + ToListAsync(Segment) combines its sets into one query and + then counts and reads it exactly as a Filter does.

    @@ -198,6 +200,16 @@ export default function Page() { + + + + + + + @@ -212,6 +224,16 @@ export default function Page() { + + + + + + + @@ -226,6 +248,16 @@ export default function Page() { + + + + + + + @@ -233,9 +265,34 @@ export default function Page() { + + + + + + +
    Yes Docs
    ToListAsync (Filter, token) + .ToListAsync<T>(Filter, CancellationToken) /{" "} + .ToListAsync<T>(Filter, bool getQueryString, CancellationToken) + Task<FilterResult<T>>YesDocs
    ToListDynamic (Filter) .ToListDynamic<T>(Filter, bool getQueryString = false)Yes Docs
    ToListAsyncDynamic (Filter, token) + .ToListAsyncDynamic<T>(Filter, CancellationToken) /{" "} + .ToListAsyncDynamic<T>(Filter, bool getQueryString, CancellationToken) + Task<FilterResult<dynamic>>YesDocs
    ToList (Summary) .ToList<T>(Summary, bool getQueryString = false)Yes Docs
    ToListAsync (Summary, token) + .ToListAsync<T>(Summary, CancellationToken) /{" "} + .ToListAsync<T>(Summary, bool getQueryString, CancellationToken) + Task<SummaryResult>YesDocs
    ToListAsync (Segment) .ToListAsync<T>(Segment segment)Yes (only) Docs
    ToListAsync (Segment, token).ToListAsync<T>(Segment, CancellationToken)Task<SegmentResult<T>>Yes (only)Docs
    + + ToListAsync and ToListAsyncDynamic with a{" "} + Filter, ToListAsync with a{" "} + Summary and ToListAsync with a{" "} + Segment each have overloads that take a{" "} + CancellationToken. The token reaches the count and the + read. The overloads sit beside the 3.1 signatures, which are unchanged, + so code compiled against 3.1 still binds.{" "} + ToListAsync(filter, default) no longer compiles, because{" "} + default fits both getQueryString and the + token; neither do ToListAsyncDynamic(filter, default) and{" "} + ToListAsync(summary, default). Write false, a + token, or a named argument. And a reflection lookup of{" "} + ToListAsyncDynamic by name alone now finds three methods + where it found one: pass the parameter types to{" "} + Type.GetMethod. + + Segment operations are async-only. There is no synchronous ToList<T>(Segment) variant. The set @@ -254,7 +311,9 @@ export default function Page() { for unit tests and in-process pipelines. Nothing else has one: there is no ToListDynamic(Summary), and no async or composable method works on an IEnumerable<T> source. Counting - those three, the library ships 21 extension methods. + those three and the seven overloads that take a{" "} + CancellationToken, the library ships 28{" "} + extension methods.

    @@ -268,7 +327,7 @@ export default function Page() { - Every one of the 21 begins by asking the policy layer whether this type + Every one of the 28 begins by asking the policy layer whether this type may be queried at all. A type marked{" "} [DwEntity(RequirePolicy = true)] throws{" "} PolicyException with PolicyRequired when it is diff --git a/OfficialWebsite/app/docs/extensions/to-list-async-dynamic-filter/page.tsx b/OfficialWebsite/app/docs/extensions/to-list-async-dynamic-filter/page.tsx index 6596940..0f19684 100644 --- a/OfficialWebsite/app/docs/extensions/to-list-async-dynamic-filter/page.tsx +++ b/OfficialWebsite/app/docs/extensions/to-list-async-dynamic-filter/page.tsx @@ -7,7 +7,7 @@ import Callout from "@/components/Callout"; export const metadata: Metadata = { title: ".ToListAsyncDynamic(Filter)", description: - "Async EF Core entry — materialize a Filter using SelectDynamic and ToDynamicListAsync, returning Task>.", + "Async EF Core entry — materialize a Filter using SelectDynamic, EF Core's CountAsync and ToListAsync, returning Task>, with overloads that take a CancellationToken.", alternates: { canonical: "https://doc.dynamicwhere.com/docs/extensions/to-list-async-dynamic-filter/" }, }; @@ -20,8 +20,9 @@ export default function Page() { .ToListDynamic<T>(Filter) - . Uses EF Core's CountAsync() and{" "} - ToDynamicListAsync() under the hood. + . Counts with EF Core's CountAsync() and reads with EF + Core's own ToListAsync(). Since 3.2.0 two more overloads + take a CancellationToken, which reaches both.

    Signature

    @@ -29,6 +30,20 @@ export default function Page() { this IQueryable query, Filter filter, bool getQueryString = false) + where T : class + +// 3.2.0 +public static Task> ToListAsyncDynamic( + this IQueryable query, + Filter filter, + CancellationToken cancellationToken) + where T : class + +public static Task> ToListAsyncDynamic( + this IQueryable query, + Filter filter, + bool getQueryString, + CancellationToken cancellationToken) where T : class`}     @@ -58,6 +73,15 @@ export default function Page() { QueryString + + + + + +
    cancellationTokenCancellationToken– + Cancels the count and the read. The overload without it passes{" "} + CancellationToken.None +
    @@ -65,14 +89,29 @@ export default function Page() {
    • Where applied on the typed query.
    • - CountAsync() on the typed query → TotalCount. + CountAsync(cancellationToken) on the typed query →{" "} + TotalCount.
    • Order applied on the typed query.
    • Page applied on the typed query.
    • SelectDynamic projection applied last.
      • -
    • ToDynamicListAsync() materializes the result.
      • +
    • + EF Core's ToListAsync(cancellationToken) materializes + the result, called for the query's element type: the class the + projection generates, or T when Selects is + null. +
    + + The read used to run Dynamic LINQ's{" "} + ToDynamicListAsync(), which reads synchronously on a + thread-pool thread. It now runs through EF Core's{" "} + ToListAsync(), so the database command runs + asynchronously and a canceled token reaches it. The rows are the same. + Like the count, the read needs an EF Core async provider. + + Ordering and pagination are applied on the strongly-typed{" "} IQueryable<T> before the dynamic @@ -89,6 +128,32 @@ export default function Page() { nested dynamic objects and collections. +

    Cancellation

    +

    + The two overloads that take a CancellationToken are new in + 3.2.0. The token reaches the count and the read, so a canceled token + stops whichever of the two is running, and the call throws{" "} + OperationCanceledException. EF Core's{" "} + TaskCanceledException derives from it. The overload without + a token passes CancellationToken.None. +

    +

    + They are overloads, not an optional parameter added to the old + signature. The 3.1 signature is unchanged, so code compiled against 3.1 + still binds. The guarded handle that{" "} + ApplyPolicy{" "} + returns has the same overloads. +

    + + default fits both bool getQueryString and{" "} + CancellationToken, so the compiler reports the call as + ambiguous (CS0121). Write false, a token, or a named + argument. + + {`await db.Products.ToListAsyncDynamic(filter, default); // CS0121: ambiguous +await db.Products.ToListAsyncDynamic(filter, cancellationToken); // the token overload +await db.Products.ToListAsyncDynamic(filter, true, cancellationToken); // the SQL and a token`} +

    Returns

    Task<FilterResult<dynamic>>. diff --git a/OfficialWebsite/app/docs/extensions/to-list-async-filter/page.tsx b/OfficialWebsite/app/docs/extensions/to-list-async-filter/page.tsx index ac8eb3c..fd218cf 100644 --- a/OfficialWebsite/app/docs/extensions/to-list-async-filter/page.tsx +++ b/OfficialWebsite/app/docs/extensions/to-list-async-filter/page.tsx @@ -7,7 +7,7 @@ import Callout from "@/components/Callout"; export const metadata: Metadata = { title: ".ToListAsync(Filter)", description: - "Async EF Core entry point — materialize a Filter against an IQueryable using CountAsync and ToListAsync.", + "Async EF Core entry point — materialize a Filter against an IQueryable using CountAsync and ToListAsync, with overloads that take a CancellationToken.", alternates: { canonical: "https://doc.dynamicwhere.com/docs/extensions/to-list-async-filter/" }, }; @@ -21,7 +21,8 @@ export default function Page() { .ToList<T>(Filter) . Uses EF Core's CountAsync() and{" "} - ToListAsync() under the hood. + ToListAsync() under the hood. Since 3.2.0 two more + overloads take a CancellationToken, which reaches both.

    Signature

    @@ -29,6 +30,20 @@ export default function Page() { this IQueryable query, Filter filter, bool getQueryString = false) + where T : class + +// 3.2.0 +public static Task> ToListAsync( + this IQueryable query, + Filter filter, + CancellationToken cancellationToken) + where T : class + +public static Task> ToListAsync( + this IQueryable query, + Filter filter, + bool getQueryString, + CancellationToken cancellationToken) where T : class`} @@ -58,6 +73,15 @@ export default function Page() { FilterResult.QueryString + + + + + +
    cancellationTokenCancellationToken– + Cancels the count and the read. The overload without it passes{" "} + CancellationToken.None +
    @@ -65,12 +89,13 @@ export default function Page() {
    • Where applied on the typed query.
    • - CountAsync() on the typed query → TotalCount. + CountAsync(cancellationToken) on the typed query →{" "} + TotalCount.
    • Order applied on the typed query.
    • Page applied on the typed query.
    • Select projection applied last.
      • -
    • ToListAsync() materializes the result.
      • +
    • ToListAsync(cancellationToken) materializes the result.
    @@ -86,6 +111,33 @@ export default function Page() { database provider. +

    Cancellation

    +

    + The two overloads that take a CancellationToken are new in + 3.2.0. The token reaches the count and the read, so a canceled token + stops whichever of the two is running, and the call throws{" "} + OperationCanceledException. EF Core's{" "} + TaskCanceledException derives from it. The overload without + a token passes CancellationToken.None. +

    +

    + They are overloads, not an optional parameter added to the old + signature. The 3.1 signature is unchanged, so code compiled against 3.1 + still binds. The guarded handle that{" "} + ApplyPolicy{" "} + returns has the same overloads. +

    + + default fits both bool getQueryString and{" "} + CancellationToken, so the compiler reports the call as + ambiguous (CS0121). Write false, a token, or a named + argument. + + {`await db.Customers.ToListAsync(filter, default); // CS0121: ambiguous +await db.Customers.ToListAsync(filter, cancellationToken); // the token overload +await db.Customers.ToListAsync(filter, getQueryString: false); // no token +await db.Customers.ToListAsync(filter, true, cancellationToken); // the SQL and a token`} +

    Returns

    Task< @@ -115,6 +167,18 @@ Console.WriteLine(result.QueryString);`} return Results.Ok(result); });`} +

    + Cancel with the request. A minimal API binds a{" "} + CancellationToken parameter to{" "} + HttpContext.RequestAborted, so a client that disconnects + cancels the count or the read. +

    + {`app.MapPost("/customers/search", async (Filter filter, AppDbContext db, CancellationToken cancellationToken) => +{ + var result = await db.Customers.ToListAsync(filter, cancellationToken); + return Results.Ok(result); +});`} +

    See also

    • diff --git a/OfficialWebsite/app/docs/extensions/to-list-async-segment/page.tsx b/OfficialWebsite/app/docs/extensions/to-list-async-segment/page.tsx index 9c50828..9429240 100644 --- a/OfficialWebsite/app/docs/extensions/to-list-async-segment/page.tsx +++ b/OfficialWebsite/app/docs/extensions/to-list-async-segment/page.tsx @@ -7,7 +7,7 @@ import Callout from "@/components/Callout"; export const metadata: Metadata = { title: ".ToListAsync(Segment)", description: - "Async-only segment entry — combine the ConditionSets with Union / Intersect / Except into one query, then order, page and project it in the database like a filter.", + "Async-only segment entry — combine the ConditionSets with Union / Intersect / Except into one query, then order, page and project it in the database like a filter. An overload takes a CancellationToken.", alternates: { canonical: "https://doc.dynamicwhere.com/docs/extensions/to-list-async-segment/" }, }; @@ -38,6 +38,13 @@ export default function Page() { {`public static Task> ToListAsync( this IQueryable query, Segment segment) + where T : class + +// 3.2.0 +public static Task> ToListAsync( + this IQueryable query, + Segment segment, + CancellationToken cancellationToken) where T : class`} @@ -59,6 +66,14 @@ export default function Page() { Selects, Orders, Page + + + + +
      cancellationTokenCancellationToken + Cancels the count and the read. New in 3.2.0; the overload + without it passes CancellationToken.None +
      @@ -85,7 +100,18 @@ export default function Page() { Only the requested page is read, and an order field need not be selected.
      • +
    • + The overload that takes a CancellationToken passes it to + that count and that read. A canceled token stops whichever of the two + is running, and the call throws OperationCanceledException. +
    +

    + A segment takes no getQueryString, so{" "} + ToListAsync(segment, default) is not ambiguous: it binds + the token overload and passes CancellationToken.None. The + 3.1 signature is unchanged, so code compiled against 3.1 still binds. +

    Which rows belong is decided in the database, not by object reference, so a tracking query, an AsNoTracking() query and a query with{" "} diff --git a/OfficialWebsite/app/docs/extensions/to-list-async-summary/page.tsx b/OfficialWebsite/app/docs/extensions/to-list-async-summary/page.tsx index cf97876..3272b41 100644 --- a/OfficialWebsite/app/docs/extensions/to-list-async-summary/page.tsx +++ b/OfficialWebsite/app/docs/extensions/to-list-async-summary/page.tsx @@ -7,7 +7,7 @@ import Callout from "@/components/Callout"; export const metadata: Metadata = { title: ".ToListAsync(Summary)", description: - "Async EF Core entry — materialize a Summary against an IQueryable and return Task.", + "Async EF Core entry — materialize a Summary against an IQueryable and return Task, with overloads that take a CancellationToken.", alternates: { canonical: "https://doc.dynamicwhere.com/docs/extensions/to-list-async-summary/" }, }; @@ -20,8 +20,10 @@ export default function Page() { .ToList<T>(Summary) - . The group count runs synchronously; only the data read is async, via - Dynamic LINQ's ToDynamicListAsync(). + . On an EF Core query it counts the groups with EF Core's{" "} + CountAsync() and reads them with EF Core's{" "} + ToListAsync(). Since 3.2.0 two more overloads take a{" "} + CancellationToken, which reaches both.

    Signature

    @@ -29,6 +31,20 @@ export default function Page() { this IQueryable query, Summary summary, bool getQueryString = false) + where T : class + +// 3.2.0 +public static Task ToListAsync( + this IQueryable query, + Summary summary, + CancellationToken cancellationToken) + where T : class + +public static Task ToListAsync( + this IQueryable query, + Summary summary, + bool getQueryString, + CancellationToken cancellationToken) where T : class`} @@ -58,6 +74,15 @@ export default function Page() { SummaryResult.QueryString + + + + + +
    cancellationTokenCancellationToken– + Cancels the count and the read. The overload without it passes{" "} + CancellationToken.None +
    @@ -67,13 +92,33 @@ export default function Page() {
  • Group applied — produces grouped dynamic intermediate.
  • Having applied — fields must reference aggregate aliases.
  • - Count() on the grouped query → TotalCount. - This count is synchronous, not awaited. + The grouped query is counted → TotalCount. On an EF Core + query this is EF Core's CountAsync(cancellationToken).
  • Order applied on the grouped query.
  • Page applied on the grouped query.
    • -
  • Async materialization as List<dynamic>.
    • +
  • + Async materialization as List<dynamic>. On an EF + Core query this is EF Core's{" "} + ToListAsync(cancellationToken). +
    • +

    + A source whose provider is not EF Core's — rows in memory through{" "} + AsQueryable(), for one — keeps the reads it had in 3.1: a + synchronous Count(), then Dynamic LINQ's{" "} + ToDynamicListAsync(), which reads synchronously on a + thread-pool thread. A token that is already canceled still stops it + before the count. +

    + + + Until 3.2.0 the group count ran synchronously and only the read was + awaited, through ToDynamicListAsync(), on every provider. On + an EF Core query both now run through EF Core's asynchronous + operators, so a canceled token reaches the database. The count and the + rows are the same. + Dotted GroupBy fields like Category.Name{" "} @@ -82,6 +127,32 @@ export default function Page() { use the dotted form — the library handles alias mapping internally. +

    Cancellation

    +

    + The two overloads that take a CancellationToken are new in + 3.2.0. The token reaches the count and the read. On an EF Core query a + canceled token stops whichever of the two is running, and the call + throws OperationCanceledException. EF Core's{" "} + TaskCanceledException derives from it. The overload without + a token passes CancellationToken.None. +

    +

    + They are overloads, not an optional parameter added to the old + signature. The 3.1 signature is unchanged, so code compiled against 3.1 + still binds. The guarded handle that{" "} + ApplyPolicy{" "} + returns has the same overloads. +

    + + default fits both bool getQueryString and{" "} + CancellationToken, so the compiler reports the call as + ambiguous (CS0121). Write false, a token, or a named + argument. + + {`await db.Products.ToListAsync(summary, default); // CS0121: ambiguous +await db.Products.ToListAsync(summary, cancellationToken); // the token overload +await db.Products.ToListAsync(summary, true, cancellationToken); // the SQL and a token`} +

    Returns

    Task< diff --git a/OfficialWebsite/app/docs/policies/admin/page.tsx b/OfficialWebsite/app/docs/policies/admin/page.tsx index a468b60..bb4f0f5 100644 --- a/OfficialWebsite/app/docs/policies/admin/page.tsx +++ b/OfficialWebsite/app/docs/policies/admin/page.tsx @@ -169,6 +169,20 @@ export default function Page() { nothing is audited, so an operator checking a rule does not fill the audit trail with reads that never happened.

    +

    + A simulation has no source, so it reads the type as a source it cannot + see into. That shows in a clause that sends no Selects: + every denial beneath a member counts, and the projection it shows keeps + only the members that hold a value, a collection of values included. A + guarded query keeps what its own source carries — over a projected row, + the objects its initializer assigns; over an entity, its columns, owned + and complex members, asking only about the denials whose value it + loads; over rows in memory, values only. So the simulated clause can + list fewer members than the query returns, and can show a projection + an entity query does not need. PolicySimulator reads a + type the same way. See{" "} + A request that sends no Selects. +

    From a ClaimsPrincipal

    {`var caller = await httpContext.GetPolicyContextAsync(claimsOptions); diff --git a/OfficialWebsite/app/docs/policies/attributes/page.tsx b/OfficialWebsite/app/docs/policies/attributes/page.tsx index 15e1b2c..646f9b1 100644 --- a/OfficialWebsite/app/docs/policies/attributes/page.tsx +++ b/OfficialWebsite/app/docs/policies/attributes/page.tsx @@ -72,7 +72,8 @@ public class Ticket and ToListAsyncDynamic with a Filter, to{" "} ToListAsync with a Segment, to the composable{" "} Filter and FilterDynamic, and to the composable{" "} - Page on a source nothing has ordered or projected. + Page on a source nothing has ordered and whose projection + hides no field the default names.
  • It never applies outside the guarded handle. A core method on a plain{" "} @@ -89,7 +90,8 @@ public class Ticket {`db.Tickets.OrderBy(t => t.Title).ApplyPolicy(caller)`} — or an Order was composed on the guarded handle first, as in{" "} {`guarded.Order(order).Page(page)`} — even when the policy - dropped every order that call sent. An in-memory + dropped every order that call sent. Since 3.2.0 a Filter{" "} + composed on the handle with orders counts the same way. An in-memory sequence sorted before ApplyPolicy is not recognised as ordered, because it reaches the policy as a query with no{" "} OrderBy in it, so it takes the default; send that order with @@ -98,14 +100,55 @@ public class Ticket Order.
  • - A projected query takes no default. A Select anywhere in the - chain — the guarded Select, as in{" "} - {`guarded.Select(fields).Page(page)`}, or a projection made - before ApplyPolicy — leaves the query in its own order: a - default applied after a projection can name a field the projection left - out, which EF Core cannot translate. + A projection made before ApplyPolicy takes the default only + when it cannot hide a field the default names. Only the outermost{" "} + Select of the chain counts, because it makes the rows the + default orders. Since 3.2.0 it hides nothing when it builds{" "} + T itself in an object initializer and assigns every field + the default names, at every level of a nested path, with nothing EF + Core would compute on the client: "Owner.Name"{" "} + needs {`Owner = new OwnerRow { Name = … }`}. EF Core then + translates the order, because each field is a member the projection + assigned. +
    • +
  • + Any other projection leaves the query in its own order, as every + projection did in 3.1.0: a constructor with arguments, a default field + the initializer does not assign, a nested path through anything but an + initializer, or a default field computed by the application's own + method, such as {`Label = Decorate(r.Code)`}. EF Core + evaluates such a method on the client, where it can project the value + but cannot order by it. A default applied to any of these could name a + field EF Core cannot translate. +
    • +
  • + A projection composed on the guarded handle takes no default, even when + it keeps every field the default names. The guarded{" "} + Select, as in{" "} + {`guarded.Select(fields).Page(page)`}, and a guarded{" "} + Filter whose Selects is set leave the rest of + the chain unordered. The default is for the rows the caller's + source makes.
    • + {`[DwEntity(RequirePolicy = true, DefaultOrder = "CreatedAt desc, Id")] +public class TicketRow +{ + public int Id { get; set; } + public DateTime CreatedAt { get; set; } + public string Title { get; set; } = string.Empty; +} + +// Takes the default: the initializer builds TicketRow and assigns CreatedAt and Id. +var ordered = db.Tickets + .Select(t => new TicketRow { Id = t.Id, CreatedAt = t.CreatedAt, Title = t.Title }) + .ApplyPolicy(caller); + +// Keeps its own order: CreatedAt is not assigned, so the default could name +// a member these rows do not carry. +var unordered = db.Tickets + .Select(t => new TicketRow { Id = t.Id, Title = t.Title }) + .ApplyPolicy(caller);`}

    The default is gated like any order. A field in it that this caller may not order by is left out — never refused, because the caller did not send it — @@ -176,6 +219,13 @@ public class Ticket // and cannot sweep for one it does not. [DwOperators(Allow = new[] { Operator.Equal, Operator.In })] public string EmployeeCode { get; set; }`} + + A request that sends no Selects would return the whole row. + So a field denied for Select — at the top of the type, or + beneath a member where its value can reach the result — makes a guarded + query project the allowed members instead. See{" "} + A request that sends no Selects. +

    Injection

    diff --git a/OfficialWebsite/app/docs/policies/configuration/page.tsx b/OfficialWebsite/app/docs/policies/configuration/page.tsx index 4377193..77496cf 100644 --- a/OfficialWebsite/app/docs/policies/configuration/page.tsx +++ b/OfficialWebsite/app/docs/policies/configuration/page.tsx @@ -6,7 +6,7 @@ import Callout from "@/components/Callout"; export const metadata: Metadata = { title: "Policy Configuration — options, caps, tiers and defaults", - description: "Every DynamicWhere.ex policy option and cap with its default: tiers, dry run, the trace on a result, refusal auditing, hash salt, store failure modes, query cost budget, MinGroupSize, plus startup validation and the twenty-two error codes.", + description: "Every DynamicWhere.ex policy option and cap with its default: tiers, what a denied field does to a projection, dry run, the trace on a result, refusal auditing, hash salt, store failure modes, query cost budget, MinGroupSize, plus startup validation and the twenty-two error codes.", keywords: ["DwPolicyOptions", "DwCaps", "MaxQueryCost", "MinGroupSize", "policy configuration"], alternates: { canonical: "https://doc.dynamicwhere.com/docs/policies/configuration/" }, }; @@ -64,6 +64,299 @@ export default function Page() { IncludeTraceInResult is false.

    + +

    + A Selects entry can name a navigation, such as{" "} + "Lines", rather than the fields beneath it. With + nothing denied beneath it, the entry is kept as written. With a denied + field beneath it, the Convenience tier replaces the entry + with the allowed fields beneath it, and the Strict tier + refuses it. +

    +
    + + + + + + + + +
    Selects names a navigationConvenienceStrict
    with nothing denied beneath itKept wholeKept whole
    with a denied field beneath itReplaced by the allowed fields beneath itFieldDeniedForSelect
    whose key, Lines.Id, is denied (3.2.0)FieldDeniedForSelectFieldDeniedForSelect
    with a denied field beneath it, where the narrowing cannot be built (3.2.0)FieldDeniedForSelectFieldDeniedForSelect
    whose type holds a field denied for Select that no path names (3.2.0)Narrowed away, where it can be narrowedFieldDeniedForSelect
    +
      +
    • + The fields beneath a member are read the way the attribute walker + reads them: through any collection type, and no deeper than its four + segments. Since 3.2.0 a member typed{" "} + IReadOnlyList<T>,{" "} + IReadOnlyCollection<T>,{" "} + Collection<T> or an application's own + collection no longer hides the denials beneath it. The providers' + own rules are asked too, so a denied property with no setter and a + rule on a path reached through a cycle count. +
      • +
    • + A narrowing that cannot be built as it was gated is refused in both + tiers (3.2.0). The core's typed projection adds the key,{" "} + Id, of every nested node it builds, so a navigation + narrowed around its own denied key would get the key back. The core + reads a path only through an array, List<T>,{" "} + IList<T>, ICollection<T>,{" "} + IEnumerable<T>, HashSet<T> or{" "} + ISet<T>, so a narrowing through any other + collection fails its validation. And some members cannot be narrowed + at all: a column, a complex property or a member stored as JSON, which + EF Core reads whole; a member of a row in memory; and a member a + projection builds some way the core cannot narrow. +
      • +
    • + A member that carries everything its type holds — a member of a + projected or in-memory row, or an entity's column, owned or + complex member — can hold a field denied for Select that + no path names: one deeper than four segments, or inside a framework + generic such as Dictionary<string, T>. The{" "} + Strict tier refuses such a member, and the{" "} + Convenience tier narrows it away (3.2.0); where it cannot + be narrowed, as a member of a row in memory cannot, both tiers refuse + it. An entity's navigation loads only its own entity, so only its + paths are asked about. +
      • +
    • + A navigation named through another, Main.Lead, gates the + key of every node it passes through, which the core's projection + adds, as a dotted path to a value always did (3.2.0). A denied key + refuses the projection. Naming a field beside a denied key,{" "} + Lines.Name when Lines.Id is denied, was + already refused in both tiers. +
      • +
    • + Under Convenience the refusal names the denied key, the + first denied field beneath the member, or, for a denial no path names, + the member itself. Under Strict its{" "} + FieldPath is "*", as on every field + refusal. The trace records the reason either way. +
      • +
    + +

    A request that sends no Selects

    +

    + A request that sends no Selects returns whole rows, denied + fields included, because the core projects only when{" "} + Selects is set. So when a field denied for{" "} + Select could reach the result, a guarded query synthesizes + the projection itself. It does so in both tiers, typed and dynamic, for + a whole Filter and for a Segment. A clause + composed on its own, such as Where, Order or{" "} + Page, synthesizes nothing. +

    +

    + The projection keeps the allowed members: what an + unguarded call would return, less what the policy withholds. Before + 3.2.0 it kept the allowed scalars only, and only a simple field denied + at the top of the type asked for it — see{" "} + breaking changes. +

    + +

    When a projection is needed

    +
      +
    • + A field denied at the top of T always asks for one, + whatever it holds: a scalar, a blob, a list, an owned object or a JSON + column. +
      • +
    • + A field denied beneath a member asks for one when its value can reach + the result. On an entity, that is beneath a column, an owned or + complex member, or a navigation something loads: an{" "} + Include or ThenInclude on the query, an + automatic include, or a lazy loader — EF Core's proxies, or an + injected ILazyLoader — which fills a navigation after the + query. An Include written in a form the library cannot + read counts as loading every navigation. On a row a projection builds, + it is beneath a member the initializer assigns. On a row in memory, it + is beneath any member. +
      • +
    • + A denial beneath a navigation nothing loads never leaves the database, + so it asks for no projection. An entity whose only denials sit beneath + such navigations is read as it was in 3.1.0. +
      • +
    • + A member whose type can hold a field denied for Select{" "} + that no path names — deeper than the walker's four segments, or + inside a framework generic such as{" "} + Dictionary<string, T> — asks for one too, and so does + a member typed object, when what it holds can reach the + result. +
      • +
    • + The denials beneath a member come from the providers' rules as + well as from walking the type, so a denied property with no setter, a + rule on a path reached through a cycle, and a rule deeper than the + walk all count. +
      • +
    • + A forced scope beneath a member asks for no projection on its own. It + filters the rows that hold the member, as it always has. When a + projection is needed anyway, the member is left out whole. +
      • +
    + +

    What it keeps

    +

    + A member holding a value — a simple type, or a collection of one such as{" "} + byte[], string[] or{" "} + List<string> — is kept when it is allowed and the + source carries it. A member holding an object, or a list of them, is + kept whole, narrowed or left out whole, as below, and only where the + source carries it: +

    + + + + + + + + + + + + + + + + + + + +
    SourceValues keptObjects kept
    A projection that builds its rows before ApplyPolicy: the outermost Select constructs the row, in an object initializer or with a constructor, as in {`db.Roles.Select(r => new RoleRow { … })`}Every member the initializer assigns; every member when a constructor builds the rowThe same
    An entity query, or a Select that hands back an entity, as in {`db.Orders.Select(o => o.Customer)`}Every member EF Core mapsIts columns, converted and JSON ones included, its owned members and, on EF Core 8 or later, its complex properties, read from the EF Core model
    Rows in memory, as in roles.ApplyPolicy(caller)Every memberNone
    +

    + A value EF Core does not map is left out: computing it would make EF + Core read the whole entity, the denied columns included, and it holds + only its initial value anyway. A source the library cannot read — no EF + Core model, and neither a projection it can see into nor rows in memory + — keeps values only, as in 3.1.0, and every denial beneath a member + counts. +

    + +

    Whole, narrowed or left out whole

    +

    A member holding an object that the source carries is:

    +
      +
    • + kept whole when nothing beneath it is denied, nothing + in its type is denied or typed object, no forced scope is + beneath it, and no transform beneath it lands on a property with no + setter; +
      • +
    • + narrowed otherwise, to the allowed fields beneath it, + four segments deep, as a caller naming it would get it, where the + core's narrowing translates: an object the projection's + initializer builds, a list a subquery reads into a type the core can + bind (not an array or a set), a navigation that is neither complex nor + stored as JSON, or an entity's owned member not stored as JSON. A + field beneath it that can hold what the policy cannot name is left + out; +
      • +
    • + left out whole otherwise, and recorded as{" "} + Dropped on Select with a reason that starts{" "} + left out whole. +
      • +
    + {`left out whole: a scope forced beneath it cannot be applied to what it holds +left out whole: it is a column, which EF Core reads whole +left out whole: it is a complex property, which EF Core cannot narrow +left out whole: it is stored as JSON, which EF Core cannot narrow +left out whole: the projection builds it in a way the core cannot narrow +left out whole: the projection would add its key 'Contents.Id', which is denied +left out whole: the core cannot project 'Contents.Code' +left out whole: nothing beneath it may be selected +left out whole: it can hold what the policy cannot name`} +

    + The last one is recorded on the field beneath the member that the + narrowing leaves out. +

    + +

    Never kept

    +
      +
    • + An entity's navigation, included or not: projecting it would load + it. So once a denial needs a projection, an included or automatically + included navigation is not returned. Name it in Selects{" "} + to get it, narrowed. +
      • +
    • + An object held by a row in memory: a kept object is the caller's + own, and a transform beneath it would change it in place. The + projection's rows are new and hold no member of an object type, + so the source objects are left as they were. +
      • +
    • + A member with no setter, and a member named with one of the + expression parser's own words. +
      • +
    + +

    Other rules

    +
      +
    • + A narrowed reference that is null in the source comes back as an empty + object, as it does for a caller's own dotted{" "} + Selects. +
      • +
    • + A narrowed member carries every allowed field beneath it. An entity + reached beneath it therefore has its own navigations projected, and so + loaded, whether or not the source included them, exactly as when{" "} + Selects names the member. +
      • +
    • + Each denied field whose value can reach the result, at the top or + beneath, is recorded as Dropped on Select. +
      • +
    • + It never throws for a denied field, in either tier. It throws{" "} + AllSelectsDenied only when no field is left. When nothing + asks for a projection, Selects stays null and the query is + the one an unguarded call runs. +
      • +
    • + A typed query projects into T, so T needs a + public parameterless constructor, or the query fails with{" "} + SelectTypeMustHaveParameterlessConstructor. The dynamic + terminals do not need one. +
      • +
    • + A dry run synthesizes nothing. It records the denials and returns the + rows whole. +
      • +
    • + A simulation has no source, so it reads T as a source it + cannot see into: every denial beneath a member counts, and the + projection it shows keeps only members holding a value — see{" "} + Simulate. +
      • +
    + + A member typed object is opaque to the policy: a + synthesized projection leaves it out, and naming it returns whatever it + holds. A framework generic holding a policed type, such as{" "} + Dictionary<string, LineDto>, has no paths beneath it: + naming it is refused under Strict and narrowed away under{" "} + Convenience, or refused there too where the member cannot + be narrowed, and a synthesized projection leaves it out. + Hold such values in a list of the policed type instead. + + + A forced scope declared on a list's element type filters the rows + that hold the list, never its elements. Selects naming the + list returns every element, those the scope excludes included, as in + every release; a synthesized projection leaves such a list out. Scope + the elements where the row is built. + +

    The trace on a result

    Every guarded query records a PolicyTrace: what the policy diff --git a/OfficialWebsite/app/docs/policies/page.tsx b/OfficialWebsite/app/docs/policies/page.tsx index ef2ef98..eb6b86e 100644 --- a/OfficialWebsite/app/docs/policies/page.tsx +++ b/OfficialWebsite/app/docs/policies/page.tsx @@ -95,6 +95,45 @@ public class Employee public JsonDocument? WorkSchedule { get; set; } }`} +

    The guarded handle

    +

    + ApplyPolicy returns a PolicyQueryable<T>. + Its terminals mirror the core's: ToList,{" "} + ToListAsync, ToListDynamic and{" "} + ToListAsyncDynamic with a Filter,{" "} + ToList and ToListAsync with a{" "} + Summary, and ToListAsync with a{" "} + Segment. Each one sanitizes the request, runs it, and + transforms the rows. +

    +

    + Since 3.2.0 every asynchronous terminal on the handle also has + overloads that take a CancellationToken, as the{" "} + core's do. The + policy is applied first, so a refusal is thrown whatever the token + says. The token then reaches the count and the read. The overloads sit + beside the 3.1 signatures, which are unchanged. +

    + {`// PolicyQueryable: new in 3.2.0, beside the overloads without a token +Task> ToListAsync(Filter filter, CancellationToken cancellationToken) +Task> ToListAsync(Filter filter, bool getQueryString, CancellationToken cancellationToken) +Task> ToListAsyncDynamic(Filter filter, CancellationToken cancellationToken) +Task> ToListAsyncDynamic(Filter filter, bool getQueryString, CancellationToken cancellationToken) +Task ToListAsync(Summary summary, CancellationToken cancellationToken) +Task ToListAsync(Summary summary, bool getQueryString, CancellationToken cancellationToken) +Task> ToListAsync(Segment segment, CancellationToken cancellationToken)`} + {`// In a minimal API, a CancellationToken parameter is the request's own: +// a client that disconnects cancels the count or the read. +var result = await db.Employees.ApplyPolicy(caller).ToListAsync(filter, cancellationToken);`} + + default fits both bool getQueryString and{" "} + CancellationToken, on the handle as on the core, so the + call is ambiguous. The same goes for{" "} + ToListAsyncDynamic(filter, default) and{" "} + ToListAsync(summary, default). Write false, a + token, or a named argument. + +

    Six features, per field

    Every decision is made for one field and one feature:{" "} diff --git a/OfficialWebsite/app/docs/policies/security/page.tsx b/OfficialWebsite/app/docs/policies/security/page.tsx index ef0580a..cc06fd8 100644 --- a/OfficialWebsite/app/docs/policies/security/page.tsx +++ b/OfficialWebsite/app/docs/policies/security/page.tsx @@ -6,7 +6,7 @@ import Callout from "@/components/Callout"; export const metadata: Metadata = { title: "Security & k-anonymity — the eight inference channels", - description: "How DynamicWhere.ex closes the disclosure channels no per-field rule closes on its own: set-operation reconstruction, singleton-group aggregates, cardinality probes, sort-and-page binary search, SQL leakage, and schema probing through refusals.", + description: "How DynamicWhere.ex closes the disclosure channels no per-field rule closes on its own — set-operation reconstruction, singleton-group aggregates, cardinality probes, sort-and-page binary search, SQL leakage, and schema probing through refusals — and the denials the gate could not see until 3.2.0.", keywords: ["k-anonymity", "MinGroupSize", "inference attack", "data disclosure", "aggregate disclosure", "EF Core security"], alternates: { canonical: "https://doc.dynamicwhere.com/docs/policies/security/" }, }; @@ -18,8 +18,9 @@ export default function Page() {

    Denying a field is easy. The hard part is the set of ways a caller can learn a value without reading it. Six such channels follow, then - two bypasses that are not channels; each has a test that reproduces the - attack and goes red if the control is removed. + two bypasses that are not channels, then the requests that, until + 3.2.0, carried out a denied value the gate could not see; each has a + test that reproduces the attack and goes red if the control is removed.

    @@ -198,6 +199,102 @@ true order. Add [DwNoOrder] unless that is intended.`} +

    Denials the gate could not see

    +

    + A denied field often sits on a type the query reaches through a member: + a secret on each line of an order, a code inside a nested object. The + denial holds on every path that reaches it, and it has to hold whether + or not the caller names the member. Until 3.2.0 each request below + carried a denied value out. All are closed. +

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    AttackControl
    Send no Selects, on a type whose only denied fields sit beneath a member + A guarded query synthesizes a projection whenever the denied value + can reach the result, and narrows the member around it or leaves + the member out. Only a simple field denied at the top of{" "} + T used to synthesize one, so the whole row came back + with the denied value in it: in a list or nested object of a row + projected before ApplyPolicy, in a row held in memory, + and in an entity's included, automatically included, lazily + loaded or owned member — in both tiers. A denial beneath a + navigation nothing loads never leaves the database, so it asks for + nothing. See{" "} + A request that sends no Selects. +
    Send no Selects, on a type whose denied field holds no simple value: a blob, a list, an owned object, a JSON column + A field denied at the top of T asks for the projection + whatever it holds. Such a field used to be passed over, so with + nothing else denied the whole row came back with it. +
    Name a navigation whose key, Id, is denied + Refused with FieldDeniedForSelect in both tiers. The + core's typed projection adds the key of every nested node it + builds, so the convenience tier used to narrow the key away and + get it back. A navigation named through another,{" "} + Main.Lead, now gates the key of Main as + well, which the projection adds. +
    Name a member typed IReadOnlyList<T>, or another collection the core does not unwrap, with a denied field beneath it + Refused with FieldDeniedForSelect in both tiers. The + projection gate now reads collections the way the attribute walker + does. It used to read them through a narrower list, found nothing + beneath such a member, and returned every field, the denied ones + included, in both tiers. +
    Name a member whose type holds a denied field no path reaches: deeper than four segments, or inside a framework generic such as Dictionary<string, T> + Refused under Strict, and narrowed away under{" "} + Convenience, or refused there too where the member + cannot be narrowed. The gate also reads the rules themselves, + so a denied property with no setter and a rule on a path reached + through a cycle are found beneath a named member too. +
    Put the policed type in an application namespace that starts with System, such as SystemsCorp.Payroll + Policed. The walker read any namespace starting with{" "} + System as the framework's and put no policy + beneath its types, so a [DwDenied] field there was + returned, filterable and sortable. Only System and the + namespaces beneath it are the framework's now. +
    + + A member typed object is opaque to the policy: a + synthesized projection leaves it out, and naming it returns whatever it + holds. A framework generic holding a policed type, such as{" "} + Dictionary<string, LineDto>, has no paths beneath it: + naming it is refused under Strict and narrowed away under{" "} + Convenience, or refused there too where the member cannot + be narrowed, and a synthesized projection leaves it out. + Hold such values in a list of the policed type instead. + + + A forced scope declared on a list's element type filters the rows + that hold the list, never its elements. Selects naming the + list returns every element, those the scope excludes included, as in + every release; a synthesized projection leaves such a list out. Scope + the elements where the row is built. + +

    Getting the posture right

    • Use DwTier.Strict unless you need getQueryString.
      • @@ -208,6 +305,8 @@ true order. Add [DwNoOrder] unless that is intended.`}
    • Run DwPolicy.ValidateModel(...) at startup and treat its warnings as a checklist.
    • Put [DwEntity(RequirePolicy = true)] on anything sensitive, so a DynamicWhere call that forgets ApplyPolicy fails loudly.
    • Prefer [DwOperators] over allowing free filtering on a protected field.
      • +
    • Hold a policed type in a list, never in a dictionary, another framework generic or a member typed object. The policy has no paths into any of them.
      • +
    • Scope a list's elements where the row is built. A forced scope on the element type filters the rows that hold the list, never the elements.
    • Set DwCaps.DefaultPageSize if the API does not page for itself. It ships off, and the request MaxPageSize never bounded is the one that sent no page at all.
    • Keep DwCaps.MaxConditionSets near the number of sets your clients really send. A set with no conditions passes every other cap, and every set adds a condition or a subquery to the statement a segment becomes.
    diff --git a/OfficialWebsite/app/page.tsx b/OfficialWebsite/app/page.tsx index 1395e6a..f508894 100644 --- a/OfficialWebsite/app/page.tsx +++ b/OfficialWebsite/app/page.tsx @@ -219,7 +219,7 @@ export default function HomePage() { Everything you need to query dynamically

    - Four packages. Seventeen extension methods. Three composable shapes (Filter, Segment, Summary), and a policy layer deciding who sees what. + Four packages. Twenty-eight extension methods. Three composable shapes (Filter, Segment, Summary), and a policy layer deciding who sees what.

    @@ -309,7 +309,7 @@ export default function HomePage() { {[ { title: "Installation", desc: "Add to your project in seconds.", href: "/docs/installation" }, { title: "Quick Start", desc: "A working filter in 30 lines.", href: "/docs/quick-start" }, - { title: "Extension Methods", desc: "All 17 methods, one place.", href: "/docs/extensions" }, + { title: "Extension Methods", desc: "All 28 methods, one place.", href: "/docs/extensions" }, { title: "JSON Cookbook", desc: "13 copy-pasteable examples.", href: "/docs/examples" }, { title: "Enums Reference", desc: "Every DataType & Operator.", href: "/docs/enums" }, { title: "Classes Reference", desc: "Condition → Filter → Result.", href: "/docs/classes" }, diff --git a/OfficialWebsite/public/llms.txt b/OfficialWebsite/public/llms.txt index dfe0299..9c8fc14 100644 --- a/OfficialWebsite/public/llms.txt +++ b/OfficialWebsite/public/llms.txt @@ -2283,7 +2283,7 @@ SELECT a navigation that cannot be narrowed around a denied the core cannot project (3.2.0) throw throw FieldDeniedForSelect SELECT a.b when the key a.Id is denied throw throw FieldDeniedForSelect every requested SELECT dropped throw - AllSelectsDenied -no Selects while a field is denied for Select, at any depth allowed members allowed members +no Selects while a denied field can reach the result (3.2.0) allowed members allowed members GROUP BY a denied field throw throw FieldDeniedForGroup AGGREGATE a denied field, or a transformed field lacking AllowAggregate on a stage throw throw FieldDeniedForAggregate @@ -4898,7 +4898,8 @@ MemoryCalculationInput whatever it holds. A framework generic holding a policed type (`Dictionary`) has no paths beneath it: naming it is refused under Strict and narrowed away under Convenience, and a synthesized projection leaves it out. -- A simulation reads T as an entity query (section 23). +- A simulation reads T as a source it cannot see into (section 23): every denial beneath a member counts, and a + synthesized `Clause.Selects` keeps only members holding a value. --- diff --git a/README.md b/README.md index 25e026e..0ffa896 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,7 @@ Stop concatenating LINQ predicates by hand. Your front-end sends one JSON shape; - **JSON in → `IQueryable` out.** No string LINQ. No manual expression trees. - **Three composable shapes** — `Filter`, `Segment`, `Summary` — cover where, set operations, and group-by reporting. -- **Twenty-one extension methods** on `IQueryable` and `IEnumerable`. +- **Twenty-eight extension methods** on `IQueryable` and `IEnumerable`, every async one with overloads that take a `CancellationToken`. - **Nested navigation** through references and collections, with auto-wrapped `.Any()` lambdas where needed. - **Heterogeneous `Condition.Values`** — pass raw numbers, booleans, strings; normalized per `DataType`. - **Thread-safe reflection cache** with FIFO / LRU / LFU eviction and five tuned presets. @@ -140,7 +140,7 @@ That's the whole loop. Full walk-through in **[Quick Start](https://doc.dynamicw | **[`Segment`](https://doc.dynamicwhere.com/docs/classes/segment)** | set1 ∪/∩/∖ set2 ∪/∩/∖ set3 → order → page | UNION / INTERSECT / EXCEPT across multiple condition sets | | **[`Summary`](https://doc.dynamicwhere.com/docs/classes/summary)** | where → group → having → order → page | Aggregate reporting (`GROUP BY` + `SUM` / `AVG` / `COUNT` …) | -### Twenty-one extension methods +### Twenty-eight extension methods Projection, filtering, composition, and materialization on `IQueryable` and `IEnumerable`: @@ -151,6 +151,8 @@ Projection, filtering, composition, and materialization on `IQueryable` and ` | **Composition** | `.Order` · `.Page` · `.Group` · `.Filter` · `.FilterDynamic` · `.Summary` | | **Materialization** | `.ToList(Filter)` · `.ToListAsync(Filter)` · `.ToListDynamic(Filter)` · `.ToListAsyncDynamic(Filter)` · `.ToList(Summary)` · `.ToListAsync(Summary)` · `.ToListAsync(Segment)` | +Every async terminal also has overloads that take a `CancellationToken`, which reaches the count and the read. `ToList(Filter)`, `ToListDynamic(Filter)` and `ToList(Summary)` also run on an `IEnumerable`. + Full signatures, validations, and return types → **[Extension Methods Reference](https://doc.dynamicwhere.com/docs/extensions)**. ### Operators & data types @@ -363,7 +365,7 @@ The complete reference — every enum, class, extension method, validation rule, | [Getting Started](https://doc.dynamicwhere.com/docs) | Introduction, installation, quick start | | [Enums](https://doc.dynamicwhere.com/docs/enums) | Every DataType, Operator, Connector, Direction, Intersection, Aggregator, Cache enum | | [Classes](https://doc.dynamicwhere.com/docs/classes) | Condition, ConditionGroup, ConditionSet, OrderBy, GroupBy, AggregateBy, PageBy, Filter, Segment, Summary, Result types | -| [Extension Methods](https://doc.dynamicwhere.com/docs/extensions) | All 17 methods with signatures, validations, examples | +| [Extension Methods](https://doc.dynamicwhere.com/docs/extensions) | All 28 methods with signatures, validations, examples | | [Validation Rules](https://doc.dynamicwhere.com/docs/validation) | What's checked and what throws | | [JSON Cookbook](https://doc.dynamicwhere.com/docs/examples) | 13 copy-pasteable end-to-end examples | | [Field-Level Policies](https://doc.dynamicwhere.com/docs/policies) | Attributes, precedence, masking, dynamic rules, admin API, k-anonymity | @@ -373,6 +375,21 @@ The complete reference — every enum, class, extension method, validation rule, --- +## Version 3.2.0 highlights + +**Upgrade note — the security fixes and the three changes alter what code written for 3.1.0 does, and the new overloads can stop a call from compiling. Read these before bumping.** + +- **Fixed (security): a field denied beneath a member reached a caller who sent no `Selects`.** A guarded query synthesizes a projection for a denied field, and it did so only when a simple field at the top of the type was denied. With every denial beneath a member, the whole row came back with the denied value in it: in a list or nested object of a row projected before `ApplyPolicy`, in a row held in memory, and in an entity's included, automatically included, lazily loaded or owned member — typed and dynamic, in both tiers, for a `Filter` and a `Segment`. Such a denial now synthesizes the projection whenever its value can reach the result. On an entity that means beneath a column, an owned or complex member, or a navigation the query loads through `Include`, an automatic include or a lazy loader. A denial beneath a navigation nothing loads never leaves the database, and the entity is read exactly as before. +- **Fixed (security): a denied member that holds no simple value came back.** A field denied at the top of the type whose own type is not a simple value — a byte array, a list, an owned object, a JSON column — synthesized no projection either, so with nothing else denied it came back. +- **Fixed (security): an application namespace starting with `System` got no policy.** The attribute walker read any namespace starting with "System" as the framework's, so an application namespace such as `SystemsCorp.Payroll` got no policy beneath its types, and a `[DwDenied]` field there was returned, filterable and sortable. Only `System` and the namespaces beneath it are the framework's now. +- **Fixed (security): a navigation narrowed around its own denied key got the key back.** Under the convenience tier, `Selects` naming a navigation whose key (`Id`) is denied was narrowed to the allowed fields beneath it, and the core's typed projection added the key back. Such a narrowing is refused with `FieldDeniedForSelect` in both tiers, as naming a sibling of the key already was. A navigation named through another, such as `Main.Lead`, now gates the key of `Main`, which the projection adds; it did not. +- **Fixed (security): `Selects` could name a member whose denials the gate did not see.** A member typed as a collection the core does not unwrap — `IReadOnlyList`, `IReadOnlyCollection`, `Collection` or an application's own — returned every field beneath it, denied ones included, in both tiers, because the projection gate read collections through a narrower list than the attribute walker. It reads them the same way now, and a narrowing the core cannot project is refused with `FieldDeniedForSelect`. Denials beneath a named member are also read from the policy's own rules, so a denied property with no setter and a rule on a path reached through a cycle are found. A field denied deeper than the walker reaches, or inside a framework collection such as `Dictionary`, is refused under the strict tier and narrowed away under the convenience tier, or refused there too where the member cannot be narrowed. +- **Changed: the synthesized projection keeps what the source carries.** It kept simple fields only, so every nested object and list of a row projected before `ApplyPolicy` came back null or empty as soon as any field was denied. A row a projection builds — the outermost `Select` constructs it, in an object initializer or with a constructor, as in `db.Roles.Select(r => new RoleRow { … })` — keeps the members its initializer assigns. An entity, or a `Select` that hands back an entity such as `db.Orders.Select(o => o.Customer)`, keeps its mapped columns, converted and JSON ones included, its owned and complex members, and every collection of simple values such as `byte[]` or `List`. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, recorded as `Dropped` with a reason starting `left out whole`. An entity's navigations, the objects of a row in memory, and a value EF Core does not map are left out, and the type needs a public parameterless constructor for the projection, as it already did. +- **Changed: `[DwEntity(DefaultOrder)]` reaches a projection that builds the row.** A guarded query over a projected source takes the default when the outermost `Select` builds the type in an object initializer and assigns every field the default names, at every level of a nested path, with nothing EF Core would compute on the client: `Select(t => new TicketRow { Id = t.Id, CreatedAt = t.CreatedAt })` for `"CreatedAt desc, Id"`. Any other projection still leaves the query in its own order. A `Select`, or a `Filter` with `Selects`, composed on the guarded handle keeps the rest of the chain unordered, and a composed `Filter` that sent orders gets no default later in the chain, as a composed `Order` already did not. +- **Changed: the async dynamic `Filter` and the async `Summary` read through EF Core.** `ToListAsyncDynamic` and `ToListAsync(Summary)` read with EF Core's `ToListAsync`, and the summary counts with `CountAsync`. They used to read synchronously on a thread-pool thread, so on an EF Core query a canceled token now reaches the database. A provider that is not EF Core's keeps the synchronous read. +- **New: a `CancellationToken` on every async terminal**, guarded and unguarded: `ToListAsync` and `ToListAsyncDynamic` with a `Filter`, `ToListAsync` with a `Summary`, and `ToListAsync` with a `Segment`. The token reaches the count and the read. The overloads sit beside the 3.1 signatures, which are unchanged, so code compiled against 3.1 still binds. `ToListAsync(filter, default)`, `ToListAsyncDynamic(filter, default)` and `ToListAsync(summary, default)` no longer compile, because `default` fits both `getQueryString` and the token: write `false`, a token, or a named argument. A reflection lookup of `ToListAsyncDynamic` by name alone now finds three methods where it found one. +- **Known limits.** Once a projection is needed, an entity's navigations are left out, included ones too; name one in `Selects` to get it, narrowed. A forced scope declared on a list's element type filters the rows that hold the list, never its elements, so `Selects` naming the list returns every element and a synthesized projection leaves the list out; scope the elements where the row is built. A member typed `object` is opaque to the policy, and a framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: hold such values in a list of the policed type. `/simulate` has no source, so it reads the type as one it cannot see into: every denial beneath a member counts, and the projection it shows keeps only members holding a value. + ## Version 3.1.0 highlights **Upgrade note — eleven behaviour changes, listed first. Read these before bumping.** From 6c9e171c736454ac69d3e789282823054578337a Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 06:47:05 +0300 Subject: [PATCH 11/22] fix(policies): read what a query loads from the model, and what a member can hold A security re-review of b71b8c4 found nine ways a value denied for Select still reached the result. Four were new in 3.2.0, members it kept whole that 3.1.0 dropped; five were already in 3.1.0. A docs review found a tenth and an over-block. All fixed: - A member declared as a base type or interface holds its subtypes. Their denied fields are read too: every loaded subtype for a projected or in-memory row, the model's derived types for an entity. A hierarchy root with a derived type's denial is projected to the root type; an abstract one is refused. - A "*" deny with exact allows denies every path the walk never asks about: past four segments, around a cycle, a property with no setter. Such a member is never kept whole, and triggers the projection when its value loads. - A rule spelled in another letter case reaches a projected row's assigned member. - A collection that is not generic, and a framework interface, can hold anything. - An include named from another root, re-rooted by Select, SelectMany or Join, and a projection hidden behind another Select, count as loading every navigation. - An initializer after a constructor with arguments counts every member as assigned. - A lazy loader the constructor takes, kept in a field or any property, counts. - What a named or included entity navigation carries is read from the model: its columns (a converted Dictionary of a policed type included), owned chain at any depth, and what loads beneath it. A denial beneath a navigation nothing loads no longer asks for a projection, which had dropped every Include in a connected model. - A member the model does not map holds nothing EF Core read, and asks for nothing. - DefaultOrder applies only to a column: Regex.Replace and other framework calls made the guarded query throw where the unguarded one ran. Co-Authored-By: Claude Opus 5 --- .../Policies/ProjectionReachTests.cs | 1119 +++++++++++++++++ .../Policies/ProjectionReviewTests.cs | 25 + .../Policies/Source/DefaultOrder.cs | 128 +- .../Policies/Source/FilterSanitizer.cs | 337 ++++- .../Policies/Source/KnownSubtypes.cs | 100 ++ DynamicWhere.ex/Policies/Source/RowShape.cs | 278 +++- DynamicWhere.ex/Source/QueryRoot.cs | 75 +- 7 files changed, 1936 insertions(+), 126 deletions(-) create mode 100644 DynamicWhere.Tests/Policies/ProjectionReachTests.cs create mode 100644 DynamicWhere.ex/Policies/Source/KnownSubtypes.cs diff --git a/DynamicWhere.Tests/Policies/ProjectionReachTests.cs b/DynamicWhere.Tests/Policies/ProjectionReachTests.cs new file mode 100644 index 0000000..ac9277e --- /dev/null +++ b/DynamicWhere.Tests/Policies/ProjectionReachTests.cs @@ -0,0 +1,1119 @@ +using System.Collections; +using System.ComponentModel.DataAnnotations.Schema; +using System.Text.Json; +using System.Text.Json.Serialization; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Classes.Core; +using DynamicWhere.ex.Enums; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.DTOs; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; + +namespace DynamicWhere.Tests.Policies +{ + // --------------------------------------------------------------------------------- the database + + public class RcCustomer + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public List Cards { get; set; } = new(); + + public List Orders { get; set; } = new(); + } + + public class RcCard + { + public int Id { get; set; } + + public int RcCustomerId { get; set; } + + public string Label { get; set; } = string.Empty; + + [DwDenied] + public string? Pan { get; set; } + } + + public class RcOrder + { + public int Id { get; set; } + + public string Code { get; set; } = string.Empty; + + public int RcCustomerId { get; set; } + + public RcCustomer? Customer { get; set; } + + public List Lines { get; set; } = new(); + } + + public class RcLine + { + public int Id { get; set; } + + public int RcOrderId { get; set; } + + public string Sku { get; set; } = string.Empty; + + [DwDenied] + public string? Cost { get; set; } + } + + public class RcPerson + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + [DwDenied] + public string? TaxId { get; set; } + } + + public class RcAccount + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public int RcPersonId { get; set; } + + public RcPerson? Person { get; set; } + } + + /// The root of a hierarchy whose derived type declares a denied field. + public class RcParty + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + } + + public class RcCompany : RcParty + { + [DwDenied] + public string? TaxSecret { get; set; } + } + + public class RcDeal + { + public int Id { get; set; } + + public string Note { get; set; } = string.Empty; + + public int RcPartyId { get; set; } + + public RcParty? Party { get; set; } + } + + /// An abstract root, which no projection can build. + public abstract class RcAsset + { + public int Id { get; set; } + + public string Label { get; set; } = string.Empty; + } + + public class RcVault : RcAsset + { + [DwDenied] + public string? Combination { get; set; } + } + + /// A value EF Core converts to text, holding a type with a denied field. + public class RcCharge + { + public string Sku { get; set; } = string.Empty; + + [DwDenied] + public string? Amount { get; set; } + } + + public class RcClient + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public Dictionary Charges { get; set; } = new(); + } + + public class RcPurchase + { + public int Id { get; set; } + + public int RcClientId { get; set; } + + public RcClient? Client { get; set; } + } + + /// A navigation whose entity owns a chain deeper than the walker goes. + public class RcHolding + { + public int Id { get; set; } + + public int RcKeeperId { get; set; } + + public RcKeeper? Keeper { get; set; } + } + + public class RcKeeper + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public RcProfile Profile { get; set; } = new(); + } + + public class RcProfile + { + public string A { get; set; } = string.Empty; + + public RcDetail Detail { get; set; } = new(); + } + + public class RcDetail + { + public string B { get; set; } = string.Empty; + + public RcInner Inner { get; set; } = new(); + } + + public class RcInner + { + public string C { get; set; } = string.Empty; + + [DwDenied] + public string? Secret { get; set; } + } + + /// An automatically included navigation with a denied field. + public class RcBadgeHolder + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public int RcBadgeId { get; set; } + + public RcBadge? Badge { get; set; } + } + + public class RcBadge + { + public int Id { get; set; } + + public string Label { get; set; } = string.Empty; + + [DwDenied] + public string? Pin { get; set; } + } + + /// Members EF Core does not map, which hold only what the class puts there. + public class RcBag + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public int RcBagOwnerId { get; set; } + + public RcBagOwner? Owner { get; set; } + + [NotMapped] + public object? Extra { get; set; } + + [NotMapped] + public RcScratch Scratch { get; set; } = new(); + } + + public class RcScratch + { + [DwDenied] + public string? Cost { get; set; } + } + + public class RcBagOwner + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + } + + /// An owned member with a mapped property that has no setter. + public class RcShop + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public string? Notes { get; set; } + + public RcAddress Address { get; set; } = new(); + } + + public class RcAddress + { + public RcAddress() + { + } + + public RcAddress(string city, string? zip) + { + City = city; + Zip = zip; + } + + public string City { get; set; } = string.Empty; + + public string? Zip { get; } + } + + /// An owned member every path of which the walk asks about, and an included navigation. + public class RcStore + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public RcPlace Place { get; set; } = new(); + + public int RcStoreKeeperId { get; set; } + + public RcStoreKeeper? Keeper { get; set; } + } + + public class RcPlace + { + public string City { get; set; } = string.Empty; + + public string Zone { get; set; } = string.Empty; + } + + public class RcStoreKeeper + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + } + + public sealed class ProjectionReachContext : DbContext + { + private readonly SqliteConnection _connection; + + public ProjectionReachContext(SqliteConnection connection) => _connection = connection; + + public DbSet Customers => Set(); + + public DbSet Orders => Set(); + + public DbSet People => Set(); + + public DbSet Accounts => Set(); + + public DbSet Parties => Set(); + + public DbSet Deals => Set(); + + public DbSet Assets => Set(); + + public DbSet Purchases => Set(); + + public DbSet Holdings => Set(); + + public DbSet BadgeHolders => Set(); + + public DbSet Bags => Set(); + + public DbSet Shops => Set(); + + public DbSet Stores => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + protected override void OnModelCreating(ModelBuilder model) + { + model.Entity(); + model.Entity(); + model.Entity().Property(c => c.Charges).HasConversion( + value => JsonSerializer.Serialize(value, (JsonSerializerOptions?)null), + text => JsonSerializer.Deserialize>(text, (JsonSerializerOptions?)null)!); + model.Entity().OwnsOne( + k => k.Profile, p => p.OwnsOne(x => x.Detail, d => d.OwnsOne(y => y.Inner))); + model.Entity().Navigation(h => h.Badge).AutoInclude(); + model.Entity().OwnsOne(s => s.Address, a => a.Property(x => x.Zip)); + model.Entity().OwnsOne(s => s.Place); + } + } + + // --------------------------------------------------------------------------------- lazy loaders + + /// A lazy loader delegate the constructor takes and keeps in a field. + public class RcFieldBlog + { + private readonly Action? _loader; + private List? _posts; + + public RcFieldBlog() + { + } + + private RcFieldBlog(Action lazyLoader) => _loader = lazyLoader; + + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public List Posts + { + get + { + _loader?.Invoke(this, nameof(Posts)); + + return _posts ??= new List(); + } + set => _posts = value; + } + } + + public class RcFieldPost + { + public int Id { get; set; } + + public int RcFieldBlogId { get; set; } + + public string Title { get; set; } = string.Empty; + + [DwDenied] + public string? Draft { get; set; } + } + + /// A lazy loader delegate kept in a property not named LazyLoader. + public class RcNamedBlog + { + private List? _posts; + + public RcNamedBlog() + { + } + + private RcNamedBlog(Action lazyLoader) => Loader = lazyLoader; + + private Action? Loader { get; set; } + + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public List Posts + { + get + { + Loader?.Invoke(this, nameof(Posts)); + + return _posts ??= new List(); + } + set => _posts = value; + } + } + + public class RcNamedPost + { + public int Id { get; set; } + + public int RcNamedBlogId { get; set; } + + public string Title { get; set; } = string.Empty; + + [DwDenied] + public string? Draft { get; set; } + } + + public sealed class ProjectionReachLazyContext : DbContext + { + private readonly SqliteConnection _connection; + + public ProjectionReachLazyContext(SqliteConnection connection) => _connection = connection; + + public DbSet FieldBlogs => Set(); + + public DbSet NamedBlogs => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + protected override void OnModelCreating(ModelBuilder model) + { + model.Entity().Navigation(b => b.Posts).HasField("_posts"); + model.Entity().Navigation(b => b.Posts).HasField("_posts"); + } + } + + // --------------------------------------------------------------------------------- rows + + /// A row whose member a constructor sets, before its initializer runs. + public class RcHolder + { + public RcHolder() + { + } + + public RcHolder(RcPerson person) => Person = person; + + public int Id { get; set; } + + public RcPerson? Person { get; set; } + } + + public class RcDealRow + { + public int Id { get; set; } + + [DwDenied] + public string? Note { get; set; } + + public RcParty? Party { get; set; } + } + + public sealed class RcDeepRow + { + public int Id { get; set; } + + public string? Tenant { get; set; } + + public RcL1? First { get; set; } + } + + public sealed class RcL1 + { + public RcL2? Next { get; set; } + } + + public sealed class RcL2 + { + public RcL3? Next { get; set; } + } + + public sealed class RcL3 + { + public string Name { get; set; } = string.Empty; + + public RcL4? Next { get; set; } + } + + public sealed class RcL4 + { + public string? Secret { get; set; } + } + + /// A row holding policed objects through a collection that is not generic. + public sealed class RcBagRow + { + public int Id { get; set; } + + [DwDenied] + public string? Note { get; set; } + + public IEnumerable? Items { get; set; } + } + + public sealed class RcNode + { + public string Name { get; set; } = string.Empty; + + public RcNode? Next { get; set; } + } + + public sealed class RcLink + { + public int Id { get; set; } + + public RcNode? Head { get; set; } + } + + /// Rows in memory whose subtype declares a denied field. + public class RcAnimal + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + } + + public class RcDog : RcAnimal + { + [DwDenied] + public string? Chip { get; set; } + } + + // --------------------------------------------------------------------------------- the tests + + /// + /// What a denied value can reach, read from what the source actually loads, and what a member can + /// hold that no path names: a derived type's field, a path a policy denying by default never names, + /// a collection that is not generic. Each test asserts the safe outcome, so a failing one is a leak. + /// + public sealed class ProjectionReachTests : IDisposable + { + private static readonly JsonSerializerOptions Json = new() { ReferenceHandler = ReferenceHandler.IgnoreCycles }; + + private readonly SqliteConnection _connection; + private readonly ProjectionReachContext _db; + + public ProjectionReachTests() + { + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + + _db = new ProjectionReachContext(_connection); + _db.Database.EnsureCreated(); + + _db.Customers.Add(new RcCustomer + { + Name = "C1", + Cards = { new RcCard { Label = "visa", Pan = "pan-secret" } }, + Orders = { new RcOrder { Code = "O1", Lines = { new RcLine { Sku = "S1", Cost = "cost-secret" } } } } + }); + _db.Accounts.Add(new RcAccount { Name = "A1", Person = new RcPerson { Name = "P1", TaxId = "tax-secret" } }); + _db.Deals.Add(new RcDeal { Note = "note", Party = new RcCompany { Name = "Acme", TaxSecret = "company-secret" } }); + _db.Assets.Add(new RcVault { Label = "V1", Combination = "combination-secret" }); + _db.Purchases.Add(new RcPurchase + { + Client = new RcClient { Name = "K1", Charges = { ["a"] = new RcCharge { Sku = "S1", Amount = "amount-secret" } } } + }); + _db.Holdings.Add(new RcHolding + { + Keeper = new RcKeeper + { + Name = "O1", + Profile = new RcProfile { A = "a", Detail = new RcDetail { B = "b", Inner = new RcInner { C = "c", Secret = "deep-secret" } } } + } + }); + _db.BadgeHolders.Add(new RcBadgeHolder { Name = "H1", Badge = new RcBadge { Label = "gold", Pin = "pin-secret" } }); + _db.Bags.Add(new RcBag { Name = "B1", Owner = new RcBagOwner { Name = "W1" } }); + _db.Shops.Add(new RcShop { Name = "S1", Notes = "notes", Address = new RcAddress("Basra", "zip-secret") }); + _db.Stores.Add(new RcStore { Name = "T1", Place = new RcPlace { City = "Basra", Zone = "Z1" }, Keeper = new RcStoreKeeper { Name = "K" } }); + + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static DwPolicyContext Caller() => + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1").WithValue("TenantId", 1); + + private static PolicyQueryable Guard(IQueryable source, DwTier tier = DwTier.Strict, params IDwPolicyProvider[] more) + where T : class => + source.ApplyPolicy( + Caller(), + new DwPolicyOptions { Tier = tier, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() }.Concat(more).ToArray())); + + private static Filter Selecting(params string[] fields) => new() { Selects = fields.ToList() }; + + /// A policy that denies Select on every field except those it names. + private static FakePolicyProvider DenyingAllBut(params string[] allowed) + { + FakePolicyProvider rules = new FakePolicyProvider() + .Add("*", PolicyFeature.Select, PolicyEffect.Deny, PolicyLevel.DynamicGlobal); + + foreach (string path in allowed) + { + rules.Add(path, PolicyFeature.Select, PolicyEffect.Allow, PolicyLevel.DynamicGlobal); + } + + return rules; + } + + /// Everything a row holds, as a serializer would send it. + private static string Sent(object? rows) => JsonSerializer.Serialize(rows, Json); + + /// + /// True when anything reachable from a value holds the text, read by each object's runtime type: a + /// serializer that writes the declared type would miss a derived type's field. + /// + private static bool Holds(object? value, string text) + { + HashSet seen = new(ReferenceEqualityComparer.Instance); + Stack pending = new(); + + pending.Push(value); + + while (pending.Count > 0) + { + object? current = pending.Pop(); + + if (current is null || current is ValueType) + { + continue; + } + + if (current is string held) + { + if (held == text) + { + return true; + } + + continue; + } + + if (!seen.Add(current)) + { + continue; + } + + if (current is IEnumerable items) + { + foreach (object? item in items) + { + pending.Push(item); + } + + continue; + } + + foreach (System.Reflection.PropertyInfo property in current.GetType().GetProperties()) + { + if (property.GetIndexParameters().Length == 0 && property.CanRead) + { + pending.Push(property.GetValue(current)); + } + } + } + + return false; + } + + private static void Refused(Action query) + { + PolicyException refusal = Assert.Throws(query); + + Assert.Equal(PolicyErrorCode.FieldDeniedForSelect, refusal.ErrorCode); + } + + // ---------------------------------------------------------------- includes read from another root + + /// + /// A later operator that reaches the rows through a navigation keeps what the includes loaded, named + /// from the query's root: Select(o => o.Customer) after an include of the customer's cards. + /// + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public async Task An_include_named_from_another_root_counts_as_loading_every_navigation(DwTier tier) + { + IQueryable source = _db.Orders + .Include(o => o.Customer).ThenInclude(c => c!.Cards) + .Select(o => o.Customer!); + + Assert.Contains("pan-secret", Sent(source.AsNoTracking().ToList())); + + Assert.DoesNotContain("pan-secret", Sent(Guard(source, tier).ToList(new Filter()).Data)); + Assert.DoesNotContain("pan-secret", Sent(Guard(source, tier).ToListDynamic(new Filter()).Data)); + Assert.DoesNotContain("pan-secret", Sent((await Guard(source, tier).ToListAsync(new Filter(), CancellationToken.None)).Data)); + Assert.DoesNotContain("pan-secret", Sent((await Guard(source, tier).ToListAsync(new Segment(), CancellationToken.None)).Data)); + + IQueryable named = _db.Orders.Include("Customer.Cards").Select(o => o.Customer!); + + Assert.DoesNotContain("pan-secret", Sent(Guard(named, tier).ToList(new Filter()).Data)); + } + + [Fact] + public void An_include_named_from_another_root_is_read_through_SelectMany_and_Join() + { + IQueryable flattened = _db.Customers + .Include(c => c.Orders).ThenInclude(o => o.Lines) + .SelectMany(c => c.Orders); + IQueryable selected = _db.Customers + .Include(c => c.Orders).ThenInclude(o => o.Lines) + .SelectMany(c => c.Orders, (c, o) => o); + IQueryable joined = _db.Orders + .Include(o => o.Customer).ThenInclude(c => c!.Cards) + .Join(_db.Accounts, o => o.Id, a => a.Id, (o, a) => o.Customer!); + + Assert.Contains("cost-secret", Sent(flattened.AsNoTracking().ToList())); + + Assert.DoesNotContain("cost-secret", Sent(Guard(flattened).ToList(new Filter()).Data)); + Assert.DoesNotContain("cost-secret", Sent(Guard(selected).ToList(new Filter()).Data)); + Assert.DoesNotContain("pan-secret", Sent(Guard(joined).ToList(new Filter()).Data)); + } + + // ---------------------------------------------------------------- projections the outer Select hides + + /// + /// A projection behind a Select that builds nothing, a member of an anonymous row or a + /// conditional still loads what it assigns, although the rows are the entity's type. + /// + [Fact] + public void A_projection_behind_another_Select_is_not_read_as_an_entity_query() + { + IQueryable identity = _db.Accounts + .Select(a => new RcAccount { Id = a.Id, Name = a.Name, Person = a.Person }) + .Select(x => x); + IQueryable member = _db.Accounts + .Select(a => new { Row = new RcAccount { Id = a.Id, Name = a.Name, Person = a.Person } }) + .Select(x => x.Row); + IQueryable conditional = _db.Accounts + .Select(a => a.Id > 0 + ? new RcAccount { Id = a.Id, Name = a.Name, Person = a.Person } + : new RcAccount { Id = a.Id, Name = a.Name }); + + Assert.Contains("tax-secret", Sent(identity.ToList())); + + Assert.DoesNotContain("tax-secret", Sent(Guard(identity).ToList(new Filter()).Data)); + Assert.DoesNotContain("tax-secret", Sent(Guard(member).ToList(new Filter()).Data)); + Assert.DoesNotContain("tax-secret", Sent(Guard(conditional).ToList(new Filter()).Data)); + } + + /// + /// A projection beneath a Join does not make the join's rows: they are the result selector's, + /// which here puts the person back. The rows are read as the entity they are, every navigation loaded. + /// + [Fact] + public void A_projection_beneath_a_Join_is_not_the_one_that_makes_the_rows() + { + IQueryable rows = _db.Accounts + .Select(a => new RcAccount { Id = a.Id, RcPersonId = a.RcPersonId }) + .Join(_db.People, a => a.RcPersonId, p => p.Id, (a, p) => new RcAccount { Id = a.Id, Person = p }); + + RowShape shape = RowShape.Of(rows); + + Assert.Equal(RowKind.Entity, shape.Kind); + Assert.True(shape.Materializes("Person.TaxId")); + Assert.Contains("tax-secret", Sent(rows.ToList())); + + // EF Core may refuse to read a member the result selector did not assign; it never returns the value. + Exception? refused = Record.Exception(() => Assert.DoesNotContain("tax-secret", Sent(Guard(rows).ToList(new Filter()).Data))); + + Assert.True(refused is null or InvalidOperationException, refused?.ToString()); + } + + /// A constructor with arguments sets members its initializer never names. + [Fact] + public void An_initializer_after_a_constructor_with_arguments_counts_every_member_as_assigned() + { + IQueryable rows = _db.People.Select(p => new RcHolder(p) { Id = p.Id }); + + Assert.Contains("tax-secret", Sent(rows.ToList())); + Assert.DoesNotContain("tax-secret", Sent(Guard(rows).ToList(new Filter()).Data)); + } + + // ---------------------------------------------------------------- lazy loaders the model keeps no record of + + /// + /// A lazy loader delegate the constructor takes gets no service property, whether the class keeps it + /// in a field or in a property of another name; it still fills the navigation after the query. + /// + [Fact] + public void A_lazy_loader_kept_where_the_model_has_no_record_of_it_counts_as_loading() + { + using SqliteConnection connection = new("DataSource=:memory:"); + connection.Open(); + + using ProjectionReachLazyContext db = new(connection); + db.Database.EnsureCreated(); + db.FieldBlogs.Add(new RcFieldBlog { Name = "B1", Posts = { new RcFieldPost { Title = "T", Draft = "field-draft" } } }); + db.NamedBlogs.Add(new RcNamedBlog { Name = "B2", Posts = { new RcNamedPost { Title = "T", Draft = "named-draft" } } }); + db.SaveChanges(); + db.ChangeTracker.Clear(); + + RcFieldBlog field = Guard(db.FieldBlogs).ToList(new Filter()).Data.Single(); + RcNamedBlog named = Guard(db.NamedBlogs).ToList(new Filter()).Data.Single(); + + Assert.DoesNotContain(field.Posts, post => post.Draft == "field-draft"); + Assert.DoesNotContain(named.Posts, post => post.Draft == "named-draft"); + } + + // ---------------------------------------------------------------- a rule in another letter case + + /// + /// A rule spelled in another letter case, on a path deeper than the walk, reaches a projected row's + /// assigned member: whether or not anything else is denied, it asks for the projection. + /// + [Theory] + [InlineData(false)] + [InlineData(true)] + public void A_rule_in_another_letter_case_reaches_an_assigned_member(bool somethingElseDenied) + { + IQueryable rows = _db.Customers.Select(c => new RcDeepRow + { + Id = c.Id, + Tenant = c.Name, + First = new RcL1 { Next = new RcL2 { Next = new RcL3 { Name = c.Name, Next = new RcL4 { Secret = "camel-secret" } } } } + }); + + FakePolicyProvider rules = new FakePolicyProvider() + .Add("first.next.next.next.secret", PolicyFeature.Select, PolicyEffect.Deny, PolicyLevel.DynamicGlobal); + + if (somethingElseDenied) + { + rules.Add("tenant", PolicyFeature.Select, PolicyEffect.Deny, PolicyLevel.DynamicGlobal); + } + + RcDeepRow row = Guard(rows, DwTier.Strict, rules).ToList(new Filter()).Data.Single(); + + Assert.Null(row.First?.Next?.Next?.Next?.Secret); + Assert.Equal("C1", row.First?.Next?.Next?.Name); + } + + // ---------------------------------------------------------------- what a member can hold + + /// A collection that is not generic holds objects, whatever they carry, so it is never clean. + [Fact] + public void A_collection_that_is_not_generic_is_never_kept_whole() + { + IQueryable rows = _db.Customers.Select(c => new RcBagRow { Id = c.Id, Note = c.Name, Items = c.Cards.ToList() }); + + PolicyQueryable guarded = Guard(rows); + RcBagRow row = guarded.ToList(new Filter()).Data.Single(); + + Assert.Null(row.Items); + Assert.Contains(guarded.LastTrace!.Decisions, d => d.FieldPath == "Items" && d.Action == PolicyAction.Dropped); + } + + /// + /// A member declared as a base type holds a derived entity, whose own denied field the base type + /// never names: a projected row narrows it to the base type rather than keeping it whole. + /// + [Fact] + public void A_member_declared_as_a_base_type_is_narrowed_to_it() + { + IQueryable rows = _db.Deals.Select(d => new RcDealRow { Id = d.Id, Note = d.Note, Party = d.Party }); + + Assert.True(Holds(rows.ToList(), "company-secret")); + + RcDealRow row = Guard(rows).ToList(new Filter()).Data.Single(); + + Assert.False(Holds(row, "company-secret")); + Assert.IsNotType(row.Party); + Assert.Equal("Acme", row.Party!.Name); + } + + /// + /// A query over the root of a hierarchy returns each row as its derived type. A derived type's denied + /// field asks for the projection, which builds the root type. + /// + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void A_denied_field_of_a_derived_type_projects_the_root(DwTier tier) + { + PolicyQueryable guarded = Guard(_db.Parties, tier); + RcParty party = guarded.ToList(new Filter()).Data.Single(); + + Assert.IsNotType(party); + Assert.Equal("Acme", party.Name); + Assert.Contains(guarded.LastTrace!.Decisions, d => d.FieldPath == "TaxSecret" && d.Action == PolicyAction.Dropped); + } + + /// An abstract root cannot be built, so a query that needs a projection over one is refused. + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void An_abstract_root_that_needs_a_projection_is_refused(DwTier tier) + { + Refused(() => Guard(_db.Assets, tier).ToList(new Filter())); + + // Its concrete type can be queried, and is projected like any other. + Assert.Null(Guard(_db.Assets.OfType(), tier).ToList(new Filter()).Data.Single().Combination); + } + + /// Rows in memory can be any loaded subtype, and are projected to the rows' type. + [Fact] + public void Rows_in_memory_of_a_subtype_with_a_denied_field_are_projected() + { + RcAnimal[] rows = { new RcDog { Id = 1, Name = "Rex", Chip = "chip-secret" } }; + + RcAnimal row = Guard(rows.AsQueryable()).ToList(new Filter()).Data.Single(); + + Assert.IsNotType(row); + Assert.Equal("Rex", row.Name); + } + + /// + /// A navigation named in Selects, or included, declared as the root of a hierarchy, holds the + /// derived type's denied field. Strict refuses naming it; Convenience narrows it to the root type. + /// An included one is left out of the projection it asks for. + /// + [Fact] + public void A_navigation_declared_as_a_base_type_carries_the_derived_type() + { + Refused(() => Guard(_db.Deals, DwTier.Strict).ToList(Selecting("Id", "Party"))); + + RcDeal named = Guard(_db.Deals, DwTier.Convenience).ToList(Selecting("Id", "Party")).Data.Single(); + RcDeal included = Guard(_db.Deals.Include(d => d.Party)).ToList(new Filter()).Data.Single(); + + Assert.True(Holds(_db.Deals.Include(d => d.Party).AsNoTracking().ToList(), "company-secret")); + Assert.Equal("Acme", named.Party!.Name); + Assert.False(Holds(named, "company-secret")); + Assert.False(Holds(included, "company-secret")); + } + + // ---------------------------------------------------------------- a policy that denies what it does not name + + /// + /// Under a "*" deny, a path the walk never asks about is denied: one around a cycle, one + /// deeper than the walk, and a property with no setter. + /// + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Under_a_deny_by_default_policy_a_member_named_is_never_kept_whole(DwTier tier) + { + RcLink[] links = { new() { Id = 1, Head = new RcNode { Name = "n1", Next = new RcNode { Name = "n2-secret" } } } }; + + // In memory a member cannot be narrowed, so it is refused in both tiers. + Refused(() => Guard(links.AsQueryable(), tier, DenyingAllBut("Id", "Head", "Head.Name")).ToList(Selecting("Id", "Head"))); + + FakePolicyProvider shop = DenyingAllBut("Id", "Name", "Address", "Address.City"); + + if (tier == DwTier.Strict) + { + Refused(() => Guard(_db.Shops, tier, shop).ToList(Selecting("Id", "Address"))); + } + else + { + RcShop row = Guard(_db.Shops, tier, shop).ToList(Selecting("Id", "Address")).Data.Single(); + + Assert.Equal(("Basra", (string?)null), (row.Address.City, row.Address.Zip)); + } + } + + [Fact] + public void Under_a_deny_by_default_policy_a_synthesized_projection_narrows_what_the_walk_misses() + { + IQueryable deep = _db.Customers.Select(c => new RcDeepRow + { + Id = c.Id, + Tenant = c.Name, + First = new RcL1 { Next = new RcL2 { Next = new RcL3 { Name = c.Name, Next = new RcL4 { Secret = "deep-secret" } } } } + }); + + RcDeepRow row = Guard(deep, DwTier.Strict, DenyingAllBut("Id", "First", "First.Next", "First.Next.Next", "First.Next.Next.Name")) + .ToList(new Filter()).Data.Single(); + + Assert.Null(row.First?.Next?.Next?.Next?.Secret); + Assert.Equal("C1", row.First?.Next?.Next?.Name); + + // Notes denied, and with nothing else denied: the owned address is narrowed either way. + foreach (FakePolicyProvider rules in new[] + { + DenyingAllBut("Id", "Name", "Address", "Address.City"), + DenyingAllBut("Id", "Name", "Notes", "Address", "Address.City") + }) + { + RcShop shop = Guard(_db.Shops, DwTier.Strict, rules).ToList(new Filter()).Data.Single(); + + Assert.Equal(("Basra", (string?)null), (shop.Address.City, shop.Address.Zip)); + } + } + + /// + /// A deny-by-default policy that names every path of an owned member asks for nothing: the entity is + /// read as loaded, its included navigation with it. + /// + [Fact] + public void Under_a_deny_by_default_policy_a_member_the_walk_covers_is_kept() + { + FakePolicyProvider rules = DenyingAllBut( + "Id", "Name", "Place", "Place.City", "Place.Zone", "RcStoreKeeperId", "Keeper", "Keeper.Id", "Keeper.Name"); + + PolicyQueryable guarded = Guard(_db.Stores.Include(s => s.Keeper), DwTier.Strict, rules); + RcStore store = guarded.ToList(new Filter()).Data.Single(); + + Assert.Equal(("Z1", "K"), (store.Place.Zone, store.Keeper?.Name)); + Assert.DoesNotContain(guarded.LastTrace!.Decisions, d => d.Action == PolicyAction.Dropped); + } + + // ---------------------------------------------------------------- what a navigation's entity loads + + /// + /// Naming a navigation loads its entity whole, its owned chain included, however deep. Strict refuses + /// it; Convenience narrows it to the paths the walk can name. + /// + [Fact] + public void A_navigation_named_whose_owned_chain_holds_a_denial_past_the_walk_is_not_returned_whole() + { + Refused(() => Guard(_db.Holdings, DwTier.Strict).ToList(Selecting("Id", "Keeper"))); + + RcHolding holding = Guard(_db.Holdings, DwTier.Convenience).ToList(Selecting("Id", "Keeper")).Data.Single(); + + Assert.Equal("O1", holding.Keeper!.Name); + Assert.DoesNotContain("deep-secret", Sent(holding)); + } + + /// A column of a navigation's entity holding a framework collection of a policed type. + [Fact] + public void A_navigation_named_whose_column_holds_a_policed_type_is_not_returned_whole() + { + Assert.Contains("amount-secret", Sent(_db.Purchases.Include(p => p.Client).ToList())); + + Refused(() => Guard(_db.Purchases, DwTier.Strict).ToList(Selecting("Id", "Client"))); + Refused(() => Guard(_db.Purchases, DwTier.Strict).ToList(Selecting("Id", "Client.Charges"))); + + // Convenience narrows the navigation around the column, and the column itself to nothing. + RcPurchase purchase = Guard(_db.Purchases, DwTier.Convenience).ToList(Selecting("Id", "Client")).Data.Single(); + PolicyQueryable guarded = Guard(_db.Purchases, DwTier.Convenience); + RcPurchase column = guarded.ToList(Selecting("Id", "Client.Charges")).Data.Single(); + + Assert.Equal("K1", purchase.Client!.Name); + Assert.False(Holds(purchase, "amount-secret")); + Assert.False(Holds(column, "amount-secret")); + Assert.Contains(guarded.LastTrace!.Decisions, d => d.FieldPath == "Client.Charges" && d.Action == PolicyAction.Dropped); + } + + /// + /// An included navigation whose own navigation holds the denial, unloaded, asks for nothing: the + /// entity comes back as loaded, the included navigation with it. + /// + [Fact] + public void An_included_navigation_with_a_denial_only_beneath_what_it_does_not_load_is_kept() + { + PolicyQueryable guarded = Guard(_db.Orders.Include(o => o.Customer)); + RcOrder order = guarded.ToList(new Filter()).Data.Single(); + + Assert.Equal("C1", order.Customer?.Name); + Assert.DoesNotContain(guarded.LastTrace!.Decisions, d => d.Action == PolicyAction.Dropped); + } + + /// An automatically included navigation loads with every query, and so does its denied field. + [Fact] + public void An_automatically_included_navigation_counts_as_loaded() + { + Assert.Contains("pin-secret", Sent(_db.BadgeHolders.ToList())); + + PolicyQueryable guarded = Guard(_db.BadgeHolders); + RcBadgeHolder holder = guarded.ToList(new Filter()).Data.Single(); + + Assert.DoesNotContain("pin-secret", Sent(holder)); + Assert.Contains(guarded.LastTrace!.Decisions, d => d.FieldPath == "Badge.Pin" && d.Action == PolicyAction.Dropped); + } + + /// + /// A member EF Core does not map holds only what the class puts there, never a value the query read, + /// so neither an object nor a policed type there asks for a projection. + /// + [Fact] + public void A_member_the_model_does_not_map_asks_for_nothing() + { + PolicyQueryable guarded = Guard(_db.Bags.Include(b => b.Owner)); + RcBag bag = guarded.ToList(new Filter()).Data.Single(); + + Assert.Equal("W1", bag.Owner?.Name); + Assert.DoesNotContain(guarded.LastTrace!.Decisions, d => d.Action == PolicyAction.Dropped); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs b/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs index d633612..69b4433 100644 --- a/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs +++ b/DynamicWhere.Tests/Policies/ProjectionReviewTests.cs @@ -1,4 +1,5 @@ using System.ComponentModel.DataAnnotations.Schema; +using System.Text.RegularExpressions; using DynamicWhere.ex.Classes.Complex; using DynamicWhere.ex.Classes.Core; using DynamicWhere.ex.Enums; @@ -698,6 +699,30 @@ public void A_default_computed_by_an_application_method_is_not_applied() private static string Decorate(string code) => "[" + code + "]"; + /// + /// A default applies only to a column: a framework method EF Core cannot translate, such as + /// Regex.Replace, computes the field on the client, and ordering by it would make the guarded + /// query fail where the unguarded one ran. A column read directly, through a navigation or through + /// EF.Property is ordered by. + /// + [Fact] + public void A_default_applies_only_to_a_column_the_projection_assigns() + { + IQueryable replaced = _db.Roles.Select(r => new RvLabelRow { Id = r.Id, Label = Regex.Replace(r.Code, "R", "X") }); + IQueryable joined = _db.Roles.Select(r => new RvLabelRow { Id = r.Id, Label = r.Code + "!" }); + IQueryable direct = _db.Roles.Select(r => new RvLabelRow { Id = r.Id, Label = r.Code }); + IQueryable navigated = _db.Roles.Select(r => new RvLabelRow { Id = r.Id, Label = r.Keeper!.Name }); + IQueryable byName = _db.Roles.Select(r => new RvLabelRow { Id = r.Id, Label = EF.Property(r, "Code") }); + + Assert.True(DefaultOrder.HidesDefault(replaced.Expression, typeof(RvLabelRow))); + Assert.True(DefaultOrder.HidesDefault(joined.Expression, typeof(RvLabelRow))); + Assert.False(DefaultOrder.HidesDefault(direct.Expression, typeof(RvLabelRow))); + Assert.False(DefaultOrder.HidesDefault(navigated.Expression, typeof(RvLabelRow))); + Assert.False(DefaultOrder.HidesDefault(byName.Expression, typeof(RvLabelRow))); + + Assert.Equal("X1", Guard(replaced).ToList(new Filter()).Data.Single().Label); + } + /// A member named with a word the parser keeps cannot be projected, so it is skipped. [Fact] public void A_member_named_with_a_parser_word_is_skipped() diff --git a/DynamicWhere.ex/Policies/Source/DefaultOrder.cs b/DynamicWhere.ex/Policies/Source/DefaultOrder.cs index 24e9be5..8884656 100644 --- a/DynamicWhere.ex/Policies/Source/DefaultOrder.cs +++ b/DynamicWhere.ex/Policies/Source/DefaultOrder.cs @@ -7,6 +7,8 @@ using DynamicWhere.ex.Optimization.Cache.Source; using DynamicWhere.ex.Policies.Attributes; using DynamicWhere.ex.Source; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Metadata; namespace DynamicWhere.ex.Policies.Source; @@ -97,10 +99,35 @@ or nameof(Queryable.ThenBy) or nameof(Queryable.ThenByDescending)) /// which answers. /// internal static bool BuildsRows(Expression expression) => - OutermostSelect(expression) is { } select + RowSelect(expression) is { } select && StripQuotes(select.Arguments[1]) is LambdaExpression selector && StripConversions(selector.Body) is MemberInitExpression or NewExpression; + /// + /// The Select that makes a query's rows: the outermost one, when every call after it hands back + /// the rows it was given. Null when nothing projects the rows, or when a call such as a + /// SelectMany or a Join reaches the rows through it, so that its members are not theirs. + /// + internal static MethodCallExpression? RowSelect(Expression expression) + { + for (Expression? node = expression; node is MethodCallExpression call; + node = call.Arguments.Count > 0 ? call.Arguments[0] : null) + { + if ((call.Method.DeclaringType == typeof(Queryable) || call.Method.DeclaringType == typeof(Enumerable)) + && call.Method.Name == nameof(Queryable.Select)) + { + return call; + } + + if (!QueryRoot.KeepsRows(call)) + { + return null; + } + } + + return null; + } + /// /// True when a projection along the chain could have left out a field the type's default names. /// @@ -112,10 +139,10 @@ internal static bool BuildsRows(Expression expression) => /// /// Only the outermost projection is read, because it makes the rows the default orders. It hides /// nothing when it builds the type itself in an initializer — new Row { Code = t.Code } — - /// and assigns every field the default names, at every level of a nested path. EF Core can then - /// translate the order, since each field is a member the projection assigned. Anything else, a - /// constructor with arguments, a member it does not assign or a nested path through something - /// other than an initializer, leaves the query in whatever order it had. + /// and assigns every field the default names, at every level of a nested path, a column of the + /// entity it reads on EF Core. EF Core can then translate the order. Anything else, a constructor + /// with arguments, a member it does not assign, a value it computes or a nested path through + /// something other than an initializer, leaves the query in whatever order it had. /// /// internal static bool HidesDefault(Expression expression, Type type) @@ -132,9 +159,15 @@ internal static bool HidesDefault(Expression expression, Type type) return true; } + // On EF Core a default field must be a column the database can order by. In memory anything can + // be ordered. + ColumnReader? columns = QueryRoot.Model(expression) is { } model + ? new ColumnReader(selector.Parameters[0], model.FindEntityType(selector.Parameters[0].Type)) + : null; + foreach (Entry entry in For(type)) { - if (!Assigns(initializer.Bindings, entry.Field.Split('.'), 0)) + if (!Assigns(initializer.Bindings, entry.Field.Split('.'), 0, columns)) { return true; } @@ -160,7 +193,7 @@ internal static bool HidesDefault(Expression expression, Type type) } /// True when the bindings assign the path, following nested initializers down it. - private static bool Assigns(IEnumerable bindings, string[] path, int depth) + private static bool Assigns(IEnumerable bindings, string[] path, int depth, ColumnReader? columns) { MemberBinding? binding = bindings.FirstOrDefault( candidate => string.Equals(candidate.Member.Name, path[depth], StringComparison.Ordinal)); @@ -172,66 +205,81 @@ private static bool Assigns(IEnumerable bindings, string[] path, if (depth == path.Length - 1) { - return binding is not MemberAssignment { Expression: var assigned } || Translatable(assigned); + return binding is MemberAssignment { Expression: var assigned } + && (columns is null || columns.Column(assigned)); } return binding switch { MemberAssignment { Expression: var assigned } when StripConversions(assigned) is MemberInitExpression nested => - Assigns(nested.Bindings, path, depth + 1), - MemberMemberBinding nested => Assigns(nested.Bindings, path, depth + 1), + Assigns(nested.Bindings, path, depth + 1, columns), + MemberMemberBinding nested => Assigns(nested.Bindings, path, depth + 1, columns), _ => false }; } /// - /// True when EF Core can order by what an expression computes: nothing in it calls a method outside - /// the framework and EF Core, or invokes a delegate. + /// Reads whether EF Core can order by what a projection assigns: a column of the entity the + /// projection reads, directly, through reference navigations, or through EF.Property. /// /// - /// An assigned field is not enough on its own. Label = Decorate(r.Code) assigns the field - /// the default names, and EF Core evaluates the application's own method on the client, where it can - /// project the value but cannot order by it. Such a default is left out, as the projection left the - /// query unordered before. + /// An assigned field is not enough on its own. Label = Decorate(r.Code) assigns the field the + /// default names, and EF Core evaluates the application's own method on the client, where it can + /// project the value but cannot order by it; so it does Regex.Replace, a member the model does + /// not map, and any number of framework methods a given provider cannot translate. Anything but a + /// column is therefore left out of the default, as the projection left the query unordered before: + /// a default must never be the reason a query that ran unguarded fails. /// - private static bool Translatable(Expression expression) + private sealed class ColumnReader { - ClientCallFinder finder = new(); + private readonly ParameterExpression _row; + private readonly IEntityType? _source; - finder.Visit(expression); - - return !finder.Found; - } + internal ColumnReader(ParameterExpression row, IEntityType? source) + { + _row = row; + _source = source; + } - /// Finds a call EF Core would have to evaluate on the client. - private sealed class ClientCallFinder : ExpressionVisitor - { - internal bool Found { get; private set; } + /// True when the expression reads one column the model maps. + internal bool Column(Expression expression) + { + expression = StripConversions(expression); - public override Expression? Visit(Expression? node) => Found ? node : base.Visit(node); + return Read(expression) is ({ } owner, { } name) && owner.FindProperty(name) is not null; + } - protected override Expression VisitMethodCall(MethodCallExpression node) + /// The entity a navigation expression reaches, through reference navigations only. + private IEntityType? EntityOf(Expression expression) { - string space = node.Method.DeclaringType?.Namespace ?? string.Empty; + expression = StripConversions(expression); - if (!(space == "System" || space.StartsWith("System.", StringComparison.Ordinal) - || space == "Microsoft.EntityFrameworkCore" || space.StartsWith("Microsoft.EntityFrameworkCore.", StringComparison.Ordinal))) + if (expression == _row) { - Found = true; - - return node; + return _source; } - return base.VisitMethodCall(node); + // A reference, read the way EF Core 6 reads IsCollection: the property that moved in EF Core 8 + // is not bound, so the check holds on either. + return Read(expression) is ({ } owner, { } name) + && owner.FindNavigation(name) is { } navigation + && (navigation.IsOnDependent || navigation.ForeignKey.IsUnique) + ? navigation.TargetEntityType + : null; } - protected override Expression VisitInvocation(InvocationExpression node) + /// The entity a member read is made on, and the member's name. + private (IEntityType? Owner, string? Name) Read(Expression expression) => expression switch { - Found = true; - - return node; - } + MemberExpression { Member: PropertyInfo property, Expression: { } owner } => + (EntityOf(owner), property.Name), + MethodCallExpression { Method.Name: nameof(EF.Property) } call + when call.Method.DeclaringType == typeof(EF) + && call.Arguments[1] is ConstantExpression { Value: string name } => + (EntityOf(call.Arguments[0]), name), + _ => (null, null) + }; } internal static Expression StripQuotes(Expression expression) => diff --git a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs index 029f697..57188a7 100644 --- a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs +++ b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs @@ -1,4 +1,5 @@ -using System.Collections.Concurrent; +using System.Collections; +using System.Collections.Concurrent; using System.Reflection; using DynamicWhere.ex.Classes.Complex; using DynamicWhere.ex.Classes.Core; @@ -1710,13 +1711,11 @@ void Keep(string path) } } - // A member that carries everything its type holds can hold a denied field no path names: - // beyond the walker's depth, or inside a framework collection. An entity's navigation loads - // only its own entity, which the paths above already cover. + // A member can carry a denied field no path names: beyond the walker's depth, inside a + // framework collection, declared by a subtype, or, under a policy that denies whatever it + // does not name, anywhere the walk does not reach. On an entity, only what loads counts. bool hidden = cause is null - && rows.LoadsWhole(field) - && gate.DeclaredType(field) is { } type - && Gate.Facts(type).DeniesSelect + && gate.Unnamed(field, rows).DeniesSelect && gate.RefuseHidden(field); if (cause is null && !hidden && !gate.ReadOnlyTransformBeneath(field)) @@ -1755,7 +1754,7 @@ void Keep(string path) { // Projected whole by the core, a field holding a framework collection of a policed type // carries the denied fields the policy cannot name inside it. - if (gate.DeclaredType(path) is { } leaf && !Gate.HoldsValue(leaf) && Gate.Facts(leaf).DeniesSelect + if (gate.DeclaredType(path) is { } leaf && !Gate.HoldsValue(leaf) && gate.Unnamed(path, rows).DeniesSelect && gate.RefuseHidden(path)) { continue; @@ -1926,13 +1925,14 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) (List survivors, List<(string Path, FieldPolicy Policy)> denied) = gate.Beneath(name, PolicyFeature.None); - Gate.TypeFacts facts = Gate.Facts(member.PropertyType); - bool forced = gate.ForcedBeneath(name); - // Only a denial whose value can reach the result asks for a projection: one beneath a - // navigation nothing loads never leaves the database. + // navigation nothing loads never leaves the database. What no path names is asked about + // only where the source carries the member at all, and on an entity only where it loads. List<(string Path, FieldPolicy Policy)> reached = denied.Where(d => rows.Materializes(d.Path)).ToList(); - bool opens = rows.Materializes(name + SegmentSeparator + name); + Gate.TypeFacts unnamed = rows.Materializes(name + SegmentSeparator + name) + ? gate.Unnamed(name, rows) + : default; + bool forced = gate.ForcedBeneath(name); foreach ((string path, FieldPolicy beneath) in reached) { @@ -1942,7 +1942,7 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) // A forced scope beneath a member does not ask for a projection on its own. It filters the // rows that hold the member, which is what it has always done for a list returned whole, // and asking would leave out every included list of a scoped child type. - if (reached.Count > 0 || (opens && facts.Opaque)) + if (reached.Count > 0 || unnamed.Opaque) { anyDenied = true; } @@ -1952,7 +1952,7 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) continue; } - if (reached.Count == 0 && !facts.Opaque && !forced && !gate.ReadOnlyTransformBeneath(name)) + if (reached.Count == 0 && !unnamed.Opaque && !forced && !gate.ReadOnlyTransformBeneath(name)) { allowed.Add(name); @@ -1962,7 +1962,7 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) string? reason = forced ? "left out whole: a scope forced beneath it cannot be applied to what it holds" : rows.Narrows(name) - ? Narrowed(name, survivors, gate, allowed) + ? Narrowed(name, survivors, gate, rows, allowed) : "left out whole: " + rows.WhyNotNarrowed(name); if (reason is not null) @@ -1971,6 +1971,15 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) } } + // A row can be a subtype of T, whose own members T's never name. The projection builds T, so it + // leaves them out; one this caller may not have is what asks for it. + foreach (string path in gate.DerivedDenials(rows)) + { + anyDenied = true; + + gate.LeaveOut(path, "left out: a type derived from the row's type declares it, and the projection builds the row's type"); + } + if (!anyDenied || gate.IsDryRun) { return null; @@ -1981,6 +1990,16 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) throw gate.Exception(WholeClause, PolicyFeature.Select, PolicyErrorCode.AllSelectsDenied, null); } + // The core builds each row with T's parameterless constructor. Without one there is no projection + // to withhold anything with, and the rows are not handed over whole instead. + if (gate.EntityType.IsAbstract || gate.EntityType.GetConstructor(Type.EmptyTypes) is null) + { + gate.RefuseNarrowing( + WholeClause, null, + $"the rows must be projected to withhold a field, and the core cannot build '{gate.EntityType.Name}': " + + "it is abstract or has no public parameterless constructor"); + } + return allowed; } @@ -1994,13 +2013,13 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) /// whole. The builder adds the key of every node it narrows, so a denied key leaves the member out, /// and so does a path the core cannot project. /// - private static string? Narrowed(string member, List survivors, Gate gate, List allowed) + private static string? Narrowed(string member, List survivors, Gate gate, RowShape rows, List allowed) { List kept = new(survivors.Count); foreach (string path in survivors) { - if (gate.DeclaredType(path) is { } type && !Gate.HoldsValue(type) && Gate.Facts(type).Opaque) + if (gate.DeclaredType(path) is { } type && !Gate.HoldsValue(type) && gate.Unnamed(path, rows).Opaque) { gate.LeaveOut(path, "left out whole: it can hold what the policy cannot name"); @@ -2878,45 +2897,221 @@ internal bool ReadOnlyTransformBeneath(string member) } /// - /// What a type can hold that no path of the policy's reaches, read once per type. + /// What the value of a member can carry that the paths the walk asks about do not name: a field + /// denied for Select, and a member that can hold an object of any type. + /// + /// + /// On an entity query the model says what loads beneath the member + /// (). Each loaded member is asked about by its path, and by + /// its own attribute where no fragment reaches it: deeper than the walker goes, or declared by a + /// type the model derives. A value EF Core reads whole is read through its type. Any other source + /// can carry whatever the member's type can hold, its subtypes included. Under a policy that + /// denies every field it does not name, a path the walk never asks about is a denied one. + /// + internal TypeFacts Unnamed(string member, RowShape rows) + { + if (rows.LoadedBeneath(member) is { } loaded) + { + bool denies = false; + bool holdsObject = false; + + foreach (Loaded entry in loaded) + { + denies |= Denies(entry.Path, entry.Property); + + if (entry.Whole && !HoldsValue(entry.Property.PropertyType)) + { + TypeFacts whole = Carried(entry.Property.PropertyType, entry.Path); + + denies |= whole.DeniesSelect; + holdsObject |= whole.HoldsObject; + } + } + + return new TypeFacts(denies, holdsObject, loaded.Count > 0); + } + + // A path that names nothing is refused long before this; were it not, nothing is shown clean. + return DeclaredType(member) is { } type + ? Carried(type, member) + : new TypeFacts(true, true, true); + } + + /// What a value of a type, held at a path, can carry that no path names. + private TypeFacts Carried(Type type, string path) + { + TypeFacts facts = Facts(type); + + return DeniesByDefault && !Walks(type, path.Split(SegmentSeparator).Length) + ? facts with { DeniesSelect = true } + : facts; + } + + /// + /// True when this caller may not select a member reached along a path: its policy says so, or, + /// where no fragment reaches the path, an attribute on the member does. + /// + /// + /// The walker puts a fragment on every path of the declared types up to its depth, a member + /// with no setter and a path around a cycle included. Past its depth, or on a member only a + /// derived type declares, the attribute is all there is. + /// + private bool Denies(string path, PropertyInfo property) + { + if (!PolicyFor(path, PolicyFeature.None).Allows(PolicyFeature.Select)) + { + return true; + } + + return (path.Split(SegmentSeparator).Length > AttributePolicyProvider.MaxDepth + || Property(_entityType, path) is null) + && property.GetCustomAttributes(inherit: true) + .Any(attribute => (attribute.Features & PolicyFeature.Select) != 0); + } + + /// + /// Every member a type derived from T declares that a row can carry and this caller may not + /// have, or that carries what no path names. A projection builds T itself and leaves them all + /// out, so each is a reason to project. + /// + /// + /// A projection builds exactly T. An entity query materializes each row as the type the database + /// says it is, which the model knows. Any other source can hold any subtype loaded. + /// + internal List DerivedDenials(RowShape rows) + { + List found = new(); + + if (rows.Kind == RowKind.Projected) + { + return found; + } + + if (rows.LoadedByDerived() is { } loaded) + { + foreach (Loaded entry in loaded) + { + if (Denies(entry.Path, entry.Property) + || (entry.Whole && !HoldsValue(entry.Property.PropertyType) + && Carried(entry.Property.PropertyType, entry.Path).Opaque)) + { + found.Add(entry.Path); + } + } + + return found; + } + + foreach (Type subtype in KnownSubtypes.Of(_entityType)) + { + foreach (PropertyInfo property in subtype.GetProperties(BindingFlags.Public | BindingFlags.Instance)) + { + if (!property.CanRead + || property.GetIndexParameters().Length != 0 + || property.DeclaringType!.IsAssignableFrom(_entityType) + || found.Contains(property.Name, StringComparer.Ordinal)) + { + continue; + } + + if (Denies(property.Name, property) + || (!HoldsValue(property.PropertyType) && Carried(property.PropertyType, property.Name).Opaque)) + { + found.Add(property.Name); + } + } + } + + return found; + } + + /// + /// True when a "*" rule denies Select on every field this caller is not granted by name, + /// so that a path the walk never asks about is a denied one. /// - internal static TypeFacts Facts(Type type) => FactsByType.GetOrAdd(type, Scan); + internal bool DeniesByDefault => + _deniesByDefault ??= Fragments.Any(fragment => fragment.IsWildcard) + && !PolicyFor(UnnamedPath, PolicyFeature.None).Allows(PolicyFeature.Select); + + private bool? _deniesByDefault; + + /// A path no member can have, so that only the "*" rules match it. + private const string UnnamedPath = ""; + + /// The type the rows are. + internal Type EntityType => _entityType; + + /// + /// What a type can hold that no path of the policy's reaches, read once per type and again once + /// another assembly has loaded, since that can add subtypes. + /// + internal static TypeFacts Facts(Type type) + { + int epoch = KnownSubtypes.Epoch; + + if (FactsByType.TryGetValue(type, out (int Epoch, TypeFacts Facts) known) && known.Epoch == epoch) + { + return known.Facts; + } + + TypeFacts facts = Scan(type); + + FactsByType[type] = (epoch, facts); - private static readonly ConcurrentDictionary FactsByType = new(); + return facts; + } + + private static readonly ConcurrentDictionary FactsByType = new(); /// /// Reads every type a value of this type can hold, however deep, for a field denied for Select - /// and for a member typed . + /// and for a member that can hold an object of any type. /// /// - /// Wider than the walker on purpose. The walker stops four segments deep and does not enter a - /// framework type, so a field denied beneath either has no path, and resolves as allowed. This - /// follows every property, a read-only one included, every collection, and the type arguments of - /// a framework generic such as Dictionary<string, T>, and remembers the types it has - /// read rather than counting levels. A member typed can hold anything, so - /// it is reported rather than read. + /// Wider than the walker on purpose. The walker stops four segments deep, does not enter a + /// framework type and reads declared types only, so a field denied beneath any of those has no + /// path, and resolves as allowed. This follows every property, a read-only one included, every + /// collection, the type arguments of a framework generic such as Dictionary<string, T>, + /// and every loaded subtype of a type it reads, and remembers the types it has read rather than + /// counting levels. A member that can hold anything is reported rather than read. /// private static TypeFacts Scan(Type type) { bool denies = false; bool holdsObject = false; + bool holdsMembers = false; HashSet seen = new(); Stack pending = new(); + void Read(Type candidate) + { + if (seen.Add(candidate)) + { + pending.Push(candidate); + } + } + void Enqueue(Type candidate) { Type peeled = AttributePolicyProvider.Peeled(candidate); - if (peeled == typeof(object)) + if (HoldsAnything(peeled)) { holdsObject = true; return; } - if (AttributePolicyProvider.NavigationTypeOf(candidate) is { } navigation && seen.Add(navigation)) + if (AttributePolicyProvider.NavigationTypeOf(candidate) is { } navigation) { - pending.Push(navigation); + holdsMembers = true; + + Read(navigation); + + foreach (Type subtype in KnownSubtypes.Of(navigation)) + { + Read(subtype); + } } if (peeled.IsGenericType && AttributePolicyProvider.IsFramework(peeled)) @@ -2949,20 +3144,94 @@ void Enqueue(Type candidate) } } - return new TypeFacts(denies, holdsObject); + return new TypeFacts(denies, holdsObject, holdsMembers); } + /// + /// True for a type whose value can be an object of any type: itself, a + /// framework interface such as IComparable, and a collection that is not generic, such as + /// ArrayList, Hashtable, IEnumerable or . + /// + private static bool HoldsAnything(Type peeled) => + peeled == typeof(object) + || peeled == typeof(ValueType) + || (AttributePolicyProvider.IsFramework(peeled) + && !peeled.IsGenericType + && (peeled.IsInterface || (peeled != typeof(string) && typeof(IEnumerable).IsAssignableFrom(peeled)))); + /// /// What a type can hold that the policy cannot name a path to. /// /// A field denied for Select somewhere in what it can hold. - /// A member typed somewhere in what it can hold. - internal readonly record struct TypeFacts(bool DeniesSelect, bool HoldsObject) + /// A member that can hold an object of any type somewhere in what it can hold. + /// An application type's members somewhere in what it can hold. + internal readonly record struct TypeFacts(bool DeniesSelect, bool HoldsObject, bool HoldsMembers) { /// True when either fact holds, so the type cannot be shown to be free of policy. internal bool Opaque => DeniesSelect || HoldsObject; } + /// + /// True when the walk beneath a member of this type, the given number of segments deep, asks + /// about every path a value of it can hold. + /// + /// + /// The walk skips a property with no setter, stops four segments deep and at a type it has + /// already passed through, reads declared types only and takes a framework type whole. Any of + /// those beneath a member leaves a path it never asks about, which a policy denying every field + /// it does not name denies. + /// + internal static bool Walks(Type type, int depth) + { + int epoch = KnownSubtypes.Epoch; + + if (WalkedByType.TryGetValue((type, depth), out (int Epoch, bool Walked) known) && known.Epoch == epoch) + { + return known.Walked; + } + + bool walked = Walk(type, depth, new HashSet()); + + WalkedByType[(type, depth)] = (epoch, walked); + + return walked; + } + + private static readonly ConcurrentDictionary<(Type Type, int Depth), (int Epoch, bool Walked)> WalkedByType = new(); + + private static bool Walk(Type type, int depth, HashSet path) + { + if (AttributePolicyProvider.NavigationTypeOf(type) is not { } node) + { + // A value, or a framework type the walk takes whole: it asks about nothing beneath it. + TypeFacts facts = Facts(type); + + return !facts.HoldsObject && !facts.HoldsMembers; + } + + if (depth >= AttributePolicyProvider.MaxDepth || !path.Add(node) || KnownSubtypes.Of(node).Count > 0) + { + return false; + } + + foreach (PropertyInfo property in node.GetProperties(BindingFlags.Public | BindingFlags.Instance)) + { + if (!property.CanRead || property.GetIndexParameters().Length != 0) + { + continue; + } + + if (!property.CanWrite || !Walk(property.PropertyType, depth + 1, path)) + { + return false; + } + } + + path.Remove(node); + + return true; + } + /// /// The declared type at a path, read the way the walker reads it: through any collection, one /// segment at a time. Null when a segment names nothing. diff --git a/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs b/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs new file mode 100644 index 0000000..7afc1f2 --- /dev/null +++ b/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs @@ -0,0 +1,100 @@ +using System.Collections.Concurrent; +using System.Reflection; + +namespace DynamicWhere.ex.Policies.Source; + +/// +/// The types loaded into the process that derive from a type or implement it: what a member declared as +/// that type can hold at run time. +/// +/// +/// A policy is read from declared types, and a member declared as a base class or an interface holds +/// whichever subtype its value is. A [DwDenied] field a subtype declares is therefore carried by a +/// member the policy reads as clean, unless the subtypes are read too. +/// +/// Every object's type is loaded before the object exists, so the loaded assemblies are the whole answer +/// for any value a query returns. Only an assembly that is the type's own, or references it, can declare +/// a subtype, which keeps the framework's and every unrelated library's assemblies out of the search. +/// Loading another assembly may add subtypes, so each answer is kept only until the next one loads; +/// an assembly emitted at run time is not searched, since it cannot declare a policy attribute a +/// compiled type does not already carry. +/// +/// +internal static class KnownSubtypes +{ + private static readonly ConcurrentDictionary Found = new(); + + private static int _epoch; + + static KnownSubtypes() => + AppDomain.CurrentDomain.AssemblyLoad += (_, loaded) => + { + if (!loaded.LoadedAssembly.IsDynamic) + { + Interlocked.Increment(ref _epoch); + } + }; + + /// Changes whenever an assembly loads, and with it any answer read from the subtypes. + internal static int Epoch => Volatile.Read(ref _epoch); + + /// Every loaded type, other than the type itself, whose values the type's members can hold. + internal static IReadOnlyList Of(Type type) + { + if (type.IsSealed || type.IsValueType || type.IsGenericParameter || type == typeof(object)) + { + return Array.Empty(); + } + + int epoch = Epoch; + + if (Found.TryGetValue(type, out (int Epoch, Type[] Types) known) && known.Epoch == epoch) + { + return known.Types; + } + + Type[] types = Search(type); + + Found[type] = (epoch, types); + + return types; + } + + private static Type[] Search(Type type) + { + string? home = type.Assembly.GetName().Name; + List found = new(); + + foreach (Assembly assembly in AppDomain.CurrentDomain.GetAssemblies()) + { + if (assembly.IsDynamic + || (assembly != type.Assembly + && !assembly.GetReferencedAssemblies().Any(reference => reference.Name == home))) + { + continue; + } + + foreach (Type candidate in TypesOf(assembly)) + { + if (candidate != type && !candidate.ContainsGenericParameters && type.IsAssignableFrom(candidate)) + { + found.Add(candidate); + } + } + } + + return found.ToArray(); + } + + private static IEnumerable TypesOf(Assembly assembly) + { + try + { + return assembly.GetTypes(); + } + catch (ReflectionTypeLoadException partial) + { + return partial.Types.OfType(); + } + } +} diff --git a/DynamicWhere.ex/Policies/Source/RowShape.cs b/DynamicWhere.ex/Policies/Source/RowShape.cs index a4c14b0..ef046c1 100644 --- a/DynamicWhere.ex/Policies/Source/RowShape.cs +++ b/DynamicWhere.ex/Policies/Source/RowShape.cs @@ -1,6 +1,8 @@ using System.Collections; +using System.Collections.Concurrent; using System.Linq.Expressions; using System.Reflection; +using System.Runtime.CompilerServices; using DynamicWhere.ex.Optimization.Cache.Source; using DynamicWhere.ex.Source; using Microsoft.EntityFrameworkCore; @@ -23,7 +25,8 @@ namespace DynamicWhere.ex.Policies.Source; /// An entity query loads the entity's columns, its owned and complex members, and a navigation /// only when something loads it: an Include, an automatic include or a lazy loader. A value /// beneath a navigation nothing loads never reaches the result, so a denial there needs no -/// projection; and projecting the navigation would load it. +/// projection; and projecting the navigation would load it. A member the model does not map is +/// never filled by EF Core at all. /// /// /// A projection holds exactly the members its initializer assigns. The others hold defaults, and @@ -36,7 +39,7 @@ namespace DynamicWhere.ex.Policies.Source; /// /// /// Read from the query itself, never from the rows, so the sanitizer stays pure: this is decided -/// once, before it runs. +/// once, before it runs. Where the query does not say, every value counts as reaching the result. /// internal sealed class RowShape { @@ -45,19 +48,24 @@ internal sealed class RowShape /// /// A source this library cannot read: no model, not a projection it can see into, not rows in - /// memory. Every value beneath a member is taken to reach the result, and only members holding a - /// value are kept, as in 3.1.0. + /// memory. Every value beneath a member is taken to reach the result. /// internal static readonly RowShape Unknown = new(RowKind.Entity); /// Rows in memory. internal static readonly RowShape InMemory = new(RowKind.InMemory); + /// Whether a class can be handed a lazy loader, read once per class. + private static readonly ConcurrentDictionary TakesLoader = new(); + + /// The complex properties of each entity type, read once per entity type. + private static readonly ConditionalWeakTable> ComplexByType = new(); + private readonly HashSet? _assigned; - private readonly HashSet _narrowable = new(StringComparer.Ordinal); - private readonly HashSet _kept = new(StringComparer.Ordinal); - private readonly HashSet _includes = new(StringComparer.Ordinal); - private readonly Dictionary _loadedWhole = new(StringComparer.Ordinal); + private readonly HashSet _narrowable = new(StringComparer.OrdinalIgnoreCase); + private readonly HashSet _kept = new(StringComparer.OrdinalIgnoreCase); + private readonly HashSet _includes = new(StringComparer.OrdinalIgnoreCase); + private readonly Dictionary _loadedWhole = new(StringComparer.OrdinalIgnoreCase); private readonly IEntityType? _entity; private readonly bool _includeUnknown; @@ -70,11 +78,11 @@ private RowShape(RowKind kind, HashSet? assigned, IEnumerable na _narrowable.UnionWith(narrowable); } - private RowShape(IEntityType entity, HashSet includes, bool includeUnknown) + private RowShape(IEntityType entity, IEnumerable includes, bool includeUnknown) : this(RowKind.Entity) { _entity = entity; - _includes = includes; + _includes.UnionWith(includes); _includeUnknown = includeUnknown; foreach (IProperty property in entity.GetProperties()) @@ -159,26 +167,16 @@ internal bool Narrows(string member) => _ => _loadedWhole.TryGetValue(member, out string? reason) ? reason : "narrowing it needs EF Core" }; - /// - /// True when a member named whole can carry anything its type can hold. An entity's navigation - /// cannot: projecting it loads that entity and what the entity owns, never the navigations beneath - /// it. Its columns, owned and complex members, and every member of any other source, can. - /// - internal bool LoadsWhole(string path) - { - string member = path.Split('.')[0]; - - return Kind != RowKind.Entity || _entity is null || _kept.Contains(member); - } - /// /// True when the value at a path beneath the root can reach the result: every one in memory or in /// a projection's assigned member, and, on an entity, one every navigation on the way to is loaded. /// /// /// A navigation is loaded when it is owned, included, automatically included, or reachable by a - /// lazy loader. Anything the model cannot answer for counts as loaded, which only ever asks for a - /// projection that turns out not to be needed. + /// lazy loader. A member the model does not map is one EF Core never fills, so nothing reaches the + /// result through it. Anything the model cannot answer for counts as loaded, which only ever asks for + /// a projection that turns out not to be needed. A rule may spell a path in any letter case, so each + /// segment is read through the member it names. /// internal bool Materializes(string path) { @@ -199,50 +197,195 @@ internal bool Materializes(string path) for (int i = 0; i < segments.Length - 1; i++) { - prefix = i == 0 ? segments[0] : $"{prefix}.{segments[i]}"; - - if (current is null) + if (current is null || CacheReflection.FindProperty(current.ClrType, segments[i]) is not { } member) { return true; } + string name = member.Name; + + prefix = i == 0 ? name : $"{prefix}.{name}"; + // A column holding an object, or a complex property, is read whole with its row. - if (current.FindProperty(segments[i]) is not null || ComplexProperties(current).Contains(segments[i])) + if (current.FindProperty(name) is not null || ComplexProperties(current).Contains(name)) { return true; } INavigationBase? navigation = - (INavigationBase?)current.FindNavigation(segments[i]) ?? current.FindSkipNavigation(segments[i]); + (INavigationBase?)current.FindNavigation(name) ?? current.FindSkipNavigation(name); + + if (navigation is null || !Loads(current, navigation, prefix)) + { + return false; + } + + current = navigation.TargetEntityType; + } + + return true; + } + + /// + /// Every member the value at a path loads beneath it, each with its path from the root, when an + /// entity query's model can say; null when the value can carry anything its type holds. + /// + /// + /// Naming a member in a projection loads it, so the path's own segments are not asked about. Beneath + /// it the model decides: an entity's columns, owned and complex members, and the navigations + /// something loads, and the same for every type the model derives from it, whose rows EF Core + /// materializes as that type. A member the model does not map holds nothing EF Core read. A lazy + /// loader in reach can fill any navigation, which bounds nothing, and so does a query whose includes + /// cannot be read. + /// + internal List? LoadedBeneath(string path) + { + if (Kind != RowKind.Entity || _entity is null || _includeUnknown) + { + return null; + } + + IEntityType entity = _entity; + string prefix = string.Empty; + string[] segments = path.Split('.'); + + for (int i = 0; i < segments.Length; i++) + { + if (CacheReflection.FindProperty(entity.ClrType, segments[i]) is not { } member) + { + return null; + } + + prefix = i == 0 ? member.Name : $"{prefix}.{member.Name}"; + + // A value read whole: what it holds is its type's to say, and a path inside it names part of it. + if (entity.FindProperty(member.Name) is not null || ComplexProperties(entity).Contains(member.Name)) + { + return i == segments.Length - 1 ? new List { new(prefix, member, true) } : null; + } + + INavigationBase? navigation = + (INavigationBase?)entity.FindNavigation(member.Name) ?? entity.FindSkipNavigation(member.Name); if (navigation is null) { - return true; + return new List(); } - bool loaded = (navigation is INavigation owned && IsOwned(owned)) - || navigation.IsEagerLoaded - || _includeUnknown - || _includes.Contains(prefix) - || LoadsLazily(current); + entity = navigation.TargetEntityType; + } + + List loaded = new(); - if (!loaded) + return Collect(entity, prefix, loaded, new HashSet<(IEntityType, string?)>(), derivedOnly: false) + ? loaded + : null; + } + + /// + /// Every member an entity query's rows load that a type the model derives from the root declares, + /// each with what it loads beneath it; null when a lazy loader or an unreadable include bounds + /// nothing, and for any other source. + /// + /// + /// EF Core materializes each row as the type the database says it is, so a query over the root of a + /// hierarchy returns the derived types' columns too, which the root's own members never name. + /// + internal List? LoadedByDerived() + { + if (Kind != RowKind.Entity || _entity is null || _includeUnknown) + { + return null; + } + + List loaded = new(); + + return Collect(_entity, string.Empty, loaded, new HashSet<(IEntityType, string?)>(), derivedOnly: true) + ? loaded + : null; + } + + /// + /// Collects what an entity loads beneath a path, its derived types' members included. False when a + /// lazy loader can fill any of its navigations. + /// + private bool Collect( + IEntityType entity, string prefix, List loaded, HashSet<(IEntityType, string?)> seen, bool derivedOnly) + { + // Off an include's path, what loads depends only on the type, so a type is read once there. + if (!seen.Add((entity, _includes.Contains(prefix) ? prefix : null))) + { + return true; + } + + foreach (IEntityType type in entity.GetDerivedTypesInclusive()) + { + if (derivedOnly && type == entity) + { + continue; + } + + if (LoadsLazily(type)) { return false; } - current = navigation.TargetEntityType; + foreach (PropertyInfo member in type.ClrType.GetProperties(BindingFlags.Public | BindingFlags.Instance)) + { + // A derived type's inherited members are the base type's, read once from there. + if (!member.CanRead + || member.GetIndexParameters().Length != 0 + || (type != entity && member.DeclaringType!.IsAssignableFrom(entity.ClrType))) + { + continue; + } + + string path = prefix.Length == 0 ? member.Name : $"{prefix}.{member.Name}"; + + if (type.FindProperty(member.Name) is not null || ComplexProperties(type).Contains(member.Name)) + { + loaded.Add(new Loaded(path, member, true)); + + continue; + } + + INavigationBase? navigation = + (INavigationBase?)type.FindNavigation(member.Name) ?? type.FindSkipNavigation(member.Name); + + if (navigation is null || !Loads(type, navigation, path)) + { + continue; + } + + loaded.Add(new Loaded(path, member, false)); + + if (!Collect(navigation.TargetEntityType, path, loaded, seen, derivedOnly: false)) + { + return false; + } + } } return true; } + /// True when something loads a navigation of an entity reached along a path. + private bool Loads(IEntityType owner, INavigationBase navigation, string path) => + (navigation is INavigation owned && IsOwned(owned)) + || navigation.IsEagerLoaded + || _includeUnknown + || _includes.Contains(path) + || LoadsLazily(owner); + /// Reads the shape of a guarded query's source. /// /// Rows in memory first, because a projection over them is still in memory. Then a projection that /// builds its rows, which is how a caller makes its own row type out of entities. What is left is an - /// entity query, or a projection handing back entities, Select(o => o.Customer), both read - /// from the EF Core model; without one, the source is . + /// entity query, read from the EF Core model; without one, the source is . A + /// chain that reaches its rows through anything but the root's own, Select(o => o.Customer), + /// a SelectMany, a Join or a projection behind another Select, holds entities + /// whose navigations were loaded from another root or by a projection, so every navigation counts as + /// loaded there. /// internal static RowShape Of(IQueryable source) where T : class { @@ -261,8 +404,8 @@ internal static RowShape Of(IQueryable source) where T : class return Unknown; } - HashSet includes = new(StringComparer.Ordinal); - bool unknown = !ReadIncludes(source.Expression, includes); + HashSet includes = new(StringComparer.OrdinalIgnoreCase); + bool unknown = !ReadIncludes(source.Expression, includes) || QueryRoot.Reshapes(source.Expression); return new RowShape(entity, includes, unknown); } @@ -273,17 +416,18 @@ internal static RowShape Of(IQueryable source) where T : class /// private static RowShape Projected(IQueryable source) where T : class { - MethodCallExpression select = DefaultOrder.OutermostSelect(source.Expression)!; + MethodCallExpression select = DefaultOrder.RowSelect(source.Expression)!; LambdaExpression selector = (LambdaExpression)DefaultOrder.StripQuotes(select.Arguments[1]); Expression body = DefaultOrder.StripConversions(selector.Body); - // A constructor with arguments says nothing about which member each argument sets. - if (body is not MemberInitExpression initializer) + // A constructor with arguments says nothing about which member each argument sets, whether or + // not an initializer follows it: every member counts as assigned, and none can be narrowed. + if (body is not MemberInitExpression initializer || initializer.NewExpression.Arguments.Count > 0) { return new RowShape(RowKind.Projected, null, Array.Empty()); } - HashSet assigned = new(StringComparer.Ordinal); + HashSet assigned = new(StringComparer.OrdinalIgnoreCase); List narrowable = new(); ParameterExpression row = selector.Parameters[0]; @@ -474,11 +618,36 @@ private static bool IsOwned(INavigation navigation) => /// /// True when a lazy loader can fill this entity's navigations after the query: EF Core's proxies and - /// an injected ILazyLoader both give it a service property. + /// an injected ILazyLoader give it a service property, and a loader the class takes in its + /// constructor, the delegate form above all, may be kept in a field or a property of any name, + /// where the model has no record of it. /// private static bool LoadsLazily(IEntityType entity) => - entity.GetServiceProperties().Any(service => - service.ClrType == typeof(ILazyLoader) || service.ClrType == typeof(Action)); + entity.GetServiceProperties().Any(service => IsLoader(service.ClrType)) + || TakesLoader.GetOrAdd(entity.ClrType, HoldsLoader); + + private static bool HoldsLoader(Type type) + { + const BindingFlags any = BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance; + + if (type.GetConstructors(any).Any(constructor => constructor.GetParameters().Any(p => IsLoader(p.ParameterType)))) + { + return true; + } + + for (Type? current = type; current is not null && current != typeof(object); current = current.BaseType) + { + if (current.GetFields(any | BindingFlags.DeclaredOnly).Any(field => IsLoader(field.FieldType)) + || current.GetProperties(any | BindingFlags.DeclaredOnly).Any(property => IsLoader(property.PropertyType))) + { + return true; + } + } + + return false; + } + + private static bool IsLoader(Type type) => type == typeof(ILazyLoader) || type == typeof(Action); /// /// The names of an entity type's complex properties, which EF Core 8 introduced and loads with the @@ -488,9 +657,12 @@ private static bool LoadsLazily(IEntityType entity) => /// Read by reflection because the library compiles against EF Core 6. The method is declared on an /// interface the entity type implements, so the interfaces are searched rather than the class. /// - private static HashSet ComplexProperties(IEntityType entityType) + private static HashSet ComplexProperties(IEntityType entityType) => + ComplexByType.GetValue(entityType, ReadComplexProperties); + + private static HashSet ReadComplexProperties(IEntityType entityType) { - HashSet names = new(StringComparer.Ordinal); + HashSet names = new(StringComparer.OrdinalIgnoreCase); foreach (Type contract in entityType.GetType().GetInterfaces()) { @@ -519,6 +691,12 @@ private static HashSet ComplexProperties(IEntityType entityType) } } +/// A member a value loads, with its path from the root. +/// The member's path from the root. +/// The member. +/// True when EF Core reads it whole: a column or a complex property. +internal readonly record struct Loaded(string Path, PropertyInfo Property, bool Whole); + /// Where a guarded query's rows come from. internal enum RowKind { diff --git a/DynamicWhere.ex/Source/QueryRoot.cs b/DynamicWhere.ex/Source/QueryRoot.cs index acfcf5e..a500a14 100644 --- a/DynamicWhere.ex/Source/QueryRoot.cs +++ b/DynamicWhere.ex/Source/QueryRoot.cs @@ -24,13 +24,84 @@ internal static class QueryRoot /// The entity type the model maps for , or null when the query is not an /// EF Core query or the model does not map that type. /// - internal static IEntityType? EntityType(Expression expression, Type type) + internal static IEntityType? EntityType(Expression expression, Type type) => Model(expression)?.FindEntityType(type); + + /// The EF Core model behind a query, or null when the query is not an EF Core query. + internal static IModel? Model(Expression expression) { RootFinder finder = new(); finder.Visit(expression); - return finder.EntityType?.Model.FindEntityType(type); + return finder.EntityType?.Model; + } + + /// + /// True when a call along a query's chain hands back the rows it was given, fewer of them or in + /// another order, rather than rows it builds or reaches through them. + /// + /// + /// A method this does not know counts as building its rows, which only ever reads a query more + /// warily than it needs. + /// + internal static bool KeepsRows(MethodCallExpression call) + { + Type? declaring = call.Method.DeclaringType; + + if (declaring == typeof(Queryable) || declaring == typeof(Enumerable)) + { + return call.Method.Name switch + { + nameof(Queryable.Where) or nameof(Queryable.OrderBy) or nameof(Queryable.OrderByDescending) + or nameof(Queryable.ThenBy) or nameof(Queryable.ThenByDescending) or "Order" or "OrderDescending" + or nameof(Queryable.Skip) or nameof(Queryable.Take) or nameof(Queryable.SkipWhile) + or nameof(Queryable.TakeWhile) or "SkipLast" or "TakeLast" or nameof(Queryable.Distinct) + or "DistinctBy" or nameof(Queryable.Reverse) or nameof(Queryable.AsQueryable) + or nameof(Queryable.OfType) or nameof(Queryable.Cast) => true, + + // Each of these returns the rows of both sources, and the other one is read the same way. + nameof(Queryable.Concat) or nameof(Queryable.Union) or "UnionBy" or nameof(Queryable.Intersect) + or "IntersectBy" or nameof(Queryable.Except) or "ExceptBy" => true, + + // With no argument of its own, only a missing row is added. + nameof(Queryable.DefaultIfEmpty) => call.Arguments.Count == 1, + + _ => false + }; + } + + // EF Core's own operators that load, track or tag the rows without changing which they are. + return declaring?.Namespace == "Microsoft.EntityFrameworkCore" + && call.Method.Name is "Include" or "ThenInclude" or "AsNoTracking" + or "AsNoTrackingWithIdentityResolution" or "AsTracking" or "IgnoreQueryFilters" + or "IgnoreAutoIncludes" or "TagWith" or "TagWithCallSite" or "AsSplitQuery" or "AsSingleQuery"; + } + + /// + /// True when some call along the chain, or along the other source of a set operation, builds its + /// rows or reaches them through the rows it was given: a Select, a SelectMany, a + /// Join, a GroupBy, or anything does not know. + /// + internal static bool Reshapes(Expression expression) + { + for (Expression? node = expression; node is MethodCallExpression call; + node = call.Arguments.Count > 0 ? call.Arguments[0] : null) + { + if (!KeepsRows(call)) + { + return true; + } + + if (call.Arguments.Count > 1 + && call.Method.Name is nameof(Queryable.Concat) or nameof(Queryable.Union) or "UnionBy" + or nameof(Queryable.Intersect) or "IntersectBy" or nameof(Queryable.Except) or "ExceptBy" + && Reshapes(call.Arguments[1])) + { + return true; + } + } + + return false; } /// Finds the first query root that names an entity type. From 0aba4bbc943d5df63f258d650929c409d88e3006 Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 06:52:47 +0300 Subject: [PATCH 12/22] fix(policies): leave an abstract root to the terminal that builds it The refusal added for a projection over an abstract T, or one with no parameterless constructor, also refused the dynamic terminals, which build their own class and return the root's allowed members. The typed terminals already fail closed there with SelectTypeMustHaveParameterlessConstructor, before any row is read. Co-Authored-By: Claude Opus 5 --- .../Policies/ProjectionReachTests.cs | 17 ++++++++++++++--- .../Policies/Source/FilterSanitizer.cs | 13 ------------- 2 files changed, 14 insertions(+), 16 deletions(-) diff --git a/DynamicWhere.Tests/Policies/ProjectionReachTests.cs b/DynamicWhere.Tests/Policies/ProjectionReachTests.cs index ac9277e..2edf395 100644 --- a/DynamicWhere.Tests/Policies/ProjectionReachTests.cs +++ b/DynamicWhere.Tests/Policies/ProjectionReachTests.cs @@ -919,13 +919,24 @@ public void A_denied_field_of_a_derived_type_projects_the_root(DwTier tier) Assert.Contains(guarded.LastTrace!.Decisions, d => d.FieldPath == "TaxSecret" && d.Action == PolicyAction.Dropped); } - /// An abstract root cannot be built, so a query that needs a projection over one is refused. + /// + /// An abstract root cannot be built, so the typed terminal refuses the projection a derived type's denial + /// asks for, as it refuses any type with no public parameterless constructor. The dynamic terminal builds + /// its own class and returns the root's allowed members. + /// [Theory] [InlineData(DwTier.Strict)] [InlineData(DwTier.Convenience)] - public void An_abstract_root_that_needs_a_projection_is_refused(DwTier tier) + public void An_abstract_root_that_needs_a_projection_is_never_returned_whole(DwTier tier) { - Refused(() => Guard(_db.Assets, tier).ToList(new Filter())); + LogicException refusal = Assert.Throws(() => Guard(_db.Assets, tier).ToList(new Filter())); + + Assert.Equal(ErrorCode.SelectTypeMustHaveParameterlessConstructor, refusal.Message); + + List rows = Guard(_db.Assets, tier).ToListDynamic(new Filter()).Data!; + + Assert.Equal("V1", (string)Assert.Single(rows).Label); + Assert.False(Holds(rows, "combination-secret")); // Its concrete type can be queried, and is projected like any other. Assert.Null(Guard(_db.Assets.OfType(), tier).ToList(new Filter()).Data.Single().Combination); diff --git a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs index 57188a7..30b3854 100644 --- a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs +++ b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs @@ -1990,16 +1990,6 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) throw gate.Exception(WholeClause, PolicyFeature.Select, PolicyErrorCode.AllSelectsDenied, null); } - // The core builds each row with T's parameterless constructor. Without one there is no projection - // to withhold anything with, and the rows are not handed over whole instead. - if (gate.EntityType.IsAbstract || gate.EntityType.GetConstructor(Type.EmptyTypes) is null) - { - gate.RefuseNarrowing( - WholeClause, null, - $"the rows must be projected to withhold a field, and the core cannot build '{gate.EntityType.Name}': " + - "it is abstract or has no public parameterless constructor"); - } - return allowed; } @@ -3038,9 +3028,6 @@ internal List DerivedDenials(RowShape rows) /// A path no member can have, so that only the "*" rules match it. private const string UnnamedPath = ""; - /// The type the rows are. - internal Type EntityType => _entityType; - /// /// What a type can hold that no path of the policy's reaches, read once per type and again once /// another assembly has loaded, since that can add subtypes. From 075d507cf8ef1e9647d43d8a75ac40cfc1c40ac1 Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 06:57:04 +0300 Subject: [PATCH 13/22] docs: what a query loads, subtypes, deny-by-default, and the review's corrections The agent reference, the site, README, DOC.md and the release notes say what 6c9e171 and 0aba4bb changed: - Section 17 reads what a member carries from the source: the model for an entity, so only what loads counts, and every loaded subtype otherwise. - Every navigation counts as loaded on a chain the library cannot read. - Unmapped members ask for nothing. - A "*" deny denies what the walk never asks. - Rows of a derived type come back as T. The docs review of f7e1cc6 found claims the code did not support, now corrected: - Named entity navigations were said to be closed. - The async rationale was wrong: only the Summary count was synchronous. - DefaultOrder is now a column-only rule. - Convenience refuses a framework generic it cannot narrow. - Naming a navigation under Strict needs its fields. - The trace list is split. - Constructor-built rows are described consistently. - A transform can narrow a clean navigation. - History wording is fixed. - The dynamic read falls back outside EF Core. - llms.txt is written by hand. The version gate's phrase follows that wording. Co-Authored-By: Claude Opus 5 --- DynamicWhere.ex/DOC.md | 41 ++-- DynamicWhere.ex/DynamicWhere.ex.csproj | 12 +- DynamicWhere.ex/Source/AsyncReads.cs | 10 +- OfficialWebsite/app/docs/ai/page.tsx | 4 +- .../app/docs/breaking-changes/page.tsx | 114 ++++++---- OfficialWebsite/app/docs/extensions/page.tsx | 5 +- .../to-list-async-dynamic-filter/page.tsx | 12 +- .../extensions/to-list-async-summary/page.tsx | 15 +- .../app/docs/policies/attributes/page.tsx | 26 ++- .../app/docs/policies/configuration/page.tsx | 130 +++++++---- .../app/docs/policies/security/page.tsx | 65 +++++- OfficialWebsite/public/llms.txt | 204 ++++++++++++------ README.md | 13 +- build/check-version.ps1 | 4 +- 14 files changed, 444 insertions(+), 211 deletions(-) diff --git a/DynamicWhere.ex/DOC.md b/DynamicWhere.ex/DOC.md index 0177d43..e127a04 100644 --- a/DynamicWhere.ex/DOC.md +++ b/DynamicWhere.ex/DOC.md @@ -657,7 +657,7 @@ In-memory variant — wraps the collection with `AsQueryable()` then delegates t ### `.ToListAsyncDynamic(Filter filter, bool getQueryString = false)` -Async version of `ToListDynamic(Filter)`. Counts with EF Core's `CountAsync()` and, since 3.2.0, reads with EF Core's own `ToListAsync()`, called for the query's element type: the class the projection generates, or `T` when `Selects` is null. It used to read with Dynamic LINQ's `ToDynamicListAsync()`, which reads synchronously on a thread-pool thread; the database command now runs asynchronously and a canceled token reaches it. The rows are the same. +Async version of `ToListDynamic(Filter)`. Counts with EF Core's `CountAsync()` and, since 3.2.0, reads with EF Core's own `ToListAsync()`, called for the query's element type: the class the projection generates, or `T` when `Selects` is null. It used to read with Dynamic LINQ's `ToDynamicListAsync()`, asynchronous as well but with no token to pass on; on an EF Core query a canceled token now reaches the database. The rows are the same. Only the count needs an EF Core async provider; on any other provider the read falls back to Dynamic LINQ's. Since 3.2.0 two overloads take a `CancellationToken`, which reaches the count and the read: `.ToListAsyncDynamic(Filter filter, CancellationToken cancellationToken)` and `.ToListAsyncDynamic(Filter filter, bool getQueryString, CancellationToken cancellationToken)`. See [Cancellation](#cancellation). @@ -691,7 +691,7 @@ In-memory variant for summary operations. ### `.ToListAsync(Summary summary, bool getQueryString = false)` -Async version of `ToList(Summary)`. On an EF Core query it counts the groups with EF Core's `CountAsync()` and reads them with EF Core's `ToListAsync()`. Until 3.2.0 the count ran synchronously and the read went through Dynamic LINQ's `ToDynamicListAsync()`, a synchronous read on a thread-pool thread; on an EF Core query a canceled token now reaches the database. The count and the rows are the same. A source whose provider is not EF Core's, such as rows in memory through `AsQueryable()`, keeps the synchronous count and read. +Async version of `ToList(Summary)`. On an EF Core query it counts the groups with EF Core's `CountAsync()` and reads them with EF Core's `ToListAsync()`. Until 3.2.0 the count ran synchronously and the read went through Dynamic LINQ's `ToDynamicListAsync()`, which had no token to pass on; on an EF Core query a canceled token now reaches the database. The count and the rows are the same. A source whose provider is not EF Core's, such as rows in memory through `AsQueryable()`, keeps the synchronous count and Dynamic LINQ's read, on the calling thread. Since 3.2.0 two overloads take a `CancellationToken`, which reaches the count and the read: `.ToListAsync(Summary summary, CancellationToken cancellationToken)` and `.ToListAsync(Summary summary, bool getQueryString, CancellationToken cancellationToken)`. See [Cancellation](#cancellation). @@ -1630,7 +1630,7 @@ It is never applied: - by an unguarded call. The core extension methods on a plain `IQueryable` or `IEnumerable`, and a query taken out through `AsUnguardedQueryable()`, ignore the attribute and behave exactly as in 3.0; - when the caller sends orders — the default is not appended to them as a tiebreak; - to an `IQueryable` that is already ordered, whether before it was guarded (`db.Tickets.OrderBy(t => t.Title).ApplyPolicy(caller)`) or by a composed `Order` earlier in the chain — even one whose every order the policy dropped, because the caller still sent orders. Since 3.2.0 a `Filter` composed on the handle that sent orders counts the same way. An in-memory sequence sorted with LINQ to Objects before `ApplyPolicy` is not seen as ordered, because `AsQueryable()` hides the sort, so the default replaces that order; -- to a source whose projection could hide a field the default names. Only the outermost `Select` of the chain counts, because it makes the rows the default orders. Since 3.2.0 it hides nothing when it builds `T` itself in an object initializer, `Select(t => new TicketRow { CreatedAt = t.CreatedAt, Id = t.Id, … })`, and assigns every field the default names, at every level of a nested path (`"Owner.Name"` needs `Owner = new OwnerRow { Name = … }`), with nothing EF Core would compute on the client; the default then applies, because EF Core translates an order through a member the projection assigned. A constructor with arguments, a default field the initializer does not assign, a nested path through anything but an initializer, or a default field computed by the application's own method (`Label = Decorate(r.Code)`, which EF Core evaluates on the client, where it can project the value but cannot order by it) leaves the query in its own order, as every projection did in 3.1.0: a default applied there could name a field EF Core cannot translate; +- to a source whose projection could hide a field the default names. Only the outermost `Select` of the chain counts, because it makes the rows the default orders. Since 3.2.0 it hides nothing when it builds `T` itself in an object initializer, `Select(t => new TicketRow { CreatedAt = t.CreatedAt, Id = t.Id, … })`, and assigns every field the default names a column, at every level of a nested path (`"Owner.Name"` needs `Owner = new OwnerRow { Name = … }`). On EF Core a column is a member the model maps on the entity the `Select` reads, read directly, through reference navigations (`t.Owner.Name`) or through `EF.Property` (a shadow property included); in memory any assigned field is one. The default then applies, because EF Core translates an order by a column. A constructor with arguments, a default field the initializer does not assign, a nested path through anything but an initializer, a member the model does not map, or a default field the projection computes, by the application's own method (`Label = Decorate(r.Code)`), a framework one such as `Regex.Replace` or `ToUpper`, or an operator, leaves the query in its own order, as every projection did in 3.1.0: which of these EF Core can order by depends on the provider, and a default must never be the reason a query that ran unguarded fails; - after a projection composed on the handle. The guarded `Select`, or a guarded `Filter` whose `Selects` is set, leaves the rest of the chain unordered even when it keeps every default field, so `guarded.Select(["Id", "Title"]).Page(page)` on a type ordered by `Priority` pages unordered, as in 3.0.0. The default is for the rows the caller's source makes; - to a `Summary`, or by the composable `Where`, `Select`, `Order` or `Group`. @@ -1674,15 +1674,15 @@ A `Selects` entry can name a navigation, such as `"Lines"`, rather than the fiel | `Selects` names a navigation | `Convenience` | `Strict` | |---|---|---| -| with nothing denied beneath it | Kept whole | Kept whole | +| with nothing denied beneath it | Kept whole, or narrowed around a transform that lands on a property with no setter (3.2.0) | Kept whole, or narrowed the same way | | with a denied field beneath it | Replaced by the allowed fields beneath it | `FieldDeniedForSelect` | | whose key, `Lines.Id`, is denied (3.2.0) | `FieldDeniedForSelect` | `FieldDeniedForSelect` | | with a denied field beneath it, where the narrowing cannot be built (3.2.0) | `FieldDeniedForSelect` | `FieldDeniedForSelect` | -| whose type holds a field denied for `Select` that no path names (3.2.0) | Narrowed away, where it can be narrowed | `FieldDeniedForSelect` | +| that can carry a field denied for `Select` that no path names: past four segments, in a framework generic, on a subtype, or unasked under a `"*"` deny (3.2.0) | Narrowed to the allowed fields where the core can narrow it; `FieldDeniedForSelect` where it cannot | `FieldDeniedForSelect` | - The fields beneath a member are read the way the attribute walker reads them: through any collection type, and no deeper than its four segments. Since 3.2.0 a member typed `IReadOnlyList`, `IReadOnlyCollection`, `Collection` or an application's own collection no longer hides the denials beneath it. The providers' own rules are asked too, so a denied property with no setter and a rule on a path reached through a cycle count. - A narrowing that cannot be built as it was gated is refused in both tiers (3.2.0). The core's typed projection adds the key, `Id`, of every nested node it builds, so a navigation narrowed around its own denied key would get the key back. The core reads a path only through an array, `List`, `IList`, `ICollection`, `IEnumerable`, `HashSet` or `ISet`, so a narrowing through any other collection fails its validation. And some members cannot be narrowed at all: a column, a complex property or a member stored as JSON, which EF Core reads whole; a member of a row in memory; and a member a projection builds some way the core cannot narrow. -- A member that carries everything its type holds — a member of a projected or in-memory row, or an entity's column, owned or complex member — can hold a field denied for `Select` that no path names: one deeper than four segments, or inside a framework generic such as `Dictionary`. The `Strict` tier refuses such a member, and the `Convenience` tier narrows it away (3.2.0); where it cannot be narrowed, as a member of a row in memory cannot, both tiers refuse it. An entity's navigation loads only its own entity, so only its paths are asked about. +- A named member can also carry a field denied for `Select` that no path names (3.2.0): one deeper than four segments, one inside a framework generic such as `Dictionary`, or one a subtype of the member's type declares — a derived entity, a subclass, an interface's implementation. What it can carry is read from the source. On an entity it is read from the EF Core model, so only what loads counts: the navigation's columns, a converted one included, its owned chain at any depth, and the navigations beneath it an include, an automatic include or a lazy loader fills, for its type and every type the model derives from it. On a projected or in-memory row it is the member's type and every loaded subtype of it. Under a policy with a `"*"` deny, a path the walk never asks about — past four segments, around a cycle, with no setter, or on a subtype — is a denied one as well. The `Strict` tier refuses such a member. The `Convenience` tier narrows it where the core can, which builds the declared type and so drops a subtype's fields; a path naming a framework generic itself narrows to nothing and is dropped. Where the core cannot narrow it — a column at the top of `T`, a member of a row in memory — both tiers refuse it. - A navigation named through another, `Main.Lead`, gates the key of every node it passes through, which the core's projection adds, as a dotted path to a value always did (3.2.0). A denied key refuses the projection. Naming a field beside a denied key, `Lines.Name` when `Lines.Id` is denied, was already refused in both tiers. - Under `Convenience` the refusal names the denied key, the first denied field beneath the member, or, for a denial no path names, the member itself. Under `Strict` its `FieldPath` is `"*"`, as on every field refusal. The trace records the reason either way. @@ -1695,9 +1695,12 @@ The projection keeps the **allowed members**: what an unguarded call would retur **When a projection is needed.** - A field denied at the top of `T` always asks for one, whatever it holds: a scalar, a blob, a list, an owned object or a JSON column. -- A field denied beneath a member asks for one when its value can reach the result. On an entity, that is beneath a column, an owned or complex member, or a navigation something loads: an `Include` or `ThenInclude` on the query, an automatic include, or a lazy loader — EF Core's proxies, or an injected `ILazyLoader` — which fills a navigation after the query. An `Include` written in a form the library cannot read counts as loading every navigation. On a row a projection builds, it is beneath a member the initializer assigns. On a row in memory, it is beneath any member. +- A field denied beneath a member asks for one when its value can reach the result. On an entity, that is beneath a column, an owned or complex member, or a navigation something loads: an `Include` or `ThenInclude` on the query, an automatic include, or a lazy loader — EF Core's proxies, an injected `ILazyLoader`, or a loader delegate or `ILazyLoader` the constructor takes and keeps in a field or any property — which fills a navigation after the query. On a row a projection builds, it is beneath a member the initializer assigns; a constructor with arguments, with or without an initializer after it, counts every member as assigned. On a row in memory, it is beneath any member. A rule may spell the path in any letter case. +- Every navigation counts as loaded where the library cannot read which the query loads: an `Include` in a form it cannot read, one off the query's own chain, and a chain that reaches its rows through anything but the root's own rows — `Select(o => o.Customer)`, a `SelectMany`, a `Join`, or a projection behind another `Select` (an identity `Select`, a member of an anonymous row, a conditional). EF Core still applies includes named from the root to the entities such a chain reaches. - A denial beneath a navigation nothing loads never leaves the database, so it asks for no projection. An entity whose only denials sit beneath such navigations is read as it was in 3.1.0. -- A member whose type can hold a field denied for `Select` that no path names — deeper than the walker's four segments, or inside a framework generic such as `Dictionary` — asks for one too, and so does a member typed `object`, when what it holds can reach the result. +- A member whose value can hold a field denied for `Select` that no path names — deeper than the walker's four segments, inside a framework generic such as `Dictionary`, or declared by a subtype of its type — asks for one too, read as for a named member, so on an entity only what loads counts. So does a member that can hold an object of any type: one typed `object`, a framework interface such as `IComparable`, or a collection that is not generic (`IEnumerable`, `ArrayList`, `Array`). Under a `"*"` deny, so does a member whose value can hold a path the walk never asks about. +- A row can be a subtype of `T`. On an entity, a member a type the model derives from `T` declares, and what loads beneath it, counts as one of `T`'s own would; on a row in memory, a member any loaded subtype declares does. The projection builds `T` and leaves them out, recorded with a reason starting `left out: a type derived`. A row a projection builds is exactly `T`. +- A member EF Core does not map holds only what the class puts there, never a value the query read, so it asks for nothing. - The denials beneath a member come from the providers' rules as well as from walking the type, so a denied property with no setter, a rule on a path reached through a cycle, and a rule deeper than the walk all count. - A forced scope beneath a member asks for no projection on its own. It filters the rows that hold the member, as it always has. When a projection is needed anyway, the member is left out whole. @@ -1705,7 +1708,7 @@ The projection keeps the **allowed members**: what an unguarded call would retur | Source | Values kept | Objects kept | |---|---|---| -| A projection that builds its rows before `ApplyPolicy`: the outermost `Select` constructs the row, in an object initializer or with a constructor, as in `db.Roles.Select(r => new RoleRow { … })` | Every member the initializer assigns; every member when a constructor builds the row | The same | +| A projection that builds its rows before `ApplyPolicy`: the outermost `Select` constructs the row, in an object initializer or with a constructor, as in `db.Roles.Select(r => new RoleRow { … })` | Every member the initializer assigns; every member when a constructor with arguments builds the row, with or without an initializer after it | The same | | An entity query, or a `Select` that hands back an entity, as in `db.Orders.Select(o => o.Customer)` | Every member EF Core maps | Its columns, converted and JSON ones included, its owned members and, on EF Core 8 or later, its complex properties, read from the EF Core model | | Rows in memory, as in `roles.ApplyPolicy(caller)` | Every member | None | @@ -1713,8 +1716,8 @@ A value EF Core does not map is left out: computing it would make EF Core read t **Whole, narrowed or left out whole.** A member holding an object that the source carries is: -- **kept whole** when nothing beneath it is denied, nothing in its type is denied or typed `object`, no forced scope is beneath it, and no transform beneath it lands on a property with no setter; -- **narrowed** otherwise, to the allowed fields beneath it, four segments deep, as a caller naming it would get it, where the core's narrowing translates: an object the projection's initializer builds, a list a subquery reads into a type the core can bind (not an array or a set), a navigation that is neither complex nor stored as JSON, or an entity's owned member not stored as JSON. A field beneath it that can hold what the policy cannot name is left out; +- **kept whole** when nothing beneath it is denied, nothing its value can hold is denied (its subtypes included) or can hold an object of any type, under a `"*"` deny the walk asks about every path beneath it, no forced scope is beneath it, and no transform beneath it lands on a property with no setter; +- **narrowed** otherwise, to the allowed fields beneath it, four segments deep, as a caller naming it would get it, where the core's narrowing translates: an object the projection's initializer builds, a list a subquery reads into a type the core can bind (not an array or a set), a navigation that is neither complex nor stored as JSON, or an entity's owned member not stored as JSON. The narrowing builds the declared type, so a subtype's fields are dropped. A field beneath it that can hold what the policy cannot name is left out; - **left out whole** otherwise, and recorded as `Dropped` on `Select` with a reason that starts `left out whole`. ```text @@ -1733,7 +1736,7 @@ The last one is recorded on the field beneath the member that the narrowing leav **Never kept.** -- An entity's navigation, included or not: projecting it would load it. So once a denial needs a projection, an included or automatically included navigation is not returned. Name it in `Selects` to get it, narrowed. +- An entity's navigation, included or not: projecting it would load it. So once a denial needs a projection, an included or automatically included navigation is not returned. Under `Convenience`, name it in `Selects` to get it narrowed; under `Strict`, name its allowed fields. - An object held by a row in memory: a kept object is the caller's own, and a transform beneath it would change it in place. The projection's rows are new and hold no member of an object type, so the source objects are left as they were. - A member with no setter, and a member named with one of the expression parser's own words. @@ -1747,7 +1750,9 @@ The last one is recorded on the field beneath the member that the narrowing leav - A dry run synthesizes nothing. It records the denials and returns the rows whole. - A simulation has no source, so it reads `T` as a source it cannot see into: every denial beneath a member counts, and the projection it shows keeps only members holding a value (see [Administration](#administration)). -**What the policy cannot see into.** A member typed `object` is opaque to the policy: a synthesized projection leaves it out, and naming it returns whatever it holds. A framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: naming it is refused under `Strict` and narrowed away under `Convenience`, or refused there too where the member cannot be narrowed, and a synthesized projection leaves it out. Hold such values in a list of the policed type instead. +**What the policy cannot see into.** A member typed `object`, a framework interface or a collection that is not generic, such as `IEnumerable`, `ArrayList` or `Array`, is opaque to the policy: a synthesized projection leaves it out, and naming it returns whatever it holds. A framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: naming it is refused in both tiers where the core cannot narrow it, at the top of `T` or on a row in memory, narrowed away under `Convenience` beneath a navigation, and a synthesized projection leaves it out. Hold such values in a list of the policed type instead. A member EF Core does not map holds what the class computes, and the policy reads nothing into it: a getter that copies a denied column is the application's to withhold. + +**A projection builds the declared type.** A query over the root of a hierarchy whose derived type declares a denied field comes back as root-type rows, the derived types' allowed fields dropped too. Over an abstract root the typed terminals fail with `SelectTypeMustHaveParameterlessConstructor`, and the dynamic ones return the root's members. Query the derived type, `OfType()`, to keep its fields. Under a `"*"` deny, a member is kept whole only when the walk asks about every path beneath it. **A forced scope on a list's element type filters rows, not elements.** A forced scope declared on a list's element type filters the rows that hold the list, never its elements. `Selects` naming the list returns every element, those the scope excludes included, as in every release; a synthesized projection leaves such a list out. Scope the elements where the row is built. @@ -2311,9 +2316,11 @@ All validation errors throw `LogicException` (inherits `Exception`) with one of Fixed (security): a field denied only beneath a member, none at the top of `T`, synthesized nothing, so the whole row came back with the denied value in it — in a list or nested object of a row projected before `ApplyPolicy`, in a row held in memory, and in an entity's included, automatically included, lazily loaded or owned member — typed and dynamic, in both tiers, for a `Filter` and a `Segment`. Such a denial now synthesizes the projection whenever its value can reach the result: on an entity, beneath a column, an owned or complex member, or a navigation the query loads through `Include`, an automatic include or a lazy loader; on a projected row, beneath a member the initializer assigns; in memory, beneath any member. A denial beneath a navigation nothing loads never leaves the database, and the entity is read exactly as in 3.1.0. + Fixed (security): what a query loads was read too narrowly. An include named from the root and reached through `Select(o => o.Customer)`, `SelectMany` or `Join`, a projection behind another `Select`, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or a property of any name each loaded a denied value the gate read as unloaded. A field a subtype declares, a derived entity's or a subclass's held by a base-typed member, was not read at all, and under a `"*"` deny a path the walk never asked about was allowed. Each came back. Now every navigation counts as loaded on such a chain, the subtypes are read, and such a path is denied. When a type the model derives from `T`, or a loaded subclass of a row in memory, declares a denied field, the rows are projected to `T`, dropping a derived type's allowed fields too; over an abstract `T` the typed terminals fail with `SelectTypeMustHaveParameterlessConstructor` and the dynamic ones return its members. + Fixed (security): a field denied at the top of `T` whose own type is not a simple value — a byte array, a list, an owned object, a JSON column — synthesized no projection either, so with nothing else denied the whole row came back with it. - Changed: the projection kept simple fields only, so as soon as any field was denied, every nested object and list of a row projected before `ApplyPolicy` came back null or empty, and so did an entity's columns holding an object, its owned and complex members and its collections of simple values. A row a projection builds now keeps the members its initializer assigns; an entity keeps its mapped columns, converted and JSON ones included, its owned and complex members, and every collection of simple values; rows in memory keep their values. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, with a `Dropped` decision whose reason starts `left out whole`. An entity's navigations, included ones too, the objects of a row in memory, and a value EF Core does not map are left out; name a navigation in `Selects` to get it, narrowed. A forced scope beneath a member asks for no projection on its own, and a projection needed for another reason leaves such a member out whole. A typed query needs `T` to have a public parameterless constructor for the projection, as it already did (point 1). + Changed: the projection kept simple fields only, so as soon as any field was denied, every nested object and list of a row projected before `ApplyPolicy` came back null or empty, and so did an entity's columns holding an object, its owned and complex members and its collections of simple values. A row a projection builds now keeps the members its initializer assigns; an entity keeps its mapped columns, converted and JSON ones included, its owned and complex members, and every collection of simple values; rows in memory keep their values. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, with a `Dropped` decision whose reason starts `left out whole`. An entity's navigations, included ones too, the objects of a row in memory, and a value EF Core does not map are left out; under `Convenience` name a navigation in `Selects` to get it narrowed, and under `Strict` name its allowed fields. A forced scope beneath a member asks for no projection on its own, and a projection needed for another reason leaves such a member out whole. A typed query needs `T` to have a public parameterless constructor for the projection, as it already did (point 1). 21. **`Selects` Naming a Member Is Gated Against Every Denial Beneath It** When `Selects` names a navigation with a denied field beneath it, the `Convenience` tier replaces the entry with the allowed fields beneath it, and the `Strict` tier refuses it. Since 3.2.0 the gate finds every denial beneath the member, and refuses, with `FieldDeniedForSelect`, a narrowing it cannot build as gated. See [A navigation named in Selects](#a-navigation-named-in-selects). @@ -2325,12 +2332,12 @@ All validation errors throw `LogicException` (inherits `Exception`) with one of Fixed (security): a member of a projected or in-memory row whose type holds a field denied for `Select` that no path names — deeper than four segments, or inside a framework generic such as `Dictionary` — was returned whole. The strict tier refuses it now, and the convenience tier narrows it away; where it cannot be narrowed, both tiers refuse it. Denials beneath a named member are also read from the policy's own rules, so a denied property with no setter and a rule on a path reached through a cycle are found. A member that cannot be narrowed at all — a column, a complex property or a JSON-stored member, a member of a row in memory, or one a projection builds some way the core cannot narrow — is refused in both tiers when something beneath it is denied. A request that sends no `Selects` is not refused for such a member: its synthesized projection narrows it or leaves it out whole. 22. **`DefaultOrder` Reaches a Projection That Builds `T`** - In 3.1.0 a `Select` anywhere in the chain kept a guarded query in its own order. Since 3.2.0 only the outermost `Select` counts, and when it builds `T` in an object initializer that assigns every field the default names, at every level of a nested path, with nothing EF Core would compute on the client, the default applies: `db.Tickets.Select(t => new TicketRow { Id = t.Id, CreatedAt = t.CreatedAt, Title = t.Title }).ApplyPolicy(caller)` on a `TicketRow` declaring `"CreatedAt desc, Id"` was unordered and is now ordered. A constructor with arguments, a default field the initializer does not assign, a nested path through anything but an initializer, or a default field computed by the application's own method, which EF Core evaluates on the client and cannot order by, still leaves the query in its own order. A `Select`, or a `Filter` with `Selects`, composed on the guarded handle keeps the rest of the chain unordered, and a composed `Filter` that sent orders gets no default later in the chain, even when the policy dropped every one of them, as a composed `Order` already did not. See [Default order](#default-order). + In 3.1.0 a `Select` anywhere in the chain kept a guarded query in its own order. Since 3.2.0 only the outermost `Select` counts, and when it builds `T` in an object initializer that assigns every field the default names a column, at every level of a nested path, the default applies: `db.Tickets.Select(t => new TicketRow { Id = t.Id, CreatedAt = t.CreatedAt, Title = t.Title }).ApplyPolicy(caller)` on a `TicketRow` declaring `"CreatedAt desc, Id"` was unordered and is now ordered. A column is a member the EF Core model maps on the entity the `Select` reads, read directly, through reference navigations or through `EF.Property`. A value the projection computes, by any method or operator, a member the model does not map, a constructor with arguments, a default field the initializer does not assign, or a nested path through anything but an initializer still leaves the query in its own order. A `Select`, or a `Filter` with `Selects`, composed on the guarded handle keeps the rest of the chain unordered, and a composed `Filter` that sent orders gets no default later in the chain, even when the policy dropped every one of them, as a composed `Order` already did not. See [Default order](#default-order). 23. **Every Async Terminal Takes a `CancellationToken`** - Since 3.2.0 `ToListAsync` and `ToListAsyncDynamic` with a `Filter`, `ToListAsync` with a `Summary`, and `ToListAsync` with a `Segment` have overloads that take a `CancellationToken`, guarded and unguarded, and the token reaches the count and the read. The 3.1 signatures are unchanged, so code compiled against 3.1 still binds, but `ToListAsync(filter, default)`, `ToListAsyncDynamic(filter, default)` and `ToListAsync(summary, default)` no longer compile: `default` fits both `getQueryString` and the token (CS0121). Write `false`, a token, or a named argument. A reflection lookup of `ToListAsyncDynamic` by name alone now finds three methods where it found one. See [Cancellation](#cancellation). + Since 3.2.0 `ToListAsync` and `ToListAsyncDynamic` with a `Filter`, `ToListAsync` with a `Summary`, and `ToListAsync` with a `Segment` have overloads that take a `CancellationToken`, guarded and unguarded, and the token reaches the count and the read. The 3.1 signatures are unchanged, so code compiled against 3.1 still binds, but `ToListAsync(filter, default)`, `ToListAsyncDynamic(filter, default)` and `ToListAsync(summary, default)` no longer compile: `default` fits both `getQueryString` and the token (CS0121). Write `false`, a token, or a named argument. A reflection lookup of `ToListAsyncDynamic` by name alone now finds three methods where it found one, and one of `ToListAsync` finds more than it did. See [Cancellation](#cancellation). - Changed: `ToListAsyncDynamic` and the async `Summary` read through EF Core's `ToListAsync`, and the async `Summary` counts through `CountAsync`. They used to read synchronously on a thread-pool thread, and the summary counted synchronously, so on an EF Core query a canceled token now reaches the database. The rows and the counts are the same. A provider that is not EF Core's keeps the synchronous read. + Changed: `ToListAsyncDynamic` and the async `Summary` read through EF Core's `ToListAsync` instead of Dynamic LINQ's `ToDynamicListAsync`, which had no token to pass on, and the async `Summary` counts through `CountAsync` where it counted synchronously. So on an EF Core query a canceled token now reaches the database. The rows and the counts are the same. A provider that is not EF Core's keeps Dynamic LINQ's read, on the calling thread. 24. **A Type in a Namespace That Starts with `System` Is Policed** The attribute walker does not descend into the framework's own types, which carry no policy attributes. Until 3.2.0 it took any namespace whose name started with `System` for the framework's, so an application namespace such as `SystemsCorp.Payroll` or `SystemX.Domain` got no policy beneath its types, and a `[DwDenied]` field on such a type, reached through a member, was returned, filterable and sortable. Fixed (security): only `System` and the namespaces beneath it are the framework's now, so a guarded request that filtered on, sorted by or selected such a field is refused or dropped, as for any denied field. diff --git a/DynamicWhere.ex/DynamicWhere.ex.csproj b/DynamicWhere.ex/DynamicWhere.ex.csproj index 76437e8..61cab00 100644 --- a/DynamicWhere.ex/DynamicWhere.ex.csproj +++ b/DynamicWhere.ex/DynamicWhere.ex.csproj @@ -52,21 +52,23 @@ Security fix: a field denied for Select only beneath a member was not enforced when the caller sent no Selects. A guarded query synthesizes a projection for a denied field, and it did so only when a simple field at the top of the type was denied. With every denial beneath a member, the whole row came back with the denied value in it: in a list or nested object of a row projected before ApplyPolicy, in a row held in memory, and in an entity's included, automatically included, lazily loaded or owned member, typed and dynamic, in both tiers, for a Filter and a Segment. Such a denial now synthesizes the projection whenever its value can reach the result. On an entity that means beneath a column, an owned or complex member, or a navigation the query loads through Include, an automatic include or a lazy loader; a denial beneath a navigation nothing loads never leaves the database, and the entity is read exactly as before. +Security fix: what a query loads, and what a member holds, was read too narrowly. An include named from the root and reached through Select(o => o.Customer), SelectMany or Join, a projection behind another Select, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or any property each loaded a denied value the gate read as unloaded. A field a subtype declares, a derived entity's or a subclass's held by a base-typed member, was not read at all, and under a "*" deny with exact allows a path the walk never asked about (past four segments, around a cycle, with no setter) resolved as allowed. Selects naming an entity navigation returned a denial in its owned chain past four segments or in a converted Dictionary<string, T> column. All of these are read now: from the EF Core model for an entity, so only what loads counts, and from every loaded subtype for a projected or in-memory row. Rows whose derived type declares a denied field come back as the queried type. + Security fix: a field denied at the top of a type whose own type is not a simple value, such as a byte array, a list, an owned object or a JSON column, synthesized no projection either, so with nothing else denied it came back. Security fix: the attribute walker read any namespace starting with "System" as the framework's, so an application namespace such as SystemsCorp.Payroll got no policy beneath its types, and a [DwDenied] field there was returned, filterable and sortable. Only System and the namespaces beneath it are treated as the framework's now. Security fix: under the convenience tier, Selects naming a navigation whose element key (Id) is denied was narrowed to the allowed fields beneath it, and the core's typed projection added the key back. Such a narrowing is refused with FieldDeniedForSelect in both tiers, as naming a sibling of the key already was. A navigation named through another, such as Main.Lead, now gates the key of Main, which the projection adds; it did not. -Security fix: Selects naming a member typed as a collection the core does not unwrap, such as IReadOnlyList<T>, returned every field beneath it, denied ones included, in both tiers. The projection gate read collections through a narrower list than the attribute walker that puts policy on the fields beneath; it now reads them the same way. Denials beneath a named member are also read from the policy's own rules, so a denied property with no setter and a rule on a path reached through a cycle are found, and a field denied deeper than the walker reaches or inside a framework collection such as Dictionary<string, T> is refused under the strict tier and narrowed away under the convenience tier. +Security fix: Selects naming a member typed as a collection the core does not unwrap, such as IReadOnlyList<T>, returned every field beneath it, denied ones included, in both tiers. The projection gate read collections through a narrower list than the attribute walker that puts policy on the fields beneath; it now reads them the same way. Denials beneath a named member are also read from the policy's own rules, so a denied property with no setter and a rule on a path reached through a cycle are found, and a member carrying a field denied where no path reaches it, deeper than the walker, inside a framework collection such as Dictionary<string, T> or on a subtype, is refused under the strict tier, and under the convenience tier narrowed where the core can narrow it and refused where it cannot. -Behaviour change: the projection synthesized for a guarded query that sends no Selects keeps what the source carries. It used to keep simple fields only, so every nested object and list of a row projected before ApplyPolicy came back null or empty as soon as any field was denied. A row a projection builds keeps the members its initializer assigns. An entity keeps its mapped columns, converted and JSON ones included, its owned and complex members, and every collection of simple values such as byte[] or List<string>. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, recorded as Dropped with a reason starting "left out whole". An entity's navigations, the objects of a row in memory, and a value EF Core does not map are left out, and the type needs a public parameterless constructor for the projection, as it already did. +Behaviour change: the projection synthesized for a guarded query that sends no Selects keeps what the source carries. It used to keep simple fields only, so every nested object and list of a row projected before ApplyPolicy came back null or empty as soon as any field was denied. A row a projection builds keeps the members its initializer assigns. An entity keeps its mapped columns, converted and JSON ones included, its owned and complex members, and every collection of simple values such as byte[] or List<string>. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, recorded as Dropped with a reason starting "left out whole". An entity's navigations, the objects of a row in memory, and a value EF Core does not map are left out, and the type needs a public parameterless constructor for the typed projection, as it already did. A member EF Core does not map asks for no projection. -Behaviour change: [DwEntity(DefaultOrder)] applies to a projected source whose outermost Select builds the type in an object initializer and assigns every field the default names, at every level of a nested path, with nothing EF Core would have to compute on the client. Any other projection still leaves the query in its own order. A Select, or a Filter with Selects, composed on the guarded handle keeps the rest of the chain unordered, and a composed Filter that sent orders gets no default later in the chain, as a composed Order already did not. +Behaviour change: [DwEntity(DefaultOrder)] applies to a projected source whose outermost Select builds the type in an object initializer and assigns every field the default names a column, at every level of a nested path: a mapped member read directly, through reference navigations or through EF.Property. A computed value or any other projection still leaves the query in its own order. A Select, or a Filter with Selects, composed on the guarded handle keeps the rest of the chain unordered, and a composed Filter that sent orders gets no default later in the chain, as a composed Order already did not. -New: every asynchronous terminal has overloads taking a CancellationToken, guarded and unguarded: ToListAsync and ToListAsyncDynamic with a Filter, ToListAsync with a Summary, and ToListAsync with a Segment. The token reaches both the count and the read. The overloads sit beside the 3.1 signatures, which are unchanged, so code compiled against 3.1 still binds. ToListAsync(filter, default) no longer compiles, because default fits both overloads, and a reflection lookup of one of these methods by name alone now finds several. +New: every asynchronous terminal has overloads taking a CancellationToken, guarded and unguarded: ToListAsync and ToListAsyncDynamic with a Filter, ToListAsync with a Summary, and ToListAsync with a Segment. The token reaches both the count and the read. The overloads sit beside the 3.1 signatures, which are unchanged, so code compiled against 3.1 still binds. ToListAsync(filter, default) no longer compiles, because default fits both overloads, and a reflection lookup of one of these methods by name alone finds more overloads than it did. -Change: ToListAsyncDynamic and the asynchronous Summary read through EF Core's ToListAsync, and the Summary counts through CountAsync. They used to read synchronously on a thread-pool thread, so on an EF Core query a canceled token now reaches the database. A provider that is not EF Core's keeps the synchronous read. +Change: ToListAsyncDynamic and the asynchronous Summary read through EF Core's ToListAsync instead of Dynamic LINQ's ToDynamicListAsync, which had no token to pass on, and the Summary counts through CountAsync where it counted synchronously. So on an EF Core query a canceled token now reaches the database. A provider that is not EF Core's keeps Dynamic LINQ's read, on the calling thread. v3.1.0 — Date comparisons that work on every date member, segments combined in the database, five new caps, a stable code where a sentence used to be, preparation enforced whether or not a store is configured, a policy bypass through members named Root, It or Parent closed, a long In list that ended the process fixed, the names the expression parser keeps refused by name, a strict tier that no longer discloses its trace or which fields exist, forced predicates that can let rows with no value through, a declared default order for guarded queries, and refused queries written to the audit. diff --git a/DynamicWhere.ex/Source/AsyncReads.cs b/DynamicWhere.ex/Source/AsyncReads.cs index 92e1508..47692f2 100644 --- a/DynamicWhere.ex/Source/AsyncReads.cs +++ b/DynamicWhere.ex/Source/AsyncReads.cs @@ -12,12 +12,12 @@ namespace DynamicWhere.ex.Source; /// /// /// EF Core's own asynchronous operators are generic in the element type, and a dynamic query's element -/// type is a class generated for its projection, so they are reached by reflection. Through them the -/// provider runs the command asynchronously and a cancellation reaches the database. System.Linq.Dynamic.Core's -/// ToDynamicListAsync, which these replace on an EF Core query, runs the synchronous read on a -/// thread-pool thread and checks the token only before it starts. +/// type is only known at run time, so they are reached by reflection. Through them a cancellation reaches +/// the database. They replace System.Linq.Dynamic.Core's ToDynamicListAsync on an EF Core query, and +/// Count(), which counted a grouping synchronously. /// -/// Any other provider, rows in memory among them, keeps the synchronous read it had. +/// Any other provider, rows in memory among them, keeps ToDynamicListAsync, which reads on the +/// calling thread. /// /// internal static class AsyncReads diff --git a/OfficialWebsite/app/docs/ai/page.tsx b/OfficialWebsite/app/docs/ai/page.tsx index 61eba2c..fbc40d9 100644 --- a/OfficialWebsite/app/docs/ai/page.tsx +++ b/OfficialWebsite/app/docs/ai/page.tsx @@ -136,8 +136,8 @@ DynamicWhere.ex code. It is the complete API surface.`} - The reference is generated against version 3.2.0 from the source, and - checked by running the library, so it states behaviour — including the + The reference is written against version 3.2.0 from the source, by + hand, and checked by running the library, so it states behaviour — including the parts that are deliberately blunt, such as neither hashing nor tokenization hiding equality. Where a page in these docs disagrees with it, the file is the one to trust. For the reasoning behind a rule, the human diff --git a/OfficialWebsite/app/docs/breaking-changes/page.tsx b/OfficialWebsite/app/docs/breaking-changes/page.tsx index 5eeb120..543138f 100644 --- a/OfficialWebsite/app/docs/breaking-changes/page.tsx +++ b/OfficialWebsite/app/docs/breaking-changes/page.tsx @@ -1115,12 +1115,17 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say a column, an owned or complex member, or a navigation the query loads through an Include, an automatic include or a lazy loader; on a projected row, beneath a member the initializer assigns; - in memory, beneath any member. It does so in both tiers, typed and - dynamic, for a Filter and a Segment. Until - 3.2.0 only a simple field denied at the top of T asked for - one. A denial beneath a navigation nothing loads never leaves the - database, so an entity whose only denials sit there is read exactly as - in 3.1.0. + in memory, beneath any member. A chain that reaches its rows through + a navigation, a SelectMany, a Join or a + projection behind another Select counts every navigation + as loaded. A field a subtype of T declares, and one a + subtype of a member's type declares, count too. It does so in both + tiers, typed and dynamic, for a Filter and a{" "} + Segment. Until 3.2.0 only a simple field denied at the top + of T asked for one. A denial beneath a navigation nothing + loads never leaves the database, and a member EF Core does not map + holds nothing it read, so an entity whose only denials sit there is + read exactly as in 3.1.0.
  • What. The allowed members, which replace the allowed @@ -1143,6 +1148,28 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say owned member — typed and dynamic, in both tiers, for a{" "} Filter and a Segment. + + An include named from the root and reached through{" "} + {`Select(o => o.Customer)`}, SelectMany or{" "} + Join, a projection behind another Select, an + initializer after a constructor with arguments, and a lazy loader the + constructor takes and keeps in a field or a property of any name each + loaded a denied value the gate read as unloaded. A field a subtype + declares — a derived entity's, or a subclass's held by a + base-typed member — was not read at all, and under a{" "} + "*" deny a path the walk never asked about was + allowed. Each came back. + + + When a type the model derives from T, or a loaded subclass + of a row in memory, declares a denied field, the rows are projected to{" "} + T, so a derived type's allowed fields are dropped too, + and a member declared as a base type is narrowed to it. Over an abstract{" "} + T the typed terminals fail with{" "} + SelectTypeMustHaveParameterlessConstructor; the dynamic ones + return its members. Query the derived type,{" "} + {`OfType()`}, to keep its fields. + A field denied at the top of T whose own type is not a simple value — a byte array, a list, an owned object, a JSON column — @@ -1160,8 +1187,10 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say Once a projection is needed it leaves out an entity's navigations, - included ones too, since projecting one would load it: name the - navigation in Selects to get it, narrowed. It leaves out + included ones too, since projecting one would load it: under{" "} + Convenience name the navigation in Selects to + get it narrowed, and under Strict name its allowed fields. + It leaves out the objects a row in memory holds, since a kept object is the caller's own and a transform would change it in place, and a value EF Core does not map, which EF Core could compute only by reading the @@ -1233,16 +1262,18 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say - A member of a projected or in-memory row whose type holds a field - denied where no path reaches it: deeper than four segments, or - inside a framework generic such as{" "} - Dictionary<string, T> + A member that carries a field denied where no path reaches it: + deeper than four segments, inside a framework generic such as{" "} + Dictionary<string, T>, declared by a subtype of + its type, in an entity navigation's owned chain or converted + column, or, under a "*" deny, on a path the + walk never asks about Returned, the denied field included. - Refused under Strict, narrowed away under{" "} - Convenience, and refused in both tiers where it - cannot be narrowed. + Refused under Strict. Under Convenience{" "} + narrowed where the core can narrow it, which builds the declared + type, and refused where it cannot. @@ -1268,14 +1299,16 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say projection narrows the member or leaves it out whole (point 25). - A member typed object is opaque to the policy: a + A member typed object, a framework interface or a + collection that is not generic, such as IEnumerable,{" "} + ArrayList or Array, is opaque to the policy: a synthesized projection leaves it out, and naming it returns whatever it holds. A framework generic holding a policed type, such as{" "} Dictionary<string, LineDto>, has no paths beneath it: - naming it is refused under Strict and narrowed away under{" "} - Convenience, or refused there too where the member cannot - be narrowed, and a synthesized projection leaves it out. - Hold such values in a list of the policed type instead. + naming it is refused in both tiers where the core cannot narrow it, + narrowed away under Convenience beneath a navigation, and a + synthesized projection leaves it out. Hold such values in a list of the + policed type instead.

    27. DefaultOrder Reaches a Projection That Builds T

    @@ -1287,12 +1320,16 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say counts, because it makes the rows the default orders. When it builds{" "} T in an object initializer and assigns every field the{" "} [DwEntity(DefaultOrder)]{" "} - names, at every level of a nested path, with nothing EF Core would - compute on the client, the default applies. A constructor with - arguments, a default field the initializer does not assign, a nested - path through anything but an initializer, or a default field computed - by the application's own method, which EF Core evaluates on the - client and cannot order by, still leaves the query in its own order. + names a column, at every level of a nested path, the default applies. A + column is a member the EF Core model maps on the entity the{" "} + Select reads, read directly, through reference navigations + or through EF.Property; in memory any assigned field is + one. A value the projection computes, by any method or operator, even + one EF Core could translate, a member the model does not map, a + constructor with arguments, a default field the initializer does not + assign, or a nested path through anything but an initializer still + leaves the query in its own order: ordering by it could fail where the + unguarded query ran.

    {`[DwEntity(DefaultOrder = "CreatedAt desc, Id")] public class TicketRow @@ -1343,11 +1380,11 @@ var rows = db.Tickets with a Segment. The token reaches the count and the read. The overloads sit beside the 3.1 signatures, which are unchanged, so code compiled against 3.1 still binds. That brings the extension methods to - 28. A reflection lookup by name alone now finds several methods where - it found one — ToListAsyncDynamic, on the extension class - and on the guarded handle — so Type.GetMethod given only - the name throws AmbiguousMatchException; pass the - parameter types. + 28. A reflection lookup by name alone finds more overloads than it did, + and where it found one — ToListAsyncDynamic, on the + extension class and on the guarded handle — it now finds several, so{" "} + Type.GetMethod given only the name throws{" "} + AmbiguousMatchException; pass the parameter types.

    default fits both bool getQueryString and the @@ -1359,14 +1396,15 @@ var rows = db.Tickets await query.ToListAsync(filter, false); // as 3.1 read it await query.ToListAsync(filter, cancellationToken); // the new overload`}     - + ToListAsyncDynamic and the async Summary read - through EF Core's ToListAsync, and the async{" "} - Summary counts through CountAsync. They used to - read synchronously on a thread-pool thread, and the summary counted - synchronously, so on an EF Core query a canceled token now reaches the - database. The rows and the counts are the same. A provider that is not - EF Core's keeps the synchronous read. + through EF Core's ToListAsync instead of Dynamic + LINQ's ToDynamicListAsync, which had no token to pass + on, and the async Summary counts through{" "} + CountAsync where it counted synchronously. So on an EF Core + query a canceled token now reaches the database. The rows and the counts + are the same. A provider that is not EF Core's keeps Dynamic + LINQ's read, on the calling thread.

    29. A Type in a Namespace That Starts with System Is Policed

    diff --git a/OfficialWebsite/app/docs/extensions/page.tsx b/OfficialWebsite/app/docs/extensions/page.tsx index 1b4c94e..253a81f 100644 --- a/OfficialWebsite/app/docs/extensions/page.tsx +++ b/OfficialWebsite/app/docs/extensions/page.tsx @@ -169,9 +169,10 @@ export default function Page() { Core's CountAsync and read with EF Core's{" "} ToListAsync. Since 3.2.0 that includes the dynamic{" "} Filter's read and the Summary's count - and read, which used to run synchronously.{" "} + and read: the count used to run synchronously, and the reads through + Dynamic LINQ, with no token to pass on.{" "} ToListAsync(Summary) on a provider that is not EF - Core's keeps its synchronous count and read, and{" "} + Core's keeps its synchronous count and Dynamic LINQ's read, and{" "} ToListAsync(Segment) combines its sets into one query and then counts and reads it exactly as a Filter does.

    diff --git a/OfficialWebsite/app/docs/extensions/to-list-async-dynamic-filter/page.tsx b/OfficialWebsite/app/docs/extensions/to-list-async-dynamic-filter/page.tsx index 0f19684..f8e595c 100644 --- a/OfficialWebsite/app/docs/extensions/to-list-async-dynamic-filter/page.tsx +++ b/OfficialWebsite/app/docs/extensions/to-list-async-dynamic-filter/page.tsx @@ -103,13 +103,13 @@ public static Task> ToListAsyncDynamic(
    • - + The read used to run Dynamic LINQ's{" "} - ToDynamicListAsync(), which reads synchronously on a - thread-pool thread. It now runs through EF Core's{" "} - ToListAsync(), so the database command runs - asynchronously and a canceled token reaches it. The rows are the same. - Like the count, the read needs an EF Core async provider. + ToDynamicListAsync(), asynchronous as well but with no token + to pass on. On an EF Core query it now runs through EF Core's{" "} + ToListAsync(), so a canceled token reaches the database. The + rows are the same. Only the count needs an EF Core async provider; on + any other provider the read falls back to Dynamic LINQ's. diff --git a/OfficialWebsite/app/docs/extensions/to-list-async-summary/page.tsx b/OfficialWebsite/app/docs/extensions/to-list-async-summary/page.tsx index 3272b41..3a309d3 100644 --- a/OfficialWebsite/app/docs/extensions/to-list-async-summary/page.tsx +++ b/OfficialWebsite/app/docs/extensions/to-list-async-summary/page.tsx @@ -107,17 +107,16 @@ public static Task ToListAsync( A source whose provider is not EF Core's — rows in memory through{" "} AsQueryable(), for one — keeps the reads it had in 3.1: a synchronous Count(), then Dynamic LINQ's{" "} - ToDynamicListAsync(), which reads synchronously on a - thread-pool thread. A token that is already canceled still stops it - before the count. + ToDynamicListAsync(), which reads on the calling thread. A + token that is already canceled still stops it before the count.

    - Until 3.2.0 the group count ran synchronously and only the read was - awaited, through ToDynamicListAsync(), on every provider. On - an EF Core query both now run through EF Core's asynchronous - operators, so a canceled token reaches the database. The count and the - rows are the same. + Until 3.2.0 the group count ran synchronously, and the read went through + Dynamic LINQ's ToDynamicListAsync(), which had no token + to pass on, on every provider. On an EF Core query both now run through + EF Core's asynchronous operators, so a canceled token reaches the + database. The count and the rows are the same. diff --git a/OfficialWebsite/app/docs/policies/attributes/page.tsx b/OfficialWebsite/app/docs/policies/attributes/page.tsx index 646f9b1..0d666d8 100644 --- a/OfficialWebsite/app/docs/policies/attributes/page.tsx +++ b/OfficialWebsite/app/docs/policies/attributes/page.tsx @@ -105,21 +105,27 @@ public class Ticket Select of the chain counts, because it makes the rows the default orders. Since 3.2.0 it hides nothing when it builds{" "} T itself in an object initializer and assigns every field - the default names, at every level of a nested path, with nothing EF - Core would compute on the client: "Owner.Name"{" "} - needs {`Owner = new OwnerRow { Name = … }`}. EF Core then - translates the order, because each field is a member the projection - assigned. + the default names a column, at every level of a nested path:{" "} + "Owner.Name" needs{" "} + {`Owner = new OwnerRow { Name = … }`}. On EF Core a column + is a member the model maps on the entity the Select reads, + read directly (t.Code), through reference navigations + (t.Owner.Name) or through EF.Property, a + shadow property included; in memory any assigned field is one. EF Core + then translates the order.
  • Any other projection leaves the query in its own order, as every projection did in 3.1.0: a constructor with arguments, a default field the initializer does not assign, a nested path through anything but an - initializer, or a default field computed by the application's own - method, such as {`Label = Decorate(r.Code)`}. EF Core - evaluates such a method on the client, where it can project the value - but cannot order by it. A default applied to any of these could name a - field EF Core cannot translate. + initializer, a member the model does not map, or a default field the + projection computes, by the application's own method{" "} + ({`Label = Decorate(r.Code)`}), a framework one such as{" "} + Regex.Replace or ToUpper, or an operator. EF + Core evaluates some of these on the client, where it can project the + value but cannot order by it, and which ones it translates depends on + the provider, so none is ordered by: a default must never be the reason + a query that ran unguarded fails.
  • A projection composed on the guarded handle takes no default, even when diff --git a/OfficialWebsite/app/docs/policies/configuration/page.tsx b/OfficialWebsite/app/docs/policies/configuration/page.tsx index 77496cf..6be4d91 100644 --- a/OfficialWebsite/app/docs/policies/configuration/page.tsx +++ b/OfficialWebsite/app/docs/policies/configuration/page.tsx @@ -76,11 +76,11 @@ export default function Page() { - + - +
    Selects names a navigationConvenienceStrict
    with nothing denied beneath itKept wholeKept whole
    with nothing denied beneath itKept whole, or narrowed around a transform that lands on a property with no setter (3.2.0)Kept whole, or narrowed the same way
    with a denied field beneath itReplaced by the allowed fields beneath itFieldDeniedForSelect
    whose key, Lines.Id, is denied (3.2.0)FieldDeniedForSelectFieldDeniedForSelect
    with a denied field beneath it, where the narrowing cannot be built (3.2.0)FieldDeniedForSelectFieldDeniedForSelect
    whose type holds a field denied for Select that no path names (3.2.0)Narrowed away, where it can be narrowedFieldDeniedForSelect
    that can carry a field denied for Select that no path names: past four segments, in a framework generic, on a subtype, or unasked under a "*" deny (3.2.0)Narrowed to the allowed fields where the core can narrow it; FieldDeniedForSelect where it cannotFieldDeniedForSelect
      @@ -110,16 +110,25 @@ export default function Page() { projection builds some way the core cannot narrow.
    • - A member that carries everything its type holds — a member of a - projected or in-memory row, or an entity's column, owned or - complex member — can hold a field denied for Select that - no path names: one deeper than four segments, or inside a framework - generic such as Dictionary<string, T>. The{" "} - Strict tier refuses such a member, and the{" "} - Convenience tier narrows it away (3.2.0); where it cannot - be narrowed, as a member of a row in memory cannot, both tiers refuse - it. An entity's navigation loads only its own entity, so only its - paths are asked about. + A named member can also carry a field denied for Select{" "} + that no path names (3.2.0): one deeper than four segments, one inside a + framework generic such as Dictionary<string, T>, or + one a subtype of the member's type declares — a derived entity, a + subclass, an interface's implementation. What it can carry is read + from the source. On an entity it is read from the EF Core model, so + only what loads counts: the navigation's columns, a converted one + included, its owned chain at any depth, and the navigations beneath it + an include, an automatic include or a lazy loader fills, for its type + and every type the model derives from it. On a projected or in-memory + row it is the member's type and every loaded subtype of it. Under a + policy with a "*" deny, a path the walk never asks + about — past four segments, around a cycle, with no setter, or on a + subtype — is a denied one as well. The Strict tier refuses + such a member. The Convenience tier narrows it where the + core can, which builds the declared type and so drops a subtype's + fields; a path naming a framework generic itself narrows to nothing and + is dropped. Where the core cannot narrow it — a column at the top of{" "} + T, a member of a row in memory — both tiers refuse it.
    • A navigation named through another, Main.Lead, gates the @@ -169,12 +178,25 @@ export default function Page() { the result. On an entity, that is beneath a column, an owned or complex member, or a navigation something loads: an{" "} Include or ThenInclude on the query, an - automatic include, or a lazy loader — EF Core's proxies, or an - injected ILazyLoader — which fills a navigation after the - query. An Include written in a form the library cannot - read counts as loading every navigation. On a row a projection builds, - it is beneath a member the initializer assigns. On a row in memory, it - is beneath any member. + automatic include, or a lazy loader — EF Core's proxies, an + injected ILazyLoader, or a loader delegate or{" "} + ILazyLoader the constructor takes and keeps in a field or + any property — which fills a navigation after the query. On a row a + projection builds, it is beneath a member the initializer assigns; a + constructor with arguments, with or without an initializer after it, + counts every member as assigned. On a row in memory, it is beneath any + member. A rule may spell the path in any letter case. +
      • +
    • + Every navigation counts as loaded where the library cannot read which + the query loads: an Include in a form it cannot read, one + off the query's own chain, and a chain that reaches its rows + through anything but the root's own rows —{" "} + {`Select(o => o.Customer)`}, a SelectMany, a{" "} + Join, or a projection behind another Select{" "} + (an identity Select, a member of an anonymous row, a + conditional). EF Core still applies includes named from the root to + the entities such a chain reaches.
    • A denial beneath a navigation nothing loads never leaves the database, @@ -182,12 +204,30 @@ export default function Page() { such navigations is read as it was in 3.1.0.
    • - A member whose type can hold a field denied for Select{" "} - that no path names — deeper than the walker's four segments, or + A member whose value can hold a field denied for Select{" "} + that no path names — deeper than the walker's four segments, inside a framework generic such as{" "} - Dictionary<string, T> — asks for one too, and so does - a member typed object, when what it holds can reach the - result. + Dictionary<string, T>, or declared by a subtype of + its type — asks for one too, read as for a named member, so on an + entity only what loads counts. So does a member that can hold an object + of any type: one typed object, a framework interface such + as IComparable, or a collection that is not generic + (IEnumerable, ArrayList,{" "} + Array). Under a "*" deny, so does a + member whose value can hold a path the walk never asks about. +
      • +
    • + A row can be a subtype of T. On an entity, a member a type + the model derives from T declares, and what loads beneath + it, counts as one of T's own would; on a row in memory, + a member any loaded subtype declares does. The projection builds{" "} + T and leaves them out, recorded with a reason starting{" "} + left out: a type derived. A row a projection builds is + exactly T. +
      • +
    • + A member EF Core does not map holds only what the class puts there, + never a value the query read, so it asks for nothing.
    • The denials beneath a member come from the providers' rules as @@ -216,7 +256,7 @@ export default function Page() { A projection that builds its rows before ApplyPolicy: the outermost Select constructs the row, in an object initializer or with a constructor, as in {`db.Roles.Select(r => new RoleRow { … })`} - Every member the initializer assigns; every member when a constructor builds the row + Every member the initializer assigns; every member when a constructor with arguments builds the row, with or without an initializer after it The same @@ -245,9 +285,10 @@ export default function Page() {
      • kept whole when nothing beneath it is denied, nothing - in its type is denied or typed object, no forced scope is - beneath it, and no transform beneath it lands on a property with no - setter; + its value can hold is denied (its subtypes included) or can hold an + object of any type, under a "*" deny the walk asks + about every path beneath it, no forced scope is beneath it, and no + transform beneath it lands on a property with no setter;
      • narrowed otherwise, to the allowed fields beneath it, @@ -255,9 +296,10 @@ export default function Page() { core's narrowing translates: an object the projection's initializer builds, a list a subquery reads into a type the core can bind (not an array or a set), a navigation that is neither complex nor - stored as JSON, or an entity's owned member not stored as JSON. A - field beneath it that can hold what the policy cannot name is left - out; + stored as JSON, or an entity's owned member not stored as JSON. + The narrowing builds the declared type, so a subtype's fields are + dropped. A field beneath it that can hold what the policy cannot name + is left out;
      • left out whole otherwise, and recorded as{" "} @@ -284,8 +326,9 @@ left out whole: it can hold what the policy cannot name`}
      • An entity's navigation, included or not: projecting it would load it. So once a denial needs a projection, an included or automatically - included navigation is not returned. Name it in Selects{" "} - to get it, narrowed. + included navigation is not returned. Under Convenience, + name it in Selects to get it narrowed; under{" "} + Strict, name its allowed fields.
      • An object held by a row in memory: a kept object is the caller's @@ -340,14 +383,27 @@ left out whole: it can hold what the policy cannot name`}
      - A member typed object is opaque to the policy: a + A member typed object, a framework interface or a + collection that is not generic, such as IEnumerable,{" "} + ArrayList or Array, is opaque to the policy: a synthesized projection leaves it out, and naming it returns whatever it holds. A framework generic holding a policed type, such as{" "} Dictionary<string, LineDto>, has no paths beneath it: - naming it is refused under Strict and narrowed away under{" "} - Convenience, or refused there too where the member cannot - be narrowed, and a synthesized projection leaves it out. - Hold such values in a list of the policed type instead. + naming it is refused in both tiers where the core cannot narrow it, at + the top of T or on a row in memory, narrowed away under{" "} + Convenience beneath a navigation, and a synthesized + projection leaves it out. Hold such values in a list of the policed type + instead. + + + A query over the root of a hierarchy whose derived type declares a + denied field comes back as root-type rows, the derived types' + allowed fields dropped too. Over an abstract root the typed terminals + fail with SelectTypeMustHaveParameterlessConstructor, and + the dynamic ones return the root's members. Query the derived type,{" "} + {`OfType()`}, to keep its fields. Under a{" "} + "*" deny, a member is kept whole only when the walk + asks about every path beneath it. A forced scope declared on a list's element type filters the rows diff --git a/OfficialWebsite/app/docs/policies/security/page.tsx b/OfficialWebsite/app/docs/policies/security/page.tsx index cc06fd8..f5c1023 100644 --- a/OfficialWebsite/app/docs/policies/security/page.tsx +++ b/OfficialWebsite/app/docs/policies/security/page.tsx @@ -256,13 +256,52 @@ true order. Add [DwNoOrder] unless that is intended.`} - Name a member whose type holds a denied field no path reaches: deeper than four segments, or inside a framework generic such as Dictionary<string, T> + Name a member that carries a denied field no path reaches: deeper than four segments, inside a framework generic such as Dictionary<string, T>, or, on an entity's navigation, in its owned chain or a converted column - Refused under Strict, and narrowed away under{" "} - Convenience, or refused there too where the member - cannot be narrowed. The gate also reads the rules themselves, - so a denied property with no setter and a rule on a path reached - through a cycle are found beneath a named member too. + Refused under Strict. Under Convenience it + is narrowed where the core can narrow it and refused where it + cannot. What the member carries is read from the source — from the + EF Core model for an entity, so only what loads counts. The gate + also reads the rules themselves, so a denied property with no + setter and a rule on a path reached through a cycle are found + beneath a named member too. + + + + Include a navigation from the root, then reach the rows through it — {`Select(o => o.Customer)`}, SelectMany, Join — or hide a projection behind another Select + + Every navigation counts as loaded on such a chain, since EF Core + still applies includes named from the root to the entities it + reaches, and the library cannot read which. The includes used to be + read against the wrong root, so the denied value beneath them was + returned. + + + + Let a lazy loader fill a navigation after the query: a loader delegate or ILazyLoader the constructor takes, kept in a field or a property of any name + + Counts as loading every navigation, as EF Core's proxies and an + injected ILazyLoader property already did. The model + keeps no record of such a loader, so the navigation it filled came + back with the denied value. + + + + Declare the denied field on a subtype — a derived entity, a subclass, an interface's implementation — and read it through the base type: a query over the hierarchy's root, or a member declared as the base type + + The subtypes are read too: the types the EF Core model derives for + an entity, every loaded subtype for a projected or in-memory row. + Such rows are projected to T and such members narrowed + to the declared type; a named one is refused under{" "} + Strict. The policy used to read the declared type only. + + + + Under a "*" deny with exact allows, reach a path the walk never asks about: past four segments, around a cycle, a property with no setter + + Such a path is denied, so a member holding one is narrowed, left out + or refused. It used to resolve as allowed, so the member was + returned whole, named or not. @@ -278,14 +317,18 @@ true order. Add [DwNoOrder] unless that is intended.`} - A member typed object is opaque to the policy: a + A member typed object, a framework interface or a + collection that is not generic, such as IEnumerable,{" "} + ArrayList or Array, is opaque to the policy: a synthesized projection leaves it out, and naming it returns whatever it holds. A framework generic holding a policed type, such as{" "} Dictionary<string, LineDto>, has no paths beneath it: - naming it is refused under Strict and narrowed away under{" "} - Convenience, or refused there too where the member cannot - be narrowed, and a synthesized projection leaves it out. - Hold such values in a list of the policed type instead. + naming it is refused in both tiers where the core cannot narrow it, + narrowed away under Convenience beneath a navigation, and a + synthesized projection leaves it out. Hold such values in a list of the + policed type instead. A member EF Core does not map holds what the class + computes, and the policy reads nothing into it: a getter that copies a + denied column is the application's to withhold. A forced scope declared on a list's element type filters the rows diff --git a/OfficialWebsite/public/llms.txt b/OfficialWebsite/public/llms.txt index 9c8fc14..f869b1f 100644 --- a/OfficialWebsite/public/llms.txt +++ b/OfficialWebsite/public/llms.txt @@ -804,10 +804,11 @@ ToListDynamic, ToListAsyncDynamic Where -> COUNT(where-only query) -> build Or - Every call runs two queries: a count of the filtered set (ignoring Page) and the data query. - In the dynamic pair an invalid `Orders`, `Page` or `Selects` throws after the count has already run. - `ToListAsync` uses EF Core `CountAsync` / `ToListAsync`. `ToListAsyncDynamic` uses EF Core `CountAsync`, then - EF Core's `ToListAsync` over the projection's generated type, so the read runs asynchronously (3.2.0); it used to - run Dynamic LINQ's `ToDynamicListAsync`, a synchronous read on a thread-pool thread. Both need an EF Core async - provider: on a plain `list.AsQueryable()` they throw `InvalidOperationException` ("The provider for the source - 'IQueryable' doesn't implement 'IAsyncQueryProvider'…"). Use the synchronous methods in memory. + EF Core's `ToListAsync` over the query's element type: `T` when `Selects` is null, the projection's generated + class otherwise (3.2.0). It used to read through Dynamic LINQ's `ToDynamicListAsync`, asynchronous as well but + with no token to pass on. Both need an EF Core async provider for the count: on a plain + `list.AsQueryable()` they throw `InvalidOperationException` ("The provider for the source 'IQueryable' doesn't + implement 'IAsyncQueryProvider'…"). Use the synchronous methods in memory. - The overloads taking a `CancellationToken` (3.2.0) pass it to the count and to the read, so a canceled token stops whichever is running and the call throws `OperationCanceledException` (EF Core's `TaskCanceledException` derives from it). The overloads without a token pass `CancellationToken.None`. @@ -1838,11 +1839,14 @@ public class Ticket { … } - to a source whose projection could hide a default field. Only the outermost `Select` of the chain counts, because it makes the rows the default orders. Since 3.2.0 it hides nothing when it builds T itself in an object initializer, `Select(t => new Row { Code = t.Code, … })`, and assigns every field the default names, at every - level of a nested path (`"Owner.Name"` needs `Owner = new OwnerRow { Name = … }`); the default then applies, - because EF Core translates an order through a member the projection assigned. A constructor with arguments, a - default field the initializer does not assign, or a nested path through anything but an initializer leaves the - query in its own order, as every projection did in 3.1.0: a default applied there could name a field the - projection left out, which EF Core cannot translate; + level of a nested path (`"Owner.Name"` needs `Owner = new OwnerRow { Name = … }`), a column. On EF Core a column + is a member the model maps on the entity the `Select` reads, read directly, through reference navigations + (`t.Owner.Name`) or through `EF.Property` (a shadow property included); the default then applies, because EF + Core translates an order by a column the projection assigned. In memory any assigned field is ordered by. A + value the projection computes, by a method (`Regex.Replace`, `ToUpper`, the application's own) or an operator + (`t.First + " " + t.Last`), a member the model does not map, a constructor with arguments, a default field the + initializer does not assign, or a nested path through anything but an initializer leaves the query in its own + order, as every projection did in 3.1.0: ordering by it could fail where the unguarded query ran; - after a projection composed on the handle: the guarded `Select`, or a guarded `Filter` whose `Selects` is set, leaves the rest of the chain unordered even when it keeps every default field, so `guarded.Select(["Id", "Title"]).Page(page)` pages as it did in 3.0.0, unordered. The default is for the rows the @@ -2282,6 +2286,11 @@ SELECT a navigation that cannot be narrowed around a denied field: its key a.Id is denied, or a path through it is one the core cannot project (3.2.0) throw throw FieldDeniedForSelect SELECT a.b when the key a.Id is denied throw throw FieldDeniedForSelect +SELECT a member that can carry a denied field no path names: + past four segments, in a framework generic, on a subtype, + or unasked under a "*" deny (3.2.0) allowed leaves, throw FieldDeniedForSelect + or throw where it + cannot be narrowed every requested SELECT dropped throw - AllSelectsDenied no Selects while a denied field can reach the result (3.2.0) allowed members allowed members GROUP BY a denied field throw throw FieldDeniedForGroup @@ -2307,16 +2316,26 @@ DefaultOrder field denied for Segment, in a Segment (3.1.0) left out l - "drop" removes the entry and records `Dropped`. A Strict throw records `Denied`. - The guarded composable `Order(...)` and `Select(...)` drop and throw the same way. - "allowed leaves": when `Selects` names a navigation with a denied field beneath it, it is replaced by the allowed - leaf paths beneath it. A navigation with nothing denied beneath it is kept as written. + leaf paths beneath it. A navigation with nothing denied beneath it is kept as written, unless a transform beneath it + lands on a property with no setter, which the outbound walk could not write back: it is then narrowed around that + property (3.2.0). - The fields beneath are read the way the attribute walker reads them (3.2.0): through any collection type, so a member typed `IReadOnlyList` or an application's own collection no longer hides its denied fields, and no deeper than the walker's four segments. The providers' fragments are asked too, so a denied property with no setter and a rule on a path reached through a cycle count (3.2.0). - - A named member that can carry anything its type holds, a member of a projected or in-memory row or an entity's - column, owned or complex member, and whose type holds a field denied for Select that no path names (deeper than - four segments, or inside a framework generic such as `Dictionary`): Strict refuses it with - `FieldDeniedForSelect`, Convenience narrows it away (3.2.0). An entity's navigation loads only its own entity, so - only the paths are asked about there. + - A named member can also carry a field denied for Select that no path names (3.2.0): deeper than four segments, + inside a framework generic such as `Dictionary`, or declared by a subtype of the member's type (a + derived entity, a subclass, an interface's implementation). What it can carry is read from the source. On an + entity it is read from the EF Core model, so only what loads counts: the navigation's columns, a converted one + included, its owned chain at any depth, and the navigations beneath it that an include, an automatic include or + a lazy loader fills, for its type and every type the model derives from it. On a projected or in-memory row it + is the member's type and every loaded subtype of it. Under a policy with a `"*"` deny, a path the walk never + asks about (past four segments, around a cycle, with no setter, or on a subtype) is a denied one as well. + - Such a member is refused with `FieldDeniedForSelect` under Strict. Under Convenience it is narrowed to the + allowed leaves where the core can narrow the path's first member, which builds the declared type and so drops a + subtype's fields; a path that names a framework generic itself narrows to nothing and is dropped. Where the core + cannot narrow it (a column, complex or JSON member at the top of T, or a member of a row in memory) it is + refused in both tiers. - A narrowing that cannot be built as gated is refused with `FieldDeniedForSelect`, in both tiers (3.2.0). The core's typed projection adds the key (`Id`) of every nested node it builds, so a narrowing whose nodes carry a denied key would return it; a path through a collection the core does not unwrap fails its validation; and a @@ -2329,17 +2348,32 @@ DefaultOrder field denied for Segment, in a Segment (3.1.0) left out l single-clause composable call. - A denial at the top of T always counts, whatever the member holds: a scalar, a blob, a list, an owned object, a JSON column (3.2.0; 3.1.0 asked only about simple members, so a denied `byte[]` or owned member came back). - - A denial beneath a member counts when its value can reach the result: - - on an entity, beneath a column, an owned or complex member, or a navigation something loads: an `Include` or - `ThenInclude` on the query (a form the library cannot read, or an include off the query's own chain such as on - a join's inner source, counts as loading every navigation), an automatic include, or a lazy loader (proxies, - or an injected `ILazyLoader`, whose navigations fill after the query). A - denial beneath a navigation nothing loads never leaves the database and needs no projection, so a connected - model is read as it was in 3.1.0; - - on a row a projection builds, beneath a member its initializer assigns; - - on a row in memory, beneath any member. - - So does a member whose type can hold a field denied for Select that no path names, deeper than the walker's four - segments or inside a framework generic such as `Dictionary`, and a member typed `object`. + - A denial beneath a member counts when its value can reach the result. A rule may spell its path in any letter + case. + - On an entity: beneath a column, an owned or complex member, or a navigation something loads. That is an + `Include` or `ThenInclude` on the query, an automatic include, or a lazy loader: proxies, an injected + `ILazyLoader`, or a loader delegate or `ILazyLoader` the constructor takes, kept in a field or any property, + whose navigations fill after the query. Every navigation counts as loaded when the library cannot read which + the query loads: an include in a form it cannot read, an include off the query's own chain (on a join's inner + source, say), and a chain that reaches its rows through anything but the root's own rows, such as + `Select(o => o.Customer)`, a `SelectMany`, a `Join`, or a projection behind another `Select` (an identity + `Select`, a member of an anonymous row, a conditional), since EF Core applies includes named from the root to + the entities it reaches. A denial beneath a navigation nothing loads never leaves the database and needs no + projection, so a connected model is read as it was in 3.1.0. A member EF Core does not map holds only what the + class puts there, never a value the query read, and asks for nothing. + - On a row a projection builds: beneath a member its initializer assigns. A constructor with arguments, with or + without an initializer after it, counts every member as assigned. + - On a row in memory: beneath any member. + - So does a member whose value can hold a field denied for Select that no path names (deeper than the walker's + four segments, inside a framework generic such as `Dictionary`, or declared by a subtype of its + type), read as for a named member above, so on an entity only what loads counts. So does a member that can hold + an object of any type: one typed `object`, a framework interface such as `IComparable`, or a collection that is + not generic (`IEnumerable`, `ArrayList`, `Hashtable`, `Array`). Under a policy with a `"*"` deny, so does a + member whose value can hold a path the walk never asks about. + - A row can be a subtype of T. On an entity, each member a type the model derives from T declares, and what loads + beneath it, counts as one of T's own would; on a row in memory, each member any loaded subtype declares. The + projection builds T, so it leaves them all out, recorded as `Dropped` with a reason starting `left out: a type + derived`. A row a projection builds is exactly T. - The denials beneath a member come from the providers' fragments as well as from walking the type, so a denied property with no setter, a rule on a path reached through a cycle, and a rule deeper than the walk all count. - A forced scope beneath a member asks for no projection on its own: it filters the rows that hold the member, as it @@ -2352,26 +2386,33 @@ DefaultOrder field denied for Segment, in a Segment (3.1.0) left out l read the whole entity, the denied columns included, and it holds only its initial value anyway; - every allowed member holding an object or a list of them that the source carries: a member a projection's initializer assigns, and an entity's columns (converted or JSON), owned and complex members. It is kept whole - when nothing beneath it is denied, nothing in its type is denied or typed `object`, no forced scope is beneath - it and no transform beneath it lands on a property with no setter; + when nothing beneath it is denied, nothing its value can hold is denied or can hold an object of any type, + under a `"*"` deny the walk asks about every path beneath it, no forced scope is beneath it, and no transform + beneath it lands on a property with no setter; - otherwise it is narrowed to the allowed leaves beneath it, to four segments, as a caller naming it would get, when the core's narrowing translates: an object the projection's initializer builds, a list a subquery reads into a type the core can bind (not an array or a set), a navigation that is neither complex nor stored as JSON, - or an entity's owned member not stored as JSON. A leaf that can hold what the policy cannot name is left out; + or an entity's owned member not stored as JSON. The narrowing builds the member's declared type, so a + subtype's fields are dropped. A leaf that can hold what the policy cannot name is left out; - otherwise it is left out whole, recorded as `Dropped` on `Select` with a reason starting `left out whole`: a scope forced beneath it; a column, complex property or JSON-stored member, which EF Core reads whole; one the - projection builds some other way, by a constructor, a conditional or an unassigned member; a denied key the - core's projection would add back; a path the core cannot project; nothing beneath it left to select; - - Never kept: an entity's navigation, included or not, since projecting it would load it (name it in `Selects` to - get it, narrowed); an object held by a row in memory, since a kept object is the caller's own and a transform - would change it in place; a member with no setter; a member named with one of the parser's words. - - A `Select` handing back an entity, `Select(o => o.Customer)`, builds no row and is read as an entity query. + projection builds some other way, by a constructor with arguments, a conditional or an unassigned member; a + denied key the core's projection would add back; a path the core cannot project; nothing beneath it left to + select; + - Never kept: an entity's navigation, included or not, since projecting it would load it (under Convenience, name + it in `Selects` to get it narrowed; under Strict, name its allowed fields); an object held by a row in memory, + since a kept object is the caller's own and a transform would change it in place; a member with no setter; a + member named with one of the parser's words. + - A `Select` handing back an entity, `Select(o => o.Customer)`, builds no row and is read as an entity query, with + every navigation counted as loaded (above). - A narrowed reference that is null in the source comes back as an empty object, as it does for a caller's own dotted `Selects`. - A narrowed member carries every allowed leaf beneath it. An entity reached beneath it has its own navigations projected and so loaded, which the source may not have included, exactly as `Selects` naming the member does. - Each denied field whose value can reach the result, at the top or beneath, is recorded as `Dropped` on `Select`. - - It never throws for the denied field. It throws `AllSelectsDenied` only when no field is left. + - It never throws for the denied field. It throws `AllSelectsDenied` only when no field is left. A typed terminal + projects into T, so a T with no public parameterless constructor, an abstract one among them, fails there with + `SelectTypeMustHaveParameterlessConstructor`; the dynamic terminals build their own class. - With nothing counted, `Selects` stays null and the query is the one an unguarded call runs. - A path that matches nothing on `T` (3.1.0): - Convenience, and dry run in either tier: validation throws `LogicException` @@ -2797,8 +2838,13 @@ What is recorded Dropped a field removed from a Convenience request (Select, Order); one per field Dropped a DefaultOrder field left out for this caller (Order), both tiers, and in a Segment a field denied for Segment too; Reason starts "left out of the default order" (3.1.0) - Dropped a field a synthesized projection leaves out (Select), at the top or beneath a member, and a - member it leaves out whole because it cannot be narrowed; Reason starts "left out whole" (3.2.0) + Dropped a denied field a synthesized projection leaves out (Select), at the top or beneath a member; + Reason names the policy's sources, such as "DwDeniedAttribute (sealed)" (3.2.0) + Dropped a member a synthesized projection leaves out whole (Select); Reason starts "left out whole" (3.2.0) + Dropped a member a type derived from T declares, which a synthesized projection building T leaves out + (Select); Reason starts "left out: a type derived" (3.2.0) + Dropped a member a Convenience caller named that can hold a denied field no path names (Select); Reason + "it holds a field denied for Select where no path can name it" (3.2.0) Denied a refusal: Strict denial, cap, cost, query string ("*"), required filter, segment inference, MaxAuditEvents; the throw follows unless dry run. Under Strict a name that matches nothing is Denied under that name, Reason "names nothing on " (3.1.0) @@ -4240,7 +4286,8 @@ Read before generating policy attributes. pages, and an `IQueryable` ordered before `ApplyPolicy`, or by a composed `Order` before `Page`, keeps its own order; a list sorted in memory before `ApplyPolicy` is not seen as ordered and gets the default, so send the order with the filter. A projected query takes the default only when its outermost `Select` builds T in an - object initializer assigning every default field (3.2.0); the guarded `Select` composed afterwards never does. + object initializer assigning every default field a column (3.2.0); a computed value, even one EF Core could + translate, leaves it unordered, and the guarded `Select` composed afterwards never takes it. A default field the caller may not order by is left out without an error. End every order meant for paging with a unique field, such as the key. 38. **A `[DwEntity]` on a derived type replaces its base type's.** The attribute allows one per type, and .NET @@ -4690,8 +4737,9 @@ MemoryCalculationInput ``` 3.2.0 A projected row keeps its members, denials the gate could not see are enforced, a default order that - reaches projected rows, and a CancellationToken on every async terminal. The bullets marked "Behaviour - change" change what code written for 3.1.0 does. + reaches projected rows, and a CancellationToken on every async terminal. The security fixes refuse or + withhold what 3.1.0 returned; the bullets marked "Behaviour change" also change what a correct query + returns; and one call form stops compiling (the token bullet). - Security fix and behaviour change. With no Selects, a field denied for Select only beneath a member, none at the top of T, synthesized no projection, so the whole row came back with the denied value in it: in a list or nested object of a row projected before ApplyPolicy, in a row in memory, and in an entity's included, @@ -4699,8 +4747,25 @@ MemoryCalculationInput value can reach the result, in both tiers, typed and dynamic, for a Filter and a Segment. On an entity that means beneath a column, an owned or complex member, or a navigation the query loads; a denial beneath a navigation nothing loads never leaves the database, and the entity is read as in 3.1.0. - - Security fix. A field denied at the top of T whose type is not a simple value, a blob, a list, an owned - object or a JSON column, synthesized no projection either, so with nothing else denied it came back. + - Security fix. Where the query hides what loads, every navigation now counts as loaded: an include named from + the root and re-rooted by Select(o => o.Customer), SelectMany or Join, which EF Core still applies, and a + projection behind another Select (an identity Select, a member of an anonymous row, a conditional). So does + a lazy loader the constructor takes, delegate or ILazyLoader, kept in a field or a property of any name, and + an initializer after a constructor with arguments counts every member as assigned. Each returned the denied + value. + - Security fix and behaviour change. A member declared as a base type or an interface holds its subtypes, + whose denied fields the declared type never names. They are read now: the types the EF Core model derives, + for an entity, and every loaded subtype for a projected or in-memory row. A query over the root of a + hierarchy whose derived type declares a denied field, an included or named base-typed navigation, and a + base-typed member of a row, all returned it. Such rows are projected to T and such members narrowed to the + declared type, which drops the subtype's fields, allowed ones too. Over an abstract T the typed terminals + then fail with SelectTypeMustHaveParameterlessConstructor, as for any T they cannot build; the dynamic + terminals return the root's allowed members. + - Security fix. Under a "*" deny with exact allows, a path the walk never asked about (past four segments, + around a cycle, with no setter) resolved as allowed, so a member holding one was returned whole, named or + not. Such a member is refused, narrowed or projected as one with a denial beneath it is. + - Security fix. A field denied at the top of T whose type is not a simple value (a blob, a list, an owned + object or a JSON column, say) synthesized no projection either, so with nothing else denied it came back. - Security fix. The attribute walker read any namespace starting with "System" as the framework's, so an application's SystemsCorp.Payroll got no fragment beneath its types, and a [DwDenied] field there was returned, filterable and sortable. Only System and the namespaces beneath it are the framework's now. @@ -4712,30 +4777,34 @@ MemoryCalculationInput IReadOnlyCollection, Collection or an application's own) returned every field beneath it, denied ones included, in both tiers: the projection gate read collections through a narrower list than the attribute walker. It reads them as the walker does, and a narrowing the core cannot project is refused. - - Security fix. Selects naming a member of a projected or in-memory row whose type holds a field denied for - Select that no path names, deeper than four segments or inside a framework generic such as Dictionary, returned it. Strict refuses it; Convenience narrows it away. A denied property with no setter, and a rule on - a path reached through a cycle, are found beneath a named member too. + - Security fix. Selects naming a member that carries a field denied for Select no path names returned it: + deeper than four segments, inside a framework generic such as Dictionary, or, on a named entity + navigation, in its owned chain or a converted column. What a member carries is read from the source, from + the EF Core model for an entity. Strict refuses it; Convenience narrows it where the core can, and refuses it + where it cannot. A denied property with no setter, and a rule on a path reached through a cycle, are found + beneath a named member too. - Behaviour change. The synthesized projection keeps what the source carries. A row a projection builds keeps its assigned nested objects and lists; an entity keeps its columns, converted and JSON ones included, and its owned and complex members, and every member holding a collection of simple values (byte[], List). In 3.1.0 all of these came back null or empty whenever a field was denied. A member is kept whole when nothing it can hold is denied, narrowed around a denial where the core's narrowing translates, and otherwise left out whole with a "left out whole" Dropped decision. An entity's navigations and the objects of a row in memory - are left out, as before; a value EF Core does not map is left out too. + are left out, as before; a value EF Core does not map is left out too, and asks for nothing. - Behaviour change. [DwEntity(DefaultOrder)] applies to a projected source whose outermost Select builds T - in an object initializer assigning every field the default names, with nothing EF Core would compute on the - client; any other projection still leaves the query in its own order. A Select, or a Filter with Selects, - composed on the guarded handle keeps the rest of the chain unordered. A composed Filter that sent orders - gets no default later in the chain, as a composed Order already did not. + in an object initializer assigning every field the default names a column: a mapped member, read directly, + through reference navigations or through EF.Property. A computed value or any other projection still leaves + the query in its own order. A Select, or a Filter with Selects, composed on the guarded handle keeps the rest + of the chain unordered. A composed Filter that sent orders gets no default later in the chain, as a composed + Order already did not. - Every async terminal has overloads taking a CancellationToken, guarded and unguarded: ToListAsync and ToListAsyncDynamic with a Filter, ToListAsync with a Summary, and ToListAsync with a Segment. The 3.1 signatures are unchanged, so code compiled against 3.1 still binds. The token reaches the count and the - read. ToListAsync(filter, default) no longer compiles, and a reflection lookup of one of these methods by - name alone now finds several. - - Behaviour change. ToListAsyncDynamic and the async Summary read through EF Core's ToListAsync, and the async - Summary counts through CountAsync, instead of reading synchronously; on an EF Core query a canceled token - now reaches the database. A provider that is not EF Core's keeps the synchronous read. + read. ToListAsync(filter, default) no longer compiles, since default fits both bool and CancellationToken, + and a reflection lookup of one of these methods by name alone finds more overloads than it did. + - Behaviour change. The async Summary counts through EF Core's CountAsync, where it counted synchronously, and + it and ToListAsyncDynamic read through EF Core's ToListAsync instead of Dynamic LINQ's ToDynamicListAsync, + so on an EF Core query a canceled token reaches the database. A provider that is not EF Core's keeps Dynamic + LINQ's read, on the calling thread. 3.1.0 Dates rebuilt, segments combined in the database, five new caps, two new error codes, preparation enforced, a strict tier that discloses less, declared default orders, forced predicates that admit null, audited refusals, and long In lists that no longer end the process. The eleven bullets marked "Behaviour change" @@ -4890,14 +4959,25 @@ MemoryCalculationInput before EF Core translates it. `Contains` / `StartsWith` / `EndsWith` work only on string members. - Cache configuration changes are eventually consistent: calls already running finish with the options they read. - With no `Selects`, a guarded query over an entity leaves out every navigation EF Core does not own once a denial - needs a projection (section 17), an included one too. Name the navigation in `Selects` to get it, narrowed. + needs a projection (section 17), an included one too. Under Convenience, name the navigation in `Selects` to get + it narrowed; under Strict, name its allowed fields. - A forced scope declared on a list's element type filters the rows that hold the list, never its elements. Selects naming the list returns every element, those the scope excludes included, as in every release; a synthesized projection leaves such a list out. Scope the elements where the row is built. -- A member typed `object` is opaque to the policy: a synthesized projection leaves it out, and naming it returns - whatever it holds. A framework generic holding a policed type (`Dictionary`) has no paths - beneath it: naming it is refused under Strict and narrowed away under Convenience, and a synthesized projection - leaves it out. +- A member typed `object`, a framework interface or a collection that is not generic (`IEnumerable`, `ArrayList`, + `Array`) is opaque to the policy: a synthesized projection leaves it out, and naming it returns whatever it holds. + A framework generic holding a policed type (`Dictionary`) has no paths beneath it: naming it is + refused in both tiers where the core cannot narrow it (at the top of T, or on a row in memory), narrowed away + under Convenience beneath a navigation, and a synthesized projection leaves it out. +- A projection builds the declared type. A query over the root of a hierarchy whose derived type declares a denied + field comes back as root-type rows, the derived types' allowed fields dropped too; over an abstract root the typed + terminals fail with `SelectTypeMustHaveParameterlessConstructor` and the dynamic ones return the root's members. + Query the derived type (`OfType()`) to keep its fields. Subtypes are read from the assemblies loaded + when the query runs, which hold every type a row can have. +- Under a `"*"` deny, a member is kept whole only when the walk asks about every path beneath it: a type with a + property that has no setter, a cycle, a subtype or anything past four segments is narrowed, left out or refused. +- An entity member EF Core does not map holds what the class computes, and the policy reads nothing into it: a + getter that copies a denied column is the application's to withhold. - A simulation reads T as a source it cannot see into (section 23): every denial beneath a member counts, and a synthesized `Clause.Selects` keeps only members holding a value. diff --git a/README.md b/README.md index 0ffa896..1315965 100644 --- a/README.md +++ b/README.md @@ -380,15 +380,16 @@ The complete reference — every enum, class, extension method, validation rule, **Upgrade note — the security fixes and the three changes alter what code written for 3.1.0 does, and the new overloads can stop a call from compiling. Read these before bumping.** - **Fixed (security): a field denied beneath a member reached a caller who sent no `Selects`.** A guarded query synthesizes a projection for a denied field, and it did so only when a simple field at the top of the type was denied. With every denial beneath a member, the whole row came back with the denied value in it: in a list or nested object of a row projected before `ApplyPolicy`, in a row held in memory, and in an entity's included, automatically included, lazily loaded or owned member — typed and dynamic, in both tiers, for a `Filter` and a `Segment`. Such a denial now synthesizes the projection whenever its value can reach the result. On an entity that means beneath a column, an owned or complex member, or a navigation the query loads through `Include`, an automatic include or a lazy loader. A denial beneath a navigation nothing loads never leaves the database, and the entity is read exactly as before. +- **Fixed (security): what a query loads, and what a member holds, was read too narrowly.** An include named from the root and reached through `Select(o => o.Customer)`, `SelectMany` or `Join`, a projection behind another `Select`, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or any property each loaded a denied value the gate read as unloaded. A field a subtype declares — a derived entity's, a subclass's held by a base-typed member — was not read at all, and under a `"*"` deny with exact allows a path the walk never asked about (past four segments, around a cycle, with no setter) resolved as allowed. `Selects` naming an entity navigation returned a denial in its owned chain past four segments or in a converted `Dictionary` column. All of these are read now: from the EF Core model for an entity, so only what loads counts, and from every loaded subtype for a projected or in-memory row. - **Fixed (security): a denied member that holds no simple value came back.** A field denied at the top of the type whose own type is not a simple value — a byte array, a list, an owned object, a JSON column — synthesized no projection either, so with nothing else denied it came back. - **Fixed (security): an application namespace starting with `System` got no policy.** The attribute walker read any namespace starting with "System" as the framework's, so an application namespace such as `SystemsCorp.Payroll` got no policy beneath its types, and a `[DwDenied]` field there was returned, filterable and sortable. Only `System` and the namespaces beneath it are the framework's now. - **Fixed (security): a navigation narrowed around its own denied key got the key back.** Under the convenience tier, `Selects` naming a navigation whose key (`Id`) is denied was narrowed to the allowed fields beneath it, and the core's typed projection added the key back. Such a narrowing is refused with `FieldDeniedForSelect` in both tiers, as naming a sibling of the key already was. A navigation named through another, such as `Main.Lead`, now gates the key of `Main`, which the projection adds; it did not. -- **Fixed (security): `Selects` could name a member whose denials the gate did not see.** A member typed as a collection the core does not unwrap — `IReadOnlyList`, `IReadOnlyCollection`, `Collection` or an application's own — returned every field beneath it, denied ones included, in both tiers, because the projection gate read collections through a narrower list than the attribute walker. It reads them the same way now, and a narrowing the core cannot project is refused with `FieldDeniedForSelect`. Denials beneath a named member are also read from the policy's own rules, so a denied property with no setter and a rule on a path reached through a cycle are found. A field denied deeper than the walker reaches, or inside a framework collection such as `Dictionary`, is refused under the strict tier and narrowed away under the convenience tier, or refused there too where the member cannot be narrowed. -- **Changed: the synthesized projection keeps what the source carries.** It kept simple fields only, so every nested object and list of a row projected before `ApplyPolicy` came back null or empty as soon as any field was denied. A row a projection builds — the outermost `Select` constructs it, in an object initializer or with a constructor, as in `db.Roles.Select(r => new RoleRow { … })` — keeps the members its initializer assigns. An entity, or a `Select` that hands back an entity such as `db.Orders.Select(o => o.Customer)`, keeps its mapped columns, converted and JSON ones included, its owned and complex members, and every collection of simple values such as `byte[]` or `List`. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, recorded as `Dropped` with a reason starting `left out whole`. An entity's navigations, the objects of a row in memory, and a value EF Core does not map are left out, and the type needs a public parameterless constructor for the projection, as it already did. -- **Changed: `[DwEntity(DefaultOrder)]` reaches a projection that builds the row.** A guarded query over a projected source takes the default when the outermost `Select` builds the type in an object initializer and assigns every field the default names, at every level of a nested path, with nothing EF Core would compute on the client: `Select(t => new TicketRow { Id = t.Id, CreatedAt = t.CreatedAt })` for `"CreatedAt desc, Id"`. Any other projection still leaves the query in its own order. A `Select`, or a `Filter` with `Selects`, composed on the guarded handle keeps the rest of the chain unordered, and a composed `Filter` that sent orders gets no default later in the chain, as a composed `Order` already did not. -- **Changed: the async dynamic `Filter` and the async `Summary` read through EF Core.** `ToListAsyncDynamic` and `ToListAsync(Summary)` read with EF Core's `ToListAsync`, and the summary counts with `CountAsync`. They used to read synchronously on a thread-pool thread, so on an EF Core query a canceled token now reaches the database. A provider that is not EF Core's keeps the synchronous read. -- **New: a `CancellationToken` on every async terminal**, guarded and unguarded: `ToListAsync` and `ToListAsyncDynamic` with a `Filter`, `ToListAsync` with a `Summary`, and `ToListAsync` with a `Segment`. The token reaches the count and the read. The overloads sit beside the 3.1 signatures, which are unchanged, so code compiled against 3.1 still binds. `ToListAsync(filter, default)`, `ToListAsyncDynamic(filter, default)` and `ToListAsync(summary, default)` no longer compile, because `default` fits both `getQueryString` and the token: write `false`, a token, or a named argument. A reflection lookup of `ToListAsyncDynamic` by name alone now finds three methods where it found one. -- **Known limits.** Once a projection is needed, an entity's navigations are left out, included ones too; name one in `Selects` to get it, narrowed. A forced scope declared on a list's element type filters the rows that hold the list, never its elements, so `Selects` naming the list returns every element and a synthesized projection leaves the list out; scope the elements where the row is built. A member typed `object` is opaque to the policy, and a framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: hold such values in a list of the policed type. `/simulate` has no source, so it reads the type as one it cannot see into: every denial beneath a member counts, and the projection it shows keeps only members holding a value. +- **Fixed (security): `Selects` could name a member whose denials the gate did not see.** A member typed as a collection the core does not unwrap — `IReadOnlyList`, `IReadOnlyCollection`, `Collection` or an application's own — returned every field beneath it, denied ones included, in both tiers, because the projection gate read collections through a narrower list than the attribute walker. It reads them the same way now, and a narrowing the core cannot project is refused with `FieldDeniedForSelect`. Denials beneath a named member are also read from the policy's own rules, so a denied property with no setter and a rule on a path reached through a cycle are found. A member carrying a field denied where no path reaches it — deeper than the walker, inside a framework collection such as `Dictionary`, or on a subtype — is refused under the strict tier, and under the convenience tier narrowed where the core can narrow it and refused where it cannot. +- **Changed: the synthesized projection keeps what the source carries.** It kept simple fields only, so every nested object and list of a row projected before `ApplyPolicy` came back null or empty as soon as any field was denied. A row a projection builds — the outermost `Select` constructs it, in an object initializer or with a constructor, as in `db.Roles.Select(r => new RoleRow { … })` — keeps the members its initializer assigns. An entity, or a `Select` that hands back an entity such as `db.Orders.Select(o => o.Customer)`, keeps its mapped columns, converted and JSON ones included, its owned and complex members, and every collection of simple values such as `byte[]` or `List`. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, recorded as `Dropped` with a reason starting `left out whole`. An entity's navigations, the objects of a row in memory, and a value EF Core does not map are left out, and the type needs a public parameterless constructor for the typed projection, as it already did. When a derived type in the model, or a loaded subclass of a row in memory, declares a denied field, the rows come back as the queried type, so a derived type's allowed fields are dropped too; query the derived type with `OfType()` to keep them. +- **Changed: `[DwEntity(DefaultOrder)]` reaches a projection that builds the row.** A guarded query over a projected source takes the default when the outermost `Select` builds the type in an object initializer and assigns every field the default names a column, at every level of a nested path — a mapped member read directly, through reference navigations or through `EF.Property`: `Select(t => new TicketRow { Id = t.Id, CreatedAt = t.CreatedAt })` for `"CreatedAt desc, Id"`. A computed value or any other projection still leaves the query in its own order. A `Select`, or a `Filter` with `Selects`, composed on the guarded handle keeps the rest of the chain unordered, and a composed `Filter` that sent orders gets no default later in the chain, as a composed `Order` already did not. +- **Changed: the async dynamic `Filter` and the async `Summary` read through EF Core.** `ToListAsyncDynamic` and `ToListAsync(Summary)` read with EF Core's `ToListAsync` instead of Dynamic LINQ's `ToDynamicListAsync`, which had no token to pass on, and the summary counts with `CountAsync` where it counted synchronously. So on an EF Core query a canceled token now reaches the database. A provider that is not EF Core's keeps Dynamic LINQ's read, on the calling thread. +- **New: a `CancellationToken` on every async terminal**, guarded and unguarded: `ToListAsync` and `ToListAsyncDynamic` with a `Filter`, `ToListAsync` with a `Summary`, and `ToListAsync` with a `Segment`. The token reaches the count and the read. The overloads sit beside the 3.1 signatures, which are unchanged, so code compiled against 3.1 still binds. `ToListAsync(filter, default)`, `ToListAsyncDynamic(filter, default)` and `ToListAsync(summary, default)` no longer compile, because `default` fits both `getQueryString` and the token: write `false`, a token, or a named argument. A reflection lookup of `ToListAsyncDynamic` by name alone now finds three methods where it found one, and one of `ToListAsync` finds more than it did. +- **Known limits.** Once a projection is needed, an entity's navigations are left out, included ones too; under the convenience tier name one in `Selects` to get it narrowed, and under the strict tier name its allowed fields. A forced scope declared on a list's element type filters the rows that hold the list, never its elements, so `Selects` naming the list returns every element and a synthesized projection leaves the list out; scope the elements where the row is built. A member typed `object`, a framework interface or a collection that is not generic (`IEnumerable`, `ArrayList`) is opaque to the policy, and a framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: hold such values in a list of the policed type. A member EF Core does not map holds what the class computes, and the policy reads nothing into it. `/simulate` has no source, so it reads the type as one it cannot see into: every denial beneath a member counts, and the projection it shows keeps only members holding a value. ## Version 3.1.0 highlights diff --git a/build/check-version.ps1 b/build/check-version.ps1 index 311f68f..7d5e900 100644 --- a/build/check-version.ps1 +++ b/build/check-version.ps1 @@ -103,10 +103,10 @@ $targets = [ordered]@{ "Version ($semver) . targets net6\.0", "DynamicWhere\.ex --version ($semver)" ) - # The page that serves that reference says which version it was generated against, in prose + # The page that serves that reference says which version it was written against, in prose # none of the patterns above reach. 'OfficialWebsite/app/docs/ai/page.tsx' = @( - "generated against version ($semver)" + "written against version ($semver)" ) } From fce9c1648328c832ae1f12923b617c5158ae6120 Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 06:59:06 +0300 Subject: [PATCH 14/22] perf(policies): read the loaded assemblies' references once per load epoch Each subtype search asked every loaded assembly for its references and its types. A scan over a connected model searches once per type it reaches, so the first query repeated that work for every one. The references are now read once per assembly-load epoch, and an assembly's types only when a search reaches an assembly that could declare a subtype. Co-Authored-By: Claude Opus 5 --- .../Policies/Source/KnownSubtypes.cs | 54 ++++++++++++++----- 1 file changed, 42 insertions(+), 12 deletions(-) diff --git a/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs b/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs index 7afc1f2..7949a0a 100644 --- a/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs +++ b/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs @@ -26,6 +26,8 @@ internal static class KnownSubtypes private static int _epoch; + private static Snapshot? _snapshot; + static KnownSubtypes() => AppDomain.CurrentDomain.AssemblyLoad += (_, loaded) => { @@ -53,32 +55,30 @@ internal static IReadOnlyList Of(Type type) return known.Types; } - Type[] types = Search(type); + Type[] types = Search(type, Assemblies(epoch)); Found[type] = (epoch, types); return types; } - private static Type[] Search(Type type) + private static Type[] Search(Type type, IEnumerable assemblies) { - string? home = type.Assembly.GetName().Name; + string home = type.Assembly.GetName().Name ?? string.Empty; List found = new(); - foreach (Assembly assembly in AppDomain.CurrentDomain.GetAssemblies()) + foreach (Candidate candidate in assemblies) { - if (assembly.IsDynamic - || (assembly != type.Assembly - && !assembly.GetReferencedAssemblies().Any(reference => reference.Name == home))) + if (candidate.Assembly != type.Assembly && !candidate.References.Contains(home)) { continue; } - foreach (Type candidate in TypesOf(assembly)) + foreach (Type declared in candidate.Types.Value) { - if (candidate != type && !candidate.ContainsGenericParameters && type.IsAssignableFrom(candidate)) + if (declared != type && !declared.ContainsGenericParameters && type.IsAssignableFrom(declared)) { - found.Add(candidate); + found.Add(declared); } } } @@ -86,7 +86,33 @@ private static Type[] Search(Type type) return found.ToArray(); } - private static IEnumerable TypesOf(Assembly assembly) + /// + /// The loaded assemblies that could declare a subtype, each with the names it references, read once + /// per epoch; an assembly's types are read only when a search reaches it. + /// + private static Candidate[] Assemblies(int epoch) + { + if (Volatile.Read(ref _snapshot) is { } snapshot && snapshot.Epoch == epoch) + { + return snapshot.Candidates; + } + + Candidate[] candidates = AppDomain.CurrentDomain.GetAssemblies() + .Where(assembly => !assembly.IsDynamic) + .Select(assembly => new Candidate( + assembly, + new HashSet( + assembly.GetReferencedAssemblies().Select(reference => reference.Name ?? string.Empty), + StringComparer.Ordinal), + new Lazy(() => TypesOf(assembly)))) + .ToArray(); + + Volatile.Write(ref _snapshot, new Snapshot(epoch, candidates)); + + return candidates; + } + + private static Type[] TypesOf(Assembly assembly) { try { @@ -94,7 +120,11 @@ private static IEnumerable TypesOf(Assembly assembly) } catch (ReflectionTypeLoadException partial) { - return partial.Types.OfType(); + return partial.Types.OfType().ToArray(); } } + + private sealed record Candidate(Assembly Assembly, HashSet References, Lazy Types); + + private sealed record Snapshot(int Epoch, Candidate[] Candidates); } From 203f5c94426d84fd9da0ca27a9f351ccabdd739e Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 07:48:19 +0300 Subject: [PATCH 15/22] fix(policies): what a member can hold, read without over-blocking ordinary queries A security review and a blast-radius review of 6c9e171 pulled in opposite directions: nine more ways a denied value still reached the result, and five classes of ordinary queries projected, refused or broken for nothing. Both are answered here. Security fixes: - A member the model does not map counts as loaded again. A getter over a mapped field or a private auto-included navigation hands out what EF Core read. - A [DwDenied] on an override, or on an interface's implementation, applies to the member through the base type or the interface. The attribute walker reads it, so Where, Order, Group and Select all see it, and its cache follows the subtype index. - The subtype index covers open generic subtypes and applications' subclasses of framework classes (Exception, Stream). A type parameter left open holds anything. - A rule on a path through a subtype's member, or through one of two members differing only in case, is enforced beneath a member instead of dropped as stale. - A projection that builds a subtype of T is projected to T when the subtype declares a denial. - An async lazy-loader delegate and an injected DbContext count as loading. - An application's collection that implements only IEnumerable holds anything. Precision: - A member that can hold an object of any type no longer asks for a projection. An entity's values come from the database and never count as holding one. - A reshaped chain counts every navigation as loaded only when it has an include or builds an object; otherwise the model decides. - A projected member is read as the type its initializer constructs. - Under a "*" deny, each path the walk skips is asked of the policy, not refused. - An initializer after a constructor with arguments narrows its own bindings again. - A member a projection leaves out that the unguarded call would have returned is recorded in the trace. The subtype index is built once per load of an assembly that could declare a subtype, never for framework or resource assemblies. The two reviews' probes are kept as ReviewLeakTests, ReviewOverBlockTests and ReviewDefaultOrderTests. A few fail-closed outcomes are asserted as such: in-memory rows where any loaded subtype declares a denial, and a "*" deny whose subtype paths are not named. Co-Authored-By: Claude Opus 5 --- .../Policies/ProjectionReachTests.cs | 145 ++- .../Policies/ReviewDefaultOrderTests.cs | 190 +++ .../Policies/ReviewLeakTests.cs | 746 ++++++++++++ .../Policies/ReviewLeakTests2.cs | 450 ++++++++ .../Policies/ReviewLeakTests3.cs | 174 +++ .../Policies/ReviewLeakTests4.cs | 142 +++ .../Policies/ReviewLeakTests5.cs | 121 ++ .../Policies/ReviewLeakTests6.cs | 96 ++ .../Policies/ReviewOverBlockTests.cs | 1014 +++++++++++++++++ .../Resolution/AttributePolicyProvider.cs | 156 ++- .../Policies/Source/FilterSanitizer.cs | 406 +++++-- .../Policies/Source/KnownSubtypes.cs | 174 ++- DynamicWhere.ex/Policies/Source/RowShape.cs | 143 ++- DynamicWhere.ex/Source/QueryRoot.cs | 61 + 14 files changed, 3803 insertions(+), 215 deletions(-) create mode 100644 DynamicWhere.Tests/Policies/ReviewDefaultOrderTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewLeakTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewLeakTests2.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewLeakTests3.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewLeakTests4.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewLeakTests5.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewLeakTests6.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewOverBlockTests.cs diff --git a/DynamicWhere.Tests/Policies/ProjectionReachTests.cs b/DynamicWhere.Tests/Policies/ProjectionReachTests.cs index 2edf395..c50748e 100644 --- a/DynamicWhere.Tests/Policies/ProjectionReachTests.cs +++ b/DynamicWhere.Tests/Policies/ProjectionReachTests.cs @@ -218,7 +218,7 @@ public class RcBadge public string? Pin { get; set; } } - /// Members EF Core does not map, which hold only what the class puts there. + /// A member EF Core does not map that can hold an object of any type. public class RcBag { public int Id { get; set; } @@ -231,15 +231,6 @@ public class RcBag [NotMapped] public object? Extra { get; set; } - - [NotMapped] - public RcScratch Scratch { get; set; } = new(); - } - - public class RcScratch - { - [DwDenied] - public string? Cost { get; set; } } public class RcBagOwner @@ -306,6 +297,48 @@ public class RcStoreKeeper public string Name { get; set; } = string.Empty; } + /// An owned chain whose last owned type exposes a mapped field through a getter the model does not map. + public class RcRoom + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public RcShelf Shelf { get; set; } = new(); + } + + public class RcShelf + { + public string Label { get; set; } = string.Empty; + + public RcBox Box { get; set; } = new(); + } + + public class RcBox + { + public string Label { get; set; } = string.Empty; + + public RcLid Lid { get; set; } = new(); + } + + public class RcLid + { + private string? _code; + + public string Colour { get; set; } = string.Empty; + + [NotMapped] + public RcCodeView Code => new() { Value = _code }; + + public void Seal(string code) => _code = code; + } + + public class RcCodeView + { + [DwDenied] + public string? Value { get; set; } + } + public sealed class ProjectionReachContext : DbContext { private readonly SqliteConnection _connection; @@ -338,6 +371,8 @@ public sealed class ProjectionReachContext : DbContext public DbSet Stores => Set(); + public DbSet Rooms => Set(); + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); protected override void OnModelCreating(ModelBuilder model) @@ -352,6 +387,8 @@ protected override void OnModelCreating(ModelBuilder model) model.Entity().Navigation(h => h.Badge).AutoInclude(); model.Entity().OwnsOne(s => s.Address, a => a.Property(x => x.Zip)); model.Entity().OwnsOne(s => s.Place); + model.Entity().OwnsOne( + r => r.Shelf, s => s.OwnsOne(x => x.Box, b => b.OwnsOne(y => y.Lid, l => l.Property("_code")))); } } @@ -539,6 +576,32 @@ public sealed class RcLink public RcNode? Head { get; set; } } + /// A base type whose subtype's field a rule denies, no attribute anywhere. + public class RcPet + { + public string Name { get; set; } = string.Empty; + } + + public class RcHamster : RcPet + { + public string? Tag { get; set; } + } + + public class RcCage + { + public int Id { get; set; } + + public RcPet? Pet { get; set; } + } + + /// A row in memory holding an object of any type, with nothing denied anywhere. + public class RcNote + { + public int Id { get; set; } + + public object? Payload { get; set; } + } + /// Rows in memory whose subtype declares a denied field. public class RcAnimal { @@ -601,6 +664,10 @@ public ProjectionReachTests() _db.Shops.Add(new RcShop { Name = "S1", Notes = "notes", Address = new RcAddress("Basra", "zip-secret") }); _db.Stores.Add(new RcStore { Name = "T1", Place = new RcPlace { City = "Basra", Zone = "Z1" }, Keeper = new RcStoreKeeper { Name = "K" } }); + RcRoom room = new() { Name = "R1", Shelf = new RcShelf { Label = "s", Box = new RcBox { Label = "b", Lid = new RcLid { Colour = "red" } } } }; + room.Shelf.Box.Lid.Seal("lid-secret"); + _db.Rooms.Add(room); + _db.SaveChanges(); _db.ChangeTracker.Clear(); } @@ -1114,11 +1181,63 @@ public void An_automatically_included_navigation_counts_as_loaded() } /// - /// A member EF Core does not map holds only what the class puts there, never a value the query read, - /// so neither an object nor a policed type there asks for a projection. + /// A getter the model does not map, past the walker's four segments in an owned chain, hands out a + /// mapped field: what it returns counts as loaded, so its denied field asks for the projection. + /// + [Fact] + public void A_getter_the_model_does_not_map_past_the_walk_counts_as_loaded() + { + Assert.True(Holds(_db.Rooms.AsNoTracking().ToList(), "lid-secret")); + + PolicyQueryable guarded = Guard(_db.Rooms); + RcRoom room = guarded.ToList(new Filter()).Data.Single(); + + Assert.False(Holds(room, "lid-secret")); + Assert.Equal("red", room.Shelf.Box.Lid.Colour); + } + + /// + /// A rule on a field only a subtype declares, reached through a member declared as the base type, is + /// enforced like an attribute there would be: named or not, the field does not come back. + /// + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void A_rule_on_a_subtype_field_through_a_base_typed_member_is_enforced(DwTier tier) + { + RcCage[] rows = { new() { Id = 1, Pet = new RcHamster { Name = "Ham", Tag = "tag-by-rule" } } }; + FakePolicyProvider rules = new FakePolicyProvider() + .Add("Pet.Tag", PolicyFeature.Select, PolicyEffect.Deny, PolicyLevel.DynamicGlobal); + + Assert.False(Holds(Guard(rows.AsQueryable(), tier, rules).ToList(new Filter()).Data, "tag-by-rule")); + + Exception? named = Record.Exception(() => + Assert.False(Holds(Guard(rows.AsQueryable(), tier, rules).ToList(Selecting("Id", "Pet")).Data, "tag-by-rule"))); + + Assert.True(named is null or PolicyException, named?.ToString()); + } + + /// + /// With nothing denied, a row in memory holding an object of any type is returned as it is: such a + /// member asks for no projection on its own. + /// + [Fact] + public void A_row_in_memory_holding_an_object_with_nothing_denied_is_returned_as_it_is() + { + RcNote[] rows = { new() { Id = 1, Payload = new Dictionary { ["a"] = 1 } } }; + + PolicyQueryable guarded = Guard(rows.AsQueryable()); + + Assert.Same(rows[0], guarded.ToList(new Filter()).Data.Single()); + Assert.DoesNotContain(guarded.LastTrace!.Decisions, d => d.Action == PolicyAction.Dropped); + } + + /// + /// A member that can hold an object of any type asks for no projection: the policy cannot see into it + /// whether or not one is built. The entity comes back as loaded, its included navigation with it. /// [Fact] - public void A_member_the_model_does_not_map_asks_for_nothing() + public void A_member_that_can_hold_anything_asks_for_nothing() { PolicyQueryable guarded = Guard(_db.Bags.Include(b => b.Owner)); RcBag bag = guarded.ToList(new Filter()).Data.Single(); diff --git a/DynamicWhere.Tests/Policies/ReviewDefaultOrderTests.cs b/DynamicWhere.Tests/Policies/ReviewDefaultOrderTests.cs new file mode 100644 index 0000000..81c5b6e --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewDefaultOrderTests.cs @@ -0,0 +1,190 @@ +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Classes.Core; +using DynamicWhere.ex.Enums; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Xunit.Abstractions; + +// DCMP's DW-13 shapes after 6c9e171's column-only rule: a projected row must still take its declared +// default when it assigns every default field a mapped column, directly, through a reference +// navigation or through EF.Property on a shadow property. + +namespace DynamicWhere.Tests.Policies +{ + public class ZbTask + { + public int Id { get; set; } + + public string Code { get; set; } = string.Empty; + + public int? ZbTaskGroupId { get; set; } + + public ZbTaskGroup? Group { get; set; } + } + + public class ZbTaskGroup + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + } + + [DwEntity(DefaultOrder = "Code desc")] + public class ZbTaskRow + { + public int Id { get; set; } + + public string Code { get; set; } = string.Empty; + } + + /// DCMP's shape with a denied member, so the guard also synthesizes a projection. + [DwEntity(DefaultOrder = "Code desc")] + public class ZbTaskSecretRow + { + public int Id { get; set; } + + public string Code { get; set; } = string.Empty; + + [DwDenied] + public string? Secret { get; set; } + } + + [DwEntity(DefaultOrder = "Seq")] + public class ZbTaskSeqRow + { + public int Id { get; set; } + + public short Seq { get; set; } + } + + [DwEntity(DefaultOrder = "GroupName desc")] + public class ZbTaskGroupRow + { + public int Id { get; set; } + + public string GroupName { get; set; } = string.Empty; + } + + public sealed class ZbDefaultOrderContext : DbContext + { + private readonly SqliteConnection _connection; + + public ZbDefaultOrderContext(SqliteConnection connection) => _connection = connection; + + public DbSet Tasks => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + protected override void OnModelCreating(ModelBuilder model) => + model.Entity().Property("status_seq"); + } + + public sealed class ReviewDefaultOrderTests : IDisposable + { + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZbDefaultOrderContext _db; + + public ReviewDefaultOrderTests(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + + _db = new ZbDefaultOrderContext(_connection); + _db.Database.EnsureCreated(); + + // Inserted B, C, A: none of the expected orders is the insertion order. + foreach ((string code, short seq, string group) in new[] { ("B", (short)2, "g2"), ("C", (short)1, "g3"), ("A", (short)3, "g1") }) + { + ZbTask task = new() { Code = code, Group = new ZbTaskGroup { Name = group } }; + + _db.Tasks.Add(task); + _db.Entry(task).Property("status_seq").CurrentValue = seq; + } + + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static PolicyQueryable Guard(IQueryable source) where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = DwTier.Strict, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + private string Log(string label, IEnumerable ids) + { + string order = string.Join(",", ids.Select(id => _db.Tasks.AsNoTracking().Single(t => t.Id == id).Code)); + string line = $"{label}: {order}"; + + _out.WriteLine(line); + ZbProbeLog.Write(line); + + return order; + } + + [Fact] + public void Zb_G1_a_projected_row_assigning_a_column_takes_Code_desc() + { + IQueryable rows = _db.Tasks.Select(t => new ZbTaskRow { Id = t.Id, Code = t.Code }); + + Assert.Equal("C,B,A", Log("G1 typed", Guard(rows).ToList(new Filter()).Data.Select(r => r.Id))); + Assert.Equal("C,B", Log("G1 paged", Guard(rows).ToList(new Filter { Page = new PageBy { PageNumber = 1, PageSize = 2 } }).Data.Select(r => r.Id))); + } + + [Fact] + public void Zb_G1b_a_projected_row_with_a_denied_member_still_takes_Code_desc() + { + IQueryable rows = _db.Tasks.Select(t => new ZbTaskSecretRow { Id = t.Id, Code = t.Code, Secret = t.Code }); + + Assert.Equal("C,B,A", Log("G1b typed", Guard(rows).ToList(new Filter()).Data.Select(r => r.Id))); + } + + [Fact] + public void Zb_G2_a_projected_row_bound_through_EF_Property_takes_its_default() + { + IQueryable rows = _db.Tasks.Select(t => new ZbTaskSeqRow { Id = t.Id, Seq = EF.Property(t, "status_seq") }); + + Assert.Equal("C,B,A", Log("G2 typed", Guard(rows).ToList(new Filter()).Data.Select(r => r.Id))); + } + + [Fact] + public void Zb_G3_a_projected_row_reading_through_a_reference_navigation_takes_its_default() + { + IQueryable rows = _db.Tasks.Select(t => new ZbTaskGroupRow { Id = t.Id, GroupName = t.Group!.Name }); + + Assert.Equal("C,B,A", Log("G3 typed", Guard(rows).ToList(new Filter()).Data.Select(r => r.Id))); + } + + /// Informational: a computed value is left unordered by design since 6c9e171. + [Fact] + public void Zb_G4_a_projected_row_assigning_a_computed_value() + { + IQueryable rows = _db.Tasks.Select(t => new ZbTaskRow { Id = t.Id, Code = t.Code.Trim() }); + + Log("G4 typed (Trim)", Guard(rows).ToList(new Filter()).Data.Select(r => r.Id)); + } + + /// Informational: a projection over an anonymous intermediate row. + [Fact] + public void Zb_G5_a_projected_row_over_an_anonymous_intermediate() + { + IQueryable rows = _db.Tasks.Select(t => new { t.Id, t.Code }).Select(x => new ZbTaskRow { Id = x.Id, Code = x.Code }); + + Log("G5 typed (anonymous intermediate)", Guard(rows).ToList(new Filter()).Data.Select(r => r.Id)); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewLeakTests.cs b/DynamicWhere.Tests/Policies/ReviewLeakTests.cs new file mode 100644 index 0000000..b85da7f --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewLeakTests.cs @@ -0,0 +1,746 @@ +using System.Collections; +using System.ComponentModel.DataAnnotations.Schema; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.DTOs; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; + +namespace DynamicWhere.Tests.Policies +{ + // ============================================================================ round 3 probes: models + + // ---- P1: an unmapped getter over a private, mapped field ---------------------------------------- + + public class ZrEmployee + { + private string? _band; + + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + /// Computed from a private field EF Core maps as a column; the model does not map this member. + [NotMapped] + public ZrPay Pay => new() { Grade = "G", Band = _band }; + + public void SetBand(string band) => _band = band; + } + + public class ZrPay + { + public string? Grade { get; set; } + + [DwDenied] + public string? Band { get; set; } + } + + // ---- P1b: the same inside an owned type ----------------------------------------------------------- + + public class ZrClinic + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public ZrContact Contact { get; set; } = new(); + } + + public class ZrContact + { + private string? _phone; + + public string City { get; set; } = string.Empty; + + [NotMapped] + public ZrPhoneView Phone => new() { Number = _phone }; + + public void SetPhone(string phone) => _phone = phone; + } + + public class ZrPhoneView + { + [DwDenied] + public string? Number { get; set; } + } + + // ---- P2: an unmapped getter exposing a private, automatically included navigation ------------------ + + public class ZrLedger + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + private List Entries { get; set; } = new(); + + [NotMapped] + public IReadOnlyList Items => Entries; + + public void Add(ZrEntry entry) => Entries.Add(entry); + } + + public class ZrEntry + { + public int Id { get; set; } + + public int ZrLedgerId { get; set; } + + public string Memo { get; set; } = string.Empty; + + [DwDenied] + public string? Amount { get; set; } + } + + /// An entity that includes the ledger, so the ledger is a loaded navigation beneath the row. + public class ZrBook + { + public int Id { get; set; } + + public string Title { get; set; } = string.Empty; + + public int ZrLedgerId { get; set; } + + public ZrLedger? Ledger { get; set; } + } + + // ---- P3: generic subtypes, which the subtype search never lists ----------------------------------- + + public class ZrCreature + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + } + + public class ZrTagged : ZrCreature + { + [DwDenied] + public string? Secret { get; set; } + + public TTag? Tag { get; set; } + } + + public class ZrPen + { + public int Id { get; set; } + + public string Label { get; set; } = string.Empty; + + public ZrCreature? Pet { get; set; } + } + + /// An EF Core hierarchy whose derived entity is a closed generic type. + public class ZrOrg + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + } + + public class ZrGenOrg : ZrOrg + { + [DwDenied] + public string? Secret { get; set; } + } + + /// A derived entity whose field no attribute denies, for a rule to deny instead. + public class ZrBank : ZrOrg + { + public string? Swift { get; set; } + } + + public class ZrLoan + { + public int Id { get; set; } + + public string Note { get; set; } = string.Empty; + + public int ZrOrgId { get; set; } + + public ZrOrg? Org { get; set; } + } + + public class ZrLoanRow + { + public int Id { get; set; } + + public string Note { get; set; } = string.Empty; + + public ZrOrg? Org { get; set; } + } + + // ---- P4: a member typed as a framework base class --------------------------------------------------- + + public class ZrDeclinedException : Exception + { + public ZrDeclinedException() + : base("declined") + { + } + + [DwDenied] + public string? Pan { get; set; } + } + + public class ZrJobResult + { + public int Id { get; set; } + + public string Status { get; set; } = string.Empty; + + public Exception? Error { get; set; } + } + + // ---- P5: a rule on a subtype's field, reached through a base-typed member --------------------------- + + public class ZrCat : ZrCreature + { + public string? Microchip { get; set; } + } + + // ---- P8: two members differing only in letter case ------------------------------------------------- + + public class ZrCaseRow + { + public int Id { get; set; } + + public ZrCardInfo INFO { get; set; } = new(); + + public ZrSafeInfo Info { get; set; } = new(); + } + + public class ZrCardInfo + { + [DwDenied] + public string? Pan { get; set; } + } + + public class ZrSafeInfo + { + public string? Label { get; set; } + } + + // ---- P6: a Concat of two projections --------------------------------------------------------------- + + public class ZrAcctRow + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public ZrPersonView? Person { get; set; } + } + + public class ZrPersonView + { + public string? Name { get; set; } + + [DwDenied] + public string? TaxId { get; set; } + } + + // ---- P9: many-to-many with an explicit join entity carrying a denied payload ------------------------ + + public class ZrPost + { + public int Id { get; set; } + + public string Title { get; set; } = string.Empty; + + public List Tags { get; set; } = new(); + + public List PostTags { get; set; } = new(); + } + + public class ZrTag + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public List Posts { get; set; } = new(); + + public List PostTags { get; set; } = new(); + } + + public class ZrPostTag + { + public int ZrPostId { get; set; } + + public int ZrTagId { get; set; } + + public ZrPost? Post { get; set; } + + public ZrTag? Tag { get; set; } + + [DwDenied] + public string? AddedBy { get; set; } + } + + public sealed class ZrProbeContext : DbContext + { + private readonly SqliteConnection _connection; + + public ZrProbeContext(SqliteConnection connection) => _connection = connection; + + public DbSet Employees => Set(); + + public DbSet Clinics => Set(); + + public DbSet Ledgers => Set(); + + public DbSet Books => Set(); + + public DbSet Orgs => Set(); + + public DbSet Loans => Set(); + + public DbSet Posts => Set(); + + public DbSet Tags => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + protected override void OnModelCreating(ModelBuilder model) + { + model.Entity().Property("_band"); + model.Entity().OwnsOne(c => c.Contact, o => o.Property("_phone")); + model.Entity(b => + { + b.HasMany("Entries").WithOne().HasForeignKey(e => e.ZrLedgerId); + b.Navigation("Entries").AutoInclude(); + }); + model.Entity>(); + model.Entity(); + model.Entity() + .HasMany(p => p.Tags) + .WithMany(t => t.Posts) + .UsingEntity( + j => j.HasOne(pt => pt.Tag).WithMany(t => t.PostTags).HasForeignKey(pt => pt.ZrTagId), + j => j.HasOne(pt => pt.Post).WithMany(p => p.PostTags).HasForeignKey(pt => pt.ZrPostId), + j => j.HasKey(pt => new { pt.ZrPostId, pt.ZrTagId })); + } + } + + // ============================================================================ round 3 probes: tests + + /// + /// Round 3 adversarial probes. Every assertion states the safe outcome, so a red test is a leak. A refusal with + /// FieldDeniedForSelect is a safe outcome. + /// + public sealed class ReviewLeakTests : IDisposable + { + private readonly SqliteConnection _connection; + private readonly ZrProbeContext _db; + + public ReviewLeakTests() + { + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + + _db = new ZrProbeContext(_connection); + _db.Database.EnsureCreated(); + + ZrEmployee employee = new() { Name = "E1" }; + employee.SetBand("band-secret"); + _db.Employees.Add(employee); + + ZrClinic clinic = new() { Name = "C1", Contact = new ZrContact { City = "Basra" } }; + clinic.Contact.SetPhone("phone-secret"); + _db.Clinics.Add(clinic); + + ZrLedger ledger = new() { Name = "L1" }; + ledger.Add(new ZrEntry { Memo = "m", Amount = "amount-secret" }); + _db.Ledgers.Add(ledger); + _db.Books.Add(new ZrBook { Title = "B1", Ledger = ledger }); + + _db.Loans.Add(new ZrLoan { Note = "gen", Org = new ZrGenOrg { Name = "G", Secret = "generic-entity-secret" } }); + _db.Loans.Add(new ZrLoan { Note = "bank", Org = new ZrBank { Name = "K", Swift = "swift-by-rule" } }); + + ZrPost post = new() { Title = "P1" }; + ZrTag tag = new() { Name = "T1" }; + _db.Posts.Add(post); + _db.Tags.Add(tag); + _db.SaveChanges(); + _db.Set().Add(new ZrPostTag { ZrPostId = post.Id, ZrTagId = tag.Id, AddedBy = "join-secret" }); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static DwPolicyContext Caller() => + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1").WithValue("TenantId", 1); + + private static PolicyQueryable Guard(IQueryable source, DwTier tier = DwTier.Strict, params IDwPolicyProvider[] more) + where T : class => + source.ApplyPolicy( + Caller(), + new DwPolicyOptions { Tier = tier, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() }.Concat(more).ToArray())); + + private static Filter Selecting(params string[] fields) => new() { Selects = fields.ToList() }; + + private static FakePolicyProvider Denying(params string[] paths) + { + FakePolicyProvider rules = new(); + + foreach (string path in paths) + { + rules.Add(path, PolicyFeature.Select, PolicyEffect.Deny, PolicyLevel.DynamicGlobal); + } + + return rules; + } + + private static FakePolicyProvider DenyingAllBut(params string[] allowed) + { + FakePolicyProvider rules = new FakePolicyProvider() + .Add("*", PolicyFeature.Select, PolicyEffect.Deny, PolicyLevel.DynamicGlobal); + + foreach (string path in allowed) + { + rules.Add(path, PolicyFeature.Select, PolicyEffect.Allow, PolicyLevel.DynamicGlobal); + } + + return rules; + } + + /// + /// True when anything reachable from a value holds the text, read by each object's runtime type, through + /// every readable public property (getter-only ones included, as a serializer writes them). + /// + private static bool Holds(object? value, string text) + { + HashSet seen = new(ReferenceEqualityComparer.Instance); + Stack pending = new(); + + pending.Push(value); + + while (pending.Count > 0) + { + object? current = pending.Pop(); + + if (current is null || current is ValueType || current is System.Reflection.MemberInfo) + { + continue; + } + + if (current is string held) + { + if (held == text) + { + return true; + } + + continue; + } + + if (!seen.Add(current)) + { + continue; + } + + if (current is IEnumerable items) + { + foreach (object? item in items) + { + pending.Push(item); + } + + continue; + } + + foreach (System.Reflection.PropertyInfo property in current.GetType().GetProperties()) + { + if (property.GetIndexParameters().Length != 0 || !property.CanRead) + { + continue; + } + + object? read; + + try + { + read = property.GetValue(current); + } + catch + { + continue; + } + + pending.Push(read); + } + } + + return false; + } + + /// Runs a guarded read; a FieldDeniedForSelect refusal is safe and reads as no rows. + private static object? SafeRead(Func read) + { + try + { + return read(); + } + catch (PolicyException refusal) when (refusal.ErrorCode == PolicyErrorCode.FieldDeniedForSelect) + { + return null; + } + } + + // ------------------------------------------------------------------------------------------------ P1 + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P1_an_unmapped_getter_over_a_private_mapped_field_does_not_carry_the_denied_value(DwTier tier) + { + // Unguarded, the entity carries the value through the getter. + Assert.True(Holds(_db.Employees.AsNoTracking().ToList(), "band-secret")); + + Assert.False(Holds(SafeRead(() => Guard(_db.Employees, tier).ToList(new Filter()).Data), "band-secret")); + } + + [Fact] + public void Zr_P1_dynamic_terminal() + { + Assert.False(Holds(SafeRead(() => Guard(_db.Employees).ToListDynamic(new Filter()).Data), "band-secret")); + } + + [Fact] + public async Task Zr_P1_segment_terminal() + { + object? rows = null; + + try + { + rows = (await Guard(_db.Employees).ToListAsync(new Segment(), CancellationToken.None)).Data; + } + catch (PolicyException refusal) when (refusal.ErrorCode == PolicyErrorCode.FieldDeniedForSelect) + { + } + + Assert.False(Holds(rows, "band-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P1b_an_owned_type_with_an_unmapped_getter_over_a_private_mapped_field(DwTier tier) + { + Assert.True(Holds(_db.Clinics.AsNoTracking().ToList(), "phone-secret")); + + Assert.False(Holds(SafeRead(() => Guard(_db.Clinics, tier).ToList(new Filter()).Data), "phone-secret")); + } + + // ------------------------------------------------------------------------------------------------ P2 + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P2_an_unmapped_getter_exposing_a_private_auto_included_navigation(DwTier tier) + { + Assert.True(Holds(_db.Ledgers.AsNoTracking().ToList(), "amount-secret")); + + Assert.False(Holds(SafeRead(() => Guard(_db.Ledgers, tier).ToList(new Filter()).Data), "amount-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P2b_the_same_beneath_an_included_navigation(DwTier tier) + { + IQueryable source = _db.Books.Include(b => b.Ledger); + + Assert.True(Holds(source.AsNoTracking().ToList(), "amount-secret")); + + Assert.False(Holds(SafeRead(() => Guard(source, tier).ToList(new Filter()).Data), "amount-secret")); + } + + // ------------------------------------------------------------------------------------------------ P3 + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P3_rows_in_memory_of_a_generic_subtype_with_a_denied_field(DwTier tier) + { + ZrCreature[] rows = { new ZrTagged { Id = 1, Name = "Rex", Secret = "generic-secret", Tag = 7 } }; + + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable(), tier).ToList(new Filter()).Data), "generic-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P3b_a_base_typed_member_in_memory_holding_a_generic_subtype(DwTier tier) + { + ZrPen[] rows = { new() { Id = 1, Label = "pen", Pet = new ZrTagged { Id = 2, Name = "Rex", Secret = "generic-secret" } } }; + + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable(), tier).ToList(new Filter()).Data), "generic-secret")); + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable(), tier).ToList(Selecting("Id", "Pet")).Data), "generic-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P3c_a_projected_member_holding_a_closed_generic_derived_entity(DwTier tier) + { + IQueryable rows = _db.Loans.Select(l => new ZrLoanRow { Id = l.Id, Note = l.Note, Org = l.Org }); + + Assert.True(Holds(rows.ToList(), "generic-entity-secret")); + + Assert.False(Holds(SafeRead(() => Guard(rows, tier).ToList(new Filter()).Data), "generic-entity-secret")); + Assert.False(Holds(SafeRead(() => Guard(rows, tier).ToList(Selecting("Id", "Org")).Data), "generic-entity-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P3c_control_the_entity_query_reads_the_model(DwTier tier) + { + IQueryable source = _db.Loans.Include(l => l.Org); + + Assert.True(Holds(source.AsNoTracking().ToList(), "generic-entity-secret")); + Assert.False(Holds(SafeRead(() => Guard(source, tier).ToList(new Filter()).Data), "generic-entity-secret")); + Assert.False(Holds(SafeRead(() => Guard(_db.Orgs, tier).ToList(new Filter()).Data), "generic-entity-secret")); + } + + // ------------------------------------------------------------------------------------------------ P4 + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P4_a_member_typed_as_a_framework_base_class_holding_an_application_subtype(DwTier tier) + { + ZrJobResult[] rows = { new() { Id = 1, Status = "failed", Error = new ZrDeclinedException { Pan = "exception-secret" } } }; + + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable(), tier).ToList(new Filter()).Data), "exception-secret")); + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable(), tier).ToList(Selecting("Id", "Error")).Data), "exception-secret")); + } + + [Fact] + public void Zr_P4b_under_a_deny_by_default_policy() + { + ZrJobResult[] rows = { new() { Id = 1, Status = "failed", Error = new ZrDeclinedException { Pan = "exception-secret" } } }; + + FakePolicyProvider rules = DenyingAllBut("Id", "Error"); + + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable(), DwTier.Strict, rules).ToList(Selecting("Id", "Error")).Data), "exception-secret")); + } + + // ------------------------------------------------------------------------------------------------ P5 + + [Fact] + public void Zr_P5_control_a_rule_on_a_subtype_field_at_the_root_is_enforced() + { + ZrCreature[] rows = { new ZrCat { Id = 1, Name = "Tom", Microchip = "chip-by-rule" } }; + + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable(), DwTier.Strict, Denying("Microchip")).ToList(new Filter()).Data), "chip-by-rule")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P5_a_rule_on_a_subtype_field_through_a_base_typed_member_in_memory(DwTier tier) + { + ZrPen[] rows = { new() { Id = 1, Label = "pen", Pet = new ZrCat { Id = 2, Name = "Tom", Microchip = "chip-by-rule" } } }; + + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable(), tier, Denying("Pet.Microchip")).ToList(new Filter()).Data), "chip-by-rule")); + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable(), tier, Denying("Pet.Microchip")).ToList(Selecting("Id", "Pet")).Data), "chip-by-rule")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P5_control_the_entity_query_enforces_the_rule(DwTier tier) + { + Assert.True(Holds(_db.Loans.Include(l => l.Org).AsNoTracking().ToList(), "swift-by-rule")); + + Assert.False(Holds(SafeRead(() => Guard(_db.Loans.Include(l => l.Org), tier, Denying("Org.Swift")).ToList(new Filter()).Data), "swift-by-rule")); + Assert.False(Holds(SafeRead(() => Guard(_db.Loans, tier, Denying("Org.Swift")).ToList(Selecting("Id", "Org")).Data), "swift-by-rule")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P5_a_rule_on_a_subtype_field_through_a_projected_member(DwTier tier) + { + IQueryable rows = _db.Loans.Select(l => new ZrLoanRow { Id = l.Id, Note = l.Note, Org = l.Org }); + + Assert.False(Holds(SafeRead(() => Guard(rows, tier, Denying("Org.Swift")).ToList(new Filter()).Data), "swift-by-rule")); + Assert.False(Holds(SafeRead(() => Guard(rows, tier, Denying("Org.Swift")).ToList(Selecting("Id", "Org")).Data), "swift-by-rule")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P5_a_rule_on_a_subtype_field_through_the_composable_Select(DwTier tier) + { + Assert.False(Holds( + SafeRead(() => Guard(_db.Loans, tier, Denying("Org.Swift")).Select(new List { "Id", "Org" }).ToList(new Filter()).Data), + "swift-by-rule")); + } + + // ------------------------------------------------------------------------------------------------ P6 + + [Fact] + public void Zr_P6_a_Concat_of_two_projections_assigning_different_members() + { + IQueryable rows = _db.Employees + .Select(e => new ZrAcctRow { Id = e.Id, Name = e.Name }) + .Concat(_db.Employees.Select(e => new ZrAcctRow + { + Id = e.Id, + Name = e.Name, + Person = new ZrPersonView { Name = e.Name, TaxId = "concat-secret" } + })); + + Exception? unguarded = Record.Exception(() => rows.ToList()); + + if (unguarded is not null) + { + // EF Core cannot run it at all, so nothing can leak through it. + return; + } + + Assert.False(Holds(SafeRead(() => Guard(rows).ToList(new Filter()).Data), "concat-secret")); + } + + // ------------------------------------------------------------------------------------------------ P8 + + [Fact] + public void Zr_P8_two_members_differing_only_in_letter_case() + { + ZrCaseRow[] rows = { new() { Id = 1, INFO = new ZrCardInfo { Pan = "case-secret" }, Info = new ZrSafeInfo { Label = "ok" } } }; + + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable()).ToList(new Filter()).Data), "case-secret")); + } + + // ------------------------------------------------------------------------------------------------ P9 + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P9_including_a_skip_navigation_does_not_carry_the_join_entity_payload(DwTier tier) + { + IQueryable source = _db.Posts.Include(p => p.Tags); + + bool unguardedLeaks = Holds(source.AsNoTracking().ToList(), "join-secret"); + + Assert.False(Holds(SafeRead(() => Guard(source, tier).ToList(new Filter()).Data), "join-secret"), + $"unguarded leaks: {unguardedLeaks}"); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewLeakTests2.cs b/DynamicWhere.Tests/Policies/ReviewLeakTests2.cs new file mode 100644 index 0000000..80f0fea --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewLeakTests2.cs @@ -0,0 +1,450 @@ +using System.Collections; +using System.Reflection; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.DTOs; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Xunit.Abstractions; + +namespace DynamicWhere.Tests.Policies +{ + // ============================================================================ round 3, batch 2: models + + // ---- P10: a derived type overrides a member its base declares, and denies it there ---------------- + + public class ZrVehicle + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public virtual string? Code { get; set; } + } + + public class ZrArmored : ZrVehicle + { + /// Overridden to deny it here; it reads and writes the base's storage, which EF Core maps. + [DwDenied] + public override string? Code + { + get => base.Code; + set => base.Code = value; + } + } + + public class ZrConvoy + { + public int Id { get; set; } + + public string Route { get; set; } = string.Empty; + + public int ZrVehicleId { get; set; } + + public ZrVehicle? Lead { get; set; } + } + + public class ZrConvoyRow + { + public int Id { get; set; } + + public ZrVehicle? Lead { get; set; } + } + + // ---- P11: a projection that builds a subtype of T -------------------------------------------------- + + public class ZrOrgDto + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + } + + public class ZrBankDto : ZrOrgDto + { + [DwDenied] + public string? Swift { get; set; } + } + + // ---- P3d: a base type whose only subtype is generic, under a policy denying what it does not name --- + + public class ZrShape + { + public int Id { get; set; } + + public string Kind { get; set; } = string.Empty; + } + + public class ZrShapeOf : ZrShape + { + public string? Hidden { get; set; } + } + + public class ZrCanvas + { + public int Id { get; set; } + + public ZrShape? Shape { get; set; } + } + + // ---- P12: an asynchronous lazy-loader delegate ------------------------------------------------------ + + public class ZrAsyncBlog + { + private readonly Func? _loader; + private List? _posts; + + public ZrAsyncBlog() + { + } + + private ZrAsyncBlog(Func lazyLoader) => _loader = lazyLoader; + + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public List Posts + { + get + { + _loader?.Invoke(this, CancellationToken.None, nameof(Posts)).GetAwaiter().GetResult(); + + return _posts ??= new List(); + } + set => _posts = value; + } + + public bool HasLoader => _loader is not null; + } + + public class ZrAsyncPost + { + public int Id { get; set; } + + public int ZrAsyncBlogId { get; set; } + + public string Title { get; set; } = string.Empty; + + [DwDenied] + public string? Draft { get; set; } + } + + public sealed class ZrProbeContext2 : DbContext + { + private readonly SqliteConnection _connection; + + public ZrProbeContext2(SqliteConnection connection) => _connection = connection; + + public DbSet Vehicles => Set(); + + public DbSet Convoys => Set(); + + public DbSet AsyncBlogs => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + protected override void OnModelCreating(ModelBuilder model) + { + model.Entity(); + model.Entity().Navigation(b => b.Posts).HasField("_posts"); + model.Entity().Ignore(b => b.HasLoader); + } + } + + // ============================================================================ round 3, batch 2: tests + + public sealed class ReviewLeakTests2 : IDisposable + { + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZrProbeContext2 _db; + + public ReviewLeakTests2(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + + _db = new ZrProbeContext2(_connection); + _db.Database.EnsureCreated(); + + _db.Convoys.Add(new ZrConvoy { Route = "R1", Lead = new ZrArmored { Name = "A1", Code = "override-secret" } }); + _db.AsyncBlogs.Add(new ZrAsyncBlog { Name = "B1", Posts = { new ZrAsyncPost { Title = "T", Draft = "async-draft" } } }); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static PolicyQueryable Guard(IQueryable source, DwTier tier = DwTier.Strict, params IDwPolicyProvider[] more) + where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = tier, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() }.Concat(more).ToArray())); + + private static Filter Selecting(params string[] fields) => new() { Selects = fields.ToList() }; + + private static FakePolicyProvider DenyingAllBut(params string[] allowed) + { + FakePolicyProvider rules = new FakePolicyProvider() + .Add("*", PolicyFeature.Select, PolicyEffect.Deny, PolicyLevel.DynamicGlobal); + + foreach (string path in allowed) + { + rules.Add(path, PolicyFeature.Select, PolicyEffect.Allow, PolicyLevel.DynamicGlobal); + } + + return rules; + } + + private static bool Holds(object? value, string text) + { + HashSet seen = new(ReferenceEqualityComparer.Instance); + Stack pending = new(); + + pending.Push(value); + + while (pending.Count > 0) + { + object? current = pending.Pop(); + + if (current is null || current is ValueType || current is MemberInfo) + { + continue; + } + + if (current is string held) + { + if (held == text) + { + return true; + } + + continue; + } + + if (!seen.Add(current)) + { + continue; + } + + if (current is IEnumerable items) + { + foreach (object? item in items) + { + pending.Push(item); + } + + continue; + } + + foreach (PropertyInfo property in current.GetType().GetProperties()) + { + if (property.GetIndexParameters().Length != 0 || !property.CanRead) + { + continue; + } + + try + { + pending.Push(property.GetValue(current)); + } + catch + { + // A getter that throws holds nothing readable. + } + } + } + + return false; + } + + private static object? SafeRead(Func read) + { + try + { + return read(); + } + catch (PolicyException refusal) when (refusal.ErrorCode == PolicyErrorCode.FieldDeniedForSelect) + { + return null; + } + } + + // ------------------------------------------------------------------------------------------------ P10 + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P10_rows_in_memory_of_a_subtype_that_denies_an_overridden_member(DwTier tier) + { + ZrVehicle[] rows = { new ZrArmored { Id = 1, Name = "A1", Code = "override-secret" } }; + + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable(), tier).ToList(new Filter()).Data), "override-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P10_an_entity_hierarchy_root_whose_derived_type_denies_an_overridden_column(DwTier tier) + { + Assert.True(Holds(_db.Vehicles.AsNoTracking().ToList(), "override-secret")); + + Assert.False(Holds(SafeRead(() => Guard(_db.Vehicles, tier).ToList(new Filter()).Data), "override-secret")); + Assert.False(Holds(SafeRead(() => Guard(_db.Vehicles, tier).ToListDynamic(new Filter()).Data), "override-secret")); + } + + /// The same member read through the base type is also groupable, so a summary returns it as a key. + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P10_a_summary_grouped_by_the_overridden_member(DwTier tier) + { + Summary summary = new() + { + GroupBy = new DynamicWhere.ex.Classes.Core.GroupBy + { + Fields = new List { "Code" }, + AggregateBy = new List + { + new() { Alias = "Total", Aggregator = DynamicWhere.ex.Enums.Aggregator.Count } + } + } + }; + + string sent; + + try + { + sent = System.Text.Json.JsonSerializer.Serialize(Guard(_db.Vehicles, tier).ToList(summary).Data); + } + catch (PolicyException refusal) when (refusal.ErrorCode is PolicyErrorCode.FieldDeniedForGroup or PolicyErrorCode.FieldDeniedForSelect) + { + return; + } + + _out.WriteLine(sent); + + Assert.DoesNotContain("override-secret", sent); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P10_an_included_base_typed_navigation(DwTier tier) + { + IQueryable source = _db.Convoys.Include(c => c.Lead); + + Assert.True(Holds(source.AsNoTracking().ToList(), "override-secret")); + + Assert.False(Holds(SafeRead(() => Guard(source, tier).ToList(new Filter()).Data), "override-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P10_a_base_typed_navigation_named_in_Selects(DwTier tier) + { + Assert.False(Holds(SafeRead(() => Guard(_db.Convoys, tier).ToList(Selecting("Id", "Lead")).Data), "override-secret")); + Assert.False(Holds(SafeRead(() => Guard(_db.Convoys, tier).ToList(Selecting("Id", "Lead.Code")).Data), "override-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P10_control_a_projected_base_typed_member_is_scanned(DwTier tier) + { + IQueryable rows = _db.Convoys.Select(c => new ZrConvoyRow { Id = c.Id, Lead = c.Lead }); + + Assert.False(Holds(SafeRead(() => Guard(rows, tier).ToList(new Filter()).Data), "override-secret")); + } + + // ------------------------------------------------------------------------------------------------ P11 + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P11_a_projection_that_builds_a_subtype_of_T(DwTier tier) + { + // A repository returning IQueryable from a projection into the derived DTO. + IQueryable covariant = _db.Convoys.Select(c => new ZrBankDto { Id = c.Id, Name = c.Route, Swift = c.Route + "-swift-secret" }); + IQueryable declared = _db.Convoys.Select(c => new ZrBankDto { Id = c.Id, Name = c.Route, Swift = c.Route + "-swift-secret" }); + + Assert.True(Holds(covariant.ToList(), "R1-swift-secret")); + + Assert.False(Holds(SafeRead(() => Guard(covariant, tier).ToList(new Filter()).Data), "R1-swift-secret")); + Assert.False(Holds(SafeRead(() => Guard(declared, tier).ToList(new Filter()).Data), "R1-swift-secret")); + Assert.False(Holds(SafeRead(() => Guard(covariant, tier).ToListDynamic(new Filter()).Data), "R1-swift-secret")); + } + + [Fact] + public void Zr_P11_control_the_same_rows_in_memory_are_projected() + { + ZrOrgDto[] rows = { new ZrBankDto { Id = 1, Name = "R1", Swift = "R1-swift-secret" } }; + + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable()).ToList(new Filter()).Data), "R1-swift-secret")); + } + + // ------------------------------------------------------------------------------------------------ P3d + + [Fact] + public void Zr_P3d_a_generic_only_subtype_under_a_deny_by_default_policy() + { + ZrCanvas[] rows = { new() { Id = 1, Shape = new ZrShapeOf { Id = 2, Kind = "k", Hidden = "unnamed-secret" } } }; + + FakePolicyProvider rules = DenyingAllBut("Id", "Shape", "Shape.Id", "Shape.Kind"); + + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable(), DwTier.Strict, rules).ToList(Selecting("Id", "Shape")).Data), "unnamed-secret")); + } + + [Fact] + public void Zr_P3d_control_a_non_generic_subtype_under_the_same_policy_is_refused() + { + ZrPen[] rows = { new() { Id = 1, Label = "pen", Pet = new ZrCat { Id = 2, Name = "Tom", Microchip = "unnamed-secret" } } }; + + FakePolicyProvider rules = DenyingAllBut("Id", "Pet", "Pet.Id", "Pet.Name"); + + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable(), DwTier.Strict, rules).ToList(Selecting("Id", "Pet")).Data), "unnamed-secret")); + } + + // ------------------------------------------------------------------------------------------------ P12 + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P12_an_asynchronous_lazy_loader_delegate(DwTier tier) + { + ZrAsyncBlog unguarded = _db.AsyncBlogs.AsNoTracking().ToList().Single(); + + try + { + _out.WriteLine($"EF injected the async loader: {unguarded.HasLoader}; unguarded posts: {unguarded.Posts.Count}"); + } + catch (InvalidOperationException detached) + { + // EF Core 6 refuses to lazy-load an entity read with AsNoTracking, so nothing can load after the query. + _out.WriteLine($"no lazy loading of an untracked entity here: {detached.Message}"); + + return; + } + + ZrAsyncBlog guarded = Guard(_db.AsyncBlogs, tier).ToList(new Filter()).Data.Single(); + + _out.WriteLine($"guarded row has loader: {guarded.HasLoader}"); + + Assert.DoesNotContain(guarded.Posts, post => post.Draft == "async-draft"); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewLeakTests3.cs b/DynamicWhere.Tests/Policies/ReviewLeakTests3.cs new file mode 100644 index 0000000..a0420f5 --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewLeakTests3.cs @@ -0,0 +1,174 @@ +using System.Collections; +using System.Reflection; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; + +namespace DynamicWhere.Tests.Policies +{ + // ============================================================================ round 3, batch 3: models + + /// An application collection that implements only the non-generic IEnumerable. + public sealed class ZrLooseBag : IEnumerable + { + private readonly List _items = new(); + + public void Add(object item) => _items.Add(item); + + public IEnumerator GetEnumerator() => _items.GetEnumerator(); + } + + public class ZrVoucher + { + public string Label { get; set; } = string.Empty; + + [DwDenied] + public string? Code { get; set; } + } + + public class ZrWallet + { + public int Id { get; set; } + + public string Owner { get; set; } = string.Empty; + + public ZrLooseBag Vouchers { get; set; } = new(); + } + + // ============================================================================ round 3, batch 3: tests + + public sealed class ReviewLeakTests3 : IDisposable + { + private readonly SqliteConnection _connection; + private readonly ZrProbeContext _db; + + public ReviewLeakTests3() + { + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + + _db = new ZrProbeContext(_connection); + _db.Database.EnsureCreated(); + + _db.Loans.Add(new ZrLoan { Note = "gen", Org = new ZrGenOrg { Name = "G", Secret = "generic-entity-secret" } }); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static PolicyQueryable Guard(IQueryable source, DwTier tier = DwTier.Strict, params IDwPolicyProvider[] more) + where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = tier, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() }.Concat(more).ToArray())); + + private static Filter Selecting(params string[] fields) => new() { Selects = fields.ToList() }; + + private static bool Holds(object? value, string text) + { + HashSet seen = new(ReferenceEqualityComparer.Instance); + Stack pending = new(); + + pending.Push(value); + + while (pending.Count > 0) + { + object? current = pending.Pop(); + + if (current is null || current is ValueType || current is MemberInfo) + { + continue; + } + + if (current is string held) + { + if (held == text) + { + return true; + } + + continue; + } + + if (!seen.Add(current)) + { + continue; + } + + if (current is IEnumerable items) + { + foreach (object? item in items) + { + pending.Push(item); + } + + continue; + } + + foreach (PropertyInfo property in current.GetType().GetProperties()) + { + if (property.GetIndexParameters().Length != 0 || !property.CanRead) + { + continue; + } + + try + { + pending.Push(property.GetValue(current)); + } + catch + { + // A getter that throws holds nothing readable. + } + } + } + + return false; + } + + private static object? SafeRead(Func read) + { + try + { + return read(); + } + catch (PolicyException refusal) when (refusal.ErrorCode == PolicyErrorCode.FieldDeniedForSelect) + { + return null; + } + } + + // ------------------------------------------------------------------------------------------------ P3e + + /// + /// An entity query re-rooted by Select reads no model, so its derived types come from the subtype search, + /// which never lists a generic one. + /// + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P3e_an_entity_query_re_rooted_by_Select_holding_a_closed_generic_derived_entity(DwTier tier) + { + IQueryable source = _db.Loans.Select(l => l.Org!); + + Assert.True(Holds(source.AsNoTracking().ToList(), "generic-entity-secret")); + + Assert.False(Holds(SafeRead(() => Guard(source, tier).ToList(new Filter()).Data), "generic-entity-secret")); + } + + // ------------------------------------------------------------------------------------------------ P13 + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewLeakTests4.cs b/DynamicWhere.Tests/Policies/ReviewLeakTests4.cs new file mode 100644 index 0000000..84c30bd --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewLeakTests4.cs @@ -0,0 +1,142 @@ +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; + +namespace DynamicWhere.Tests.Policies +{ + // ---- P10b: an interface declares the member; the implementing class denies it ------------------------- + + public interface IZrAccount + { + int Id { get; } + + string Holder { get; } + + string? Iban { get; } + } + + public class ZrAccountEntity : IZrAccount + { + public int Id { get; set; } + + public string Holder { get; set; } = string.Empty; + + [DwDenied] + public string? Iban { get; set; } + } + + public class ZrStatement + { + public int Id { get; set; } + + public IZrAccount? Account { get; set; } + } + + public sealed class ZrProbeContext4 : DbContext + { + private readonly SqliteConnection _connection; + + public ZrProbeContext4(SqliteConnection connection) => _connection = connection; + + public DbSet Accounts => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + } + + public sealed class ReviewLeakTests4 : IDisposable + { + private readonly SqliteConnection _connection; + private readonly ZrProbeContext4 _db; + + public ReviewLeakTests4() + { + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + _db = new ZrProbeContext4(_connection); + _db.Database.EnsureCreated(); + _db.Accounts.Add(new ZrAccountEntity { Holder = "H", Iban = "iban-secret" }); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static PolicyQueryable Guard(IQueryable source, DwTier tier = DwTier.Strict) + where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = tier, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + private static bool Leaks(Func> read) + { + try + { + return read().Any(row => row is ZrAccountEntity { Iban: "iban-secret" } + || row is ZrStatement { Account: ZrAccountEntity { Iban: "iban-secret" } }); + } + catch (PolicyException refusal) when (refusal.ErrorCode is PolicyErrorCode.FieldDeniedForSelect or PolicyErrorCode.AllSelectsDenied) + { + return false; + } + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P10b_rows_in_memory_read_through_the_interface_that_declares_the_denied_member(DwTier tier) + { + IZrAccount[] rows = { new ZrAccountEntity { Id = 1, Holder = "H", Iban = "iban-secret" } }; + + Assert.False(Leaks(() => Guard(rows.AsQueryable(), tier).ToList(new Filter()).Data)); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P10b_an_entity_query_read_through_the_interface(DwTier tier) + { + IQueryable source = _db.Accounts; + + Exception? unguarded = Record.Exception(() => source.AsNoTracking().ToList()); + + if (unguarded is not null) + { + // EF Core cannot run the query through the interface at all. + return; + } + + Assert.False(Leaks(() => Guard(source, tier).ToList(new Filter()).Data)); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P10b_a_member_typed_as_the_interface(DwTier tier) + { + ZrStatement[] rows = { new() { Id = 1, Account = new ZrAccountEntity { Id = 2, Holder = "H", Iban = "iban-secret" } } }; + + Assert.False(Leaks(() => Guard(rows.AsQueryable(), tier).ToList(new Filter()).Data)); + Assert.False(Leaks(() => Guard(rows.AsQueryable(), tier).ToList(new Filter { Selects = new List { "Id", "Account" } }).Data)); + } + + [Fact] + public void Zr_P10b_control_the_class_itself_is_policed() + { + ZrAccountEntity[] rows = { new() { Id = 1, Holder = "H", Iban = "iban-secret" } }; + + Assert.False(Leaks(() => Guard(rows.AsQueryable()).ToList(new Filter()).Data)); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewLeakTests5.cs b/DynamicWhere.Tests/Policies/ReviewLeakTests5.cs new file mode 100644 index 0000000..920786e --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewLeakTests5.cs @@ -0,0 +1,121 @@ +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Xunit.Abstractions; + +namespace DynamicWhere.Tests.Policies +{ + // ---- P12b: an entity that takes the DbContext in its constructor and loads its navigation with it ----- + + public class ZrCtxBlog + { + private readonly ZrProbeContext5? _context; + private List? _posts; + + public ZrCtxBlog() + { + } + + private ZrCtxBlog(ZrProbeContext5 context) => _context = context; + + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + /// Loaded on first read through the context EF Core injected, the pattern EF Core documents. + public List Posts + { + get => _posts ??= _context?.Set().AsNoTracking().Where(p => p.ZrCtxBlogId == Id).ToList() + ?? new List(); + set => _posts = value; + } + } + + public class ZrCtxPost + { + public int Id { get; set; } + + public int ZrCtxBlogId { get; set; } + + public string Title { get; set; } = string.Empty; + + [DwDenied] + public string? Draft { get; set; } + } + + public sealed class ZrProbeContext5 : DbContext + { + private readonly SqliteConnection _connection; + + public ZrProbeContext5(SqliteConnection connection) => _connection = connection; + + public DbSet Blogs => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + protected override void OnModelCreating(ModelBuilder model) => + model.Entity().Navigation(b => b.Posts).HasField("_posts"); + } + + public sealed class ReviewLeakTests5 : IDisposable + { + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZrProbeContext5 _db; + + public ReviewLeakTests5(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + _db = new ZrProbeContext5(_connection); + _db.Database.EnsureCreated(); + _db.Blogs.Add(new ZrCtxBlog { Name = "B1", Posts = { new ZrCtxPost { Title = "T", Draft = "context-draft" } } }); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static PolicyQueryable Guard(IQueryable source, DwTier tier = DwTier.Strict) + where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = tier, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P12b_a_navigation_its_entity_loads_through_an_injected_DbContext(DwTier tier) + { + ZrCtxBlog unguarded = _db.Blogs.AsNoTracking().ToList().Single(); + + _out.WriteLine($"unguarded drafts: {string.Join(",", unguarded.Posts.Select(p => p.Draft))}"); + + ZrCtxBlog guarded; + + try + { + guarded = Guard(_db.Blogs, tier).ToList(new Filter()).Data.Single(); + } + catch (PolicyException refusal) when (refusal.ErrorCode == PolicyErrorCode.FieldDeniedForSelect) + { + return; + } + + Assert.DoesNotContain(guarded.Posts, post => post.Draft == "context-draft"); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewLeakTests6.cs b/DynamicWhere.Tests/Policies/ReviewLeakTests6.cs new file mode 100644 index 0000000..5c6ea69 --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewLeakTests6.cs @@ -0,0 +1,96 @@ +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.DTOs; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Xunit.Abstractions; + +namespace DynamicWhere.Tests.Policies +{ + // ---- P1c: a top-level denial forces the projection, and the owned member is still kept whole ----------- + + public class ZrClinic2 + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + [DwDenied] + public string? Notes { get; set; } + + public ZrContact Contact { get; set; } = new(); + } + + public sealed class ZrProbeContext6 : DbContext + { + private readonly SqliteConnection _connection; + + public ZrProbeContext6(SqliteConnection connection) => _connection = connection; + + public DbSet Clinics => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + protected override void OnModelCreating(ModelBuilder model) => + model.Entity().OwnsOne(c => c.Contact, o => o.Property("_phone")); + } + + public sealed class ReviewLeakTests6 : IDisposable + { + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZrProbeContext6 _db; + + public ReviewLeakTests6(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + _db = new ZrProbeContext6(_connection); + _db.Database.EnsureCreated(); + + ZrClinic2 clinic = new() { Name = "C1", Notes = "notes-secret", Contact = new ZrContact { City = "Basra" } }; + clinic.Contact.SetPhone("phone-secret"); + _db.Clinics.Add(clinic); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zr_P1c_an_owned_member_kept_whole_by_a_synthesized_projection(DwTier tier) + { + PolicyQueryable guarded = _db.Clinics.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = tier }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + ZrClinic2 row = guarded.ToList(new Filter()).Data.Single(); + + foreach (PolicyDecision decision in guarded.LastTrace!.Decisions) + { + _out.WriteLine($"trace: {decision.FieldPath} {decision.Feature} {decision.Action} {decision.Reason}"); + } + + // The projection ran: the top-level denial is withheld. + Assert.Null(row.Notes); + Assert.Equal("Basra", row.Contact.City); + + // And the owned member it kept whole carries the value its unmapped getter exposes. + Assert.Null(row.Contact.Phone.Number); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewOverBlockTests.cs b/DynamicWhere.Tests/Policies/ReviewOverBlockTests.cs new file mode 100644 index 0000000..adcfc26 --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewOverBlockTests.cs @@ -0,0 +1,1014 @@ +using System.Collections; +using System.Text.Json; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Classes.Core; +using DynamicWhere.ex.Enums; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.DTOs; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Xunit.Abstractions; +using Zb.Geo; + +// Probes for over-blocking in 6c9e171. Every test asserts what an ordinary query should get when +// nothing denied can reach the result (or what 3.1.0 / f7e1cc6 returned), so a failing test is a +// candidate over-block. Each test also writes its outcome, run with a detailed console logger. + +namespace Zb.Geo +{ + /// + /// Shaped like NetTopologySuite's Geometry: an application-namespace type with an object member + /// (NTS has Geometry.UserData) and subtypes. + /// + public abstract class ZbGeometry + { + public int Srid { get; set; } + + public object? UserData { get; set; } + } + + public class ZbPoint : ZbGeometry + { + public double X { get; set; } + + public double Y { get; set; } + } + + public class ZbPolygon : ZbGeometry + { + public List Shell { get; set; } = new(); + } +} + +namespace DynamicWhere.Tests.Policies +{ + // ------------------------------------------------------------ columns holding objects, no denials + + public class ZbTicket + { + public int Id { get; set; } + + public string Title { get; set; } = string.Empty; + + public Dictionary Extra { get; set; } = new(); + } + + public class ZbComment + { + public int Id { get; set; } + + public string Text { get; set; } = string.Empty; + + public int ZbTicketId { get; set; } + + public ZbTicket? Ticket { get; set; } + } + + public class ZbSensor + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public BitArray Flags { get; set; } = new(4); + } + + public class ZbSite + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public ZbPoint? Location { get; set; } + } + + public class ZbVisit + { + public int Id { get; set; } + + public string Note { get; set; } = string.Empty; + + public int ZbSiteId { get; set; } + + public ZbSite? Site { get; set; } + } + + public class ZbProfileHolder + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public ZbSettings Settings { get; set; } = new(); + } + + public class ZbSettings + { + public string Theme { get; set; } = string.Empty; + + public Dictionary Prefs { get; set; } = new(); + } + + // ------------------------------------------------------------ reshaped chains + + public class ZbCustomer + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public List Cards { get; set; } = new(); + + public List Orders { get; set; } = new(); + } + + public class ZbCard + { + public int Id { get; set; } + + public int ZbCustomerId { get; set; } + + public string Label { get; set; } = string.Empty; + + [DwDenied] + public string? Pan { get; set; } + } + + public class ZbOrder + { + public int Id { get; set; } + + public string Code { get; set; } = string.Empty; + + public int ZbCustomerId { get; set; } + + public ZbCustomer? Customer { get; set; } + + public List Lines { get; set; } = new(); + } + + public class ZbLine + { + public int Id { get; set; } + + public int ZbOrderId { get; set; } + + public string Sku { get; set; } = string.Empty; + + public string? Cost { get; set; } + } + + /// A constructor-bound entity: EF Core binds it, and it has no parameterless constructor. + public class ZbMember + { + public ZbMember(int id, string name) + { + Id = id; + Name = name; + } + + public int Id { get; private set; } + + public string Name { get; private set; } + + public List Badges { get; private set; } = new(); + } + + public class ZbBadge + { + public int Id { get; set; } + + public int ZbMemberId { get; set; } + + public string Label { get; set; } = string.Empty; + + [DwDenied] + public string? Pin { get; set; } + } + + public class ZbLink + { + public int Id { get; set; } + + public int ZbMemberId { get; set; } + + public ZbMember? Member { get; set; } + } + + /// A shopper whose tier is auto-included and holds nothing denied; a denial sits beneath unloaded wallets. + public class ZbShopper + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public int ZbTierId { get; set; } + + public ZbTier? Tier { get; set; } + + public List Wallets { get; set; } = new(); + } + + public class ZbTier + { + public int Id { get; set; } + + public string Label { get; set; } = string.Empty; + } + + public class ZbWallet + { + public int Id { get; set; } + + public int ZbShopperId { get; set; } + + [DwDenied] + public string? Iban { get; set; } + } + + public class ZbBasket + { + public int Id { get; set; } + + public int ZbShopperId { get; set; } + + public ZbShopper? Shopper { get; set; } + } + + // ------------------------------------------------------------ hierarchies + + public abstract class ZbPayment + { + public int Id { get; set; } + + public long Cents { get; set; } + } + + public class ZbCardPayment : ZbPayment + { + [DwDenied] + public string? Pan { get; set; } + } + + public class ZbCashPayment : ZbPayment + { + public string Till { get; set; } = string.Empty; + } + + /// An abstract root; no denial anywhere, one derived type holds a JSON bag. + public abstract class ZbEvent + { + public int Id { get; set; } + + public string Kind { get; set; } = string.Empty; + } + + public class ZbClickEvent : ZbEvent + { + public Dictionary Data { get; set; } = new(); + } + + public class ZbViewEvent : ZbEvent + { + public string Page { get; set; } = string.Empty; + } + + /// A concrete root; no denial anywhere, one derived type holds a JSON bag. + public class ZbDoc + { + public int Id { get; set; } + + public string Title { get; set; } = string.Empty; + } + + public class ZbRichDoc : ZbDoc + { + public Dictionary Meta { get; set; } = new(); + } + + // ------------------------------------------------------------ rows + + public interface IZbContact + { + string Name { get; set; } + } + + public class ZbContactRow : IZbContact + { + public string Name { get; set; } = string.Empty; + } + + /// Another implementor of the interface, somewhere in the application, with a denial. + public class ZbVipContact : IZbContact + { + public string Name { get; set; } = string.Empty; + + [DwDenied] + public string? Phone { get; set; } + } + + public class ZbCustomerRow + { + public int Id { get; set; } + + public IZbContact? Contact { get; set; } + } + + public class ZbPersonDto + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public ZbAddressDto? Address { get; set; } + } + + /// A subclass of the DTO elsewhere in the application, with a denial. + public class ZbStaffDto : ZbPersonDto + { + [DwDenied] + public string? Salary { get; set; } + } + + public class ZbAddressDto + { + public string City { get; set; } = string.Empty; + } + + public class ZbOrderRow + { + public ZbOrderRow() + { + } + + public ZbOrderRow(int id) => Id = id; + + public int Id { get; set; } + + public string Code { get; set; } = string.Empty; + + public List Lines { get; set; } = new(); + } + + public class ZbLineRow + { + public string Sku { get; set; } = string.Empty; + + [DwDenied] + public string? Cost { get; set; } + } + + public class ZbPlaceDto + { + public string City { get; set; } = string.Empty; + + public string Display => City.ToUpperInvariant(); + } + + public class ZbPlaceRow + { + public int Id { get; set; } + + public ZbPlaceDto? Place { get; set; } + } + + public class ZbZoneDto + { + public string Code { get; set; } = string.Empty; + } + + /// A subclass with no denial at all. + public class ZbSubZoneDto : ZbZoneDto + { + public string Parent { get; set; } = string.Empty; + } + + public class ZbZoneRow + { + public int Id { get; set; } + + public ZbZoneDto? Zone { get; set; } + } + + // ------------------------------------------------------------ the database + + public sealed class ZbProbeContext : DbContext + { + private readonly SqliteConnection _connection; + + public ZbProbeContext(SqliteConnection connection) => _connection = connection; + + public DbSet Tickets => Set(); + + public DbSet Comments => Set(); + + public DbSet Sensors => Set(); + + public DbSet Sites => Set(); + + public DbSet Visits => Set(); + + public DbSet ProfileHolders => Set(); + + public DbSet Customers => Set(); + + public DbSet Orders => Set(); + + public DbSet Members => Set(); + + public DbSet Links => Set(); + + public DbSet Payments => Set(); + + public DbSet Events => Set(); + + public DbSet Docs => Set(); + + public DbSet Shoppers => Set(); + + public DbSet Baskets => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + protected override void OnModelCreating(ModelBuilder model) + { + model.Entity().Property(t => t.Extra).HasConversion( + value => JsonSerializer.Serialize(value, (JsonSerializerOptions?)null), + text => JsonSerializer.Deserialize>(text, (JsonSerializerOptions?)null)!); + + model.Entity().Property(s => s.Flags).HasConversion( + value => string.Concat(value.Cast().Select(bit => bit ? '1' : '0')), + text => new BitArray(text.Select(c => c == '1').ToArray())); + + model.Entity().Property(s => s.Location).HasConversion( + value => value == null ? null : $"{value.X};{value.Y}", + text => text == null ? null : new ZbPoint { X = double.Parse(text.Split(';', StringSplitOptions.None)[0]), Y = double.Parse(text.Split(';', StringSplitOptions.None)[1]) }); + + model.Entity().OwnsOne(p => p.Settings, s => s.Property(x => x.Prefs).HasConversion( + value => JsonSerializer.Serialize(value, (JsonSerializerOptions?)null), + text => JsonSerializer.Deserialize>(text, (JsonSerializerOptions?)null)!)); + + model.Entity().HasMany(m => m.Badges).WithOne().HasForeignKey(b => b.ZbMemberId); + + model.Entity(); + model.Entity(); + + model.Entity().Property(e => e.Data).HasConversion( + value => JsonSerializer.Serialize(value, (JsonSerializerOptions?)null), + text => JsonSerializer.Deserialize>(text, (JsonSerializerOptions?)null)!); + model.Entity(); + + model.Entity().Navigation(s => s.Tier).AutoInclude(); + + model.Entity().Property(d => d.Meta).HasConversion( + value => JsonSerializer.Serialize(value, (JsonSerializerOptions?)null), + text => JsonSerializer.Deserialize>(text, (JsonSerializerOptions?)null)!); + } + } + + // ------------------------------------------------------------ the probes + + /// Appends a probe's outcome to a file named for the tree and the EF Core version it ran on. + internal static class ZbProbeLog + { + private static readonly object Gate = new(); + + internal static string Tag + { + get + { + string where = AppContext.BaseDirectory; + string tree = where.Contains("base-8a8d7fd") ? "base-8a8d7fd" + : where.Contains("parent-f7e1cc6") ? "parent-f7e1cc6" + : where.Contains("head-fce9c16") ? "head-fce9c16" + : "6c9e171"; + + return $"{tree}-ef{typeof(DbContext).Assembly.GetName().Version!.Major}"; + } + } + + internal static void Write(string line) + { + string directory = "/private/tmp/claude-501/-Users-sajadh92-Developer-Project-DynamicWhere-ex/8f118b4d-a861-42ce-b1d4-baa9f4b933e3/scratchpad/probes-b3"; + + if (!Directory.Exists(directory)) + { + return; + } + + lock (Gate) + { + File.AppendAllText(Path.Combine(directory, $"outcomes-{Tag}.txt"), line + Environment.NewLine); + } + } + } + + public sealed class ReviewOverBlockTests : IDisposable + { + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZbProbeContext _db; + + public ReviewOverBlockTests(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + + _db = new ZbProbeContext(_connection); + _db.Database.EnsureCreated(); + + ZbTicket ticket = new() { Title = "T1", Extra = { ["colour"] = "red" } }; + + _db.Tickets.Add(ticket); + _db.Comments.Add(new ZbComment { Text = "c1", Ticket = ticket }); + _db.Sensors.Add(new ZbSensor { Name = "S1", Flags = new BitArray(new[] { true, false, true, true }) }); + + ZbSite site = new() { Name = "Basra", Location = new ZbPoint { X = 30.5, Y = 47.8 } }; + + _db.Sites.Add(site); + _db.Visits.Add(new ZbVisit { Note = "v1", Site = site }); + _db.ProfileHolders.Add(new ZbProfileHolder { Name = "P1", Settings = new ZbSettings { Theme = "dark", Prefs = { ["lang"] = "ar" } } }); + + ZbCustomer withOrders = new() + { + Name = "C1", + Cards = { new ZbCard { Label = "visa", Pan = "pan-secret" } }, + Orders = { new ZbOrder { Code = "O1", Lines = { new ZbLine { Sku = "S1", Cost = "9" } } } } + }; + + _db.Customers.Add(withOrders); + _db.Customers.Add(new ZbCustomer { Name = "C2-no-orders" }); + + ZbMember member = new(0, "M1"); + + member.Badges.Add(new ZbBadge { Label = "gold", Pin = "pin-secret" }); + _db.Members.Add(member); + _db.Links.Add(new ZbLink { Member = member }); + + _db.Payments.Add(new ZbCardPayment { Cents = 100, Pan = "payment-pan-secret" }); + _db.Payments.Add(new ZbCashPayment { Cents = 200, Till = "till-1" }); + + _db.Events.Add(new ZbClickEvent { Kind = "click", Data = { ["x"] = 1 } }); + _db.Events.Add(new ZbViewEvent { Kind = "view", Page = "/home" }); + + ZbShopper shopper = new() { Name = "S1", Tier = new ZbTier { Label = "gold" }, Wallets = { new ZbWallet { Iban = "iban-secret" } } }; + + _db.Shoppers.Add(shopper); + _db.Baskets.Add(new ZbBasket { Shopper = shopper }); + + _db.Docs.Add(new ZbDoc { Title = "plain" }); + _db.Docs.Add(new ZbRichDoc { Title = "rich", Meta = { ["pages"] = 3 } }); + + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static DwPolicyContext Caller() => + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1").WithValue("TenantId", 1); + + private static PolicyQueryable Guard(IQueryable source, DwTier tier = DwTier.Strict, params IDwPolicyProvider[] more) + where T : class => + source.ApplyPolicy( + Caller(), + new DwPolicyOptions { Tier = tier, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() }.Concat(more).ToArray())); + + private static FakePolicyProvider DenyingAllBut(params string[] allowed) + { + FakePolicyProvider rules = new FakePolicyProvider() + .Add("*", PolicyFeature.Select, PolicyEffect.Deny, PolicyLevel.DynamicGlobal); + + foreach (string path in allowed) + { + rules.Add(path, PolicyFeature.Select, PolicyEffect.Allow, PolicyLevel.DynamicGlobal); + } + + return rules; + } + + private static string Dropped(PolicyQueryable guarded) where T : class => + string.Join(" | ", guarded.LastTrace?.Decisions + .Where(d => d.Action != PolicyAction.Allowed) + .Select(d => $"{d.FieldPath} {d.Action}: {d.Reason}") ?? Array.Empty()); + + private static bool AnyDropped(PolicyQueryable guarded) where T : class => + guarded.LastTrace?.Decisions.Any(d => d.Action == PolicyAction.Dropped) ?? false; + + /// Runs a guarded call and describes what came back, or what it threw. + private string Outcome(string label, PolicyQueryable guarded, Func, IEnumerable> run) + where T : class + { + string line; + + try + { + List rows = run(guarded).Cast().ToList(); + + line = $"{label}: OK rows={rows.Count} types=[{string.Join(",", rows.Select(r => r?.GetType().Name ?? "null"))}]" + + $" json={JsonSerializer.Serialize(rows)} decisions=[{Dropped(guarded)}]"; + } + catch (Exception e) + { + string code = e is PolicyException refusal ? refusal.ErrorCode.ToString() : string.Empty; + + line = $"{label}: THREW {e.GetType().Name} {code} {e.Message.Split('\n')[0]} decisions=[{Dropped(guarded)}]"; + } + + _out.WriteLine(line); + ZbProbeLog.Write(line); + + return line; + } + + // ================================================================ A: columns that hold objects + + [Fact] + public void Zb_A1_entity_with_an_object_bag_column_and_no_denial_is_returned_as_loaded() + { + PolicyQueryable guarded = Guard(_db.Tickets); + + Outcome("A1 typed", Guard(_db.Tickets), g => g.ToList(new Filter()).Data); + + ZbTicket ticket = guarded.ToList(new Filter()).Data.Single(); + + Assert.False(AnyDropped(guarded), Dropped(guarded)); + Assert.True(ticket.Extra.ContainsKey("colour")); + } + + [Fact] + public void Zb_A2_entity_with_a_BitArray_column_and_no_denial_is_returned_as_loaded() + { + PolicyQueryable guarded = Guard(_db.Sensors); + + Outcome("A2 typed", Guard(_db.Sensors), g => g.ToList(new Filter()).Data); + + ZbSensor sensor = guarded.ToList(new Filter()).Data.Single(); + + Assert.False(AnyDropped(guarded), Dropped(guarded)); + Assert.Equal(4, sensor.Flags.Length); + Assert.True(sensor.Flags[3]); + } + + [Fact] + public void Zb_A3_entity_with_a_geometry_like_column_and_no_denial_is_returned_as_loaded() + { + PolicyQueryable guarded = Guard(_db.Sites); + + Outcome("A3 typed", Guard(_db.Sites), g => g.ToList(new Filter()).Data); + + ZbSite site = guarded.ToList(new Filter()).Data.Single(); + + Assert.False(AnyDropped(guarded), Dropped(guarded)); + Assert.NotNull(site.Location); + } + + [Fact] + public void Zb_A4_an_included_navigation_whose_entity_holds_an_object_bag_is_kept() + { + PolicyQueryable guarded = Guard(_db.Comments.Include(c => c.Ticket)); + + Outcome("A4 typed", Guard(_db.Comments.Include(c => c.Ticket)), g => g.ToList(new Filter()).Data); + + ZbComment comment = guarded.ToList(new Filter()).Data.Single(); + + Assert.False(AnyDropped(guarded), Dropped(guarded)); + Assert.NotNull(comment.Ticket); + } + + [Fact] + public void Zb_A5_an_included_navigation_whose_entity_holds_a_geometry_is_kept() + { + PolicyQueryable guarded = Guard(_db.Visits.Include(v => v.Site)); + + Outcome("A5 typed", Guard(_db.Visits.Include(v => v.Site)), g => g.ToList(new Filter()).Data); + + ZbVisit visit = guarded.ToList(new Filter()).Data.Single(); + + Assert.False(AnyDropped(guarded), Dropped(guarded)); + Assert.NotNull(visit.Site); + } + + [Fact] + public void Zb_A6_an_owned_member_holding_an_object_bag_is_kept() + { + PolicyQueryable guarded = Guard(_db.ProfileHolders); + + Outcome("A6 typed", Guard(_db.ProfileHolders), g => g.ToList(new Filter()).Data); + + ZbProfileHolder holder = guarded.ToList(new Filter()).Data.Single(); + + Assert.False(AnyDropped(guarded), Dropped(guarded)); + Assert.True(holder.Settings.Prefs.ContainsKey("lang")); + } + + // ================================================================ B: reshaped chains, nothing loaded beneath + + [Fact] + public void Zb_B1_Select_to_a_navigation_with_nothing_loaded_beneath_is_not_projected() + { + IQueryable source = _db.Orders.Select(o => o.Customer!); + + Outcome("B1 unguarded", Guard(_db.Customers.Where(c => false)), _ => source.ToList()); + Outcome("B1 typed", Guard(source), g => g.ToList(new Filter()).Data); + _db.ChangeTracker.Clear(); + + PolicyQueryable guarded = Guard(source); + ZbCustomer customer = guarded.ToList(new Filter()).Data.Single(); + + Assert.Equal("C1", customer.Name); + Assert.False(AnyDropped(guarded), Dropped(guarded)); + } + + [Fact] + public void Zb_B2_SelectMany_with_nothing_loaded_beneath_is_not_projected() + { + IQueryable source = _db.Customers.SelectMany(c => c.Orders); + + Outcome("B2 typed", Guard(source), g => g.ToList(new Filter()).Data); + _db.ChangeTracker.Clear(); + + PolicyQueryable guarded = Guard(source); + + Assert.Single(guarded.ToList(new Filter()).Data); + Assert.False(AnyDropped(guarded), Dropped(guarded)); + } + + [Fact] + public void Zb_B3_Join_with_nothing_loaded_beneath_is_not_projected() + { + IQueryable source = _db.Orders.Join(_db.Customers, o => o.ZbCustomerId, c => c.Id, (o, c) => c); + + Outcome("B3 typed", Guard(source), g => g.ToList(new Filter()).Data); + + PolicyQueryable guarded = Guard(source); + + Assert.Single(guarded.ToList(new Filter()).Data); + Assert.False(AnyDropped(guarded), Dropped(guarded)); + } + + [Fact] + public void Zb_B4_GroupBy_First_with_nothing_loaded_beneath_runs_unprojected() + { + // EF Core 6 cannot count this shape, guarded or not; the core's count fails there before the policy has a say. + if (typeof(DbContext).Assembly.GetName().Version!.Major < 7) + { + return; + } + + IQueryable source = _db.Orders.GroupBy(o => o.ZbCustomerId).Select(g => g.OrderBy(o => o.Id).First()); + + Outcome("B4 unguarded", Guard(_db.Orders.Where(o => false)), _ => source.ToList()); + Outcome("B4 typed", Guard(source), g => g.ToList(new Filter()).Data); + Outcome("B4 typed paged", Guard(source), g => g.ToList(new Filter { Page = new PageBy { PageNumber = 1, PageSize = 10 } }).Data); + + PolicyQueryable guarded = Guard(source); + + Assert.Single(guarded.ToList(new Filter()).Data); + Assert.False(AnyDropped(guarded), Dropped(guarded)); + } + + [Fact] + public void Zb_B5_Select_to_a_constructor_bound_entity_with_nothing_loaded_beneath_runs() + { + IQueryable source = _db.Links.Select(l => l.Member!); + + Outcome("B5 unguarded", Guard(_db.Members.Where(m => false)), _ => source.ToList()); + Outcome("B5 typed", Guard(source), g => g.ToList(new Filter()).Data); + Outcome("B5 dynamic", Guard(source), g => g.ToListDynamic(new Filter()).Data); + Outcome("B5 direct typed (no reshape)", Guard(_db.Members), g => g.ToList(new Filter()).Data); + + Assert.Equal("M1", Guard(source).ToList(new Filter()).Data.Single().Name); + } + + [Fact] + public void Zb_B6_Select_FirstOrDefault_with_a_missing_row_runs_as_it_does_unguarded() + { + IQueryable source = _db.Customers.OrderBy(c => c.Id).Select(c => c.Orders.OrderBy(o => o.Id).FirstOrDefault()!); + + Outcome("B6 unguarded", Guard(_db.Orders.Where(o => false)), _ => source.ToList()); + Outcome("B6 typed", Guard(source), g => g.ToList(new Filter()).Data); + + Assert.Equal(2, Guard(source).ToList(new Filter()).Data.Count); + } + + [Fact] + public void Zb_B7_left_join_through_DefaultIfEmpty_runs_as_it_does_unguarded() + { + IQueryable source = _db.Customers.SelectMany(c => c.Orders.DefaultIfEmpty(), (c, o) => o!); + + Outcome("B7 unguarded", Guard(_db.Orders.Where(o => false)), _ => source.ToList()); + Outcome("B7 typed", Guard(source), g => g.ToList(new Filter()).Data); + + Assert.Equal(2, Guard(source).ToList(new Filter()).Data.Count); + } + + /// + /// The needless projection over a reshaped chain drops an auto-included navigation that holds nothing + /// denied: only the unloaded wallets hold a denial. + /// + [Fact] + public void Zb_B9_Select_to_a_navigation_keeps_its_auto_included_navigation() + { + IQueryable source = _db.Baskets.Select(b => b.Shopper!); + + Outcome("B9 unguarded", Guard(_db.Shoppers.Where(s => false)), _ => source.AsNoTracking().ToList()); + Outcome("B9 typed", Guard(source), g => g.ToList(new Filter()).Data); + Outcome("B9 direct (no reshape)", Guard(_db.Shoppers), g => g.ToList(new Filter()).Data); + + Assert.Equal("gold", Guard(_db.Shoppers).ToList(new Filter()).Data.Single().Tier?.Label); + Assert.Equal("gold", Guard(source).ToList(new Filter()).Data.Single().Tier?.Label); + } + + /// + /// Informational: the same chains with a top-level denial, which forces the synthesized projection in + /// every version. Shows whether the exceptions above are the projection's own fragility. + /// + [Fact] + public void Zb_B8_the_same_chains_with_a_top_level_denial_in_every_version() + { + FakePolicyProvider code = new FakePolicyProvider().Add("Code", PolicyFeature.Select, PolicyEffect.Deny, PolicyLevel.DynamicGlobal); + + Outcome("B8 FirstOrDefault + top-level denial", Guard(_db.Customers.OrderBy(c => c.Id).Select(c => c.Orders.OrderBy(o => o.Id).FirstOrDefault()!), DwTier.Strict, code), g => g.ToList(new Filter()).Data); + Outcome("B8 DefaultIfEmpty + top-level denial", Guard(_db.Customers.SelectMany(c => c.Orders.DefaultIfEmpty(), (c, o) => o!), DwTier.Strict, code), g => g.ToList(new Filter()).Data); + Outcome("B8 GroupBy First + top-level denial", Guard(_db.Orders.GroupBy(o => o.ZbCustomerId).Select(g => g.OrderBy(o => o.Id).First()), DwTier.Strict, code), g => g.ToList(new Filter()).Data); + } + + // ================================================================ C: hierarchies + + [Fact] + public void Zb_C1_an_abstract_root_with_no_denial_anywhere_runs() + { + Outcome("C1 typed", Guard(_db.Events), g => g.ToList(new Filter()).Data); + Outcome("C1 dynamic", Guard(_db.Events), g => g.ToListDynamic(new Filter()).Data); + + List events = Guard(_db.Events).ToList(new Filter()).Data; + + Assert.Contains(events, e => e is ZbClickEvent click && click.Data.ContainsKey("x")); + } + + [Fact] + public void Zb_C2_a_concrete_root_with_no_denial_anywhere_keeps_its_derived_rows() + { + PolicyQueryable guarded = Guard(_db.Docs); + + Outcome("C2 typed", Guard(_db.Docs), g => g.ToList(new Filter()).Data); + + List docs = guarded.ToList(new Filter()).Data; + + Assert.False(AnyDropped(guarded), Dropped(guarded)); + Assert.Contains(docs, d => d is ZbRichDoc rich && rich.Meta.ContainsKey("pages")); + } + + /// Informational: an abstract root whose derived type declares a denied field. + [Fact] + public void Zb_C3_an_abstract_root_whose_derived_type_has_a_denial() + { + string typed = Outcome("C3 typed", Guard(_db.Payments), g => g.ToList(new Filter()).Data); + string dynamic = Outcome("C3 dynamic", Guard(_db.Payments), g => g.ToListDynamic(new Filter()).Data); + string convenience = Outcome("C3 typed convenience", Guard(_db.Payments, DwTier.Convenience), g => g.ToList(new Filter()).Data); + + Assert.DoesNotContain("THREW", dynamic); + } + + // ================================================================ D: interfaces and DTO base classes + + [Fact] + public void Zb_D1_a_projected_member_declared_as_an_interface_keeps_the_value_it_was_built_with() + { + IQueryable rows = _db.Customers.OrderBy(c => c.Id) + .Select(c => new ZbCustomerRow { Id = c.Id, Contact = new ZbContactRow { Name = c.Name } }); + + Outcome("D1 typed", Guard(rows), g => g.ToList(new Filter()).Data); + + PolicyQueryable guarded = Guard(rows); + ZbCustomerRow row = guarded.ToList(new Filter()).Data.First(); + + Assert.Equal("C1", row.Contact?.Name); + Assert.False(AnyDropped(guarded), Dropped(guarded)); + } + + [Fact] + public void Zb_D2_an_in_memory_member_declared_as_an_interface_whose_other_implementation_is_denied_is_left_out() + { + ZbCustomerRow[] rows = { new() { Id = 1, Contact = new ZbContactRow { Name = "n1" } } }; + + Outcome("D2 typed", Guard(rows.AsQueryable()), g => g.ToList(new Filter()).Data); + + PolicyQueryable guarded = Guard(rows.AsQueryable()); + ZbCustomerRow row = guarded.ToList(new Filter()).Data.Single(); + + // In memory the member can hold any implementation, one with a denied field among them, and an + // object of a row in memory is never kept once a projection is needed. + Assert.Null(row.Contact); + Assert.True(AnyDropped(guarded), Dropped(guarded)); + } + + [Fact] + public void Zb_D3_in_memory_rows_of_a_DTO_a_subclass_of_which_has_a_denial_are_projected() + { + ZbPersonDto[] rows = { new() { Id = 1, Name = "n1", Address = new ZbAddressDto { City = "Basra" } } }; + + Outcome("D3 typed", Guard(rows.AsQueryable()), g => g.ToList(new Filter()).Data); + + PolicyQueryable guarded = Guard(rows.AsQueryable()); + ZbPersonDto row = guarded.ToList(new Filter()).Data.Single(); + + // A loaded subclass declares a denied field, and a row in memory can be one: the rows are projected + // to the declared type, which leaves their objects out. + Assert.NotSame(rows[0], row); + Assert.Null(row.Address); + Assert.Equal("n1", row.Name); + } + + // ================================================================ E: constructor plus initializer + + [Fact] + public void Zb_E1_an_initializer_after_a_constructor_still_narrows_the_members_it_assigns() + { + IQueryable rows = _db.Orders.Select(o => new ZbOrderRow(o.Id) + { + Code = o.Code, + Lines = o.Lines.Select(l => new ZbLineRow { Sku = l.Sku, Cost = l.Cost }).ToList() + }); + + Outcome("E1 typed", Guard(rows), g => g.ToList(new Filter()).Data); + + ZbOrderRow row = Guard(rows).ToList(new Filter()).Data.Single(); + + Assert.Equal("O1", row.Code); + Assert.Equal("S1", Assert.Single(row.Lines).Sku); + Assert.Null(row.Lines[0].Cost); + } + + // ================================================================ F: deny-by-default, every path allowed + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zb_F1_deny_by_default_allowing_every_path_keeps_an_in_memory_member_with_a_computed_property(DwTier tier) + { + ZbPlaceRow[] rows = { new() { Id = 1, Place = new ZbPlaceDto { City = "Basra" } } }; + FakePolicyProvider rules = DenyingAllBut("Id", "Place", "Place.City", "Place.Display"); + + Outcome($"F1 {tier} named", Guard(rows.AsQueryable(), tier, rules), g => g.ToList(new Filter { Selects = new List { "Id", "Place" } }).Data); + Outcome($"F1 {tier} none", Guard(rows.AsQueryable(), tier, rules), g => g.ToList(new Filter()).Data); + + ZbPlaceRow row = Guard(rows.AsQueryable(), tier, rules).ToList(new Filter { Selects = new List { "Id", "Place" } }).Data.Single(); + + Assert.Equal("Basra", row.Place?.City); + } + + [Fact] + public void Zb_F2_deny_by_default_allowing_every_path_keeps_a_projected_member_with_a_computed_property() + { + IQueryable rows = _db.Customers.OrderBy(c => c.Id).Select(c => new ZbPlaceRow { Id = c.Id, Place = new ZbPlaceDto { City = c.Name } }); + FakePolicyProvider rules = DenyingAllBut("Id", "Place", "Place.City", "Place.Display"); + + Outcome("F2 none", Guard(rows, DwTier.Strict, rules), g => g.ToList(new Filter()).Data); + Outcome("F2 named", Guard(rows, DwTier.Strict, rules), g => g.ToList(new Filter { Selects = new List { "Id", "Place" } }).Data); + + PolicyQueryable guarded = Guard(rows, DwTier.Strict, rules); + ZbPlaceRow row = guarded.ToList(new Filter()).Data.First(); + + Assert.Equal("C1", row.Place?.City); + + // Naming the member, every path of which the policy allows, is not refused. + ZbPlaceRow named = Guard(rows, DwTier.Strict, rules).ToList(new Filter { Selects = new List { "Id", "Place" } }).Data.First(); + + Assert.Equal("C1", named.Place?.City); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zb_F3_deny_by_default_keeps_a_member_only_when_every_subtype_path_is_named(DwTier tier) + { + ZbZoneRow[] rows = { new() { Id = 1, Zone = new ZbZoneDto { Code = "Z1" } } }; + FakePolicyProvider rules = DenyingAllBut("Id", "Zone", "Zone.Code"); + + Outcome($"F3 {tier} named", Guard(rows.AsQueryable(), tier, rules), g => g.ToList(new Filter { Selects = new List { "Id", "Zone" } }).Data); + + // The subclass declares Parent, which the policy does not name, and a row in memory can hold it. + Assert.Throws(() => Guard(rows.AsQueryable(), tier, rules).ToList(new Filter { Selects = new List { "Id", "Zone" } })); + + // Named too, every path a Zone can hold is granted, and the member is kept whole. + FakePolicyProvider named = DenyingAllBut("Id", "Zone", "Zone.Code", "Zone.Parent"); + ZbZoneRow row = Guard(rows.AsQueryable(), tier, named).ToList(new Filter { Selects = new List { "Id", "Zone" } }).Data.Single(); + + Assert.Equal("Z1", row.Zone?.Code); + } + } +} diff --git a/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs b/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs index c75de5a..402c241 100644 --- a/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs +++ b/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs @@ -6,6 +6,7 @@ using DynamicWhere.ex.Policies.Context; using DynamicWhere.ex.Policies.DTOs; using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Source; namespace DynamicWhere.ex.Policies.Resolution; @@ -13,7 +14,8 @@ namespace DynamicWhere.ex.Policies.Resolution; /// Produces policy fragments by reflecting over the attributes on a type. /// /// -/// The result is identical for every caller, so it is computed once per type and cached. An +/// The result is identical for every caller, so it is computed once per type and cached, until an +/// assembly that could declare a subtype loads (see the denials below). An /// attribute with Overridable = false lands at and /// nothing at runtime can replace it; one with Overridable = true lands at /// , the least authoritative level, and acts only as @@ -21,7 +23,7 @@ namespace DynamicWhere.ex.Policies.Resolution; /// public sealed class AttributePolicyProvider : IDwPolicyProvider { - private static readonly ConcurrentDictionary> Cache = new(); + private static readonly ConcurrentDictionary Fragments)> Cache = new(); /// /// How many navigation segments a generated field path may contain. Matches the default @@ -38,7 +40,19 @@ public IReadOnlyList GetFragments(Type entityType, DwPolicyConte throw new ArgumentNullException(nameof(entityType)); } - return Cache.GetOrAdd(entityType, Build); + int epoch = KnownSubtypes.Epoch; + + if (Cache.TryGetValue(entityType, out (int Epoch, IReadOnlyList Fragments) known) + && known.Epoch == epoch) + { + return known.Fragments; + } + + IReadOnlyList fragments = Build(entityType); + + Cache[entityType] = (epoch, fragments); + + return fragments; } /// @@ -107,6 +121,11 @@ private static void Walk( fragments.Add(ToFragment(path, attribute)); } + foreach (DwDenyAttribute attribute in DenialsElsewhere(type, property)) + { + fragments.Add(ToFragment(path, attribute)); + } + foreach (DwOperatorsAttribute attribute in property.GetCustomAttributes(inherit: true)) { fragments.Add(ToFragment(path, attribute)); @@ -180,6 +199,137 @@ private static void Walk( } } + /// + /// The denials a member carries from another declaration of it: the interface member it implements, + /// and, in a type loaded below the one walked, the override or the implementation that a row of that + /// type runs. + /// + /// + /// Attributes are read from the declaration walked, and inheritance only reaches up the chain. A row + /// read through a base type or an interface is still the subtype it is, though, and its member returns + /// what the subtype's declaration returns: a [DwDenied] on override Code, or on the + /// class's implementation of IAccount.Iban, was never seen through the base path, which filtered, + /// sorted, grouped and returned it. Each such denial applies to the path for every row, since a + /// projection cannot withhold a field from some rows only. + /// + internal static IEnumerable DenialsElsewhere(Type type, PropertyInfo property) + { + if (property.GetGetMethod() is not { } getter) + { + yield break; + } + + if (!type.IsInterface) + { + foreach (Type contract in type.GetInterfaces()) + { + if (Declaration(Implemented(type, contract, getter), contract) is { } declared) + { + foreach (DwDenyAttribute attribute in declared.GetCustomAttributes(inherit: false)) + { + yield return attribute; + } + } + } + } + + foreach (Type subtype in KnownSubtypes.Of(type)) + { + PropertyInfo? below = type.IsInterface + ? Implementation(subtype, type, getter) + : Override(subtype, getter); + + if (below is null) + { + continue; + } + + foreach (DwDenyAttribute attribute in below.GetCustomAttributes(inherit: false)) + { + yield return attribute; + } + } + } + + /// The property a subtype declares that overrides a getter, or null. + private static PropertyInfo? Override(Type subtype, MethodInfo getter) + { + if (!getter.IsVirtual) + { + return null; + } + + MethodInfo root = getter.GetBaseDefinition(); + + foreach (PropertyInfo candidate in subtype.GetProperties( + BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.DeclaredOnly)) + { + if (candidate.GetGetMethod(nonPublic: true) is { } overriding + && Same(overriding.GetBaseDefinition(), root) + && !Same(overriding, getter)) + { + return candidate; + } + } + + return null; + } + + /// The property a type declares that implements an interface's getter, or null. + private static PropertyInfo? Implementation(Type subtype, Type contract, MethodInfo getter) + { + if (subtype.IsInterface || Implemented(subtype, contract, getter) is not { } target) + { + return null; + } + + return target.DeclaringType? + .GetProperties(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.DeclaredOnly) + .FirstOrDefault(candidate => candidate.GetGetMethod(nonPublic: true) is { } method && Same(method, target)); + } + + /// + /// For a class member, the interface getter it implements; for an interface getter, the method a type + /// implements it with. Null when there is none, or when the runtime cannot map an open generic type. + /// + private static MethodInfo? Implemented(Type type, Type contract, MethodInfo getter) + { + InterfaceMapping map; + + try + { + map = type.GetInterfaceMap(contract); + } + catch (Exception exception) when (exception is ArgumentException or InvalidOperationException or NotSupportedException) + { + return null; + } + + for (int i = 0; i < map.InterfaceMethods.Length; i++) + { + if (Same(map.InterfaceMethods[i], getter)) + { + return map.TargetMethods[i]; + } + + if (Same(map.TargetMethods[i], getter)) + { + return map.InterfaceMethods[i]; + } + } + + return null; + } + + /// The interface property an interface getter belongs to. + private static PropertyInfo? Declaration(MethodInfo? method, Type contract) => + method is null || method.DeclaringType != contract + ? null + : contract.GetProperties().FirstOrDefault(candidate => candidate.GetGetMethod() is { } getter && Same(getter, method)); + + private static bool Same(MethodInfo left, MethodInfo right) => + left.MetadataToken == right.MetadataToken && left.Module == right.Module; + /// /// How many collection layers peels off a single property type /// before it stops and walks whatever it has reached. diff --git a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs index 30b3854..3cc6ee5 100644 --- a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs +++ b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs @@ -1892,6 +1892,7 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) private static List? SynthesizedProjection(Gate gate, RowShape rows) { List allowed = new(); + List<(string Member, string Reason)> uncarried = new(); bool anyDenied = false; foreach (PropertyInfo member in gate.Members()) @@ -1941,14 +1942,27 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) // A forced scope beneath a member does not ask for a projection on its own. It filters the // rows that hold the member, which is what it has always done for a list returned whole, - // and asking would leave out every included list of a scoped child type. - if (reached.Count > 0 || unnamed.Opaque) + // and asking would leave out every included list of a scoped child type. Nor does a member + // that can hold an object of any type: the policy cannot see into it whether or not a + // projection is built, and asking would drop every such column of every query. + if (reached.Count > 0 || unnamed.DeniesSelect) { anyDenied = true; } if (!assignable || !rows.Carries(name)) { + // Left out because a projection cannot keep it. Worth a line in the trace only where the + // unguarded call would have returned it: an included navigation, an object in memory. + if (rows.Materializes(name + SegmentSeparator + name)) + { + uncarried.Add((name, !assignable + ? "left out: a projection cannot assign it" + : rows.Kind == RowKind.InMemory + ? "left out: a projection cannot keep an object a row in memory holds" + : "left out: it is a navigation, which projecting would load")); + } + continue; } @@ -1985,6 +1999,11 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) return null; } + foreach ((string name, string reason) in uncarried) + { + gate.LeaveOut(name, reason); + } + if (allowed.Count == 0) { throw gate.Exception(WholeClause, PolicyFeature.Select, PolicyErrorCode.AllSelectsDenied, null); @@ -2029,6 +2048,11 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) return $"left out whole: the core cannot project '{unbuilt}'"; } + if (gate.Unbuildable(member, kept) is { } node) + { + return $"left out whole: the core cannot build '{node.Name}', which it narrows into"; + } + if (kept.Count == 0) { return "left out whole: nothing beneath it may be selected"; @@ -2832,7 +2856,7 @@ private static void Descend( if (fragment.IsWildcard || !fragment.FieldPath.StartsWith(prefix, StringComparison.OrdinalIgnoreCase) || !asked.Add(fragment.FieldPath) - || Property(_entityType, fragment.FieldPath) is null) + || !Resolves(_entityType, fragment.FieldPath)) { continue; } @@ -2848,6 +2872,85 @@ private static void Descend( return (survivors, denied); } + /// + /// True when a path names a member, in any letter case, of the type or of any subtype a value along + /// it can be: a rule on Org.Swift names a field of the bank a member declared as an + /// organisation holds, and a type with both Info and INFO has two members the path + /// could name. + /// + private static bool Resolves(Type root, string path) + { + List current = new() { root }; + string[] segments = path.Split(SegmentSeparator); + + for (int i = 0; i < segments.Length; i++) + { + List next = new(); + bool named = false; + + foreach (Type type in current) + { + foreach (Type candidate in KnownSubtypes.Of(type).Prepend(type)) + { + foreach (PropertyInfo property in candidate.GetProperties(BindingFlags.Public | BindingFlags.Instance)) + { + if (!string.Equals(property.Name, segments[i], StringComparison.OrdinalIgnoreCase)) + { + continue; + } + + named = true; + + if (AttributePolicyProvider.NavigationTypeOf(property.PropertyType) is { } nested + && !next.Contains(nested)) + { + next.Add(nested); + } + } + } + } + + if (!named || (i < segments.Length - 1 && next.Count == 0)) + { + return false; + } + + current = next; + } + + return true; + } + + /// + /// The first type a narrowing would build a node as that the core's typed projection cannot + /// construct, an interface, an abstract class or one without a public parameterless constructor, + /// which it skips; null when it can build them all. + /// + internal Type? Unbuildable(string member, IEnumerable paths) + { + HashSet nodes = new(StringComparer.OrdinalIgnoreCase) { member }; + + foreach (string path in paths) + { + for (int cut = path.IndexOf(SegmentSeparator); cut >= 0; cut = path.IndexOf(SegmentSeparator, cut + 1)) + { + nodes.Add(path[..cut]); + } + } + + foreach (string node in nodes) + { + if (DeclaredType(node) is { } declared + && AttributePolicyProvider.NavigationTypeOf(declared) is { } type + && (type.IsAbstract || type.GetConstructor(Type.EmptyTypes) is null)) + { + return type; + } + } + + return null; + } + /// Every fragment the providers hold for this type and caller, read once per query. private IReadOnlyList Fragments => _fragments ??= _resolver.Fragments(_entityType, _context); @@ -2894,45 +2997,50 @@ internal bool ReadOnlyTransformBeneath(string member) /// On an entity query the model says what loads beneath the member /// (). Each loaded member is asked about by its path, and by /// its own attribute where no fragment reaches it: deeper than the walker goes, or declared by a - /// type the model derives. A value EF Core reads whole is read through its type. Any other source - /// can carry whatever the member's type can hold, its subtypes included. Under a policy that - /// denies every field it does not name, a path the walk never asks about is a denied one. + /// type the model derives. A value EF Core reads whole is read through its type. What it read + /// from the database holds no object of the application's, so an entity's members never count as + /// holding one. A projection's member is read as the type the projection constructs it as, when + /// it says. Any other value can carry whatever the member's type can hold, its subtypes included. + /// Under a policy that denies every field it does not name, a path the walk never asks about is a + /// denied one unless the policy names it. /// internal TypeFacts Unnamed(string member, RowShape rows) { if (rows.LoadedBeneath(member) is { } loaded) { bool denies = false; - bool holdsObject = false; foreach (Loaded entry in loaded) { - denies |= Denies(entry.Path, entry.Property); - - if (entry.Whole && !HoldsValue(entry.Property.PropertyType)) - { - TypeFacts whole = Carried(entry.Property.PropertyType, entry.Path); - - denies |= whole.DeniesSelect; - holdsObject |= whole.HoldsObject; - } + denies |= Denies(entry.Path, entry.Property) + || (entry.Whole + && !HoldsValue(entry.Property.PropertyType) + && Carried(entry.Property.PropertyType, entry.Path, exact: false).DeniesSelect); } - return new TypeFacts(denies, holdsObject, loaded.Count > 0); + return new TypeFacts(denies, false, loaded.Count > 0); + } + + if (!member.Contains(SegmentSeparator) && rows.BuiltType(member) is { } built) + { + return Carried(built, member, exact: true); } // A path that names nothing is refused long before this; were it not, nothing is shown clean. return DeclaredType(member) is { } type - ? Carried(type, member) + ? Carried(type, member, exact: false) : new TypeFacts(true, true, true); } - /// What a value of a type, held at a path, can carry that no path names. - private TypeFacts Carried(Type type, string path) + /// + /// What a value of a type, held at a path, can carry that no path names. An exact type is the one + /// the value was constructed as, so its own subtypes cannot be it. + /// + private TypeFacts Carried(Type type, string path, bool exact) { - TypeFacts facts = Facts(type); + TypeFacts facts = Facts(type, exact); - return DeniesByDefault && !Walks(type, path.Split(SegmentSeparator).Length) + return DeniesByDefault && !Granted(type, path, exact) ? facts with { DeniesSelect = true } : facts; } @@ -2943,8 +3051,9 @@ private TypeFacts Carried(Type type, string path) /// /// /// The walker puts a fragment on every path of the declared types up to its depth, a member - /// with no setter and a path around a cycle included. Past its depth, or on a member only a - /// derived type declares, the attribute is all there is. + /// with no setter, a path around a cycle, and a denial an override or an implementation declares + /// included. Past its depth, or on a member only a derived type declares, the attribute is all + /// there is. /// private bool Denies(string path, PropertyInfo property) { @@ -2961,12 +3070,13 @@ private bool Denies(string path, PropertyInfo property) /// /// Every member a type derived from T declares that a row can carry and this caller may not - /// have, or that carries what no path names. A projection builds T itself and leaves them all - /// out, so each is a reason to project. + /// have, or that carries a denied field no path names. A projection builds T itself and leaves + /// them all out, so each is a reason to project. /// /// - /// A projection builds exactly T. An entity query materializes each row as the type the database - /// says it is, which the model knows. Any other source can hold any subtype loaded. + /// A projection builds the type its initializer constructs, which may be a subtype of T. An entity + /// query materializes each row as the type the database says it is, which the model knows. Any + /// other source can hold any subtype loaded. /// internal List DerivedDenials(RowShape rows) { @@ -2974,6 +3084,11 @@ internal List DerivedDenials(RowShape rows) if (rows.Kind == RowKind.Projected) { + if (rows.Built is { } built && built != _entityType && _entityType.IsAssignableFrom(built)) + { + Declared(built, found); + } + return found; } @@ -2983,7 +3098,7 @@ internal List DerivedDenials(RowShape rows) { if (Denies(entry.Path, entry.Property) || (entry.Whole && !HoldsValue(entry.Property.PropertyType) - && Carried(entry.Property.PropertyType, entry.Path).Opaque)) + && Carried(entry.Property.PropertyType, entry.Path, exact: false).DeniesSelect)) { found.Add(entry.Path); } @@ -2994,25 +3109,32 @@ internal List DerivedDenials(RowShape rows) foreach (Type subtype in KnownSubtypes.Of(_entityType)) { - foreach (PropertyInfo property in subtype.GetProperties(BindingFlags.Public | BindingFlags.Instance)) + Declared(subtype, found); + } + + return found; + } + + /// Adds each member a subtype declares below T that this caller may not have. + private void Declared(Type subtype, List found) + { + foreach (PropertyInfo property in subtype.GetProperties(BindingFlags.Public | BindingFlags.Instance)) + { + if (!property.CanRead + || property.GetIndexParameters().Length != 0 + || property.DeclaringType!.IsAssignableFrom(_entityType) + || found.Contains(property.Name, StringComparer.Ordinal)) { - if (!property.CanRead - || property.GetIndexParameters().Length != 0 - || property.DeclaringType!.IsAssignableFrom(_entityType) - || found.Contains(property.Name, StringComparer.Ordinal)) - { - continue; - } + continue; + } - if (Denies(property.Name, property) - || (!HoldsValue(property.PropertyType) && Carried(property.PropertyType, property.Name).Opaque)) - { - found.Add(property.Name); - } + if (Denies(property.Name, property) + || (!HoldsValue(property.PropertyType) + && Carried(property.PropertyType, property.Name, exact: false).DeniesSelect)) + { + found.Add(property.Name); } } - - return found; } /// @@ -3028,27 +3150,101 @@ internal List DerivedDenials(RowShape rows) /// A path no member can have, so that only the "*" rules match it. private const string UnnamedPath = ""; + /// + /// Under a policy denying every field it does not name, true when every path beneath a member of + /// this type that the walk never asks about is one the policy grants by name. + /// + /// + /// The walk skips a property with no setter, stops four segments deep and at a type it already + /// passed through, reads declared types only and takes a framework type whole. Each path it skips + /// is asked about here, a subtype's members included; one the policy does not name is denied. + /// Around a cycle the paths never end, and inside a framework type holding an application type's + /// members no path can name them, so neither is ever granted. + /// + private bool Granted(Type type, string path, bool exact) + { + if (!_granted.TryGetValue((type, path, exact), out bool granted)) + { + granted = Grant(type, path, exact, new HashSet()); + _granted[(type, path, exact)] = granted; + } + + return granted; + } + + private readonly Dictionary<(Type, string, bool), bool> _granted = new(); + + private bool Grant(Type type, string path, bool exact, HashSet onPath) + { + if (AttributePolicyProvider.NavigationTypeOf(type) is not { } node) + { + TypeFacts facts = Facts(type, exact: false); + + return !facts.HoldsObject && !facts.HoldsMembers; + } + + if (!onPath.Add(node)) + { + return false; + } + + int depth = path.Split(SegmentSeparator).Length; + IEnumerable<(PropertyInfo Property, bool Declared)> members = node + .GetProperties(BindingFlags.Public | BindingFlags.Instance) + .Select(property => (property, true)); + + if (!exact) + { + members = members.Concat(KnownSubtypes.Of(node) + .SelectMany(subtype => subtype.GetProperties(BindingFlags.Public | BindingFlags.Instance)) + .Where(property => !property.DeclaringType!.IsAssignableFrom(node)) + .Select(property => (property, false))); + } + + foreach ((PropertyInfo property, bool declared) in members) + { + if (!property.CanRead || property.GetIndexParameters().Length != 0) + { + continue; + } + + string child = $"{path}{SegmentSeparator}{property.Name}"; + bool walked = declared && property.CanWrite && depth < AttributePolicyProvider.MaxDepth; + + if ((!walked && Denies(child, property)) + || (!HoldsValue(property.PropertyType) && !Grant(property.PropertyType, child, false, onPath))) + { + return false; + } + } + + onPath.Remove(node); + + return true; + } + /// /// What a type can hold that no path of the policy's reaches, read once per type and again once - /// another assembly has loaded, since that can add subtypes. + /// another assembly has loaded, since that can add subtypes. An exact type is the one a value was + /// constructed as, so its own subtypes are not read. /// - internal static TypeFacts Facts(Type type) + internal static TypeFacts Facts(Type type, bool exact = false) { int epoch = KnownSubtypes.Epoch; - if (FactsByType.TryGetValue(type, out (int Epoch, TypeFacts Facts) known) && known.Epoch == epoch) + if (FactsByType.TryGetValue((type, exact), out (int Epoch, TypeFacts Facts) known) && known.Epoch == epoch) { return known.Facts; } - TypeFacts facts = Scan(type); + TypeFacts facts = Scan(type, exact); - FactsByType[type] = (epoch, facts); + FactsByType[(type, exact)] = (epoch, facts); return facts; } - private static readonly ConcurrentDictionary FactsByType = new(); + private static readonly ConcurrentDictionary<(Type, bool), (int Epoch, TypeFacts Facts)> FactsByType = new(); /// /// Reads every type a value of this type can hold, however deep, for a field denied for Select @@ -3059,10 +3255,11 @@ internal static TypeFacts Facts(Type type) /// framework type and reads declared types only, so a field denied beneath any of those has no /// path, and resolves as allowed. This follows every property, a read-only one included, every /// collection, the type arguments of a framework generic such as Dictionary<string, T>, - /// and every loaded subtype of a type it reads, and remembers the types it has read rather than - /// counting levels. A member that can hold anything is reported rather than read. + /// and every loaded subtype of a type it reads, a framework class's included, and remembers the + /// types it has read rather than counting levels. A member that can hold anything is reported + /// rather than read. /// - private static TypeFacts Scan(Type type) + private static TypeFacts Scan(Type type, bool exact) { bool denies = false; bool holdsObject = false; @@ -3078,7 +3275,17 @@ void Read(Type candidate) } } - void Enqueue(Type candidate) + void Subtypes(Type of) + { + foreach (Type subtype in KnownSubtypes.Of(of)) + { + holdsMembers = true; + + Read(subtype); + } + } + + void Enqueue(Type candidate, bool root) { Type peeled = AttributePolicyProvider.Peeled(candidate); @@ -3095,22 +3302,28 @@ void Enqueue(Type candidate) Read(navigation); - foreach (Type subtype in KnownSubtypes.Of(navigation)) + if (!(root && exact)) { - Read(subtype); + Subtypes(navigation); } } + else if (peeled.IsClass && !peeled.IsSealed && !peeled.IsGenericType && !(root && exact)) + { + // A framework class an application can derive from, an Exception or a Stream: the + // subtype a value holds is read, since the class itself carries no policy. + Subtypes(peeled); + } if (peeled.IsGenericType && AttributePolicyProvider.IsFramework(peeled)) { foreach (Type argument in peeled.GetGenericArguments()) { - Enqueue(argument); + Enqueue(argument, false); } } } - Enqueue(type); + Enqueue(type, true); while (pending.Count > 0 && !(denies && holdsObject)) { @@ -3127,7 +3340,7 @@ void Enqueue(Type candidate) denies = true; } - Enqueue(property.PropertyType); + Enqueue(property.PropertyType, false); } } @@ -3135,16 +3348,18 @@ void Enqueue(Type candidate) } /// - /// True for a type whose value can be an object of any type: itself, a - /// framework interface such as IComparable, and a collection that is not generic, such as - /// ArrayList, Hashtable, IEnumerable or . + /// True for a type whose value can be an object of any type: itself, a type + /// parameter an open generic subtype leaves unbound, a framework interface such as + /// IComparable, and a collection that is not generic, such as ArrayList, + /// IEnumerable, or an application's own. /// private static bool HoldsAnything(Type peeled) => peeled == typeof(object) || peeled == typeof(ValueType) - || (AttributePolicyProvider.IsFramework(peeled) - && !peeled.IsGenericType - && (peeled.IsInterface || (peeled != typeof(string) && typeof(IEnumerable).IsAssignableFrom(peeled)))); + || peeled.IsGenericParameter + || (!peeled.IsGenericType + && ((AttributePolicyProvider.IsFramework(peeled) && peeled.IsInterface) + || (peeled != typeof(string) && typeof(IEnumerable).IsAssignableFrom(peeled)))); /// /// What a type can hold that the policy cannot name a path to. @@ -3158,67 +3373,6 @@ internal readonly record struct TypeFacts(bool DeniesSelect, bool HoldsObject, b internal bool Opaque => DeniesSelect || HoldsObject; } - /// - /// True when the walk beneath a member of this type, the given number of segments deep, asks - /// about every path a value of it can hold. - /// - /// - /// The walk skips a property with no setter, stops four segments deep and at a type it has - /// already passed through, reads declared types only and takes a framework type whole. Any of - /// those beneath a member leaves a path it never asks about, which a policy denying every field - /// it does not name denies. - /// - internal static bool Walks(Type type, int depth) - { - int epoch = KnownSubtypes.Epoch; - - if (WalkedByType.TryGetValue((type, depth), out (int Epoch, bool Walked) known) && known.Epoch == epoch) - { - return known.Walked; - } - - bool walked = Walk(type, depth, new HashSet()); - - WalkedByType[(type, depth)] = (epoch, walked); - - return walked; - } - - private static readonly ConcurrentDictionary<(Type Type, int Depth), (int Epoch, bool Walked)> WalkedByType = new(); - - private static bool Walk(Type type, int depth, HashSet path) - { - if (AttributePolicyProvider.NavigationTypeOf(type) is not { } node) - { - // A value, or a framework type the walk takes whole: it asks about nothing beneath it. - TypeFacts facts = Facts(type); - - return !facts.HoldsObject && !facts.HoldsMembers; - } - - if (depth >= AttributePolicyProvider.MaxDepth || !path.Add(node) || KnownSubtypes.Of(node).Count > 0) - { - return false; - } - - foreach (PropertyInfo property in node.GetProperties(BindingFlags.Public | BindingFlags.Instance)) - { - if (!property.CanRead || property.GetIndexParameters().Length != 0) - { - continue; - } - - if (!property.CanWrite || !Walk(property.PropertyType, depth + 1, path)) - { - return false; - } - } - - path.Remove(node); - - return true; - } - /// /// The declared type at a path, read the way the walker reads it: through any collection, one /// segment at a time. Null when a segment names nothing. diff --git a/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs b/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs index 7949a0a..066c2ad 100644 --- a/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs +++ b/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs @@ -13,31 +13,39 @@ namespace DynamicWhere.ex.Policies.Source; /// member the policy reads as clean, unless the subtypes are read too. /// /// Every object's type is loaded before the object exists, so the loaded assemblies are the whole answer -/// for any value a query returns. Only an assembly that is the type's own, or references it, can declare -/// a subtype, which keeps the framework's and every unrelated library's assemblies out of the search. -/// Loading another assembly may add subtypes, so each answer is kept only until the next one loads; -/// an assembly emitted at run time is not searched, since it cannot declare a policy attribute a -/// compiled type does not already carry. +/// for any value a query returns. The framework's own assemblies are left out: none of them declares a +/// subtype of an application's type, and a framework type carries no policy. Every other assembly is +/// indexed once, each type under every base class and interface it has, so a search is a lookup. A +/// generic type is indexed under its definition as well as its own instantiation, and a subtype that is +/// itself generic is returned open, since the instantiation a row holds is not known. +/// +/// +/// Loading another such assembly may add subtypes, so the index is rebuilt after one loads. An assembly +/// emitted at run time, or one holding only resources, cannot declare one. /// /// internal static class KnownSubtypes { - private static readonly ConcurrentDictionary Found = new(); + private static readonly object Building = new(); + + /// Every assembly seen loading, in case one raises its event before the domain lists it. + private static readonly ConcurrentDictionary SeenLoading = new(); private static int _epoch; - private static Snapshot? _snapshot; + private static Index? _index; static KnownSubtypes() => AppDomain.CurrentDomain.AssemblyLoad += (_, loaded) => { - if (!loaded.LoadedAssembly.IsDynamic) + if (Searched(loaded.LoadedAssembly)) { + SeenLoading.TryAdd(loaded.LoadedAssembly, 0); Interlocked.Increment(ref _epoch); } }; - /// Changes whenever an assembly loads, and with it any answer read from the subtypes. + /// Changes whenever an assembly that could declare a subtype loads, and with it every answer read from one. internal static int Epoch => Volatile.Read(ref _epoch); /// Every loaded type, other than the type itself, whose values the type's members can hold. @@ -48,68 +56,134 @@ internal static IReadOnlyList Of(Type type) return Array.Empty(); } - int epoch = Epoch; + Index index = Current(); + List? found = null; - if (Found.TryGetValue(type, out (int Epoch, Type[] Types) known) && known.Epoch == epoch) + if (index.Descendants.TryGetValue(type, out Type[]? direct)) { - return known.Types; + found = new List(direct); } - Type[] types = Search(type, Assemblies(epoch)); + // A subtype declared over the definition, class Tagged : Base, may be any instantiation of it; + // and a type still open itself may be any instantiation, so every subtype of the definition counts. + if (type.IsGenericType + && index.Descendants.TryGetValue(type.GetGenericTypeDefinition(), out Type[]? byDefinition)) + { + found ??= new List(); - Found[type] = (epoch, types); + foreach (Type candidate in byDefinition) + { + if ((type.ContainsGenericParameters || candidate.ContainsGenericParameters) && !found.Contains(candidate)) + { + found.Add(candidate); + } + } + } - return types; + return found is null ? Array.Empty() : found; } - private static Type[] Search(Type type, IEnumerable assemblies) + /// The index for the assemblies loaded now, built once for each epoch. + private static Index Current() { - string home = type.Assembly.GetName().Name ?? string.Empty; - List found = new(); + if (Volatile.Read(ref _index) is { } index && index.Epoch == Epoch) + { + return index; + } - foreach (Candidate candidate in assemblies) + lock (Building) { - if (candidate.Assembly != type.Assembly && !candidate.References.Contains(home)) + if (_index is { } built && built.Epoch == Epoch) + { + return built; + } + + // The epoch is read before the assemblies, so one loading meanwhile rebuilds the index again. + int epoch = Epoch; + Index fresh = Build(epoch); + + Volatile.Write(ref _index, fresh); + + return fresh; + } + } + + private static Index Build(int epoch) + { + HashSet assemblies = new(AppDomain.CurrentDomain.GetAssemblies().Where(Searched)); + + assemblies.UnionWith(SeenLoading.Keys); + + Dictionary> descendants = new(); + + void Add(Type ancestor, Type type) + { + Type key = ancestor.IsGenericType && ancestor.ContainsGenericParameters + ? ancestor.GetGenericTypeDefinition() + : ancestor; + + if (!descendants.TryGetValue(key, out List? list)) { - continue; + descendants[key] = list = new List(); } - foreach (Type declared in candidate.Types.Value) + list.Add(type); + + if (key != ancestor || !ancestor.IsGenericType) { - if (declared != type && !declared.ContainsGenericParameters && type.IsAssignableFrom(declared)) + return; + } + + // A closed ancestor, Base, is also one instantiation of its definition. + Type definition = ancestor.GetGenericTypeDefinition(); + + if (!descendants.TryGetValue(definition, out List? byDefinition)) + { + descendants[definition] = byDefinition = new List(); + } + + byDefinition.Add(type); + } + + foreach (Assembly assembly in assemblies) + { + foreach (Type type in TypesOf(assembly)) + { + for (Type? ancestor = type.BaseType; ancestor is not null && ancestor != typeof(object); ancestor = ancestor.BaseType) { - found.Add(declared); + Add(ancestor, type); + } + + foreach (Type contract in Interfaces(type)) + { + Add(contract, type); } } } - return found.ToArray(); + return new Index(epoch, descendants.ToDictionary(entry => entry.Key, entry => entry.Value.Distinct().ToArray())); } - /// - /// The loaded assemblies that could declare a subtype, each with the names it references, read once - /// per epoch; an assembly's types are read only when a search reaches it. - /// - private static Candidate[] Assemblies(int epoch) + /// An assembly that can declare a subtype of an application's type. + private static bool Searched(Assembly assembly) { - if (Volatile.Read(ref _snapshot) is { } snapshot && snapshot.Epoch == epoch) + if (assembly.IsDynamic) { - return snapshot.Candidates; + return false; } - Candidate[] candidates = AppDomain.CurrentDomain.GetAssemblies() - .Where(assembly => !assembly.IsDynamic) - .Select(assembly => new Candidate( - assembly, - new HashSet( - assembly.GetReferencedAssemblies().Select(reference => reference.Name ?? string.Empty), - StringComparer.Ordinal), - new Lazy(() => TypesOf(assembly)))) - .ToArray(); + AssemblyName name = assembly.GetName(); + + if (!string.IsNullOrEmpty(name.CultureName)) + { + return false; + } - Volatile.Write(ref _snapshot, new Snapshot(epoch, candidates)); + string simple = name.Name ?? string.Empty; - return candidates; + return !(simple is "mscorlib" or "netstandard" or "System" or "WindowsBase" + || simple.StartsWith("System.", StringComparison.Ordinal) + || simple.StartsWith("Microsoft.", StringComparison.Ordinal)); } private static Type[] TypesOf(Assembly assembly) @@ -124,7 +198,17 @@ private static Type[] TypesOf(Assembly assembly) } } - private sealed record Candidate(Assembly Assembly, HashSet References, Lazy Types); + private static Type[] Interfaces(Type type) + { + try + { + return type.GetInterfaces(); + } + catch (TypeLoadException) + { + return Array.Empty(); + } + } - private sealed record Snapshot(int Epoch, Candidate[] Candidates); + private sealed record Index(int Epoch, Dictionary Descendants); } diff --git a/DynamicWhere.ex/Policies/Source/RowShape.cs b/DynamicWhere.ex/Policies/Source/RowShape.cs index ef046c1..24c7f9a 100644 --- a/DynamicWhere.ex/Policies/Source/RowShape.cs +++ b/DynamicWhere.ex/Policies/Source/RowShape.cs @@ -26,7 +26,7 @@ namespace DynamicWhere.ex.Policies.Source; /// only when something loads it: an Include, an automatic include or a lazy loader. A value /// beneath a navigation nothing loads never reaches the result, so a denial there needs no /// projection; and projecting the navigation would load it. A member the model does not map is -/// never filled by EF Core at all. +/// computed by the class, from whatever EF Core loaded into it, so it counts as holding its type. /// /// /// A projection holds exactly the members its initializer assigns. The others hold defaults, and @@ -62,6 +62,7 @@ internal sealed class RowShape private static readonly ConditionalWeakTable> ComplexByType = new(); private readonly HashSet? _assigned; + private readonly Dictionary _built = new(StringComparer.OrdinalIgnoreCase); private readonly HashSet _narrowable = new(StringComparer.OrdinalIgnoreCase); private readonly HashSet _kept = new(StringComparer.OrdinalIgnoreCase); private readonly HashSet _includes = new(StringComparer.OrdinalIgnoreCase); @@ -71,11 +72,18 @@ internal sealed class RowShape private RowShape(RowKind kind) => Kind = kind; - private RowShape(RowKind kind, HashSet? assigned, IEnumerable narrowable) + private RowShape( + RowKind kind, HashSet? assigned, IEnumerable narrowable, Type? built, IDictionary members) : this(kind) { _assigned = assigned; _narrowable.UnionWith(narrowable); + Built = built; + + foreach (KeyValuePair member in members) + { + _built[member.Key] = member.Value; + } } private RowShape(IEntityType entity, IEnumerable includes, bool includeUnknown) @@ -125,6 +133,20 @@ private RowShape(IEntityType entity, IEnumerable includes, bool includeU /// Where the rows come from. internal RowKind Kind { get; } + /// + /// The type a projection constructs its rows as, which may be a subtype of the query's element type; + /// null for any other source. + /// + internal Type? Built { get; } + + /// + /// The type a projection constructs a member's value as, or each element of a list it builds, when + /// the initializer says: Contact = new ContactRow { … } holds a ContactRow whatever the + /// member is declared as. Null when the value is read from somewhere else. + /// + internal Type? BuiltType(string member) => + Kind == RowKind.Projected && _built.TryGetValue(member, out Type? type) ? type : null; + /// /// True when a synthesized projection may keep a member holding an object: one a projection /// assigned, or an entity's column, owned or complex member. Never an entity's navigation, which @@ -173,10 +195,11 @@ internal bool Narrows(string member) => /// /// /// A navigation is loaded when it is owned, included, automatically included, or reachable by a - /// lazy loader. A member the model does not map is one EF Core never fills, so nothing reaches the - /// result through it. Anything the model cannot answer for counts as loaded, which only ever asks for - /// a projection that turns out not to be needed. A rule may spell a path in any letter case, so each - /// segment is read through the member it names. + /// lazy loader. A member the model does not map is computed by the class, and a getter over a mapped + /// field or a private navigation hands out what EF Core loaded, so it counts as loaded too. Anything + /// the model cannot answer for counts as loaded, which only ever asks for a projection that turns out + /// not to be needed. A rule may spell a path in any letter case, so each segment is read through the + /// member it names. /// internal bool Materializes(string path) { @@ -215,7 +238,12 @@ internal bool Materializes(string path) INavigationBase? navigation = (INavigationBase?)current.FindNavigation(name) ?? current.FindSkipNavigation(name); - if (navigation is null || !Loads(current, navigation, prefix)) + if (navigation is null) + { + return true; + } + + if (!Loads(current, navigation, prefix)) { return false; } @@ -234,9 +262,9 @@ internal bool Materializes(string path) /// Naming a member in a projection loads it, so the path's own segments are not asked about. Beneath /// it the model decides: an entity's columns, owned and complex members, and the navigations /// something loads, and the same for every type the model derives from it, whose rows EF Core - /// materializes as that type. A member the model does not map holds nothing EF Core read. A lazy - /// loader in reach can fill any navigation, which bounds nothing, and so does a query whose includes - /// cannot be read. + /// materializes as that type. A member the model does not map is read whole, as a column is: its + /// getter can hand out anything the entity holds. A lazy loader in reach can fill any navigation, + /// which bounds nothing, and so does a query whose includes cannot be read. /// internal List? LoadedBeneath(string path) { @@ -267,9 +295,10 @@ internal bool Materializes(string path) INavigationBase? navigation = (INavigationBase?)entity.FindNavigation(member.Name) ?? entity.FindSkipNavigation(member.Name); + // A member the model does not map holds what its getter computes: read it whole, as its type. if (navigation is null) { - return new List(); + return i == segments.Length - 1 ? new List { new(prefix, member, true) } : null; } entity = navigation.TargetEntityType; @@ -352,7 +381,15 @@ private bool Collect( INavigationBase? navigation = (INavigationBase?)type.FindNavigation(member.Name) ?? type.FindSkipNavigation(member.Name); - if (navigation is null || !Loads(type, navigation, path)) + // Not mapped: whatever its getter computes from what EF Core loaded, read as its type. + if (navigation is null) + { + loaded.Add(new Loaded(path, member, true)); + + continue; + } + + if (!Loads(type, navigation, path)) { continue; } @@ -383,9 +420,11 @@ private bool Loads(IEntityType owner, INavigationBase navigation, string path) = /// builds its rows, which is how a caller makes its own row type out of entities. What is left is an /// entity query, read from the EF Core model; without one, the source is . A /// chain that reaches its rows through anything but the root's own, Select(o => o.Customer), - /// a SelectMany, a Join or a projection behind another Select, holds entities - /// whose navigations were loaded from another root or by a projection, so every navigation counts as - /// loaded there. + /// a SelectMany, a Join or a GroupBy, holds entities EF Core loaded along + /// another path. Its includes name paths from another root, which EF Core still applies, and a + /// projection inside it, such as one behind an identity Select, loads whatever it assigns: with + /// either, every navigation counts as loaded. With neither, only an automatic include or a lazy + /// loader fills one, which the model says. /// internal static RowShape Of(IQueryable source) where T : class { @@ -405,7 +444,8 @@ internal static RowShape Of(IQueryable source) where T : class } HashSet includes = new(StringComparer.OrdinalIgnoreCase); - bool unknown = !ReadIncludes(source.Expression, includes) || QueryRoot.Reshapes(source.Expression); + bool unknown = !ReadIncludes(source.Expression, includes, out int included) + || (QueryRoot.Reshapes(source.Expression) && (included > 0 || QueryRoot.Builds(source.Expression))); return new RowShape(entity, includes, unknown); } @@ -419,33 +459,74 @@ private static RowShape Projected(IQueryable source) where T : class MethodCallExpression select = DefaultOrder.RowSelect(source.Expression)!; LambdaExpression selector = (LambdaExpression)DefaultOrder.StripQuotes(select.Arguments[1]); Expression body = DefaultOrder.StripConversions(selector.Body); + Dictionary built = new(StringComparer.OrdinalIgnoreCase); - // A constructor with arguments says nothing about which member each argument sets, whether or - // not an initializer follows it: every member counts as assigned, and none can be narrowed. - if (body is not MemberInitExpression initializer || initializer.NewExpression.Arguments.Count > 0) + if (body is not MemberInitExpression initializer) { - return new RowShape(RowKind.Projected, null, Array.Empty()); + // A constructor with arguments says nothing about which member each argument sets: every + // member counts as assigned, and none can be narrowed. + return new RowShape(RowKind.Projected, null, Array.Empty(), body.Type, built); } - HashSet assigned = new(StringComparer.OrdinalIgnoreCase); List narrowable = new(); ParameterExpression row = selector.Parameters[0]; bool efCore = source.Provider is IAsyncQueryProvider; IEntityType? rowSource = efCore ? QueryRoot.EntityType(source.Expression, row.Type) : null; + HashSet? assigned = initializer.NewExpression.Arguments.Count > 0 + ? null + : new HashSet(StringComparer.OrdinalIgnoreCase); foreach (MemberBinding binding in initializer.Bindings) { - assigned.Add(binding.Member.Name); + // After a constructor with arguments every member counts as assigned, since the constructor + // may set any of them; the initializer's own bindings still say what they hold. + assigned?.Add(binding.Member.Name); + + if (binding is not MemberAssignment assignment) + { + continue; + } + + if (BuiltBy(assignment.Expression) is { } type) + { + built[binding.Member.Name] = type; + } // The narrowing reads the member through EF.Property, so only EF Core can run it. - if (efCore && binding is MemberAssignment assignment && CanNarrow(assignment, row, rowSource)) + if (efCore && CanNarrow(assignment, row, rowSource)) { narrowable.Add(binding.Member.Name); } } - return new RowShape(RowKind.Projected, assigned, narrowable); + return new RowShape(RowKind.Projected, assigned, narrowable, initializer.Type, built); + } + + /// + /// The type an assigned value is constructed as, new ContactRow { … }, or each element of a + /// list a subquery builds, o.Lines.Select(l => new LineRow { … }).ToList(); null otherwise. + /// + private static Type? BuiltBy(Expression expression) + { + Expression value = DefaultOrder.StripConversions(expression); + + if (value is MemberInitExpression or NewExpression) + { + return value.Type; + } + + if (value is MethodCallExpression { Method.Name: "ToList" or "ToArray" or "ToHashSet" } read + && (read.Method.DeclaringType == typeof(Enumerable) || read.Method.DeclaringType == typeof(Queryable)) + && DefaultOrder.StripConversions(read.Arguments[0]) is MethodCallExpression { Method.Name: "Select" } select + && select.Arguments.Count == 2 + && DefaultOrder.StripQuotes(select.Arguments[1]) is LambdaExpression { Body: var element } + && DefaultOrder.StripConversions(element) is MemberInitExpression or NewExpression) + { + return DefaultOrder.StripConversions(element).Type; + } + + return null; } /// @@ -492,7 +573,7 @@ private static bool CanNarrow(MemberAssignment assignment, ParameterExpression r /// with its prefixes. False when one names its path in a form this cannot read, so every navigation /// counts as loaded. /// - private static bool ReadIncludes(Expression expression, HashSet includes) + private static bool ReadIncludes(Expression expression, HashSet includes, out int included) { List calls = new(); @@ -511,6 +592,8 @@ private static bool ReadIncludes(Expression expression, HashSet includes counter.Visit(expression); + included = counter.Count; + bool readable = counter.Count == calls.Count; string? current = null; @@ -619,8 +702,8 @@ private static bool IsOwned(INavigation navigation) => /// /// True when a lazy loader can fill this entity's navigations after the query: EF Core's proxies and /// an injected ILazyLoader give it a service property, and a loader the class takes in its - /// constructor, the delegate form above all, may be kept in a field or a property of any name, - /// where the model has no record of it. + /// constructor, either delegate form above all, may be kept in a field or a property of any name, + /// where the model has no record of it. An injected DbContext can load anything. /// private static bool LoadsLazily(IEntityType entity) => entity.GetServiceProperties().Any(service => IsLoader(service.ClrType)) @@ -647,7 +730,11 @@ private static bool HoldsLoader(Type type) return false; } - private static bool IsLoader(Type type) => type == typeof(ILazyLoader) || type == typeof(Action); + private static bool IsLoader(Type type) => + type == typeof(ILazyLoader) + || type == typeof(Action) + || type == typeof(Func) + || typeof(DbContext).IsAssignableFrom(type); /// /// The names of an entity type's complex properties, which EF Core 8 introduced and loads with the diff --git a/DynamicWhere.ex/Source/QueryRoot.cs b/DynamicWhere.ex/Source/QueryRoot.cs index a500a14..329b861 100644 --- a/DynamicWhere.ex/Source/QueryRoot.cs +++ b/DynamicWhere.ex/Source/QueryRoot.cs @@ -104,6 +104,67 @@ or nameof(Queryable.Intersect) or "IntersectBy" or nameof(Queryable.Except) or " return false; } + /// + /// True when a call along the chain that does not know constructs an object in + /// one of its lambdas: a projection, such as the one behind Select(x => x), an anonymous row + /// or a conditional, assigns what it builds, and loads whatever navigation it assigns. + /// + internal static bool Builds(Expression expression) + { + ConstructionFinder finder = new(); + + for (Expression? node = expression; node is MethodCallExpression call; + node = call.Arguments.Count > 0 ? call.Arguments[0] : null) + { + if (call.Arguments.Count > 1 + && call.Method.Name is nameof(Queryable.Concat) or nameof(Queryable.Union) or "UnionBy" + or nameof(Queryable.Intersect) or "IntersectBy" or nameof(Queryable.Except) or "ExceptBy" + && Builds(call.Arguments[1])) + { + return true; + } + + if (KeepsRows(call)) + { + continue; + } + + foreach (Expression argument in call.Arguments.Skip(1)) + { + finder.Visit(argument); + + if (finder.Found) + { + return true; + } + } + } + + return false; + } + + /// Finds an object construction anywhere in an expression. + private sealed class ConstructionFinder : ExpressionVisitor + { + internal bool Found { get; private set; } + + public override Expression? Visit(Expression? node) => Found ? node : base.Visit(node); + + protected override Expression VisitNew(NewExpression node) + { + Found = true; + + return node; + } + + protected override Expression VisitMemberInit(MemberInitExpression node) + { + Found = true; + + return node; + } + } + /// Finds the first query root that names an entity type. private sealed class RootFinder : ExpressionVisitor { From 279f45467917b65916396f80e624b30d302be4d1 Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 07:52:04 +0300 Subject: [PATCH 16/22] docs: subtypes, overrides, unmapped getters and what asks for nothing The agent reference, the site, README, DOC.md and the release notes say what 203f5c9 changed: - A deny-family attribute on an override, on an implementation, or on the interface member a class implements applies to the path. - Subtypes include open generics and applications' subclasses of framework classes. - A projection constructing a subtype of T is read as it. - A rule through a subtype's member is enforced. - Unmapped members count as loaded and are read as their type. - An injected DbContext and the async loader delegate count as loaders. - A reshaped chain counts every navigation only with an include or a built object. - A member that can hold anything asks for no projection, and an entity keeps it. - Under a "*" deny, a skipped path is asked of the policy. - A member a projection leaves out that the unguarded call returned is in the trace. New limits: rows in memory are projected whenever a loaded subtype declares a denial, and a default order does not reach a projection over an intermediate row. Co-Authored-By: Claude Opus 5 --- DynamicWhere.ex/DOC.md | 24 ++-- DynamicWhere.ex/DynamicWhere.ex.csproj | 4 +- .../app/docs/breaking-changes/page.tsx | 43 +++--- .../app/docs/policies/attributes/page.tsx | 13 +- .../app/docs/policies/configuration/page.tsx | 121 +++++++++------- .../app/docs/policies/security/page.tsx | 35 +++-- OfficialWebsite/public/llms.txt | 135 ++++++++++++------ README.md | 6 +- 8 files changed, 244 insertions(+), 137 deletions(-) diff --git a/DynamicWhere.ex/DOC.md b/DynamicWhere.ex/DOC.md index e127a04..4d2ca69 100644 --- a/DynamicWhere.ex/DOC.md +++ b/DynamicWhere.ex/DOC.md @@ -1682,7 +1682,7 @@ A `Selects` entry can name a navigation, such as `"Lines"`, rather than the fiel - The fields beneath a member are read the way the attribute walker reads them: through any collection type, and no deeper than its four segments. Since 3.2.0 a member typed `IReadOnlyList`, `IReadOnlyCollection`, `Collection` or an application's own collection no longer hides the denials beneath it. The providers' own rules are asked too, so a denied property with no setter and a rule on a path reached through a cycle count. - A narrowing that cannot be built as it was gated is refused in both tiers (3.2.0). The core's typed projection adds the key, `Id`, of every nested node it builds, so a navigation narrowed around its own denied key would get the key back. The core reads a path only through an array, `List`, `IList`, `ICollection`, `IEnumerable`, `HashSet` or `ISet`, so a narrowing through any other collection fails its validation. And some members cannot be narrowed at all: a column, a complex property or a member stored as JSON, which EF Core reads whole; a member of a row in memory; and a member a projection builds some way the core cannot narrow. -- A named member can also carry a field denied for `Select` that no path names (3.2.0): one deeper than four segments, one inside a framework generic such as `Dictionary`, or one a subtype of the member's type declares — a derived entity, a subclass, an interface's implementation. What it can carry is read from the source. On an entity it is read from the EF Core model, so only what loads counts: the navigation's columns, a converted one included, its owned chain at any depth, and the navigations beneath it an include, an automatic include or a lazy loader fills, for its type and every type the model derives from it. On a projected or in-memory row it is the member's type and every loaded subtype of it. Under a policy with a `"*"` deny, a path the walk never asks about — past four segments, around a cycle, with no setter, or on a subtype — is a denied one as well. The `Strict` tier refuses such a member. The `Convenience` tier narrows it where the core can, which builds the declared type and so drops a subtype's fields; a path naming a framework generic itself narrows to nothing and is dropped. Where the core cannot narrow it — a column at the top of `T`, a member of a row in memory — both tiers refuse it. +- A named member can also carry a field denied for `Select` that no path names (3.2.0): one deeper than four segments, one inside a framework generic such as `Dictionary`, or one a subtype of the member's type declares — a derived entity, a subclass, an interface's implementation. What it can carry is read from the source. On an entity it is read from the EF Core model, so only what loads counts: the navigation's columns, a converted one included, its owned chain at any depth, the navigations beneath it an include, an automatic include or a lazy loader fills, and each member the model does not map, read as its type, since its getter can hand out what EF Core loaded — for its type and every type the model derives from it. On a projected row it is the type the initializer constructs the member as, when it says, and otherwise the member's type and every loaded subtype of it, as on a row in memory. Under a policy with a `"*"` deny, a path the walk never asks about — past four segments, with no setter, or on a subtype — is a denied one unless the policy names it; around a cycle it always is. The `Strict` tier refuses such a member. The `Convenience` tier narrows it where the core can, which builds the declared type and so drops a subtype's fields; a path naming a framework generic itself narrows to nothing and is dropped. Where the core cannot narrow it — a column at the top of `T`, a member of a row in memory — both tiers refuse it. - A navigation named through another, `Main.Lead`, gates the key of every node it passes through, which the core's projection adds, as a dotted path to a value always did (3.2.0). A denied key refuses the projection. Naming a field beside a denied key, `Lines.Name` when `Lines.Id` is denied, was already refused in both tiers. - Under `Convenience` the refusal names the denied key, the first denied field beneath the member, or, for a denial no path names, the member itself. Under `Strict` its `FieldPath` is `"*"`, as on every field refusal. The trace records the reason either way. @@ -1695,12 +1695,13 @@ The projection keeps the **allowed members**: what an unguarded call would retur **When a projection is needed.** - A field denied at the top of `T` always asks for one, whatever it holds: a scalar, a blob, a list, an owned object or a JSON column. -- A field denied beneath a member asks for one when its value can reach the result. On an entity, that is beneath a column, an owned or complex member, or a navigation something loads: an `Include` or `ThenInclude` on the query, an automatic include, or a lazy loader — EF Core's proxies, an injected `ILazyLoader`, or a loader delegate or `ILazyLoader` the constructor takes and keeps in a field or any property — which fills a navigation after the query. On a row a projection builds, it is beneath a member the initializer assigns; a constructor with arguments, with or without an initializer after it, counts every member as assigned. On a row in memory, it is beneath any member. A rule may spell the path in any letter case. -- Every navigation counts as loaded where the library cannot read which the query loads: an `Include` in a form it cannot read, one off the query's own chain, and a chain that reaches its rows through anything but the root's own rows — `Select(o => o.Customer)`, a `SelectMany`, a `Join`, or a projection behind another `Select` (an identity `Select`, a member of an anonymous row, a conditional). EF Core still applies includes named from the root to the entities such a chain reaches. +- A field denied beneath a member asks for one when its value can reach the result. On an entity, that is beneath a column, an owned or complex member, or a navigation something loads: an `Include` or `ThenInclude` on the query, an automatic include, or a lazy loader — EF Core's proxies, an injected `ILazyLoader`, a loader delegate or `ILazyLoader` the constructor takes and keeps in a field or any property, the asynchronous loader delegate of EF Core 7, or an injected `DbContext` — which fills a navigation after the query. On a row a projection builds, it is beneath a member the initializer assigns; a constructor with arguments counts every member as assigned, and an initializer after it still says what its own bindings hold. On a row in memory, it is beneath any member. A rule may spell the path in any letter case. +- Every navigation counts as loaded where the library cannot read which the query loads: an `Include` in a form it cannot read, one off the query's own chain, and a chain that reaches its rows through anything but the root's own rows — `Select(o => o.Customer)`, a `SelectMany`, a `Join`, a `GroupBy` — when it also has an include, which EF Core applies from the root to the entities it reaches, or builds an object in one of its lambdas, as a projection behind an identity `Select`, a member of an anonymous row or a conditional does. Such a chain with neither is read from the model. - A denial beneath a navigation nothing loads never leaves the database, so it asks for no projection. An entity whose only denials sit beneath such navigations is read as it was in 3.1.0. -- A member whose value can hold a field denied for `Select` that no path names — deeper than the walker's four segments, inside a framework generic such as `Dictionary`, or declared by a subtype of its type — asks for one too, read as for a named member, so on an entity only what loads counts. So does a member that can hold an object of any type: one typed `object`, a framework interface such as `IComparable`, or a collection that is not generic (`IEnumerable`, `ArrayList`, `Array`). Under a `"*"` deny, so does a member whose value can hold a path the walk never asks about. -- A row can be a subtype of `T`. On an entity, a member a type the model derives from `T` declares, and what loads beneath it, counts as one of `T`'s own would; on a row in memory, a member any loaded subtype declares does. The projection builds `T` and leaves them out, recorded with a reason starting `left out: a type derived`. A row a projection builds is exactly `T`. -- A member EF Core does not map holds only what the class puts there, never a value the query read, so it asks for nothing. +- A member whose value can hold a field denied for `Select` that no path names — deeper than the walker's four segments, inside a framework generic such as `Dictionary`, or declared by a subtype of its type — asks for one too, read as for a named member, so on an entity only what loads counts. Under a `"*"` deny, so does a member whose value can hold a path the walk never asks about and the policy does not name. +- A member that can hold an object of any type — one typed `object`, a framework interface such as `IComparable`, or a collection that is not generic, such as `IEnumerable`, `ArrayList` or an application's own — asks for nothing on its own: the policy cannot see into it whether or not a projection is built. +- A row can be a subtype of `T`. On an entity, a member a type the model derives from `T` declares, and what loads beneath it, counts as one of `T`'s own would; on a row in memory, a member any loaded subtype declares does; on a row a projection builds, a member the type its initializer constructs declares below `T`. The projection builds `T` and leaves them out, recorded with a reason starting `left out: a type derived`. A subtype is any type loaded outside the framework's assemblies that derives from the type or implements it, an open generic one and an application's subclass of `Exception` included; a rule on a path through a subtype's member counts as a rule on the declared type's own path does. +- A member EF Core does not map counts as loaded: its getter can hand out a mapped field or a private navigation, so its type is read whole. - The denials beneath a member come from the providers' rules as well as from walking the type, so a denied property with no setter, a rule on a path reached through a cycle, and a rule deeper than the walk all count. - A forced scope beneath a member asks for no projection on its own. It filters the rows that hold the member, as it always has. When a projection is needed anyway, the member is left out whole. @@ -1716,7 +1717,7 @@ A value EF Core does not map is left out: computing it would make EF Core read t **Whole, narrowed or left out whole.** A member holding an object that the source carries is: -- **kept whole** when nothing beneath it is denied, nothing its value can hold is denied (its subtypes included) or can hold an object of any type, under a `"*"` deny the walk asks about every path beneath it, no forced scope is beneath it, and no transform beneath it lands on a property with no setter; +- **kept whole** when nothing beneath it is denied, nothing its value can hold is denied (its subtypes included), it cannot hold an object of any type (asked of a projected row or a row in memory: what an entity read from the database never holds one), under a `"*"` deny every path beneath it the walk skips is one the policy names, no forced scope is beneath it, and no transform beneath it lands on a property with no setter; - **narrowed** otherwise, to the allowed fields beneath it, four segments deep, as a caller naming it would get it, where the core's narrowing translates: an object the projection's initializer builds, a list a subquery reads into a type the core can bind (not an array or a set), a navigation that is neither complex nor stored as JSON, or an entity's owned member not stored as JSON. The narrowing builds the declared type, so a subtype's fields are dropped. A field beneath it that can hold what the policy cannot name is left out; - **left out whole** otherwise, and recorded as `Dropped` on `Select` with a reason that starts `left out whole`. @@ -1728,6 +1729,7 @@ left out whole: it is stored as JSON, which EF Core cannot narrow left out whole: the projection builds it in a way the core cannot narrow left out whole: the projection would add its key 'Contents.Id', which is denied left out whole: the core cannot project 'Contents.Code' +left out whole: the core cannot build 'IContact', which it narrows into left out whole: nothing beneath it may be selected left out whole: it can hold what the policy cannot name ``` @@ -1736,7 +1738,7 @@ The last one is recorded on the field beneath the member that the narrowing leav **Never kept.** -- An entity's navigation, included or not: projecting it would load it. So once a denial needs a projection, an included or automatically included navigation is not returned. Under `Convenience`, name it in `Selects` to get it narrowed; under `Strict`, name its allowed fields. +- An entity's navigation, included or not: projecting it would load it. So once a denial needs a projection, an included or automatically included navigation is not returned, and the trace records it as `Dropped` with a reason starting `left out:`. Under `Convenience`, name it in `Selects` to get it narrowed; under `Strict`, name its allowed fields. - An object held by a row in memory: a kept object is the caller's own, and a transform beneath it would change it in place. The projection's rows are new and hold no member of an object type, so the source objects are left as they were. - A member with no setter, and a member named with one of the expression parser's own words. @@ -1750,9 +1752,9 @@ The last one is recorded on the field beneath the member that the narrowing leav - A dry run synthesizes nothing. It records the denials and returns the rows whole. - A simulation has no source, so it reads `T` as a source it cannot see into: every denial beneath a member counts, and the projection it shows keeps only members holding a value (see [Administration](#administration)). -**What the policy cannot see into.** A member typed `object`, a framework interface or a collection that is not generic, such as `IEnumerable`, `ArrayList` or `Array`, is opaque to the policy: a synthesized projection leaves it out, and naming it returns whatever it holds. A framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: naming it is refused in both tiers where the core cannot narrow it, at the top of `T` or on a row in memory, narrowed away under `Convenience` beneath a navigation, and a synthesized projection leaves it out. Hold such values in a list of the policed type instead. A member EF Core does not map holds what the class computes, and the policy reads nothing into it: a getter that copies a denied column is the application's to withhold. +**What the policy cannot see into.** A member typed `object`, a framework interface or a collection that is not generic, such as `IEnumerable`, `ArrayList` or an application's own, is opaque to the policy. It never asks for a projection; when one is needed anyway, a projected row or a row in memory leaves it out and an entity keeps it; and naming it returns whatever it holds. A framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: naming it is refused in both tiers where the core cannot narrow it, at the top of `T` or on a row in memory, narrowed away under `Convenience` beneath a navigation, and a synthesized projection leaves it out. Hold such values in a list of the policed type instead. A member EF Core does not map is read as its type, since its getter can hand out what EF Core loaded; a getter that copies a denied column into a type with no denial is the application's to withhold. -**A projection builds the declared type.** A query over the root of a hierarchy whose derived type declares a denied field comes back as root-type rows, the derived types' allowed fields dropped too. Over an abstract root the typed terminals fail with `SelectTypeMustHaveParameterlessConstructor`, and the dynamic ones return the root's members. Query the derived type, `OfType()`, to keep its fields. Under a `"*"` deny, a member is kept whole only when the walk asks about every path beneath it. +**A projection builds the declared type.** A query over the root of a hierarchy whose derived type declares a denied field comes back as root-type rows, the derived types' allowed fields dropped too. Over an abstract root the typed terminals fail with `SelectTypeMustHaveParameterlessConstructor`, and the dynamic ones return the root's members. Query the derived type, `OfType()`, to keep its fields. Rows in memory can be any loaded subtype, and the policy does not look at the rows: when a subtype declares a denied field, they are projected and their objects left out, even if no row is that subtype. Under a `"*"` deny, a member is kept whole only when every path beneath it the walk skips is one the policy names. A `[DwDenied]` on an override, or on a class's implementation of an interface member, applies to the path through the base type or the interface, on every row and in every clause. **A forced scope on a list's element type filters rows, not elements.** A forced scope declared on a list's element type filters the rows that hold the list, never its elements. `Selects` naming the list returns every element, those the scope excludes included, as in every release; a synthesized projection leaves such a list out. Scope the elements where the row is built. @@ -2316,7 +2318,7 @@ All validation errors throw `LogicException` (inherits `Exception`) with one of Fixed (security): a field denied only beneath a member, none at the top of `T`, synthesized nothing, so the whole row came back with the denied value in it — in a list or nested object of a row projected before `ApplyPolicy`, in a row held in memory, and in an entity's included, automatically included, lazily loaded or owned member — typed and dynamic, in both tiers, for a `Filter` and a `Segment`. Such a denial now synthesizes the projection whenever its value can reach the result: on an entity, beneath a column, an owned or complex member, or a navigation the query loads through `Include`, an automatic include or a lazy loader; on a projected row, beneath a member the initializer assigns; in memory, beneath any member. A denial beneath a navigation nothing loads never leaves the database, and the entity is read exactly as in 3.1.0. - Fixed (security): what a query loads was read too narrowly. An include named from the root and reached through `Select(o => o.Customer)`, `SelectMany` or `Join`, a projection behind another `Select`, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or a property of any name each loaded a denied value the gate read as unloaded. A field a subtype declares, a derived entity's or a subclass's held by a base-typed member, was not read at all, and under a `"*"` deny a path the walk never asked about was allowed. Each came back. Now every navigation counts as loaded on such a chain, the subtypes are read, and such a path is denied. When a type the model derives from `T`, or a loaded subclass of a row in memory, declares a denied field, the rows are projected to `T`, dropping a derived type's allowed fields too; over an abstract `T` the typed terminals fail with `SelectTypeMustHaveParameterlessConstructor` and the dynamic ones return its members. + Fixed (security): what a query loads was read too narrowly. An include named from the root and reached through `Select(o => o.Customer)`, `SelectMany` or `Join`, a projection behind another `Select`, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or a property of any name each loaded a denied value the gate read as unloaded. An injected `DbContext` or EF Core 7's asynchronous loader delegate did too. A field a subtype declares, a derived entity's or a subclass's held by a base-typed member, was not read at all, nor was a `[DwDenied]` on an override or on an interface member's implementation, and under a `"*"` deny a path the walk never asked about was allowed. Each came back. Now every navigation counts as loaded on such a chain, the subtypes are read, and such a path is denied. When a type the model derives from `T`, or a loaded subclass of a row in memory, declares a denied field, the rows are projected to `T`, dropping a derived type's allowed fields too; over an abstract `T` the typed terminals fail with `SelectTypeMustHaveParameterlessConstructor` and the dynamic ones return its members. Fixed (security): a field denied at the top of `T` whose own type is not a simple value — a byte array, a list, an owned object, a JSON column — synthesized no projection either, so with nothing else denied the whole row came back with it. diff --git a/DynamicWhere.ex/DynamicWhere.ex.csproj b/DynamicWhere.ex/DynamicWhere.ex.csproj index 61cab00..22cc870 100644 --- a/DynamicWhere.ex/DynamicWhere.ex.csproj +++ b/DynamicWhere.ex/DynamicWhere.ex.csproj @@ -52,7 +52,7 @@ Security fix: a field denied for Select only beneath a member was not enforced when the caller sent no Selects. A guarded query synthesizes a projection for a denied field, and it did so only when a simple field at the top of the type was denied. With every denial beneath a member, the whole row came back with the denied value in it: in a list or nested object of a row projected before ApplyPolicy, in a row held in memory, and in an entity's included, automatically included, lazily loaded or owned member, typed and dynamic, in both tiers, for a Filter and a Segment. Such a denial now synthesizes the projection whenever its value can reach the result. On an entity that means beneath a column, an owned or complex member, or a navigation the query loads through Include, an automatic include or a lazy loader; a denial beneath a navigation nothing loads never leaves the database, and the entity is read exactly as before. -Security fix: what a query loads, and what a member holds, was read too narrowly. An include named from the root and reached through Select(o => o.Customer), SelectMany or Join, a projection behind another Select, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or any property each loaded a denied value the gate read as unloaded. A field a subtype declares, a derived entity's or a subclass's held by a base-typed member, was not read at all, and under a "*" deny with exact allows a path the walk never asked about (past four segments, around a cycle, with no setter) resolved as allowed. Selects naming an entity navigation returned a denial in its owned chain past four segments or in a converted Dictionary<string, T> column. All of these are read now: from the EF Core model for an entity, so only what loads counts, and from every loaded subtype for a projected or in-memory row. Rows whose derived type declares a denied field come back as the queried type. +Security fix: what a query loads, and what a member holds, was read too narrowly. An include named from the root and reached through Select(o => o.Customer), SelectMany or Join, a projection behind another Select, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or any property each loaded a denied value the gate read as unloaded. An injected DbContext and EF Core 7's asynchronous loader delegate loaded one too. A field a subtype declares, a derived entity's, a subclass's held by a base-typed member or an open generic one's, was not read at all, nor was a [DwDenied] on an override or on an interface member's implementation, and under a "*" deny with exact allows a path the walk never asked about (past four segments, around a cycle, with no setter) resolved as allowed. Selects naming an entity navigation returned a denial in its owned chain past four segments or in a converted Dictionary<string, T> column. All of these are read now: from the EF Core model for an entity, so only what loads counts, and from every loaded subtype for a projected or in-memory row. Rows whose derived type declares a denied field come back as the queried type. A denial on an override or an implementation applies to the base type's or the interface's path, on every row and in every clause. Security fix: a field denied at the top of a type whose own type is not a simple value, such as a byte array, a list, an owned object or a JSON column, synthesized no projection either, so with nothing else denied it came back. @@ -62,7 +62,7 @@ Security fix: under the convenience tier, Selects naming a navigation whose elem Security fix: Selects naming a member typed as a collection the core does not unwrap, such as IReadOnlyList<T>, returned every field beneath it, denied ones included, in both tiers. The projection gate read collections through a narrower list than the attribute walker that puts policy on the fields beneath; it now reads them the same way. Denials beneath a named member are also read from the policy's own rules, so a denied property with no setter and a rule on a path reached through a cycle are found, and a member carrying a field denied where no path reaches it, deeper than the walker, inside a framework collection such as Dictionary<string, T> or on a subtype, is refused under the strict tier, and under the convenience tier narrowed where the core can narrow it and refused where it cannot. -Behaviour change: the projection synthesized for a guarded query that sends no Selects keeps what the source carries. It used to keep simple fields only, so every nested object and list of a row projected before ApplyPolicy came back null or empty as soon as any field was denied. A row a projection builds keeps the members its initializer assigns. An entity keeps its mapped columns, converted and JSON ones included, its owned and complex members, and every collection of simple values such as byte[] or List<string>. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, recorded as Dropped with a reason starting "left out whole". An entity's navigations, the objects of a row in memory, and a value EF Core does not map are left out, and the type needs a public parameterless constructor for the typed projection, as it already did. A member EF Core does not map asks for no projection. +Behaviour change: the projection synthesized for a guarded query that sends no Selects keeps what the source carries. It used to keep simple fields only, so every nested object and list of a row projected before ApplyPolicy came back null or empty as soon as any field was denied. A row a projection builds keeps the members its initializer assigns. An entity keeps its mapped columns, converted and JSON ones included, its owned and complex members, and every collection of simple values such as byte[] or List<string>. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, recorded as Dropped with a reason starting "left out whole". An entity's navigations, the objects of a row in memory, and a value EF Core does not map are left out, and the type needs a public parameterless constructor for the typed projection, as it already did. A member that can hold an object of any type, a geometry, a JSON bag or a BitArray column say, asks for no projection on its own; a member EF Core does not map is read as its type, since its getter can hand out what EF Core loaded; and a chain reaching its rows through a navigation, SelectMany, Join or GroupBy counts every navigation as loaded only when it has an include or builds an object. Behaviour change: [DwEntity(DefaultOrder)] applies to a projected source whose outermost Select builds the type in an object initializer and assigns every field the default names a column, at every level of a nested path: a mapped member read directly, through reference navigations or through EF.Property. A computed value or any other projection still leaves the query in its own order. A Select, or a Filter with Selects, composed on the guarded handle keeps the rest of the chain unordered, and a composed Filter that sent orders gets no default later in the chain, as a composed Order already did not. diff --git a/OfficialWebsite/app/docs/breaking-changes/page.tsx b/OfficialWebsite/app/docs/breaking-changes/page.tsx index 543138f..a3509fc 100644 --- a/OfficialWebsite/app/docs/breaking-changes/page.tsx +++ b/OfficialWebsite/app/docs/breaking-changes/page.tsx @@ -1116,16 +1116,17 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say through an Include, an automatic include or a lazy loader; on a projected row, beneath a member the initializer assigns; in memory, beneath any member. A chain that reaches its rows through - a navigation, a SelectMany, a Join or a - projection behind another Select counts every navigation - as loaded. A field a subtype of T declares, and one a - subtype of a member's type declares, count too. It does so in both - tiers, typed and dynamic, for a Filter and a{" "} - Segment. Until 3.2.0 only a simple field denied at the top - of T asked for one. A denial beneath a navigation nothing - loads never leaves the database, and a member EF Core does not map - holds nothing it read, so an entity whose only denials sit there is - read exactly as in 3.1.0. + a navigation, a SelectMany, a Join or a{" "} + GroupBy counts every navigation as loaded when it also has + an include or builds an object in a lambda. A field a subtype of{" "} + T declares, one a subtype of a member's type + declares, and one beneath a member EF Core does not map, count too. A + member that can hold an object of any type asks for nothing on its + own. It does so in both tiers, typed and dynamic, for a{" "} + Filter and a Segment. Until 3.2.0 only a + simple field denied at the top of T asked for one. A + denial beneath a navigation nothing loads never leaves the database, so + an entity whose only denials sit there is read exactly as in 3.1.0.
    • What. The allowed members, which replace the allowed @@ -1154,9 +1155,12 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say Join, a projection behind another Select, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or a property of any name each - loaded a denied value the gate read as unloaded. A field a subtype - declares — a derived entity's, or a subclass's held by a - base-typed member — was not read at all, and under a{" "} + loaded a denied value the gate read as unloaded, and so did an injected{" "} + DbContext or EF Core 7's asynchronous loader delegate. + A field a subtype declares — a derived entity's, or a + subclass's held by a base-typed member — was not read at all, nor + was a [DwDenied] on an override or on an interface + member's implementation, and under a{" "} "*" deny a path the walk never asked about was allowed. Each came back. @@ -1168,7 +1172,11 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say T the typed terminals fail with{" "} SelectTypeMustHaveParameterlessConstructor; the dynamic ones return its members. Query the derived type,{" "} - {`OfType()`}, to keep its fields. + {`OfType()`}, to keep its fields. Rows in memory + can be any loaded subtype, so there the rows are projected whenever one + declares a denied field. A [DwDenied] on an override or on + an interface member's implementation denies the base path for + every row, in every clause. A field denied at the top of T whose own type is not a @@ -1301,9 +1309,10 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say A member typed object, a framework interface or a collection that is not generic, such as IEnumerable,{" "} - ArrayList or Array, is opaque to the policy: a - synthesized projection leaves it out, and naming it returns whatever it - holds. A framework generic holding a policed type, such as{" "} + ArrayList or an application's own, is opaque to the + policy: it never asks for a projection, a synthesized projection over a + projected row or rows in memory leaves it out, and naming it returns + whatever it holds. A framework generic holding a policed type, such as{" "} Dictionary<string, LineDto>, has no paths beneath it: naming it is refused in both tiers where the core cannot narrow it, narrowed away under Convenience beneath a navigation, and a diff --git a/OfficialWebsite/app/docs/policies/attributes/page.tsx b/OfficialWebsite/app/docs/policies/attributes/page.tsx index 0d666d8..92cebda 100644 --- a/OfficialWebsite/app/docs/policies/attributes/page.tsx +++ b/OfficialWebsite/app/docs/policies/attributes/page.tsx @@ -112,7 +112,8 @@ public class Ticket read directly (t.Code), through reference navigations (t.Owner.Name) or through EF.Property, a shadow property included; in memory any assigned field is one. EF Core - then translates the order. + then translates the order. A projection over an anonymous or other + intermediate row reads no entity, so it takes no default.
    • Any other projection leaves the query in its own order, as every @@ -232,6 +233,16 @@ public string EmployeeCode { get; set; }`} query project the allowed members instead. See{" "} A request that sends no Selects. + + An access-control attribute on another declaration of a member applies + to its path too: on the interface member a class implements, on an + override a subtype declares, and on the implementation a type gives an + interface member. A row read through the base type or the interface is + still that subtype, so the denial holds for the path on every row, in + every clause. Until 3.2.0 only the declaration walked, and the + attributes above it, were read. The other attributes are still read from + the declaration walked only. +

      Injection

      diff --git a/OfficialWebsite/app/docs/policies/configuration/page.tsx b/OfficialWebsite/app/docs/policies/configuration/page.tsx index 6be4d91..911f7ce 100644 --- a/OfficialWebsite/app/docs/policies/configuration/page.tsx +++ b/OfficialWebsite/app/docs/policies/configuration/page.tsx @@ -117,13 +117,17 @@ export default function Page() { subclass, an interface's implementation. What it can carry is read from the source. On an entity it is read from the EF Core model, so only what loads counts: the navigation's columns, a converted one - included, its owned chain at any depth, and the navigations beneath it - an include, an automatic include or a lazy loader fills, for its type - and every type the model derives from it. On a projected or in-memory - row it is the member's type and every loaded subtype of it. Under a - policy with a "*" deny, a path the walk never asks - about — past four segments, around a cycle, with no setter, or on a - subtype — is a denied one as well. The Strict tier refuses + included, its owned chain at any depth, the navigations beneath it an + include, an automatic include or a lazy loader fills, and each member + the model does not map, read as its type, since its getter can hand + out what EF Core loaded — for its type and every type the model + derives from it. On a projected row it is the type the initializer + constructs the member as, when it says, and otherwise the + member's type and every loaded subtype of it, as on a row in + memory. Under a policy with a "*" deny, a path the + walk never asks about — past four segments, with no setter, or on a + subtype — is a denied one unless the policy names it; around a cycle it + always is. The Strict tier refuses such a member. The Convenience tier narrows it where the core can, which builds the declared type and so drops a subtype's fields; a path naming a framework generic itself narrows to nothing and @@ -179,13 +183,15 @@ export default function Page() { complex member, or a navigation something loads: an{" "} Include or ThenInclude on the query, an automatic include, or a lazy loader — EF Core's proxies, an - injected ILazyLoader, or a loader delegate or{" "} + injected ILazyLoader, a loader delegate or{" "} ILazyLoader the constructor takes and keeps in a field or - any property — which fills a navigation after the query. On a row a - projection builds, it is beneath a member the initializer assigns; a - constructor with arguments, with or without an initializer after it, - counts every member as assigned. On a row in memory, it is beneath any - member. A rule may spell the path in any letter case. + any property, the asynchronous loader delegate of EF Core 7, or an + injected DbContext — which fills a navigation after the + query. On a row a projection builds, it is beneath a member the + initializer assigns; a constructor with arguments counts every member + as assigned, and an initializer after it still says what its own + bindings hold. On a row in memory, it is beneath any member. A rule may + spell the path in any letter case.
    • Every navigation counts as loaded where the library cannot read which @@ -193,10 +199,11 @@ export default function Page() { off the query's own chain, and a chain that reaches its rows through anything but the root's own rows —{" "} {`Select(o => o.Customer)`}, a SelectMany, a{" "} - Join, or a projection behind another Select{" "} - (an identity Select, a member of an anonymous row, a - conditional). EF Core still applies includes named from the root to - the entities such a chain reaches. + Join, a GroupBy — when it also has an + include, which EF Core applies from the root to the entities it + reaches, or builds an object in one of its lambdas, as a projection + behind an identity Select, a member of an anonymous row or + a conditional does. Such a chain with neither is read from the model.
    • A denial beneath a navigation nothing loads never leaves the database, @@ -209,25 +216,36 @@ export default function Page() { inside a framework generic such as{" "} Dictionary<string, T>, or declared by a subtype of its type — asks for one too, read as for a named member, so on an - entity only what loads counts. So does a member that can hold an object - of any type: one typed object, a framework interface such - as IComparable, or a collection that is not generic - (IEnumerable, ArrayList,{" "} - Array). Under a "*" deny, so does a - member whose value can hold a path the walk never asks about. + entity only what loads counts. Under a{" "} + "*" deny, so does a member whose value can hold a + path the walk never asks about and the policy does not name. +
      • +
    • + A member that can hold an object of any type — one typed{" "} + object, a framework interface such as{" "} + IComparable, or a collection that is not generic, such + as IEnumerable, ArrayList or an + application's own — asks for nothing on its own: the policy cannot + see into it whether or not a projection is built.
    • A row can be a subtype of T. On an entity, a member a type the model derives from T declares, and what loads beneath it, counts as one of T's own would; on a row in memory, - a member any loaded subtype declares does. The projection builds{" "} - T and leaves them out, recorded with a reason starting{" "} - left out: a type derived. A row a projection builds is - exactly T. + a member any loaded subtype declares does; on a row a projection + builds, a member the type its initializer constructs declares below{" "} + T. The projection builds T and leaves them + out, recorded with a reason starting{" "} + left out: a type derived. A subtype is any type loaded + outside the framework's assemblies that derives from the type or + implements it, an open generic one and an application's subclass + of Exception included; a rule on a path through a + subtype's member counts as a rule on the declared type's own + path does.
    • - A member EF Core does not map holds only what the class puts there, - never a value the query read, so it asks for nothing. + A member EF Core does not map counts as loaded: its getter can hand out + a mapped field or a private navigation, so its type is read whole.
    • The denials beneath a member come from the providers' rules as @@ -285,10 +303,12 @@ export default function Page() {
      • kept whole when nothing beneath it is denied, nothing - its value can hold is denied (its subtypes included) or can hold an - object of any type, under a "*" deny the walk asks - about every path beneath it, no forced scope is beneath it, and no - transform beneath it lands on a property with no setter; + its value can hold is denied (its subtypes included), it cannot hold an + object of any type (asked of a projected row or a row in memory: what + an entity read from the database never holds one), under a{" "} + "*" deny every path beneath it the walk skips is + one the policy names, no forced scope is beneath it, and no transform + beneath it lands on a property with no setter;
      • narrowed otherwise, to the allowed fields beneath it, @@ -314,6 +334,7 @@ left out whole: it is stored as JSON, which EF Core cannot narrow left out whole: the projection builds it in a way the core cannot narrow left out whole: the projection would add its key 'Contents.Id', which is denied left out whole: the core cannot project 'Contents.Code' +left out whole: the core cannot build 'IContact', which it narrows into left out whole: nothing beneath it may be selected left out whole: it can hold what the policy cannot name`}

        @@ -326,9 +347,10 @@ left out whole: it can hold what the policy cannot name`}

      • An entity's navigation, included or not: projecting it would load it. So once a denial needs a projection, an included or automatically - included navigation is not returned. Under Convenience, - name it in Selects to get it narrowed; under{" "} - Strict, name its allowed fields. + included navigation is not returned, and the trace records it as{" "} + Dropped with a reason starting left out:. + Under Convenience, name it in Selects to get + it narrowed; under Strict, name its allowed fields.
      • An object held by a row in memory: a kept object is the caller's @@ -385,15 +407,15 @@ left out whole: it can hold what the policy cannot name`} A member typed object, a framework interface or a collection that is not generic, such as IEnumerable,{" "} - ArrayList or Array, is opaque to the policy: a - synthesized projection leaves it out, and naming it returns whatever it - holds. A framework generic holding a policed type, such as{" "} - Dictionary<string, LineDto>, has no paths beneath it: - naming it is refused in both tiers where the core cannot narrow it, at - the top of T or on a row in memory, narrowed away under{" "} - Convenience beneath a navigation, and a synthesized - projection leaves it out. Hold such values in a list of the policed type - instead. + ArrayList or an application's own, is opaque to the + policy. It never asks for a projection; when one is needed anyway, a + projected row or a row in memory leaves it out and an entity keeps it; + and naming it returns whatever it holds. A framework generic holding a + policed type, such as Dictionary<string, LineDto>, has + no paths beneath it: naming it is refused in both tiers where the core + cannot narrow it, narrowed away under Convenience beneath a + navigation, and a synthesized projection leaves it out. Hold such values + in a list of the policed type instead. A query over the root of a hierarchy whose derived type declares a @@ -401,9 +423,12 @@ left out whole: it can hold what the policy cannot name`} allowed fields dropped too. Over an abstract root the typed terminals fail with SelectTypeMustHaveParameterlessConstructor, and the dynamic ones return the root's members. Query the derived type,{" "} - {`OfType()`}, to keep its fields. Under a{" "} - "*" deny, a member is kept whole only when the walk - asks about every path beneath it. + {`OfType()`}, to keep its fields. Rows in memory + can be any loaded subtype, and the policy does not look at the rows: + when a subtype declares a denied field, they are projected and their + objects left out, even if no row is that subtype. Under a{" "} + "*" deny, a member is kept whole only when every + path beneath it the walk skips is one the policy names. A forced scope declared on a list's element type filters the rows diff --git a/OfficialWebsite/app/docs/policies/security/page.tsx b/OfficialWebsite/app/docs/policies/security/page.tsx index f5c1023..cc2e4bf 100644 --- a/OfficialWebsite/app/docs/policies/security/page.tsx +++ b/OfficialWebsite/app/docs/policies/security/page.tsx @@ -290,10 +290,23 @@ true order. Add [DwNoOrder] unless that is intended.`}
      • + + + + @@ -319,16 +332,18 @@ true order. Add [DwNoOrder] unless that is intended.`} A member typed object, a framework interface or a collection that is not generic, such as IEnumerable,{" "} - ArrayList or Array, is opaque to the policy: a - synthesized projection leaves it out, and naming it returns whatever it - holds. A framework generic holding a policed type, such as{" "} + ArrayList or an application's own, is opaque to the + policy: it never asks for a projection, a synthesized projection over a + projected row or rows in memory leaves it out, and naming it returns + whatever it holds. A framework generic holding a policed type, such as{" "} Dictionary<string, LineDto>, has no paths beneath it: naming it is refused in both tiers where the core cannot narrow it, narrowed away under Convenience beneath a navigation, and a synthesized projection leaves it out. Hold such values in a list of the - policed type instead. A member EF Core does not map holds what the class - computes, and the policy reads nothing into it: a getter that copies a - denied column is the application's to withhold. + policed type instead. A member EF Core does not map is read as its type, + since its getter can hand out what EF Core loaded; a getter that copies a + denied column into a type with no denial is the application's to + withhold. A forced scope declared on a list's element type filters the rows diff --git a/OfficialWebsite/public/llms.txt b/OfficialWebsite/public/llms.txt index f869b1f..11ab5e9 100644 --- a/OfficialWebsite/public/llms.txt +++ b/OfficialWebsite/public/llms.txt @@ -1976,6 +1976,12 @@ No named attribute refuses `Segment`: write `[DwDeny(PolicyFeature.Segment)]`. - `[DwDeny(PolicyFeature.None)]` refuses nothing, and nothing reports it. - `[DwNoSelect]` leaves the field filterable, sortable, groupable and aggregatable, so it can still be counted. +- One on another declaration of the member applies to the path too (3.2.0): on the interface member a class + implements, on an override a loaded subtype declares, and on the implementation a loaded type gives an interface + member. A row read through the base type or the interface is still that subtype, and its member returns what the + subtype's declaration returns, so the denial holds for the path on every row, Where, Order, Group and Select + alike. Only the deny family is read this way; aliases, transforms and the other attributes are read from the + declaration walked. ### DwOperators @@ -2327,10 +2333,13 @@ DefaultOrder field denied for Segment, in a Segment (3.1.0) left out l inside a framework generic such as `Dictionary`, or declared by a subtype of the member's type (a derived entity, a subclass, an interface's implementation). What it can carry is read from the source. On an entity it is read from the EF Core model, so only what loads counts: the navigation's columns, a converted one - included, its owned chain at any depth, and the navigations beneath it that an include, an automatic include or - a lazy loader fills, for its type and every type the model derives from it. On a projected or in-memory row it - is the member's type and every loaded subtype of it. Under a policy with a `"*"` deny, a path the walk never - asks about (past four segments, around a cycle, with no setter, or on a subtype) is a denied one as well. + included, its owned chain at any depth, the navigations beneath it that an include, an automatic include or a + lazy loader fills, and each member the model does not map, read as its type, since its getter can hand out what + EF Core loaded; for its type and every type the model derives from it. On a projected row it is the type the + initializer constructs the member as, when it says (`Contact = new ContactRow { … }`), and otherwise the + member's type and every loaded subtype of it, as on a row in memory. Under a policy with a `"*"` deny, a path + the walk never asks about (past four segments, with no setter, or on a subtype) is a denied one unless the + policy names it; around a cycle, where the paths never end, it always is. - Such a member is refused with `FieldDeniedForSelect` under Strict. Under Convenience it is narrowed to the allowed leaves where the core can narrow the path's first member, which builds the declared type and so drops a subtype's fields; a path that names a framework generic itself narrows to nothing and is dropped. Where the core @@ -2352,28 +2361,37 @@ DefaultOrder field denied for Segment, in a Segment (3.1.0) left out l case. - On an entity: beneath a column, an owned or complex member, or a navigation something loads. That is an `Include` or `ThenInclude` on the query, an automatic include, or a lazy loader: proxies, an injected - `ILazyLoader`, or a loader delegate or `ILazyLoader` the constructor takes, kept in a field or any property, - whose navigations fill after the query. Every navigation counts as loaded when the library cannot read which - the query loads: an include in a form it cannot read, an include off the query's own chain (on a join's inner - source, say), and a chain that reaches its rows through anything but the root's own rows, such as - `Select(o => o.Customer)`, a `SelectMany`, a `Join`, or a projection behind another `Select` (an identity - `Select`, a member of an anonymous row, a conditional), since EF Core applies includes named from the root to - the entities it reaches. A denial beneath a navigation nothing loads never leaves the database and needs no - projection, so a connected model is read as it was in 3.1.0. A member EF Core does not map holds only what the - class puts there, never a value the query read, and asks for nothing. - - On a row a projection builds: beneath a member its initializer assigns. A constructor with arguments, with or - without an initializer after it, counts every member as assigned. + `ILazyLoader`, a loader delegate or `ILazyLoader` the constructor takes, kept in a field or any property, the + asynchronous loader delegate EF Core 7 added, or an injected `DbContext`, any of which fills a navigation + after the query. Every navigation counts as loaded when the library cannot read which the query loads: an + include in a form it cannot read, an include off the query's own chain (on a join's inner source, say), and a + chain that reaches its rows through anything but the root's own rows (`Select(o => o.Customer)`, a + `SelectMany`, a `Join`, a `GroupBy`) when it also has an include, which EF Core applies from the root to the + entities it reaches, or builds an object in one of its lambdas (a projection behind an identity `Select`, a + member of an anonymous row, a conditional), which loads whatever it assigns. Such a chain with neither is read + from the model. A denial beneath a navigation nothing loads never leaves the database and needs no projection, + so a connected model is read as it was in 3.1.0. A member EF Core does not map counts as loaded: its getter can + hand out a mapped field or a private navigation, so its type is read whole. + - On a row a projection builds: beneath a member its initializer assigns. A constructor with arguments counts + every member as assigned; an initializer after it still says what its own bindings hold. - On a row in memory: beneath any member. - So does a member whose value can hold a field denied for Select that no path names (deeper than the walker's four segments, inside a framework generic such as `Dictionary`, or declared by a subtype of its - type), read as for a named member above, so on an entity only what loads counts. So does a member that can hold - an object of any type: one typed `object`, a framework interface such as `IComparable`, or a collection that is - not generic (`IEnumerable`, `ArrayList`, `Hashtable`, `Array`). Under a policy with a `"*"` deny, so does a - member whose value can hold a path the walk never asks about. + type), read as for a named member above, so on an entity only what loads counts. Under a policy with a `"*"` + deny, so does a member whose value can hold a path the walk never asks about and the policy does not name. + - A member that can hold an object of any type, one typed `object`, a framework interface such as `IComparable`, + an unbound type parameter, or a collection that is not generic (`IEnumerable`, `ArrayList`, `Array`, an + application's own), asks for nothing on its own: the policy cannot see into it whether or not a projection is + built. - A row can be a subtype of T. On an entity, each member a type the model derives from T declares, and what loads - beneath it, counts as one of T's own would; on a row in memory, each member any loaded subtype declares. The - projection builds T, so it leaves them all out, recorded as `Dropped` with a reason starting `left out: a type - derived`. A row a projection builds is exactly T. + beneath it, counts as one of T's own would; on a row in memory, each member any loaded subtype declares; on a + row a projection builds, each member the type its initializer constructs declares below T. The projection builds + T, so it leaves them all out, recorded as `Dropped` with a reason starting `left out: a type derived`. + - A subtype is any type loaded outside the framework's own assemblies that derives from the type or implements it: + an open generic one, `Tagged : Creature`, and an application's subclass of a framework class, an `Exception` + or a `Stream`, included. A rule on a path through a subtype's member (`Org.Swift`, where `Swift` is the bank's), + or through one of two members whose names differ only in letter case, counts beneath a member as a rule on the + declared type's own path does. - The denials beneath a member come from the providers' fragments as well as from walking the type, so a denied property with no setter, a rule on a path reached through a cycle, and a rule deeper than the walk all count. - A forced scope beneath a member asks for no projection on its own: it filters the rows that hold the member, as it @@ -2386,9 +2404,10 @@ DefaultOrder field denied for Segment, in a Segment (3.1.0) left out l read the whole entity, the denied columns included, and it holds only its initial value anyway; - every allowed member holding an object or a list of them that the source carries: a member a projection's initializer assigns, and an entity's columns (converted or JSON), owned and complex members. It is kept whole - when nothing beneath it is denied, nothing its value can hold is denied or can hold an object of any type, - under a `"*"` deny the walk asks about every path beneath it, no forced scope is beneath it, and no transform - beneath it lands on a property with no setter; + when nothing beneath it is denied, nothing its value can hold is denied, it cannot hold an object of any type + (asked of a projected row or a row in memory: what an entity read from the database never holds one), under a + `"*"` deny every path beneath it the walk skips is one the policy names, no forced scope is beneath it, and no + transform beneath it lands on a property with no setter; - otherwise it is narrowed to the allowed leaves beneath it, to four segments, as a caller naming it would get, when the core's narrowing translates: an object the projection's initializer builds, a list a subquery reads into a type the core can bind (not an array or a set), a navigation that is neither complex nor stored as JSON, @@ -2397,14 +2416,16 @@ DefaultOrder field denied for Segment, in a Segment (3.1.0) left out l - otherwise it is left out whole, recorded as `Dropped` on `Select` with a reason starting `left out whole`: a scope forced beneath it; a column, complex property or JSON-stored member, which EF Core reads whole; one the projection builds some other way, by a constructor with arguments, a conditional or an unassigned member; a - denied key the core's projection would add back; a path the core cannot project; nothing beneath it left to - select; + denied key the core's projection would add back; a path the core cannot project; a node type the core cannot + construct (an interface, an abstract class, one with no public parameterless constructor); nothing beneath it + left to select; - Never kept: an entity's navigation, included or not, since projecting it would load it (under Convenience, name it in `Selects` to get it narrowed; under Strict, name its allowed fields); an object held by a row in memory, since a kept object is the caller's own and a transform would change it in place; a member with no setter; a - member named with one of the parser's words. + member named with one of the parser's words. Each one the unguarded call would have returned, an included + navigation or an object in memory, is recorded as `Dropped` with a reason starting `left out:` (3.2.0). - A `Select` handing back an entity, `Select(o => o.Customer)`, builds no row and is read as an entity query, with - every navigation counted as loaded (above). + every navigation counted as loaded when the query has an include (above). - A narrowed reference that is null in the source comes back as an empty object, as it does for a caller's own dotted `Selects`. - A narrowed member carries every allowed leaf beneath it. An entity reached beneath it has its own navigations @@ -2843,6 +2864,8 @@ What is recorded Dropped a member a synthesized projection leaves out whole (Select); Reason starts "left out whole" (3.2.0) Dropped a member a type derived from T declares, which a synthesized projection building T leaves out (Select); Reason starts "left out: a type derived" (3.2.0) + Dropped a member a synthesized projection cannot keep that the unguarded call would have returned, an + included navigation or an object of a row in memory (Select); Reason starts "left out:" (3.2.0) Dropped a member a Convenience caller named that can hold a denied field no path names (Select); Reason "it holds a field denied for Select where no path can name it" (3.2.0) Denied a refusal: Strict denial, cap, cost, query string ("*"), required filter, segment inference, @@ -4751,19 +4774,27 @@ MemoryCalculationInput the root and re-rooted by Select(o => o.Customer), SelectMany or Join, which EF Core still applies, and a projection behind another Select (an identity Select, a member of an anonymous row, a conditional). So does a lazy loader the constructor takes, delegate or ILazyLoader, kept in a field or a property of any name, and - an initializer after a constructor with arguments counts every member as assigned. Each returned the denied - value. + an initializer after a constructor with arguments counts every member as assigned. So do an injected + DbContext and EF Core 7's asynchronous loader delegate. Each returned the denied value. A reshaped chain + with no include and no object built in its lambdas is still read from the model, so it is not projected + for a denial beneath a navigation it does not load. - Security fix and behaviour change. A member declared as a base type or an interface holds its subtypes, whose denied fields the declared type never names. They are read now: the types the EF Core model derives, - for an entity, and every loaded subtype for a projected or in-memory row. A query over the root of a - hierarchy whose derived type declares a denied field, an included or named base-typed navigation, and a - base-typed member of a row, all returned it. Such rows are projected to T and such members narrowed to the - declared type, which drops the subtype's fields, allowed ones too. Over an abstract T the typed terminals - then fail with SelectTypeMustHaveParameterlessConstructor, as for any T they cannot build; the dynamic - terminals return the root's allowed members. + for an entity, and every loaded subtype for a projected or in-memory row, an open generic one and an + application's subclass of a framework class included. A query over the root of a hierarchy whose derived + type declares a denied field, an included or named base-typed navigation, a base-typed member of a row, and + a projection constructing a subtype of T, all returned it. Such rows are projected to T and such members + narrowed to the declared type, which drops the subtype's fields, allowed ones too. Over an abstract T the + typed terminals then fail with SelectTypeMustHaveParameterlessConstructor, as for any T they cannot build; + the dynamic terminals return the root's allowed members. A rule on a subtype's field through a base-typed + member was dropped as naming nothing; it is enforced. + - Security fix. A deny-family attribute on an override, on the implementation of an interface member, or on + the interface member a class implements, was read only from its own declaration, so the base type's or the + interface's path filtered, sorted, grouped and returned the value. It applies to the path now. - Security fix. Under a "*" deny with exact allows, a path the walk never asked about (past four segments, - around a cycle, with no setter) resolved as allowed, so a member holding one was returned whole, named or - not. Such a member is refused, narrowed or projected as one with a denial beneath it is. + around a cycle, with no setter, on a subtype) resolved as allowed, so a member holding one was returned + whole, named or not. Each such path is asked of the policy now; one it does not name is denied, and a + member holding one is refused, narrowed or projected as one with a denial beneath it is. - Security fix. A field denied at the top of T whose type is not a simple value (a blob, a list, an owned object or a JSON column, say) synthesized no projection either, so with nothing else denied it came back. - Security fix. The attribute walker read any namespace starting with "System" as the framework's, so an @@ -4789,7 +4820,11 @@ MemoryCalculationInput 3.1.0 all of these came back null or empty whenever a field was denied. A member is kept whole when nothing it can hold is denied, narrowed around a denial where the core's narrowing translates, and otherwise left out whole with a "left out whole" Dropped decision. An entity's navigations and the objects of a row in memory - are left out, as before; a value EF Core does not map is left out too, and asks for nothing. + are left out, as before, and each one the unguarded call would have returned is recorded as Dropped; a + value EF Core does not map is left out too. A member that can hold an object of any type, a geometry, a + JSON bag or a BitArray column say, asks for no projection on its own, and an entity keeps it whole. A + projected member is read as the type its initializer constructs, and an initializer after a constructor + with arguments narrows its own bindings. - Behaviour change. [DwEntity(DefaultOrder)] applies to a projected source whose outermost Select builds T in an object initializer assigning every field the default names a column: a mapped member, read directly, through reference navigations or through EF.Property. A computed value or any other projection still leaves @@ -4965,7 +5000,9 @@ MemoryCalculationInput naming the list returns every element, those the scope excludes included, as in every release; a synthesized projection leaves such a list out. Scope the elements where the row is built. - A member typed `object`, a framework interface or a collection that is not generic (`IEnumerable`, `ArrayList`, - `Array`) is opaque to the policy: a synthesized projection leaves it out, and naming it returns whatever it holds. + `Array`, an application's own) is opaque to the policy. It never asks for a projection; when one is needed anyway, + a projected row or a row in memory leaves it out and an entity keeps it (what EF Core read from the database holds + no application object); and naming it returns whatever it holds. A framework generic holding a policed type (`Dictionary`) has no paths beneath it: naming it is refused in both tiers where the core cannot narrow it (at the top of T, or on a row in memory), narrowed away under Convenience beneath a navigation, and a synthesized projection leaves it out. @@ -4974,10 +5011,18 @@ MemoryCalculationInput terminals fail with `SelectTypeMustHaveParameterlessConstructor` and the dynamic ones return the root's members. Query the derived type (`OfType()`) to keep its fields. Subtypes are read from the assemblies loaded when the query runs, which hold every type a row can have. -- Under a `"*"` deny, a member is kept whole only when the walk asks about every path beneath it: a type with a - property that has no setter, a cycle, a subtype or anything past four segments is narrowed, left out or refused. -- An entity member EF Core does not map holds what the class computes, and the policy reads nothing into it: a - getter that copies a denied column is the application's to withhold. +- Under a `"*"` deny, a member is kept whole only when every path beneath it the walk skips, one with no setter, + on a subtype or past four segments, is one the policy names; around a cycle it never is. Otherwise it is + narrowed, left out or refused. +- A member EF Core does not map counts as loaded and is read as its type, since its getter can hand out what EF + Core loaded: a denied field in its type asks for a projection, which leaves the member out, since the projection + cannot assign it. A getter that copies a denied column into a type with no denial is the application's to + withhold. +- Rows in memory can be any loaded subtype, and the policy does not look at the rows. When a subtype of T declares + a denied field, or a subtype or implementation of a member's type does, the rows are projected and their objects + left out, even if no row is that subtype. +- A default order reaches a projected row only through columns of the entity its `Select` reads: a projection over + an anonymous or other intermediate row takes no default. - A simulation reads T as a source it cannot see into (section 23): every denial beneath a member counts, and a synthesized `Clause.Selects` keeps only members holding a value. diff --git a/README.md b/README.md index 1315965..000daf7 100644 --- a/README.md +++ b/README.md @@ -380,16 +380,16 @@ The complete reference — every enum, class, extension method, validation rule, **Upgrade note — the security fixes and the three changes alter what code written for 3.1.0 does, and the new overloads can stop a call from compiling. Read these before bumping.** - **Fixed (security): a field denied beneath a member reached a caller who sent no `Selects`.** A guarded query synthesizes a projection for a denied field, and it did so only when a simple field at the top of the type was denied. With every denial beneath a member, the whole row came back with the denied value in it: in a list or nested object of a row projected before `ApplyPolicy`, in a row held in memory, and in an entity's included, automatically included, lazily loaded or owned member — typed and dynamic, in both tiers, for a `Filter` and a `Segment`. Such a denial now synthesizes the projection whenever its value can reach the result. On an entity that means beneath a column, an owned or complex member, or a navigation the query loads through `Include`, an automatic include or a lazy loader. A denial beneath a navigation nothing loads never leaves the database, and the entity is read exactly as before. -- **Fixed (security): what a query loads, and what a member holds, was read too narrowly.** An include named from the root and reached through `Select(o => o.Customer)`, `SelectMany` or `Join`, a projection behind another `Select`, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or any property each loaded a denied value the gate read as unloaded. A field a subtype declares — a derived entity's, a subclass's held by a base-typed member — was not read at all, and under a `"*"` deny with exact allows a path the walk never asked about (past four segments, around a cycle, with no setter) resolved as allowed. `Selects` naming an entity navigation returned a denial in its owned chain past four segments or in a converted `Dictionary` column. All of these are read now: from the EF Core model for an entity, so only what loads counts, and from every loaded subtype for a projected or in-memory row. +- **Fixed (security): what a query loads, and what a member holds, was read too narrowly.** An include named from the root and reached through `Select(o => o.Customer)`, `SelectMany` or `Join`, a projection behind another `Select`, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or any property each loaded a denied value the gate read as unloaded. An injected `DbContext` and EF Core 7's asynchronous loader delegate loaded one too. A field a subtype declares — a derived entity's, a subclass's held by a base-typed member, an open generic one's — was not read at all, nor was a `[DwDenied]` on an override or on an interface member's implementation, and under a `"*"` deny with exact allows a path the walk never asked about (past four segments, around a cycle, with no setter) resolved as allowed. `Selects` naming an entity navigation returned a denial in its owned chain past four segments or in a converted `Dictionary` column. All of these are read now: from the EF Core model for an entity, so only what loads counts, and from every loaded subtype for a projected or in-memory row. A denial on an override or an implementation applies to the base type's or the interface's path, on every row and in every clause. - **Fixed (security): a denied member that holds no simple value came back.** A field denied at the top of the type whose own type is not a simple value — a byte array, a list, an owned object, a JSON column — synthesized no projection either, so with nothing else denied it came back. - **Fixed (security): an application namespace starting with `System` got no policy.** The attribute walker read any namespace starting with "System" as the framework's, so an application namespace such as `SystemsCorp.Payroll` got no policy beneath its types, and a `[DwDenied]` field there was returned, filterable and sortable. Only `System` and the namespaces beneath it are the framework's now. - **Fixed (security): a navigation narrowed around its own denied key got the key back.** Under the convenience tier, `Selects` naming a navigation whose key (`Id`) is denied was narrowed to the allowed fields beneath it, and the core's typed projection added the key back. Such a narrowing is refused with `FieldDeniedForSelect` in both tiers, as naming a sibling of the key already was. A navigation named through another, such as `Main.Lead`, now gates the key of `Main`, which the projection adds; it did not. - **Fixed (security): `Selects` could name a member whose denials the gate did not see.** A member typed as a collection the core does not unwrap — `IReadOnlyList`, `IReadOnlyCollection`, `Collection` or an application's own — returned every field beneath it, denied ones included, in both tiers, because the projection gate read collections through a narrower list than the attribute walker. It reads them the same way now, and a narrowing the core cannot project is refused with `FieldDeniedForSelect`. Denials beneath a named member are also read from the policy's own rules, so a denied property with no setter and a rule on a path reached through a cycle are found. A member carrying a field denied where no path reaches it — deeper than the walker, inside a framework collection such as `Dictionary`, or on a subtype — is refused under the strict tier, and under the convenience tier narrowed where the core can narrow it and refused where it cannot. -- **Changed: the synthesized projection keeps what the source carries.** It kept simple fields only, so every nested object and list of a row projected before `ApplyPolicy` came back null or empty as soon as any field was denied. A row a projection builds — the outermost `Select` constructs it, in an object initializer or with a constructor, as in `db.Roles.Select(r => new RoleRow { … })` — keeps the members its initializer assigns. An entity, or a `Select` that hands back an entity such as `db.Orders.Select(o => o.Customer)`, keeps its mapped columns, converted and JSON ones included, its owned and complex members, and every collection of simple values such as `byte[]` or `List`. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, recorded as `Dropped` with a reason starting `left out whole`. An entity's navigations, the objects of a row in memory, and a value EF Core does not map are left out, and the type needs a public parameterless constructor for the typed projection, as it already did. When a derived type in the model, or a loaded subclass of a row in memory, declares a denied field, the rows come back as the queried type, so a derived type's allowed fields are dropped too; query the derived type with `OfType()` to keep them. +- **Changed: the synthesized projection keeps what the source carries.** It kept simple fields only, so every nested object and list of a row projected before `ApplyPolicy` came back null or empty as soon as any field was denied. A row a projection builds — the outermost `Select` constructs it, in an object initializer or with a constructor, as in `db.Roles.Select(r => new RoleRow { … })` — keeps the members its initializer assigns. An entity, or a `Select` that hands back an entity such as `db.Orders.Select(o => o.Customer)`, keeps its mapped columns, converted and JSON ones included, its owned and complex members, and every collection of simple values such as `byte[]` or `List`. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, recorded as `Dropped` with a reason starting `left out whole`. An entity's navigations, the objects of a row in memory, and a value EF Core does not map are left out, and the type needs a public parameterless constructor for the typed projection, as it already did. When a derived type in the model, or a loaded subclass of a row in memory, declares a denied field, the rows come back as the queried type, so a derived type's allowed fields are dropped too; query the derived type with `OfType()` to keep them. A member that can hold an object of any type, a geometry, a JSON bag or a `BitArray` column say, asks for no projection on its own, and a chain that reaches its rows through a navigation, `SelectMany`, `Join` or `GroupBy` counts every navigation as loaded only when it has an include or builds an object. - **Changed: `[DwEntity(DefaultOrder)]` reaches a projection that builds the row.** A guarded query over a projected source takes the default when the outermost `Select` builds the type in an object initializer and assigns every field the default names a column, at every level of a nested path — a mapped member read directly, through reference navigations or through `EF.Property`: `Select(t => new TicketRow { Id = t.Id, CreatedAt = t.CreatedAt })` for `"CreatedAt desc, Id"`. A computed value or any other projection still leaves the query in its own order. A `Select`, or a `Filter` with `Selects`, composed on the guarded handle keeps the rest of the chain unordered, and a composed `Filter` that sent orders gets no default later in the chain, as a composed `Order` already did not. - **Changed: the async dynamic `Filter` and the async `Summary` read through EF Core.** `ToListAsyncDynamic` and `ToListAsync(Summary)` read with EF Core's `ToListAsync` instead of Dynamic LINQ's `ToDynamicListAsync`, which had no token to pass on, and the summary counts with `CountAsync` where it counted synchronously. So on an EF Core query a canceled token now reaches the database. A provider that is not EF Core's keeps Dynamic LINQ's read, on the calling thread. - **New: a `CancellationToken` on every async terminal**, guarded and unguarded: `ToListAsync` and `ToListAsyncDynamic` with a `Filter`, `ToListAsync` with a `Summary`, and `ToListAsync` with a `Segment`. The token reaches the count and the read. The overloads sit beside the 3.1 signatures, which are unchanged, so code compiled against 3.1 still binds. `ToListAsync(filter, default)`, `ToListAsyncDynamic(filter, default)` and `ToListAsync(summary, default)` no longer compile, because `default` fits both `getQueryString` and the token: write `false`, a token, or a named argument. A reflection lookup of `ToListAsyncDynamic` by name alone now finds three methods where it found one, and one of `ToListAsync` finds more than it did. -- **Known limits.** Once a projection is needed, an entity's navigations are left out, included ones too; under the convenience tier name one in `Selects` to get it narrowed, and under the strict tier name its allowed fields. A forced scope declared on a list's element type filters the rows that hold the list, never its elements, so `Selects` naming the list returns every element and a synthesized projection leaves the list out; scope the elements where the row is built. A member typed `object`, a framework interface or a collection that is not generic (`IEnumerable`, `ArrayList`) is opaque to the policy, and a framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: hold such values in a list of the policed type. A member EF Core does not map holds what the class computes, and the policy reads nothing into it. `/simulate` has no source, so it reads the type as one it cannot see into: every denial beneath a member counts, and the projection it shows keeps only members holding a value. +- **Known limits.** Once a projection is needed, an entity's navigations are left out, included ones too; under the convenience tier name one in `Selects` to get it narrowed, and under the strict tier name its allowed fields. A forced scope declared on a list's element type filters the rows that hold the list, never its elements, so `Selects` naming the list returns every element and a synthesized projection leaves the list out; scope the elements where the row is built. A member typed `object`, a framework interface or a collection that is not generic (`IEnumerable`, `ArrayList`) is opaque to the policy, and a framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: hold such values in a list of the policed type. A member EF Core does not map is read as its type, since its getter can hand out what EF Core loaded. Rows in memory can be any loaded subtype, so they are projected whenever one declares a denied field. `/simulate` has no source, so it reads the type as one it cannot see into: every denial beneath a member counts, and the projection it shows keeps only members holding a value. ## Version 3.1.0 highlights From ca3f1549eab0915b4cf2b18facbed0e09becc4ae Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 09:25:47 +0300 Subject: [PATCH 17/22] fix(policies): read a denial from every declaration a row runs, and what a lambda hands its rows A [DwDenied] on another declaration of a member was missed in several places: an open generic class implementing a generic interface, an explicit implementation reached through a framework generic, an interface a subtype adds over a member it inherits, an interface declaration read by the gate's scan or its fallback for a subtype's own member, an override of the setter alone, and a member a subtype hides with new. The walker now reads every declaration a row can run, once per member and load epoch, and the gate and the startup scan read the same. A reshaped chain counted as building its rows only when a lambda constructed an object. An application's method, a captured query with its own include or projection, and an object captured from memory hand a row what no include accounts for, and count now. A value that only feeds a predicate or a key no longer counts. Also: a converted column that can hold any object is left out when a projection is built for another field; an application's own non-generic collection has its own members read; an open generic subtype over another closed instantiation is no longer one of a type's subtypes; the trace records a member left out only when a projection is built; the subtype index no longer holds on to assemblies the domain already lists. Co-Authored-By: Claude Opus 5 --- .../Policies/ReviewConvertedColumnTests.cs | 144 +++ .../Policies/ReviewDenialsElsewhereTests.cs | 1131 +++++++++++++++++ .../Policies/ReviewLeakTests7.cs | 228 ++++ .../Policies/ReviewObjectMemberTests.cs | 453 +++++++ .../Policies/ReviewOpaqueCollectionTests.cs | 180 +++ .../Policies/ReviewOverBlockTests.cs | 16 + .../Policies/ReviewReshapeTests.cs | 298 +++++ .../Policies/ReviewSubtypeIndexTests.cs | 136 ++ DynamicWhere.ex/Policies/DTOs/PolicyTrace.cs | 9 + .../Resolution/AttributePolicyProvider.cs | 225 +++- .../Policies/Source/FilterSanitizer.cs | 82 +- .../Policies/Source/KnownSubtypes.cs | 39 +- DynamicWhere.ex/Policies/Source/RowShape.cs | 16 +- .../Validation/PolicyModelValidator.cs | 8 +- DynamicWhere.ex/Source/QueryRoot.cs | 208 ++- 15 files changed, 3070 insertions(+), 103 deletions(-) create mode 100644 DynamicWhere.Tests/Policies/ReviewConvertedColumnTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewDenialsElsewhereTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewLeakTests7.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewObjectMemberTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewOpaqueCollectionTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewReshapeTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewSubtypeIndexTests.cs diff --git a/DynamicWhere.Tests/Policies/ReviewConvertedColumnTests.cs b/DynamicWhere.Tests/Policies/ReviewConvertedColumnTests.cs new file mode 100644 index 0000000..ec106b0 --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewConvertedColumnTests.cs @@ -0,0 +1,144 @@ +using System.Collections; +using System.Reflection; +using System.Text.Json; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Xunit.Abstractions; + +namespace DynamicWhere.Tests.Policies +{ + // ============================================================================ converted columns holding policed types + + /// The denial is declared on the interface member, which applies to the class's member too. + public interface IZvVaultCard + { + [DwDenied] + string? Pan { get; } + } + + public class ZvVaultCard : IZvVaultCard + { + public string Label { get; set; } = string.Empty; + + public string? Pan { get; set; } + } + + /// Control: the same card with the attribute on its own member. + public class ZvVaultCardDirect + { + public string Label { get; set; } = string.Empty; + + [DwDenied] + public string? Pan { get; set; } + } + + public class ZvVault + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + [DwDenied] + public string? Note { get; set; } + + public Dictionary Cards { get; set; } = new(); + + public Dictionary DirectCards { get; set; } = new(); + } + + public sealed class ZvConvertedContext : DbContext + { + private readonly SqliteConnection _connection; + + public ZvConvertedContext(SqliteConnection connection) => _connection = connection; + + public DbSet Vaults => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + protected override void OnModelCreating(ModelBuilder model) + { + model.Entity().Property(v => v.Cards).HasConversion( + value => JsonSerializer.Serialize(value, (JsonSerializerOptions?)null), + text => JsonSerializer.Deserialize>(text, (JsonSerializerOptions?)null)!); + model.Entity().Property(v => v.DirectCards).HasConversion( + value => JsonSerializer.Serialize(value, (JsonSerializerOptions?)null), + text => JsonSerializer.Deserialize>(text, (JsonSerializerOptions?)null)!); + } + } + + public sealed class ReviewConvertedColumnTests : IDisposable + { + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZvConvertedContext _db; + + public ReviewConvertedColumnTests(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + _db = new ZvConvertedContext(_connection); + _db.Database.EnsureCreated(); + _db.Vaults.Add(new ZvVault + { + Name = "V1", + Note = "note-secret", + Cards = { ["a"] = new ZvVaultCard { Label = "visa", Pan = "iface-column-pan" } }, + DirectCards = { ["b"] = new ZvVaultCardDirect { Label = "mc", Pan = "direct-column-pan" } } + }); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static PolicyQueryable Guard(IQueryable source, DwTier tier) where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = tier, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + /// + /// 3.1.0 left every non-scalar member out once Note asked for a projection. The column is now kept whole when + /// the policy reads nothing denied in it; a denial declared on the interface member is not read there. + /// + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zv_F1_a_converted_column_holding_a_type_whose_interface_denies_a_member(DwTier tier) + { + PolicyQueryable guarded = Guard(_db.Vaults, tier); + string sent; + + try + { + sent = JsonSerializer.Serialize(guarded.ToList(new Filter()).Data); + } + catch (PolicyException refusal) when (refusal.ErrorCode == PolicyErrorCode.FieldDeniedForSelect) + { + sent = "refused"; + } + + _out.WriteLine("sent: " + sent + " decisions: " + + string.Join(" | ", guarded.LastTrace?.Decisions.Where(d => d.Action != PolicyAction.Allowed) + .Select(d => $"{d.FieldPath} {d.Action}") ?? Array.Empty())); + + Assert.DoesNotContain("note-secret", sent); + Assert.DoesNotContain("direct-column-pan", sent); + Assert.DoesNotContain("iface-column-pan", sent); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewDenialsElsewhereTests.cs b/DynamicWhere.Tests/Policies/ReviewDenialsElsewhereTests.cs new file mode 100644 index 0000000..421e742 --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewDenialsElsewhereTests.cs @@ -0,0 +1,1131 @@ +using System.Collections; +using System.Reflection; +using System.Text.Json; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Classes.Core; +using DynamicWhere.ex.Enums; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.DTOs; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using DynamicWhere.ex.Policies.Storage; +using DynamicWhere.ex.Policies.Validation; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Xunit.Abstractions; + +namespace DynamicWhere.Tests.Policies +{ + // ============================================================================ denials on other declarations: models + + // ---- A1: a generic interface, implemented by an open generic class that denies the member ---------- + + public interface IZvHolder + { + int Id { get; } + + string? Secret { get; } + } + + public class ZvHolderImpl : IZvHolder + { + public int Id { get; set; } + + [DwDenied] + public string? Secret { get; set; } + } + + /// Control: the same interface shape, implemented by a closed class. + public interface IZvClosedHolder + { + int Id { get; } + + string? Secret { get; } + } + + public class ZvIntClosedHolder : IZvClosedHolder + { + public int Id { get; set; } + + [DwDenied] + public string? Secret { get; set; } + } + + /// Control: a plain interface implemented by an open generic class. + public interface IZvPlainHolder + { + int Id { get; } + + string? Secret { get; } + } + + public class ZvGenPlainHolder : IZvPlainHolder + { + public int Id { get; set; } + + [DwDenied] + public string? Secret { get; set; } + } + + public class ZvHolderRow + { + public int Id { get; set; } + + public IZvHolder? Holder { get; set; } + } + + // ---- A1b: a generic base class whose open generic subclass overrides and denies --------------------- + + public class ZvBox + { + public int Id { get; set; } + + public virtual string? Code { get; set; } + } + + public class ZvSecretBox : ZvBox + { + [DwDenied] + public override string? Code + { + get => base.Code; + set => base.Code = value; + } + } + + // ---- A2: an explicit interface implementation that denies the member -------------------------------- + + public interface IZvLocker + { + int Id { get; } + + string? Pin { get; } + } + + public class ZvLocker : IZvLocker + { + private string? _pin; + + public int Id { get; set; } + + [DwDenied] + string? IZvLocker.Pin => _pin; + + public void Seal(string pin) => _pin = pin; + } + + public class ZvLockerRoom + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public Dictionary Lockers { get; set; } = new(); + } + + // ---- A3: an interface member's denial, on a member a subtype of the queried type implements --------- + + public interface IZvTaxed + { + [DwDenied] + string? TaxId { get; } + } + + public class ZvFirm + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + } + + public class ZvTaxedFirm : ZvFirm, IZvTaxed + { + public string? TaxId { get; set; } + } + + /// Control: a subtype with the attribute on its own member, on a base of its own. + public class ZvFirmB + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + } + + public class ZvDirectTaxedFirm : ZvFirmB + { + [DwDenied] + public string? TaxNumber { get; set; } + } + + public class ZvFirmHolder + { + public int Id { get; set; } + + public ZvFirm? Firm { get; set; } + } + + // ---- A4: an interface member's denial, reached through a framework generic ------------------------- + + public interface IZvCardish + { + [DwDenied] + string? Pan { get; } + } + + public class ZvCardImpl : IZvCardish + { + public string Label { get; set; } = string.Empty; + + public string? Pan { get; set; } + } + + public class ZvWallet + { + public int Id { get; set; } + + public string Owner { get; set; } = string.Empty; + + public Dictionary Cards { get; set; } = new(); + } + + public class ZvCardDirect + { + public string Label { get; set; } = string.Empty; + + [DwDenied] + public string? Pan { get; set; } + } + + public class ZvWalletDirect + { + public int Id { get; set; } + + public string Owner { get; set; } = string.Empty; + + public Dictionary Cards { get; set; } = new(); + } + + // ---- A5: an override of the setter alone ---------------------------------------------------------- + + public class ZvGadget + { + public int Id { get; set; } + + public virtual string? Code { get; set; } + } + + public class ZvSetOnlyGadget : ZvGadget + { + [DwDenied] + public override string? Code + { + set => base.Code = value; + } + } + + // ---- A6: a member hidden with new rather than overridden -------------------------------------------- + + public class ZvDevice + { + public int Id { get; set; } + + public string? Serial { get; set; } + } + + public class ZvHiddenDevice : ZvDevice + { + [DwDenied] + public new string? Serial + { + get => base.Serial; + set => base.Serial = value; + } + } + + /// A member hidden with new that keeps its own value, as new usually does. + public class ZvDevice2 + { + public int Id { get; set; } + + public string? Serial { get; set; } + } + + public class ZvHiddenDevice2 : ZvDevice2 + { + [DwDenied] + public new string? Serial { get; set; } + } + + // ---- A7: an override two levels down, of an abstract member ---------------------------------------- + + public abstract class ZvInstrument + { + public int Id { get; set; } + + public abstract string? Code { get; set; } + } + + public class ZvMidInstrument : ZvInstrument + { + public override string? Code { get; set; } + } + + public class ZvLeafInstrument : ZvMidInstrument + { + [DwDenied] + public override string? Code + { + get => base.Code; + set => base.Code = value; + } + } + + // ---- A8: an overridable and a sealed denial on a subtype's overrides -------------------------------- + + public class ZvMeter + { + public int Id { get; set; } + + public virtual string? Reading { get; set; } + + public virtual string? Serial { get; set; } + } + + public class ZvSmartMeter : ZvMeter + { + [DwNoWhere(Overridable = true)] + public override string? Reading + { + get => base.Reading; + set => base.Reading = value; + } + + [DwNoWhere] + public override string? Serial + { + get => base.Serial; + set => base.Serial = value; + } + } + + public class ZvAnalogMeter : ZvMeter + { + } + + // ---- A9: a transform on a subtype's override, which is documented as read from the declaration walked + + public class ZvContactCard + { + public int Id { get; set; } + + public virtual string? Phone { get; set; } + } + + public class ZvMaskedContactCard : ZvContactCard + { + [DwMask(MaskStrategy.Full)] + public override string? Phone + { + get => base.Phone; + set => base.Phone = value; + } + } + + // ---- A10: a rule on a subtype's member through a base-typed member, spelled in another letter case + + public class ZvKennel + { + public int Id { get; set; } + + public ZvKennelPet? Pet { get; set; } + } + + public class ZvKennelPet + { + public string Name { get; set; } = string.Empty; + } + + public class ZvKennelHamster : ZvKennelPet + { + public string? Tag { get; set; } + } + + // ---- B1: siblings, and a concrete type's own subtype ---------------------------------------------- + + public class ZvCar + { + public int Id { get; set; } + + public virtual string? Plate { get; set; } + } + + public class ZvArmoredCar : ZvCar + { + [DwDenied] + public override string? Plate + { + get => base.Plate; + set => base.Plate = value; + } + } + + public class ZvTaxi : ZvCar + { + } + + public class ZvLimo : ZvTaxi + { + [DwNoOrder] + public override string? Plate + { + get => base.Plate; + set => base.Plate = value; + } + } + + // ---- B2: a widely shared interface, one implementer denies ------------------------------------------- + + public interface IZvNamed + { + string Name { get; } + } + + public class ZvCity : IZvNamed + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + } + + public class ZvSpy : IZvNamed + { + public int Id { get; set; } + + [DwDenied] + public string Name { get; set; } = string.Empty; + } + + public class ZvPin + { + public int Id { get; set; } + + public IZvNamed? Place { get; set; } + } + + // ---- B3: the startup scan over a default order a subtype's override denies overridably ------------- + + [DwEntity(DefaultOrder = "Rank")] + public class ZvRanked + { + public int Id { get; set; } + + public virtual int Rank { get; set; } + } + + public class ZvSecretlyRanked : ZvRanked + { + [DwNoOrder(Overridable = true)] + public override int Rank + { + get => base.Rank; + set => base.Rank = value; + } + } + + /// Control: the same denial on the type's own member, which the scan reads as a warning. + [DwEntity(DefaultOrder = "Rank")] + public class ZvOwnRanked + { + public int Id { get; set; } + + [DwNoOrder(Overridable = true)] + public int Rank { get; set; } + } + + // ---- A3 on EF Core: a TPH root whose derived entity implements the interface ------------------------ + + public class ZvEfFirm + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + } + + public class ZvEfTaxedFirm : ZvEfFirm, IZvTaxed + { + public string? TaxId { get; set; } + } + + /// An entity implementing the shared interface one unrelated DTO denies. + public class ZvEfCity : IZvNamed + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + } + + /// An entity the model maps alone. + public class ZvEfVehicle + { + public int Id { get; set; } + + public virtual string? Code { get; set; } + } + + /// A class the model does not map, which EF Core can never materialize for ZvEfVehicle's set. + public class ZvVehicleView : ZvEfVehicle + { + [DwDenied] + public override string? Code + { + get => base.Code; + set => base.Code = value; + } + } + + public sealed class ZvDenialsContext : DbContext + { + private readonly SqliteConnection _connection; + + public ZvDenialsContext(SqliteConnection connection) => _connection = connection; + + public DbSet Firms => Set(); + + public DbSet Cities => Set(); + + public DbSet Vehicles => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + protected override void OnModelCreating(ModelBuilder model) => model.Entity(); + } + + // ============================================================================ denials on other declarations: tests + + /// + /// The denials read from another declaration of a member: an interface member, an implementation, an override + /// and a member hidden with new, through every path a row reaches them by. + /// + public sealed class ReviewDenialsElsewhereTests : IDisposable + { + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZvDenialsContext _db; + + public ReviewDenialsElsewhereTests(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + _db = new ZvDenialsContext(_connection); + _db.Database.EnsureCreated(); + _db.Firms.Add(new ZvEfTaxedFirm { Name = "Acme", TaxId = "ef-tax-secret" }); + _db.Firms.Add(new ZvEfFirm { Name = "Plain" }); + _db.Cities.Add(new ZvEfCity { Name = "Basra" }); + _db.Vehicles.Add(new ZvEfVehicle { Code = "V-1" }); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static PolicyQueryable Guard(IQueryable source, DwTier tier = DwTier.Strict, params IDwPolicyProvider[] more) + where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = tier, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() }.Concat(more).ToArray())); + + private static Filter Where(string field, string value) => new() + { + ConditionGroup = new ConditionGroup + { + Conditions = { new Condition { Field = field, DataType = DataType.Text, Operator = Operator.Equal, Values = { value } } } + } + }; + + private static Filter OrderedBy(string field) => new() { Orders = new List { new() { Field = field } } }; + + private static Summary GroupedBy(string field) => new() + { + GroupBy = new GroupBy + { + Fields = new List { field }, + AggregateBy = new List { new() { Alias = "Total", Aggregator = Aggregator.Count } } + } + }; + + /// The error code a guarded call refused with, or null when it ran. + private string Code(Action run) + { + try + { + run(); + + return "ran"; + } + catch (PolicyException refusal) + { + return refusal.ErrorCode.ToString(); + } + catch (Exception other) + { + return $"{other.GetType().Name}: {other.Message.Split('\n')[0]}"; + } + } + + private static IEnumerable Denials(Type type, string path) => + new AttributePolicyProvider().GetFragments(type, new DwPolicyContext()) + .Where(f => f.Effect == PolicyEffect.Deny && string.Equals(f.FieldPath, path, StringComparison.Ordinal)); + + /// Everything reachable from a value, read by runtime type, including explicit interface members. + private static bool Holds(object? value, string text) + { + HashSet seen = new(ReferenceEqualityComparer.Instance); + Stack pending = new(); + + pending.Push(value); + + while (pending.Count > 0) + { + object? current = pending.Pop(); + + if (current is null || current is ValueType || current is MemberInfo) + { + continue; + } + + if (current is string held) + { + if (held == text) + { + return true; + } + + continue; + } + + if (!seen.Add(current)) + { + continue; + } + + if (current is IDictionary map) + { + foreach (object? item in map.Values) + { + pending.Push(item); + } + + continue; + } + + if (current is IEnumerable items) + { + foreach (object? item in items) + { + pending.Push(item); + } + + continue; + } + + foreach (PropertyInfo property in current.GetType().GetProperties(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance)) + { + if (property.GetIndexParameters().Length != 0 || !property.CanRead) + { + continue; + } + + try + { + pending.Push(property.GetValue(current)); + } + catch + { + // A getter that throws holds nothing readable. + } + } + } + + return false; + } + + private static object? SafeRead(Func read) + { + try + { + return read(); + } + catch (PolicyException refusal) when (refusal.ErrorCode is PolicyErrorCode.FieldDeniedForSelect or PolicyErrorCode.AllSelectsDenied) + { + return null; + } + catch (LogicException) + { + // The core refused to build the projection: nothing returned. + return null; + } + } + + // ------------------------------------------------------------------------------------------------ A1 + + [Fact] + public void Zv_A1_a_generic_interface_implemented_by_an_open_generic_class_carries_its_denial() + { + _out.WriteLine("IZvHolder.Secret denials: " + Denials(typeof(IZvHolder), "Secret").Count()); + _out.WriteLine("control IZvClosedHolder.Secret denials: " + Denials(typeof(IZvClosedHolder), "Secret").Count()); + _out.WriteLine("control IZvPlainHolder.Secret denials: " + Denials(typeof(IZvPlainHolder), "Secret").Count()); + + Assert.NotEmpty(Denials(typeof(IZvClosedHolder), "Secret")); + Assert.NotEmpty(Denials(typeof(IZvPlainHolder), "Secret")); + Assert.NotEmpty(Denials(typeof(IZvHolder), "Secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zv_A1_filtering_a_generic_interface_on_the_member_its_open_generic_implementation_denies(DwTier tier) + { + IZvHolder[] rows = { new ZvHolderImpl { Id = 1, Secret = "generic-iface-secret" } }; + + string where = Code(() => Guard(rows.AsQueryable(), tier).ToList(Where("Secret", "generic-iface-secret"))); + string order = Code(() => Guard(rows.AsQueryable(), tier).ToList(OrderedBy("Secret"))); + string group = Code(() => Guard(rows.AsQueryable(), tier).ToList(GroupedBy("Secret"))); + + _out.WriteLine($"where={where} order={order} group={group}"); + + Assert.Equal(nameof(PolicyErrorCode.FieldDeniedForWhere), where); + Assert.Equal(nameof(PolicyErrorCode.FieldDeniedForGroup), group); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zv_A1_rows_read_through_a_generic_interface_whose_open_generic_implementation_denies(DwTier tier) + { + IZvHolder[] rows = { new ZvHolderImpl { Id = 1, Secret = "generic-iface-secret" } }; + + PolicyQueryable> guarded = Guard(rows.AsQueryable(), tier); + object? data = SafeRead(() => guarded.ToList(new Filter()).Data); + + _out.WriteLine("result: " + JsonSerializer.Serialize(data) + " decisions: " + + string.Join(" | ", guarded.LastTrace?.Decisions.Select(d => $"{d.FieldPath} {d.Action}") ?? Array.Empty())); + + Assert.False(Holds(data, "generic-iface-secret")); + } + + [Fact] + public void Zv_A1_a_member_typed_as_the_generic_interface() + { + ZvHolderRow[] rows = { new() { Id = 1, Holder = new ZvHolderImpl { Id = 2, Secret = "generic-member-secret" } } }; + + string where = Code(() => Guard(rows.AsQueryable()).ToList(Where("Holder.Secret", "generic-member-secret"))); + object? whole = SafeRead(() => Guard(rows.AsQueryable()).ToList(new Filter()).Data); + + _out.WriteLine($"where on Holder.Secret={where}; whole result holds it={Holds(whole, "generic-member-secret")}"); + + Assert.Equal(nameof(PolicyErrorCode.FieldDeniedForWhere), where); + Assert.False(Holds(whole, "generic-member-secret")); + } + + [Fact] + public void Zv_A1_control_a_closed_implementation_of_the_generic_interface_is_refused() + { + IZvClosedHolder[] rows = { new ZvIntClosedHolder { Id = 1, Secret = "closed-secret" } }; + + Assert.Equal(nameof(PolicyErrorCode.FieldDeniedForWhere), Code(() => Guard(rows.AsQueryable()).ToList(Where("Secret", "closed-secret")))); + } + + [Fact] + public void Zv_A1b_a_generic_base_class_whose_open_generic_subclass_overrides_and_denies() + { + ZvBox[] rows = { new ZvSecretBox { Id = 1, Code = "box-secret" } }; + + _out.WriteLine("ZvBox.Code denials: " + Denials(typeof(ZvBox), "Code").Count()); + + Assert.Equal(nameof(PolicyErrorCode.FieldDeniedForWhere), Code(() => Guard(rows.AsQueryable()).ToList(Where("Code", "box-secret")))); + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable()).ToList(new Filter()).Data), "box-secret")); + } + + // ------------------------------------------------------------------------------------------------ A2 + + [Fact] + public void Zv_A2_an_explicit_interface_implementation_denies_the_interface_path() + { + ZvLocker locker = new() { Id = 1 }; + locker.Seal("pin-secret"); + IZvLocker[] rows = { locker }; + + _out.WriteLine("IZvLocker.Pin denials: " + Denials(typeof(IZvLocker), "Pin").Count()); + + Assert.Equal(nameof(PolicyErrorCode.FieldDeniedForWhere), Code(() => Guard(rows.AsQueryable()).ToList(Where("Pin", "pin-secret")))); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zv_A2_an_explicit_implementation_held_in_a_framework_generic(DwTier tier) + { + ZvLocker locker = new() { Id = 1 }; + locker.Seal("pin-secret"); + ZvLockerRoom[] rows = { new() { Id = 1, Name = "R", Lockers = { ["a"] = locker } } }; + + object? unguarded = rows; + Assert.Contains("pin-secret", JsonSerializer.Serialize(unguarded)); + + object? guarded = SafeRead(() => Guard(rows.AsQueryable(), tier).ToList(new Filter()).Data); + object? named = SafeRead(() => Guard(rows.AsQueryable(), tier).ToList(new Filter { Selects = new List { "Id", "Lockers" } }).Data); + + _out.WriteLine("whole: " + JsonSerializer.Serialize(guarded)); + _out.WriteLine("named: " + JsonSerializer.Serialize(named)); + + Assert.DoesNotContain("pin-secret", JsonSerializer.Serialize(guarded)); + Assert.DoesNotContain("pin-secret", JsonSerializer.Serialize(named)); + } + + // ------------------------------------------------------------------------------------------------ A3 + + [Fact] + public void Zv_A3_control_the_class_that_implements_the_interface_is_policed() + { + ZvTaxedFirm[] rows = { new() { Id = 1, Name = "Acme", TaxId = "tax-secret" } }; + + Assert.NotEmpty(Denials(typeof(ZvTaxedFirm), "TaxId")); + Assert.Equal(nameof(PolicyErrorCode.FieldDeniedForWhere), Code(() => Guard(rows.AsQueryable()).ToList(Where("TaxId", "tax-secret")))); + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable()).ToList(new Filter()).Data), "tax-secret")); + } + + [Fact] + public void Zv_A3_control_a_subtypes_own_attribute_through_the_base_type() + { + ZvFirmB[] rows = { new ZvDirectTaxedFirm { Id = 1, Name = "Acme", TaxNumber = "direct-tax-secret" } }; + + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable()).ToList(new Filter()).Data), "direct-tax-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zv_A3_rows_in_memory_read_through_a_base_type_whose_subtype_implements_the_denying_interface(DwTier tier) + { + ZvFirm[] rows = { new ZvTaxedFirm { Id = 1, Name = "Acme", TaxId = "tax-secret" } }; + + PolicyQueryable guarded = Guard(rows.AsQueryable(), tier); + object? data = SafeRead(() => guarded.ToList(new Filter()).Data); + + _out.WriteLine("types: " + string.Join(",", ((IEnumerable?)data ?? Array.Empty()).Cast().Select(r => r.GetType().Name))); + + Assert.False(Holds(data, "tax-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zv_A3_a_base_typed_member_holding_the_subtype(DwTier tier) + { + ZvFirmHolder[] rows = { new() { Id = 1, Firm = new ZvTaxedFirm { Id = 2, Name = "Acme", TaxId = "tax-secret" } } }; + + object? whole = SafeRead(() => Guard(rows.AsQueryable(), tier).ToList(new Filter()).Data); + object? named = SafeRead(() => Guard(rows.AsQueryable(), tier).ToList(new Filter { Selects = new List { "Id", "Firm" } }).Data); + + Assert.False(Holds(whole, "tax-secret")); + Assert.False(Holds(named, "tax-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zv_A3_an_entity_hierarchy_root_whose_derived_entity_implements_the_denying_interface(DwTier tier) + { + Assert.True(Holds(_db.Firms.AsNoTracking().ToList(), "ef-tax-secret")); + + PolicyQueryable guarded = Guard(_db.Firms, tier); + object? data = SafeRead(() => guarded.ToList(new Filter()).Data); + + _out.WriteLine("types: " + string.Join(",", ((IEnumerable?)data ?? Array.Empty()).Cast().Select(r => r.GetType().Name))); + + Assert.False(Holds(data, "ef-tax-secret")); + Assert.False(Holds(SafeRead(() => Guard(_db.Firms, tier).ToListDynamic(new Filter()).Data), "ef-tax-secret")); + } + + [Fact] + public void Zv_A3_control_the_derived_entity_queried_itself_is_policed() + { + Assert.Equal(nameof(PolicyErrorCode.FieldDeniedForWhere), Code(() => Guard(_db.Firms.OfType()).ToList(Where("TaxId", "x")))); + Assert.False(Holds(SafeRead(() => Guard(_db.Firms.OfType()).ToList(new Filter()).Data), "ef-tax-secret")); + } + + // ------------------------------------------------------------------------------------------------ A4 + + [Fact] + public void Zv_A4_control_a_framework_generic_of_a_class_that_denies_its_own_member() + { + ZvWalletDirect[] rows = { new() { Id = 1, Owner = "O", Cards = { ["a"] = new ZvCardDirect { Label = "visa", Pan = "direct-pan-secret" } } } }; + + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable()).ToList(new Filter()).Data), "direct-pan-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zv_A4_a_framework_generic_of_a_class_whose_interface_denies_the_member(DwTier tier) + { + ZvWallet[] rows = { new() { Id = 1, Owner = "O", Cards = { ["a"] = new ZvCardImpl { Label = "visa", Pan = "iface-pan-secret" } } } }; + + object? whole = SafeRead(() => Guard(rows.AsQueryable(), tier).ToList(new Filter()).Data); + object? named = SafeRead(() => Guard(rows.AsQueryable(), tier).ToList(new Filter { Selects = new List { "Id", "Cards" } }).Data); + + _out.WriteLine("whole: " + JsonSerializer.Serialize(whole)); + _out.WriteLine("named: " + JsonSerializer.Serialize(named)); + + Assert.False(Holds(whole, "iface-pan-secret")); + Assert.False(Holds(named, "iface-pan-secret")); + } + + // ------------------------------------------------------------------------------------------------ A5 + + [Fact] + public void Zv_A5_an_override_of_the_setter_alone() + { + PropertyInfo declared = typeof(ZvSetOnlyGadget).GetProperty("Code")!; + + _out.WriteLine($"ZvSetOnlyGadget.Code: declaring={declared.DeclaringType!.Name} canRead={declared.CanRead}"); + _out.WriteLine("ZvSetOnlyGadget.Code denials: " + Denials(typeof(ZvSetOnlyGadget), "Code").Count()); + _out.WriteLine("ZvGadget.Code denials: " + Denials(typeof(ZvGadget), "Code").Count()); + + ZvGadget[] rows = { new ZvSetOnlyGadget { Id = 1, Code = "setter-secret" } }; + ZvSetOnlyGadget[] own = { new() { Id = 1, Code = "setter-secret" } }; + + string direct = Code(() => Guard(own.AsQueryable()).ToList(Where("Code", "setter-secret"))); + string through = Code(() => Guard(rows.AsQueryable()).ToList(Where("Code", "setter-secret"))); + + _out.WriteLine($"queried as the subtype: {direct}; through the base type: {through}"); + + object? data = SafeRead(() => Guard(rows.AsQueryable()).ToList(new Filter()).Data); + string sent = JsonSerializer.Serialize(data); + + // The runtime type's PropertyInfo has no getter, so a reflection walk misses it; a serializer does not. + _out.WriteLine("through the base type, the result sent: " + sent); + + // The same member of the same rows: whatever the subtype says, the base path should say too. + Assert.Equal(direct, through); + Assert.DoesNotContain("setter-secret", sent); + } + + // ------------------------------------------------------------------------------------------------ A6 + + [Fact] + public void Zv_A6_a_member_hidden_with_new_denies_the_base_path() + { + ZvDevice[] rows = { new ZvHiddenDevice { Id = 1, Serial = "hidden-secret" } }; + ZvHiddenDevice[] own = { new() { Id = 1, Serial = "hidden-secret" } }; + + string direct = Code(() => Guard(own.AsQueryable()).ToList(Where("Serial", "hidden-secret"))); + string through = Code(() => Guard(rows.AsQueryable()).ToList(Where("Serial", "hidden-secret"))); + object? data = SafeRead(() => Guard(rows.AsQueryable()).ToList(new Filter()).Data); + + _out.WriteLine($"queried as the subtype: {direct}; through the base type: {through}; result holds: {Holds(data, "hidden-secret")}"); + + // Whether the new member reads the one it hides cannot be told, so the base path is denied too. + Assert.Equal(nameof(PolicyErrorCode.FieldDeniedForWhere), direct); + Assert.Equal(direct, through); + Assert.False(Holds(data, "hidden-secret")); + } + + [Fact] + public void Zv_A6_a_member_hidden_with_new_that_keeps_its_own_value() + { + ZvDevice2[] rows = { new ZvHiddenDevice2 { Id = 1, Serial = "hidden-own-secret" } }; + ((ZvDevice2)rows[0]).Serial = "public-serial"; + + PolicyQueryable guarded = Guard(rows.AsQueryable()); + object? data = SafeRead(() => guarded.ToList(new Filter()).Data); + + _out.WriteLine("types: " + string.Join(",", ((IEnumerable?)data ?? Array.Empty()).Cast().Select(r => r.GetType().Name)) + + "; decisions: " + string.Join(" | ", guarded.LastTrace?.Decisions.Select(d => $"{d.FieldPath} {d.Action}") ?? Array.Empty())); + + // A row of the subtype still holds the hidden member's own value; a runtime-type serializer writes it. + Assert.False(Holds(data, "hidden-own-secret")); + } + + // ------------------------------------------------------------------------------------------------ A7 + + [Fact] + public void Zv_A7_an_override_two_levels_down_of_an_abstract_member() + { + ZvInstrument[] root = { new ZvLeafInstrument { Id = 1, Code = "leaf-secret" } }; + ZvMidInstrument[] mid = { new ZvLeafInstrument { Id = 1, Code = "leaf-secret" } }; + + Assert.Equal(nameof(PolicyErrorCode.FieldDeniedForWhere), Code(() => Guard(root.AsQueryable()).ToList(Where("Code", "leaf-secret")))); + Assert.Equal(nameof(PolicyErrorCode.FieldDeniedForWhere), Code(() => Guard(mid.AsQueryable()).ToList(Where("Code", "leaf-secret")))); + Assert.False(Holds(SafeRead(() => Guard(mid.AsQueryable()).ToList(new Filter()).Data), "leaf-secret")); + } + + // ------------------------------------------------------------------------------------------------ A8 + + [Fact] + public void Zv_A8_a_rule_lifts_an_overridable_subtype_denial_and_not_a_sealed_one() + { + ZvMeter[] rows = { new ZvSmartMeter { Id = 1, Reading = "r", Serial = "s" } }; + FakePolicyProvider allow = new FakePolicyProvider() + .Add("Reading", PolicyFeature.Where, PolicyEffect.Allow, PolicyLevel.DynamicGlobal) + .Add("Serial", PolicyFeature.Where, PolicyEffect.Allow, PolicyLevel.DynamicGlobal); + + string readingWithout = Code(() => Guard(rows.AsQueryable()).ToList(Where("Reading", "r"))); + string readingLifted = Code(() => Guard(rows.AsQueryable(), DwTier.Strict, allow).ToList(Where("Reading", "r"))); + string serialLifted = Code(() => Guard(rows.AsQueryable(), DwTier.Strict, allow).ToList(Where("Serial", "s"))); + + _out.WriteLine($"Reading unlifted={readingWithout} lifted={readingLifted}; Serial with an allow rule={serialLifted}"); + + Assert.Equal(nameof(PolicyErrorCode.FieldDeniedForWhere), readingWithout); + Assert.Equal("ran", readingLifted); + Assert.Equal(nameof(PolicyErrorCode.FieldDeniedForWhere), serialLifted); + } + + [Fact] + public void Zv_A8_the_store_refuses_a_rule_on_the_base_path_a_subtype_seals_and_not_on_a_sibling() + { + PolicyRule onBase = new(DwSubjectKind.Global, null, typeof(ZvMeter).FullName!, "Serial", PolicyFeature.Where, PolicyEffect.Allow); + PolicyRule onSibling = new(DwSubjectKind.Global, null, typeof(ZvAnalogMeter).FullName!, "Serial", PolicyFeature.Where, PolicyEffect.Allow); + PolicyRule overridable = new(DwSubjectKind.Global, null, typeof(ZvMeter).FullName!, "Reading", PolicyFeature.Where, PolicyEffect.Allow); + + Func resolve = name => name == typeof(ZvMeter).FullName ? typeof(ZvMeter) + : name == typeof(ZvAnalogMeter).FullName ? typeof(ZvAnalogMeter) : null; + + Exception? baseRefusal = Record.Exception(() => SealedFields.Refuse(onBase, resolve, "rule")); + Exception? siblingRefusal = Record.Exception(() => SealedFields.Refuse(onSibling, resolve, "rule")); + Exception? overridableRefusal = Record.Exception(() => SealedFields.Refuse(overridable, resolve, "rule")); + + _out.WriteLine($"base: {baseRefusal?.Message}"); + _out.WriteLine($"sibling: {siblingRefusal?.Message}"); + _out.WriteLine($"overridable: {overridableRefusal?.Message}"); + + Assert.IsType(baseRefusal); + Assert.Null(siblingRefusal); + Assert.Null(overridableRefusal); + } + + // ------------------------------------------------------------------------------------------------ A9 + + // ------------------------------------------------------------------------------------------------ A10 + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zv_A10_a_rule_on_a_subtype_member_spelled_in_another_case_is_enforced(DwTier tier) + { + ZvKennel[] rows = { new() { Id = 1, Pet = new ZvKennelHamster { Name = "Ham", Tag = "case-tag-secret" } } }; + FakePolicyProvider rules = new FakePolicyProvider() + .Add("pet.TAG", PolicyFeature.Select, PolicyEffect.Deny, PolicyLevel.DynamicGlobal); + + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable(), tier, rules).ToList(new Filter()).Data), "case-tag-secret")); + Assert.False(Holds(SafeRead(() => Guard(rows.AsQueryable(), tier, rules) + .ToList(new Filter { Selects = new List { "Id", "Pet" } }).Data), "case-tag-secret")); + } + + // ------------------------------------------------------------------------------------------------ B1 + + [Fact] + public void Zv_B1_a_sibling_override_does_not_reach_a_concrete_type_and_its_own_subtype_does() + { + ZvTaxi[] taxis = { new() { Id = 1, Plate = "taxi-plate" } }; + + Assert.Empty(Denials(typeof(ZvTaxi), "Plate").Where(f => (f.Features & PolicyFeature.Where) != 0)); + Assert.Equal("ran", Code(() => Guard(taxis.AsQueryable()).ToList(Where("Plate", "taxi-plate")))); + Assert.Equal("ran", Code(() => Guard(taxis.AsQueryable()).ToList(GroupedBy("Plate")))); + Assert.True(Holds(Guard(taxis.AsQueryable()).ToList(new Filter()).Data, "taxi-plate")); + + // ZvLimo, a subtype of ZvTaxi, denies ordering on Plate: that reaches ZvTaxi's path. + Assert.Equal(nameof(PolicyErrorCode.FieldDeniedForOrder), Code(() => Guard(taxis.AsQueryable()).ToList(OrderedBy("Plate")))); + + // ZvArmoredCar, a sibling, denies everything: the base path is denied, ZvTaxi's is not. + ZvCar[] cars = { new ZvTaxi { Id = 1, Plate = "taxi-plate" } }; + Assert.Equal(nameof(PolicyErrorCode.FieldDeniedForWhere), Code(() => Guard(cars.AsQueryable()).ToList(Where("Plate", "taxi-plate")))); + } + + // ------------------------------------------------------------------------------------------------ B2 + + [Fact] + public void Zv_B2_one_implementer_of_a_shared_interface_denies_the_interface_path_for_every_row() + { + ZvCity[] cities = { new() { Id = 1, Name = "Basra" } }; + IZvNamed[] named = { new ZvCity { Id = 1, Name = "Basra" } }; + ZvPin[] pins = { new() { Id = 1, Place = new ZvCity { Id = 2, Name = "Basra" } } }; + + string city = Code(() => Guard(cities.AsQueryable()).ToList(Where("Name", "Basra"))); + string iface = Code(() => Guard(named.AsQueryable()).ToList(Where("Name", "Basra"))); + string member = Code(() => Guard(pins.AsQueryable()).ToList(Where("Place.Name", "Basra"))); + string whole = Code(() => Guard(pins.AsQueryable()).ToList(new Filter())); + + _out.WriteLine($"class={city} interface={iface} interface-typed member={member} whole row={whole}"); + + Assert.Equal("ran", city); + } + + /// + /// An EF Core query over one entity set, read through the interface the entity implements: an unrelated + /// implementer's denial (ZvSpy, not even an entity) decides the interface path for every row. + /// + [Fact] + public void Zv_B2_an_entity_set_read_through_a_shared_interface_takes_an_unrelated_implementers_denial() + { + IQueryable cities = _db.Cities; + + string concrete = Code(() => Guard(_db.Cities).ToList(Where("Name", "Basra"))); + string where = Code(() => Guard(cities).ToList(Where("Name", "Basra"))); + string whole = Code(() => Guard(cities).ToList(new Filter())); + string dynamic = Code(() => Guard(cities).ToListDynamic(new Filter())); + + _out.WriteLine($"concrete set Where={concrete}; through the interface: Where={where} ToList={whole} ToListDynamic={dynamic}"); + + Assert.Equal("ran", concrete); + } + + /// + /// A class the EF Core model does not map overrides and denies an entity's member. EF Core never returns + /// one from the entity's set, but the walk reads every loaded subtype, so the entity's own path is denied: + /// a documented limit, which fails closed. + /// + [Fact] + public void Zv_B5_a_subclass_EF_Core_does_not_map_still_denies_the_entitys_path() + { + Assert.Null(_db.Model.FindEntityType(typeof(ZvVehicleView))); + + string where = Code(() => Guard(_db.Vehicles).ToList(Where("Code", "V-1"))); + object? data = SafeRead(() => Guard(_db.Vehicles).ToList(new Filter()).Data); + + _out.WriteLine($"Where={where} whole={JsonSerializer.Serialize(data)}"); + + Assert.Equal(nameof(PolicyErrorCode.FieldDeniedForWhere), where); + Assert.False(Holds(data, "V-1")); + } + + // ------------------------------------------------------------------------------------------------ B3 + + [Fact] + public void Zv_B3_the_startup_scan_reads_a_subtypes_overridable_denial_of_a_default_order_as_a_warning() + { + PolicyModelReport subtype = PolicyModelValidator.Inspect(new[] { typeof(ZvRanked) }); + PolicyModelReport own = PolicyModelValidator.Inspect(new[] { typeof(ZvOwnRanked) }); + + _out.WriteLine("subtype's override: errors=[" + string.Join(" | ", subtype.Errors) + "] warnings=[" + string.Join(" | ", subtype.Warnings) + "]"); + _out.WriteLine("own member: errors=[" + string.Join(" | ", own.Errors) + "] warnings=[" + string.Join(" | ", own.Warnings) + "]"); + + // The control: an overridable denial on the type's own member is a warning. + Assert.Empty(own.Errors); + Assert.NotEmpty(own.Warnings); + + // The same overridable denial on a subtype's override should read the same way. + Assert.Empty(subtype.Errors); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewLeakTests7.cs b/DynamicWhere.Tests/Policies/ReviewLeakTests7.cs new file mode 100644 index 0000000..6a32ad8 --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewLeakTests7.cs @@ -0,0 +1,228 @@ +using System.Collections; +using System.Reflection; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Classes.Core; +using DynamicWhere.ex.Enums; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; + +namespace DynamicWhere.Tests.Policies +{ + // ---- an interface a subtype adds over a member it inherits --------------------------------------------- + + public interface IZwCarded + { + [DwDenied] + string? Pan { get; } + } + + public class ZwAccount + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public string? Pan { get; set; } + } + + /// Declares nothing of its own: the base class's member implements the denied interface member. + public class ZwCardAccount : ZwAccount, IZwCarded + { + } + + // ---- an interface implemented by an override whose base declaration denies ---------------------------- + + public interface IZwPinned + { + int Id { get; } + + string? Pin { get; } + } + + public class ZwPinBase + { + public int Id { get; set; } + + [DwDenied] + public virtual string? Pin { get; set; } + } + + public class ZwPinned : ZwPinBase, IZwPinned + { + public override string? Pin + { + get => base.Pin; + set => base.Pin = value; + } + } + + public class ZwPinHolder + { + public int Id { get; set; } + + public string Label { get; set; } = string.Empty; + + public List Pins { get; set; } = new(); + } + + // ---- one class implementing two instantiations of a generic interface, one of them denied -------------- + + public interface IZwSlot + { + string? Code { get; } + } + + public class ZwTwoSlots : IZwSlot, IZwSlot + { + public int Id { get; set; } + + public string? Open { get; set; } + + public string? Sealed { get; set; } + + string? IZwSlot.Code => Open; + + [DwDenied] + string? IZwSlot.Code => Sealed; + } + + /// + /// A denial on another declaration of a member than the one walked: an interface a subtype adds over a + /// member it inherits, and an implementation that is an override of a denied member. + /// + public class ReviewLeakTests7 + { + private static PolicyQueryable Guard(IQueryable source, DwTier tier = DwTier.Strict) + where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = tier }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + private static Filter Where(string field, string value) => new() + { + ConditionGroup = new ConditionGroup + { + Conditions = { new Condition { Field = field, DataType = DataType.Text, Operator = Operator.Equal, Values = { value } } } + } + }; + + /// True when a value, read by runtime type and through every collection, holds the text. + private static bool Holds(object? value, string text) + { + HashSet seen = new(ReferenceEqualityComparer.Instance); + Stack pending = new(); + + pending.Push(value); + + while (pending.Count > 0) + { + object? current = pending.Pop(); + + if (current is null || current is ValueType) + { + continue; + } + + if (current is string held) + { + if (held == text) + { + return true; + } + + continue; + } + + if (!seen.Add(current)) + { + continue; + } + + if (current is IEnumerable items) + { + foreach (object? item in items) + { + pending.Push(item); + } + + continue; + } + + foreach (PropertyInfo property in current.GetType().GetProperties(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance)) + { + if (property.GetIndexParameters().Length == 0 && property.CanRead) + { + pending.Push(property.GetValue(current)); + } + } + } + + return false; + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void An_interface_a_subtype_adds_over_an_inherited_member_denies_the_base_path(DwTier tier) + { + ZwAccount[] rows = { new ZwCardAccount { Id = 1, Name = "a", Pan = "inherited-pan-secret" } }; + + PolicyException refusal = Assert.ThrowsAny( + () => Guard(rows.AsQueryable(), tier).ToList(Where("Pan", "inherited-pan-secret"))); + + Assert.Equal(PolicyErrorCode.FieldDeniedForWhere, refusal.ErrorCode); + Assert.False(Holds(Guard(rows.AsQueryable(), tier).ToList(new Filter()).Data, "inherited-pan-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void An_implementation_that_overrides_a_denied_member_denies_the_interface_path(DwTier tier) + { + IZwPinned[] rows = { new ZwPinned { Id = 1, Pin = "override-pin-secret" } }; + + PolicyException refusal = Assert.ThrowsAny( + () => Guard(rows.AsQueryable(), tier).ToList(Where("Pin", "override-pin-secret"))); + + Assert.Equal(PolicyErrorCode.FieldDeniedForWhere, refusal.ErrorCode); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void A_list_of_the_interface_is_not_returned_holding_the_denied_value(DwTier tier) + { + ZwPinHolder[] rows = + { + new() { Id = 1, Label = "desk", Pins = { new ZwPinned { Id = 2, Pin = "listed-pin-secret" } } } + }; + + object? data = Guard(rows.AsQueryable(), tier).ToList(new Filter()).Data; + + Assert.False(Holds(data, "listed-pin-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void A_denial_on_one_instantiation_of_a_generic_interface_leaves_the_other_alone(DwTier tier) + { + ZwTwoSlots row = new() { Id = 1, Open = "open", Sealed = "sealed-secret" }; + IZwSlot[] open = { row }; + IZwSlot[] sealedRows = { row }; + + Assert.Single(Guard(open.AsQueryable(), tier).ToList(Where("Code", "open")).Data); + + PolicyException refusal = Assert.ThrowsAny( + () => Guard(sealedRows.AsQueryable(), tier).ToList(Where("Code", "sealed-secret"))); + + Assert.Equal(PolicyErrorCode.FieldDeniedForWhere, refusal.ErrorCode); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewObjectMemberTests.cs b/DynamicWhere.Tests/Policies/ReviewObjectMemberTests.cs new file mode 100644 index 0000000..363fcc8 --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewObjectMemberTests.cs @@ -0,0 +1,453 @@ +using System.Collections; +using System.ComponentModel.DataAnnotations.Schema; +using System.Reflection; +using System.Text.Json; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Xunit.Abstractions; + +namespace DynamicWhere.Tests.Policies +{ + // ============================================================================ members that can hold any object: models + + /// The application type an event's payload column is converted to and from. + public class ZvCardIssued + { + public string Label { get; set; } = string.Empty; + + [DwDenied] + public string? Pan { get; set; } + } + + /// An event whose payload column is typed object and converted to an application type. + public class ZvEvent + { + public int Id { get; set; } + + public string Kind { get; set; } = string.Empty; + + public object? Payload { get; set; } + } + + /// The same, with a top-level denial that asks for a projection on its own. + public class ZvAuditedEvent + { + public int Id { get; set; } + + public string Kind { get; set; } = string.Empty; + + [DwDenied] + public string? Actor { get; set; } + + public object? Payload { get; set; } + } + + /// An owned member holding the converted payload, beside a top-level denial. + public class ZvEnvelope + { + public int Id { get; set; } + + public string Kind { get; set; } = string.Empty; + + [DwDenied] + public string? Actor { get; set; } + + public ZvMeta Meta { get; set; } = new(); + } + + public class ZvMeta + { + public string Tag { get; set; } = string.Empty; + + public object? Payload { get; set; } + } + + /// An unmapped getter typed object, over a private automatically included navigation. + public class ZvKeyring + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + private List Keys { get; set; } = new(); + + [NotMapped] + public object Items => Keys; + + public void Add(ZvKey key) => Keys.Add(key); + } + + /// Control: the same getter typed as the element list, which the policy reads. + public class ZvTypedKeyring + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + private List Keys { get; set; } = new(); + + [NotMapped] + public IReadOnlyList Items => Keys; + + public void Add(ZvTypedKey key) => Keys.Add(key); + } + + public class ZvKey + { + public int Id { get; set; } + + public int ZvKeyringId { get; set; } + + public string Label { get; set; } = string.Empty; + + [DwDenied] + public string? Secret { get; set; } + } + + public class ZvTypedKey + { + public int Id { get; set; } + + public int ZvTypedKeyringId { get; set; } + + public string Label { get; set; } = string.Empty; + + [DwDenied] + public string? Secret { get; set; } + } + + /// + /// An entity whose unmapped member the application fills once EF Core has read it, beside a top-level denial. + /// + public class ZvNotedEvent + { + public int Id { get; set; } + + public string Kind { get; set; } = string.Empty; + + [DwDenied] + public string? Actor { get; set; } + + [NotMapped] + public object? Note { get; set; } + } + + /// The same, one level down: an owned member whose unmapped member the application fills. + public class ZvNoteHolder + { + public int Id { get; set; } + + public string Kind { get; set; } = string.Empty; + + [DwDenied] + public string? Actor { get; set; } + + public ZvNoteMeta Meta { get; set; } = new(); + } + + public class ZvNoteMeta + { + public string Tag { get; set; } = string.Empty; + + [NotMapped] + public object? Note { get; set; } + } + + public sealed class ZvObjectContext : DbContext + { + private readonly SqliteConnection _connection; + + public ZvObjectContext(SqliteConnection connection) => _connection = connection; + + public DbSet Events => Set(); + + public DbSet AuditedEvents => Set(); + + public DbSet Envelopes => Set(); + + public DbSet Keyrings => Set(); + + public DbSet TypedKeyrings => Set(); + + public DbSet NotedEvents => Set(); + + public DbSet NoteHolders => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + private static string Write(object? value) => JsonSerializer.Serialize(value, (JsonSerializerOptions?)null); + + private static object? Read(string text) => JsonSerializer.Deserialize(text, (JsonSerializerOptions?)null); + + protected override void OnModelCreating(ModelBuilder model) + { + model.Entity().Property(e => e.Payload).HasConversion(v => Write(v), s => Read(s)); + model.Entity().Property(e => e.Payload).HasConversion(v => Write(v), s => Read(s)); + model.Entity().OwnsOne(e => e.Meta, m => m.Property(x => x.Payload).HasConversion(v => Write(v), s => Read(s))); + model.Entity().OwnsOne(h => h.Meta); + model.Entity(b => + { + b.HasMany("Keys").WithOne().HasForeignKey(k => k.ZvKeyringId); + b.Navigation("Keys").AutoInclude(); + }); + model.Entity(b => + { + b.HasMany("Keys").WithOne().HasForeignKey(k => k.ZvTypedKeyringId); + b.Navigation("Keys").AutoInclude(); + }); + } + } + + // ============================================================================ members that can hold any object: tests + + /// + /// A member typed object asks for no projection. What EF Core materializes holds no application object, but a + /// converted column can, so once a projection is built for another reason it is left out rather than kept + /// whole; an unmapped getter typed as the list it hands out is read as that list. + /// + public sealed class ReviewObjectMemberTests : IDisposable + { + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZvObjectContext _db; + + public ReviewObjectMemberTests(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + _db = new ZvObjectContext(_connection); + _db.Database.EnsureCreated(); + + _db.Events.Add(new ZvEvent { Kind = "CardIssued", Payload = new ZvCardIssued { Label = "visa", Pan = "event-pan-secret" } }); + _db.AuditedEvents.Add(new ZvAuditedEvent + { + Kind = "CardIssued", Actor = "actor-secret", Payload = new ZvCardIssued { Label = "visa", Pan = "audited-pan-secret" } + }); + _db.Envelopes.Add(new ZvEnvelope + { + Kind = "CardIssued", Actor = "actor-secret", + Meta = new ZvMeta { Tag = "t", Payload = new ZvCardIssued { Label = "visa", Pan = "owned-pan-secret" } } + }); + + ZvKeyring keyring = new() { Name = "K1" }; + keyring.Add(new ZvKey { Label = "front", Secret = "key-secret" }); + _db.Keyrings.Add(keyring); + + ZvTypedKeyring typed = new() { Name = "K2" }; + typed.Add(new ZvTypedKey { Label = "back", Secret = "typed-key-secret" }); + _db.TypedKeyrings.Add(typed); + + _db.NotedEvents.Add(new ZvNotedEvent { Kind = "Noted", Actor = "noted-actor-secret" }); + _db.NoteHolders.Add(new ZvNoteHolder { Kind = "Held", Actor = "holder-actor-secret", Meta = new ZvNoteMeta { Tag = "t1" } }); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static PolicyQueryable Guard(IQueryable source, DwTier tier = DwTier.Strict) + where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = tier, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + private static bool Holds(object? value, string text) + { + HashSet seen = new(ReferenceEqualityComparer.Instance); + Stack pending = new(); + + pending.Push(value); + + while (pending.Count > 0) + { + object? current = pending.Pop(); + + if (current is null || current is ValueType || current is MemberInfo) + { + continue; + } + + if (current is string held) + { + if (held == text) + { + return true; + } + + continue; + } + + if (!seen.Add(current)) + { + continue; + } + + if (current is IEnumerable items) + { + foreach (object? item in items) + { + pending.Push(item); + } + + continue; + } + + foreach (PropertyInfo property in current.GetType().GetProperties()) + { + if (property.GetIndexParameters().Length != 0 || !property.CanRead) + { + continue; + } + + try + { + pending.Push(property.GetValue(current)); + } + catch + { + // A getter that throws holds nothing readable. + } + } + } + + return false; + } + + private static object? SafeRead(Func read) + { + try + { + return read(); + } + catch (PolicyException refusal) when (refusal.ErrorCode is PolicyErrorCode.FieldDeniedForSelect or PolicyErrorCode.AllSelectsDenied) + { + return null; + } + } + + private string Describe(PolicyQueryable guarded, object? data) where T : class => + "sent=" + JsonSerializer.Serialize(data) + " decisions=[" + + string.Join(" | ", guarded.LastTrace?.Decisions + .Where(d => d.Action != PolicyAction.Allowed) + .Select(d => $"{d.FieldPath} {d.Action}: {d.Reason}") ?? Array.Empty()) + "]"; + + // ------------------------------------------------------------------------ a converted column typed object + + /// + /// 3.1.0 synthesized a projection of scalars only whenever a top-level field was denied, so this column was + /// dropped with the denied Actor. Now a synthesized projection keeps an entity's object column whole. + /// + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zv_D5_a_converted_object_column_beside_a_top_level_denial(DwTier tier) + { + Assert.True(Holds(_db.AuditedEvents.AsNoTracking().ToList(), "audited-pan-secret")); + + PolicyQueryable guarded = Guard(_db.AuditedEvents, tier); + object? data = SafeRead(() => guarded.ToList(new Filter()).Data); + + _out.WriteLine(Describe(guarded, data)); + + Assert.False(Holds(data, "actor-secret")); + Assert.False(Holds(data, "audited-pan-secret")); + } + + /// An owned member holding the converted column, beside a top-level denial. + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zv_D5_an_owned_member_holding_a_converted_object_column_beside_a_top_level_denial(DwTier tier) + { + Assert.True(Holds(_db.Envelopes.AsNoTracking().ToList(), "owned-pan-secret")); + + PolicyQueryable guarded = Guard(_db.Envelopes, tier); + object? data = SafeRead(() => guarded.ToList(new Filter()).Data); + + _out.WriteLine(Describe(guarded, data)); + + Assert.False(Holds(data, "owned-pan-secret")); + } + + // ------------------------------------------------------------------------ an unmapped getter typed object + + [Fact] + public void Zv_D6_control_an_unmapped_getter_typed_as_the_list_is_read() + { + Assert.True(Holds(_db.TypedKeyrings.AsNoTracking().ToList(), "typed-key-secret")); + + PolicyQueryable guarded = Guard(_db.TypedKeyrings); + object? data = SafeRead(() => guarded.ToList(new Filter()).Data); + + _out.WriteLine(Describe(guarded, data)); + + Assert.False(Holds(data, "typed-key-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zv_D5_an_unmapped_member_the_application_fills_beside_a_top_level_denial(DwTier tier) + { + _db.ChangeTracker.Clear(); + _db.ChangeTracker.Tracked += (_, tracked) => + { + if (tracked.FromQuery && tracked.Entry.Entity is ZvNotedEvent noted) + { + noted.Note = new ZvCardIssued { Label = "memo", Pan = "unmapped-pan-secret" }; + } + }; + + Assert.True(Holds(_db.NotedEvents.ToList(), "unmapped-pan-secret")); + + _db.ChangeTracker.Clear(); + + object? data = Guard(_db.NotedEvents, tier).ToList(new Filter()).Data; + + Assert.False(Holds(data, "noted-actor-secret")); + Assert.False(Holds(data, "unmapped-pan-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zv_D5_an_owned_member_whose_unmapped_member_the_application_fills(DwTier tier) + { + _db.ChangeTracker.Clear(); + _db.ChangeTracker.Tracked += (_, tracked) => + { + if (tracked.FromQuery && tracked.Entry.Entity is ZvNoteMeta meta) + { + meta.Note = new ZvCardIssued { Label = "memo", Pan = "owned-unmapped-secret" }; + } + }; + + Assert.True(Holds(_db.NoteHolders.ToList(), "owned-unmapped-secret")); + + _db.ChangeTracker.Clear(); + + PolicyQueryable guarded = Guard(_db.NoteHolders, tier); + object? data = guarded.ToList(new Filter()).Data; + + _out.WriteLine(Describe(guarded, data)); + + Assert.False(Holds(data, "holder-actor-secret")); + Assert.False(Holds(data, "owned-unmapped-secret")); + Assert.Contains("t1", JsonSerializer.Serialize(data)); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewOpaqueCollectionTests.cs b/DynamicWhere.Tests/Policies/ReviewOpaqueCollectionTests.cs new file mode 100644 index 0000000..301d96c --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewOpaqueCollectionTests.cs @@ -0,0 +1,180 @@ +using System.Collections; +using System.Reflection; +using System.Text.Json; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Xunit.Abstractions; + +namespace DynamicWhere.Tests.Policies +{ + // ============================================================================ an application class that implements IEnumerable + + /// + /// An application type with its own policed members that also enumerates, non-generically. It can hold + /// anything, and its own members are still read. + /// + public class ZvLedgerBag : IEnumerable + { + public string Owner { get; set; } = string.Empty; + + [DwDenied] + public string? AccountNo { get; set; } + + public IEnumerator GetEnumerator() => Array.Empty().GetEnumerator(); + } + + public class ZvBagHolder + { + public int Id { get; set; } + + public Dictionary Bags { get; set; } = new(); + } + + public class ZvBagBase + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + } + + public class ZvBagSub : ZvBagBase + { + public ZvLedgerBag? Bag { get; set; } + } + + public sealed class ReviewOpaqueCollectionTests + { + private readonly ITestOutputHelper _out; + + public ReviewOpaqueCollectionTests(ITestOutputHelper output) => _out = output; + + private static PolicyQueryable Guard(IQueryable source, DwTier tier = DwTier.Strict) where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = tier, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + /// Everything reachable, read by runtime type; an IEnumerable is read as its properties too. + private static bool Holds(object? value, string text) + { + HashSet seen = new(ReferenceEqualityComparer.Instance); + Stack pending = new(); + + pending.Push(value); + + while (pending.Count > 0) + { + object? current = pending.Pop(); + + if (current is null || current is ValueType || current is MemberInfo) + { + continue; + } + + if (current is string held) + { + if (held == text) + { + return true; + } + + continue; + } + + if (!seen.Add(current)) + { + continue; + } + + if (current is IDictionary map) + { + foreach (object? item in map.Values) + { + pending.Push(item); + } + + continue; + } + + if (current is IEnumerable items && current.GetType().Namespace?.StartsWith("System", StringComparison.Ordinal) == true) + { + foreach (object? item in items) + { + pending.Push(item); + } + + continue; + } + + foreach (PropertyInfo property in current.GetType().GetProperties()) + { + if (property.GetIndexParameters().Length != 0 || !property.CanRead) + { + continue; + } + + try + { + pending.Push(property.GetValue(current)); + } + catch + { + // A getter that throws holds nothing readable. + } + } + } + + return false; + } + + private static object? SafeRead(Func read) + { + try + { + return read(); + } + catch (PolicyException refusal) when (refusal.ErrorCode is PolicyErrorCode.FieldDeniedForSelect or PolicyErrorCode.AllSelectsDenied) + { + return null; + } + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zv_E1_an_enumerable_application_type_inside_a_framework_generic(DwTier tier) + { + ZvBagHolder[] rows = { new() { Id = 1, Bags = { ["a"] = new ZvLedgerBag { Owner = "o", AccountNo = "acct-secret" } } } }; + + PolicyQueryable guarded = Guard(rows.AsQueryable(), tier); + object? data = SafeRead(() => guarded.ToList(new Filter()).Data); + + _out.WriteLine("holds: " + Holds(data, "acct-secret") + "; decisions: " + + string.Join(" | ", guarded.LastTrace?.Decisions.Where(d => d.Action != PolicyAction.Allowed).Select(d => $"{d.FieldPath} {d.Action}") ?? Array.Empty())); + + Assert.False(Holds(data, "acct-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zv_E2_rows_in_memory_whose_subtype_holds_an_enumerable_application_type(DwTier tier) + { + ZvBagBase[] rows = { new ZvBagSub { Id = 1, Name = "n", Bag = new ZvLedgerBag { Owner = "o", AccountNo = "acct-secret" } } }; + + PolicyQueryable guarded = Guard(rows.AsQueryable(), tier); + object? data = SafeRead(() => guarded.ToList(new Filter()).Data); + + _out.WriteLine("holds: " + Holds(data, "acct-secret") + "; types: " + + string.Join(",", ((IEnumerable?)data ?? Array.Empty()).Cast().Select(r => r.GetType().Name))); + + Assert.False(Holds(data, "acct-secret")); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewOverBlockTests.cs b/DynamicWhere.Tests/Policies/ReviewOverBlockTests.cs index adcfc26..d971457 100644 --- a/DynamicWhere.Tests/Policies/ReviewOverBlockTests.cs +++ b/DynamicWhere.Tests/Policies/ReviewOverBlockTests.cs @@ -746,6 +746,22 @@ public void Zb_B2_SelectMany_with_nothing_loaded_beneath_is_not_projected() Assert.False(AnyDropped(guarded), Dropped(guarded)); } + [Fact] + public void Zb_B10_a_filter_inside_a_reshaping_lambda_loads_nothing() + { + // A method call, and a value constructed, that only feed a predicate hand a row nothing. + IQueryable source = _db.Customers.SelectMany(c => c.Orders.Where( + o => EF.Functions.Like(o.Code, "%") && o.Id < new DateTime(2100, 1, 1).Year)); + + Outcome("B10 typed", Guard(source), g => g.ToList(new Filter()).Data); + _db.ChangeTracker.Clear(); + + PolicyQueryable guarded = Guard(source); + + Assert.Single(guarded.ToList(new Filter()).Data); + Assert.False(AnyDropped(guarded), Dropped(guarded)); + } + [Fact] public void Zb_B3_Join_with_nothing_loaded_beneath_is_not_projected() { diff --git a/DynamicWhere.Tests/Policies/ReviewReshapeTests.cs b/DynamicWhere.Tests/Policies/ReviewReshapeTests.cs new file mode 100644 index 0000000..efed892 --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewReshapeTests.cs @@ -0,0 +1,298 @@ +using System.Collections; +using System.Reflection; +using System.Text.Json; +using System.Text.Json.Serialization; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using DynamicWhere.ex.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Xunit.Abstractions; + +namespace DynamicWhere.Tests.Policies +{ + // ============================================================================ reshaped chains: models + + public class ZvShopper + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public List Cards { get; set; } = new(); + } + + public class ZvPayCard + { + public int Id { get; set; } + + public int ZvShopperId { get; set; } + + public string Label { get; set; } = string.Empty; + + [DwDenied] + public string? Pan { get; set; } + } + + public class ZvPurchase + { + public int Id { get; set; } + + public string Code { get; set; } = string.Empty; + + public int ZvShopperId { get; set; } + + public ZvShopper? Shopper { get; set; } + } + + /// What an application's mapper does: builds the row type from what the query hands it. + public static class ZvMapper + { + public static ZvShopper Shopper(ZvShopper shopper, List cards) => + new() { Id = shopper.Id, Name = shopper.Name, Cards = cards }; + } + + public sealed class ZvReshapeContext : DbContext + { + private readonly SqliteConnection _connection; + + public ZvReshapeContext(SqliteConnection connection) => _connection = connection; + + public DbSet Shoppers => Set(); + + public DbSet Purchases => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + } + + // ============================================================================ reshaped chains: tests + + /// + /// A reshaped chain with no include and no object built in its lambdas is read from the model. These are + /// chains that load a navigation anyway: an application's method builds the row, or a lambda captures a query + /// with its own projection or include. An EF Core translation failure of the unguarded query is reported, + /// not asserted. + /// + public sealed class ReviewReshapeTests : IDisposable + { + private static readonly JsonSerializerOptions Json = new() { ReferenceHandler = ReferenceHandler.IgnoreCycles }; + + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZvReshapeContext _db; + + public ReviewReshapeTests(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + _db = new ZvReshapeContext(_connection); + _db.Database.EnsureCreated(); + + ZvShopper shopper = new() { Name = "S1", Cards = { new ZvPayCard { Label = "visa", Pan = "reshape-pan-secret" } } }; + + _db.Shoppers.Add(shopper); + _db.Purchases.Add(new ZvPurchase { Code = "P1", Shopper = shopper }); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static PolicyQueryable Guard(IQueryable source, DwTier tier = DwTier.Strict) + where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = tier, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + private static string Sent(object? rows) => JsonSerializer.Serialize(rows, Json); + + /// Runs the unguarded query and the guarded one, and reports what each returned. + private (string Unguarded, string Guarded) Run(string label, IQueryable source, DwTier tier = DwTier.Strict) where T : class + { + string unguarded; + string guarded; + + try + { + unguarded = Sent(source.AsNoTracking().ToList()); + } + catch (Exception e) + { + unguarded = $"THREW {e.GetType().Name}: {e.Message.Split('\n')[0]}"; + } + + try + { + PolicyQueryable handle = Guard(source, tier); + + guarded = Sent(handle.ToList(new Filter()).Data) + " decisions=[" + + string.Join(" | ", handle.LastTrace?.Decisions.Where(d => d.Action != PolicyAction.Allowed) + .Select(d => $"{d.FieldPath} {d.Action}") ?? Array.Empty()) + "]"; + } + catch (Exception e) + { + guarded = $"THREW {e.GetType().Name}: {e.Message.Split('\n')[0]}"; + } + + RowShape shape = RowShape.Of(source); + + _out.WriteLine($"{label}: shape={shape.Kind} materializes(Cards.Pan)={shape.Materializes("Cards.Pan")} " + + $"reshapes={QueryRoot.Reshapes(source.Expression)} builds={QueryRoot.Builds(source.Expression)}"); + _out.WriteLine($"{label}: unguarded={unguarded}"); + _out.WriteLine($"{label}: guarded={guarded}"); + + return (unguarded, guarded); + } + + // ------------------------------------------------------------------------ D1: a mapper builds the row + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zv_D1_a_reshaped_chain_whose_row_a_method_builds_from_a_loaded_collection(DwTier tier) + { + IQueryable source = _db.Purchases.Select(p => ZvMapper.Shopper(p.Shopper!, p.Shopper!.Cards.ToList())); + + (string unguarded, string guarded) = Run("D1", source, tier); + + Assert.Contains("reshape-pan-secret", unguarded); + Assert.DoesNotContain("reshape-pan-secret", guarded); + } + + // ------------------------------------------------------------------------ D2: the projection is behind a closure + + [Fact] + public void Zv_D2_a_projection_captured_in_a_closure_inside_a_SelectMany() + { + IQueryable built = _db.Shoppers.Select(s => new ZvShopper { Id = s.Id, Name = s.Name, Cards = s.Cards }); + IQueryable source = _db.Purchases.SelectMany(p => built.Where(s => s.Id == p.ZvShopperId)); + + (string unguarded, string guarded) = Run("D2 SelectMany", source); + + if (unguarded.StartsWith("THREW", StringComparison.Ordinal)) + { + return; + } + + Assert.DoesNotContain("reshape-pan-secret", guarded); + } + + [Fact] + public void Zv_D2_a_projection_captured_in_a_closure_inside_a_Select() + { + IQueryable built = _db.Shoppers.Select(s => new ZvShopper { Id = s.Id, Name = s.Name, Cards = s.Cards }); + IQueryable source = _db.Purchases.Select(p => built.First(s => s.Id == p.ZvShopperId)); + + (string unguarded, string guarded) = Run("D2 Select", source); + + if (unguarded.StartsWith("THREW", StringComparison.Ordinal)) + { + return; + } + + Assert.DoesNotContain("reshape-pan-secret", guarded); + } + + /// An object captured from memory holds whatever it holds, which no include accounts for. + [Fact] + public void Zv_D2_an_object_captured_from_memory_inside_a_Select() + { + ZvShopper keeper = new() { Id = 99, Name = "keeper", Cards = { new ZvPayCard { Label = "mem", Pan = "memory-pan-secret" } } }; + IQueryable source = _db.Purchases.Select(p => keeper); + + (string unguarded, string guarded) = Run("D2 memory", source); + + if (unguarded.StartsWith("THREW", StringComparison.Ordinal)) + { + return; + } + + Assert.Contains("memory-pan-secret", unguarded); + Assert.DoesNotContain("memory-pan-secret", guarded); + } + + /// The same, with an include in place of the projection: the include is behind the closure too. + [Fact] + public void Zv_D2_an_include_captured_in_a_closure_inside_a_Select() + { + IQueryable withCards = _db.Shoppers.Include(s => s.Cards); + IQueryable source = _db.Purchases.Select(p => withCards.First(s => s.Id == p.ZvShopperId)); + + (string unguarded, string guarded) = Run("D2 include", source); + + if (unguarded.StartsWith("THREW", StringComparison.Ordinal)) + { + return; + } + + Assert.DoesNotContain("reshape-pan-secret", guarded); + } + + // ------------------------------------------------------------------------ ordinary EF Core queries and the epoch + + // ------------------------------------------------------------------------ D3/D4: ruled-out shapes + + /// An object built only inside a Where subquery loads nothing into the rows. + [Fact] + public void Zv_D3_an_object_built_only_inside_a_Where_subquery() + { + IQueryable source = _db.Purchases + .Where(p => _db.Shoppers.Select(s => new { s.Id, s.Name }).Any(x => x.Id == p.ZvShopperId)) + .Select(p => p.Shopper!); + + (string unguarded, string guarded) = Run("D3", source); + + Assert.DoesNotContain("reshape-pan-secret", unguarded); + Assert.DoesNotContain("reshape-pan-secret", guarded); + } + + /// A Join whose inner source projects: the construction is an argument of the chain, so it is seen. + [Fact] + public void Zv_D4_a_Join_whose_inner_source_projects() + { + IQueryable source = _db.Purchases.Join( + _db.Shoppers.Select(s => new ZvShopper { Id = s.Id, Name = s.Name, Cards = s.Cards }), + p => p.ZvShopperId, + s => s.Id, + (p, s) => s); + + (string unguarded, string guarded) = Run("D4", source); + + if (unguarded.StartsWith("THREW", StringComparison.Ordinal)) + { + return; + } + + Assert.DoesNotContain("reshape-pan-secret", guarded); + } + + /// A Join whose inner source is a projection held in a variable: still an argument, still seen. + [Fact] + public void Zv_D4_a_Join_whose_inner_source_is_a_projection_in_a_variable() + { + IQueryable built = _db.Shoppers.Select(s => new ZvShopper { Id = s.Id, Name = s.Name, Cards = s.Cards }); + IQueryable source = _db.Purchases.Join(built, p => p.ZvShopperId, s => s.Id, (p, s) => s); + + (string unguarded, string guarded) = Run("D4 variable", source); + + if (unguarded.StartsWith("THREW", StringComparison.Ordinal)) + { + return; + } + + Assert.DoesNotContain("reshape-pan-secret", guarded); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewSubtypeIndexTests.cs b/DynamicWhere.Tests/Policies/ReviewSubtypeIndexTests.cs new file mode 100644 index 0000000..f894168 --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewSubtypeIndexTests.cs @@ -0,0 +1,136 @@ +using System.Collections; +using System.Diagnostics; +using System.Reflection; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.DTOs; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Xunit.Abstractions; + +namespace DynamicWhere.Tests.Policies +{ + // ============================================================================ the subtype index: models + + public class ZvIntBox : ZvBox + { + } + + public class ZvStringBox : ZvBox + { + } + + public class ZvMidBox : ZvBox + { + } + + public class ZvLeafBox : ZvMidBox + { + } + + /// An open generic type whose ancestor is closed: it can never be a ZvBox<int>. + public class ZvFixedBox : ZvBox + { + [DwNoGroup] + public override string? Code + { + get => base.Code; + set => base.Code = value; + } + } + + public class ZvDerivedHolder : ZvHolderImpl + { + } + + /// The base type a late-loaded assembly derives from; nothing else in this assembly does. + public class ZvLateBase + { + public int Id { get; set; } + + public virtual string? Code { get; set; } + } + + public class ZvJobRow + { + public int Id { get; set; } + + public string Status { get; set; } = string.Empty; + + public Exception? Error { get; set; } + } + + // ============================================================================ the subtype index: tests + + public sealed class ReviewSubtypeIndexTests + { + private readonly ITestOutputHelper _out; + + public ReviewSubtypeIndexTests(ITestOutputHelper output) => _out = output; + + private static PolicyQueryable Guard(IQueryable source) where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = DwTier.Strict, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + private static int Denials(Type type, string path, PolicyFeature feature) => + new AttributePolicyProvider().GetFragments(type, new DwPolicyContext()) + .Count(f => f.Effect == PolicyEffect.Deny && f.FieldPath == path && (f.Features & feature) != 0); + + // ------------------------------------------------------------------------------------------------ C1 + + [Fact] + public void Zv_C1_generic_subtypes_are_indexed_under_the_instantiations_they_can_be() + { + IReadOnlyList ofIntBox = KnownSubtypes.Of(typeof(ZvBox)); + IReadOnlyList ofHolder = KnownSubtypes.Of(typeof(IZvHolder)); + + _out.WriteLine("Of(ZvBox): " + string.Join(", ", ofIntBox.Select(t => t.Name))); + _out.WriteLine("Of(IZvHolder): " + string.Join(", ", ofHolder.Select(t => t.Name))); + + Assert.Contains(typeof(ZvIntBox), ofIntBox); + Assert.Contains(typeof(ZvLeafBox), ofIntBox); + Assert.Contains(typeof(ZvSecretBox<>), ofIntBox); + Assert.Contains(typeof(ZvMidBox<>), ofIntBox); + Assert.DoesNotContain(typeof(ZvStringBox), ofIntBox); + + Assert.Contains(typeof(ZvHolderImpl<>), ofHolder); + Assert.Contains(typeof(ZvDerivedHolder<>), ofHolder); + Assert.Contains(typeof(ZvGenPlainHolder<>), KnownSubtypes.Of(typeof(IZvPlainHolder))); + } + + /// What the interface map says for an open generic implementation and a closed interface. + [Fact] + public void Zv_C1_the_interface_map_of_an_open_generic_implementation_for_a_closed_interface() + { + Exception? closed = Record.Exception(() => typeof(ZvHolderImpl<>).GetInterfaceMap(typeof(IZvHolder))); + Exception? open = Record.Exception(() => typeof(ZvHolderImpl<>).GetInterfaceMap(typeof(ZvHolderImpl<>).GetInterfaces()[0])); + + _out.WriteLine($"GetInterfaceMap(ZvHolderImpl<>, IZvHolder): {closed?.GetType().Name ?? "ok"} {closed?.Message}"); + _out.WriteLine($"GetInterfaceMap(ZvHolderImpl<>, IZvHolder): {open?.GetType().Name ?? "ok"}"); + + Assert.NotNull(closed); + Assert.Null(open); + } + + /// + /// An open generic type whose ancestor is a different closed instantiation is not a subtype of this one, + /// so its override's denial does not reach a path no row of it can be on. + /// + [Fact] + public void Zv_C1_an_open_generic_type_over_another_closed_instantiation_is_not_its_subtype() + { + _out.WriteLine("Of(ZvBox) holds ZvFixedBox<>: " + KnownSubtypes.Of(typeof(ZvBox)).Contains(typeof(ZvFixedBox<>))); + _out.WriteLine("ZvBox.Code group denials: " + Denials(typeof(ZvBox), "Code", PolicyFeature.Group)); + _out.WriteLine("ZvBox.Code group denials: " + Denials(typeof(ZvBox), "Code", PolicyFeature.Group)); + + // ZvFixedBox derives from ZvBox only; a ZvBox is never one, and a ZvBox can be. + Assert.DoesNotContain(typeof(ZvFixedBox<>), KnownSubtypes.Of(typeof(ZvBox))); + Assert.Contains(typeof(ZvFixedBox<>), KnownSubtypes.Of(typeof(ZvBox))); + } + } +} diff --git a/DynamicWhere.ex/Policies/DTOs/PolicyTrace.cs b/DynamicWhere.ex/Policies/DTOs/PolicyTrace.cs index ae88f09..4f5dd44 100644 --- a/DynamicWhere.ex/Policies/DTOs/PolicyTrace.cs +++ b/DynamicWhere.ex/Policies/DTOs/PolicyTrace.cs @@ -64,6 +64,15 @@ public void Add(PolicyDecision decision) _decisions.Add(decision); } + /// How many decisions have been recorded. + internal int Count => _decisions.Count; + + /// + /// Withdraws every decision recorded after the first : the steps of an action + /// that was then not taken, such as members left out of a projection that is not built. + /// + internal void Withdraw(int count) => _decisions.RemoveRange(count, _decisions.Count - count); + /// Remembers that the caller reached a canonical path by writing another name. internal void RecordSpelling(string canonicalPath, string spoken) => _spoken[canonicalPath] = spoken; diff --git a/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs b/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs index 402c241..57f5288 100644 --- a/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs +++ b/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs @@ -200,9 +200,9 @@ private static void Walk( } /// - /// The denials a member carries from another declaration of it: the interface member it implements, - /// and, in a type loaded below the one walked, the override or the implementation that a row of that - /// type runs. + /// The denials a member carries from another declaration of it: an interface member it implements, + /// and, in a type loaded below the one walked, the override, the member hidden with new, or the + /// implementation that a row of that type runs. /// /// /// Attributes are read from the declaration walked, and inheritance only reaches up the chain. A row @@ -211,19 +211,95 @@ private static void Walk( /// class's implementation of IAccount.Iban, was never seen through the base path, which filtered, /// sorted, grouped and returned it. Each such denial applies to the path for every row, since a /// projection cannot withhold a field from some rows only. + /// + /// Every declaration a row can run counts: an override of either accessor, an implementation declared + /// explicitly, inherited from a base class or by an open generic class, and an interface a subtype adds + /// over a member it inherits. So does a member a subtype hides with new: whether it reads the + /// member it hides cannot be told from outside, and a row serialized as its own type writes it under the + /// same name. + /// /// - internal static IEnumerable DenialsElsewhere(Type type, PropertyInfo property) + internal static IReadOnlyList DenialsElsewhere(Type type, PropertyInfo property) + { + int epoch = KnownSubtypes.Epoch; + + if (Elsewhere.TryGetValue((type, property), out (int Epoch, DwDenyAttribute[] Denials) known) && known.Epoch == epoch) + { + return known.Denials; + } + + DwDenyAttribute[] denials = ReadDenialsElsewhere(type, property).ToArray(); + + Elsewhere[(type, property)] = (epoch, denials); + + return denials; + } + + /// The denials of each member's other declarations, read once for each type and member until another assembly loads. + private static readonly ConcurrentDictionary<(Type Type, PropertyInfo Property), (int Epoch, DwDenyAttribute[] Denials)> Elsewhere = new(); + + private static IEnumerable ReadDenialsElsewhere(Type type, PropertyInfo property) { - if (property.GetGetMethod() is not { } getter) + MethodInfo[] accessors = property.GetAccessors(nonPublic: true); + + if (accessors.Length == 0) { yield break; } - if (!type.IsInterface) + // Many rows reach one declaration, an interface every subtype implements, and it is read once. + HashSet read = new() { property }; + + foreach (Type row in KnownSubtypes.Of(type).Prepend(type)) { - foreach (Type contract in type.GetInterfaces()) + if (row.IsInterface) + { + continue; + } + + // What a row of this type runs for the member: the member, or what the row declares in its + // place; read through an interface, the row's implementation of it. + List runs = new(); + + if (type.IsInterface) { - if (Declaration(Implemented(type, contract, getter), contract) is { } declared) + foreach (PropertyInfo implementation in Implementations(row, type, accessors)) + { + runs.AddRange(implementation.GetAccessors(nonPublic: true)); + + if (read.Add(implementation)) + { + foreach (DwDenyAttribute attribute in implementation.GetCustomAttributes(inherit: true)) + { + yield return attribute; + } + } + } + } + else + { + runs.AddRange(accessors); + + if (row != type) + { + foreach (PropertyInfo below in Redeclarations(row, property)) + { + runs.AddRange(below.GetAccessors(nonPublic: true)); + + if (read.Add(below)) + { + foreach (DwDenyAttribute attribute in below.GetCustomAttributes(inherit: false)) + { + yield return attribute; + } + } + } + } + } + + foreach (PropertyInfo declared in Declarations(row, runs, accessors)) + { + if (read.Add(declared)) { foreach (DwDenyAttribute attribute in declared.GetCustomAttributes(inherit: false)) { @@ -232,100 +308,109 @@ internal static IEnumerable DenialsElsewhere(Type type, Propert } } } + } + + /// + /// What a subtype declares in a member's place: an override of either accessor, or a member of the same + /// name that hides it. Both carry the member's name. + /// + private static IEnumerable Redeclarations(Type subtype, PropertyInfo property) => + subtype + .GetProperties(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.DeclaredOnly) + .Where(candidate => candidate.Name == property.Name); + + /// + /// The properties a class implements an interface's member with. A class that is itself open generic + /// implements the interface over its own type parameters, which may be any instantiation, so that one + /// is read too. + /// + private static IEnumerable Implementations(Type row, Type contract, MethodInfo[] accessors) + { + List found = new(); - foreach (Type subtype in KnownSubtypes.Of(type)) + foreach (Type implemented in row.GetInterfaces()) { - PropertyInfo? below = type.IsInterface - ? Implementation(subtype, type, getter) - : Override(subtype, getter); + bool same = implemented == contract + || (implemented.IsGenericType && contract.IsGenericType + && (implemented.ContainsGenericParameters || contract.ContainsGenericParameters) + && implemented.GetGenericTypeDefinition() == contract.GetGenericTypeDefinition()); - if (below is null) + if (!same || Map(row, implemented) is not { } map) { continue; } - foreach (DwDenyAttribute attribute in below.GetCustomAttributes(inherit: false)) + for (int i = 0; i < map.InterfaceMethods.Length; i++) { - yield return attribute; + if (accessors.Any(accessor => Same(map.InterfaceMethods[i], accessor)) + && Declaring(map.TargetMethods[i]) is { } implementation + && !found.Contains(implementation)) + { + found.Add(implementation); + } } } + + return found; } - /// The property a subtype declares that overrides a getter, or null. - private static PropertyInfo? Override(Type subtype, MethodInfo getter) + /// + /// The interface properties a class implements with any of the accessors it runs, other than the member + /// walked itself. + /// + private static IEnumerable Declarations(Type row, List runs, MethodInfo[] own) { - if (!getter.IsVirtual) - { - return null; - } + MethodInfo[] roots = runs.Select(accessor => accessor.GetBaseDefinition()).ToArray(); + List found = new(); - MethodInfo root = getter.GetBaseDefinition(); - - foreach (PropertyInfo candidate in subtype.GetProperties( - BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.DeclaredOnly)) + foreach (Type contract in row.GetInterfaces()) { - if (candidate.GetGetMethod(nonPublic: true) is { } overriding - && Same(overriding.GetBaseDefinition(), root) - && !Same(overriding, getter)) + if (Map(row, contract) is not { } map) { - return candidate; + continue; } - } - return null; - } + for (int i = 0; i < map.TargetMethods.Length; i++) + { + MethodInfo declared = map.InterfaceMethods[i]; - /// The property a type declares that implements an interface's getter, or null. - private static PropertyInfo? Implementation(Type subtype, Type contract, MethodInfo getter) - { - if (subtype.IsInterface || Implemented(subtype, contract, getter) is not { } target) - { - return null; + if (roots.Any(root => Same(map.TargetMethods[i].GetBaseDefinition(), root)) + && !own.Any(accessor => Same(accessor, declared)) + && Declaring(declared) is { } property + && !found.Contains(property)) + { + found.Add(property); + } + } } - return target.DeclaringType? - .GetProperties(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.DeclaredOnly) - .FirstOrDefault(candidate => candidate.GetGetMethod(nonPublic: true) is { } method && Same(method, target)); + return found; } /// - /// For a class member, the interface getter it implements; for an interface getter, the method a type - /// implements it with. Null when there is none, or when the runtime cannot map an open generic type. + /// A class's interface map, or null when the runtime cannot map it, as for some open generic types. + /// Read once for each class and interface. /// - private static MethodInfo? Implemented(Type type, Type contract, MethodInfo getter) - { - InterfaceMapping map; - - try - { - map = type.GetInterfaceMap(contract); - } - catch (Exception exception) when (exception is ArgumentException or InvalidOperationException or NotSupportedException) + private static InterfaceMapping? Map(Type type, Type contract) => + Maps.GetOrAdd((type, contract), static key => { - return null; - } - - for (int i = 0; i < map.InterfaceMethods.Length; i++) - { - if (Same(map.InterfaceMethods[i], getter)) + try { - return map.TargetMethods[i]; + return key.Type.GetInterfaceMap(key.Contract); } - - if (Same(map.TargetMethods[i], getter)) + catch (Exception exception) when (exception is ArgumentException or InvalidOperationException or NotSupportedException) { - return map.InterfaceMethods[i]; + return null; } - } + }); - return null; - } + private static readonly ConcurrentDictionary<(Type Type, Type Contract), InterfaceMapping?> Maps = new(); - /// The interface property an interface getter belongs to. - private static PropertyInfo? Declaration(MethodInfo? method, Type contract) => - method is null || method.DeclaringType != contract - ? null - : contract.GetProperties().FirstOrDefault(candidate => candidate.GetGetMethod() is { } getter && Same(getter, method)); + /// The property an accessor belongs to, found on the type that declares the accessor. + private static PropertyInfo? Declaring(MethodInfo accessor) => + accessor.DeclaringType? + .GetProperties(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.DeclaredOnly) + .FirstOrDefault(candidate => candidate.GetAccessors(nonPublic: true).Any(method => Same(method, accessor))); private static bool Same(MethodInfo left, MethodInfo right) => left.MetadataToken == right.MetadataToken && left.Module == right.Module; diff --git a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs index 3cc6ee5..258cb98 100644 --- a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs +++ b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs @@ -1894,6 +1894,7 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) List allowed = new(); List<(string Member, string Reason)> uncarried = new(); bool anyDenied = false; + int recorded = gate.TraceCount; foreach (PropertyInfo member in gate.Members()) { @@ -1994,7 +1995,16 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) gate.LeaveOut(path, "left out: a type derived from the row's type declares it, and the projection builds the row's type"); } - if (!anyDenied || gate.IsDryRun) + // A member that asks for nothing itself is still left out, or narrowed, as the projection would build + // it. With no projection built nothing was, and the trace says what the query did. + if (!anyDenied) + { + gate.WithdrawTrace(recorded); + + return null; + } + + if (gate.IsDryRun) { return null; } @@ -2997,10 +3007,11 @@ internal bool ReadOnlyTransformBeneath(string member) /// On an entity query the model says what loads beneath the member /// (). Each loaded member is asked about by its path, and by /// its own attribute where no fragment reaches it: deeper than the walker goes, or declared by a - /// type the model derives. A value EF Core reads whole is read through its type. What it read - /// from the database holds no object of the application's, so an entity's members never count as - /// holding one. A projection's member is read as the type the projection constructs it as, when - /// it says. Any other value can carry whatever the member's type can hold, its subtypes included. + /// type the model derives. A value EF Core reads whole is read through its type. What EF Core + /// materializes from the database holds no object of the application's; what a value converter + /// hands back is the application's code, and holds what its type allows. A + /// projection's member is read as the type the projection constructs it as, when it says. Any + /// other value can carry whatever the member's type can hold, its subtypes included. /// Under a policy that denies every field it does not name, a path the walk never asks about is a /// denied one unless the policy names it. /// @@ -3009,16 +3020,19 @@ internal TypeFacts Unnamed(string member, RowShape rows) if (rows.LoadedBeneath(member) is { } loaded) { bool denies = false; + bool holdsObject = false; foreach (Loaded entry in loaded) { - denies |= Denies(entry.Path, entry.Property) - || (entry.Whole - && !HoldsValue(entry.Property.PropertyType) - && Carried(entry.Property.PropertyType, entry.Path, exact: false).DeniesSelect); + TypeFacts carried = entry.Whole && !HoldsValue(entry.Property.PropertyType) + ? Carried(entry.Property.PropertyType, entry.Path, exact: false) + : default; + + denies |= Denies(entry.Path, entry.Property) || carried.DeniesSelect; + holdsObject |= entry.Converted && carried.HoldsObject; } - return new TypeFacts(denies, false, loaded.Count > 0); + return new TypeFacts(denies, holdsObject, loaded.Count > 0); } if (!member.Contains(SegmentSeparator) && rows.BuiltType(member) is { } built) @@ -3052,8 +3066,8 @@ private TypeFacts Carried(Type type, string path, bool exact) /// /// The walker puts a fragment on every path of the declared types up to its depth, a member /// with no setter, a path around a cycle, and a denial an override or an implementation declares - /// included. Past its depth, or on a member only a derived type declares, the attribute is all - /// there is. + /// included. Past its depth, or on a member only a derived type declares, the attributes are all + /// there is, read from every declaration a row can run. /// private bool Denies(string path, PropertyInfo property) { @@ -3064,10 +3078,34 @@ private bool Denies(string path, PropertyInfo property) return (path.Split(SegmentSeparator).Length > AttributePolicyProvider.MaxDepth || Property(_entityType, path) is null) - && property.GetCustomAttributes(inherit: true) - .Any(attribute => (attribute.Features & PolicyFeature.Select) != 0); + && DeclaresDenial(property); } + /// + /// True when a member's attributes deny Select: its own, inherited ones, and those of another + /// declaration a row of the type it was read from can run, an interface member or a subtype's + /// override among them. Read once for each member, and again once another assembly has loaded. + /// + private static bool DeclaresDenial(PropertyInfo property) + { + int epoch = KnownSubtypes.Epoch; + + if (DenialsByMember.TryGetValue(property, out (int Epoch, bool Denies) known) && known.Epoch == epoch) + { + return known.Denies; + } + + bool denies = property.GetCustomAttributes(inherit: true) + .Concat(AttributePolicyProvider.DenialsElsewhere(property.ReflectedType ?? property.DeclaringType!, property)) + .Any(attribute => (attribute.Features & PolicyFeature.Select) != 0); + + DenialsByMember[property] = (epoch, denies); + + return denies; + } + + private static readonly ConcurrentDictionary DenialsByMember = new(); + /// /// Every member a type derived from T declares that a row can carry and this caller may not /// have, or that carries a denied field no path names. A projection builds T itself and leaves @@ -3293,7 +3331,12 @@ void Enqueue(Type candidate, bool root) { holdsObject = true; - return; + // An application's own collection that is not generic can hold anything, and it still + // declares members of its own, which are read as any other type's are. + if (peeled.IsGenericParameter || AttributePolicyProvider.IsFramework(peeled)) + { + return; + } } if (AttributePolicyProvider.NavigationTypeOf(candidate) is { } navigation) @@ -3334,8 +3377,7 @@ void Enqueue(Type candidate, bool root) continue; } - if (property.GetCustomAttributes(inherit: true) - .Any(attribute => (attribute.Features & PolicyFeature.Select) != 0)) + if (!denies && DeclaresDenial(property)) { denies = true; } @@ -3802,6 +3844,12 @@ internal void DenySegmentInference(string fieldPath, FieldPolicy policy) throw Exception(fieldPath, PolicyFeature.Segment, PolicyErrorCode.FieldDeniedForSegment, policy); } + /// How many decisions the query's trace holds. + internal int TraceCount => _trace.Count; + + /// Withdraws the decisions recorded after the first . + internal void WithdrawTrace(int count) => _trace.Withdraw(count); + /// /// Records that a synthesized projection left a member out whole, and why: something beneath it /// is denied and the member cannot be narrowed. diff --git a/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs b/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs index 066c2ad..000f22c 100644 --- a/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs +++ b/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs @@ -73,7 +73,7 @@ internal static IReadOnlyList Of(Type type) foreach (Type candidate in byDefinition) { - if ((type.ContainsGenericParameters || candidate.ContainsGenericParameters) && !found.Contains(candidate)) + if ((type.ContainsGenericParameters || CanBe(candidate, type)) && !found.Contains(candidate)) { found.Add(candidate); } @@ -83,6 +83,34 @@ internal static IReadOnlyList Of(Type type) return found is null ? Array.Empty() : found; } + /// + /// True when some instantiation of an open generic subtype derives from or implements a closed type: + /// its own base or interface of that definition is the type, or is still open. One over another + /// instantiation, class Fixed<T> : Base<string>, never holds a Base<int>. + /// + private static bool CanBe(Type candidate, Type type) + { + if (!candidate.ContainsGenericParameters) + { + return false; + } + + Type definition = type.GetGenericTypeDefinition(); + IEnumerable ancestors = type.IsInterface ? Interfaces(candidate) : BaseTypes(candidate); + + return ancestors.Any(ancestor => ancestor.IsGenericType + && ancestor.GetGenericTypeDefinition() == definition + && (ancestor == type || ancestor.ContainsGenericParameters)); + } + + private static IEnumerable BaseTypes(Type type) + { + for (Type? ancestor = type.BaseType; ancestor is not null; ancestor = ancestor.BaseType) + { + yield return ancestor; + } + } + /// The index for the assemblies loaded now, built once for each epoch. private static Index Current() { @@ -112,7 +140,14 @@ private static Index Build(int epoch) { HashSet assemblies = new(AppDomain.CurrentDomain.GetAssemblies().Where(Searched)); - assemblies.UnionWith(SeenLoading.Keys); + // One the domain lists now needs no holding on to: this index reads it, and every later one will. + foreach (Assembly seen in SeenLoading.Keys) + { + if (!assemblies.Add(seen)) + { + SeenLoading.TryRemove(seen, out _); + } + } Dictionary> descendants = new(); diff --git a/DynamicWhere.ex/Policies/Source/RowShape.cs b/DynamicWhere.ex/Policies/Source/RowShape.cs index 24c7f9a..5170ebe 100644 --- a/DynamicWhere.ex/Policies/Source/RowShape.cs +++ b/DynamicWhere.ex/Policies/Source/RowShape.cs @@ -289,7 +289,7 @@ internal bool Materializes(string path) // A value read whole: what it holds is its type's to say, and a path inside it names part of it. if (entity.FindProperty(member.Name) is not null || ComplexProperties(entity).Contains(member.Name)) { - return i == segments.Length - 1 ? new List { new(prefix, member, true) } : null; + return i == segments.Length - 1 ? new List { new(prefix, member, true, Converted(entity, member.Name)) } : null; } INavigationBase? navigation = @@ -373,7 +373,7 @@ private bool Collect( if (type.FindProperty(member.Name) is not null || ComplexProperties(type).Contains(member.Name)) { - loaded.Add(new Loaded(path, member, true)); + loaded.Add(new Loaded(path, member, true, Converted(type, member.Name))); continue; } @@ -406,6 +406,12 @@ private bool Collect( return true; } + /// + /// True when a column's value comes from a value converter, which is the application's code and can hand + /// back an object of any type its member's type allows. + /// + private static bool Converted(IEntityType type, string name) => type.FindProperty(name)?.GetValueConverter() is not null; + /// True when something loads a navigation of an entity reached along a path. private bool Loads(IEntityType owner, INavigationBase navigation, string path) => (navigation is INavigation owned && IsOwned(owned)) @@ -782,7 +788,11 @@ private static HashSet ReadComplexProperties(IEntityType entityType) /// The member's path from the root. /// The member. /// True when EF Core reads it whole: a column or a complex property. -internal readonly record struct Loaded(string Path, PropertyInfo Property, bool Whole); +/// +/// True when a value converter hands back its value: the application's code, which can return an object of any +/// type the member's type allows. +/// +internal readonly record struct Loaded(string Path, PropertyInfo Property, bool Whole, bool Converted = false); /// Where a guarded query's rows come from. internal enum RowKind diff --git a/DynamicWhere.ex/Policies/Validation/PolicyModelValidator.cs b/DynamicWhere.ex/Policies/Validation/PolicyModelValidator.cs index 8779dc7..5b9b16f 100644 --- a/DynamicWhere.ex/Policies/Validation/PolicyModelValidator.cs +++ b/DynamicWhere.ex/Policies/Validation/PolicyModelValidator.cs @@ -195,15 +195,18 @@ private static void CheckDefaultOrder(Type type, List errors, List - /// True when the member a path ends on is denied a feature by its own attributes, and every one of - /// those attributes is overridable. + /// True when the member a path ends on is denied a feature by its attributes, and every one of those + /// attributes is overridable. They include those of the member's other declarations: an interface + /// member it implements, and a subtype's override. /// private static bool DeniedOnlyOverridably(Type type, string path, PolicyFeature feature) { PropertyInfo? member = null; + Type owner = type; foreach (string segment in path.Split('.')) { + owner = type; member = CacheReflection.FindProperty(type, segment); if (member is null) @@ -216,6 +219,7 @@ private static bool DeniedOnlyOverridably(Type type, string path, PolicyFeature List denials = member! .GetCustomAttributes(inherit: true) + .Concat(Resolution.AttributePolicyProvider.DenialsElsewhere(owner, member!)) .Where(attribute => (attribute.Features & feature) == feature) .ToList(); diff --git a/DynamicWhere.ex/Source/QueryRoot.cs b/DynamicWhere.ex/Source/QueryRoot.cs index 329b861..d326d85 100644 --- a/DynamicWhere.ex/Source/QueryRoot.cs +++ b/DynamicWhere.ex/Source/QueryRoot.cs @@ -1,6 +1,9 @@ using System.Collections.Concurrent; using System.Linq.Expressions; using System.Reflection; +using DynamicWhere.ex.Optimization.Cache.Source; +using DynamicWhere.ex.Policies.Resolution; +using Microsoft.EntityFrameworkCore; using Microsoft.EntityFrameworkCore.Metadata; namespace DynamicWhere.ex.Source; @@ -105,13 +108,28 @@ or nameof(Queryable.Intersect) or "IntersectBy" or nameof(Queryable.Except) or " } /// - /// True when a call along the chain that does not know constructs an object in - /// one of its lambdas: a projection, such as the one behind Select(x => x), an anonymous row - /// or a conditional, assigns what it builds, and loads whatever navigation it assigns. + /// True when a call along the chain that does not know can hand its rows an + /// object the query's own includes do not account for: one a lambda constructs, one an application's + /// method returns, and one a lambda captured, another query's rows among them. /// - internal static bool Builds(Expression expression) + /// + /// A projection, such as the one behind Select(x => x), an anonymous row or a conditional, + /// assigns what it builds, and loads whatever navigation it assigns. A method can return anything. A + /// query captured in a variable becomes part of the query EF Core runs, with its own includes and + /// projections, and an object captured from memory holds whatever it holds. A value, such as a + /// predicate's result, a key or a date, carries none of them, so what only feeds one is not read. + /// + internal static bool Builds(Expression expression) => Builds(expression, depth: 0); + + private static bool Builds(Expression expression, int depth) { - ConstructionFinder finder = new(); + // A captured query that captures itself would never end; one this deep is counted as building. + if (depth > MaxCapturedDepth) + { + return true; + } + + ForeignFinder finder = new(depth); for (Expression? node = expression; node is MethodCallExpression call; node = call.Arguments.Count > 0 ? call.Arguments[0] : null) @@ -119,7 +137,7 @@ internal static bool Builds(Expression expression) if (call.Arguments.Count > 1 && call.Method.Name is nameof(Queryable.Concat) or nameof(Queryable.Union) or "UnionBy" or nameof(Queryable.Intersect) or "IntersectBy" or nameof(Queryable.Except) or "ExceptBy" - && Builds(call.Arguments[1])) + && Builds(call.Arguments[1], depth)) { return true; } @@ -143,12 +161,23 @@ or nameof(Queryable.Intersect) or "IntersectBy" or nameof(Queryable.Except) or " return false; } - /// Finds an object construction anywhere in an expression. - private sealed class ConstructionFinder : ExpressionVisitor + /// How many captured queries deep reads before it stops. + private const int MaxCapturedDepth = 8; + + /// + /// Finds, anywhere in an expression, what can hand a row an object its query does not load: an object + /// constructed, an application's method's result, and an object or a query captured from outside. + /// + private sealed class ForeignFinder : ExpressionVisitor { + private readonly int _depth; + + internal ForeignFinder(int depth) => _depth = depth; + internal bool Found { get; private set; } - public override Expression? Visit(Expression? node) => Found ? node : base.Visit(node); + public override Expression? Visit(Expression? node) => + Found || node is null || IsValue(node.Type) ? node : base.Visit(node); protected override Expression VisitNew(NewExpression node) { @@ -163,6 +192,167 @@ protected override Expression VisitMemberInit(MemberInitExpression node) return node; } + + protected override Expression VisitListInit(ListInitExpression node) + { + Found = true; + + return node; + } + + protected override Expression VisitNewArray(NewArrayExpression node) + { + Found = true; + + return node; + } + + protected override Expression VisitMethodCall(MethodCallExpression node) + { + if (Reads(node.Method)) + { + return base.VisitMethodCall(node); + } + + Found = true; + + return node; + } + + protected override Expression VisitMember(MemberExpression node) + { + if (!IsCaptured(node)) + { + return base.VisitMember(node); + } + + Found = !TryEvaluate(node, out object? value) || Holds(value, _depth); + + return node; + } + + protected override Expression VisitConstant(ConstantExpression node) + { + Found = Holds(node.Value, _depth); + + return node; + } + } + + /// + /// True for a method that hands back what it was given, or reads it: LINQ's operators, EF Core's + /// EF.Property and query operators, and a context's Set, which names a query root. + /// + private static bool Reads(MethodInfo method) + { + Type? declaring = method.DeclaringType; + + return declaring == typeof(Queryable) + || declaring == typeof(Enumerable) + || declaring == typeof(EF) + || declaring == typeof(EntityFrameworkQueryableExtensions) + || (declaring == typeof(DbContext) && method.Name == nameof(DbContext.Set)); + } + + /// True when a value of the type holds only values: a string, a number, a date, or a collection of them. + private static bool IsValue(Type type) => CacheReflection.IsSimpleType(AttributePolicyProvider.Peeled(type)); + + /// True for a member read off something the lambda captured, or off nothing, rather than off its parameters. + private static bool IsCaptured(MemberExpression node) + { + Expression? target = node.Expression; + + while (target is MemberExpression member) + { + target = member.Expression; + } + + return target is null or ConstantExpression; + } + + /// Reads a captured member's value as EF Core does before it runs the query. + private static bool TryEvaluate(MemberExpression node, out object? value) + { + value = null; + + object? target = null; + + if (node.Expression is ConstantExpression constant) + { + target = constant.Value; + } + else if (node.Expression is MemberExpression member && !TryEvaluate(member, out target)) + { + return false; + } + + try + { + value = node.Member switch + { + FieldInfo field => field.GetValue(target), + PropertyInfo property => property.GetValue(target), + _ => null + }; + + return node.Member is FieldInfo or PropertyInfo; + } + catch (Exception exception) when (exception is TargetException or TargetInvocationException + or ArgumentException or InvalidOperationException + or NotSupportedException or MemberAccessException) + { + return false; + } + } + + /// + /// True when a captured value can hand a row an object its query does not load: an object in memory, + /// or a query with its own includes or projections, or one over rows in memory. + /// + private static bool Holds(object? value, int depth) + { + if (value is null || value is DbContext || IsValue(value.GetType())) + { + return false; + } + + if (value is not IQueryable query) + { + return true; + } + + if (query.Provider is EnumerableQuery) + { + return !IsValue(query.ElementType); + } + + IncludeFinder includes = new(); + + includes.Visit(query.Expression); + + return includes.Found || Builds(query.Expression, depth + 1); + } + + /// Finds an Include or a ThenInclude anywhere in a query. + private sealed class IncludeFinder : ExpressionVisitor + { + internal bool Found { get; private set; } + + public override Expression? Visit(Expression? node) => Found ? node : base.Visit(node); + + protected override Expression VisitMethodCall(MethodCallExpression node) + { + if (node.Method.DeclaringType == typeof(EntityFrameworkQueryableExtensions) + && node.Method.Name is nameof(EntityFrameworkQueryableExtensions.Include) + or nameof(EntityFrameworkQueryableExtensions.ThenInclude)) + { + Found = true; + + return node; + } + + return base.VisitMethodCall(node); + } } /// Finds the first query root that names an entity type. From bdbe55c9e385ec69988a40052a2e3e9dd730a882 Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 09:25:53 +0300 Subject: [PATCH 18/22] docs: other declarations, captured queries, converted columns and the limits left open The override rule now names a member hidden with new, a setter-only override and the implementations an open generic class or a subtype gives. A reshaped chain's lambda counts when it hands its rows an object from an application's method or a captured query or object. A converted column that can hold any object is left out when a projection is built. The limits add a subclass EF Core does not map, opaque converted and unmapped object members with nothing else denied, and unloadable load contexts. Co-Authored-By: Claude Opus 5 --- DynamicWhere.ex/DOC.md | 12 ++-- DynamicWhere.ex/DynamicWhere.ex.csproj | 2 +- .../app/docs/breaking-changes/page.tsx | 20 ++++-- .../app/docs/policies/attributes/page.tsx | 16 +++-- .../app/docs/policies/configuration/page.tsx | 24 +++++-- .../app/docs/policies/security/page.tsx | 7 +- OfficialWebsite/public/llms.txt | 67 ++++++++++++------- README.md | 4 +- 8 files changed, 96 insertions(+), 56 deletions(-) diff --git a/DynamicWhere.ex/DOC.md b/DynamicWhere.ex/DOC.md index 4d2ca69..2a3d98e 100644 --- a/DynamicWhere.ex/DOC.md +++ b/DynamicWhere.ex/DOC.md @@ -1645,7 +1645,7 @@ A field the default keeps is a use of that field. One audited for `Order`, by `[ `DwPolicy.ValidateModel(options, types)` inspects the policy attributes on the given types and throws `InvalidOperationException` listing every error; `PolicyModelValidator.Inspect(types, options)` returns the same `PolicyModelReport` — `Errors`, `Warnings`, `IsValid` — without throwing. Called at startup, either one lets a misconfiguration fail the deployment rather than a caller's request. Since 3.1.0 the scan also reports: - every `[DwForceWhere]` resolution would refuse: `Value` and `ContextValue` both set or both missing on a comparison, either one set on a null check, a member whose type has no `DataType`, and `AllowNull` with `IsNull` / `IsNotNull` or on a member that can never be null. Before, these surfaced on the first query that resolved them; -- every `[DwEntity(DefaultOrder = ...)]` entry a guarded query would skip or leave out. An entry that is not a field optionally followed by `asc` or `desc` is an error; a field whose name starts with one of the words the expression parser keeps is an error, judged before the type is asked whether it has the member, because it may well have it — `"{Type}: DefaultOrder names '{field}', which starts with a name the expression parser keeps for itself, so no query can use it. Rename the member."`; a field the type does not have is a warning; a field no query can order by, such as a collection of entities, is an error; a field the type's own attributes seal against ordering is an error, because every guarded query would leave it out. A field denied for ordering only by attributes marked `Overridable = true` is a warning, because a rule can lift the denial for some callers — `"{Type}: DefaultOrder names '{field}', which its attributes deny for ordering unless a rule allows it, so guarded queries leave it out until one does."` — and so is a field the attributes deny for segments: `"{Type}: DefaultOrder names '{field}', which its attributes deny for segments, so guarded segments leave it out."` +- every `[DwEntity(DefaultOrder = ...)]` entry a guarded query would skip or leave out. An entry that is not a field optionally followed by `asc` or `desc` is an error; a field whose name starts with one of the words the expression parser keeps is an error, judged before the type is asked whether it has the member, because it may well have it — `"{Type}: DefaultOrder names '{field}', which starts with a name the expression parser keeps for itself, so no query can use it. Rename the member."`; a field the type does not have is a warning; a field no query can order by, such as a collection of entities, is an error; a field the type's own attributes seal against ordering is an error, because every guarded query would leave it out; the attributes include those of the member's other declarations, an interface member it implements and a subtype's override. A field denied for ordering only by attributes marked `Overridable = true` is a warning, because a rule can lift the denial for some callers — `"{Type}: DefaultOrder names '{field}', which its attributes deny for ordering unless a rule allows it, so guarded queries leave it out until one does."` — and so is a field the attributes deny for segments: `"{Type}: DefaultOrder names '{field}', which its attributes deny for segments, so guarded segments leave it out."` ### Blocked-action semantics @@ -1696,7 +1696,7 @@ The projection keeps the **allowed members**: what an unguarded call would retur - A field denied at the top of `T` always asks for one, whatever it holds: a scalar, a blob, a list, an owned object or a JSON column. - A field denied beneath a member asks for one when its value can reach the result. On an entity, that is beneath a column, an owned or complex member, or a navigation something loads: an `Include` or `ThenInclude` on the query, an automatic include, or a lazy loader — EF Core's proxies, an injected `ILazyLoader`, a loader delegate or `ILazyLoader` the constructor takes and keeps in a field or any property, the asynchronous loader delegate of EF Core 7, or an injected `DbContext` — which fills a navigation after the query. On a row a projection builds, it is beneath a member the initializer assigns; a constructor with arguments counts every member as assigned, and an initializer after it still says what its own bindings hold. On a row in memory, it is beneath any member. A rule may spell the path in any letter case. -- Every navigation counts as loaded where the library cannot read which the query loads: an `Include` in a form it cannot read, one off the query's own chain, and a chain that reaches its rows through anything but the root's own rows — `Select(o => o.Customer)`, a `SelectMany`, a `Join`, a `GroupBy` — when it also has an include, which EF Core applies from the root to the entities it reaches, or builds an object in one of its lambdas, as a projection behind an identity `Select`, a member of an anonymous row or a conditional does. Such a chain with neither is read from the model. +- Every navigation counts as loaded where the library cannot read which the query loads: an `Include` in a form it cannot read, one off the query's own chain, and a chain that reaches its rows through anything but the root's own rows — `Select(o => o.Customer)`, a `SelectMany`, a `Join`, a `GroupBy` — when it also has an include, which EF Core applies from the root to the entities it reaches, or when one of its lambdas hands its rows an object: one it builds, as a projection behind an identity `Select`, a member of an anonymous row or a conditional does; one an application's method returns; or one it captured, another query with its own include or projection, or an object in memory. What only feeds a predicate or a key is a value and hands a row nothing. Such a chain with none of these is read from the model. - A denial beneath a navigation nothing loads never leaves the database, so it asks for no projection. An entity whose only denials sit beneath such navigations is read as it was in 3.1.0. - A member whose value can hold a field denied for `Select` that no path names — deeper than the walker's four segments, inside a framework generic such as `Dictionary`, or declared by a subtype of its type — asks for one too, read as for a named member, so on an entity only what loads counts. Under a `"*"` deny, so does a member whose value can hold a path the walk never asks about and the policy does not name. - A member that can hold an object of any type — one typed `object`, a framework interface such as `IComparable`, or a collection that is not generic, such as `IEnumerable`, `ArrayList` or an application's own — asks for nothing on its own: the policy cannot see into it whether or not a projection is built. @@ -1717,7 +1717,7 @@ A value EF Core does not map is left out: computing it would make EF Core read t **Whole, narrowed or left out whole.** A member holding an object that the source carries is: -- **kept whole** when nothing beneath it is denied, nothing its value can hold is denied (its subtypes included), it cannot hold an object of any type (asked of a projected row or a row in memory: what an entity read from the database never holds one), under a `"*"` deny every path beneath it the walk skips is one the policy names, no forced scope is beneath it, and no transform beneath it lands on a property with no setter; +- **kept whole** when nothing beneath it is denied, nothing its value can hold is denied (its subtypes included), it cannot hold an object of any type (asked of a projected row, a row in memory, and an entity's column a value converter hands back: what EF Core materializes itself never holds one), under a `"*"` deny every path beneath it the walk skips is one the policy names, no forced scope is beneath it, and no transform beneath it lands on a property with no setter; - **narrowed** otherwise, to the allowed fields beneath it, four segments deep, as a caller naming it would get it, where the core's narrowing translates: an object the projection's initializer builds, a list a subquery reads into a type the core can bind (not an array or a set), a navigation that is neither complex nor stored as JSON, or an entity's owned member not stored as JSON. The narrowing builds the declared type, so a subtype's fields are dropped. A field beneath it that can hold what the policy cannot name is left out; - **left out whole** otherwise, and recorded as `Dropped` on `Select` with a reason that starts `left out whole`. @@ -1752,9 +1752,9 @@ The last one is recorded on the field beneath the member that the narrowing leav - A dry run synthesizes nothing. It records the denials and returns the rows whole. - A simulation has no source, so it reads `T` as a source it cannot see into: every denial beneath a member counts, and the projection it shows keeps only members holding a value (see [Administration](#administration)). -**What the policy cannot see into.** A member typed `object`, a framework interface or a collection that is not generic, such as `IEnumerable`, `ArrayList` or an application's own, is opaque to the policy. It never asks for a projection; when one is needed anyway, a projected row or a row in memory leaves it out and an entity keeps it; and naming it returns whatever it holds. A framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: naming it is refused in both tiers where the core cannot narrow it, at the top of `T` or on a row in memory, narrowed away under `Convenience` beneath a navigation, and a synthesized projection leaves it out. Hold such values in a list of the policed type instead. A member EF Core does not map is read as its type, since its getter can hand out what EF Core loaded; a getter that copies a denied column into a type with no denial is the application's to withhold. +**What the policy cannot see into.** A member typed `object`, a framework interface or a collection that is not generic, such as `IEnumerable`, `ArrayList` or an application's own, is opaque to the policy. It never asks for a projection; when one is needed anyway, a projected row, a row in memory, and an entity's converted column leave it out, and an entity's other columns keep it; and naming it returns whatever it holds. A converter returning an application type through a column typed `object` is opaque the same way, so type the member as what it holds. An application's own such collection still has its own members read. A framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: naming it is refused in both tiers where the core cannot narrow it, at the top of `T` or on a row in memory, narrowed away under `Convenience` beneath a navigation, and a synthesized projection leaves it out. Hold such values in a list of the policed type instead. A member EF Core does not map is read as its type, since its getter can hand out what EF Core loaded; a getter that copies a denied column into a type with no denial is the application's to withhold. -**A projection builds the declared type.** A query over the root of a hierarchy whose derived type declares a denied field comes back as root-type rows, the derived types' allowed fields dropped too. Over an abstract root the typed terminals fail with `SelectTypeMustHaveParameterlessConstructor`, and the dynamic ones return the root's members. Query the derived type, `OfType()`, to keep its fields. Rows in memory can be any loaded subtype, and the policy does not look at the rows: when a subtype declares a denied field, they are projected and their objects left out, even if no row is that subtype. Under a `"*"` deny, a member is kept whole only when every path beneath it the walk skips is one the policy names. A `[DwDenied]` on an override, or on a class's implementation of an interface member, applies to the path through the base type or the interface, on every row and in every clause. +**A projection builds the declared type.** A query over the root of a hierarchy whose derived type declares a denied field comes back as root-type rows, the derived types' allowed fields dropped too. Over an abstract root the typed terminals fail with `SelectTypeMustHaveParameterlessConstructor`, and the dynamic ones return the root's members. Query the derived type, `OfType()`, to keep its fields. Rows in memory can be any loaded subtype, and the policy does not look at the rows: when a subtype declares a denied field, they are projected and their objects left out, even if no row is that subtype. Under a `"*"` deny, a member is kept whole only when every path beneath it the walk skips is one the policy names. A `[DwDenied]` on an override, on a member a subtype hides with `new`, or on a class's implementation of an interface member, applies to the path through the base type or the interface, on every row and in every clause. **A forced scope on a list's element type filters rows, not elements.** A forced scope declared on a list's element type filters the rows that hold the list, never its elements. `Selects` naming the list returns every element, those the scope excludes included, as in every release; a synthesized projection leaves such a list out. Scope the elements where the row is built. @@ -2318,7 +2318,7 @@ All validation errors throw `LogicException` (inherits `Exception`) with one of Fixed (security): a field denied only beneath a member, none at the top of `T`, synthesized nothing, so the whole row came back with the denied value in it — in a list or nested object of a row projected before `ApplyPolicy`, in a row held in memory, and in an entity's included, automatically included, lazily loaded or owned member — typed and dynamic, in both tiers, for a `Filter` and a `Segment`. Such a denial now synthesizes the projection whenever its value can reach the result: on an entity, beneath a column, an owned or complex member, or a navigation the query loads through `Include`, an automatic include or a lazy loader; on a projected row, beneath a member the initializer assigns; in memory, beneath any member. A denial beneath a navigation nothing loads never leaves the database, and the entity is read exactly as in 3.1.0. - Fixed (security): what a query loads was read too narrowly. An include named from the root and reached through `Select(o => o.Customer)`, `SelectMany` or `Join`, a projection behind another `Select`, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or a property of any name each loaded a denied value the gate read as unloaded. An injected `DbContext` or EF Core 7's asynchronous loader delegate did too. A field a subtype declares, a derived entity's or a subclass's held by a base-typed member, was not read at all, nor was a `[DwDenied]` on an override or on an interface member's implementation, and under a `"*"` deny a path the walk never asked about was allowed. Each came back. Now every navigation counts as loaded on such a chain, the subtypes are read, and such a path is denied. When a type the model derives from `T`, or a loaded subclass of a row in memory, declares a denied field, the rows are projected to `T`, dropping a derived type's allowed fields too; over an abstract `T` the typed terminals fail with `SelectTypeMustHaveParameterlessConstructor` and the dynamic ones return its members. + Fixed (security): what a query loads was read too narrowly. An include named from the root and reached through `Select(o => o.Customer)`, `SelectMany` or `Join`, a projection behind another `Select`, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or a property of any name each loaded a denied value the gate read as unloaded. An injected `DbContext` or EF Core 7's asynchronous loader delegate did too, and so did a reshaping lambda that got its row from an application's method or from a captured query or object. A converted column typed `object` was kept whole when a projection was built for another field, and an application's own non-generic collection hid its own denied members. A field a subtype declares, a derived entity's or a subclass's held by a base-typed member, was not read at all, nor was a `[DwDenied]` on an override, on a member hidden with `new` or on an interface member's implementation, and under a `"*"` deny a path the walk never asked about was allowed. Each came back. Now every navigation counts as loaded on such a chain, the subtypes are read, and such a path is denied. When a type the model derives from `T`, or a loaded subclass of a row in memory, declares a denied field, the rows are projected to `T`, dropping a derived type's allowed fields too; over an abstract `T` the typed terminals fail with `SelectTypeMustHaveParameterlessConstructor` and the dynamic ones return its members. Fixed (security): a field denied at the top of `T` whose own type is not a simple value — a byte array, a list, an owned object, a JSON column — synthesized no projection either, so with nothing else denied the whole row came back with it. diff --git a/DynamicWhere.ex/DynamicWhere.ex.csproj b/DynamicWhere.ex/DynamicWhere.ex.csproj index 22cc870..fc15864 100644 --- a/DynamicWhere.ex/DynamicWhere.ex.csproj +++ b/DynamicWhere.ex/DynamicWhere.ex.csproj @@ -52,7 +52,7 @@ Security fix: a field denied for Select only beneath a member was not enforced when the caller sent no Selects. A guarded query synthesizes a projection for a denied field, and it did so only when a simple field at the top of the type was denied. With every denial beneath a member, the whole row came back with the denied value in it: in a list or nested object of a row projected before ApplyPolicy, in a row held in memory, and in an entity's included, automatically included, lazily loaded or owned member, typed and dynamic, in both tiers, for a Filter and a Segment. Such a denial now synthesizes the projection whenever its value can reach the result. On an entity that means beneath a column, an owned or complex member, or a navigation the query loads through Include, an automatic include or a lazy loader; a denial beneath a navigation nothing loads never leaves the database, and the entity is read exactly as before. -Security fix: what a query loads, and what a member holds, was read too narrowly. An include named from the root and reached through Select(o => o.Customer), SelectMany or Join, a projection behind another Select, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or any property each loaded a denied value the gate read as unloaded. An injected DbContext and EF Core 7's asynchronous loader delegate loaded one too. A field a subtype declares, a derived entity's, a subclass's held by a base-typed member or an open generic one's, was not read at all, nor was a [DwDenied] on an override or on an interface member's implementation, and under a "*" deny with exact allows a path the walk never asked about (past four segments, around a cycle, with no setter) resolved as allowed. Selects naming an entity navigation returned a denial in its owned chain past four segments or in a converted Dictionary<string, T> column. All of these are read now: from the EF Core model for an entity, so only what loads counts, and from every loaded subtype for a projected or in-memory row. Rows whose derived type declares a denied field come back as the queried type. A denial on an override or an implementation applies to the base type's or the interface's path, on every row and in every clause. +Security fix: what a query loads, and what a member holds, was read too narrowly. An include named from the root and reached through Select(o => o.Customer), SelectMany or Join, a projection behind another Select, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or any property each loaded a denied value the gate read as unloaded. An injected DbContext and EF Core 7's asynchronous loader delegate loaded one too, and so did a reshaping lambda that got its row from an application's method or from a captured query or object. A converted column typed object was kept whole when a projection was built for another field, and an application's own non-generic collection hid its own denied members. A field a subtype declares, a derived entity's, a subclass's held by a base-typed member or an open generic one's, was not read at all, nor was a [DwDenied] on an override, on a member hidden with new or on an interface member's implementation, and under a "*" deny with exact allows a path the walk never asked about (past four segments, around a cycle, with no setter) resolved as allowed. Selects naming an entity navigation returned a denial in its owned chain past four segments or in a converted Dictionary<string, T> column. All of these are read now: from the EF Core model for an entity, so only what loads counts, and from every loaded subtype for a projected or in-memory row. Rows whose derived type declares a denied field come back as the queried type. A denial on an override, a member hidden with new or an implementation applies to the base type's or the interface's path, on every row and in every clause. Security fix: a field denied at the top of a type whose own type is not a simple value, such as a byte array, a list, an owned object or a JSON column, synthesized no projection either, so with nothing else denied it came back. diff --git a/OfficialWebsite/app/docs/breaking-changes/page.tsx b/OfficialWebsite/app/docs/breaking-changes/page.tsx index a3509fc..6f83235 100644 --- a/OfficialWebsite/app/docs/breaking-changes/page.tsx +++ b/OfficialWebsite/app/docs/breaking-changes/page.tsx @@ -1156,11 +1156,16 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or a property of any name each loaded a denied value the gate read as unloaded, and so did an injected{" "} - DbContext or EF Core 7's asynchronous loader delegate. - A field a subtype declares — a derived entity's, or a + DbContext or EF Core 7's asynchronous loader delegate, + and a reshaping lambda that got its row from an application's + method or from a captured query or object. A converted column typed{" "} + object was kept whole when a projection was built for + another field, and an application's own non-generic collection hid + its own denied members. A field a subtype declares — a derived entity's, or a subclass's held by a base-typed member — was not read at all, nor - was a [DwDenied] on an override or on an interface - member's implementation, and under a{" "} + was a [DwDenied] on an override, on a member hidden with{" "} + new or on an interface member's implementation, and + under a{" "} "*" deny a path the walk never asked about was allowed. Each came back. @@ -1174,9 +1179,10 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say return its members. Query the derived type,{" "} {`OfType()`}, to keep its fields. Rows in memory can be any loaded subtype, so there the rows are projected whenever one - declares a denied field. A [DwDenied] on an override or on - an interface member's implementation denies the base path for - every row, in every clause. + declares a denied field. A [DwDenied] on an override, on a + member a subtype hides with new, or on an interface + member's implementation denies the base path for every row, in + every clause. A field denied at the top of T whose own type is not a diff --git a/OfficialWebsite/app/docs/policies/attributes/page.tsx b/OfficialWebsite/app/docs/policies/attributes/page.tsx index 92cebda..200b9f5 100644 --- a/OfficialWebsite/app/docs/policies/attributes/page.tsx +++ b/OfficialWebsite/app/docs/policies/attributes/page.tsx @@ -235,13 +235,15 @@ public string EmployeeCode { get; set; }`} An access-control attribute on another declaration of a member applies - to its path too: on the interface member a class implements, on an - override a subtype declares, and on the implementation a type gives an - interface member. A row read through the base type or the interface is - still that subtype, so the denial holds for the path on every row, in - every clause. Until 3.2.0 only the declaration walked, and the - attributes above it, were read. The other attributes are still read from - the declaration walked only. + to its path too: on the interface member a class or its subtype + implements with it, on an override of either accessor a subtype + declares, on a member a subtype hides with new, and on the + implementation a type gives an interface member, explicit, inherited or + declared by an open generic class. A row read through the base type or + the interface is still that subtype, so the denial holds for the path on + every row, in every clause. Until 3.2.0 only the declaration walked, and + the attributes above it, were read. The other attributes are still read + from the declaration walked only.

      Injection

      diff --git a/OfficialWebsite/app/docs/policies/configuration/page.tsx b/OfficialWebsite/app/docs/policies/configuration/page.tsx index 911f7ce..719fcf6 100644 --- a/OfficialWebsite/app/docs/policies/configuration/page.tsx +++ b/OfficialWebsite/app/docs/policies/configuration/page.tsx @@ -201,9 +201,13 @@ export default function Page() { {`Select(o => o.Customer)`}, a SelectMany, a{" "} Join, a GroupBy — when it also has an include, which EF Core applies from the root to the entities it - reaches, or builds an object in one of its lambdas, as a projection - behind an identity Select, a member of an anonymous row or - a conditional does. Such a chain with neither is read from the model. + reaches, or when one of its lambdas hands its rows an object: one it + builds, as a projection behind an identity Select, a + member of an anonymous row or a conditional does; one an + application's method returns; or one it captured, another query + with its own include or projection, or an object in memory. What only + feeds a predicate or a key is a value and hands a row nothing. Such a + chain with none of these is read from the model.
    • A denial beneath a navigation nothing loads never leaves the database, @@ -304,8 +308,9 @@ export default function Page() {
    • kept whole when nothing beneath it is denied, nothing its value can hold is denied (its subtypes included), it cannot hold an - object of any type (asked of a projected row or a row in memory: what - an entity read from the database never holds one), under a{" "} + object of any type (asked of a projected row, a row in memory, and an + entity's column a value converter hands back: what EF Core + materializes itself never holds one), under a{" "} "*" deny every path beneath it the walk skips is one the policy names, no forced scope is beneath it, and no transform beneath it lands on a property with no setter; @@ -409,8 +414,13 @@ left out whole: it can hold what the policy cannot name`} collection that is not generic, such as IEnumerable,{" "} ArrayList or an application's own, is opaque to the policy. It never asks for a projection; when one is needed anyway, a - projected row or a row in memory leaves it out and an entity keeps it; - and naming it returns whatever it holds. A framework generic holding a + projected row, a row in memory, and an entity's converted column + leave it out, and an entity's other columns keep it; + and naming it returns whatever it holds. A converter returning an + application type through a column typed object is opaque + the same way, so type the member as what it holds. An + application's own collection still has its own members read. A + framework generic holding a policed type, such as Dictionary<string, LineDto>, has no paths beneath it: naming it is refused in both tiers where the core cannot narrow it, narrowed away under Convenience beneath a diff --git a/OfficialWebsite/app/docs/policies/security/page.tsx b/OfficialWebsite/app/docs/policies/security/page.tsx index cc2e4bf..1bc493e 100644 --- a/OfficialWebsite/app/docs/policies/security/page.tsx +++ b/OfficialWebsite/app/docs/policies/security/page.tsx @@ -301,12 +301,13 @@ true order. Add [DwNoOrder] unless that is intended.`}
      • - + diff --git a/OfficialWebsite/public/llms.txt b/OfficialWebsite/public/llms.txt index 11ab5e9..bdc7735 100644 --- a/OfficialWebsite/public/llms.txt +++ b/OfficialWebsite/public/llms.txt @@ -1976,12 +1976,15 @@ No named attribute refuses `Segment`: write `[DwDeny(PolicyFeature.Segment)]`. - `[DwDeny(PolicyFeature.None)]` refuses nothing, and nothing reports it. - `[DwNoSelect]` leaves the field filterable, sortable, groupable and aggregatable, so it can still be counted. -- One on another declaration of the member applies to the path too (3.2.0): on the interface member a class - implements, on an override a loaded subtype declares, and on the implementation a loaded type gives an interface - member. A row read through the base type or the interface is still that subtype, and its member returns what the - subtype's declaration returns, so the denial holds for the path on every row, Where, Order, Group and Select - alike. Only the deny family is read this way; aliases, transforms and the other attributes are read from the - declaration walked. +- One on another declaration of the member applies to the path too (3.2.0): on the interface member a class, or a + loaded subtype of it, implements with the member; on an override of either accessor a loaded subtype declares; on + a member a loaded subtype hides with `new`; and on the implementation a loaded type gives an interface member, + explicit, inherited from a base class or declared by an open generic class. A row read through the base type or + the interface is still that subtype, and its member returns what the subtype's declaration returns, so the denial + holds for the path on every row, Where, Order, Group and Select alike. A member hidden with `new` counts because + whether it reads the member it hides cannot be told from outside, and a row serialized as its own type writes it + under the same name. Only the deny family is read this way; aliases, transforms and the other attributes are read + from the declaration walked. ### DwOperators @@ -2367,9 +2370,12 @@ DefaultOrder field denied for Segment, in a Segment (3.1.0) left out l include in a form it cannot read, an include off the query's own chain (on a join's inner source, say), and a chain that reaches its rows through anything but the root's own rows (`Select(o => o.Customer)`, a `SelectMany`, a `Join`, a `GroupBy`) when it also has an include, which EF Core applies from the root to the - entities it reaches, or builds an object in one of its lambdas (a projection behind an identity `Select`, a - member of an anonymous row, a conditional), which loads whatever it assigns. Such a chain with neither is read - from the model. A denial beneath a navigation nothing loads never leaves the database and needs no projection, + entities it reaches, or when one of its lambdas hands its rows an object: one it builds (a projection behind an + identity `Select`, a member of an anonymous row, a conditional), which loads whatever it assigns; one an + application's method returns; or one it captured, another query with its own include or projection, or an + object in memory. What only feeds a predicate or a key is a value and hands a row nothing. Such a chain with + none of these is read from the model. A denial beneath a navigation nothing loads never leaves the database + and needs no projection, so a connected model is read as it was in 3.1.0. A member EF Core does not map counts as loaded: its getter can hand out a mapped field or a private navigation, so its type is read whole. - On a row a projection builds: beneath a member its initializer assigns. A constructor with arguments counts @@ -2382,7 +2388,7 @@ DefaultOrder field denied for Segment, in a Segment (3.1.0) left out l - A member that can hold an object of any type, one typed `object`, a framework interface such as `IComparable`, an unbound type parameter, or a collection that is not generic (`IEnumerable`, `ArrayList`, `Array`, an application's own), asks for nothing on its own: the policy cannot see into it whether or not a projection is - built. + built. An application's own such collection still has its own members read, as any type's are. - A row can be a subtype of T. On an entity, each member a type the model derives from T declares, and what loads beneath it, counts as one of T's own would; on a row in memory, each member any loaded subtype declares; on a row a projection builds, each member the type its initializer constructs declares below T. The projection builds @@ -2405,7 +2411,8 @@ DefaultOrder field denied for Segment, in a Segment (3.1.0) left out l - every allowed member holding an object or a list of them that the source carries: a member a projection's initializer assigns, and an entity's columns (converted or JSON), owned and complex members. It is kept whole when nothing beneath it is denied, nothing its value can hold is denied, it cannot hold an object of any type - (asked of a projected row or a row in memory: what an entity read from the database never holds one), under a + (asked of a projected row, a row in memory, and an entity's column a value converter hands back: what EF Core + materializes itself never holds one), under a `"*"` deny every path beneath it the walk skips is one the policy names, no forced scope is beneath it, and no transform beneath it lands on a property with no setter; - otherwise it is narrowed to the allowed leaves beneath it, to four segments, as a caller naming it would get, @@ -2783,7 +2790,8 @@ Errors: - a field no query can order by, a path ending on a collection of entities: `"{Type}: DefaultOrder names '{field}', which no query can order by, so guarded queries skip it."` - a field the type's own attributes deny for ordering, unless every one of those denials is `Overridable` (then a - warning, below): + warning, below). The attributes include those of the member's other declarations, an interface member it + implements and a subtype's override (3.2.0): `"{Type}: DefaultOrder names '{field}', which its attributes deny for ordering, so every guarded query leaves it out."` - a stage that cannot be built: - `Round` or `Bucket` with `Step <= 0`, or `Truncate` with `Decimals < 0` @@ -4775,9 +4783,11 @@ MemoryCalculationInput projection behind another Select (an identity Select, a member of an anonymous row, a conditional). So does a lazy loader the constructor takes, delegate or ILazyLoader, kept in a field or a property of any name, and an initializer after a constructor with arguments counts every member as assigned. So do an injected - DbContext and EF Core 7's asynchronous loader delegate. Each returned the denied value. A reshaped chain - with no include and no object built in its lambdas is still read from the model, so it is not projected - for a denial beneath a navigation it does not load. + DbContext and EF Core 7's asynchronous loader delegate. So does a reshaped chain whose lambda hands its rows + an object an application's method returns or one it captured: another query with its own include or + projection, or an object in memory. Each returned the denied value. A reshaped chain with none of these is + still read from the model, so it is not projected for a denial beneath a navigation it does not load, and a + value that only feeds a predicate or a key no longer counts as building its rows. - Security fix and behaviour change. A member declared as a base type or an interface holds its subtypes, whose denied fields the declared type never names. They are read now: the types the EF Core model derives, for an entity, and every loaded subtype for a projected or in-memory row, an open generic one and an @@ -4788,9 +4798,10 @@ MemoryCalculationInput typed terminals then fail with SelectTypeMustHaveParameterlessConstructor, as for any T they cannot build; the dynamic terminals return the root's allowed members. A rule on a subtype's field through a base-typed member was dropped as naming nothing; it is enforced. - - Security fix. A deny-family attribute on an override, on the implementation of an interface member, or on - the interface member a class implements, was read only from its own declaration, so the base type's or the - interface's path filtered, sorted, grouped and returned the value. It applies to the path now. + - Security fix. A deny-family attribute on an override, on a member a subtype hides with new, on the + implementation of an interface member, or on the interface member a class implements, was read only from + its own declaration, so the base type's or the interface's path filtered, sorted, grouped and returned the + value. It applies to the path now. - Security fix. Under a "*" deny with exact allows, a path the walk never asked about (past four segments, around a cycle, with no setter, on a subtype) resolved as allowed, so a member holding one was returned whole, named or not. Each such path is asked of the policy now; one it does not name is denied, and a @@ -4822,9 +4833,11 @@ MemoryCalculationInput whole with a "left out whole" Dropped decision. An entity's navigations and the objects of a row in memory are left out, as before, and each one the unguarded call would have returned is recorded as Dropped; a value EF Core does not map is left out too. A member that can hold an object of any type, a geometry, a - JSON bag or a BitArray column say, asks for no projection on its own, and an entity keeps it whole. A - projected member is read as the type its initializer constructs, and an initializer after a constructor - with arguments narrows its own bindings. + JSON bag or a BitArray column say, asks for no projection on its own, and an entity keeps it whole, unless a + value converter hands back its value: a converter is the application's code, so such a column is left out. + An application's own collection that is not generic has its own members read. A projected member is read as + the type its initializer constructs, and an initializer after a constructor with arguments narrows its own + bindings. The trace records a member left out only when a projection is built. - Behaviour change. [DwEntity(DefaultOrder)] applies to a projected source whose outermost Select builds T in an object initializer assigning every field the default names a column: a mapped member, read directly, through reference navigations or through EF.Property. A computed value or any other projection still leaves @@ -5001,8 +5014,11 @@ MemoryCalculationInput projection leaves such a list out. Scope the elements where the row is built. - A member typed `object`, a framework interface or a collection that is not generic (`IEnumerable`, `ArrayList`, `Array`, an application's own) is opaque to the policy. It never asks for a projection; when one is needed anyway, - a projected row or a row in memory leaves it out and an entity keeps it (what EF Core read from the database holds - no application object); and naming it returns whatever it holds. + a projected row, a row in memory, and an entity's converted column leave it out, and an entity's other columns + keep it (what EF Core materializes itself holds no application object); and naming it + returns whatever it holds. A value converter that returns an application type through a column typed `object`, + and an unmapped getter typed `object` over a private navigation, are opaque the same way: with nothing else + denied the row comes back as loaded. Type the member as what it holds. A framework generic holding a policed type (`Dictionary`) has no paths beneath it: naming it is refused in both tiers where the core cannot narrow it (at the top of T, or on a row in memory), narrowed away under Convenience beneath a navigation, and a synthesized projection leaves it out. @@ -5021,6 +5037,11 @@ MemoryCalculationInput - Rows in memory can be any loaded subtype, and the policy does not look at the rows. When a subtype of T declares a denied field, or a subtype or implementation of a member's type does, the rows are projected and their objects left out, even if no row is that subtype. +- A deny-family attribute on an override, or on a member a subtype hides with `new`, denies the base path for every + subtype loaded, not only those the EF Core model maps: a view model deriving from an entity and overriding one of + its members decides the entity's own path. Declare such a class apart from the entity to keep the path open. +- Types from an unloadable `AssemblyLoadContext` stay referenced by the policy's caches, so the context is not + collected while the process runs. - A default order reaches a projected row only through columns of the entity its `Select` reads: a projection over an anonymous or other intermediate row takes no default. - A simulation reads T as a source it cannot see into (section 23): every denial beneath a member counts, and a diff --git a/README.md b/README.md index 000daf7..f397e99 100644 --- a/README.md +++ b/README.md @@ -380,7 +380,7 @@ The complete reference — every enum, class, extension method, validation rule, **Upgrade note — the security fixes and the three changes alter what code written for 3.1.0 does, and the new overloads can stop a call from compiling. Read these before bumping.** - **Fixed (security): a field denied beneath a member reached a caller who sent no `Selects`.** A guarded query synthesizes a projection for a denied field, and it did so only when a simple field at the top of the type was denied. With every denial beneath a member, the whole row came back with the denied value in it: in a list or nested object of a row projected before `ApplyPolicy`, in a row held in memory, and in an entity's included, automatically included, lazily loaded or owned member — typed and dynamic, in both tiers, for a `Filter` and a `Segment`. Such a denial now synthesizes the projection whenever its value can reach the result. On an entity that means beneath a column, an owned or complex member, or a navigation the query loads through `Include`, an automatic include or a lazy loader. A denial beneath a navigation nothing loads never leaves the database, and the entity is read exactly as before. -- **Fixed (security): what a query loads, and what a member holds, was read too narrowly.** An include named from the root and reached through `Select(o => o.Customer)`, `SelectMany` or `Join`, a projection behind another `Select`, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or any property each loaded a denied value the gate read as unloaded. An injected `DbContext` and EF Core 7's asynchronous loader delegate loaded one too. A field a subtype declares — a derived entity's, a subclass's held by a base-typed member, an open generic one's — was not read at all, nor was a `[DwDenied]` on an override or on an interface member's implementation, and under a `"*"` deny with exact allows a path the walk never asked about (past four segments, around a cycle, with no setter) resolved as allowed. `Selects` naming an entity navigation returned a denial in its owned chain past four segments or in a converted `Dictionary` column. All of these are read now: from the EF Core model for an entity, so only what loads counts, and from every loaded subtype for a projected or in-memory row. A denial on an override or an implementation applies to the base type's or the interface's path, on every row and in every clause. +- **Fixed (security): what a query loads, and what a member holds, was read too narrowly.** An include named from the root and reached through `Select(o => o.Customer)`, `SelectMany` or `Join`, a projection behind another `Select`, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or any property each loaded a denied value the gate read as unloaded. An injected `DbContext` and EF Core 7's asynchronous loader delegate loaded one too, and so did a reshaping lambda that got its row from an application's method or from a captured query or object. A converted column typed `object` was kept whole when a projection was built for another field, and an application's own non-generic collection hid its own denied members. A field a subtype declares — a derived entity's, a subclass's held by a base-typed member, an open generic one's — was not read at all, nor was a `[DwDenied]` on an override, on a member hidden with `new` or on an interface member's implementation, and under a `"*"` deny with exact allows a path the walk never asked about (past four segments, around a cycle, with no setter) resolved as allowed. `Selects` naming an entity navigation returned a denial in its owned chain past four segments or in a converted `Dictionary` column. All of these are read now: from the EF Core model for an entity, so only what loads counts, and from every loaded subtype for a projected or in-memory row. A denial on an override, a member hidden with `new` or an implementation applies to the base type's or the interface's path, on every row and in every clause. - **Fixed (security): a denied member that holds no simple value came back.** A field denied at the top of the type whose own type is not a simple value — a byte array, a list, an owned object, a JSON column — synthesized no projection either, so with nothing else denied it came back. - **Fixed (security): an application namespace starting with `System` got no policy.** The attribute walker read any namespace starting with "System" as the framework's, so an application namespace such as `SystemsCorp.Payroll` got no policy beneath its types, and a `[DwDenied]` field there was returned, filterable and sortable. Only `System` and the namespaces beneath it are the framework's now. - **Fixed (security): a navigation narrowed around its own denied key got the key back.** Under the convenience tier, `Selects` naming a navigation whose key (`Id`) is denied was narrowed to the allowed fields beneath it, and the core's typed projection added the key back. Such a narrowing is refused with `FieldDeniedForSelect` in both tiers, as naming a sibling of the key already was. A navigation named through another, such as `Main.Lead`, now gates the key of `Main`, which the projection adds; it did not. @@ -389,7 +389,7 @@ The complete reference — every enum, class, extension method, validation rule, - **Changed: `[DwEntity(DefaultOrder)]` reaches a projection that builds the row.** A guarded query over a projected source takes the default when the outermost `Select` builds the type in an object initializer and assigns every field the default names a column, at every level of a nested path — a mapped member read directly, through reference navigations or through `EF.Property`: `Select(t => new TicketRow { Id = t.Id, CreatedAt = t.CreatedAt })` for `"CreatedAt desc, Id"`. A computed value or any other projection still leaves the query in its own order. A `Select`, or a `Filter` with `Selects`, composed on the guarded handle keeps the rest of the chain unordered, and a composed `Filter` that sent orders gets no default later in the chain, as a composed `Order` already did not. - **Changed: the async dynamic `Filter` and the async `Summary` read through EF Core.** `ToListAsyncDynamic` and `ToListAsync(Summary)` read with EF Core's `ToListAsync` instead of Dynamic LINQ's `ToDynamicListAsync`, which had no token to pass on, and the summary counts with `CountAsync` where it counted synchronously. So on an EF Core query a canceled token now reaches the database. A provider that is not EF Core's keeps Dynamic LINQ's read, on the calling thread. - **New: a `CancellationToken` on every async terminal**, guarded and unguarded: `ToListAsync` and `ToListAsyncDynamic` with a `Filter`, `ToListAsync` with a `Summary`, and `ToListAsync` with a `Segment`. The token reaches the count and the read. The overloads sit beside the 3.1 signatures, which are unchanged, so code compiled against 3.1 still binds. `ToListAsync(filter, default)`, `ToListAsyncDynamic(filter, default)` and `ToListAsync(summary, default)` no longer compile, because `default` fits both `getQueryString` and the token: write `false`, a token, or a named argument. A reflection lookup of `ToListAsyncDynamic` by name alone now finds three methods where it found one, and one of `ToListAsync` finds more than it did. -- **Known limits.** Once a projection is needed, an entity's navigations are left out, included ones too; under the convenience tier name one in `Selects` to get it narrowed, and under the strict tier name its allowed fields. A forced scope declared on a list's element type filters the rows that hold the list, never its elements, so `Selects` naming the list returns every element and a synthesized projection leaves the list out; scope the elements where the row is built. A member typed `object`, a framework interface or a collection that is not generic (`IEnumerable`, `ArrayList`) is opaque to the policy, and a framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: hold such values in a list of the policed type. A member EF Core does not map is read as its type, since its getter can hand out what EF Core loaded. Rows in memory can be any loaded subtype, so they are projected whenever one declares a denied field. `/simulate` has no source, so it reads the type as one it cannot see into: every denial beneath a member counts, and the projection it shows keeps only members holding a value. +- **Known limits.** Once a projection is needed, an entity's navigations are left out, included ones too; under the convenience tier name one in `Selects` to get it narrowed, and under the strict tier name its allowed fields. A forced scope declared on a list's element type filters the rows that hold the list, never its elements, so `Selects` naming the list returns every element and a synthesized projection leaves the list out; scope the elements where the row is built. A member typed `object`, a framework interface or a collection that is not generic (`IEnumerable`, `ArrayList`) is opaque to the policy, and a framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: hold such values in a list of the policed type. A member EF Core does not map is read as its type, since its getter can hand out what EF Core loaded. Rows in memory can be any loaded subtype, so they are projected whenever one declares a denied field. A denial on an override or a `new` member counts for every loaded subtype, a class EF Core does not map included. `/simulate` has no source, so it reads the type as one it cannot see into: every denial beneath a member counts, and the projection it shows keeps only members holding a value. ## Version 3.1.0 highlights From 689d70e20f773b545c2c85d6f3c6d7655f4023d8 Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 10:36:49 +0300 Subject: [PATCH 19/22] fix(policies): variance, wrapped providers, shared names and collection classes; read specifications as EF Core does Security: - A variant generic interface: a member typed IFeed holds an IFeed implementation, whose denial the subtype index and the walker now read. The open generic case was a regression in ca3f154. - A provider wrapping EF Core's (LinqKit's AsExpandable, DelegateDecompiler's Decompile) got no AsNoTracking, since EF Core's extension hands such a query back unchanged: the context filled in navigations it already held, and a masked value became a pending change. The call goes into the query itself. - A converted member inside an EF Core 8 complex property is read as converted, so a projection leaves out one that can hold any object. - Two members sharing a name, one hidden with new under another type or spelled in another case: the core reads one, and a row carries both. Either one's denial now leaves the name out. - An application's own collection class, generic or not, has its own members read. Precision: - A call that reads nothing of the lambda's and returns a query or an expression (a specification, a repository's query, FromSql, Set through a context interface) is evaluated as EF Core evaluates it; a context's query function is a query root; an anonymous object carrying range variables or a composite key builds nothing. Ordinary reshaped queries read as in 3.1.0 again, and no longer throw for abstract, constructor-bound or DDD roots. - BitArray and the framework's string collections hold values. - A private member a subtype hides with new no longer denies the base path. The review tests' probe logs no longer write to a local path. Co-Authored-By: Claude Opus 5 --- .../Policies/ReviewCollectionClassTests.cs | 176 +++++ .../Policies/ReviewConverterPrecisionTests.cs | 383 +++++++++++ .../Policies/ReviewConverterTests.cs | 189 ++++++ .../ReviewDeclarationPrecisionTests.cs | 308 +++++++++ .../Policies/ReviewDefaultOrderTests.cs | 1 - .../Policies/ReviewOverBlockTests.cs | 36 - .../Policies/ReviewOwnedPrecisionTests.cs | 168 +++++ .../Policies/ReviewReshapeImpactTests.cs | 323 +++++++++ .../Policies/ReviewReshapePrecisionTests.cs | 630 ++++++++++++++++++ .../Policies/ReviewSharedNameTests.cs | 183 +++++ .../Policies/ReviewTrackingKit.cs | 201 ++++++ .../Policies/ReviewTrackingTests.cs | 325 +++++++++ .../Policies/ReviewVarianceTests.cs | 356 ++++++++++ .../ReviewComplexPropertyTests.cs | 199 ++++++ .../Resolution/AttributePolicyProvider.cs | 18 +- .../Policies/Source/FilterSanitizer.cs | 103 ++- .../Policies/Source/KnownSubtypes.cs | 10 +- .../Policies/Source/PolicyQueryable.cs | 26 +- DynamicWhere.ex/Policies/Source/RowShape.cs | 58 +- DynamicWhere.ex/Source/QueryRoot.cs | 107 ++- 20 files changed, 3742 insertions(+), 58 deletions(-) create mode 100644 DynamicWhere.Tests/Policies/ReviewCollectionClassTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewConverterPrecisionTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewConverterTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewDeclarationPrecisionTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewOwnedPrecisionTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewReshapeImpactTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewReshapePrecisionTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewSharedNameTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewTrackingKit.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewTrackingTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewVarianceTests.cs create mode 100644 DynamicWhere.Tests/ReviewComplexPropertyTests.cs diff --git a/DynamicWhere.Tests/Policies/ReviewCollectionClassTests.cs b/DynamicWhere.Tests/Policies/ReviewCollectionClassTests.cs new file mode 100644 index 0000000..b907114 --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewCollectionClassTests.cs @@ -0,0 +1,176 @@ +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Source; +using Xunit.Abstractions; + +namespace DynamicWhere.Tests.Policies +{ + // ============================================================================ an application's own generic collections: models + + /// An application collection of values that declares a member of its own. + public class ZxTagSet : List + { + [DwDenied] + public string? OwnerSecret { get; set; } + } + + public class ZxPlainCard + { + public string Label { get; set; } = string.Empty; + } + + /// An application collection of objects that declares a member of its own. + public class ZxCardList : List + { + [DwDenied] + public string? MasterKey { get; set; } + } + + /// An application dictionary that declares a member of its own. + public class ZxPropertyBag : Dictionary + { + [DwDenied] + public string? BagSecret { get; set; } + } + + public class ZxBagRow + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public ZxTagSet Tags { get; set; } = new(); + } + + public class ZxListRow + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public ZxCardList Cards { get; set; } = new(); + } + + public class ZxPropsRow + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public ZxPropertyBag Props { get; set; } = new(); + } + + /// The same, beside a top-level denial, so a projection is built for another reason. + public class ZxBagRowDenied + { + public int Id { get; set; } + + [DwDenied] + public string? Ssn { get; set; } + + public ZxTagSet Tags { get; set; } = new(); + + public ZxCardList Cards { get; set; } = new(); + } + + // ============================================================================ an application's own generic collections: tests + + /// + /// An application's own collection class, deriving from List<string>, List<T> or a Dictionary, declares + /// members beside the elements it holds, and they are read as any type's are. + /// + public sealed class ReviewCollectionClassTests + { + private readonly ITestOutputHelper _out; + + public ReviewCollectionClassTests(ITestOutputHelper output) => _out = output; + + private object? Read(IQueryable source, DwTier tier, bool dynamic = false) where T : class + { + PolicyQueryable guarded = ZxKit.Guard(source, tier); + object? data = ZxKit.SafeRead(() => dynamic ? guarded.ToListDynamic(new Filter()).Data : guarded.ToList(new Filter()).Data); + + _out.WriteLine($"{typeof(T).Name} {tier} dynamic={dynamic}: sent={ZxKit.Json(data)} trace=[{ZxKit.Trace(guarded)}]"); + + return data; + } + + [Theory] + [InlineData(DwTier.Strict, false)] + [InlineData(DwTier.Convenience, false)] + [InlineData(DwTier.Strict, true)] + public void Zx_K1_a_collection_of_values_that_declares_a_denied_member(DwTier tier, bool dynamic) + { + ZxBagRow[] rows = { new() { Id = 1, Name = "r1", Tags = new ZxTagSet { "a", "b" } } }; + rows[0].Tags.OwnerSecret = "tagset-owner-secret"; + + object? data = Read(rows.AsQueryable(), tier, dynamic); + + Assert.False(ZxKit.Holds(data, "tagset-owner-secret")); + } + + [Theory] + [InlineData(DwTier.Strict, false)] + [InlineData(DwTier.Convenience, false)] + [InlineData(DwTier.Strict, true)] + public void Zx_K2_a_collection_of_objects_that_declares_a_denied_member(DwTier tier, bool dynamic) + { + ZxListRow[] rows = { new() { Id = 1, Name = "r1", Cards = new ZxCardList { new() { Label = "visa" } } } }; + rows[0].Cards.MasterKey = "cardlist-master-secret"; + + object? data = Read(rows.AsQueryable(), tier, dynamic); + + Assert.False(ZxKit.Holds(data, "cardlist-master-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_K3_a_dictionary_that_declares_a_denied_member(DwTier tier) + { + ZxPropsRow[] rows = { new() { Id = 1, Name = "r1", Props = new ZxPropertyBag { ["k"] = "v" } } }; + rows[0].Props.BagSecret = "bag-own-secret"; + + object? data = Read(rows.AsQueryable(), tier); + + Assert.False(ZxKit.Holds(data, "bag-own-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_K4_beside_a_top_level_denial_the_projection_keeps_the_collection(DwTier tier) + { + ZxBagRowDenied[] rows = + { + new() { Id = 1, Ssn = "ssn-secret", Tags = new ZxTagSet { "a" }, Cards = new ZxCardList { new() { Label = "visa" } } } + }; + rows[0].Tags.OwnerSecret = "tagset-owner-secret-2"; + rows[0].Cards.MasterKey = "cardlist-master-secret-2"; + + object? data = Read(rows.AsQueryable(), tier); + + Assert.False(ZxKit.Holds(data, "ssn-secret")); + Assert.False(ZxKit.Holds(data, "tagset-owner-secret-2")); + Assert.False(ZxKit.Holds(data, "cardlist-master-secret-2")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_K5_naming_the_collection_of_values_in_selects(DwTier tier) + { + ZxBagRow[] rows = { new() { Id = 1, Name = "r1", Tags = new ZxTagSet { "a" } } }; + rows[0].Tags.OwnerSecret = "tagset-owner-secret-3"; + + PolicyQueryable guarded = ZxKit.Guard(rows.AsQueryable(), tier); + object? data = ZxKit.SafeRead(() => guarded.ToList(ZxKit.Selecting("Id", "Tags")).Data); + + _out.WriteLine($"{tier}: sent={ZxKit.Json(data)} trace=[{ZxKit.Trace(guarded)}]"); + + Assert.False(ZxKit.Holds(data, "tagset-owner-secret-3")); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewConverterPrecisionTests.cs b/DynamicWhere.Tests/Policies/ReviewConverterPrecisionTests.cs new file mode 100644 index 0000000..22e036c --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewConverterPrecisionTests.cs @@ -0,0 +1,383 @@ +using System.Collections; +using System.Globalization; +using System.Net; +using System.Numerics; +using System.Text.Json; +using System.Text.Json.Nodes; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.ChangeTracking; +using Xunit.Abstractions; + +// A converted column beside a top-level [DwDenied] (Secret), which builds a projection. An everyday converted column +// comes back with its value; one that can hold an object of any type is left out, since the converter is the +// application's code and the policy cannot see what it hands back. + +namespace DynamicWhere.Tests.Policies +{ + public enum ZycStatus + { + Draft, + Active + } + + public readonly record struct ZycAccountId(int Value); + + public sealed record ZycAccountKey(Guid Value); + + public sealed record ZycMoney(decimal Amount, string Currency); + + public class ZycPrefs + { + public string Theme { get; set; } = string.Empty; + + public int FontSize { get; set; } + + public List Recent { get; set; } = new(); + } + + /// A JSON envelope holding an arbitrary payload: genuinely able to hold any object. + public class ZycEnvelope + { + public string Kind { get; set; } = string.Empty; + + public object? Data { get; set; } + } + + public class ZycPoint + { + public double X { get; set; } + + public double Y { get; set; } + } + + public class ZycAccount + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + /// A top-level denial: every guarded query over the account builds a projection. + [DwDenied] + public string? Secret { get; set; } + + public ZycStatus Status { get; set; } + + public DateTime CreatedUtc { get; set; } + + public List Tags { get; set; } = new(); + + public string[] Aliases { get; set; } = Array.Empty(); + + public HashSet Codes { get; set; } = new(); + + public ZycAccountId Ref { get; set; } + + public ZycAccountKey Key { get; set; } = new(Guid.Empty); + + public ZycMoney Price { get; set; } = new(0, "IQD"); + + public ZycPrefs Prefs { get; set; } = new(); + + public List History { get; set; } = new(); + + public ZycPoint Location { get; set; } = new(); + + public Dictionary Labels { get; set; } = new(); + + public Dictionary> Groups { get; set; } = new(); + + public Dictionary Attributes { get; set; } = new(); + + public Dictionary Bag { get; set; } = new(); + + public ZycEnvelope Envelope { get; set; } = new(); + + public BitArray Flags { get; set; } = new(0); + + public Uri? Site { get; set; } + + public IPAddress? Ip { get; set; } + + public CultureInfo? Culture { get; set; } + + public Version? Schema { get; set; } + + public BigInteger Big { get; set; } + + public JsonObject? Extra { get; set; } + + public JsonElement Element { get; set; } + + public object? Setting { get; set; } + } + + public sealed class ZycContext : DbContext + { + private readonly SqliteConnection _connection; + + public ZycContext(SqliteConnection connection) => _connection = connection; + + public DbSet Accounts => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + private static string J(T value) => JsonSerializer.Serialize(value, (JsonSerializerOptions?)null); + + private static T U(string text) => JsonSerializer.Deserialize(text, (JsonSerializerOptions?)null)!; + + private static ValueComparer ByJson() => new( + (left, right) => J(left) == J(right), + value => J(value).GetHashCode(), + value => U(J(value))); + + protected override void OnModelCreating(ModelBuilder model) + { + var account = model.Entity(); + + account.Property(a => a.Status).HasConversion(); + account.Property(a => a.CreatedUtc).HasConversion(v => v.ToUniversalTime(), v => DateTime.SpecifyKind(v, DateTimeKind.Utc)); + account.Property(a => a.Tags).HasConversion(v => string.Join(',', v), v => v.Split(',', StringSplitOptions.RemoveEmptyEntries).ToList(), ByJson>()); + account.Property(a => a.Aliases).HasConversion(v => string.Join(',', v), v => v.Split(',', StringSplitOptions.RemoveEmptyEntries), ByJson()); + account.Property(a => a.Codes).HasConversion(v => J(v), v => U>(v), ByJson>()); + account.Property(a => a.Ref).HasConversion(v => v.Value, v => new ZycAccountId(v)); + account.Property(a => a.Key).HasConversion(v => v.Value, v => new ZycAccountKey(v)); + account.Property(a => a.Price).HasConversion(v => J(v), v => U(v)); + account.Property(a => a.Prefs).HasConversion(v => J(v), v => U(v), ByJson()); + account.Property(a => a.History).HasConversion(v => J(v), v => U>(v), ByJson>()); + account.Property(a => a.Location).HasConversion(v => J(v), v => U(v), ByJson()); + account.Property(a => a.Labels).HasConversion(v => J(v), v => U>(v), ByJson>()); + account.Property(a => a.Groups).HasConversion(v => J(v), v => U>>(v), ByJson>>()); + account.Property(a => a.Attributes).HasConversion(v => J(v), v => U>(v), ByJson>()); + account.Property(a => a.Bag).HasConversion(v => J(v), v => U>(v), ByJson>()); + account.Property(a => a.Envelope).HasConversion(v => J(v), v => U(v), ByJson()); + account.Property(a => a.Flags).HasConversion( + v => string.Concat(v.Cast().Select(bit => bit ? '1' : '0')), + v => new BitArray(v.Select(c => c == '1').ToArray())); + account.Property(a => a.Site).HasConversion(v => v == null ? null : v.ToString(), v => v == null ? null : new Uri(v)); + account.Property(a => a.Ip).HasConversion(v => v == null ? null : v.ToString(), v => v == null ? null : IPAddress.Parse(v)); + account.Property(a => a.Culture).HasConversion(v => v == null ? null : v.Name, v => v == null ? null : CultureInfo.GetCultureInfo(v)); + account.Property(a => a.Schema).HasConversion(v => v == null ? null : v.ToString(), v => v == null ? null : Version.Parse(v)); + account.Property(a => a.Big).HasConversion(v => v.ToString(), v => BigInteger.Parse(v)); + account.Property(a => a.Extra).HasConversion(v => v == null ? null : v.ToJsonString((JsonSerializerOptions?)null), v => v == null ? null : (JsonObject)JsonNode.Parse(v, null, default)!); + account.Property(a => a.Element).HasConversion(v => v.GetRawText(), v => JsonDocument.Parse(v, default).RootElement.Clone()); + account.Property(a => a.Setting).HasConversion(v => J(v), v => U(v)); + } + } + + public sealed class ReviewConverterPrecisionTests : IDisposable + { + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZycContext _db; + + public ReviewConverterPrecisionTests(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + _db = new ZycContext(_connection); + _db.Database.EnsureCreated(); + + _db.Accounts.Add(new ZycAccount + { + Name = "A1", + Secret = "zyc-secret", + Status = ZycStatus.Active, + CreatedUtc = new DateTime(2026, 1, 2, 3, 4, 5, DateTimeKind.Utc), + Tags = { "a", "b" }, + Aliases = new[] { "x" }, + Codes = { 7 }, + Ref = new ZycAccountId(42), + Key = new ZycAccountKey(Guid.Parse("11111111-2222-3333-4444-555555555555")), + Price = new ZycMoney(9.5m, "USD"), + Prefs = new ZycPrefs { Theme = "dark", FontSize = 12, Recent = { "r" } }, + History = { new ZycPrefs { Theme = "light" } }, + Location = new ZycPoint { X = 1, Y = 2 }, + Labels = { ["team"] = "blue" }, + Groups = { ["g"] = new List { "m" } }, + Attributes = { ["n"] = JsonDocument.Parse("5").RootElement.Clone() }, + Bag = { ["colour"] = "red" }, + Envelope = new ZycEnvelope { Kind = "note", Data = "hello" }, + Flags = new BitArray(new[] { true, false, true }), + Site = new Uri("https://example.org/a"), + Ip = IPAddress.Parse("10.0.0.1"), + Culture = CultureInfo.GetCultureInfo("ar-IQ"), + Schema = new Version(1, 2), + Big = BigInteger.Parse("123456789012345678901234567890"), + Extra = new JsonObject { ["p"] = 1 }, + Element = JsonDocument.Parse("{\"q\":2}").RootElement.Clone(), + Setting = "on" + }); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + /// What an unguarded read returns for each column, compared with the guarded read. + private static readonly Dictionary> Holds = new() + { + ["Status"] = a => a.Status == ZycStatus.Active, + ["CreatedUtc"] = a => a.CreatedUtc.Year == 2026, + ["Tags"] = a => a.Tags.SequenceEqual(new[] { "a", "b" }), + ["Aliases"] = a => a.Aliases.SequenceEqual(new[] { "x" }), + ["Codes"] = a => a.Codes.Contains(7), + ["Ref"] = a => a.Ref.Value == 42, + ["Key"] = a => a.Key.Value != Guid.Empty, + ["Price"] = a => a.Price.Amount == 9.5m, + ["Prefs"] = a => a.Prefs.Theme == "dark", + ["History"] = a => a.History.Count == 1, + ["Location"] = a => a.Location.Y == 2, + ["Labels"] = a => a.Labels.ContainsKey("team"), + ["Groups"] = a => a.Groups.ContainsKey("g"), + ["Attributes"] = a => a.Attributes.ContainsKey("n"), + ["Bag"] = a => a.Bag.ContainsKey("colour"), + ["Envelope"] = a => a.Envelope.Kind == "note", + ["Flags"] = a => a.Flags.Length == 3, + ["Site"] = a => a.Site is not null, + ["Ip"] = a => a.Ip is not null, + ["Culture"] = a => a.Culture is not null, + ["Schema"] = a => a.Schema is not null, + ["Big"] = a => !a.Big.IsZero, + ["Extra"] = a => a.Extra is not null, + ["Element"] = a => a.Element.ValueKind == JsonValueKind.Object, + ["Setting"] = a => a.Setting is not null + }; + + /// + /// Docs check (llms.txt 3.2.0 history): "The trace records a member left out only when a projection is built." + /// A dry run builds none and returns the rows whole. + /// + [Fact] + public void Zy_C_a_dry_run_returns_the_row_whole_and_records_what_it_would_leave_out() + { + PolicyQueryable guarded = _db.Accounts.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = DwTier.Strict, DryRun = true, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + ZycAccount row = guarded.ToList(new Filter()).Data.Single(); + string leftOut = string.Join(" | ", guarded.LastTrace?.Decisions + .Where(d => d.Reason?.StartsWith("left out", StringComparison.Ordinal) == true) + .Select(d => $"{d.FieldPath}: {d.Reason}") ?? Array.Empty()); + + ZyLog.Write(_out, $"C dry run: secretReturned={row.Secret is not null} bagReturned={row.Bag.Count > 0} leftOut=[{leftOut}]"); + + // A dry run drops nothing, and records what a real run would leave out. + Assert.True(row.Bag.Count > 0); + Assert.Contains("Bag: left out whole: it can hold what the policy cannot name", leftOut); + } + + [Theory] + [InlineData("Status")] + [InlineData("CreatedUtc")] + [InlineData("Tags")] + [InlineData("Aliases")] + [InlineData("Codes")] + [InlineData("Ref")] + [InlineData("Key")] + [InlineData("Price")] + [InlineData("Prefs")] + [InlineData("History")] + [InlineData("Location")] + [InlineData("Labels")] + [InlineData("Groups")] + [InlineData("Attributes")] + [InlineData("Flags")] + [InlineData("Site")] + [InlineData("Ip")] + [InlineData("Culture")] + [InlineData("Schema")] + [InlineData("Big")] + [InlineData("Extra")] + [InlineData("Element")] + public void Zy_C_a_converted_column_beside_a_top_level_denial_is_kept(string column) + { + ZycAccount unguarded = _db.Accounts.AsNoTracking().Single(); + + Assert.True(Holds[column](unguarded), "the unguarded read itself lost " + column); + + PolicyQueryable guarded = _db.Accounts.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = DwTier.Strict, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + ZycAccount row; + + try + { + row = guarded.ToList(new Filter()).Data.Single(); + } + catch (Exception e) + { + ZyLog.Write(_out, $"C {column}: THREW {e.GetType().Name} {e.Message.Split('\n')[0]}"); + + throw; + } + + string about = string.Join(" | ", guarded.LastTrace?.Decisions + .Where(d => d.Action != PolicyAction.Allowed && d.FieldPath.StartsWith(column, StringComparison.Ordinal)) + .Select(d => $"{d.FieldPath} {d.Action}: {d.Reason}") ?? Array.Empty()); + + bool kept = Holds[column](row); + + ZyLog.Write(_out, $"C {column}: {(kept ? "KEPT" : "LEFT OUT")} secretKept={row.Secret is not null} [{about}]"); + + Assert.Null(row.Secret); + Assert.True(kept, $"{column} left out: {about}"); + } + + [Theory] + [InlineData("Bag")] + [InlineData("Envelope")] + [InlineData("Setting")] + public void Zy_C_a_converted_column_that_can_hold_any_object_is_left_out_beside_a_top_level_denial(string column) + { + ZycAccount unguarded = _db.Accounts.AsNoTracking().Single(); + + Assert.True(Holds[column](unguarded), "the unguarded read itself lost " + column); + + PolicyQueryable guarded = _db.Accounts.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = DwTier.Strict, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + ZycAccount row; + + try + { + row = guarded.ToList(new Filter()).Data.Single(); + } + catch (Exception e) + { + ZyLog.Write(_out, $"C {column}: THREW {e.GetType().Name} {e.Message.Split('\n')[0]}"); + + throw; + } + + string about = string.Join(" | ", guarded.LastTrace?.Decisions + .Where(d => d.Action != PolicyAction.Allowed && d.FieldPath.StartsWith(column, StringComparison.Ordinal)) + .Select(d => $"{d.FieldPath} {d.Action}: {d.Reason}") ?? Array.Empty()); + + bool kept = Holds[column](row); + + ZyLog.Write(_out, $"C {column}: {(kept ? "KEPT" : "LEFT OUT")} secretKept={row.Secret is not null} [{about}]"); + + Assert.Null(row.Secret); + Assert.False(kept, $"{column} kept: {about}"); + Assert.Contains($"{column} Dropped: left out whole: it can hold what the policy cannot name", about); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewConverterTests.cs b/DynamicWhere.Tests/Policies/ReviewConverterTests.cs new file mode 100644 index 0000000..dac010b --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewConverterTests.cs @@ -0,0 +1,189 @@ +using System.Text.Json; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Metadata; +using Microsoft.EntityFrameworkCore.Storage.ValueConversion; +using Xunit.Abstractions; + +namespace DynamicWhere.Tests.Policies +{ + // ============================================================================ converters the model does not name: models + + public class ZxIssued + { + public string Label { get; set; } = string.Empty; + + [DwDenied] + public string? Pan { get; set; } + } + + /// The application's own converter: writes any object as JSON and reads it back as ZxIssued. + public sealed class ZxPayloadConverter : ValueConverter + { + public ZxPayloadConverter() + : base(v => Write(v), s => Read(s)) + { + } + + private static string Write(object? value) => + value is null ? "null" : JsonSerializer.Serialize(value, value.GetType()); + + private static object? Read(string text) => JsonSerializer.Deserialize(text); + } + + public class ZxConvEvent + { + public int Id { get; set; } + + public string Kind { get; set; } = string.Empty; + + [DwDenied] + public string? Actor { get; set; } + + public object? Payload { get; set; } + } + + /// How the converter is configured. + public enum ZxConverterStyle + { + /// Control: HasConversion(new converter()). + Instance, + + /// HasConversion<TConverter>(). + GenericType, + + /// HasConversion(typeof(TConverter)). + TypeObject, + + /// ConfigureConventions: Properties<object>().HaveConversion<TConverter>(). + Convention + } + + public abstract class ZxConvContextBase : DbContext + { + private readonly SqliteConnection _connection; + + protected ZxConvContextBase(SqliteConnection connection) => _connection = connection; + + public DbSet Events => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + } + + public sealed class ZxConvInstanceContext : ZxConvContextBase + { + public ZxConvInstanceContext(SqliteConnection connection) : base(connection) { } + + protected override void OnModelCreating(ModelBuilder model) => + model.Entity().Property(e => e.Payload).HasConversion(new ZxPayloadConverter()); + } + + public sealed class ZxConvGenericTypeContext : ZxConvContextBase + { + public ZxConvGenericTypeContext(SqliteConnection connection) : base(connection) { } + + protected override void OnModelCreating(ModelBuilder model) => + model.Entity().Property(e => e.Payload).HasConversion(); + } + + public sealed class ZxConvTypeObjectContext : ZxConvContextBase + { + public ZxConvTypeObjectContext(SqliteConnection connection) : base(connection) { } + + protected override void OnModelCreating(ModelBuilder model) => + model.Entity().Property(e => e.Payload).HasConversion(typeof(ZxPayloadConverter)); + } + + public sealed class ZxConvConventionContext : ZxConvContextBase + { + public ZxConvConventionContext(SqliteConnection connection) : base(connection) { } + + protected override void ConfigureConventions(ModelConfigurationBuilder configuration) => + configuration.Properties().HaveConversion(); + + protected override void OnModelCreating(ModelBuilder model) => + model.Entity().Property(e => e.Payload); + } + + // ============================================================================ converters configured every way: tests + + /// + /// A converted column that can hold any object is left out once a projection is built for another field, + /// however the converter is configured: an instance, a type, or a lambda. + /// + public sealed class ReviewConverterTests + { + private readonly ITestOutputHelper _out; + + public ReviewConverterTests(ITestOutputHelper output) => _out = output; + + private static ZxConvContextBase Create(ZxConverterStyle style, SqliteConnection connection) => style switch + { + ZxConverterStyle.Instance => new ZxConvInstanceContext(connection), + ZxConverterStyle.GenericType => new ZxConvGenericTypeContext(connection), + ZxConverterStyle.TypeObject => new ZxConvTypeObjectContext(connection), + _ => new ZxConvConventionContext(connection) + }; + + [Theory] + [InlineData(ZxConverterStyle.Instance, DwTier.Strict)] + [InlineData(ZxConverterStyle.GenericType, DwTier.Strict)] + [InlineData(ZxConverterStyle.GenericType, DwTier.Convenience)] + [InlineData(ZxConverterStyle.TypeObject, DwTier.Strict)] + [InlineData(ZxConverterStyle.Convention, DwTier.Strict)] + public void Zx_C1_a_converted_object_column_beside_a_top_level_denial(ZxConverterStyle style, DwTier tier) + { + using SqliteConnection connection = new("DataSource=:memory:"); + connection.Open(); + + ZxConvContextBase db; + + try + { + db = Create(style, connection); + db.Database.EnsureCreated(); + } + catch (Exception e) + { + _out.WriteLine($"{style}: model not built: {e.GetType().Name}: {e.Message.Split('\n')[0]}"); + + return; + } + + using (db) + { + db.Events.Add(new ZxConvEvent + { + Kind = "CardIssued", Actor = "conv-actor-secret", Payload = new ZxIssued { Label = "visa", Pan = $"conv-pan-{style}" } + }); + db.SaveChanges(); + db.ChangeTracker.Clear(); + + IProperty payload = db.Model.FindEntityType(typeof(ZxConvEvent))!.FindProperty(nameof(ZxConvEvent.Payload))!; + + _out.WriteLine($"{style}: GetValueConverter()={payload.GetValueConverter()?.GetType().Name ?? "null"} " + + $"mapping.Converter={payload.GetTypeMapping().Converter?.GetType().Name ?? "null"}"); + + Assert.True(ZxKit.Holds(db.Events.AsNoTracking().ToList(), $"conv-pan-{style}")); + + PolicyQueryable guarded = ZxKit.Guard(db.Events, tier); + object? data = ZxKit.SafeRead(() => guarded.ToList(new Filter()).Data); + + _out.WriteLine($"{style} {tier}: sent={ZxKit.Json(data)} trace=[{ZxKit.Trace(guarded)}]"); + + Assert.False(ZxKit.Holds(data, "conv-actor-secret")); + Assert.False(ZxKit.Holds(data, $"conv-pan-{style}")); + + object? dynamicData = ZxKit.SafeRead(() => ZxKit.Guard(db.Events, tier).ToListDynamic(new Filter()).Data); + + _out.WriteLine($"{style} {tier} dynamic: sent={ZxKit.Json(dynamicData)}"); + + Assert.False(ZxKit.Holds(dynamicData, $"conv-pan-{style}")); + } + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewDeclarationPrecisionTests.cs b/DynamicWhere.Tests/Policies/ReviewDeclarationPrecisionTests.cs new file mode 100644 index 0000000..9fda2b7 --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewDeclarationPrecisionTests.cs @@ -0,0 +1,308 @@ +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Classes.Core; +using DynamicWhere.ex.Enums; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Xunit.Abstractions; + +// A member that no declaration a row can run through the queried path denies stays filterable and is returned: +// a sibling implementer's denial, a private or static member of the same name, an explicit implementation. + +namespace DynamicWhere.Tests.Policies +{ + // ---- D1: one implementer of a widely shared interface denies its implementation ------------------------- + + public interface IZydAuditable + { + string? CreatedBy { get; } + } + + public class ZydInvoice : IZydAuditable + { + public int Id { get; set; } + + public string? CreatedBy { get; set; } + } + + public class ZydPayslip : IZydAuditable + { + public int Id { get; set; } + + [DwDenied] + public string? CreatedBy { get; set; } + } + + // ---- D2: a subclass hides the member with a private one ----------------------------------------------- + + public class ZydNoteDto + { + public int Id { get; set; } + + public string Text { get; set; } = string.Empty; + } + + public class ZydRedactedNoteDto : ZydNoteDto + { + [DwDenied] + private new string Text { get; set; } = string.Empty; + + public void Seal(string text) => Text = text; + } + + // ---- D3: a subclass declares a static member of the same name ------------------------------------------- + + public class ZydRateDto + { + public int Id { get; set; } + + public decimal Rate { get; set; } + } + + public class ZydRateDefaultsDto : ZydRateDto + { + [DwDenied] + public static new decimal Rate { get; set; } + } + + // ---- D4: a subclass implements an interface explicitly with a denied member of the same name ----------- + + public interface IZydCoded + { + string Code { get; } + } + + public class ZydBadgeDto + { + public int Id { get; set; } + + public string Code { get; set; } = string.Empty; + } + + public class ZydSecretBadgeDto : ZydBadgeDto, IZydCoded + { + [DwDenied] + string IZydCoded.Code => "explicit-secret"; + } + + // ---- D5: a class with its own public member and an explicit implementation of a denying interface ------ + + public interface IZydSerialed + { + [DwDenied] + string Serial { get; } + } + + public class ZydDevice : IZydSerialed + { + public int Id { get; set; } + + public string Serial { get; set; } = string.Empty; + + string IZydSerialed.Serial => "device-internal-serial"; + } + + // ---- D6: a sibling variant hides the member with new to withhold it in its own shape (documented limit) -- + + public class ZydEmployeeDto + { + public int Id { get; set; } + + public decimal Salary { get; set; } + } + + public class ZydPublicEmployeeDto : ZydEmployeeDto + { + [DwDenied] + public new decimal Salary { get; set; } + } + + // ---- D7: an entity, and a generic interface one unrelated generic class implements over another argument -- + + public interface IZydKeyed + { + TKey Key { get; } + } + + public class ZydTicket : IZydKeyed + { + public int Id { get; set; } + + public int Key { get; set; } + + public string Title { get; set; } = string.Empty; + } + + public class ZydVaultEntry : IZydKeyed + { + [DwDenied] + public string Key { get; set; } = string.Empty; + + public T? Value { get; set; } + } + + public sealed class ZydContext : DbContext + { + private readonly SqliteConnection _connection; + + public ZydContext(SqliteConnection connection) => _connection = connection; + + public DbSet Invoices => Set(); + + public DbSet Tickets => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + } + + public sealed class ReviewDeclarationPrecisionTests : IDisposable + { + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZydContext _db; + + public ReviewDeclarationPrecisionTests(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + _db = new ZydContext(_connection); + _db.Database.EnsureCreated(); + _db.Invoices.Add(new ZydInvoice { CreatedBy = "alice" }); + _db.Tickets.Add(new ZydTicket { Key = 7, Title = "T" }); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static PolicyQueryable Guard(IQueryable source) where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = DwTier.Strict, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + private static Filter Where(string field, string value) => new() + { + ConditionGroup = new ConditionGroup + { + Conditions = + { + new Condition + { + Field = field, + DataType = decimal.TryParse(value, System.Globalization.NumberStyles.Number, System.Globalization.CultureInfo.InvariantCulture, out _) ? DataType.Number : DataType.Text, + Operator = Operator.Equal, + Values = { value } + } + } + } + }; + + /// Filters on the member, then reads the rows whole; logs both and returns them. + private (string Filtered, List Rows, PolicyQueryable Guarded) Probe(string label, IQueryable source, string field, string value) + where T : class + { + string filtered; + + try + { + filtered = "ran rows=" + Guard(source).ToList(Where(field, value)).Data.Count; + } + catch (PolicyException refusal) + { + filtered = refusal.ErrorCode.ToString(); + } + + PolicyQueryable guarded = Guard(source); + List rows = guarded.ToList(new Filter()).Data; + + ZyLog.Write(_out, $"D {label}: Where({field})={filtered}; whole: dropped=[{ZyLog.Decisions(guarded)}]"); + + return (filtered, rows, guarded); + } + + [Fact] + public void Zy_D1_an_entity_implementing_a_shared_interface_is_not_denied_by_a_sibling_implementer() + { + (string filtered, List rows, PolicyQueryable guarded) = + Probe("D1 entity beside a denying sibling implementer", _db.Invoices, "CreatedBy", "alice"); + + Assert.Equal("ran rows=1", filtered); + Assert.Equal("alice", rows.Single().CreatedBy); + Assert.False(ZyLog.AnyDropped(guarded), ZyLog.Decisions(guarded)); + } + + [Fact] + public void Zy_D2_a_private_member_a_subclass_hides_with_is_never_run_through_the_base_path() + { + ZydNoteDto[] notes = { new() { Id = 1, Text = "hello" } }; + + (string filtered, List rows, PolicyQueryable guarded) = + Probe("D2 private new in a subclass", notes.AsQueryable(), "Text", "hello"); + + Assert.Equal("ran rows=1", filtered); + Assert.Equal("hello", rows.Single().Text); + Assert.False(ZyLog.AnyDropped(guarded), ZyLog.Decisions(guarded)); + } + + [Fact] + public void Zy_D3_a_static_member_of_the_same_name_is_not_a_declaration_a_row_runs() + { + ZydRateDto[] rates = { new() { Id = 1, Rate = 2.5m } }; + + (string filtered, List rows, PolicyQueryable guarded) = + Probe("D3 static new in a subclass", rates.AsQueryable(), "Rate", "2.5"); + + Assert.StartsWith("ran", filtered); + Assert.Equal(2.5m, rows.Single().Rate); + Assert.False(ZyLog.AnyDropped(guarded), ZyLog.Decisions(guarded)); + } + + [Fact] + public void Zy_D4_an_explicit_implementation_in_a_subclass_does_not_deny_the_base_member() + { + ZydBadgeDto[] badges = { new() { Id = 1, Code = "B1" } }; + + (string filtered, List rows, PolicyQueryable guarded) = + Probe("D4 explicit implementation in a subclass", badges.AsQueryable(), "Code", "B1"); + + Assert.Equal("ran rows=1", filtered); + Assert.Equal("B1", rows.Single().Code); + Assert.False(ZyLog.AnyDropped(guarded), ZyLog.Decisions(guarded)); + } + + [Fact] + public void Zy_D5_a_public_member_beside_an_explicit_implementation_of_a_denying_interface() + { + ZydDevice[] devices = { new() { Id = 1, Serial = "S1" } }; + + (string filtered, List rows, PolicyQueryable guarded) = + Probe("D5 public member beside an explicit denying implementation", devices.AsQueryable(), "Serial", "S1"); + + Assert.Equal("ran rows=1", filtered); + Assert.Equal("S1", rows.Single().Serial); + Assert.False(ZyLog.AnyDropped(guarded), ZyLog.Decisions(guarded)); + } + + [Fact] + public void Zy_D7_an_entity_implementing_a_generic_interface_is_not_denied_by_an_open_generic_over_another_argument() + { + (string filtered, List rows, PolicyQueryable guarded) = + Probe("D7 entity beside an open generic implementer of IZydKeyed", _db.Tickets, "Title", "T"); + + Assert.Equal("ran rows=1", filtered); + Assert.Equal(7, rows.Single().Key); + Assert.False(ZyLog.AnyDropped(guarded), ZyLog.Decisions(guarded)); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewDefaultOrderTests.cs b/DynamicWhere.Tests/Policies/ReviewDefaultOrderTests.cs index 81c5b6e..4679c80 100644 --- a/DynamicWhere.Tests/Policies/ReviewDefaultOrderTests.cs +++ b/DynamicWhere.Tests/Policies/ReviewDefaultOrderTests.cs @@ -131,7 +131,6 @@ private string Log(string label, IEnumerable ids) string line = $"{label}: {order}"; _out.WriteLine(line); - ZbProbeLog.Write(line); return order; } diff --git a/DynamicWhere.Tests/Policies/ReviewOverBlockTests.cs b/DynamicWhere.Tests/Policies/ReviewOverBlockTests.cs index d971457..df7bda1 100644 --- a/DynamicWhere.Tests/Policies/ReviewOverBlockTests.cs +++ b/DynamicWhere.Tests/Policies/ReviewOverBlockTests.cs @@ -474,41 +474,6 @@ protected override void OnModelCreating(ModelBuilder model) // ------------------------------------------------------------ the probes - /// Appends a probe's outcome to a file named for the tree and the EF Core version it ran on. - internal static class ZbProbeLog - { - private static readonly object Gate = new(); - - internal static string Tag - { - get - { - string where = AppContext.BaseDirectory; - string tree = where.Contains("base-8a8d7fd") ? "base-8a8d7fd" - : where.Contains("parent-f7e1cc6") ? "parent-f7e1cc6" - : where.Contains("head-fce9c16") ? "head-fce9c16" - : "6c9e171"; - - return $"{tree}-ef{typeof(DbContext).Assembly.GetName().Version!.Major}"; - } - } - - internal static void Write(string line) - { - string directory = "/private/tmp/claude-501/-Users-sajadh92-Developer-Project-DynamicWhere-ex/8f118b4d-a861-42ce-b1d4-baa9f4b933e3/scratchpad/probes-b3"; - - if (!Directory.Exists(directory)) - { - return; - } - - lock (Gate) - { - File.AppendAllText(Path.Combine(directory, $"outcomes-{Tag}.txt"), line + Environment.NewLine); - } - } - } - public sealed class ReviewOverBlockTests : IDisposable { private readonly ITestOutputHelper _out; @@ -628,7 +593,6 @@ private string Outcome(string label, PolicyQueryable guarded, Func Prefs { get; set; } = new(); + } + + public class ZyoAddress + { + public string City { get; set; } = string.Empty; + + public Dictionary Extra { get; set; } = new(); + } + + public class ZyoProfile + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + [DwDenied] + public string? Secret { get; set; } + + public ZyoSettings Settings { get; set; } = new(); + + public List Addresses { get; set; } = new(); + } + + public sealed class ZyoContext : DbContext + { + private readonly SqliteConnection _connection; + + public ZyoContext(SqliteConnection connection) => _connection = connection; + + public DbSet Profiles => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + protected override void OnModelCreating(ModelBuilder model) + { + model.Entity().OwnsOne(p => p.Settings, s => s.Property(x => x.Prefs).HasConversion( + value => JsonSerializer.Serialize(value, (JsonSerializerOptions?)null), + text => JsonSerializer.Deserialize>(text, (JsonSerializerOptions?)null)!)); + model.Entity().OwnsMany(p => p.Addresses, a => + { + a.WithOwner().HasForeignKey("ZyoProfileId"); + a.Property("Id"); + a.HasKey("Id"); + a.Property(x => x.Extra).HasConversion( + value => JsonSerializer.Serialize(value, (JsonSerializerOptions?)null), + text => JsonSerializer.Deserialize>(text, (JsonSerializerOptions?)null)!); + }); + } + } + + public sealed class ReviewOwnedPrecisionTests : IDisposable + { + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZyoContext _db; + + public ReviewOwnedPrecisionTests(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + _db = new ZyoContext(_connection); + _db.Database.EnsureCreated(); + _db.Profiles.Add(new ZyoProfile + { + Name = "P1", + Secret = "zyo-secret", + Settings = new ZyoSettings { Theme = "dark", Prefs = { ["lang"] = "ar" } }, + Addresses = { new ZyoAddress { City = "Basra", Extra = { ["floor"] = 3 } } } + }); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private (ZyoProfile? Row, string Outcome) Run() + { + PolicyQueryable guarded = _db.Profiles.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = DwTier.Strict, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + try + { + ZyoProfile row = guarded.ToList(new Filter()).Data.Single(); + string outcome = $"OK json={JsonSerializer.Serialize(row)} dropped=[{ZyLog.Decisions(guarded)}]"; + + ZyLog.Write(_out, "O owned: " + outcome); + + return (row, outcome); + } + catch (Exception e) + { + string outcome = $"THREW {e.GetType().Name} {e.Message.Split('\n')[0]} dropped=[{ZyLog.Decisions(guarded)}]"; + + ZyLog.Write(_out, "O owned: " + outcome); + + return (null, outcome); + } + } + + [Fact] + public void Zy_O1_an_owned_member_holding_a_converted_bag_keeps_its_plain_fields() + { + (ZyoProfile? row, string outcome) = Run(); + + Assert.NotNull(row); + Assert.Null(row!.Secret); + Assert.Equal("dark", row.Settings?.Theme); + } + + [Fact] + public void Zy_O2_an_owned_member_holding_a_converted_bag_is_narrowed_around_the_bag() + { + (ZyoProfile? row, string outcome) = Run(); + + Assert.NotNull(row); + Assert.NotNull(row!.Settings); + Assert.False(row.Settings!.Prefs.ContainsKey("lang"), outcome); + Assert.Contains("Settings.Prefs Dropped: left out whole: it can hold what the policy cannot name", outcome); + } + + [Fact] + public void Zy_O3_an_owned_collection_holding_a_converted_bag_keeps_its_elements() + { + (ZyoProfile? row, string outcome) = Run(); + + Assert.NotNull(row); + Assert.Equal("Basra", Assert.Single(row!.Addresses).City); + } + + [Fact] + public void Zy_O4_an_owned_collection_holding_a_converted_bag_is_narrowed_around_the_bag() + { + (ZyoProfile? row, string outcome) = Run(); + + Assert.NotNull(row); + Assert.False(Assert.Single(row!.Addresses).Extra.ContainsKey("floor"), outcome); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewReshapeImpactTests.cs b/DynamicWhere.Tests/Policies/ReviewReshapeImpactTests.cs new file mode 100644 index 0000000..67e17af --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewReshapeImpactTests.cs @@ -0,0 +1,323 @@ +using System.Linq.Expressions; +using System.Text.Json; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Xunit.Abstractions; + +// What the needless projection behind a reshaping lambda costs on shapes other than a plain entity: an abstract +// hierarchy, a concrete root with derived rows, and a constructor-bound entity. The only denial (ZyiKey.Secret) sits +// beneath Owner.Keys, which nothing loads. Each test asserts the rows come back as the unguarded query returns them. + +namespace DynamicWhere.Tests.Policies +{ + public class ZyiOwner + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public List Keys { get; set; } = new(); + + public List Payments { get; set; } = new(); + + public List Docs { get; set; } = new(); + + public List Members { get; set; } = new(); + + public List Invoices { get; set; } = new(); + } + + /// DDD style: a private constructor for EF Core, a public one for the domain, private setters. + public class ZyiInvoice + { + private ZyiInvoice() + { + } + + public ZyiInvoice(string number) => Number = number; + + public int Id { get; private set; } + + public string Number { get; private set; } = string.Empty; + + public int ZyiOwnerId { get; private set; } + + public ZyiOwner? Owner { get; private set; } + } + + public class ZyiKey + { + public int Id { get; set; } + + public int ZyiOwnerId { get; set; } + + [DwDenied] + public string? Secret { get; set; } + } + + public abstract class ZyiPayment + { + public int Id { get; set; } + + public long Cents { get; set; } + + public int ZyiOwnerId { get; set; } + + public ZyiOwner? Owner { get; set; } + } + + public class ZyiCardPayment : ZyiPayment + { + public string Last4 { get; set; } = string.Empty; + } + + public class ZyiCashPayment : ZyiPayment + { + public string Till { get; set; } = string.Empty; + } + + public class ZyiDoc + { + public int Id { get; set; } + + public string Title { get; set; } = string.Empty; + + public int ZyiOwnerId { get; set; } + + public ZyiOwner? Owner { get; set; } + } + + public class ZyiRichDoc : ZyiDoc + { + public int Pages { get; set; } + } + + /// Constructor-bound: EF Core binds it, and it has no parameterless constructor. + public class ZyiMember + { + public ZyiMember(int id, string name, int zyiOwnerId) + { + Id = id; + Name = name; + ZyiOwnerId = zyiOwnerId; + } + + public int Id { get; private set; } + + public string Name { get; private set; } + + public int ZyiOwnerId { get; private set; } + + public ZyiOwner? Owner { get; private set; } + } + + public static class ZyiSpecs + { + public static readonly Expression> Paid = p => p.Cents > 0; + + public static readonly Expression> Titled = d => d.Title != string.Empty; + + public static readonly Expression> Named = m => m.Name != string.Empty; + + public static readonly Expression> Numbered = i => i.Number != string.Empty; + } + + public sealed class ZyiContext : DbContext + { + private readonly SqliteConnection _connection; + + public ZyiContext(SqliteConnection connection) => _connection = connection; + + public DbSet Owners => Set(); + + public DbSet Payments => Set(); + + public DbSet Docs => Set(); + + public DbSet Members => Set(); + + /// A queryable function, as EF Core maps a table-valued function (not mapped here; only its shape is read). + public IQueryable DocsOf(int ownerId) => Docs.Where(d => d.ZyiOwnerId == ownerId); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + protected override void OnModelCreating(ModelBuilder model) + { + model.Entity(); + model.Entity(); + model.Entity(); + model.Entity().HasOne(m => m.Owner).WithMany(o => o.Members).HasForeignKey(m => m.ZyiOwnerId); + } + } + + public sealed class ReviewReshapeImpactTests : IDisposable + { + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZyiContext _db; + + public ReviewReshapeImpactTests(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + _db = new ZyiContext(_connection); + _db.Database.EnsureCreated(); + + ZyiOwner owner = new() + { + Name = "W1", + Keys = { new ZyiKey { Secret = "zyi-secret" } }, + Payments = { new ZyiCardPayment { Cents = 100, Last4 = "4242" }, new ZyiCashPayment { Cents = 200, Till = "T1" } }, + Docs = { new ZyiDoc { Title = "plain" }, new ZyiRichDoc { Title = "rich", Pages = 3 } }, + Invoices = { new ZyiInvoice("INV-1") } + }; + + _db.Owners.Add(owner); + _db.SaveChanges(); + _db.Members.Add(new ZyiMember(0, "M1", owner.Id)); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static PolicyQueryable Guard(IQueryable source) where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = DwTier.Strict, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + private string Typed(string label, IQueryable source) where T : class + { + string unguarded = JsonSerializer.Serialize(source.AsNoTracking().ToList().Select(r => r.GetType().Name)); + PolicyQueryable guarded = Guard(source); + string outcome; + + _db.ChangeTracker.Clear(); + + try + { + List rows = guarded.ToList(new Filter()).Data; + + outcome = $"OK types={JsonSerializer.Serialize(rows.Select(r => r.GetType().Name))} json={JsonSerializer.Serialize(rows)}"; + } + catch (Exception e) + { + outcome = $"THREW {e.GetType().Name} {(e is PolicyException p ? p.ErrorCode.ToString() : string.Empty)} {e.Message.Split('\n')[0]}"; + } + + ZyLog.Write(_out, $"I {label}: {ZyLog.Shape(source.Expression)} unguarded types={unguarded} guarded {outcome} dropped=[{ZyLog.Decisions(guarded)}]"); + + return outcome; + } + + [Fact] + public void Zy_I1_control_abstract_rows_through_a_navigation_collection() + { + string outcome = Typed("I1 control SelectMany(o => o.Payments)", _db.Owners.SelectMany(o => o.Payments)); + + Assert.StartsWith("OK", outcome); + Assert.Contains("ZyiCardPayment", outcome); + } + + [Fact] + public void Zy_I2_abstract_rows_through_a_navigation_collection_with_a_specification() + { + string outcome = Typed("I2 SelectMany(o => o.Payments.AsQueryable().Where(spec))", _db.Owners.SelectMany(o => o.Payments.AsQueryable().Where(ZyiSpecs.Paid))); + + Assert.StartsWith("OK", outcome); + Assert.Contains("ZyiCardPayment", outcome); + } + + [Fact] + public void Zy_I3_control_derived_rows_of_a_concrete_root() + { + string outcome = Typed("I3 control SelectMany(o => o.Docs)", _db.Owners.SelectMany(o => o.Docs)); + + Assert.Contains("ZyiRichDoc", outcome); + } + + [Fact] + public void Zy_I4_derived_rows_of_a_concrete_root_with_a_specification() + { + string outcome = Typed("I4 SelectMany(o => o.Docs.AsQueryable().Where(spec))", _db.Owners.SelectMany(o => o.Docs.AsQueryable().Where(ZyiSpecs.Titled))); + + Assert.Contains("ZyiRichDoc", outcome); + } + + [Fact] + public void Zy_I5_control_a_constructor_bound_entity() + { + string outcome = Typed("I5 control SelectMany(o => o.Members)", _db.Owners.SelectMany(o => o.Members)); + + Assert.StartsWith("OK", outcome); + } + + [Fact] + public void Zy_I6_a_constructor_bound_entity_with_a_specification() + { + string outcome = Typed("I6 SelectMany(o => o.Members.AsQueryable().Where(spec))", _db.Owners.SelectMany(o => o.Members.AsQueryable().Where(ZyiSpecs.Named))); + + Assert.StartsWith("OK", outcome); + } + + [Fact] + public void Zy_I9_control_a_DDD_aggregate_with_a_private_constructor() + { + string outcome = Typed("I9 control SelectMany(o => o.Invoices)", _db.Owners.SelectMany(o => o.Invoices)); + + Assert.StartsWith("OK", outcome); + Assert.Contains("INV-1", outcome); + } + + [Fact] + public void Zy_I10_a_DDD_aggregate_with_a_private_constructor_and_a_specification() + { + string outcome = Typed("I10 SelectMany(o => o.Invoices.AsQueryable().Where(spec))", _db.Owners.SelectMany(o => o.Invoices.AsQueryable().Where(ZyiSpecs.Numbered))); + + Assert.StartsWith("OK", outcome); + Assert.Contains("INV-1", outcome); + } + + [Fact] + public void Zy_I11_a_DDD_aggregate_through_a_left_join_in_query_syntax() + { + IQueryable source = + from o in _db.Owners + join i in _db.Set() on o.Id equals i.ZyiOwnerId into invoices + from i in invoices.DefaultIfEmpty() + select i; + + string outcome = Typed("I11 left join to a DDD aggregate", source); + + Assert.StartsWith("OK", outcome); + Assert.Contains("INV-1", outcome); + } + + [Fact] + public void Zy_I7_a_queryable_function_on_the_context_is_read_as_building() + { + // Shape only: EF Core maps such a method to a table-valued function; SQLite cannot run one. + IQueryable source = _db.Owners.SelectMany(o => _db.DocsOf(o.Id)); + string shape = ZyLog.Shape(source.Expression); + + ZyLog.Write(_out, $"I I7 SelectMany(o => _db.DocsOf(o.Id)) (shape only): {shape}"); + + Assert.DoesNotContain("builds=True", shape); + } + + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewReshapePrecisionTests.cs b/DynamicWhere.Tests/Policies/ReviewReshapePrecisionTests.cs new file mode 100644 index 0000000..e3534a8 --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewReshapePrecisionTests.cs @@ -0,0 +1,630 @@ +using System.Linq.Expressions; +using System.Reflection; +using System.Text.Json; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Xunit.Abstractions; + +// What an ordinary reshaped EF Core query gets when the only denial (ZyrCard.Pan) sits beneath a navigation nothing +// loads: no projection, the auto-included Tier kept, the converted Meta bag kept. Specifications, repositories, +// FromSql, a context's Set through an interface, composite keys and query-syntax joins, lets and left joins read no +// more than a plain chain does. QueryRoot's internals are reached by reflection. + +namespace DynamicWhere.Tests.Policies +{ + public class ZyrTier + { + public int Id { get; set; } + + public string Label { get; set; } = string.Empty; + } + + public class ZyrCustomer + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public string Region { get; set; } = string.Empty; + + public int ZyrTierId { get; set; } + + /// Automatically included; nothing beneath it is denied. + public ZyrTier? Tier { get; set; } + + /// A converted JSON bag, the pre-EF7 way to store one. + public Dictionary Meta { get; set; } = new(); + + /// Never loaded by any probe; the only denial is beneath it. + public List Cards { get; set; } = new(); + + public List Orders { get; set; } = new(); + } + + public class ZyrCard + { + public int Id { get; set; } + + public int ZyrCustomerId { get; set; } + + public string Label { get; set; } = string.Empty; + + [DwDenied] + public string? Pan { get; set; } + } + + public class ZyrChannel + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + } + + public class ZyrOrder + { + public int Id { get; set; } + + public string Code { get; set; } = string.Empty; + + public string Region { get; set; } = string.Empty; + + public int ZyrCustomerId { get; set; } + + /// Never loaded; Customer.Cards.Pan is the denial beneath it. + public ZyrCustomer? Customer { get; set; } + + public int ZyrChannelId { get; set; } + + /// Automatically included; nothing beneath it is denied. + public ZyrChannel? Channel { get; set; } + + /// A converted JSON bag. + public Dictionary Meta { get; set; } = new(); + } + + /// A context exposed through an interface, as clean-architecture templates do. + public interface IZyrContext + { + DbSet Set() where TEntity : class; + } + + /// What a repository layer looks like: a method and a property handing out a plain set. + public sealed class ZyrRepository + { + private readonly ZyrContext _db; + + public ZyrRepository(ZyrContext db) => _db = db; + + public IQueryable Customers() => _db.Customers; + + public IQueryable CustomerSet => _db.Customers; + } + + /// Reusable predicates, the specification pattern. + public static class ZyrSpecs + { + public static readonly Expression> Open = o => o.Code != string.Empty; + + public static Expression> InRegion(string region) => o => o.Region == region; + + public static readonly Expression> Named = c => c.Name != string.Empty; + } + + public readonly record struct ZyrRegionId(int Value); + + /// A request object a controller captures into a query. + public sealed class ZyrFilter + { + public string Region { get; set; } = string.Empty; + + public List Codes { get; set; } = new(); + } + + public sealed class ZyrContext : DbContext, IZyrContext + { + private readonly SqliteConnection _connection; + + public ZyrContext(SqliteConnection connection) => _connection = connection; + + public DbSet Customers => Set(); + + public DbSet Orders => Set(); + + public DbSet Tiers => Set(); + + public DbSet Channels => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + protected override void OnModelCreating(ModelBuilder model) + { + model.Entity().ToTable("ZyrCustomers"); + model.Entity().ToTable("ZyrOrders"); + model.Entity().Navigation(c => c.Tier).AutoInclude(); + model.Entity().Navigation(o => o.Channel).AutoInclude(); + model.Entity().Property(c => c.Meta).HasConversion( + value => JsonSerializer.Serialize(value, (JsonSerializerOptions?)null), + text => JsonSerializer.Deserialize>(text, (JsonSerializerOptions?)null)!); + model.Entity().Property(o => o.Meta).HasConversion( + value => JsonSerializer.Serialize(value, (JsonSerializerOptions?)null), + text => JsonSerializer.Deserialize>(text, (JsonSerializerOptions?)null)!); + } + } + + /// Writes a test's outcome to its output, and reads the query's shape. + internal static class ZyLog + { + internal static void Write(ITestOutputHelper output, string line) => output.WriteLine(line); + + /// QueryRoot.Reshapes / Builds, by reflection, where the tree has them. + internal static string Shape(Expression expression) + { + Type? root = typeof(Filter).Assembly.GetType("DynamicWhere.ex.Source.QueryRoot"); + + string Call(string name) + { + MethodInfo? method = root?.GetMethod(name, BindingFlags.NonPublic | BindingFlags.Static, null, new[] { typeof(Expression) }, null); + + if (method is null) + { + return "n/a"; + } + + try + { + return method.Invoke(null, new object[] { expression })!.ToString()!; + } + catch (TargetInvocationException e) + { + return "threw " + e.InnerException?.GetType().Name; + } + } + + return $"reshapes={Call("Reshapes")} builds={Call("Builds")}"; + } + + internal static string Decisions(PolicyQueryable guarded) where T : class => + string.Join(" | ", guarded.LastTrace?.Decisions + .Where(d => d.Action != PolicyAction.Allowed) + .Select(d => $"{d.FieldPath} {d.Action}: {d.Reason}") ?? Array.Empty()); + + internal static bool AnyDropped(PolicyQueryable guarded) where T : class => + guarded.LastTrace?.Decisions.Any(d => d.Action == PolicyAction.Dropped) ?? false; + } + + public sealed class ReviewReshapePrecisionTests : IDisposable + { + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZyrContext _db; + private readonly ZyrRepository _repo; + + public ReviewReshapePrecisionTests(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + _db = new ZyrContext(_connection); + _db.Database.EnsureCreated(); + _repo = new ZyrRepository(_db); + + ZyrTier gold = new() { Label = "gold" }; + ZyrChannel web = new() { Name = "web" }; + + ZyrCustomer c1 = new() + { + Name = "C1", + Region = "north", + Tier = gold, + Meta = { ["k"] = "v1" }, + Cards = { new ZyrCard { Label = "visa", Pan = "zyr-pan-secret" } } + }; + ZyrCustomer c2 = new() { Name = "C2", Region = "south", Tier = gold, Meta = { ["k"] = "v2" } }; + + c1.Orders.Add(new ZyrOrder { Code = "O1", Region = "north", Channel = web, Meta = { ["k"] = "o1" } }); + c1.Orders.Add(new ZyrOrder { Code = "O2", Region = "south", Channel = web, Meta = { ["k"] = "o2" } }); + c2.Orders.Add(new ZyrOrder { Code = "O3", Region = "south", Channel = web, Meta = { ["k"] = "o3" } }); + + _db.Customers.AddRange(c1, c2); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static PolicyQueryable Guard(IQueryable source, DwTier tier = DwTier.Strict) where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = tier, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + /// + /// Runs the unguarded query (to show EF Core translates it) and the guarded one, logs both, and returns the + /// guarded rows with the handle; null rows when the unguarded query itself does not translate. + /// + private (List? Rows, PolicyQueryable Guarded) Run(string label, IQueryable source) where T : class + { + string shape = ZyLog.Shape(source.Expression); + PolicyQueryable guarded = Guard(source); + + try + { + int count = source.AsNoTracking().ToList().Count; + + ZyLog.Write(_out, $"R {label}: unguarded OK rows={count} {shape}"); + } + catch (Exception e) + { + ZyLog.Write(_out, $"R {label}: unguarded THREW {e.GetType().Name} {e.Message.Split('\n')[0]} {shape}"); + + return (null, guarded); + } + + _db.ChangeTracker.Clear(); + + try + { + List rows = guarded.ToList(new Filter()).Data; + int tracked = _db.ChangeTracker.Entries().Count(); + + ZyLog.Write(_out, $"R {label}: guarded OK rows={rows.Count} tracked={tracked} dropped=[{ZyLog.Decisions(guarded)}]"); + + return (rows, guarded); + } + catch (Exception e) + { + string code = e is PolicyException refusal ? refusal.ErrorCode.ToString() : string.Empty; + + ZyLog.Write(_out, $"R {label}: guarded THREW {e.GetType().Name} {code} {e.Message.Split('\n')[0]} dropped=[{ZyLog.Decisions(guarded)}]"); + + throw; + } + } + + private void CustomersAsLoaded(string label, IQueryable source, int count) + { + (List? rows, PolicyQueryable guarded) = Run(label, source); + + if (rows is null) + { + return; + } + + Assert.Equal(count, rows.Count); + Assert.False(ZyLog.AnyDropped(guarded), ZyLog.Decisions(guarded)); + Assert.All(rows, row => Assert.Equal("gold", row.Tier?.Label)); + Assert.All(rows, row => Assert.True(row.Meta.ContainsKey("k"), "Meta left out")); + } + + private void OrdersAsLoaded(string label, IQueryable source, int count) + { + (List? rows, PolicyQueryable guarded) = Run(label, source); + + if (rows is null) + { + return; + } + + Assert.Equal(count, rows.Count); + Assert.False(ZyLog.AnyDropped(guarded), ZyLog.Decisions(guarded)); + Assert.All(rows, row => Assert.Equal("web", row.Channel?.Name)); + Assert.All(rows, row => Assert.True(row.Meta.ContainsKey("k"), "Meta left out")); + } + + // ------------------------------------------------------------------ controls: expected green everywhere + + [Fact] + public void Zy_R00_control_Select_to_a_navigation() + { + CustomersAsLoaded("R00 Select(o => o.Customer)", _db.Orders.Select(o => o.Customer!), 3); + } + + [Fact] + public void Zy_R01_control_SelectMany_over_a_collection() + { + OrdersAsLoaded("R01 SelectMany(c => c.Orders)", _db.Customers.SelectMany(c => c.Orders), 3); + } + + [Fact] + public void Zy_R02_control_correlated_subquery_over_a_captured_set_property() + { + CustomersAsLoaded( + "R02 SelectMany(o => _db.Customers.Where(..))", + _db.Orders.SelectMany(o => _db.Customers.Where(c => c.Id == o.ZyrCustomerId)), + 3); + } + + [Fact] + public void Zy_R03_control_correlated_subquery_over_DbContext_Set() + { + CustomersAsLoaded( + "R03 SelectMany(o => _db.Set().Where(..))", + _db.Orders.SelectMany(o => _db.Set().Where(c => c.Id == o.ZyrCustomerId)), + 3); + } + + [Fact] + public void Zy_R04_control_correlated_subquery_over_a_repository_property() + { + CustomersAsLoaded( + "R04 SelectMany(o => _repo.CustomerSet.Where(..))", + _db.Orders.SelectMany(o => _repo.CustomerSet.Where(c => c.Id == o.ZyrCustomerId)), + 3); + } + + [Fact] + public void Zy_R05_control_captured_plain_query_variable() + { + IQueryable named = _db.Customers.Where(c => c.Name != string.Empty).AsNoTracking(); + + CustomersAsLoaded( + "R05 SelectMany(o => named.Where(..))", + _db.Orders.SelectMany(o => named.Where(c => c.Id == o.ZyrCustomerId)), + 3); + } + + [Fact] + public void Zy_R06_control_captured_values_in_a_nested_predicate() + { + List regions = new() { "north", "south" }; + ZyrRegionId region = new(1); + DateTime cutoff = new(2000, 1, 1); + + OrdersAsLoaded( + "R06 nested predicate: list.Contains, struct id, DateTime, new DateTime", + _db.Customers.SelectMany(c => c.Orders.Where(o => regions.Contains(o.Region) + && region.Value > 0 + && new DateTime(2100, 1, 1) > cutoff)), + 3); + } + + [Fact] + public void Zy_R07_control_Join_on_a_single_key() + { + CustomersAsLoaded( + "R07 Join single key", + _db.Orders.Join(_db.Customers, o => o.ZyrCustomerId, c => c.Id, (o, c) => c), + 3); + } + + // ------------------------------------------------------------------ what a lambda reads without building + + [Fact] + public void Zy_R10_a_specification_variable_inside_a_correlated_subquery() + { + Expression> named = c => c.Name != string.Empty; + + CustomersAsLoaded( + "R10 SelectMany(o => _db.Customers.Where(specVariable).Where(..))", + _db.Orders.SelectMany(o => _db.Customers.Where(named).Where(c => c.Id == o.ZyrCustomerId)), + 3); + } + + [Fact] + public void Zy_R11_a_static_specification_on_a_navigation_collection() + { + OrdersAsLoaded( + "R11 SelectMany(c => c.Orders.AsQueryable().Where(ZyrSpecs.Open))", + _db.Customers.SelectMany(c => c.Orders.AsQueryable().Where(ZyrSpecs.Open)), + 3); + } + + [Fact] + public void Zy_R12_a_specification_factory_method_on_a_navigation_collection() + { + OrdersAsLoaded( + "R12 SelectMany(c => c.Orders.AsQueryable().Where(ZyrSpecs.InRegion(\"south\")))", + _db.Customers.SelectMany(c => c.Orders.AsQueryable().Where(ZyrSpecs.InRegion("south"))), + 2); + } + + [Fact] + public void Zy_R13_a_static_specification_inside_a_correlated_subquery_over_a_set() + { + CustomersAsLoaded( + "R13 SelectMany(o => _db.Customers.Where(ZyrSpecs.Named).Where(..))", + _db.Orders.SelectMany(o => _db.Customers.Where(ZyrSpecs.Named).Where(c => c.Id == o.ZyrCustomerId)), + 3); + } + + [Fact] + public void Zy_R14_a_repository_method_inside_a_correlated_subquery() + { + CustomersAsLoaded( + "R14 SelectMany(o => _repo.Customers().Where(..))", + _db.Orders.SelectMany(o => _repo.Customers().Where(c => c.Id == o.ZyrCustomerId)), + 3); + } + + [Fact] + public void Zy_R15_Set_through_a_context_interface_inside_a_correlated_subquery() + { + IZyrContext context = _db; + + CustomersAsLoaded( + "R15 SelectMany(o => iface.Set().Where(..))", + _db.Orders.SelectMany(o => context.Set().Where(c => c.Id == o.ZyrCustomerId)), + 3); + } + + [Fact] + public void Zy_R16_FromSqlRaw_inside_a_correlated_subquery() + { + CustomersAsLoaded( + "R16 SelectMany(o => _db.Customers.FromSqlRaw(..).Where(..))", + _db.Orders.SelectMany(o => _db.Customers.FromSqlRaw("SELECT * FROM ZyrCustomers").Where(c => c.Id == o.ZyrCustomerId)), + 3); + } + + [Fact] + public void Zy_R17_a_specification_in_query_syntax() + { + Expression> open = o => o.Code != string.Empty; + + IQueryable source = + from c in _db.Customers + from o in c.Orders.AsQueryable().Where(open) + select o; + + OrdersAsLoaded("R17 from c .. from o in c.Orders.AsQueryable().Where(spec) select o", source, 3); + } + + // ------------------------------------------------------------------ 3.2.0-wide: anonymous keys and carriers + + [Fact] + public void Zy_R20_Join_on_a_composite_key() + { + CustomersAsLoaded( + "R20 Join composite key", + _db.Orders.Join(_db.Customers, o => new { Id = o.ZyrCustomerId, o.Region }, c => new { c.Id, c.Region }, (o, c) => c), + 2); + } + + [Fact] + public void Zy_R21_left_join_in_query_syntax() + { + IQueryable source = + from o in _db.Orders + join c in _db.Customers on o.ZyrCustomerId equals c.Id into cs + from c in cs.DefaultIfEmpty() + select c; + + CustomersAsLoaded("R21 left join (GroupJoin + DefaultIfEmpty)", source, 3); + } + + [Fact] + public void Zy_R22_two_from_clauses_with_a_where() + { + IQueryable source = + from c in _db.Customers + from o in c.Orders + where o.Code != string.Empty + select o; + + OrdersAsLoaded("R22 from c from o in c.Orders where .. select o", source, 3); + } + + [Fact] + public void Zy_R23_GroupBy_on_a_composite_key_then_First() + { + if (typeof(DbContext).Assembly.GetName().Version!.Major < 7) + { + return; + } + + OrdersAsLoaded( + "R23 GroupBy(new { .. }).Select(g => g.OrderBy(..).First())", + _db.Orders.GroupBy(o => new { o.ZyrCustomerId, o.Region }).Select(g => g.OrderBy(o => o.Id).First()), + 3); + } + + [Fact] + public void Zy_R08_control_a_conditional_with_null() + { + CustomersAsLoaded( + "R08 Select(o => o.ZyrCustomerId > 0 ? o.Customer : null)", + _db.Orders.Select(o => o.ZyrCustomerId > 0 ? o.Customer! : null!), + 3); + } + + [Fact] + public void Zy_R09_control_captured_filter_object_in_a_nested_predicate() + { + ZyrFilter filter = new() { Region = "south", Codes = new List { "O2", "O3" } }; + + OrdersAsLoaded( + "R09 nested predicate over a captured filter object's members", + _db.Customers.SelectMany(c => c.Orders.Where(o => o.Region == filter.Region && filter.Codes.Contains(o.Code))), + 2); + } + + [Fact] + public void Zy_R18_control_two_from_clauses_without_a_where() + { + IQueryable source = + from c in _db.Customers + from o in c.Orders + select o; + + OrdersAsLoaded("R18 from c from o in c.Orders select o", source, 3); + } + + [Fact] + public void Zy_R19_control_join_in_query_syntax() + { + IQueryable source = + from o in _db.Orders + join c in _db.Customers on o.ZyrCustomerId equals c.Id + select c; + + CustomersAsLoaded("R19 from o join c .. select c", source, 3); + } + + [Fact] + public void Zy_R25_join_then_where_in_query_syntax() + { + IQueryable source = + from o in _db.Orders + join c in _db.Customers on o.ZyrCustomerId equals c.Id + where o.Code != string.Empty + select c; + + CustomersAsLoaded("R25 from o join c .. where o.Code .. select c", source, 3); + } + + /// Shape only: value-typed calls in predicates and keys (string.Format, interpolation, Convert, ToUpper, Nullable). + [Fact] + public void Zy_R30_control_value_calls_in_predicates_and_keys_are_not_read_as_building() + { + (string Label, System.Linq.Expressions.Expression Expression)[] shapes = + { + ("string.Format in a nested predicate", _db.Customers.SelectMany(c => c.Orders.Where(o => string.Format("{0}", o.Code) == "O1")).Expression), + ("interpolation in a nested predicate", _db.Customers.SelectMany(c => c.Orders.Where(o => $"{o.Code}-{o.Region}" != string.Empty)).Expression), + ("Convert in a nested predicate", _db.Customers.SelectMany(c => c.Orders.Where(o => Convert.ToString(o.ZyrCustomerId) != string.Empty)).Expression), + ("ToUpper key", _db.Orders.GroupBy(o => o.Region.ToUpper()).Select(g => g.OrderBy(o => o.Id).First()).Expression), + ("Nullable key", _db.Orders.GroupBy(o => (int?)o.ZyrCustomerId).Select(g => g.OrderBy(o => o.Id).First()).Expression), + ("EF.Functions in a nested predicate", _db.Customers.SelectMany(c => c.Orders.Where(o => EF.Functions.Like(o.Code, "O%"))).Expression) + }; + + List building = new(); + + foreach ((string label, System.Linq.Expressions.Expression expression) in shapes) + { + string shape = ZyLog.Shape(expression); + + ZyLog.Write(_out, $"R R30 {label}: {shape}"); + + if (shape.Contains("builds=True")) + { + building.Add(label); + } + } + + Assert.Empty(building); + } + + [Fact] + public void Zy_R24_let_clause() + { + IQueryable source = + from o in _db.Orders + let c = o.Customer + where c!.Name != string.Empty + select c; + + CustomersAsLoaded("R24 let c = o.Customer", source, 3); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewSharedNameTests.cs b/DynamicWhere.Tests/Policies/ReviewSharedNameTests.cs new file mode 100644 index 0000000..cb06b3e --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewSharedNameTests.cs @@ -0,0 +1,183 @@ +using System.Reflection; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Classes.Core; +using DynamicWhere.ex.Enums; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Source; +using Xunit.Abstractions; + +namespace DynamicWhere.Tests.Policies +{ + // ============================================================================ other declarations: models + + /// A base type whose denied member a subtype overrides with a covariant return type (C# 9). + public class ZxCovBase + { + protected ZxBaseCard? CardValue; + + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + [DwDenied] + public virtual ZxBaseCard? Card => CardValue; + + public void Put(ZxBaseCard card) => CardValue = card; + } + + public class ZxCovDerived : ZxCovBase + { + public override ZxDerivedCard? Card => CardValue as ZxDerivedCard; + } + + /// The same with a scalar beneath, the denial on the abstract base declaration. + public abstract class ZxCovAbstract + { + public int Id { get; set; } + + [DwDenied] + public abstract ZxBaseCard? Card { get; } + } + + public class ZxCovConcrete : ZxCovAbstract + { + public ZxDerivedCard? Held { private get; set; } + + public override ZxDerivedCard? Card => Held; + } + + // ---- a default interface member that a derived interface overrides explicitly, with the denial there ---- + + public interface IZxCoded + { + string? Code { get; } + } + + public interface IZxSecretCoded : IZxCoded + { + string? Raw { get; } + + [DwDenied] + string? IZxCoded.Code => Raw; + } + + public class ZxCodedImpl : IZxSecretCoded + { + public string? Raw { get; set; } + } + + public class ZxCodedRow + { + public int Id { get; set; } + + public IZxCoded? Coded { get; set; } + } + + // ---- T itself hides a base member with new, typed as an object holding a denied field ---- + + public class ZxHideBase + { + public int Id { get; set; } + + public string? Info { get; set; } + } + + public class ZxHideDerived : ZxHideBase + { + public new ZxPayCard? Info { get; set; } + } + + /// The same beside a top-level denial, so a projection is built. + public class ZxHideDerivedDenied : ZxHideBase + { + [DwDenied] + public string? Ssn { get; set; } + + public new ZxPayCard? Info { get; set; } + } + + // ============================================================================ other declarations: tests + + public sealed class ReviewSharedNameTests + { + private readonly ITestOutputHelper _out; + + public ReviewSharedNameTests(ITestOutputHelper output) => _out = output; + + [Fact] + public void Zx_D1_a_covariant_override_keeps_the_base_declarations_denial() + { + Assert.NotEmpty(ZxKit.Denials(typeof(ZxCovDerived), "Card")); + Assert.NotEmpty(ZxKit.Denials(typeof(ZxCovConcrete), "Card")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_D1_rows_of_the_covariant_subtype_do_not_return_the_denied_member(DwTier tier) + { + ZxCovDerived row = new() { Id = 1, Name = "r1" }; + row.Put(new ZxDerivedCard { Label = "covariant-card-secret" }); + ZxCovConcrete concrete = new() { Id = 2, Held = new ZxDerivedCard { Label = "abstract-covariant-secret" } }; + + PolicyQueryable guarded = ZxKit.Guard(new[] { row }.AsQueryable(), tier); + object? data = ZxKit.SafeRead(() => guarded.ToList(new Filter()).Data); + + _out.WriteLine($"{tier}: sent={ZxKit.Json(data)} trace=[{ZxKit.Trace(guarded)}]"); + + PolicyQueryable guarded2 = ZxKit.Guard(new[] { concrete }.AsQueryable(), tier); + object? data2 = ZxKit.SafeRead(() => guarded2.ToList(new Filter()).Data); + + _out.WriteLine($"{tier}: sent={ZxKit.Json(data2)} trace=[{ZxKit.Trace(guarded2)}]"); + + Assert.False(ZxKit.Holds(data, "covariant-card-secret")); + Assert.False(ZxKit.Holds(data2, "abstract-covariant-secret")); + } + + [Fact] + public void Zx_D4_a_denial_on_a_derived_interfaces_explicit_default_implementation() + { + ZxCodedRow[] rows = { new() { Id = 1, Coded = new ZxCodedImpl { Raw = "dim-code-secret" } } }; + + _out.WriteLine("runs: " + ((IZxCoded)rows[0].Coded!).Code); + _out.WriteLine("Coded.Code denials: " + ZxKit.Denials(typeof(ZxCodedRow), "Coded.Code").Count()); + + string where = ZxKit.Code(() => ZxKit.Guard(rows.AsQueryable()).ToList(ZxKit.Where("Coded.Code", "dim-code-secret"))); + object? data = ZxKit.SafeRead(() => ZxKit.Guard(rows.AsQueryable()).ToList(new Filter()).Data); + + _out.WriteLine($"where={where} sent={ZxKit.Json(data)}"); + + Assert.NotEqual("ran", where); + Assert.False(ZxKit.Holds(data, "dim-code-secret") && ZxKit.Json(data).Contains("dim-code-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_D5_a_row_type_that_hides_a_value_member_with_an_object_holding_a_denied_field(DwTier tier) + { + PropertyInfo[] infos = typeof(ZxHideDerived).GetProperties().Where(p => p.Name == "Info").ToArray(); + + _out.WriteLine("GetProperties order: " + string.Join(", ", infos.Select(p => $"{p.DeclaringType!.Name}.{p.Name}:{p.PropertyType.Name}"))); + _out.WriteLine("Info.Pan denials: " + ZxKit.Denials(typeof(ZxHideDerived), "Info.Pan").Count()); + + ZxHideDerived[] rows = { new() { Id = 1, Info = new ZxPayCard { Label = "visa", Pan = "hide-pan-secret" } } }; + ZxHideDerivedDenied[] denied = { new() { Id = 2, Ssn = "hide-ssn", Info = new ZxPayCard { Label = "visa", Pan = "hide-pan-secret-2" } } }; + + PolicyQueryable guarded = ZxKit.Guard(rows.AsQueryable(), tier); + object? data = ZxKit.SafeRead(() => guarded.ToList(new Filter()).Data); + object? named = ZxKit.SafeRead(() => ZxKit.Guard(rows.AsQueryable(), tier).ToList(ZxKit.Selecting("Id", "Info")).Data); + PolicyQueryable guarded2 = ZxKit.Guard(denied.AsQueryable(), tier); + object? data2 = ZxKit.SafeRead(() => guarded2.ToList(new Filter()).Data); + + _out.WriteLine($"{tier} none: sent={ZxKit.Json(data)} trace=[{ZxKit.Trace(guarded)}]"); + _out.WriteLine($"{tier} named: sent={ZxKit.Json(named)}"); + _out.WriteLine($"{tier} projected: sent={ZxKit.Json(data2)} trace=[{ZxKit.Trace(guarded2)}]"); + + Assert.False(ZxKit.Holds(data, "hide-pan-secret")); + Assert.False(ZxKit.Holds(named, "hide-pan-secret")); + Assert.False(ZxKit.Holds(data2, "hide-pan-secret-2")); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewTrackingKit.cs b/DynamicWhere.Tests/Policies/ReviewTrackingKit.cs new file mode 100644 index 0000000..bc40cd8 --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewTrackingKit.cs @@ -0,0 +1,201 @@ +using System.Collections; +using System.Reflection; +using System.Text.Json; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Classes.Core; +using DynamicWhere.ex.Enums; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.DTOs; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; + +namespace DynamicWhere.Tests.Policies +{ + /// Shared helpers for the tracking, variance, converter, shared-name and collection tests. + internal static class ZxKit + { + internal static PolicyQueryable Guard(IQueryable source, DwTier tier = DwTier.Strict) where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = tier, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + internal static Filter Where(string field, string value) => new() + { + ConditionGroup = new ConditionGroup + { + Conditions = { new Condition { Sort = 1, Field = field, DataType = DataType.Text, Operator = Operator.Equal, Values = { value } } } + } + }; + + internal static Filter Selecting(params string[] fields) => new() { Selects = fields.ToList() }; + + /// The refusal code, "ran" when it ran, or the other exception. + internal static string Code(Action run) + { + try + { + run(); + + return "ran"; + } + catch (PolicyException refusal) + { + return refusal.ErrorCode.ToString(); + } + catch (Exception other) + { + return $"{other.GetType().Name}: {other.Message.Split('\n')[0]}"; + } + } + + /// Runs a read; a policy refusal or a core refusal returns null (nothing reached the caller). + internal static object? SafeRead(Func read) + { + try + { + return read(); + } + catch (PolicyException) + { + return null; + } + catch (LogicException) + { + return null; + } + } + + internal static IEnumerable Denials(Type type, string path) => + new AttributePolicyProvider().GetFragments(type, new DwPolicyContext()) + .Where(f => f.Effect == PolicyEffect.Deny && string.Equals(f.FieldPath, path, StringComparison.OrdinalIgnoreCase)); + + internal static string Trace(PolicyQueryable guarded) where T : class => + string.Join(" | ", guarded.LastTrace?.Decisions + .Where(d => d.Action != PolicyAction.Allowed) + .Select(d => $"{d.FieldPath} {d.Action}: {d.Reason}") ?? Array.Empty()); + + internal static string Json(object? value) + { + try + { + return JsonSerializer.Serialize(value, new JsonSerializerOptions + { + ReferenceHandler = System.Text.Json.Serialization.ReferenceHandler.IgnoreCycles + }); + } + catch (Exception e) + { + return $""; + } + } + + /// Everything reachable from a value, read by runtime type (public and non-public properties and fields). + internal static bool Holds(object? value, string text) + { + HashSet seen = new(ReferenceEqualityComparer.Instance); + Stack pending = new(); + + pending.Push(value); + + while (pending.Count > 0) + { + object? current = pending.Pop(); + + if (current is null || current is MemberInfo || current is Delegate + || (current.GetType().Namespace ?? string.Empty).StartsWith("Microsoft.", StringComparison.Ordinal)) + { + continue; + } + + if (current is string held) + { + if (held.Contains(text, StringComparison.Ordinal)) + { + return true; + } + + continue; + } + + if (current.GetType().IsPrimitive || current is decimal || current is DateTime || current is Guid) + { + continue; + } + + if (!current.GetType().IsValueType && !seen.Add(current)) + { + continue; + } + + if (current is IDictionary map) + { + foreach (DictionaryEntry item in map) + { + pending.Push(item.Key); + pending.Push(item.Value); + } + } + else if (current is IEnumerable items) + { + try + { + foreach (object? item in items) + { + pending.Push(item); + } + } + catch + { + // An enumerator that throws holds nothing readable. + } + } + + const BindingFlags all = BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance; + + foreach (PropertyInfo property in current.GetType().GetProperties(all)) + { + if (property.GetIndexParameters().Length != 0 || !property.CanRead) + { + continue; + } + + try + { + pending.Push(property.GetValue(current)); + } + catch + { + // A getter that throws holds nothing readable. + } + } + + for (Type? type = current.GetType(); type is not null && type != typeof(object); type = type.BaseType) + { + if (type.Namespace is { } ns && (ns == "System" || ns.StartsWith("System.", StringComparison.Ordinal) + || ns.StartsWith("Microsoft.", StringComparison.Ordinal))) + { + continue; + } + + foreach (FieldInfo field in type.GetFields(all | BindingFlags.DeclaredOnly)) + { + try + { + pending.Push(field.GetValue(current)); + } + catch + { + // Unreadable. + } + } + } + } + + return false; + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewTrackingTests.cs b/DynamicWhere.Tests/Policies/ReviewTrackingTests.cs new file mode 100644 index 0000000..675f963 --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewTrackingTests.cs @@ -0,0 +1,325 @@ +using System.Collections; +using System.Linq.Expressions; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Source; +using DynamicWhere.ex.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Xunit.Abstractions; + +namespace DynamicWhere.Tests.Policies +{ + // ============================================================================ reshaped chains and tracking: models + + public class ZxShopper + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public List Cards { get; set; } = new(); + } + + public class ZxPayCard + { + public int Id { get; set; } + + public int ZxShopperId { get; set; } + + public string Label { get; set; } = string.Empty; + + [DwDenied] + public string? Pan { get; set; } + } + + public class ZxPurchase + { + public int Id { get; set; } + + public string Code { get; set; } = string.Empty; + + public int ZxShopperId { get; set; } + + public ZxShopper? Shopper { get; set; } + } + + /// An application helper whose result is only a flag, and which hands the row its cards on the way. + public static class ZxHydrator + { + public static bool Attach(ZxShopper shopper, List cards) + { + shopper.Cards = cards; + + return true; + } + } + + /// The same, done by a constructor whose object is only read for a flag. + public sealed class ZxAttachment + { + public ZxAttachment(ZxShopper shopper, List cards) => shopper.Cards = cards; + + public bool Done => true; + } + + public class ZxMaskedNote + { + public int Id { get; set; } + + [DwMask(MaskStrategy.Full)] + public string Body { get; set; } = string.Empty; + } + + public sealed class ZxShopContext : DbContext + { + private readonly SqliteConnection _connection; + + public ZxShopContext(SqliteConnection connection) => _connection = connection; + + public DbSet Shoppers => Set(); + + public DbSet Cards => Set(); + + public DbSet Purchases => Set(); + + public DbSet Notes => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + } + + /// + /// A queryable that forwards to another provider, as LinqKit's AsExpandable, DelegateDecompiler's Decompile and + /// similar wrappers do over an EF Core query. Its provider is not EF Core's, so EF Core's AsNoTracking leaves it + /// unchanged. + /// + public sealed class ZxForwardingQuery : IOrderedQueryable + { + private readonly IQueryable _inner; + + public ZxForwardingQuery(IQueryable inner) + { + _inner = inner; + Provider = new ZxForwardingProvider(inner.Provider); + } + + public Type ElementType => typeof(T); + + public Expression Expression => _inner.Expression; + + public IQueryProvider Provider { get; } + + public IEnumerator GetEnumerator() => _inner.GetEnumerator(); + + IEnumerator IEnumerable.GetEnumerator() => GetEnumerator(); + } + + public sealed class ZxForwardingProvider : IQueryProvider + { + private readonly IQueryProvider _inner; + + public ZxForwardingProvider(IQueryProvider inner) => _inner = inner; + + public IQueryable CreateQuery(Expression expression) + { + IQueryable created = _inner.CreateQuery(expression); + + return (IQueryable)Activator.CreateInstance(typeof(ZxForwardingQuery<>).MakeGenericType(created.ElementType), created)!; + } + + public IQueryable CreateQuery(Expression expression) => + new ZxForwardingQuery(_inner.CreateQuery(expression)); + + public object? Execute(Expression expression) => _inner.Execute(expression); + + public TResult Execute(Expression expression) => _inner.Execute(expression); + } + + // ============================================================================ reshaped chains and tracking: tests + + public sealed class ReviewTrackingTests : IDisposable + { + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZxShopContext _db; + + public ReviewTrackingTests(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + _db = new ZxShopContext(_connection); + _db.Database.EnsureCreated(); + + ZxShopper shopper = new() { Name = "S1", Cards = { new ZxPayCard { Label = "visa", Pan = "zx-pan-secret" } } }; + + _db.Shoppers.Add(shopper); + _db.Purchases.Add(new ZxPurchase { Code = "P1", Shopper = shopper }); + _db.Notes.Add(new ZxMaskedNote { Body = "original-note-body" }); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private (string Unguarded, object? Guarded) Run(string label, IQueryable source, DwTier tier, bool dynamic = false) where T : class + { + string unguarded; + + try + { + unguarded = ZxKit.Holds(source.AsNoTracking().ToList(), "zx-pan-secret") ? "HOLDS PAN" : "no pan"; + } + catch (Exception e) + { + unguarded = $"THREW {e.GetType().Name}: {e.Message.Split('\n')[0]}"; + } + + PolicyQueryable guarded = ZxKit.Guard(source, tier); + object? data; + + try + { + data = dynamic ? guarded.ToListDynamic(new Filter()).Data : guarded.ToList(new Filter()).Data; + } + catch (Exception e) + { + data = null; + _out.WriteLine($"{label}: guarded threw {e.GetType().Name}: {e.Message.Split('\n')[0]}"); + } + + RowShape shape = RowShape.Of(source); + + _out.WriteLine($"{label} {tier}: shape={shape.Kind} materializes(Cards.Pan)={shape.Materializes("Cards.Pan")} " + + $"reshapes={QueryRoot.Reshapes(source.Expression)} builds={QueryRoot.Builds(source.Expression)}"); + _out.WriteLine($"{label} {tier}: unguarded={unguarded}"); + _out.WriteLine($"{label} {tier}: guarded={ZxKit.Json(data)} trace=[{ZxKit.Trace(guarded)}]"); + + return (unguarded, data); + } + + // ------------------------------------------------------------------------ B1: a flag-typed subtree that hands the row its cards + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_B1_an_application_method_read_only_for_a_flag_in_a_conditional(DwTier tier) + { + IQueryable source = _db.Purchases + .Select(p => ZxHydrator.Attach(p.Shopper!, p.Shopper!.Cards.ToList()) ? p.Shopper! : p.Shopper!); + + (string unguarded, object? guarded) = Run("B1", source, tier); + + if (!unguarded.StartsWith("HOLDS", StringComparison.Ordinal)) + { + return; + } + + Assert.False(ZxKit.Holds(guarded, "zx-pan-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_B2_a_constructor_read_only_for_a_flag_in_a_conditional(DwTier tier) + { + IQueryable source = _db.Purchases + .Select(p => new ZxAttachment(p.Shopper!, p.Shopper!.Cards.ToList()).Done ? p.Shopper! : p.Shopper!); + + (string unguarded, object? guarded) = Run("B2", source, tier); + + if (!unguarded.StartsWith("HOLDS", StringComparison.Ordinal)) + { + return; + } + + Assert.False(ZxKit.Holds(guarded, "zx-pan-secret")); + } + + // ------------------------------------------------------------------------ T: a provider that wraps EF Core runs a tracking query + + [Theory] + [InlineData(DwTier.Strict, false)] + [InlineData(DwTier.Convenience, false)] + [InlineData(DwTier.Strict, true)] + public void Zx_T1_a_wrapped_query_over_a_context_that_already_tracks_the_cards(DwTier tier, bool dynamic) + { + // Earlier in the same unit of work, the application read the cards with tracking. + _ = _db.Cards.ToList(); + + IQueryable source = new ZxForwardingQuery(_db.Shoppers); + + (_, object? guarded) = Run("T1", source, tier, dynamic); + + Assert.False(ZxKit.Holds(guarded, "zx-pan-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_T2_a_wrapped_query_over_a_context_that_already_tracks_the_shopper_with_its_cards(DwTier tier) + { + _ = _db.Shoppers.Include(s => s.Cards).ToList(); + + IQueryable source = new ZxForwardingQuery(_db.Shoppers); + + (_, object? guarded) = Run("T2", source, tier); + + Assert.False(ZxKit.Holds(guarded, "zx-pan-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_T3_control_the_same_query_unwrapped(DwTier tier) + { + _ = _db.Cards.ToList(); + + (_, object? guarded) = Run("T3", _db.Shoppers, tier); + + Assert.False(ZxKit.Holds(guarded, "zx-pan-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_T4_a_wrapped_reshaped_query_over_tracked_cards(DwTier tier) + { + _ = _db.Cards.ToList(); + + IQueryable source = new ZxForwardingQuery(_db.Purchases).Select(p => p.Shopper!); + + (_, object? guarded) = Run("T4", source, tier); + + Assert.False(ZxKit.Holds(guarded, "zx-pan-secret")); + } + + /// + /// Guarded() detaches the query so that a transform applied to a materialized entity is never written back. + /// Through a wrapping provider the query tracks, and the masked value is the entity's pending state. + /// + [Fact] + public void Zx_T5_a_wrapped_query_leaves_a_masked_value_as_a_pending_change() + { + IQueryable source = new ZxForwardingQuery(_db.Notes); + + object? data = ZxKit.Guard(source).ToList(new Filter()).Data; + + _out.WriteLine($"sent={ZxKit.Json(data)} tracked={_db.ChangeTracker.Entries().Count()} hasChanges={_db.ChangeTracker.HasChanges()}"); + + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + + string stored = _db.Notes.AsNoTracking().Single().Body; + + _out.WriteLine($"stored after SaveChanges: {stored}"); + + Assert.Equal("original-note-body", stored); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewVarianceTests.cs b/DynamicWhere.Tests/Policies/ReviewVarianceTests.cs new file mode 100644 index 0000000..3ca4eca --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewVarianceTests.cs @@ -0,0 +1,356 @@ +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Classes.Core; +using DynamicWhere.ex.Enums; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Source; +using Xunit.Abstractions; + +namespace DynamicWhere.Tests.Policies +{ + // ============================================================================ variant generic interfaces: models + + public class ZxBaseCard + { + public string Label { get; set; } = string.Empty; + } + + public class ZxDerivedCard : ZxBaseCard + { + public string Tier { get; set; } = string.Empty; + } + + // ---- covariant, implemented only by an open generic class over a closed subtype argument ------------------- + + /// A covariant interface: an IZxOpenVar<ZxDerivedCard> is an IZxOpenVar<ZxBaseCard>. + public interface IZxOpenVar + { + T Value { get; } + + string? Code { get; } + } + + /// Every instantiation of this class is an IZxOpenVar<ZxBaseCard> through covariance. + public class ZxOpenVarImpl : IZxOpenVar + { + public ZxDerivedCard Value { get; set; } = new(); + + [DwDenied] + public string? Code { get; set; } + + [DwDenied] + public string? Secret { get; set; } + } + + public class ZxOpenVarRow + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public IZxOpenVar? Tagged { get; set; } + } + + // ---- covariant, implemented only by a CLOSED class over a subtype argument ---- + + public interface IZxClosedVar + { + T Value { get; } + + string? Code { get; } + } + + public class ZxClosedVarImpl : IZxClosedVar + { + public ZxDerivedCard Value { get; set; } = new(); + + [DwDenied] + public string? Code { get; set; } + + [DwDenied] + public string? Secret { get; set; } + } + + public class ZxClosedVarRow + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public IZxClosedVar? Tagged { get; set; } + } + + // ---- contravariant, implemented by a closed class over a BASE argument ---- + + public interface IZxSink + { + string? Code { get; } + + void Accept(T item); + } + + public class ZxBaseSink : IZxSink + { + [DwDenied] + public string? Code { get; set; } + + public void Accept(ZxBaseCard item) + { + } + } + + public class ZxSinkRow + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public IZxSink? Sink { get; set; } + } + + // ---- controls: the member typed as the exact instantiation implemented ---- + + public class ZxExactOpenRow + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public IZxOpenVar? Tagged { get; set; } + } + + // ============================================================================ variant generic interfaces: tests + + /// + /// A member typed as a variant interface holds implementations of other instantiations, which the subtype index + /// and the walker read as the runtime assigns them, covariant and contravariant alike. + /// + public sealed class ReviewVarianceTests + { + private readonly ITestOutputHelper _out; + + public ReviewVarianceTests(ITestOutputHelper output) => _out = output; + + private static Summary GroupedBy(string field) => new() + { + GroupBy = new GroupBy + { + Fields = new List { field }, + AggregateBy = new List { new() { Alias = "Total", Aggregator = Aggregator.Count } } + } + }; + + private void Report(string label, Type type) + { + _out.WriteLine($"{label}: Of({type.Name}<{type.GetGenericArguments()[0].Name}>) = " + + string.Join(", ", KnownSubtypes.Of(type).Select(t => t.Name))); + } + + [Fact] + public void Zx_V0_the_types_are_assignable_through_variance() + { + Assert.True(typeof(IZxOpenVar).IsAssignableFrom(typeof(ZxOpenVarImpl))); + Assert.True(typeof(IZxClosedVar).IsAssignableFrom(typeof(ZxClosedVarImpl))); + Assert.True(typeof(IZxSink).IsAssignableFrom(typeof(ZxBaseSink))); + + Exception? map = Record.Exception(() => typeof(ZxOpenVarImpl<>).GetInterfaceMap(typeof(IZxOpenVar))); + Exception? closedMap = Record.Exception(() => typeof(ZxClosedVarImpl).GetInterfaceMap(typeof(IZxClosedVar))); + + _out.WriteLine($"GetInterfaceMap(ZxOpenVarImpl<>, IZxOpenVar): {map?.GetType().Name ?? "ok"}"); + _out.WriteLine($"GetInterfaceMap(ZxClosedVarImpl, IZxClosedVar): {closedMap?.GetType().Name ?? "ok"}"); + } + + // ------------------------------------------------------------------------ the subtype index + + [Fact] + public void Zx_V1_open_the_index_lists_an_open_generic_implementation_over_a_subtype_argument() + { + Report("open", typeof(IZxOpenVar)); + + Assert.Contains(typeof(ZxOpenVarImpl<>), KnownSubtypes.Of(typeof(IZxOpenVar))); + } + + [Fact] + public void Zx_V1_closed_the_index_lists_a_closed_implementation_over_a_subtype_argument() + { + Report("closed", typeof(IZxClosedVar)); + + Assert.Contains(typeof(ZxClosedVarImpl), KnownSubtypes.Of(typeof(IZxClosedVar))); + } + + [Fact] + public void Zx_V1_contra_the_index_lists_a_contravariant_implementation() + { + Report("contra", typeof(IZxSink)); + + Assert.Contains(typeof(ZxBaseSink), KnownSubtypes.Of(typeof(IZxSink))); + } + + // ------------------------------------------------------------------------ the walker's fragment on the path + + [Fact] + public void Zx_V2_control_the_exact_instantiation_carries_the_implementations_denial() + { + Assert.NotEmpty(ZxKit.Denials(typeof(ZxExactOpenRow), "Tagged.Code")); + } + + [Fact] + public void Zx_V2_open_the_covariant_path_carries_the_implementations_denial() + { + _out.WriteLine("open Tagged.Code denials: " + ZxKit.Denials(typeof(ZxOpenVarRow), "Tagged.Code").Count()); + + Assert.NotEmpty(ZxKit.Denials(typeof(ZxOpenVarRow), "Tagged.Code")); + } + + [Fact] + public void Zx_V2_closed_the_covariant_path_carries_the_implementations_denial() + { + _out.WriteLine("closed Tagged.Code denials: " + ZxKit.Denials(typeof(ZxClosedVarRow), "Tagged.Code").Count()); + + Assert.NotEmpty(ZxKit.Denials(typeof(ZxClosedVarRow), "Tagged.Code")); + } + + [Fact] + public void Zx_V2_contra_the_contravariant_path_carries_the_implementations_denial() + { + _out.WriteLine("contra Sink.Code denials: " + ZxKit.Denials(typeof(ZxSinkRow), "Sink.Code").Count()); + + Assert.NotEmpty(ZxKit.Denials(typeof(ZxSinkRow), "Sink.Code")); + } + + // ------------------------------------------------------------------------ where, group, order on the denied path + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_V3_open_where_group_and_order_on_the_denied_code_are_refused(DwTier tier) + { + ZxOpenVarRow[] rows = { new() { Id = 1, Name = "r1", Tagged = new ZxOpenVarImpl { Code = "open-code-secret" } } }; + + string where = ZxKit.Code(() => ZxKit.Guard(rows.AsQueryable(), tier).ToList(ZxKit.Where("Tagged.Code", "open-code-secret"))); + object? grouped = null; + string group = ZxKit.Code(() => grouped = ZxKit.Guard(rows.AsQueryable(), tier).ToList(GroupedBy("Tagged.Code")).Data); + string order = ZxKit.Code(() => ZxKit.Guard(rows.AsQueryable(), tier).ToList(new Filter + { + Orders = new List { new() { Field = "Tagged.Code" } }, Selects = new List { "Id" } + })); + + _out.WriteLine($"open {tier}: where={where} group={group} ({ZxKit.Json(grouped)}) order={order}"); + + Assert.NotEqual("ran", where); + Assert.NotEqual("ran", group); + + if (tier == DwTier.Strict) + { + Assert.NotEqual("ran", order); + } + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_V3_closed_where_group_and_order_on_the_denied_code_are_refused(DwTier tier) + { + ZxClosedVarRow[] rows = { new() { Id = 1, Name = "r1", Tagged = new ZxClosedVarImpl { Code = "closed-code-secret" } } }; + + string where = ZxKit.Code(() => ZxKit.Guard(rows.AsQueryable(), tier).ToList(ZxKit.Where("Tagged.Code", "closed-code-secret"))); + object? grouped = null; + string group = ZxKit.Code(() => grouped = ZxKit.Guard(rows.AsQueryable(), tier).ToList(GroupedBy("Tagged.Code")).Data); + + _out.WriteLine($"closed {tier}: where={where} group={group} ({ZxKit.Json(grouped)})"); + + Assert.NotEqual("ran", where); + Assert.NotEqual("ran", group); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_V3_contra_where_on_the_denied_code_is_refused(DwTier tier) + { + ZxSinkRow[] rows = { new() { Id = 1, Name = "r1", Sink = new ZxBaseSink { Code = "contra-code-secret" } } }; + + string where = ZxKit.Code(() => ZxKit.Guard(rows.AsQueryable(), tier).ToList(ZxKit.Where("Sink.Code", "contra-code-secret"))); + + _out.WriteLine($"contra {tier}: where={where}"); + + Assert.NotEqual("ran", where); + } + + // ------------------------------------------------------------------------ rows returned with no projection asked for + + [Theory] + [InlineData(DwTier.Strict, false)] + [InlineData(DwTier.Convenience, false)] + [InlineData(DwTier.Strict, true)] + public void Zx_V4_open_rows_do_not_return_the_implementations_denied_members(DwTier tier, bool dynamic) + { + ZxOpenVarRow[] rows = + { + new() { Id = 1, Name = "r1", Tagged = new ZxOpenVarImpl { Code = "open-code-secret", Secret = "open-own-secret" } } + }; + + PolicyQueryable guarded = ZxKit.Guard(rows.AsQueryable(), tier); + object? data = ZxKit.SafeRead(() => dynamic ? guarded.ToListDynamic(new Filter()).Data : guarded.ToList(new Filter()).Data); + + _out.WriteLine($"open {tier} dynamic={dynamic}: sent={ZxKit.Json(data)} trace=[{ZxKit.Trace(guarded)}]"); + + Assert.False(ZxKit.Holds(data, "open-code-secret")); + Assert.False(ZxKit.Holds(data, "open-own-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_V4_closed_rows_do_not_return_the_implementations_denied_members(DwTier tier) + { + ZxClosedVarRow[] rows = + { + new() { Id = 1, Name = "r1", Tagged = new ZxClosedVarImpl { Code = "closed-code-secret", Secret = "closed-own-secret" } } + }; + + PolicyQueryable guarded = ZxKit.Guard(rows.AsQueryable(), tier); + object? data = ZxKit.SafeRead(() => guarded.ToList(new Filter()).Data); + + _out.WriteLine($"closed {tier}: sent={ZxKit.Json(data)} trace=[{ZxKit.Trace(guarded)}]"); + + Assert.False(ZxKit.Holds(data, "closed-code-secret")); + Assert.False(ZxKit.Holds(data, "closed-own-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_V4_contra_rows_do_not_return_the_implementations_denied_members(DwTier tier) + { + ZxSinkRow[] rows = { new() { Id = 1, Name = "r1", Sink = new ZxBaseSink { Code = "contra-code-secret" } } }; + + PolicyQueryable guarded = ZxKit.Guard(rows.AsQueryable(), tier); + object? data = ZxKit.SafeRead(() => guarded.ToList(new Filter()).Data); + + _out.WriteLine($"contra {tier}: sent={ZxKit.Json(data)} trace=[{ZxKit.Trace(guarded)}]"); + + Assert.False(ZxKit.Holds(data, "contra-code-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_V4_control_rows_typed_as_the_exact_instantiation(DwTier tier) + { + ZxExactOpenRow[] rows = + { + new() { Id = 1, Name = "r1", Tagged = new ZxOpenVarImpl { Code = "exact-code-secret", Secret = "exact-own-secret" } } + }; + + PolicyQueryable guarded = ZxKit.Guard(rows.AsQueryable(), tier); + object? data = ZxKit.SafeRead(() => guarded.ToList(new Filter()).Data); + + _out.WriteLine($"exact {tier}: sent={ZxKit.Json(data)} trace=[{ZxKit.Trace(guarded)}]"); + + Assert.False(ZxKit.Holds(data, "exact-code-secret")); + Assert.False(ZxKit.Holds(data, "exact-own-secret")); + } + } +} diff --git a/DynamicWhere.Tests/ReviewComplexPropertyTests.cs b/DynamicWhere.Tests/ReviewComplexPropertyTests.cs new file mode 100644 index 0000000..7f6f521 --- /dev/null +++ b/DynamicWhere.Tests/ReviewComplexPropertyTests.cs @@ -0,0 +1,199 @@ +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Source; +using DynamicWhere.Tests.Policies; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Xunit.Abstractions; + +namespace DynamicWhere.Tests +{ + // EF Core 8 only (complex types, JSON columns): kept at the test project's root so the EF Core 6 floor leg, + // which compiles Policies/** only, leaves it out. + + // ============================================================================ a converted member inside a complex or JSON type: models + + public class ZxComplexMeta + { + public string Tag { get; set; } = string.Empty; + + public object? Payload { get; set; } + } + + public class ZxComplexEvent + { + public int Id { get; set; } + + public string Kind { get; set; } = string.Empty; + + [DwDenied] + public string? Actor { get; set; } + + public ZxComplexMeta Meta { get; set; } = new(); + } + + public class ZxJsonMeta + { + public string Tag { get; set; } = string.Empty; + + public object? Payload { get; set; } + } + + public class ZxJsonEvent + { + public int Id { get; set; } + + public string Kind { get; set; } = string.Empty; + + [DwDenied] + public string? Actor { get; set; } + + public ZxJsonMeta? Meta { get; set; } + } + + public sealed class ZxComplexContext : DbContext + { + private readonly SqliteConnection _connection; + + public ZxComplexContext(SqliteConnection connection) => _connection = connection; + + public DbSet ComplexEvents => Set(); + + public DbSet JsonEvents => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + protected override void OnModelCreating(ModelBuilder model) + { + model.Entity().ComplexProperty(e => e.Meta, meta => meta.Property(m => m.Payload).HasConversion(new ZxPayloadConverter())); + model.Entity().OwnsOne(e => e.Meta, meta => + { + meta.ToJson(); + meta.Property(m => m.Payload).HasConversion(new ZxPayloadConverter()); + }); + } + } + + // ============================================================================ a converted member inside a complex or JSON type: tests + + /// + /// RowShape.Converted asks the entity type for the column. A complex property is read whole as its CLR type, and + /// the converter on the member inside it is never asked about. + /// + public sealed class ReviewComplexPropertyTests : IDisposable + { + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZxComplexContext _db; + + public ReviewComplexPropertyTests(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + _db = new ZxComplexContext(_connection); + _db.Database.EnsureCreated(); + _db.ComplexEvents.Add(new ZxComplexEvent + { + Kind = "CardIssued", Actor = "complex-actor-secret", + Meta = new ZxComplexMeta { Tag = "t", Payload = new ZxIssued { Label = "visa", Pan = "complex-pan-secret" } } + }); + _db.JsonEvents.Add(new ZxJsonEvent + { + Kind = "CardIssued", Actor = "json-actor-secret", + Meta = new ZxJsonMeta { Tag = "t", Payload = new ZxIssued { Label = "visa", Pan = "json-pan-secret" } } + }); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_C2_a_converted_object_member_inside_a_complex_property_beside_a_top_level_denial(DwTier tier) + { + Assert.True(ZxKit.Holds(_db.ComplexEvents.AsNoTracking().ToList(), "complex-pan-secret")); + + PolicyQueryable guarded = ZxKit.Guard(_db.ComplexEvents, tier); + object? data = ZxKit.SafeRead(() => guarded.ToList(new Filter()).Data); + + _out.WriteLine($"{tier}: sent={ZxKit.Json(data)} trace=[{ZxKit.Trace(guarded)}]"); + + Assert.False(ZxKit.Holds(data, "complex-actor-secret")); + Assert.False(ZxKit.Holds(data, "complex-pan-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_C2_dynamic_a_converted_object_member_inside_a_complex_property(DwTier tier) + { + PolicyQueryable guarded = ZxKit.Guard(_db.ComplexEvents, tier); + object? data = ZxKit.SafeRead(() => guarded.ToListDynamic(new Filter()).Data); + + _out.WriteLine($"{tier}: sent={ZxKit.Json(data)} trace=[{ZxKit.Trace(guarded)}]"); + + Assert.False(ZxKit.Holds(data, "complex-pan-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zx_C3_a_converted_object_member_inside_a_json_owned_type_beside_a_top_level_denial(DwTier tier) + { + List raw; + + try + { + raw = _db.JsonEvents.AsNoTracking().ToList(); + } + catch (Exception e) + { + _out.WriteLine($"unguarded threw: {e.GetType().Name}: {e.Message.Split('\n')[0]}"); + + return; + } + + _out.WriteLine("unguarded holds pan: " + ZxKit.Holds(raw, "json-pan-secret")); + + PolicyQueryable guarded = ZxKit.Guard(_db.JsonEvents, tier); + object? data = ZxKit.SafeRead(() => guarded.ToList(new Filter()).Data); + + _out.WriteLine($"{tier}: sent={ZxKit.Json(data)} trace=[{ZxKit.Trace(guarded)}]"); + + Assert.False(ZxKit.Holds(data, "json-actor-secret")); + Assert.False(ZxKit.Holds(data, "json-pan-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public async Task Zx_C2_segment_a_converted_object_member_inside_a_complex_property(DwTier tier) + { + PolicyQueryable guarded = ZxKit.Guard(_db.ComplexEvents, tier); + object? data; + + try + { + data = (await guarded.ToListAsync(new Segment())).Data; + } + catch (Exception e) + { + _out.WriteLine($"{tier}: segment threw {e.GetType().Name}: {e.Message.Split('\n')[0]}"); + + return; + } + + _out.WriteLine($"{tier}: segment sent={ZxKit.Json(data)} trace=[{ZxKit.Trace(guarded)}]"); + + Assert.False(ZxKit.Holds(data, "complex-pan-secret")); + } + } +} diff --git a/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs b/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs index 57f5288..0051ac8 100644 --- a/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs +++ b/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs @@ -311,18 +311,19 @@ private static IEnumerable ReadDenialsElsewhere(Type type, Prop } /// - /// What a subtype declares in a member's place: an override of either accessor, or a member of the same - /// name that hides it. Both carry the member's name. + /// What a subtype declares in a member's place: an override of either accessor, or a public member of the + /// same name that hides it. Both carry the member's name. One the subtype keeps to itself hides nothing a + /// caller reads, and no serializer writes it. /// private static IEnumerable Redeclarations(Type subtype, PropertyInfo property) => subtype - .GetProperties(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.DeclaredOnly) + .GetProperties(BindingFlags.Public | BindingFlags.Instance | BindingFlags.DeclaredOnly) .Where(candidate => candidate.Name == property.Name); /// - /// The properties a class implements an interface's member with. A class that is itself open generic - /// implements the interface over its own type parameters, which may be any instantiation, so that one - /// is read too. + /// The properties a class implements an interface's member with: through the interface itself, through an + /// instantiation variance lets stand for it, IFeed<VisaCard> for IFeed<Card> with out T, and, for + /// a class that is itself open generic, through one over its own type parameters, which may be any. /// private static IEnumerable Implementations(Type row, Type contract, MethodInfo[] accessors) { @@ -332,8 +333,9 @@ private static IEnumerable Implementations(Type row, Type contract { bool same = implemented == contract || (implemented.IsGenericType && contract.IsGenericType - && (implemented.ContainsGenericParameters || contract.ContainsGenericParameters) - && implemented.GetGenericTypeDefinition() == contract.GetGenericTypeDefinition()); + && implemented.GetGenericTypeDefinition() == contract.GetGenericTypeDefinition() + && (implemented.ContainsGenericParameters || contract.ContainsGenericParameters + || contract.IsAssignableFrom(implemented))); if (!same || Map(row, implemented) is not { } map) { diff --git a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs index 258cb98..ee83e2a 100644 --- a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs +++ b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs @@ -1910,6 +1910,19 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) continue; } + // Two members share the name, one hidden with new under another type or spelled in another case: + // the core reads one of them, and a row carries both. What either can hold is asked about, and a + // projection, which cannot tell them apart, leaves the name out. + if (gate.SharedName(name) is { Shared: true } shared + && (shared.Denies || gate.Beneath(name, PolicyFeature.None).Denied.Any(d => rows.Materializes(d.Path)))) + { + anyDenied = true; + + gate.LeaveOut(name, "left out: the type declares more than one member of this name, which a projection cannot tell apart"); + + continue; + } + // A projection assigns only a member with a setter, and no path can name a member the // expression parser keeps a word for; such a member is still asked about below. bool assignable = member.CanWrite && !ReservedNames.Starts(name); @@ -1978,7 +1991,9 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) ? "left out whole: a scope forced beneath it cannot be applied to what it holds" : rows.Narrows(name) ? Narrowed(name, survivors, gate, rows, allowed) - : "left out whole: " + rows.WhyNotNarrowed(name); + : reached.Count == 0 && !unnamed.DeniesSelect && unnamed.HoldsObject + ? "left out whole: it can hold what the policy cannot name" + : "left out whole: " + rows.WhyNotNarrowed(name); if (reason is not null) { @@ -2724,7 +2739,65 @@ internal IEnumerable Members() /// carry a policy, and on an entity it is a column the unguarded call loads. /// internal static bool HoldsValue(Type type) => - CacheReflection.IsSimpleType(AttributePolicyProvider.Peeled(type)); + CacheReflection.IsSimpleType(AttributePolicyProvider.Peeled(type)) + && !OwnsMembers(Nullable.GetUnderlyingType(type) ?? type); + + /// + /// True for an application's own collection class, one deriving from List<string> or a + /// Dictionary say, that declares members beside the elements it holds. + /// + private static bool OwnsMembers(Type type) => + type.IsClass + && !type.IsArray + && !AttributePolicyProvider.IsFramework(type) + && AttributePolicyProvider.Peeled(type) != type + && type.GetProperties(BindingFlags.Public | BindingFlags.Instance) + .Any(property => property.GetIndexParameters().Length == 0 + && property.DeclaringType is { } declaring + && !AttributePolicyProvider.IsFramework(declaring)); + + /// The names more than one readable member of a type shares, compared as the core compares names. + private static readonly ConcurrentDictionary> SharedNames = new(); + + private static HashSet ReadSharedNames(Type type) => + new(type.GetProperties(BindingFlags.Public | BindingFlags.Instance) + .Where(property => property.CanRead && property.GetIndexParameters().Length == 0) + .GroupBy(property => property.Name, StringComparer.OrdinalIgnoreCase) + .Where(group => group.Count() > 1) + .Select(group => group.Key), + StringComparer.OrdinalIgnoreCase); + + /// + /// Whether T declares more than one member of a name, as the core compares names, one of them holding + /// more than a value; and if so, whether what one of them holds is denied. + /// + /// + /// A member hidden with new under another type, or two names differing only in case. The core + /// reads one of them by name, and a row carries every one, which a serializer writes. + /// + internal (bool Shared, bool Denies) SharedName(string name) + { + if (!SharedNames.GetOrAdd(_entityType, ReadSharedNames).Contains(name)) + { + return (false, false); + } + + List variants = _entityType + .GetProperties(BindingFlags.Public | BindingFlags.Instance) + .Where(property => property.CanRead + && property.GetIndexParameters().Length == 0 + && string.Equals(property.Name, name, StringComparison.OrdinalIgnoreCase)) + .ToList(); + + if (variants.TrueForAll(variant => HoldsValue(variant.PropertyType))) + { + return (false, false); + } + + return (true, variants.Exists(variant => Denies(name, variant) + || (!HoldsValue(variant.PropertyType) + && Carried(variant.PropertyType, name, exact: false).DeniesSelect))); + } /// /// True when a validated path names a navigation rather than a scalar. @@ -3326,6 +3399,20 @@ void Subtypes(Type of) void Enqueue(Type candidate, bool root) { Type peeled = AttributePolicyProvider.Peeled(candidate); + Type unwrapped = Nullable.GetUnderlyingType(candidate) ?? candidate; + + // An application's own collection class declares members beside the elements it holds. + if (OwnsMembers(unwrapped)) + { + holdsMembers = true; + + Read(unwrapped); + + if (!(root && exact)) + { + Subtypes(unwrapped); + } + } if (HoldsAnything(peeled)) { @@ -3401,7 +3488,17 @@ private static bool HoldsAnything(Type peeled) => || peeled.IsGenericParameter || (!peeled.IsGenericType && ((AttributePolicyProvider.IsFramework(peeled) && peeled.IsInterface) - || (peeled != typeof(string) && typeof(IEnumerable).IsAssignableFrom(peeled)))); + || (peeled != typeof(string) && typeof(IEnumerable).IsAssignableFrom(peeled) + && !ValueCollections.Contains(peeled)))); + + /// The framework's collections that are not generic yet hold values only: bits and strings. + private static readonly HashSet ValueCollections = new() + { + typeof(BitArray), + typeof(System.Collections.Specialized.StringCollection), + typeof(System.Collections.Specialized.StringDictionary), + typeof(System.Collections.Specialized.NameValueCollection) + }; /// /// What a type can hold that the policy cannot name a path to. diff --git a/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs b/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs index 000f22c..4b111e1 100644 --- a/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs +++ b/DynamicWhere.ex/Policies/Source/KnownSubtypes.cs @@ -84,15 +84,17 @@ internal static IReadOnlyList Of(Type type) } /// - /// True when some instantiation of an open generic subtype derives from or implements a closed type: - /// its own base or interface of that definition is the type, or is still open. One over another + /// True when a type indexed under a generic type's definition can be a value of one instantiation of it. + /// A closed type can when the runtime says so, which reads variance: a member typed + /// IFeed<Card>, with out T, holds an IFeed<VisaCard>. An open generic one can when its own + /// base or interface of that definition is still open, or is one of those. One over another /// instantiation, class Fixed<T> : Base<string>, never holds a Base<int>. /// private static bool CanBe(Type candidate, Type type) { if (!candidate.ContainsGenericParameters) { - return false; + return type.IsAssignableFrom(candidate); } Type definition = type.GetGenericTypeDefinition(); @@ -100,7 +102,7 @@ private static bool CanBe(Type candidate, Type type) return ancestors.Any(ancestor => ancestor.IsGenericType && ancestor.GetGenericTypeDefinition() == definition - && (ancestor == type || ancestor.ContainsGenericParameters)); + && (ancestor.ContainsGenericParameters || type.IsAssignableFrom(ancestor))); } private static IEnumerable BaseTypes(Type type) diff --git a/DynamicWhere.ex/Policies/Source/PolicyQueryable.cs b/DynamicWhere.ex/Policies/Source/PolicyQueryable.cs index 68156be..fb617ea 100644 --- a/DynamicWhere.ex/Policies/Source/PolicyQueryable.cs +++ b/DynamicWhere.ex/Policies/Source/PolicyQueryable.cs @@ -11,6 +11,8 @@ using DynamicWhere.ex.Source; using Microsoft.EntityFrameworkCore; using System.Linq.Dynamic.Core; +using System.Linq.Expressions; +using System.Reflection; namespace DynamicWhere.ex.Policies.Source; @@ -733,10 +735,30 @@ static bool IsGroupSize(AggregateBy aggregate) => /// point every guarded query passes through, so no individual method can forget it. /// /// Harmless on a non-EF source: the EF extension returns the query unchanged when the provider - /// is not one of its own. + /// is not one of its own. A provider that wraps EF Core's, as LinqKit's AsExpandable and + /// DelegateDecompiler's Decompile do, is not one of its own either, and passes the query on to + /// EF Core with tracking still on: the tracked entities' navigations were then filled in on the rows, + /// and a masked value became a pending change. On such a query the call goes into the query itself, + /// where EF Core reads it. /// /// - private IQueryable Guarded() => _source.AsNoTracking(); + private IQueryable Guarded() + { + IQueryable untracked = _source.AsNoTracking(); + + if (!ReferenceEquals(untracked, _source) || QueryRoot.Model(_source.Expression) is null) + { + return untracked; + } + + return _source.Provider.CreateQuery( + Expression.Call(null, AsNoTrackingMethod.MakeGenericMethod(typeof(T)), _source.Expression)); + } + + private static readonly MethodInfo AsNoTrackingMethod = typeof(EntityFrameworkQueryableExtensions) + .GetMethods() + .Single(method => method.Name == nameof(EntityFrameworkQueryableExtensions.AsNoTracking) + && method.GetParameters().Length == 1); /// /// Returns the query as a plain , outside the guard. diff --git a/DynamicWhere.ex/Policies/Source/RowShape.cs b/DynamicWhere.ex/Policies/Source/RowShape.cs index 5170ebe..131f939 100644 --- a/DynamicWhere.ex/Policies/Source/RowShape.cs +++ b/DynamicWhere.ex/Policies/Source/RowShape.cs @@ -408,9 +408,63 @@ private bool Collect( /// /// True when a column's value comes from a value converter, which is the application's code and can hand - /// back an object of any type its member's type allows. + /// back an object of any type its member's type allows; for a complex property, when a member of it does. /// - private static bool Converted(IEntityType type, string name) => type.FindProperty(name)?.GetValueConverter() is not null; + private static bool Converted(IEntityType type, string name) => + type.FindProperty(name)?.GetValueConverter() is not null || ConvertedComplex(type).Contains(name); + + /// The complex properties of an entity type with a converted member at any depth. + private static HashSet ConvertedComplex(IEntityType entityType) => + ConvertedComplexByType.GetValue(entityType, ReadConvertedComplex); + + private static readonly ConditionalWeakTable> ConvertedComplexByType = new(); + + /// + /// Reads, through reflection since EF Core 6 has none, the complex properties whose type holds a converted + /// property. One that cannot be read counts as converted. + /// + private static HashSet ReadConvertedComplex(IEntityType entityType) + { + HashSet names = new(StringComparer.OrdinalIgnoreCase); + + foreach (object complex in Items(entityType, "GetComplexProperties")) + { + if (complex.GetType().GetProperty("Name")?.GetValue(complex) is string name && HoldsConverted(complex, depth: 0)) + { + names.Add(name); + } + } + + return names; + } + + private static bool HoldsConverted(object complex, int depth) + { + if (depth > MaxComplexDepth || complex.GetType().GetProperty("ComplexType")?.GetValue(complex) is not { } type) + { + return true; + } + + return Items(type, "GetProperties").Any(property => property is not IReadOnlyProperty column || column.GetValueConverter() is not null) + || Items(type, "GetComplexProperties").Any(nested => HoldsConverted(nested, depth + 1)); + } + + /// How deep complex properties are read inside one another before the rest counts as converted. + private const int MaxComplexDepth = 8; + + /// What a model object's parameterless method of the name returns, for a method some interface of it declares. + private static IEnumerable Items(object model, string method) + { + foreach (Type contract in model.GetType().GetInterfaces()) + { + if (contract.GetMethod(method, Type.EmptyTypes) is { } found && found.Invoke(model, null) is IEnumerable items) + { + return items.Cast().ToList(); + } + } + + return Array.Empty(); + } /// True when something loads a navigation of an entity reached along a path. private bool Loads(IEntityType owner, INavigationBase navigation, string path) => diff --git a/DynamicWhere.ex/Source/QueryRoot.cs b/DynamicWhere.ex/Source/QueryRoot.cs index d326d85..f8bb6ce 100644 --- a/DynamicWhere.ex/Source/QueryRoot.cs +++ b/DynamicWhere.ex/Source/QueryRoot.cs @@ -181,6 +181,13 @@ private sealed class ForeignFinder : ExpressionVisitor protected override Expression VisitNew(NewExpression node) { + // An anonymous object carries what it is given, the range variables of a query-syntax join or + // let, or the parts of a composite key, and builds nothing of its own: what it carries is read. + if (IsAnonymous(node.Type)) + { + return base.VisitNew(node); + } + Found = true; return node; @@ -209,11 +216,22 @@ protected override Expression VisitNewArray(NewArrayExpression node) protected override Expression VisitMethodCall(MethodCallExpression node) { + // A call that reads nothing of the lambda's and hands back a query or an expression, a + // repository's query, a specification or FromSql, is evaluated before the query runs, as EF + // Core evaluates it, and what it returns is read. + if (Evaluable(node)) + { + Found = !TryEvaluate(node, out object? value) || Holds(value, _depth); + + return node; + } + if (Reads(node.Method)) { return base.VisitMethodCall(node); } + // Any other method's result is its own to say. Found = true; return node; @@ -241,7 +259,8 @@ protected override Expression VisitConstant(ConstantExpression node) /// /// True for a method that hands back what it was given, or reads it: LINQ's operators, EF Core's - /// EF.Property and query operators, and a context's Set, which names a query root. + /// EF.Property and query operators, a context's Set, which names a query root, and a + /// context's own method returning a query, which EF Core translates only as a function the model maps. /// private static bool Reads(MethodInfo method) { @@ -251,7 +270,76 @@ private static bool Reads(MethodInfo method) || declaring == typeof(Enumerable) || declaring == typeof(EF) || declaring == typeof(EntityFrameworkQueryableExtensions) - || (declaring == typeof(DbContext) && method.Name == nameof(DbContext.Set)); + || (declaring == typeof(DbContext) && method.Name == nameof(DbContext.Set)) + || (declaring is not null && typeof(DbContext).IsAssignableFrom(declaring) + && typeof(IQueryable).IsAssignableFrom(method.ReturnType)); + } + + /// True for a type the compiler generates for an anonymous object. + private static bool IsAnonymous(Type type) => + type.IsDefined(typeof(System.Runtime.CompilerServices.CompilerGeneratedAttribute), inherit: false) + && type.Name.Contains("AnonymousType", StringComparison.Ordinal); + + /// + /// True for a call EF Core evaluates before it translates the query: one that reads no parameter of the + /// lambdas around it and hands back a query or an expression, which the query then holds. + /// + private static bool Evaluable(MethodCallExpression call) + { + if (!typeof(IQueryable).IsAssignableFrom(call.Type) && !typeof(Expression).IsAssignableFrom(call.Type)) + { + return false; + } + + ParameterFinder parameters = new(); + + parameters.Visit(call); + + return !parameters.Found; + } + + /// Finds a parameter of a lambda outside the expression it is given. + private sealed class ParameterFinder : ExpressionVisitor + { + private readonly HashSet _declared = new(); + + internal bool Found { get; private set; } + + public override Expression? Visit(Expression? node) => Found ? node : base.Visit(node); + + protected override Expression VisitLambda(Expression node) + { + _declared.UnionWith(node.Parameters); + + return base.VisitLambda(node); + } + + protected override Expression VisitParameter(ParameterExpression node) + { + Found |= !_declared.Contains(node); + + return node; + } + } + + /// Runs a call EF Core would evaluate itself, and reads what it returns. + private static bool TryEvaluate(MethodCallExpression call, out object? value) + { + try + { + value = Expression.Lambda>(Expression.Convert(call, typeof(object))) + .Compile(preferInterpretation: true)(); + + return true; + } + catch (Exception) + { + // Whatever it throws, EF Core would throw when it ran the query; read here, it only means the + // call counts as handing the rows anything. + value = null; + + return false; + } } /// True when a value of the type holds only values: a string, a number, a date, or a collection of them. @@ -316,6 +404,21 @@ private static bool Holds(object? value, int depth) return false; } + if (depth > MaxCapturedDepth) + { + return true; + } + + // An expression, a specification say, becomes part of the query, and is read as though written there. + if (value is Expression expression) + { + ForeignFinder finder = new(depth + 1); + + finder.Visit(expression); + + return finder.Found; + } + if (value is not IQueryable query) { return true; From f4e2c00ec6a88f647038734cad84f344e41934ea Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 10:36:49 +0300 Subject: [PATCH 20/22] docs: variance, wrapped providers, specifications, shared names, and the review's corrections What a reshaped chain counts as building now names specifications, repository queries, FromSql, query functions and anonymous carriers. The converted-column rule covers complex properties and names its scope; the security notes no longer say 3.1.0 kept such a column. The override rule names public hiding members and variant instantiations, the validator reads them too, and a dry run's trace is described as it behaves. A guarded query through a provider wrapping EF Core's runs untracked. Co-Authored-By: Claude Opus 5 --- DynamicWhere.ex/DOC.md | 16 ++-- DynamicWhere.ex/DynamicWhere.ex.csproj | 4 +- .../app/docs/breaking-changes/page.tsx | 31 +++++--- .../app/docs/policies/attributes/page.tsx | 7 +- .../app/docs/policies/configuration/page.tsx | 33 ++++++--- .../app/docs/policies/security/page.tsx | 13 +++- OfficialWebsite/public/llms.txt | 73 ++++++++++++------- README.md | 4 +- 8 files changed, 114 insertions(+), 67 deletions(-) diff --git a/DynamicWhere.ex/DOC.md b/DynamicWhere.ex/DOC.md index 2a3d98e..2e68935 100644 --- a/DynamicWhere.ex/DOC.md +++ b/DynamicWhere.ex/DOC.md @@ -1645,7 +1645,7 @@ A field the default keeps is a use of that field. One audited for `Order`, by `[ `DwPolicy.ValidateModel(options, types)` inspects the policy attributes on the given types and throws `InvalidOperationException` listing every error; `PolicyModelValidator.Inspect(types, options)` returns the same `PolicyModelReport` — `Errors`, `Warnings`, `IsValid` — without throwing. Called at startup, either one lets a misconfiguration fail the deployment rather than a caller's request. Since 3.1.0 the scan also reports: - every `[DwForceWhere]` resolution would refuse: `Value` and `ContextValue` both set or both missing on a comparison, either one set on a null check, a member whose type has no `DataType`, and `AllowNull` with `IsNull` / `IsNotNull` or on a member that can never be null. Before, these surfaced on the first query that resolved them; -- every `[DwEntity(DefaultOrder = ...)]` entry a guarded query would skip or leave out. An entry that is not a field optionally followed by `asc` or `desc` is an error; a field whose name starts with one of the words the expression parser keeps is an error, judged before the type is asked whether it has the member, because it may well have it — `"{Type}: DefaultOrder names '{field}', which starts with a name the expression parser keeps for itself, so no query can use it. Rename the member."`; a field the type does not have is a warning; a field no query can order by, such as a collection of entities, is an error; a field the type's own attributes seal against ordering is an error, because every guarded query would leave it out; the attributes include those of the member's other declarations, an interface member it implements and a subtype's override. A field denied for ordering only by attributes marked `Overridable = true` is a warning, because a rule can lift the denial for some callers — `"{Type}: DefaultOrder names '{field}', which its attributes deny for ordering unless a rule allows it, so guarded queries leave it out until one does."` — and so is a field the attributes deny for segments: `"{Type}: DefaultOrder names '{field}', which its attributes deny for segments, so guarded segments leave it out."` +- every `[DwEntity(DefaultOrder = ...)]` entry a guarded query would skip or leave out. An entry that is not a field optionally followed by `asc` or `desc` is an error; a field whose name starts with one of the words the expression parser keeps is an error, judged before the type is asked whether it has the member, because it may well have it — `"{Type}: DefaultOrder names '{field}', which starts with a name the expression parser keeps for itself, so no query can use it. Rename the member."`; a field the type does not have is a warning; a field no query can order by, such as a collection of entities, is an error; a field the type's own attributes seal against ordering is an error, because every guarded query would leave it out; the attributes include those of the member's other declarations, an interface member it implements, a subtype's override and a public member a subtype hides with `new`. A field denied for ordering only by attributes marked `Overridable = true` is a warning, because a rule can lift the denial for some callers — `"{Type}: DefaultOrder names '{field}', which its attributes deny for ordering unless a rule allows it, so guarded queries leave it out until one does."` — and so is a field the attributes deny for segments: `"{Type}: DefaultOrder names '{field}', which its attributes deny for segments, so guarded segments leave it out."` ### Blocked-action semantics @@ -1696,7 +1696,7 @@ The projection keeps the **allowed members**: what an unguarded call would retur - A field denied at the top of `T` always asks for one, whatever it holds: a scalar, a blob, a list, an owned object or a JSON column. - A field denied beneath a member asks for one when its value can reach the result. On an entity, that is beneath a column, an owned or complex member, or a navigation something loads: an `Include` or `ThenInclude` on the query, an automatic include, or a lazy loader — EF Core's proxies, an injected `ILazyLoader`, a loader delegate or `ILazyLoader` the constructor takes and keeps in a field or any property, the asynchronous loader delegate of EF Core 7, or an injected `DbContext` — which fills a navigation after the query. On a row a projection builds, it is beneath a member the initializer assigns; a constructor with arguments counts every member as assigned, and an initializer after it still says what its own bindings hold. On a row in memory, it is beneath any member. A rule may spell the path in any letter case. -- Every navigation counts as loaded where the library cannot read which the query loads: an `Include` in a form it cannot read, one off the query's own chain, and a chain that reaches its rows through anything but the root's own rows — `Select(o => o.Customer)`, a `SelectMany`, a `Join`, a `GroupBy` — when it also has an include, which EF Core applies from the root to the entities it reaches, or when one of its lambdas hands its rows an object: one it builds, as a projection behind an identity `Select`, a member of an anonymous row or a conditional does; one an application's method returns; or one it captured, another query with its own include or projection, or an object in memory. What only feeds a predicate or a key is a value and hands a row nothing. Such a chain with none of these is read from the model. +- Every navigation counts as loaded where the library cannot read which the query loads: an `Include` in a form it cannot read, one off the query's own chain, and a chain that reaches its rows through anything but the root's own rows — `Select(o => o.Customer)`, a `SelectMany`, a `Join`, a `GroupBy` — when it also has an include, which EF Core applies from the root to the entities it reaches, or when one of its lambdas hands its rows an object: one it builds, as a projection behind an identity `Select`, or an object built inside an anonymous row or a conditional, does; one an application's method returns from what the lambda gives it; or one it captured, another query with its own include or projection, or an object in memory. A call that reads nothing of the lambda's and returns a query or an expression — a specification, a repository's query, `FromSql`, a context's `Set` through an interface — is evaluated as EF Core evaluates it, and what it returns is read; a context's own query function is a query root; an anonymous object that only carries what the rows hold, query-syntax range variables or a composite key, builds nothing; and what only feeds a predicate or a key is a value and hands a row nothing. Such a chain with none of these is read from the model. - A denial beneath a navigation nothing loads never leaves the database, so it asks for no projection. An entity whose only denials sit beneath such navigations is read as it was in 3.1.0. - A member whose value can hold a field denied for `Select` that no path names — deeper than the walker's four segments, inside a framework generic such as `Dictionary`, or declared by a subtype of its type — asks for one too, read as for a named member, so on an entity only what loads counts. Under a `"*"` deny, so does a member whose value can hold a path the walk never asks about and the policy does not name. - A member that can hold an object of any type — one typed `object`, a framework interface such as `IComparable`, or a collection that is not generic, such as `IEnumerable`, `ArrayList` or an application's own — asks for nothing on its own: the policy cannot see into it whether or not a projection is built. @@ -1710,14 +1710,14 @@ The projection keeps the **allowed members**: what an unguarded call would retur | Source | Values kept | Objects kept | |---|---|---| | A projection that builds its rows before `ApplyPolicy`: the outermost `Select` constructs the row, in an object initializer or with a constructor, as in `db.Roles.Select(r => new RoleRow { … })` | Every member the initializer assigns; every member when a constructor with arguments builds the row, with or without an initializer after it | The same | -| An entity query, or a `Select` that hands back an entity, as in `db.Orders.Select(o => o.Customer)` | Every member EF Core maps | Its columns, converted and JSON ones included, its owned members and, on EF Core 8 or later, its complex properties, read from the EF Core model | +| An entity query, or a `Select` that hands back an entity, as in `db.Orders.Select(o => o.Customer)` | Every member EF Core maps | Its columns, converted and JSON ones included, its owned members and, on EF Core 8 or later, its complex properties, read from the EF Core model; a converted value that can hold an object of any type is left out | | Rows in memory, as in `roles.ApplyPolicy(caller)` | Every member | None | A value EF Core does not map is left out: computing it would make EF Core read the whole entity, the denied columns included, and it holds only its initial value anyway. A source the library cannot read — no EF Core model, and neither a projection it can see into nor rows in memory — keeps values only, as in 3.1.0, and every denial beneath a member counts. **Whole, narrowed or left out whole.** A member holding an object that the source carries is: -- **kept whole** when nothing beneath it is denied, nothing its value can hold is denied (its subtypes included), it cannot hold an object of any type (asked of a projected row, a row in memory, and an entity's column a value converter hands back: what EF Core materializes itself never holds one), under a `"*"` deny every path beneath it the walk skips is one the policy names, no forced scope is beneath it, and no transform beneath it lands on a property with no setter; +- **kept whole** when nothing beneath it is denied, nothing its value can hold is denied (its subtypes included), it cannot hold an object of any type (asked of a projected row, a row in memory, and an entity's column a value converter hands back, directly or inside a complex property: what EF Core materializes itself never holds one), under a `"*"` deny every path beneath it the walk skips is one the policy names, no forced scope is beneath it, and no transform beneath it lands on a property with no setter; - **narrowed** otherwise, to the allowed fields beneath it, four segments deep, as a caller naming it would get it, where the core's narrowing translates: an object the projection's initializer builds, a list a subquery reads into a type the core can bind (not an array or a set), a navigation that is neither complex nor stored as JSON, or an entity's owned member not stored as JSON. The narrowing builds the declared type, so a subtype's fields are dropped. A field beneath it that can hold what the policy cannot name is left out; - **left out whole** otherwise, and recorded as `Dropped` on `Select` with a reason that starts `left out whole`. @@ -1752,9 +1752,9 @@ The last one is recorded on the field beneath the member that the narrowing leav - A dry run synthesizes nothing. It records the denials and returns the rows whole. - A simulation has no source, so it reads `T` as a source it cannot see into: every denial beneath a member counts, and the projection it shows keeps only members holding a value (see [Administration](#administration)). -**What the policy cannot see into.** A member typed `object`, a framework interface or a collection that is not generic, such as `IEnumerable`, `ArrayList` or an application's own, is opaque to the policy. It never asks for a projection; when one is needed anyway, a projected row, a row in memory, and an entity's converted column leave it out, and an entity's other columns keep it; and naming it returns whatever it holds. A converter returning an application type through a column typed `object` is opaque the same way, so type the member as what it holds. An application's own such collection still has its own members read. A framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: naming it is refused in both tiers where the core cannot narrow it, at the top of `T` or on a row in memory, narrowed away under `Convenience` beneath a navigation, and a synthesized projection leaves it out. Hold such values in a list of the policed type instead. A member EF Core does not map is read as its type, since its getter can hand out what EF Core loaded; a getter that copies a denied column into a type with no denial is the application's to withhold. +**What the policy cannot see into.** A member typed `object`, a framework interface or a collection that is not generic, such as `IEnumerable`, `ArrayList` or an application's own, is opaque to the policy. It never asks for a projection; when one is needed anyway, a projected row, a row in memory, and an entity's converted column leave it out, and an entity's other columns keep it; and naming it returns whatever it holds. A converter returning an application type through a column typed `object` is opaque the same way, so type the member as what it holds. `BitArray` and the framework's string collections hold values. An application's own collection class, generic or not, still has its own members read. Two members sharing a name, one hidden with `new` under another type or spelled in another case, are left out when either holds a denial: the core reads one of them, and a row carries both. A framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: naming it is refused in both tiers where the core cannot narrow it, at the top of `T` or on a row in memory, narrowed away under `Convenience` beneath a navigation, and a synthesized projection leaves it out. Hold such values in a list of the policed type instead. A member EF Core does not map is read as its type, since its getter can hand out what EF Core loaded; a getter that copies a denied column into a type with no denial is the application's to withhold. -**A projection builds the declared type.** A query over the root of a hierarchy whose derived type declares a denied field comes back as root-type rows, the derived types' allowed fields dropped too. Over an abstract root the typed terminals fail with `SelectTypeMustHaveParameterlessConstructor`, and the dynamic ones return the root's members. Query the derived type, `OfType()`, to keep its fields. Rows in memory can be any loaded subtype, and the policy does not look at the rows: when a subtype declares a denied field, they are projected and their objects left out, even if no row is that subtype. Under a `"*"` deny, a member is kept whole only when every path beneath it the walk skips is one the policy names. A `[DwDenied]` on an override, on a member a subtype hides with `new`, or on a class's implementation of an interface member, applies to the path through the base type or the interface, on every row and in every clause. +**A projection builds the declared type.** A query over the root of a hierarchy whose derived type declares a denied field comes back as root-type rows, the derived types' allowed fields dropped too. Over an abstract root the typed terminals fail with `SelectTypeMustHaveParameterlessConstructor`, and the dynamic ones return the root's members. Query the derived type, `OfType()`, to keep its fields. Rows in memory can be any loaded subtype, and the policy does not look at the rows: when a subtype declares a denied field, they are projected and their objects left out, even if no row is that subtype. Under a `"*"` deny, a member is kept whole only when every path beneath it the walk skips is one the policy names. A `[DwDenied]` on an override, on a public member a subtype hides with `new`, or on a class's implementation of an interface member, through a variant instantiation too, applies to the path through the base type or the interface, on every row and in every clause. **A forced scope on a list's element type filters rows, not elements.** A forced scope declared on a list's element type filters the rows that hold the list, never its elements. `Selects` naming the list returns every element, those the scope excludes included, as in every release; a synthesized projection leaves such a list out. Scope the elements where the row is built. @@ -2318,11 +2318,11 @@ All validation errors throw `LogicException` (inherits `Exception`) with one of Fixed (security): a field denied only beneath a member, none at the top of `T`, synthesized nothing, so the whole row came back with the denied value in it — in a list or nested object of a row projected before `ApplyPolicy`, in a row held in memory, and in an entity's included, automatically included, lazily loaded or owned member — typed and dynamic, in both tiers, for a `Filter` and a `Segment`. Such a denial now synthesizes the projection whenever its value can reach the result: on an entity, beneath a column, an owned or complex member, or a navigation the query loads through `Include`, an automatic include or a lazy loader; on a projected row, beneath a member the initializer assigns; in memory, beneath any member. A denial beneath a navigation nothing loads never leaves the database, and the entity is read exactly as in 3.1.0. - Fixed (security): what a query loads was read too narrowly. An include named from the root and reached through `Select(o => o.Customer)`, `SelectMany` or `Join`, a projection behind another `Select`, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or a property of any name each loaded a denied value the gate read as unloaded. An injected `DbContext` or EF Core 7's asynchronous loader delegate did too, and so did a reshaping lambda that got its row from an application's method or from a captured query or object. A converted column typed `object` was kept whole when a projection was built for another field, and an application's own non-generic collection hid its own denied members. A field a subtype declares, a derived entity's or a subclass's held by a base-typed member, was not read at all, nor was a `[DwDenied]` on an override, on a member hidden with `new` or on an interface member's implementation, and under a `"*"` deny a path the walk never asked about was allowed. Each came back. Now every navigation counts as loaded on such a chain, the subtypes are read, and such a path is denied. When a type the model derives from `T`, or a loaded subclass of a row in memory, declares a denied field, the rows are projected to `T`, dropping a derived type's allowed fields too; over an abstract `T` the typed terminals fail with `SelectTypeMustHaveParameterlessConstructor` and the dynamic ones return its members. + Fixed (security): what a query loads was read too narrowly. An include named from the root and reached through `Select(o => o.Customer)`, `SelectMany` or `Join`, a projection behind another `Select`, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or a property of any name each loaded a denied value the gate read as unloaded. An injected `DbContext` or EF Core 7's asynchronous loader delegate did too, and so did a reshaping lambda that got its row from an application's method or from a captured query or object. An application's own collection class hid its own denied members, and a guarded query through a provider wrapping EF Core's, such as LinqKit's `AsExpandable`, ran tracking, so the context filled in navigations it already held and a masked value became a pending change. A field a subtype declares, a derived entity's or a subclass's held by a base-typed member, was not read at all, nor was a `[DwDenied]` on an override, on a member hidden with `new` or on an interface member's implementation, and under a `"*"` deny a path the walk never asked about was allowed. Each came back. Now every navigation counts as loaded on such a chain, the subtypes are read, and such a path is denied. When a type the model derives from `T`, or a loaded subclass of a row in memory, declares a denied field, the rows are projected to `T`, dropping a derived type's allowed fields too; over an abstract `T` the typed terminals fail with `SelectTypeMustHaveParameterlessConstructor` and the dynamic ones return its members. Fixed (security): a field denied at the top of `T` whose own type is not a simple value — a byte array, a list, an owned object, a JSON column — synthesized no projection either, so with nothing else denied the whole row came back with it. - Changed: the projection kept simple fields only, so as soon as any field was denied, every nested object and list of a row projected before `ApplyPolicy` came back null or empty, and so did an entity's columns holding an object, its owned and complex members and its collections of simple values. A row a projection builds now keeps the members its initializer assigns; an entity keeps its mapped columns, converted and JSON ones included, its owned and complex members, and every collection of simple values; rows in memory keep their values. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, with a `Dropped` decision whose reason starts `left out whole`. An entity's navigations, included ones too, the objects of a row in memory, and a value EF Core does not map are left out; under `Convenience` name a navigation in `Selects` to get it narrowed, and under `Strict` name its allowed fields. A forced scope beneath a member asks for no projection on its own, and a projection needed for another reason leaves such a member out whole. A typed query needs `T` to have a public parameterless constructor for the projection, as it already did (point 1). + Changed: the projection kept simple fields only, so as soon as any field was denied, every nested object and list of a row projected before `ApplyPolicy` came back null or empty, and so did an entity's columns holding an object, its owned and complex members and its collections of simple values. A row a projection builds now keeps the members its initializer assigns; an entity keeps its mapped columns, converted and JSON ones included, except a converted one that can hold an object of any type, its owned and complex members, and every collection of simple values; rows in memory keep their values. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, with a `Dropped` decision whose reason starts `left out whole`. An entity's navigations, included ones too, the objects of a row in memory, and a value EF Core does not map are left out; under `Convenience` name a navigation in `Selects` to get it narrowed, and under `Strict` name its allowed fields. A forced scope beneath a member asks for no projection on its own, and a projection needed for another reason leaves such a member out whole. A typed query needs `T` to have a public parameterless constructor for the projection, as it already did (point 1). 21. **`Selects` Naming a Member Is Gated Against Every Denial Beneath It** When `Selects` names a navigation with a denied field beneath it, the `Convenience` tier replaces the entry with the allowed fields beneath it, and the `Strict` tier refuses it. Since 3.2.0 the gate finds every denial beneath the member, and refuses, with `FieldDeniedForSelect`, a narrowing it cannot build as gated. See [A navigation named in Selects](#a-navigation-named-in-selects). diff --git a/DynamicWhere.ex/DynamicWhere.ex.csproj b/DynamicWhere.ex/DynamicWhere.ex.csproj index fc15864..80debba 100644 --- a/DynamicWhere.ex/DynamicWhere.ex.csproj +++ b/DynamicWhere.ex/DynamicWhere.ex.csproj @@ -52,7 +52,7 @@ Security fix: a field denied for Select only beneath a member was not enforced when the caller sent no Selects. A guarded query synthesizes a projection for a denied field, and it did so only when a simple field at the top of the type was denied. With every denial beneath a member, the whole row came back with the denied value in it: in a list or nested object of a row projected before ApplyPolicy, in a row held in memory, and in an entity's included, automatically included, lazily loaded or owned member, typed and dynamic, in both tiers, for a Filter and a Segment. Such a denial now synthesizes the projection whenever its value can reach the result. On an entity that means beneath a column, an owned or complex member, or a navigation the query loads through Include, an automatic include or a lazy loader; a denial beneath a navigation nothing loads never leaves the database, and the entity is read exactly as before. -Security fix: what a query loads, and what a member holds, was read too narrowly. An include named from the root and reached through Select(o => o.Customer), SelectMany or Join, a projection behind another Select, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or any property each loaded a denied value the gate read as unloaded. An injected DbContext and EF Core 7's asynchronous loader delegate loaded one too, and so did a reshaping lambda that got its row from an application's method or from a captured query or object. A converted column typed object was kept whole when a projection was built for another field, and an application's own non-generic collection hid its own denied members. A field a subtype declares, a derived entity's, a subclass's held by a base-typed member or an open generic one's, was not read at all, nor was a [DwDenied] on an override, on a member hidden with new or on an interface member's implementation, and under a "*" deny with exact allows a path the walk never asked about (past four segments, around a cycle, with no setter) resolved as allowed. Selects naming an entity navigation returned a denial in its owned chain past four segments or in a converted Dictionary<string, T> column. All of these are read now: from the EF Core model for an entity, so only what loads counts, and from every loaded subtype for a projected or in-memory row. Rows whose derived type declares a denied field come back as the queried type. A denial on an override, a member hidden with new or an implementation applies to the base type's or the interface's path, on every row and in every clause. +Security fix: what a query loads, and what a member holds, was read too narrowly. An include named from the root and reached through Select(o => o.Customer), SelectMany or Join, a projection behind another Select, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or any property each loaded a denied value the gate read as unloaded. An injected DbContext and EF Core 7's asynchronous loader delegate loaded one too, and so did a reshaping lambda that got its row from an application's method or from a captured query or object. An application's own collection class hid its own denied members, and a guarded query through a provider wrapping EF Core's, such as LinqKit's AsExpandable, ran tracking, so the context filled in navigations it already held and a masked value became a pending change. A field a subtype declares, a derived entity's, a subclass's held by a base-typed member or an open generic one's, was not read at all, nor was a [DwDenied] on an override, on a member hidden with new or on an interface member's implementation, and under a "*" deny with exact allows a path the walk never asked about (past four segments, around a cycle, with no setter) resolved as allowed. Selects naming an entity navigation returned a denial in its owned chain past four segments or in a converted Dictionary<string, T> column. All of these are read now: from the EF Core model for an entity, so only what loads counts, and from every loaded subtype for a projected or in-memory row. Rows whose derived type declares a denied field come back as the queried type. A denial on an override, a public member hidden with new or an implementation, through a variant instantiation too, applies to the base type's or the interface's path, on every row and in every clause. Security fix: a field denied at the top of a type whose own type is not a simple value, such as a byte array, a list, an owned object or a JSON column, synthesized no projection either, so with nothing else denied it came back. @@ -62,7 +62,7 @@ Security fix: under the convenience tier, Selects naming a navigation whose elem Security fix: Selects naming a member typed as a collection the core does not unwrap, such as IReadOnlyList<T>, returned every field beneath it, denied ones included, in both tiers. The projection gate read collections through a narrower list than the attribute walker that puts policy on the fields beneath; it now reads them the same way. Denials beneath a named member are also read from the policy's own rules, so a denied property with no setter and a rule on a path reached through a cycle are found, and a member carrying a field denied where no path reaches it, deeper than the walker, inside a framework collection such as Dictionary<string, T> or on a subtype, is refused under the strict tier, and under the convenience tier narrowed where the core can narrow it and refused where it cannot. -Behaviour change: the projection synthesized for a guarded query that sends no Selects keeps what the source carries. It used to keep simple fields only, so every nested object and list of a row projected before ApplyPolicy came back null or empty as soon as any field was denied. A row a projection builds keeps the members its initializer assigns. An entity keeps its mapped columns, converted and JSON ones included, its owned and complex members, and every collection of simple values such as byte[] or List<string>. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, recorded as Dropped with a reason starting "left out whole". An entity's navigations, the objects of a row in memory, and a value EF Core does not map are left out, and the type needs a public parameterless constructor for the typed projection, as it already did. A member that can hold an object of any type, a geometry, a JSON bag or a BitArray column say, asks for no projection on its own; a member EF Core does not map is read as its type, since its getter can hand out what EF Core loaded; and a chain reaching its rows through a navigation, SelectMany, Join or GroupBy counts every navigation as loaded only when it has an include or builds an object. +Behaviour change: the projection synthesized for a guarded query that sends no Selects keeps what the source carries. It used to keep simple fields only, so every nested object and list of a row projected before ApplyPolicy came back null or empty as soon as any field was denied. A row a projection builds keeps the members its initializer assigns. An entity keeps its mapped columns, converted and JSON ones included except a converted one that can hold an object of any type, its owned and complex members, and every collection of simple values such as byte[] or List<string>. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, recorded as Dropped with a reason starting "left out whole". An entity's navigations, the objects of a row in memory, and a value EF Core does not map are left out, and the type needs a public parameterless constructor for the typed projection, as it already did. A member that can hold an object of any type, a geometry or a JSON bag say, asks for no projection on its own; a member EF Core does not map is read as its type, since its getter can hand out what EF Core loaded; and a chain reaching its rows through a navigation, SelectMany, Join or GroupBy counts every navigation as loaded only when it has an include, or a lambda that builds an object, gets one from an application's method, or captures a query with its own include or projection. Behaviour change: [DwEntity(DefaultOrder)] applies to a projected source whose outermost Select builds the type in an object initializer and assigns every field the default names a column, at every level of a nested path: a mapped member read directly, through reference navigations or through EF.Property. A computed value or any other projection still leaves the query in its own order. A Select, or a Filter with Selects, composed on the guarded handle keeps the rest of the chain unordered, and a composed Filter that sent orders gets no default later in the chain, as a composed Order already did not. diff --git a/OfficialWebsite/app/docs/breaking-changes/page.tsx b/OfficialWebsite/app/docs/breaking-changes/page.tsx index 6f83235..c01874a 100644 --- a/OfficialWebsite/app/docs/breaking-changes/page.tsx +++ b/OfficialWebsite/app/docs/breaking-changes/page.tsx @@ -1118,7 +1118,9 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say in memory, beneath any member. A chain that reaches its rows through a navigation, a SelectMany, a Join or a{" "} GroupBy counts every navigation as loaded when it also has - an include or builds an object in a lambda. A field a subtype of{" "} + an include, or a lambda that builds an object, gets one from an + application's method, or captures a query with its own include or + projection. A field a subtype of{" "} T declares, one a subtype of a member's type declares, and one beneath a member EF Core does not map, count too. A member that can hold an object of any type asks for nothing on its @@ -1132,7 +1134,8 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say What. The allowed members, which replace the allowed scalars. A row a projection builds keeps the members its initializer assigns. An entity keeps its mapped columns, converted and JSON ones - included, its owned and complex members, and every collection of + included except a converted one that can hold an object of any type, + its owned and complex members, and every collection of simple values such as byte[] or{" "} List<string>. Rows in memory keep their values. A member holding an object is kept whole when nothing it can hold is @@ -1158,13 +1161,16 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say loaded a denied value the gate read as unloaded, and so did an injected{" "} DbContext or EF Core 7's asynchronous loader delegate, and a reshaping lambda that got its row from an application's - method or from a captured query or object. A converted column typed{" "} - object was kept whole when a projection was built for - another field, and an application's own non-generic collection hid - its own denied members. A field a subtype declares — a derived entity's, or a + method or from a captured query or object. An application's own + collection class hid its own denied members, and a guarded query + through a provider wrapping EF Core's, such as LinqKit's{" "} + AsExpandable, ran tracking, so the context filled in + navigations it already held and a masked value became a pending change. + A field a subtype declares — a derived entity's, or a subclass's held by a base-typed member — was not read at all, nor - was a [DwDenied] on an override, on a member hidden with{" "} - new or on an interface member's implementation, and + was a [DwDenied] on an override, on a public member hidden + with new or on an interface member's implementation, + and under a{" "} "*" deny a path the walk never asked about was allowed. Each came back. @@ -1180,9 +1186,9 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say {`OfType()`}, to keep its fields. Rows in memory can be any loaded subtype, so there the rows are projected whenever one declares a denied field. A [DwDenied] on an override, on a - member a subtype hides with new, or on an interface - member's implementation denies the base path for every row, in - every clause. + public member a subtype hides with new, or on an interface + member's implementation, through a variant instantiation too, + denies the base path for every row, in every clause. A field denied at the top of T whose own type is not a @@ -1195,7 +1201,8 @@ PolicyTrace? recorded = guarded.LastTrace; // recorded whatever the setting say of a row projected before ApplyPolicy came back null or empty, and so did an entity's columns holding an object, its owned and complex members and its collections of simple values. They are - returned now, whole or narrowed. A member that cannot be narrowed is + returned now, whole or narrowed, except a converted value that can hold + an object of any type, which the policy cannot see into. A member that cannot be narrowed is left out whole, and the trace records a Dropped decision whose reason starts left out whole. diff --git a/OfficialWebsite/app/docs/policies/attributes/page.tsx b/OfficialWebsite/app/docs/policies/attributes/page.tsx index 200b9f5..0365614 100644 --- a/OfficialWebsite/app/docs/policies/attributes/page.tsx +++ b/OfficialWebsite/app/docs/policies/attributes/page.tsx @@ -237,9 +237,10 @@ public string EmployeeCode { get; set; }`} An access-control attribute on another declaration of a member applies to its path too: on the interface member a class or its subtype implements with it, on an override of either accessor a subtype - declares, on a member a subtype hides with new, and on the - implementation a type gives an interface member, explicit, inherited or - declared by an open generic class. A row read through the base type or + declares, on a public member a subtype hides with new, and + on the implementation a type gives an interface member, explicit, + inherited or declared by an open generic class, through a variant + instantiation too. A row read through the base type or the interface is still that subtype, so the denial holds for the path on every row, in every clause. Until 3.2.0 only the declaration walked, and the attributes above it, were read. The other attributes are still read diff --git a/OfficialWebsite/app/docs/policies/configuration/page.tsx b/OfficialWebsite/app/docs/policies/configuration/page.tsx index 719fcf6..b1a1601 100644 --- a/OfficialWebsite/app/docs/policies/configuration/page.tsx +++ b/OfficialWebsite/app/docs/policies/configuration/page.tsx @@ -202,12 +202,18 @@ export default function Page() { Join, a GroupBy — when it also has an include, which EF Core applies from the root to the entities it reaches, or when one of its lambdas hands its rows an object: one it - builds, as a projection behind an identity Select, a - member of an anonymous row or a conditional does; one an - application's method returns; or one it captured, another query - with its own include or projection, or an object in memory. What only - feeds a predicate or a key is a value and hands a row nothing. Such a - chain with none of these is read from the model. + builds, as a projection behind an identity Select, or an + object built inside an anonymous row or a conditional, does; one an + application's method returns from what the lambda gives it; or + one it captured, another query with its own include or projection, or + an object in memory. A call that reads nothing of the lambda's and + returns a query or an expression (a specification, a repository's + query, FromSql) is evaluated as EF Core evaluates it, and + what it returns is read; a context's own query function is a query + root; an anonymous object that only carries what the rows hold, range + variables or a composite key, builds nothing; and what only feeds a + predicate or a key is a value. Such a chain with none of these is read + from the model.
    • A denial beneath a navigation nothing loads never leaves the database, @@ -284,7 +290,7 @@ export default function Page() {
      • - + @@ -309,8 +315,9 @@ export default function Page() { kept whole when nothing beneath it is denied, nothing its value can hold is denied (its subtypes included), it cannot hold an object of any type (asked of a projected row, a row in memory, and an - entity's column a value converter hands back: what EF Core - materializes itself never holds one), under a{" "} + entity's column a value converter hands back, directly or inside a + complex property: what EF Core materializes itself never holds one), + under a{" "} "*" deny every path beneath it the walk skips is one the policy names, no forced scope is beneath it, and no transform beneath it lands on a property with no setter; @@ -418,9 +425,11 @@ left out whole: it can hold what the policy cannot name`} leave it out, and an entity's other columns keep it; and naming it returns whatever it holds. A converter returning an application type through a column typed object is opaque - the same way, so type the member as what it holds. An - application's own collection still has its own members read. A - framework generic holding a + the same way, so type the member as what it holds.{" "} + BitArray and the framework's string collections hold + values. An application's own collection class, generic or not, + still has its own members read, and two members sharing a name are left + out when either holds a denial. A framework generic holding a policed type, such as Dictionary<string, LineDto>, has no paths beneath it: naming it is refused in both tiers where the core cannot narrow it, narrowed away under Convenience beneath a diff --git a/OfficialWebsite/app/docs/policies/security/page.tsx b/OfficialWebsite/app/docs/policies/security/page.tsx index 1bc493e..40fbf3d 100644 --- a/OfficialWebsite/app/docs/policies/security/page.tsx +++ b/OfficialWebsite/app/docs/policies/security/page.tsx @@ -301,7 +301,7 @@ true order. Add [DwNoOrder] unless that is intended.`} - + + + + +
      Declare the denied field on a subtype — a derived entity, a subclass, an interface's implementation — and read it through the base type: a query over the hierarchy's root, or a member declared as the base type The subtypes are read too: the types the EF Core model derives for - an entity, every loaded subtype for a projected or in-memory row. - Such rows are projected to T and such members narrowed - to the declared type; a named one is refused under{" "} - Strict. The policy used to read the declared type only. + an entity, and for a projected or in-memory row every loaded + subtype, an open generic one and an application's subclass of a + framework class such as Exception included. Such rows + are projected to T and such members narrowed to the + declared type; a named one is refused under Strict. A + projection constructing a subtype of T is read as it, + and a rule on a subtype's field through a base-typed member is + enforced. The policy used to read the declared type only. +
      Put the [DwDenied] on an override, or on a class's implementation of an interface member, and read the member through the base type or the interface + The denial applies to the path for every row, in every clause. The + attribute walker read the declaration it walked and the attributes + above it, never an override or an implementation below, so the + base path filtered, sorted, grouped and returned the value.
      Put the [DwDenied] on an override, or on a class's implementation of an interface member, and read the member through the base type or the interfacePut the [DwDenied] on an override, on a member a subtype hides with new, or on a class's implementation of an interface member, and read the member through the base type or the interface The denial applies to the path for every row, in every clause. The attribute walker read the declaration it walked and the attributes - above it, never an override or an implementation below, so the - base path filtered, sorted, grouped and returned the value. + above it, never an override, a hiding member or an implementation + below, so the base path filtered, sorted, grouped and returned the + value.
      An entity query, or a Select that hands back an entity, as in {`db.Orders.Select(o => o.Customer)`} Every member EF Core mapsIts columns, converted and JSON ones included, its owned members and, on EF Core 8 or later, its complex properties, read from the EF Core modelIts columns, converted and JSON ones included, its owned members and, on EF Core 8 or later, its complex properties, read from the EF Core model; a converted value that can hold an object of any type is left out
      Rows in memory, as in roles.ApplyPolicy(caller)
      Put the [DwDenied] on an override, on a member a subtype hides with new, or on a class's implementation of an interface member, and read the member through the base type or the interfacePut the [DwDenied] on an override, on a public member a subtype hides with new, or on a class's implementation of an interface member, and read the member through the base type or the interface, a variant instantiation of it included The denial applies to the path for every row, in every clause. The attribute walker read the declaration it walked and the attributes @@ -310,6 +310,17 @@ true order. Add [DwNoOrder] unless that is intended.`} value.
      Guard a query through a provider that wraps EF Core's, as LinqKit's AsExpandable or DelegateDecompiler's Decompile do + The query runs untracked. EF Core's AsNoTracking{" "} + hands such a query back unchanged, so it tracked: the context filled + in navigations it already held, the denied ones included, and a + masked value became a pending change the next{" "} + SaveChanges would write. The call now goes into the + query itself. +
      Under a "*" deny with exact allows, reach a path the walk never asks about: past four segments, around a cycle, a property with no setter diff --git a/OfficialWebsite/public/llms.txt b/OfficialWebsite/public/llms.txt index bdc7735..8fc09da 100644 --- a/OfficialWebsite/public/llms.txt +++ b/OfficialWebsite/public/llms.txt @@ -1787,7 +1787,9 @@ How this differs from the unguarded surface: Rules: - Every guarded query runs over `AsNoTracking()`. Returned EF Core entities are detached, so a masked value is never - saved back. An in-memory source has no such copy: without `Selects` and with nothing denied, the rows returned are + saved back. Through a provider that wraps EF Core's, as LinqKit's `AsExpandable` and DelegateDecompiler's + `Decompile` do, EF Core's extension would hand the query back still tracking, so the call is put into the query + itself (3.2.0). An in-memory source has no such copy: without `Selects` and with nothing denied, the rows returned are the source objects themselves, transformed in place. When a projection is synthesized the rows are new, and they hold no member of an object type, so the source objects are left as they were. - Chaining keeps decisions: each composable returns a new handle, and the terminal's trace includes the earlier links' decisions. @@ -1978,8 +1980,9 @@ No named attribute refuses `Segment`: write `[DwDeny(PolicyFeature.Segment)]`. counted. - One on another declaration of the member applies to the path too (3.2.0): on the interface member a class, or a loaded subtype of it, implements with the member; on an override of either accessor a loaded subtype declares; on - a member a loaded subtype hides with `new`; and on the implementation a loaded type gives an interface member, - explicit, inherited from a base class or declared by an open generic class. A row read through the base type or + a public member a loaded subtype hides with `new`; and on the implementation a loaded type gives an interface + member, explicit, inherited from a base class or declared by an open generic class, through that interface or an + instantiation variance lets stand for it (`IFeed` for a member typed `IFeed`, with `out T`). A row read through the base type or the interface is still that subtype, and its member returns what the subtype's declaration returns, so the denial holds for the path on every row, Where, Order, Group and Select alike. A member hidden with `new` counts because whether it reads the member it hides cannot be told from outside, and a row serialized as its own type writes it @@ -2371,10 +2374,14 @@ DefaultOrder field denied for Segment, in a Segment (3.1.0) left out l chain that reaches its rows through anything but the root's own rows (`Select(o => o.Customer)`, a `SelectMany`, a `Join`, a `GroupBy`) when it also has an include, which EF Core applies from the root to the entities it reaches, or when one of its lambdas hands its rows an object: one it builds (a projection behind an - identity `Select`, a member of an anonymous row, a conditional), which loads whatever it assigns; one an - application's method returns; or one it captured, another query with its own include or projection, or an - object in memory. What only feeds a predicate or a key is a value and hands a row nothing. Such a chain with - none of these is read from the model. A denial beneath a navigation nothing loads never leaves the database + identity `Select`, or an object built inside an anonymous row or a conditional), which loads whatever it + assigns; one an application's method returns from what the lambda gives it; or one it captured, another query + with its own include or projection, or an object in memory. A call that reads nothing of the lambda's and returns + a query or an expression (a specification, a repository's query, `FromSql`, a context's `Set` through an + interface) is evaluated as EF Core evaluates it, and what it returns is read; a context's own query function is a + query root; an anonymous object that only carries what the rows hold (query-syntax range variables, a + composite key) builds nothing; and what only feeds a predicate or a key is a value and hands a row nothing. Such + a chain with none of these is read from the model. A denial beneath a navigation nothing loads never leaves the database and needs no projection, so a connected model is read as it was in 3.1.0. A member EF Core does not map counts as loaded: its getter can hand out a mapped field or a private navigation, so its type is read whole. @@ -2411,8 +2418,8 @@ DefaultOrder field denied for Segment, in a Segment (3.1.0) left out l - every allowed member holding an object or a list of them that the source carries: a member a projection's initializer assigns, and an entity's columns (converted or JSON), owned and complex members. It is kept whole when nothing beneath it is denied, nothing its value can hold is denied, it cannot hold an object of any type - (asked of a projected row, a row in memory, and an entity's column a value converter hands back: what EF Core - materializes itself never holds one), under a + (asked of a projected row, a row in memory, and an entity's column a value converter hands back, directly or + inside a complex property: what EF Core materializes itself never holds one), under a `"*"` deny every path beneath it the walk skips is one the policy names, no forced scope is beneath it, and no transform beneath it lands on a property with no setter; - otherwise it is narrowed to the allowed leaves beneath it, to four segments, as a caller naming it would get, @@ -2791,7 +2798,7 @@ Errors: `"{Type}: DefaultOrder names '{field}', which no query can order by, so guarded queries skip it."` - a field the type's own attributes deny for ordering, unless every one of those denials is `Overridable` (then a warning, below). The attributes include those of the member's other declarations, an interface member it - implements and a subtype's override (3.2.0): + implements, a subtype's override and a public member a subtype hides with `new` (3.2.0): `"{Type}: DefaultOrder names '{field}', which its attributes deny for ordering, so every guarded query leaves it out."` - a stage that cannot be built: - `Round` or `Bucket` with `Step <= 0`, or `Truncate` with `Decimals < 0` @@ -4784,10 +4791,17 @@ MemoryCalculationInput a lazy loader the constructor takes, delegate or ILazyLoader, kept in a field or a property of any name, and an initializer after a constructor with arguments counts every member as assigned. So do an injected DbContext and EF Core 7's asynchronous loader delegate. So does a reshaped chain whose lambda hands its rows - an object an application's method returns or one it captured: another query with its own include or - projection, or an object in memory. Each returned the denied value. A reshaped chain with none of these is - still read from the model, so it is not projected for a denial beneath a navigation it does not load, and a - value that only feeds a predicate or a key no longer counts as building its rows. + an object an application's method returns from the row, or one it captured: another query with its own + include or projection, or an object in memory. Each returned the denied value. A reshaped chain with none of + these is still read from the model, so it is not projected for a denial beneath a navigation it does not + load. A specification, a repository's query, FromSql and a context's Set through an interface are evaluated + as EF Core evaluates them, a context's query function is a query root, and an anonymous object carrying + range variables or a composite key, or a value that only feeds a predicate or a key, builds nothing. + - Security fix. A guarded query through a provider that wraps EF Core's, LinqKit's AsExpandable or + DelegateDecompiler's Decompile, ran tracking: EF Core's AsNoTracking hands such a query back unchanged. The + rows' navigations were then filled from entities the context already tracked, the denied ones included, + and a masked value became a pending change the next SaveChanges would write. AsNoTracking now goes into the + query itself. - Security fix and behaviour change. A member declared as a base type or an interface holds its subtypes, whose denied fields the declared type never names. They are read now: the types the EF Core model derives, for an entity, and every loaded subtype for a projected or in-memory row, an open generic one and an @@ -4798,10 +4812,10 @@ MemoryCalculationInput typed terminals then fail with SelectTypeMustHaveParameterlessConstructor, as for any T they cannot build; the dynamic terminals return the root's allowed members. A rule on a subtype's field through a base-typed member was dropped as naming nothing; it is enforced. - - Security fix. A deny-family attribute on an override, on a member a subtype hides with new, on the - implementation of an interface member, or on the interface member a class implements, was read only from - its own declaration, so the base type's or the interface's path filtered, sorted, grouped and returned the - value. It applies to the path now. + - Security fix. A deny-family attribute on an override, on a public member a subtype hides with new, on the + implementation of an interface member (through a variant instantiation too), or on the interface member a + class implements, was read only from its own declaration, so the base type's or the interface's path + filtered, sorted, grouped and returned the value. It applies to the path now. - Security fix. Under a "*" deny with exact allows, a path the walk never asked about (past four segments, around a cycle, with no setter, on a subtype) resolved as allowed, so a member holding one was returned whole, named or not. Each such path is asked of the policy now; one it does not name is denied, and a @@ -4832,12 +4846,15 @@ MemoryCalculationInput it can hold is denied, narrowed around a denial where the core's narrowing translates, and otherwise left out whole with a "left out whole" Dropped decision. An entity's navigations and the objects of a row in memory are left out, as before, and each one the unguarded call would have returned is recorded as Dropped; a - value EF Core does not map is left out too. A member that can hold an object of any type, a geometry, a - JSON bag or a BitArray column say, asks for no projection on its own, and an entity keeps it whole, unless a - value converter hands back its value: a converter is the application's code, so such a column is left out. - An application's own collection that is not generic has its own members read. A projected member is read as - the type its initializer constructs, and an initializer after a constructor with arguments narrows its own - bindings. The trace records a member left out only when a projection is built. + value EF Core does not map is left out too. A member that can hold an object of any type, a geometry or a + JSON bag say, asks for no projection on its own, and an entity keeps it whole, unless a value converter + hands back its value, directly or inside a complex property: a converter is the application's code, so + such a column is left out. BitArray and the framework's string collections hold values. An application's + own collection class, generic or not, has its own members read. Two members sharing a name, one hidden with + new under another type or spelled in another case, are left out when either holds a denial, since the core + reads one and a row carries both. A projected member is read as the type its initializer constructs, and an + initializer after a constructor with arguments narrows its own bindings. The trace records a member left + out only when a projection is built, or, in a dry run, would be. - Behaviour change. [DwEntity(DefaultOrder)] applies to a projected source whose outermost Select builds T in an object initializer assigning every field the default names a column: a mapped member, read directly, through reference navigations or through EF.Property. A computed value or any other projection still leaves @@ -5016,9 +5033,11 @@ MemoryCalculationInput `Array`, an application's own) is opaque to the policy. It never asks for a projection; when one is needed anyway, a projected row, a row in memory, and an entity's converted column leave it out, and an entity's other columns keep it (what EF Core materializes itself holds no application object); and naming it - returns whatever it holds. A value converter that returns an application type through a column typed `object`, - and an unmapped getter typed `object` over a private navigation, are opaque the same way: with nothing else - denied the row comes back as loaded. Type the member as what it holds. + returns whatever it holds. A converted column counts as able to hold anything when its type can: `object`, a + `Dictionary`, or a type with such a member. `BitArray`, `StringCollection`, `StringDictionary` and + `NameValueCollection` hold values. A value converter that returns an application type through a column typed + `object`, and an unmapped getter typed `object` over a private navigation, are opaque the same way: with nothing + else denied the row comes back as loaded. Type the member as what it holds. A framework generic holding a policed type (`Dictionary`) has no paths beneath it: naming it is refused in both tiers where the core cannot narrow it (at the top of T, or on a row in memory), narrowed away under Convenience beneath a navigation, and a synthesized projection leaves it out. diff --git a/README.md b/README.md index f397e99..045bacb 100644 --- a/README.md +++ b/README.md @@ -380,12 +380,12 @@ The complete reference — every enum, class, extension method, validation rule, **Upgrade note — the security fixes and the three changes alter what code written for 3.1.0 does, and the new overloads can stop a call from compiling. Read these before bumping.** - **Fixed (security): a field denied beneath a member reached a caller who sent no `Selects`.** A guarded query synthesizes a projection for a denied field, and it did so only when a simple field at the top of the type was denied. With every denial beneath a member, the whole row came back with the denied value in it: in a list or nested object of a row projected before `ApplyPolicy`, in a row held in memory, and in an entity's included, automatically included, lazily loaded or owned member — typed and dynamic, in both tiers, for a `Filter` and a `Segment`. Such a denial now synthesizes the projection whenever its value can reach the result. On an entity that means beneath a column, an owned or complex member, or a navigation the query loads through `Include`, an automatic include or a lazy loader. A denial beneath a navigation nothing loads never leaves the database, and the entity is read exactly as before. -- **Fixed (security): what a query loads, and what a member holds, was read too narrowly.** An include named from the root and reached through `Select(o => o.Customer)`, `SelectMany` or `Join`, a projection behind another `Select`, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or any property each loaded a denied value the gate read as unloaded. An injected `DbContext` and EF Core 7's asynchronous loader delegate loaded one too, and so did a reshaping lambda that got its row from an application's method or from a captured query or object. A converted column typed `object` was kept whole when a projection was built for another field, and an application's own non-generic collection hid its own denied members. A field a subtype declares — a derived entity's, a subclass's held by a base-typed member, an open generic one's — was not read at all, nor was a `[DwDenied]` on an override, on a member hidden with `new` or on an interface member's implementation, and under a `"*"` deny with exact allows a path the walk never asked about (past four segments, around a cycle, with no setter) resolved as allowed. `Selects` naming an entity navigation returned a denial in its owned chain past four segments or in a converted `Dictionary` column. All of these are read now: from the EF Core model for an entity, so only what loads counts, and from every loaded subtype for a projected or in-memory row. A denial on an override, a member hidden with `new` or an implementation applies to the base type's or the interface's path, on every row and in every clause. +- **Fixed (security): what a query loads, and what a member holds, was read too narrowly.** An include named from the root and reached through `Select(o => o.Customer)`, `SelectMany` or `Join`, a projection behind another `Select`, an initializer after a constructor with arguments, and a lazy loader the constructor takes and keeps in a field or any property each loaded a denied value the gate read as unloaded. An injected `DbContext` and EF Core 7's asynchronous loader delegate loaded one too, and so did a reshaping lambda that got its row from an application's method or from a captured query or object. An application's own collection class hid its own denied members, and a guarded query through a provider wrapping EF Core's, such as LinqKit's `AsExpandable`, ran tracking, so the context filled in navigations it already held and a masked value became a pending change. A field a subtype declares — a derived entity's, a subclass's held by a base-typed member, an open generic one's — was not read at all, nor was a `[DwDenied]` on an override, on a member hidden with `new` or on an interface member's implementation, and under a `"*"` deny with exact allows a path the walk never asked about (past four segments, around a cycle, with no setter) resolved as allowed. `Selects` naming an entity navigation returned a denial in its owned chain past four segments or in a converted `Dictionary` column. All of these are read now: from the EF Core model for an entity, so only what loads counts, and from every loaded subtype for a projected or in-memory row. A denial on an override, a public member hidden with `new` or an implementation, through a variant instantiation too, applies to the base type's or the interface's path, on every row and in every clause. - **Fixed (security): a denied member that holds no simple value came back.** A field denied at the top of the type whose own type is not a simple value — a byte array, a list, an owned object, a JSON column — synthesized no projection either, so with nothing else denied it came back. - **Fixed (security): an application namespace starting with `System` got no policy.** The attribute walker read any namespace starting with "System" as the framework's, so an application namespace such as `SystemsCorp.Payroll` got no policy beneath its types, and a `[DwDenied]` field there was returned, filterable and sortable. Only `System` and the namespaces beneath it are the framework's now. - **Fixed (security): a navigation narrowed around its own denied key got the key back.** Under the convenience tier, `Selects` naming a navigation whose key (`Id`) is denied was narrowed to the allowed fields beneath it, and the core's typed projection added the key back. Such a narrowing is refused with `FieldDeniedForSelect` in both tiers, as naming a sibling of the key already was. A navigation named through another, such as `Main.Lead`, now gates the key of `Main`, which the projection adds; it did not. - **Fixed (security): `Selects` could name a member whose denials the gate did not see.** A member typed as a collection the core does not unwrap — `IReadOnlyList`, `IReadOnlyCollection`, `Collection` or an application's own — returned every field beneath it, denied ones included, in both tiers, because the projection gate read collections through a narrower list than the attribute walker. It reads them the same way now, and a narrowing the core cannot project is refused with `FieldDeniedForSelect`. Denials beneath a named member are also read from the policy's own rules, so a denied property with no setter and a rule on a path reached through a cycle are found. A member carrying a field denied where no path reaches it — deeper than the walker, inside a framework collection such as `Dictionary`, or on a subtype — is refused under the strict tier, and under the convenience tier narrowed where the core can narrow it and refused where it cannot. -- **Changed: the synthesized projection keeps what the source carries.** It kept simple fields only, so every nested object and list of a row projected before `ApplyPolicy` came back null or empty as soon as any field was denied. A row a projection builds — the outermost `Select` constructs it, in an object initializer or with a constructor, as in `db.Roles.Select(r => new RoleRow { … })` — keeps the members its initializer assigns. An entity, or a `Select` that hands back an entity such as `db.Orders.Select(o => o.Customer)`, keeps its mapped columns, converted and JSON ones included, its owned and complex members, and every collection of simple values such as `byte[]` or `List`. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, recorded as `Dropped` with a reason starting `left out whole`. An entity's navigations, the objects of a row in memory, and a value EF Core does not map are left out, and the type needs a public parameterless constructor for the typed projection, as it already did. When a derived type in the model, or a loaded subclass of a row in memory, declares a denied field, the rows come back as the queried type, so a derived type's allowed fields are dropped too; query the derived type with `OfType()` to keep them. A member that can hold an object of any type, a geometry, a JSON bag or a `BitArray` column say, asks for no projection on its own, and a chain that reaches its rows through a navigation, `SelectMany`, `Join` or `GroupBy` counts every navigation as loaded only when it has an include or builds an object. +- **Changed: the synthesized projection keeps what the source carries.** It kept simple fields only, so every nested object and list of a row projected before `ApplyPolicy` came back null or empty as soon as any field was denied. A row a projection builds — the outermost `Select` constructs it, in an object initializer or with a constructor, as in `db.Roles.Select(r => new RoleRow { … })` — keeps the members its initializer assigns. An entity, or a `Select` that hands back an entity such as `db.Orders.Select(o => o.Customer)`, keeps its mapped columns, converted and JSON ones included except a converted one that can hold an object of any type, its owned and complex members, and every collection of simple values such as `byte[]` or `List`. A member holding an object is kept whole when nothing it can hold is denied, narrowed to the allowed fields where the core's narrowing translates, and otherwise left out whole, recorded as `Dropped` with a reason starting `left out whole`. An entity's navigations, the objects of a row in memory, and a value EF Core does not map are left out, and the type needs a public parameterless constructor for the typed projection, as it already did. When a derived type in the model, or a loaded subclass of a row in memory, declares a denied field, the rows come back as the queried type, so a derived type's allowed fields are dropped too; query the derived type with `OfType()` to keep them. A member that can hold an object of any type, a geometry or a JSON bag say, asks for no projection on its own, and a chain that reaches its rows through a navigation, `SelectMany`, `Join` or `GroupBy` counts every navigation as loaded only when it has an include, or a lambda that builds an object, gets one from an application's method, or captures a query with its own include or projection. - **Changed: `[DwEntity(DefaultOrder)]` reaches a projection that builds the row.** A guarded query over a projected source takes the default when the outermost `Select` builds the type in an object initializer and assigns every field the default names a column, at every level of a nested path — a mapped member read directly, through reference navigations or through `EF.Property`: `Select(t => new TicketRow { Id = t.Id, CreatedAt = t.CreatedAt })` for `"CreatedAt desc, Id"`. A computed value or any other projection still leaves the query in its own order. A `Select`, or a `Filter` with `Selects`, composed on the guarded handle keeps the rest of the chain unordered, and a composed `Filter` that sent orders gets no default later in the chain, as a composed `Order` already did not. - **Changed: the async dynamic `Filter` and the async `Summary` read through EF Core.** `ToListAsyncDynamic` and `ToListAsync(Summary)` read with EF Core's `ToListAsync` instead of Dynamic LINQ's `ToDynamicListAsync`, which had no token to pass on, and the summary counts with `CountAsync` where it counted synchronously. So on an EF Core query a canceled token now reaches the database. A provider that is not EF Core's keeps Dynamic LINQ's read, on the calling thread. - **New: a `CancellationToken` on every async terminal**, guarded and unguarded: `ToListAsync` and `ToListAsyncDynamic` with a `Filter`, `ToListAsync` with a `Summary`, and `ToListAsync` with a `Segment`. The token reaches the count and the read. The overloads sit beside the 3.1 signatures, which are unchanged, so code compiled against 3.1 still binds. `ToListAsync(filter, default)`, `ToListAsyncDynamic(filter, default)` and `ToListAsync(summary, default)` no longer compile, because `default` fits both `getQueryString` and the token: write `false`, a token, or a named argument. A reflection lookup of `ToListAsyncDynamic` by name alone now finds three methods where it found one, and one of `ToListAsync` finds more than it did. From c5e01cb56a1d5eb1b1c3aada9f433efac953495a Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 11:29:15 +0300 Subject: [PATCH 21/22] fix(policies): read an inline query where it stands, and keep a collection of values a value A join on a filtered set held its inner query inline, over a root EF Core put in the tree, and the evaluator added in 689d70e compiled it, failed on the root, and counted the chain as building: included data was dropped, and DDD, constructor-bound and abstract rows threw. A query the tree already holds is read where it stands again. An application's collection of values stays a value unless a member of its own is denied, at any collection layer, so rows in memory and a "*" deny keep it as before; one with a denied member of its own is read as an object. Members sharing a name on an entity are read from what the query loads. Rows in memory are projected when a member a base type declares, and the row type hides with new, is denied. Co-Authored-By: Claude Opus 5 --- .../Policies/ReviewGateTests.cs | 240 ++++++ .../Policies/ReviewJoinEvalTests.cs | 694 ++++++++++++++++++ .../Policies/ReviewJoinRootTests.cs | 224 ++++++ .../Policies/ReviewSharedNameEfTests.cs | 300 ++++++++ .../Policies/ReviewTrackingWrapperTests.cs | 332 +++++++++ .../Resolution/AttributePolicyProvider.cs | 25 + .../Policies/Source/FilterSanitizer.cs | 98 ++- DynamicWhere.ex/Source/QueryRoot.cs | 12 +- 8 files changed, 1908 insertions(+), 17 deletions(-) create mode 100644 DynamicWhere.Tests/Policies/ReviewGateTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewJoinEvalTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewJoinRootTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewSharedNameEfTests.cs create mode 100644 DynamicWhere.Tests/Policies/ReviewTrackingWrapperTests.cs diff --git a/DynamicWhere.Tests/Policies/ReviewGateTests.cs b/DynamicWhere.Tests/Policies/ReviewGateTests.cs new file mode 100644 index 0000000..d769b78 --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewGateTests.cs @@ -0,0 +1,240 @@ +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using Xunit.Abstractions; + +// An application's own collection classes, members sharing a name or hidden with new, and the framework's value collections. + +namespace DynamicWhere.Tests.Policies +{ + /// An application list of strings with a plain member of its own (a paged list, a tag set with a note). + public class ZwLabelSet : List + { + public string? Note { get; set; } + } + + /// The ASP.NET Core tutorial's PaginatedList shape, over strings. + public class ZwPagedNames : List + { + public int PageIndex { get; set; } + + public bool HasNextPage => false; + } + + /// An application list of strings whose own member is denied. + public class ZwSecretTags : List + { + [DwDenied] + public string? OwnerSecret { get; set; } + } + + public class ZwLabelRow + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + [DwDenied] + public string? Ssn { get; set; } + + public ZwLabelSet Labels { get; set; } = new(); + + public ZwPagedNames Names { get; set; } = new(); + } + + public class ZwPlainLabelRow + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public ZwLabelSet Labels { get; set; } = new(); + } + + public class ZwNestedTagRow + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public List Sets { get; set; } = new(); + + public ZwSecretTags[] Arr { get; set; } = Array.Empty(); + } + + public class ZwDeclaredFrameworkRow + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public List Tags { get; set; } = new(); + + public IEnumerable Seq { get; set; } = Array.Empty(); + } + + public class ZwCaseRow + { + public int Id { get; set; } + + public string? Code { get; set; } + + [DwDenied] + public string? CODE { get; set; } + } + + public class ZwHideValueBase + { + public int Id { get; set; } + + [DwDenied] + public string? Code { get; set; } + } + + public class ZwHideValueDerived : ZwHideValueBase + { + public new string? Code { get; set; } + } + + public class ZwItemsBase + { + public int Id { get; set; } + + public List? Items { get; set; } + } + + public class ZwItemsDerived : ZwItemsBase + { + public new List? Items { get; set; } + } + + public sealed class ReviewGateTests + { + private readonly ITestOutputHelper _out; + + public ReviewGateTests(ITestOutputHelper output) => _out = output; + + private static PolicyQueryable GuardWith(IQueryable source, DwTier tier, params IDwPolicyProvider[] more) where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = tier, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() }.Concat(more).ToArray())); + + private object? Read(string label, IQueryable source, DwTier tier, Filter? filter = null, bool dynamic = false, params IDwPolicyProvider[] more) where T : class + { + PolicyQueryable guarded = GuardWith(source, tier, more); + object? data = null; + string code = ZwKit.Code(() => data = dynamic ? guarded.ToListDynamic(filter ?? new Filter()).Data : guarded.ToList(filter ?? new Filter()).Data); + + _out.WriteLine($"{label} {tier} dynamic={dynamic}: code={code} sent={ZwKit.Json(data)} trace=[{ZwKit.Trace(guarded)}]"); + + return data; + } + + // ------------------------------------------------------------------ O: OwnsMembers on everyday list subclasses + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zw_O1_a_list_of_strings_subclass_with_a_plain_member_is_kept_beside_a_top_level_denial(DwTier tier) + { + ZwLabelRow[] rows = + { + new() { Id = 1, Name = "r1", Ssn = "zw-ssn-secret", Labels = new ZwLabelSet { "a", "b" }, Names = new ZwPagedNames { "x", "y" } } + }; + rows[0].Labels.Note = "note"; + + object? data = Read("O1", rows.AsQueryable(), tier); + List sent = Assert.IsType>(data); + + Assert.False(ZwKit.Holds(data, "zw-ssn-secret")); + Assert.Equal(2, sent[0].Labels.Count); + Assert.Equal(2, sent[0].Names.Count); + } + + [Fact] + public void Zw_O2_under_a_star_deny_a_list_of_strings_subclass_granted_by_name_is_kept() + { + ZwPlainLabelRow[] rows = { new() { Id = 1, Name = "r1", Labels = new ZwLabelSet { "a", "b" } } }; + + FakePolicyProvider rules = new FakePolicyProvider() + .Add("*", PolicyFeature.Select, PolicyEffect.Deny, PolicyLevel.DynamicGlobal) + .Add("Id", PolicyFeature.Select, PolicyEffect.Allow, PolicyLevel.DynamicGlobal) + .Add("Name", PolicyFeature.Select, PolicyEffect.Allow, PolicyLevel.DynamicGlobal) + .Add("Labels", PolicyFeature.Select, PolicyEffect.Allow, PolicyLevel.DynamicGlobal); + + object? whole = Read("O2 none", rows.AsQueryable(), DwTier.Convenience, null, false, rules); + object? named = Read("O2 named", rows.AsQueryable(), DwTier.Convenience, new Filter { Selects = new List { "Id", "Labels" } }, false, rules); + + List a = Assert.IsType>(whole); + List b = Assert.IsType>(named); + + Assert.Equal(2, a[0].Labels.Count); + Assert.Equal(2, b[0].Labels.Count); + } + + // ------------------------------------------------------------------ K: an application collection's own members, one level removed + + [Theory] + [InlineData(DwTier.Strict, false)] + [InlineData(DwTier.Convenience, false)] + [InlineData(DwTier.Strict, true)] + public void Zw_K1_a_list_and_an_array_of_a_collection_class_with_a_denied_member(DwTier tier, bool dynamic) + { + ZwSecretTags tags = new() { "a" }; + tags.OwnerSecret = "zw-nested-owner-secret"; + + ZwNestedTagRow[] rows = { new() { Id = 1, Name = "r1", Sets = new List { tags }, Arr = new[] { tags } } }; + + object? data = Read("K1", rows.AsQueryable(), tier, null, dynamic); + + Assert.False(ZwKit.Holds(data, "zw-nested-owner-secret")); + } + + // ------------------------------------------------------------------ S: shared names + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zw_S1_two_value_members_differing_only_in_case_the_denied_one_is_withheld(DwTier tier) + { + ZwCaseRow[] rows = { new() { Id = 1, Code = "open", CODE = "zw-case-secret" } }; + + object? data = Read("S1", rows.AsQueryable(), tier); + + Assert.False(ZwKit.Holds(data, "zw-case-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zw_S2_a_denied_value_member_hidden_with_new_by_another_value(DwTier tier) + { + ZwHideValueDerived row = new() { Id = 1, Code = "open" }; + ((ZwHideValueBase)row).Code = "zw-hidden-base-secret"; + + object? data = Read("S2", new[] { row }.AsQueryable(), tier); + + Assert.False(ZwKit.Holds(data, "zw-hidden-base-secret")); + } + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zw_S3_a_list_of_values_hidden_by_a_list_of_cards(DwTier tier) + { + ZwItemsDerived row = new() { Id = 1, Items = new List { new() { Label = "visa", Pan = "zw-items-pan-secret" } } }; + ((ZwItemsBase)row).Items = new List { "a" }; + + object? data = Read("S3", new[] { row }.AsQueryable(), tier); + object? named = Read("S3 named", new[] { row }.AsQueryable(), tier, new Filter { Selects = new List { "Id", "Items" } }); + + Assert.False(ZwKit.Holds(data, "zw-items-pan-secret")); + Assert.False(ZwKit.Holds(named, "zw-items-pan-secret")); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewJoinEvalTests.cs b/DynamicWhere.Tests/Policies/ReviewJoinEvalTests.cs new file mode 100644 index 0000000..48ae8e1 --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewJoinEvalTests.cs @@ -0,0 +1,694 @@ +using System.Collections; +using System.Data.Common; +using System.Linq.Expressions; +using System.Reflection; +using System.Text.Json; +using System.Text.Json.Serialization; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Config; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Source; +using DynamicWhere.ex.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Diagnostics; +using Xunit.Abstractions; + +// Joins on a filtered, tagged or untracked set, and anonymous carriers: a query the tree holds inline is read where it +// stands, not evaluated, so these read as plain chains. + +namespace DynamicWhere.Tests.Policies +{ + public class ZwTier + { + public int Id { get; set; } + + public string Label { get; set; } = string.Empty; + } + + public class ZwCustomer + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public string Region { get; set; } = string.Empty; + + public int ZwTierId { get; set; } + + /// Automatically included; nothing beneath it is denied. + public ZwTier? Tier { get; set; } + + /// A converted JSON bag. + public Dictionary Meta { get; set; } = new(); + + /// The only denial is beneath it; nothing includes it. + public List Cards { get; set; } = new(); + + public List Orders { get; set; } = new(); + } + + public class ZwCard + { + public int Id { get; set; } + + public int ZwCustomerId { get; set; } + + public string Label { get; set; } = string.Empty; + + [DwDenied] + public string? Pan { get; set; } + } + + public class ZwChannel + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + } + + public class ZwOrder + { + public int Id { get; set; } + + public string Code { get; set; } = string.Empty; + + public string Region { get; set; } = string.Empty; + + public int ZwCustomerId { get; set; } + + public ZwCustomer? Customer { get; set; } + + public int ZwChannelId { get; set; } + + /// Automatically included. + public ZwChannel? Channel { get; set; } + + public Dictionary Meta { get; set; } = new(); + } + + /// Counts the commands a context sends. + public sealed class ZwCommandCounter : DbCommandInterceptor + { + public int Readers; + + public override InterceptionResult ReaderExecuting(DbCommand command, CommandEventData eventData, InterceptionResult result) + { + Interlocked.Increment(ref Readers); + + return result; + } + } + + public sealed class ZwContext : DbContext + { + private readonly SqliteConnection _connection; + + public ZwContext(SqliteConnection connection, ZwCommandCounter? counter = null) + { + _connection = connection; + Counter = counter; + } + + public ZwCommandCounter? Counter { get; } + + public DbSet Customers => Set(); + + public DbSet Orders => Set(); + + public DbSet Cards => Set(); + + /// An ordinary method on the context (not a mapped function) that returns a query with an include. + public IQueryable CustomersWithCardsNamed(string name) => Customers.Include(c => c.Cards).Where(c => c.Name == name); + + protected override void OnConfiguring(DbContextOptionsBuilder options) + { + options.UseSqlite(_connection); + + if (Counter is not null) + { + options.AddInterceptors(Counter); + } + } + + protected override void OnModelCreating(ModelBuilder model) + { + model.Entity().ToTable("ZwCustomers"); + model.Entity().ToTable("ZwOrders"); + model.Entity().Navigation(c => c.Tier).AutoInclude(); + model.Entity().Navigation(o => o.Channel).AutoInclude(); + model.Entity().Property(c => c.Meta).HasConversion( + value => JsonSerializer.Serialize(value, (JsonSerializerOptions?)null), + text => JsonSerializer.Deserialize>(text, (JsonSerializerOptions?)null)!); + model.Entity().Property(o => o.Meta).HasConversion( + value => JsonSerializer.Serialize(value, (JsonSerializerOptions?)null), + text => JsonSerializer.Deserialize>(text, (JsonSerializerOptions?)null)!); + } + } + + /// A repository whose methods the guard now evaluates. + public sealed class ZwRepository + { + private readonly ZwContext _db; + private IQueryable? _level2; + + public ZwRepository(ZwContext db) => _db = db; + + public int Calls; + + public int FlipCalls; + + /// Counts its calls (a log line, a metric, an audit record). + public IQueryable Customers() + { + Interlocked.Increment(ref Calls); + + return _db.Customers; + } + + /// Resolves which customers the caller may see with a query of its own, then hands back a query. + public IQueryable Visible() + { + Interlocked.Increment(ref Calls); + + List ids = _db.Customers.AsNoTracking().Where(c => c.Region != string.Empty).Select(c => c.Id).ToList(); + + return _db.Customers.Where(c => ids.Contains(c.Id)); + } + + /// A stateful method: the first call hands back the plain set, every later one includes the cards. + public IQueryable Flip() + { + int call = Interlocked.Increment(ref FlipCalls); + + return call == 1 ? _db.Customers : _db.Customers.Include(c => c.Cards); + } + + /// The same as a property getter. + public IQueryable FlipSet + { + get + { + int call = Interlocked.Increment(ref FlipCalls); + + return call == 1 ? _db.Customers : _db.Customers.Include(c => c.Cards); + } + } + + /// A query that captures a built query two levels down. + public IQueryable Level1() + { + _level2 ??= _db.Customers.Select(c => new ZwCustomer { Id = c.Id, Name = c.Name, Region = c.Region, Cards = c.Cards }); + + IQueryable level2 = _level2; + + return _db.Customers.Where(c => c.Name != string.Empty).SelectMany(c => level2.Where(x => x.Id == c.Id)); + } + + /// Customers already in memory, handed out as a query. + public IQueryable InMemory(List held) => held.AsQueryable(); + } + + internal static class ZwKit + { + private static readonly JsonSerializerOptions JsonOptions = new() { ReferenceHandler = ReferenceHandler.IgnoreCycles }; + + internal static PolicyQueryable Guard(IQueryable source, DwTier tier = DwTier.Strict) where T : class => + source.ApplyPolicy( + new DwPolicyContext().WithSubject(DwSubjectKind.User, "u1"), + new DwPolicyOptions { Tier = tier, Caps = { MinGroupSize = 1 } }, + new PolicyResolver(new IDwPolicyProvider[] { new AttributePolicyProvider() })); + + internal static string Shape(Expression expression) + { + string Try(Func read) + { + try + { + return read().ToString(); + } + catch (Exception e) + { + return "threw " + e.GetType().Name; + } + } + + return $"reshapes={Try(() => QueryRoot.Reshapes(expression))} builds={Try(() => QueryRoot.Builds(expression))}"; + } + + internal static string Trace(PolicyQueryable guarded) where T : class => + string.Join(" | ", guarded.LastTrace?.Decisions + .Where(d => d.Action != PolicyAction.Allowed) + .Select(d => $"{d.FieldPath} {d.Action}: {d.Reason}") ?? Array.Empty()); + + internal static bool AnyDropped(PolicyQueryable guarded) where T : class => + guarded.LastTrace?.Decisions.Any(d => d.Action == PolicyAction.Dropped) ?? false; + + internal static string Json(object? value) + { + try + { + return JsonSerializer.Serialize(value, JsonOptions); + } + catch (Exception e) + { + return $""; + } + } + + internal static string Code(Action run) + { + try + { + run(); + + return "ran"; + } + catch (PolicyException refusal) + { + return $"{refusal.ErrorCode}({refusal.FieldPath}; {refusal.SourceOrigin})"; + } + catch (Exception other) + { + return $"{other.GetType().Name}: {other.Message.Split('\n')[0]}"; + } + } + + internal static object? SafeRead(Func read) + { + try + { + return read(); + } + catch (PolicyException) + { + return null; + } + catch (LogicException) + { + return null; + } + } + + /// Everything reachable from a value, read by runtime type (public and non-public properties and fields). + internal static bool Holds(object? value, string text) + { + HashSet seen = new(ReferenceEqualityComparer.Instance); + Stack pending = new(); + + pending.Push(value); + + while (pending.Count > 0) + { + object? current = pending.Pop(); + + if (current is null || current is MemberInfo || current is Delegate + || (current.GetType().Namespace ?? string.Empty).StartsWith("Microsoft.", StringComparison.Ordinal)) + { + continue; + } + + if (current is string held) + { + if (held.Contains(text, StringComparison.Ordinal)) + { + return true; + } + + continue; + } + + if (current.GetType().IsPrimitive || current is decimal || current is DateTime || current is Guid) + { + continue; + } + + if (!current.GetType().IsValueType && !seen.Add(current)) + { + continue; + } + + if (current is IDictionary map) + { + foreach (DictionaryEntry item in map) + { + pending.Push(item.Key); + pending.Push(item.Value); + } + } + else if (current is IEnumerable items) + { + try + { + foreach (object? item in items) + { + pending.Push(item); + } + } + catch + { + // Unreadable. + } + } + + const BindingFlags all = BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance; + + foreach (PropertyInfo property in current.GetType().GetProperties(all)) + { + if (property.GetIndexParameters().Length != 0 || !property.CanRead) + { + continue; + } + + try + { + pending.Push(property.GetValue(current)); + } + catch + { + // Unreadable. + } + } + + for (Type? type = current.GetType(); type is not null && type != typeof(object); type = type.BaseType) + { + if (type.Namespace is { } ns && (ns == "System" || ns.StartsWith("System.", StringComparison.Ordinal) + || ns.StartsWith("Microsoft.", StringComparison.Ordinal))) + { + continue; + } + + foreach (FieldInfo field in type.GetFields(all | BindingFlags.DeclaredOnly)) + { + try + { + pending.Push(field.GetValue(current)); + } + catch + { + // Unreadable. + } + } + } + } + + return false; + } + } + + public sealed class ReviewJoinEvalTests : IDisposable + { + private const string Secret = "zw-pan-secret"; + + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZwCommandCounter _counter = new(); + private readonly ZwContext _db; + private readonly ZwRepository _repo; + + public ReviewJoinEvalTests(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + _db = new ZwContext(_connection, _counter); + _db.Database.EnsureCreated(); + _repo = new ZwRepository(_db); + + ZwTier gold = new() { Label = "gold" }; + ZwChannel web = new() { Name = "web" }; + + ZwCustomer c1 = new() + { + Name = "C1", + Region = "north", + Tier = gold, + Meta = { ["k"] = "v1" }, + Cards = { new ZwCard { Label = "visa", Pan = Secret } } + }; + ZwCustomer c2 = new() { Name = "C2", Region = "south", Tier = gold, Meta = { ["k"] = "v2" } }; + + c1.Orders.Add(new ZwOrder { Code = "O1", Region = "north", Channel = web, Meta = { ["k"] = "o1" } }); + c1.Orders.Add(new ZwOrder { Code = "O2", Region = "south", Channel = web, Meta = { ["k"] = "o2" } }); + c2.Orders.Add(new ZwOrder { Code = "O3", Region = "south", Channel = web, Meta = { ["k"] = "o3" } }); + + _db.Customers.AddRange(c1, c2); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private (List? Rows, PolicyQueryable Guarded) Run(string label, IQueryable source) where T : class + { + string shape = ZwKit.Shape(source.Expression); + PolicyQueryable guarded = ZwKit.Guard(source); + + try + { + int count = source.AsNoTracking().ToList().Count; + + _out.WriteLine($"{label}: unguarded OK rows={count} {shape}"); + } + catch (Exception e) + { + _out.WriteLine($"{label}: unguarded THREW {e.GetType().Name} {e.Message.Split('\n')[0]} {shape}"); + + return (null, guarded); + } + + _db.ChangeTracker.Clear(); + + List rows = guarded.ToList(new Filter()).Data; + + _out.WriteLine($"{label}: guarded rows={rows.Count} trace=[{ZwKit.Trace(guarded)}]"); + + return (rows, guarded); + } + + private void CustomersAsLoaded(string label, IQueryable source, int count) + { + (List? rows, PolicyQueryable guarded) = Run(label, source); + + Assert.NotNull(rows); + Assert.Equal(count, rows!.Count); + Assert.False(ZwKit.AnyDropped(guarded), ZwKit.Trace(guarded)); + Assert.All(rows, row => Assert.Equal("gold", row.Tier?.Label)); + Assert.All(rows, row => Assert.True(row.Meta.ContainsKey("k"), "Meta left out")); + } + + private void OrdersAsLoaded(string label, IQueryable source, int count) + { + (List? rows, PolicyQueryable guarded) = Run(label, source); + + Assert.NotNull(rows); + Assert.Equal(count, rows!.Count); + Assert.False(ZwKit.AnyDropped(guarded), ZwKit.Trace(guarded)); + Assert.All(rows, row => Assert.Equal("web", row.Channel?.Name)); + Assert.All(rows, row => Assert.True(row.Meta.ContainsKey("k"), "Meta left out")); + } + + // ------------------------------------------------------------------ J: joins whose inner sequence is not a bare set + + [Fact] + public void Zw_J0_control_join_on_a_bare_set() + { + CustomersAsLoaded("J0 Join(_db.Customers)", _db.Orders.Join(_db.Customers, o => o.ZwCustomerId, c => c.Id, (o, c) => c), 3); + } + + [Fact] + public void Zw_J1_join_on_a_filtered_set() + { + CustomersAsLoaded( + "J1 Join(_db.Customers.Where(..))", + _db.Orders.Join(_db.Customers.Where(c => c.Region != string.Empty), o => o.ZwCustomerId, c => c.Id, (o, c) => c), + 3); + } + + [Fact] + public void Zw_J2_query_syntax_join_on_an_untracked_set() + { + IQueryable source = + from o in _db.Orders + join c in _db.Customers.AsNoTracking() on o.ZwCustomerId equals c.Id + select c; + + CustomersAsLoaded("J2 join c in _db.Customers.AsNoTracking()", source, 3); + } + + [Fact] + public void Zw_J3_left_join_on_a_filtered_set() + { + IQueryable source = + from o in _db.Orders + join c in _db.Customers.Where(x => x.Region != string.Empty) on o.ZwCustomerId equals c.Id into cs + from c in cs.DefaultIfEmpty() + select c; + + CustomersAsLoaded("J3 left join on a filtered set", source, 3); + } + + [Fact] + public void Zw_J4_join_on_a_filtered_set_returning_the_outer_rows() + { + OrdersAsLoaded( + "J4 Join(_db.Customers.Where(..), (o, c) => o)", + _db.Orders.Join(_db.Customers.Where(c => c.Region != string.Empty), o => o.ZwCustomerId, c => c.Id, (o, c) => o), + 3); + } + + [Fact] + public void Zw_J5_join_on_a_tagged_set() + { + CustomersAsLoaded( + "J5 Join(_db.Customers.TagWith(..))", + _db.Orders.Join(_db.Customers.TagWith("lookup"), o => o.ZwCustomerId, c => c.Id, (o, c) => c), + 3); + } + + [Fact] + public void Zw_J6_where_then_join_on_a_filtered_order_set() + { + OrdersAsLoaded( + "J6 _db.Orders.Where(..).Join(_db.Orders.Where(..), (a, b) => a)", + _db.Orders.Where(o => o.Code != string.Empty) + .Join(_db.Orders.Where(o => o.Region != string.Empty), a => a.Id, b => b.Id, (a, b) => a), + 3); + } + + // ------------------------------------------------------------------ E: what the evaluation of user code costs + + [Fact] + public void Zw_E4_a_repository_query_capturing_a_built_query_two_levels_down() + { + IQueryable source = _db.Orders.SelectMany(o => _repo.Level1().Where(c => c.Id == o.ZwCustomerId)); + string unguarded; + + try + { + unguarded = ZwKit.Holds(source.AsNoTracking().ToList(), Secret) ? "HOLDS PAN" : "no pan"; + } + catch (Exception e) + { + unguarded = $"THREW {e.GetType().Name}: {e.Message.Split('\n')[0]}"; + } + + PolicyQueryable guarded = ZwKit.Guard(source); + object? data; + + try + { + data = ZwKit.SafeRead(() => guarded.ToList(new Filter()).Data); + } + catch (InvalidOperationException) when (typeof(DbContext).Assembly.GetName().Version!.Major < 7) + { + // EF Core 6 cannot translate the projection this shape needs, so the guarded read fails closed. + return; + } + + _out.WriteLine($"E4 {ZwKit.Shape(source.Expression)} unguarded={unguarded} sent={ZwKit.Json(data)} trace=[{ZwKit.Trace(guarded)}]"); + + Assert.False(ZwKit.Holds(data, Secret)); + } + + [Fact] + public void Zw_E5_a_repository_method_handing_out_rows_in_memory() + { + List held = _db.Customers.Include(c => c.Cards).AsNoTracking().ToList(); + IQueryable source = _db.Orders.SelectMany(o => _repo.InMemory(held).Where(c => c.Id == o.ZwCustomerId)); + + string code = ZwKit.Code(() => source.AsNoTracking().ToList()); + PolicyQueryable guarded = ZwKit.Guard(source); + object? data = null; + string guardedCode = ZwKit.Code(() => data = guarded.ToList(new Filter()).Data); + + _out.WriteLine($"E5 {ZwKit.Shape(source.Expression)} unguarded={code} guarded={guardedCode} sent={ZwKit.Json(data)}"); + + Assert.False(ZwKit.Holds(data, Secret)); + } + + [Fact] + public void Zw_E6_a_context_method_that_includes_read_through_a_correlated_argument() + { + IQueryable source = _db.Orders.SelectMany(o => _db.CustomersWithCardsNamed(o.Code)); + + string code = ZwKit.Code(() => source.AsNoTracking().ToList()); + PolicyQueryable guarded = ZwKit.Guard(source); + object? data = null; + string guardedCode = ZwKit.Code(() => data = guarded.ToList(new Filter()).Data); + + _out.WriteLine($"E6 {ZwKit.Shape(source.Expression)} unguarded={code} guarded={guardedCode} sent={ZwKit.Json(data)}"); + + Assert.False(ZwKit.Holds(data, Secret)); + } + + // ------------------------------------------------------------------ A: anonymous carriers under the caller's tracking + + [Theory] + [InlineData("AsTracking")] + [InlineData("IdentityResolution")] + [InlineData("None")] + public void Zw_A1_an_anonymous_carrier_with_the_cards_in_a_context_that_tracks_them(string tracking) + { + // Earlier in the unit of work the application read the cards with tracking. + _ = _db.Cards.ToList(); + + IQueryable orders = tracking switch + { + "AsTracking" => _db.Orders.AsTracking(), + "IdentityResolution" => _db.Orders.AsNoTrackingWithIdentityResolution(), + _ => _db.Orders + }; + + IQueryable source = orders + .Select(o => new { o.Customer, Cards = o.Customer!.Cards.ToList() }) + .Select(x => x.Customer!); + + PolicyQueryable guarded = ZwKit.Guard(source); + object? data = ZwKit.SafeRead(() => guarded.ToList(new Filter()).Data); + + _out.WriteLine($"A1 {tracking} {ZwKit.Shape(source.Expression)} tracked={_db.ChangeTracker.Entries().Count()} sent={ZwKit.Json(data)} trace=[{ZwKit.Trace(guarded)}]"); + + Assert.False(ZwKit.Holds(data, Secret)); + } + + [Theory] + [InlineData("AsTracking")] + [InlineData("IdentityResolution")] + public void Zw_A2_the_carrier_is_the_last_projection_before_a_member_read(string tracking) + { + _ = _db.Cards.ToList(); + + IQueryable orders = tracking == "AsTracking" ? _db.Orders.AsTracking() : _db.Orders.AsNoTrackingWithIdentityResolution(); + + IQueryable source = orders + .Select(o => new { Order = o, Owner = o.Customer, o.Customer!.Cards }) + .Where(x => x.Cards.Count >= 0) + .Select(x => x.Owner!); + + PolicyQueryable guarded = ZwKit.Guard(source); + object? data = ZwKit.SafeRead(() => guarded.ToList(new Filter()).Data); + + _out.WriteLine($"A2 {tracking} {ZwKit.Shape(source.Expression)} sent={ZwKit.Json(data)} trace=[{ZwKit.Trace(guarded)}]"); + + Assert.False(ZwKit.Holds(data, Secret)); + } + + [Fact] + public void Zw_A3_control_the_anonymous_carrier_query_reads_as_it_did() + { + CustomersAsLoaded( + "A3 Select(o => new { o.Customer, o.Code }).Select(x => x.Customer)", + _db.Orders.Select(o => new { o.Customer, o.Code }).Select(x => x.Customer!), + 3); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewJoinRootTests.cs b/DynamicWhere.Tests/Policies/ReviewJoinRootTests.cs new file mode 100644 index 0000000..181b45c --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewJoinRootTests.cs @@ -0,0 +1,224 @@ +using System.Text.Json; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Exceptions; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Xunit.Abstractions; + +// A join on a filtered set over roots a typed projection cannot build (DDD, constructor-bound, abstract): read from the model, not projected. +// The only denial (ZwKey.Secret) sits beneath Owner.Keys, which nothing loads. + +namespace DynamicWhere.Tests.Policies +{ + public class ZwOwner + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public List Keys { get; set; } = new(); + + public List Invoices { get; set; } = new(); + + public List Members { get; set; } = new(); + + public List Payments { get; set; } = new(); + } + + public class ZwKey + { + public int Id { get; set; } + + public int ZwOwnerId { get; set; } + + [DwDenied] + public string? Secret { get; set; } + } + + /// DDD style: a private constructor for EF Core, a public one for the domain, private setters. + public class ZwInvoice + { + private ZwInvoice() + { + } + + public ZwInvoice(string number) => Number = number; + + public int Id { get; private set; } + + public string Number { get; private set; } = string.Empty; + + public int ZwOwnerId { get; private set; } + + public ZwOwner? Owner { get; private set; } + } + + /// Constructor-bound: EF Core binds it, and it has no parameterless constructor. + public class ZwMember + { + public ZwMember(int id, string name, int zwOwnerId) + { + Id = id; + Name = name; + ZwOwnerId = zwOwnerId; + } + + public int Id { get; private set; } + + public string Name { get; private set; } + + public int ZwOwnerId { get; private set; } + + public ZwOwner? Owner { get; private set; } + } + + public abstract class ZwPayment + { + public int Id { get; set; } + + public long Cents { get; set; } + + public int ZwOwnerId { get; set; } + + public ZwOwner? Owner { get; set; } + } + + public class ZwCardPayment : ZwPayment + { + public string Last4 { get; set; } = string.Empty; + } + + public class ZwCashPayment : ZwPayment + { + public string Till { get; set; } = string.Empty; + } + + public sealed class ZwOwnerContext : DbContext + { + private readonly SqliteConnection _connection; + + public ZwOwnerContext(SqliteConnection connection) => _connection = connection; + + public DbSet Owners => Set(); + + public DbSet Payments => Set(); + + public DbSet Members => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + + protected override void OnModelCreating(ModelBuilder model) + { + model.Entity(); + model.Entity(); + model.Entity().HasOne(m => m.Owner).WithMany(o => o.Members).HasForeignKey(m => m.ZwOwnerId); + } + } + + public sealed class ReviewJoinRootTests : IDisposable + { + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZwOwnerContext _db; + + public ReviewJoinRootTests(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + _db = new ZwOwnerContext(_connection); + _db.Database.EnsureCreated(); + + ZwOwner owner = new() + { + Name = "W1", + Keys = { new ZwKey { Secret = "zw-key-secret" } }, + Payments = { new ZwCardPayment { Cents = 100, Last4 = "4242" }, new ZwCashPayment { Cents = 200, Till = "T1" } }, + Invoices = { new ZwInvoice("INV-1") } + }; + + _db.Owners.Add(owner); + _db.SaveChanges(); + _db.Members.Add(new ZwMember(0, "M1", owner.Id)); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private string Typed(string label, IQueryable source) where T : class + { + string unguarded = JsonSerializer.Serialize(source.AsNoTracking().ToList().Select(r => r.GetType().Name)); + PolicyQueryable guarded = ZwKit.Guard(source); + string outcome; + + _db.ChangeTracker.Clear(); + + try + { + List rows = guarded.ToList(new Filter()).Data; + + outcome = $"OK types={JsonSerializer.Serialize(rows.Select(r => r.GetType().Name))} json={JsonSerializer.Serialize(rows)}"; + } + catch (Exception e) + { + outcome = $"THREW {e.GetType().Name} {(e is PolicyException p ? p.ErrorCode.ToString() : string.Empty)} {e.Message.Split('\n')[0]}"; + } + + _out.WriteLine($"{label}: {ZwKit.Shape(source.Expression)} unguarded types={unguarded} guarded {outcome} trace=[{ZwKit.Trace(guarded)}]"); + + return outcome; + } + + [Fact] + public void Zw_D0_control_a_DDD_aggregate_joined_on_a_bare_set() + { + string outcome = Typed("D0", _db.Owners.Join(_db.Set(), o => o.Id, i => i.ZwOwnerId, (o, i) => i)); + + Assert.StartsWith("OK", outcome); + Assert.Contains("INV-1", outcome); + } + + [Fact] + public void Zw_D1_a_DDD_aggregate_joined_on_a_filtered_set() + { + string outcome = Typed("D1", _db.Owners.Join(_db.Set().Where(i => i.Number != string.Empty), o => o.Id, i => i.ZwOwnerId, (o, i) => i)); + + Assert.StartsWith("OK", outcome); + Assert.Contains("INV-1", outcome); + } + + [Fact] + public void Zw_D2_a_constructor_bound_entity_joined_on_a_filtered_set() + { + string outcome = Typed("D2", _db.Owners.Join(_db.Members.Where(m => m.Name != string.Empty), o => o.Id, m => m.ZwOwnerId, (o, m) => m)); + + Assert.StartsWith("OK", outcome); + } + + [Fact] + public void Zw_D3_abstract_rows_joined_on_a_filtered_set() + { + string outcome = Typed("D3", _db.Owners.Join(_db.Payments.Where(p => p.Cents > 0), o => o.Id, p => p.ZwOwnerId, (o, p) => p)); + + Assert.StartsWith("OK", outcome); + Assert.Contains("ZwCardPayment", outcome); + } + + [Fact] + public void Zw_D4_a_captured_filtered_query_as_the_inner_sequence() + { + IQueryable named = _db.Members.Where(m => m.Name != string.Empty); + + string outcome = Typed("D4", _db.Owners.Join(named, o => o.Id, m => m.ZwOwnerId, (o, m) => m)); + + Assert.StartsWith("OK", outcome); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewSharedNameEfTests.cs b/DynamicWhere.Tests/Policies/ReviewSharedNameEfTests.cs new file mode 100644 index 0000000..c97c79c --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewSharedNameEfTests.cs @@ -0,0 +1,300 @@ +using System.Linq.Expressions; +using System.Text.Json; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Context; +using DynamicWhere.ex.Policies.Resolution; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Xunit.Abstractions; + +// Members sharing a name on an EF Core entity, read from what the query loads; a denied value a derived type hides with +// new, read as its base; and a join on a filtered set, which holds its query inline. + +namespace DynamicWhere.Tests.Policies +{ + public class ZwAuthorCard + { + public int Id { get; set; } + + public int ZwAuthorId { get; set; } + + [DwDenied] + public string? Pan { get; set; } + } + + public class ZwAuthor + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + /// Never loaded by any probe; the only denial is beneath it. + public List Cards { get; set; } = new(); + } + + public class ZwDocBase + { + public int Id { get; set; } + + public string Title { get; set; } = string.Empty; + + public int? ZwAuthorId { get; set; } + + public virtual ZwAuthor? Author { get; set; } + } + + /// Re-declares the navigation with new, the same type (to change an annotation, say). + public class ZwDoc : ZwDocBase + { + public new ZwAuthor? Author + { + get => base.Author; + set => base.Author = value; + } + } + + public sealed class ZwDocContext : DbContext + { + private readonly SqliteConnection _connection; + + public ZwDocContext(SqliteConnection connection) => _connection = connection; + + public DbSet Docs => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + } + + public class ZwInfoBase + { + public int Id { get; set; } + + [DwDenied] + public string? Info { get; set; } + } + + public class ZwPlainThing + { + public string Label { get; set; } = string.Empty; + } + + /// Hides the denied value with an object of a clean type. + public class ZwInfoDerived : ZwInfoBase + { + public new ZwPlainThing? Info { get; set; } + } + + /// The same beside a top-level denial, so a projection is built. + public class ZwInfoDerivedDenied : ZwInfoBase + { + [DwDenied] + public string? Ssn { get; set; } + + public new ZwPlainThing? Info { get; set; } + } + + public class ZwVipAuthor : ZwAuthor + { + public string Level { get; set; } = string.Empty; + } + + public class ZwDocBase2 + { + public int Id { get; set; } + + public string Title { get; set; } = string.Empty; + + public int? ZwAuthorId { get; set; } + + public ZwAuthor? Author { get; set; } + } + + /// Re-declares the navigation with new, under a derived type. + public class ZwDoc2 : ZwDocBase2 + { + public new ZwVipAuthor? Author { get; set; } + } + + public sealed class ZwDoc2Context : DbContext + { + private readonly SqliteConnection _connection; + + public ZwDoc2Context(SqliteConnection connection) => _connection = connection; + + public DbSet Docs => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + } + + public sealed class ReviewSharedNameEfTests + { + private readonly ITestOutputHelper _out; + + public ReviewSharedNameEfTests(ITestOutputHelper output) => _out = output; + + [Fact] + public void Zw_N1_an_entity_redeclaring_an_unloaded_navigation_with_new_is_not_projected() + { + using SqliteConnection connection = new("DataSource=:memory:"); + connection.Open(); + + using ZwDocContext db = new(connection); + string model; + + try + { + db.Database.EnsureCreated(); + model = "built"; + } + catch (Exception e) + { + model = $"THREW {e.GetType().Name}: {e.Message.Split('\n')[0]}"; + } + + _out.WriteLine($"N1 model {model}; Author members: " + string.Join(", ", typeof(ZwDoc).GetProperties().Where(p => p.Name == "Author").Select(p => p.DeclaringType!.Name))); + + if (!model.StartsWith("built", StringComparison.Ordinal)) + { + return; + } + + db.Docs.Add(new ZwDoc { Title = "d1", Author = new ZwAuthor { Name = "a1", Cards = { new ZwAuthorCard { Pan = "zw-author-pan" } } } }); + db.SaveChanges(); + db.ChangeTracker.Clear(); + + PolicyQueryable guarded = ZwKit.Guard(db.Docs); + object? data = null; + string code = ZwKit.Code(() => data = guarded.ToList(new Filter()).Data); + + _out.WriteLine($"N1 code={code} sent={ZwKit.Json(data)} trace=[{ZwKit.Trace(guarded)}]"); + + Assert.Equal("ran", code); + Assert.False(ZwKit.AnyDropped(guarded), ZwKit.Trace(guarded)); + } + + [Fact] + public void Zw_N1b_an_entity_redeclaring_a_navigation_with_new_under_a_derived_type() + { + using SqliteConnection connection = new("DataSource=:memory:"); + connection.Open(); + + using ZwDoc2Context db = new(connection); + string model; + + try + { + db.Database.EnsureCreated(); + model = "built: " + string.Join(", ", db.Model.FindEntityType(typeof(ZwDoc2))!.GetNavigations().Select(n => $"{n.Name}->{n.TargetEntityType.ClrType.Name}")); + } + catch (Exception e) + { + model = $"THREW {e.GetType().Name}: {e.Message.Split('\n')[0]}"; + } + + _out.WriteLine($"N1b model {model}"); + + if (!model.StartsWith("built", StringComparison.Ordinal)) + { + return; + } + + db.Docs.Add(new ZwDoc2 { Title = "d1", Author = new ZwVipAuthor { Name = "a1", Level = "gold", Cards = { new ZwAuthorCard { Pan = "zw-author-pan" } } } }); + db.SaveChanges(); + db.ChangeTracker.Clear(); + + PolicyQueryable guarded = ZwKit.Guard(db.Docs); + object? data = null; + string code = ZwKit.Code(() => data = guarded.ToList(new Filter()).Data); + + _out.WriteLine($"N1b code={code} sent={ZwKit.Json(data)} trace=[{ZwKit.Trace(guarded)}]"); + + // The caller includes the author (not its cards, beneath which the only denial sits). + PolicyQueryable included = ZwKit.Guard(db.Docs.Include(d => d.Author)); + List? rows = null; + string code2 = ZwKit.Code(() => rows = included.ToList(new Filter()).Data); + + _out.WriteLine($"N1b Include(d => d.Author): code={code2} sent={ZwKit.Json(rows)} trace=[{ZwKit.Trace(included)}]"); + + Assert.Equal("ran", code); + Assert.False(ZwKit.AnyDropped(guarded), ZwKit.Trace(guarded)); + Assert.Equal("ran", code2); + Assert.Equal("a1", rows![0].Author?.Name); + } + + [Fact] + public void Zw_N2_a_hidden_denied_value_serialized_through_the_base_type() + { + ZwHideValueDerived row = new() { Id = 1, Code = "open" }; + ((ZwHideValueBase)row).Code = "zw-hidden-base-secret"; + + PolicyQueryable guarded = ZwKit.Guard(new[] { row }.AsQueryable()); + List sent = guarded.ToList(new Filter()).Data; + + // An API whose response type is the base class, as a controller returning List does. + string asBase = JsonSerializer.Serialize(sent.Cast().ToList()); + + _out.WriteLine($"N2 as derived={JsonSerializer.Serialize(sent)} as base={asBase} trace=[{ZwKit.Trace(guarded)}] denials(Code)=" + + string.Join(",", new AttributePolicyProvider() + .GetFragments(typeof(ZwHideValueDerived), new DwPolicyContext()) + .Where(f => f.FieldPath == "Code").Select(f => f.Effect))); + + Assert.DoesNotContain("zw-hidden-base-secret", asBase); + } + + [Fact] + public void Zw_N3_why_a_join_on_a_filtered_set_reads_as_building() + { + using SqliteConnection connection = new("DataSource=:memory:"); + connection.Open(); + + using ZwContext db = new(connection); + IQueryable source = db.Orders.Join(db.Customers.Where(c => c.Region != string.Empty), o => o.ZwCustomerId, c => c.Id, (o, c) => c); + MethodCallExpression join = (MethodCallExpression)source.Expression; + Expression inner = join.Arguments[1]; + string evaluated; + + try + { + object? value = Expression.Lambda>(Expression.Convert(inner, typeof(object))).Compile(preferInterpretation: true)(); + + evaluated = "ok " + value?.GetType().Name; + } + catch (Exception e) + { + evaluated = $"THREW {e.GetType().Name}: {e.Message.Split('\n')[0]}"; + } + + _out.WriteLine($"N3 inner={inner.NodeType} {inner.GetType().Name} arg0={((MethodCallExpression)inner).Arguments[0].GetType().Name} evaluated: {evaluated} {ZwKit.Shape(source.Expression)}"); + + // Diagnostic: the inner sequence cannot be compiled on its own (an inline EF Core root is not reducible), and + // the guard must not read that as building. + Assert.Contains("builds=False", ZwKit.Shape(source.Expression)); + } + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public void Zw_N4_a_denied_value_hidden_by_an_object_of_a_clean_type(DwTier tier) + { + ZwInfoDerived row = new() { Id = 1, Info = new ZwPlainThing { Label = "x" } }; + ((ZwInfoBase)row).Info = "zw-info-base-secret"; + ZwInfoDerivedDenied denied = new() { Id = 2, Ssn = "ssn", Info = new ZwPlainThing { Label = "y" } }; + ((ZwInfoBase)denied).Info = "zw-info-base-secret-2"; + + PolicyQueryable guarded = ZwKit.Guard(new[] { row }.AsQueryable(), tier); + object? data = ZwKit.SafeRead(() => guarded.ToList(new Filter()).Data); + PolicyQueryable guarded2 = ZwKit.Guard(new[] { denied }.AsQueryable(), tier); + object? data2 = ZwKit.SafeRead(() => guarded2.ToList(new Filter()).Data); + + string asBase = data is List a ? JsonSerializer.Serialize(a.Cast().ToList()) : "null"; + string asBase2 = data2 is List b ? JsonSerializer.Serialize(b.Cast().ToList()) : "null"; + + _out.WriteLine($"N4 {tier}: as base={asBase} trace=[{ZwKit.Trace(guarded)}]"); + _out.WriteLine($"N4 {tier} projected: as base={asBase2} trace=[{ZwKit.Trace(guarded2)}]"); + + Assert.False(ZwKit.Holds(data, "zw-info-base-secret")); + Assert.False(ZwKit.Holds(data2, "zw-info-base-secret-2")); + } + } +} diff --git a/DynamicWhere.Tests/Policies/ReviewTrackingWrapperTests.cs b/DynamicWhere.Tests/Policies/ReviewTrackingWrapperTests.cs new file mode 100644 index 0000000..301ac80 --- /dev/null +++ b/DynamicWhere.Tests/Policies/ReviewTrackingWrapperTests.cs @@ -0,0 +1,332 @@ +using System.Collections; +using System.Linq.Expressions; +using DynamicWhere.ex.Classes.Complex; +using DynamicWhere.ex.Classes.Core; +using DynamicWhere.ex.Enums; +using DynamicWhere.ex.Policies.Attributes; +using DynamicWhere.ex.Policies.Enums; +using DynamicWhere.ex.Policies.Source; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Query; +using Xunit.Abstractions; + +// A guarded query through a provider wrapping EF Core's runs untracked: AsNoTracking goes into the query itself. + +namespace DynamicWhere.Tests.Policies +{ + public class ZwShopper + { + public int Id { get; set; } + + public string Name { get; set; } = string.Empty; + + public List Cards { get; set; } = new(); + } + + public class ZwPayCard + { + public int Id { get; set; } + + public int ZwShopperId { get; set; } + + public string Label { get; set; } = string.Empty; + + [DwDenied] + public string? Pan { get; set; } + } + + public class ZwNote + { + public int Id { get; set; } + + public string Title { get; set; } = string.Empty; + + [DwMask(MaskStrategy.Full)] + public string Body { get; set; } = string.Empty; + } + + public sealed class ZwShopContext : DbContext + { + private readonly SqliteConnection _connection; + + public ZwShopContext(SqliteConnection connection) => _connection = connection; + + public DbSet Shoppers => Set(); + + public DbSet Cards => Set(); + + public DbSet Notes => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder options) => options.UseSqlite(_connection); + } + + /// A wrapper that forwards to EF Core's provider, async included, as LinqKit's EF Core build does. + public sealed class ZwAsyncQuery : IOrderedQueryable, IAsyncEnumerable + { + private readonly IQueryable _inner; + + public ZwAsyncQuery(IQueryable inner) + { + _inner = inner; + Provider = new ZwAsyncProvider(inner.Provider); + } + + public Type ElementType => typeof(T); + + public Expression Expression => _inner.Expression; + + public IQueryProvider Provider { get; } + + public IEnumerator GetEnumerator() => _inner.GetEnumerator(); + + IEnumerator IEnumerable.GetEnumerator() => GetEnumerator(); + + public IAsyncEnumerator GetAsyncEnumerator(CancellationToken cancellationToken = default) => + ((IAsyncEnumerable)_inner).GetAsyncEnumerator(cancellationToken); + } + + public sealed class ZwAsyncProvider : IAsyncQueryProvider + { + private readonly IQueryProvider _inner; + + public ZwAsyncProvider(IQueryProvider inner) => _inner = inner; + + public IQueryable CreateQuery(Expression expression) + { + IQueryable created = _inner.CreateQuery(expression); + + return (IQueryable)Activator.CreateInstance(typeof(ZwAsyncQuery<>).MakeGenericType(created.ElementType), created)!; + } + + public IQueryable CreateQuery(Expression expression) => + new ZwAsyncQuery(_inner.CreateQuery(expression)); + + public object? Execute(Expression expression) => _inner.Execute(expression); + + public TResult Execute(Expression expression) => _inner.Execute(expression); + + public TResult ExecuteAsync(Expression expression, CancellationToken cancellationToken = default) => + ((IAsyncQueryProvider)_inner).ExecuteAsync(expression, cancellationToken); + } + + /// + /// A wrapper whose expression is a constant of itself, which its provider swaps for the inner query's before + /// handing it on: the EF Core root is not in the tree the guard reads. + /// + public sealed class ZwConstantRootQuery : IOrderedQueryable + { + public ZwConstantRootQuery(IQueryable inner, Expression? expression = null) + { + Inner = inner; + Expression = expression ?? Expression.Constant(this); + Provider = new ZwConstantRootProvider(this); + } + + internal IQueryable Inner { get; } + + public Type ElementType => typeof(T); + + public Expression Expression { get; } + + public IQueryProvider Provider { get; } + + public IEnumerator GetEnumerator() => + Inner.Provider.CreateQuery(ZwConstantRootProvider.Unwrap(Expression)).GetEnumerator(); + + IEnumerator IEnumerable.GetEnumerator() => GetEnumerator(); + } + + public sealed class ZwConstantRootProvider : IQueryProvider + { + private readonly ZwConstantRootQuery _root; + + public ZwConstantRootProvider(ZwConstantRootQuery root) => _root = root; + + internal static Expression Unwrap(Expression expression) => new Swap().Visit(expression); + + private sealed class Swap : ExpressionVisitor + { + protected override Expression VisitConstant(ConstantExpression node) => + node.Value is ZwConstantRootQuery query ? query.Inner.Expression : node; + } + + public IQueryable CreateQuery(Expression expression) => + (IQueryable)Activator.CreateInstance( + typeof(ZwConstantRootQuery<>).MakeGenericType(expression.Type.GetGenericArguments()[0]), + _root.Inner.Provider.CreateQuery(Unwrap(expression)), expression)!; + + public IQueryable CreateQuery(Expression expression) => + (IQueryable)CreateQuery(expression); + + public object? Execute(Expression expression) => _root.Inner.Provider.Execute(Unwrap(expression)); + + public TResult Execute(Expression expression) => _root.Inner.Provider.Execute(Unwrap(expression)); + } + + public sealed class ReviewTrackingWrapperTests : IDisposable + { + private const string Secret = "zw-shop-pan-secret"; + + private readonly ITestOutputHelper _out; + private readonly SqliteConnection _connection; + private readonly ZwShopContext _db; + + public ReviewTrackingWrapperTests(ITestOutputHelper output) + { + _out = output; + _connection = new SqliteConnection("DataSource=:memory:"); + _connection.Open(); + _db = new ZwShopContext(_connection); + _db.Database.EnsureCreated(); + + _db.Shoppers.Add(new ZwShopper { Name = "S1", Cards = { new ZwPayCard { Label = "visa", Pan = Secret } } }); + _db.Shoppers.Add(new ZwShopper { Name = "S2" }); + _db.Notes.Add(new ZwNote { Title = "t1", Body = "original-note-body" }); + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + } + + public void Dispose() + { + _db.Dispose(); + _connection.Dispose(); + } + + private static Segment Union() => new() + { + ConditionSets = + { + new ConditionSet + { + Sort = 1, + ConditionGroup = new ConditionGroup + { + Conditions = { new Condition { Sort = 1, Field = "Name", DataType = DataType.Text, Operator = Operator.Equal, Values = { "S1" } } } + } + }, + new ConditionSet + { + Sort = 2, + Intersection = Intersection.Union, + ConditionGroup = new ConditionGroup + { + Conditions = { new Condition { Sort = 1, Field = "Name", DataType = DataType.Text, Operator = Operator.Equal, Values = { "S2" } } } + } + } + } + }; + + private static Summary CountByName() => new() + { + GroupBy = new GroupBy + { + Fields = new List { "Name" }, + AggregateBy = new List { new() { Alias = "Total", Aggregator = Aggregator.Count } } + } + }; + + // ------------------------------------------------------------------ W: a wrapper with async support + + [Theory] + [InlineData(DwTier.Strict)] + [InlineData(DwTier.Convenience)] + public async Task Zw_W1_async_reads_through_a_wrapper_over_a_context_tracking_the_cards(DwTier tier) + { + _ = _db.Cards.ToList(); + int before = _db.ChangeTracker.Entries().Count(); + + IQueryable source = new ZwAsyncQuery(_db.Shoppers); + + object? data = null; + string code = ZwKit.Code(() => data = ZwKit.Guard(source, tier).ToListAsync(new Filter()).GetAwaiter().GetResult().Data); + string async = ZwKit.Code(() => ZwKit.Guard(source, tier).ToListAsyncDynamic(new Filter()).GetAwaiter().GetResult()); + string segment = ZwKit.Code(() => ZwKit.Guard(source, tier).ToListAsync(Union()).GetAwaiter().GetResult()); + string summary = ZwKit.Code(() => ZwKit.Guard(source, tier).ToListAsync(CountByName()).GetAwaiter().GetResult()); + int after = _db.ChangeTracker.Entries().Count(); + + _out.WriteLine($"W1 {tier}: list={code} dynamic={async} segment={segment} summary={summary} tracked {before}->{after} sent={ZwKit.Json(data)}"); + + Assert.Equal("ran", code); + Assert.Equal("ran", async); + Assert.Equal("ran", segment); + Assert.Equal("ran", summary); + Assert.False(ZwKit.Holds(data, Secret)); + } + + [Fact] + public async Task Zw_W2_a_masked_value_read_async_through_a_wrapper_is_never_a_pending_change() + { + IQueryable source = new ZwAsyncQuery(_db.Notes); + + object? data = (await ZwKit.Guard(source).ToListAsync(new Filter())).Data; + + _out.WriteLine($"W2 sent={ZwKit.Json(data)} tracked={_db.ChangeTracker.Entries().Count()} changes={_db.ChangeTracker.HasChanges()}"); + + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + + Assert.Equal("original-note-body", _db.Notes.AsNoTracking().Single().Body); + } + + [Fact] + public async Task Zw_W3_a_masked_value_read_through_a_wrapped_segment_is_never_a_pending_change() + { + IQueryable source = new ZwAsyncQuery(_db.Notes); + + Segment segment = new() + { + ConditionSets = + { + new ConditionSet + { + Sort = 1, + ConditionGroup = new ConditionGroup + { + Conditions = { new Condition { Sort = 1, Field = "Title", DataType = DataType.Text, Operator = Operator.Equal, Values = { "t1" } } } + } + } + } + }; + + object? data = (await ZwKit.Guard(source).ToListAsync(segment)).Data; + + _out.WriteLine($"W3 sent={ZwKit.Json(data)} tracked={_db.ChangeTracker.Entries().Count()} changes={_db.ChangeTracker.HasChanges()}"); + + _db.SaveChanges(); + _db.ChangeTracker.Clear(); + + Assert.Equal("original-note-body", _db.Notes.AsNoTracking().Single().Body); + } + + [Fact] + public void Zw_W4_the_callers_AsTracking_inside_a_wrapper_does_not_outlive_the_guard() + { + _ = _db.Cards.ToList(); + + IQueryable source = new ZwAsyncQuery(_db.Shoppers.AsTracking()); + object? data = ZwKit.SafeRead(() => ZwKit.Guard(source).ToList(new Filter()).Data); + + _out.WriteLine($"W4 sent={ZwKit.Json(data)} tracked={_db.ChangeTracker.Entries().Count()}"); + + Assert.False(ZwKit.Holds(data, Secret)); + } + + // ------------------------------------------------------------------ C: a wrapper whose root the guard cannot see + + [Fact] + public void Zw_C2_shoppers_through_a_wrapper_whose_expression_is_a_constant_of_itself() + { + _ = _db.Cards.ToList(); + + IQueryable source = new ZwConstantRootQuery(_db.Shoppers); + + object? data = null; + string code = ZwKit.Code(() => data = ZwKit.Guard(source).ToList(new Filter()).Data); + + _out.WriteLine($"C2 code={code} sent={ZwKit.Json(data)} tracked={_db.ChangeTracker.Entries().Count()}"); + + Assert.False(ZwKit.Holds(data, Secret)); + } + } +} diff --git a/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs b/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs index 0051ac8..f1c0730 100644 --- a/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs +++ b/DynamicWhere.ex/Policies/Resolution/AttributePolicyProvider.cs @@ -486,6 +486,31 @@ internal static Type Peeled(Type propertyType) return current; } + /// + /// The type a property holds, then each collection layer beneath it, down to the element + /// reaches: List<Tags>, then Tags, then . + /// + internal static IEnumerable Layers(Type propertyType) + { + Type current = Nullable.GetUnderlyingType(propertyType) ?? propertyType; + + yield return current; + + for (int layer = 0; layer < MaxCollectionLayers; layer++) + { + Type? element = ElementTypeOf(current); + + if (element is null) + { + yield break; + } + + current = Nullable.GetUnderlyingType(element) ?? element; + + yield return current; + } + } + /// /// Returns the element type of one collection layer — the element of an array, or the T /// of the first a type implements — or null when the type is not a diff --git a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs index ee83e2a..ede7cac 100644 --- a/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs +++ b/DynamicWhere.ex/Policies/Source/FilterSanitizer.cs @@ -1913,7 +1913,7 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) // Two members share the name, one hidden with new under another type or spelled in another case: // the core reads one of them, and a row carries both. What either can hold is asked about, and a // projection, which cannot tell them apart, leaves the name out. - if (gate.SharedName(name) is { Shared: true } shared + if (gate.SharedName(name, rows) is { Shared: true } shared && (shared.Denies || gate.Beneath(name, PolicyFeature.None).Denied.Any(d => rows.Materializes(d.Path)))) { anyDenied = true; @@ -2010,6 +2010,18 @@ private static void SynthesizeSelects(Filter filter, Gate gate, RowShape rows) gate.LeaveOut(path, "left out: a type derived from the row's type declares it, and the projection builds the row's type"); } + // A member a base type declares and T hides with new is still held by a row in memory, and read by + // anyone reading the row as the base type. A projection builds T and leaves it out. + if (rows.Kind == RowKind.InMemory) + { + foreach (string name in gate.HiddenDenials()) + { + anyDenied = true; + + gate.LeaveOut(name, "left out: a base type's member of this name, which the row's own member hides"); + } + } + // A member that asks for nothing itself is still left out, or narrowed, as the projection would build // it. With no projection built nothing was, and the trace says what the query did. if (!anyDenied) @@ -2740,11 +2752,12 @@ internal IEnumerable Members() /// internal static bool HoldsValue(Type type) => CacheReflection.IsSimpleType(AttributePolicyProvider.Peeled(type)) - && !OwnsMembers(Nullable.GetUnderlyingType(type) ?? type); + && !AttributePolicyProvider.Layers(type).Any(layer => OwnsMembers(layer) && Facts(layer).DeniesSelect); /// /// True for an application's own collection class, one deriving from List<string> or a - /// Dictionary say, that declares members beside the elements it holds. + /// Dictionary say, that declares members beside the elements it holds. A collection of values + /// stays a value unless a member of its own is denied. /// private static bool OwnsMembers(Type type) => type.IsClass @@ -2756,6 +2769,47 @@ private static bool OwnsMembers(Type type) => && property.DeclaringType is { } declaring && !AttributePolicyProvider.IsFramework(declaring)); + /// + /// The names of members a base type of T declares, T hides with new, and this caller may not have, + /// or that hold a denied field no path names. + /// + internal List HiddenDenials() + { + List found = new(); + PropertyInfo[] shown = _entityType.GetProperties(BindingFlags.Public | BindingFlags.Instance); + + for (Type? declaring = _entityType.BaseType; declaring is not null && declaring != typeof(object); declaring = declaring.BaseType) + { + foreach (PropertyInfo hidden in declaring.GetProperties(BindingFlags.Public | BindingFlags.Instance | BindingFlags.DeclaredOnly)) + { + if (!hidden.CanRead + || hidden.GetIndexParameters().Length != 0 + || found.Contains(hidden.Name, StringComparer.Ordinal) + || shown.Any(member => member.Name == hidden.Name && SameSlot(member, hidden))) + { + continue; + } + + if (DeclaresDenial(hidden) + || (!HoldsValue(hidden.PropertyType) && Carried(hidden.PropertyType, hidden.Name, exact: false).DeniesSelect)) + { + found.Add(hidden.Name); + } + } + } + + return found; + } + + /// True when two properties' getters are one method, or one overrides the other. + private static bool SameSlot(PropertyInfo left, PropertyInfo right) => + left.GetGetMethod() is { } first + && right.GetGetMethod() is { } second + && first.GetBaseDefinition() is { } a + && second.GetBaseDefinition() is { } b + && a.MetadataToken == b.MetadataToken + && a.Module == b.Module; + /// The names more than one readable member of a type shares, compared as the core compares names. private static readonly ConcurrentDictionary> SharedNames = new(); @@ -2775,7 +2829,7 @@ private static HashSet ReadSharedNames(Type type) => /// A member hidden with new under another type, or two names differing only in case. The core /// reads one of them by name, and a row carries every one, which a serializer writes. /// - internal (bool Shared, bool Denies) SharedName(string name) + internal (bool Shared, bool Denies) SharedName(string name, RowShape rows) { if (!SharedNames.GetOrAdd(_entityType, ReadSharedNames).Contains(name)) { @@ -2794,9 +2848,14 @@ private static HashSet ReadSharedNames(Type type) => return (false, false); } - return (true, variants.Exists(variant => Denies(name, variant) - || (!HoldsValue(variant.PropertyType) - && Carried(variant.PropertyType, name, exact: false).DeniesSelect))); + // An entity's model says which of them EF Core maps and what loads beneath it; any other row can + // hold what either type can. + bool beneath = rows.LoadedBeneath(name) is not null + ? Unnamed(name, rows).DeniesSelect + : variants.Exists(variant => !HoldsValue(variant.PropertyType) + && Carried(variant.PropertyType, name, exact: false).DeniesSelect); + + return (true, beneath || variants.Exists(variant => Denies(name, variant))); } /// @@ -3287,6 +3346,12 @@ private bool Granted(Type type, string path, bool exact) private bool Grant(Type type, string path, bool exact, HashSet onPath) { + // A value, a collection of them among values, holds no path to name. + if (HoldsValue(type)) + { + return true; + } + if (AttributePolicyProvider.NavigationTypeOf(type) is not { } node) { TypeFacts facts = Facts(type, exact: false); @@ -3399,18 +3464,21 @@ void Subtypes(Type of) void Enqueue(Type candidate, bool root) { Type peeled = AttributePolicyProvider.Peeled(candidate); - Type unwrapped = Nullable.GetUnderlyingType(candidate) ?? candidate; - // An application's own collection class declares members beside the elements it holds. - if (OwnsMembers(unwrapped)) + // An application's own collection class declares members beside the elements it holds, at any + // layer: the member's own type, or the element of a list or an array of it. + foreach (Type layer in AttributePolicyProvider.Layers(candidate)) { - holdsMembers = true; + if (OwnsMembers(layer)) + { + holdsMembers = true; - Read(unwrapped); + Read(layer); - if (!(root && exact)) - { - Subtypes(unwrapped); + if (!(root && exact)) + { + Subtypes(layer); + } } } diff --git a/DynamicWhere.ex/Source/QueryRoot.cs b/DynamicWhere.ex/Source/QueryRoot.cs index f8bb6ce..850c3c0 100644 --- a/DynamicWhere.ex/Source/QueryRoot.cs +++ b/DynamicWhere.ex/Source/QueryRoot.cs @@ -282,7 +282,8 @@ private static bool IsAnonymous(Type type) => /// /// True for a call EF Core evaluates before it translates the query: one that reads no parameter of the - /// lambdas around it and hands back a query or an expression, which the query then holds. + /// lambdas around it and hands back a query or an expression, which the query then holds. A query the + /// tree already holds inline, over a root EF Core put there, is read where it stands. /// private static bool Evaluable(MethodCallExpression call) { @@ -298,7 +299,7 @@ private static bool Evaluable(MethodCallExpression call) return !parameters.Found; } - /// Finds a parameter of a lambda outside the expression it is given. + /// Finds a parameter of a lambda outside the expression it is given, or a query root. private sealed class ParameterFinder : ExpressionVisitor { private readonly HashSet _declared = new(); @@ -320,6 +321,13 @@ protected override Expression VisitParameter(ParameterExpression node) return node; } + + protected override Expression VisitExtension(Expression node) + { + Found = true; + + return node; + } } /// Runs a call EF Core would evaluate itself, and reads what it returns. From b61a499883d1cb0780ff225b0be3f92052452bff Mon Sep 17 00:00:00 2001 From: Sajjad Hussain Date: Sat, 19 Sep 2026 11:29:15 +0300 Subject: [PATCH 22/22] docs: collections of values, hidden base members, and the limits the last review left open A repository or specification method runs once more per guarded read, a wrapper that hides the EF Core root runs tracking, a framework-typed member holding an application's collection class is read as the framework type, and EF Core 6 cannot translate some projections a reshaped query needs. Co-Authored-By: Claude Opus 5 --- DynamicWhere.ex/DOC.md | 2 +- .../app/docs/policies/configuration/page.tsx | 3 ++- OfficialWebsite/public/llms.txt | 19 ++++++++++++++++--- README.md | 2 +- 4 files changed, 20 insertions(+), 6 deletions(-) diff --git a/DynamicWhere.ex/DOC.md b/DynamicWhere.ex/DOC.md index 2e68935..b09bbbd 100644 --- a/DynamicWhere.ex/DOC.md +++ b/DynamicWhere.ex/DOC.md @@ -1752,7 +1752,7 @@ The last one is recorded on the field beneath the member that the narrowing leav - A dry run synthesizes nothing. It records the denials and returns the rows whole. - A simulation has no source, so it reads `T` as a source it cannot see into: every denial beneath a member counts, and the projection it shows keeps only members holding a value (see [Administration](#administration)). -**What the policy cannot see into.** A member typed `object`, a framework interface or a collection that is not generic, such as `IEnumerable`, `ArrayList` or an application's own, is opaque to the policy. It never asks for a projection; when one is needed anyway, a projected row, a row in memory, and an entity's converted column leave it out, and an entity's other columns keep it; and naming it returns whatever it holds. A converter returning an application type through a column typed `object` is opaque the same way, so type the member as what it holds. `BitArray` and the framework's string collections hold values. An application's own collection class, generic or not, still has its own members read. Two members sharing a name, one hidden with `new` under another type or spelled in another case, are left out when either holds a denial: the core reads one of them, and a row carries both. A framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: naming it is refused in both tiers where the core cannot narrow it, at the top of `T` or on a row in memory, narrowed away under `Convenience` beneath a navigation, and a synthesized projection leaves it out. Hold such values in a list of the policed type instead. A member EF Core does not map is read as its type, since its getter can hand out what EF Core loaded; a getter that copies a denied column into a type with no denial is the application's to withhold. +**What the policy cannot see into.** A member typed `object`, a framework interface or a collection that is not generic, such as `IEnumerable`, `ArrayList` or an application's own, is opaque to the policy. It never asks for a projection; when one is needed anyway, a projected row, a row in memory, and an entity's converted column leave it out, and an entity's other columns keep it; and naming it returns whatever it holds. A converter returning an application type through a column typed `object` is opaque the same way, so type the member as what it holds. `BitArray` and the framework's string collections hold values. An application's own collection class, generic or not, still has its own members read, and a collection of values stays a value unless one of them is denied. Two members sharing a name, one hidden with `new` under another type or spelled in another case, are left out when either holds a denial: the core reads one of them, and a row carries both. Rows in memory are projected when a member a base type declares, and the row type hides with `new`, is denied. A method in a reshaping lambda that builds a query from captured values runs once more per guarded read, and one that answers differently on each call is enforced as it answered the guard. A framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: naming it is refused in both tiers where the core cannot narrow it, at the top of `T` or on a row in memory, narrowed away under `Convenience` beneath a navigation, and a synthesized projection leaves it out. Hold such values in a list of the policed type instead. A member EF Core does not map is read as its type, since its getter can hand out what EF Core loaded; a getter that copies a denied column into a type with no denial is the application's to withhold. **A projection builds the declared type.** A query over the root of a hierarchy whose derived type declares a denied field comes back as root-type rows, the derived types' allowed fields dropped too. Over an abstract root the typed terminals fail with `SelectTypeMustHaveParameterlessConstructor`, and the dynamic ones return the root's members. Query the derived type, `OfType()`, to keep its fields. Rows in memory can be any loaded subtype, and the policy does not look at the rows: when a subtype declares a denied field, they are projected and their objects left out, even if no row is that subtype. Under a `"*"` deny, a member is kept whole only when every path beneath it the walk skips is one the policy names. A `[DwDenied]` on an override, on a public member a subtype hides with `new`, or on a class's implementation of an interface member, through a variant instantiation too, applies to the path through the base type or the interface, on every row and in every clause. diff --git a/OfficialWebsite/app/docs/policies/configuration/page.tsx b/OfficialWebsite/app/docs/policies/configuration/page.tsx index b1a1601..8867c36 100644 --- a/OfficialWebsite/app/docs/policies/configuration/page.tsx +++ b/OfficialWebsite/app/docs/policies/configuration/page.tsx @@ -428,7 +428,8 @@ left out whole: it can hold what the policy cannot name`} the same way, so type the member as what it holds.{" "} BitArray and the framework's string collections hold values. An application's own collection class, generic or not, - still has its own members read, and two members sharing a name are left + still has its own members read, and a collection of values stays a + value unless one of them is denied. Two members sharing a name are left out when either holds a denial. A framework generic holding a policed type, such as Dictionary<string, LineDto>, has no paths beneath it: naming it is refused in both tiers where the core diff --git a/OfficialWebsite/public/llms.txt b/OfficialWebsite/public/llms.txt index 8fc09da..4149939 100644 --- a/OfficialWebsite/public/llms.txt +++ b/OfficialWebsite/public/llms.txt @@ -4850,9 +4850,11 @@ MemoryCalculationInput JSON bag say, asks for no projection on its own, and an entity keeps it whole, unless a value converter hands back its value, directly or inside a complex property: a converter is the application's code, so such a column is left out. BitArray and the framework's string collections hold values. An application's - own collection class, generic or not, has its own members read. Two members sharing a name, one hidden with - new under another type or spelled in another case, are left out when either holds a denial, since the core - reads one and a row carries both. A projected member is read as the type its initializer constructs, and an + own collection class, generic or not, has its own members read; a collection of values stays a value unless + one of them is denied. Two members sharing a name, one hidden with new under another type or spelled in + another case, are left out when either holds a denial, since the core reads one and a row carries both. + Rows in memory are projected when a member a base type declares, and the row type hides with new, is + denied. A projected member is read as the type its initializer constructs, and an initializer after a constructor with arguments narrows its own bindings. The trace records a member left out only when a projection is built, or, in a dry run, would be. - Behaviour change. [DwEntity(DefaultOrder)] applies to a projected source whose outermost Select builds T @@ -5061,6 +5063,17 @@ MemoryCalculationInput its members decides the entity's own path. Declare such a class apart from the entity to keep the path open. - Types from an unloadable `AssemblyLoadContext` stay referenced by the policy's caches, so the context is not collected while the process runs. +- A method or property in a reshaping lambda that builds a query from captured values (a repository's query, a + specification) runs once more per guarded read, when the guard reads what it returns; one that returns a different + query on each call is enforced as it answered the guard. +- A provider that wraps EF Core's gets `AsNoTracking` only when its query's expression shows the EF Core root, as + LinqKit's and DelegateDecompiler's do; one that hides it behind its own expression runs tracking. +- A member declared as a framework collection (`IEnumerable`, `List`) that holds an application's own + collection class at run time is read as the framework type, so that class's own members are not read. Serializers + write only the elements. +- On EF Core 6, a reshaped query whose projection EF Core 6 cannot translate (a count over `GroupBy`/`First`, a + `SelectMany` over a captured query that builds its rows) fails guarded, where it ran unguarded; EF Core 7 and later + translate it. - A default order reaches a projected row only through columns of the entity its `Select` reads: a projection over an anonymous or other intermediate row takes no default. - A simulation reads T as a source it cannot see into (section 23): every denial beneath a member counts, and a diff --git a/README.md b/README.md index 045bacb..663e9fe 100644 --- a/README.md +++ b/README.md @@ -389,7 +389,7 @@ The complete reference — every enum, class, extension method, validation rule, - **Changed: `[DwEntity(DefaultOrder)]` reaches a projection that builds the row.** A guarded query over a projected source takes the default when the outermost `Select` builds the type in an object initializer and assigns every field the default names a column, at every level of a nested path — a mapped member read directly, through reference navigations or through `EF.Property`: `Select(t => new TicketRow { Id = t.Id, CreatedAt = t.CreatedAt })` for `"CreatedAt desc, Id"`. A computed value or any other projection still leaves the query in its own order. A `Select`, or a `Filter` with `Selects`, composed on the guarded handle keeps the rest of the chain unordered, and a composed `Filter` that sent orders gets no default later in the chain, as a composed `Order` already did not. - **Changed: the async dynamic `Filter` and the async `Summary` read through EF Core.** `ToListAsyncDynamic` and `ToListAsync(Summary)` read with EF Core's `ToListAsync` instead of Dynamic LINQ's `ToDynamicListAsync`, which had no token to pass on, and the summary counts with `CountAsync` where it counted synchronously. So on an EF Core query a canceled token now reaches the database. A provider that is not EF Core's keeps Dynamic LINQ's read, on the calling thread. - **New: a `CancellationToken` on every async terminal**, guarded and unguarded: `ToListAsync` and `ToListAsyncDynamic` with a `Filter`, `ToListAsync` with a `Summary`, and `ToListAsync` with a `Segment`. The token reaches the count and the read. The overloads sit beside the 3.1 signatures, which are unchanged, so code compiled against 3.1 still binds. `ToListAsync(filter, default)`, `ToListAsyncDynamic(filter, default)` and `ToListAsync(summary, default)` no longer compile, because `default` fits both `getQueryString` and the token: write `false`, a token, or a named argument. A reflection lookup of `ToListAsyncDynamic` by name alone now finds three methods where it found one, and one of `ToListAsync` finds more than it did. -- **Known limits.** Once a projection is needed, an entity's navigations are left out, included ones too; under the convenience tier name one in `Selects` to get it narrowed, and under the strict tier name its allowed fields. A forced scope declared on a list's element type filters the rows that hold the list, never its elements, so `Selects` naming the list returns every element and a synthesized projection leaves the list out; scope the elements where the row is built. A member typed `object`, a framework interface or a collection that is not generic (`IEnumerable`, `ArrayList`) is opaque to the policy, and a framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: hold such values in a list of the policed type. A member EF Core does not map is read as its type, since its getter can hand out what EF Core loaded. Rows in memory can be any loaded subtype, so they are projected whenever one declares a denied field. A denial on an override or a `new` member counts for every loaded subtype, a class EF Core does not map included. `/simulate` has no source, so it reads the type as one it cannot see into: every denial beneath a member counts, and the projection it shows keeps only members holding a value. +- **Known limits.** Once a projection is needed, an entity's navigations are left out, included ones too; under the convenience tier name one in `Selects` to get it narrowed, and under the strict tier name its allowed fields. A forced scope declared on a list's element type filters the rows that hold the list, never its elements, so `Selects` naming the list returns every element and a synthesized projection leaves the list out; scope the elements where the row is built. A member typed `object`, a framework interface or a collection that is not generic (`IEnumerable`, `ArrayList`) is opaque to the policy, and a framework generic holding a policed type, such as `Dictionary`, has no paths beneath it: hold such values in a list of the policed type. A member EF Core does not map is read as its type, since its getter can hand out what EF Core loaded. Rows in memory can be any loaded subtype, so they are projected whenever one declares a denied field. A denial on an override or a `new` member counts for every loaded subtype, a class EF Core does not map included. A repository or specification method in a reshaping lambda runs once more per guarded read, and on EF Core 6 a reshaped query whose projection EF Core 6 cannot translate fails guarded where it ran unguarded. `/simulate` has no source, so it reads the type as one it cannot see into: every denial beneath a member counts, and the projection it shows keeps only members holding a value. ## Version 3.1.0 highlights