fix(storage): bind datetimes in time comparisons so they select by value - #191
Merged
acidkill merged 2 commits intoSep 3, 2026
Conversation
Two queries bound a datetime bound as an ISO string against a datetime column. SurrealDB resolves a cross-type comparison by type rank rather than by value, so the predicate became constant - and which constant depends on the operator: datetime >= string is always true, datetime <= string always false. find_neurons(time_range=...) combines both, so it matched nothing at all, for any window. get_enhanced_stats uses the one-sided >=, so today_fibers_count reported every fiber the brain had ever held as created today - a fabricated number surfaced by smem info and the dashboard. InMemoryStorage already implemented both comparisons on Python objects, so this restores parity between the backends rather than changing the contract. Covered by live regression tests; the schemaless in-memory backend compares Python objects and cannot reproduce either fault.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
datetimecolumn, which makes the comparison resolve by type rank instead of by value. Both now bind thedatetime.find_neurons(time_range=...)returned nothing at all, for any window.today_fibers_countcounted every fiber the brain had ever held, not the ones created today.Why
created_atis adatetimecolumn on bothneuronandfiber. SurrealDB resolves a comparison between two different types by type rank rather than by value, so binding a string makes the predicate a constant - and which constant depends on the operator:The result does not depend on the string's contents, so no format of ISO string makes it work.
1.
find_neurons(time_range=...)matched nothing. The two bounds are combined withAND, and they become constants in opposite directions, so the conjunction is never satisfied - including for rows squarely inside the window. Measured onsurrealdb:v3.2.4with three neurons created at that moment:This is silent: an empty result from a time-filtered query is indistinguishable from "nothing matched". The affected readers are the temporal recall hints in
retrieval.pyand two dashboard timeline queries inserver/routes/dashboard_api.py.2.
today_fibers_countcounted everything.get_enhanced_statsuses the one-sided>=, where the constant is true, so the "created today" counter returned every fiber in the brain. Measured with three fibers - one created today, one ten days old, one over a year old - the counter returned3; with the datetime bound it returns1. This one is worse than an empty read, because it fabricates a plausible number, and it is user-facing throughsmem infoand the dashboard.I scanned every datetime-column comparison in
storage/surrealdb/:alerts.py,activity.py,tool_events.py,retrieval_trace.py,reasoning_traces.py,keyword_entity.py,review_schedules.py,depth_priors.py,typed_memory.pyand the two$cutoffqueries instore.pyall binddatetimeobjects already. The remaining.isoformat()calls in that package are read-path serialisers, not binds. These two were the only sites converting a bind to a string. (shared_store.pyalso calls.isoformat()on the same two bounds, but those are HTTP query parameters for the remote-storage client, not SurrealDB binds, and are correct as they are.)Backend parity, not new semantics.
InMemoryStoragehas always implementedstart <= created_at <= endandcreated_at >= todaydirectly on Python objects, so this change makes the SurrealDB backend agree with the backend the tests have been asserting against, rather than introducing a new contract.A behaviour change worth flagging
Fixing the filter activates a code path that has never fired against SurrealDB, and I would rather point at it than let it be discovered.
_find_similar_time_neuronin the encode pipeline uses this filter to decide whether to create aTIMEneuron. It searches a ±1h window around the referenced time (hint.midpoint) but compares it againstcreated_at, the row's insert time - two different quantities - and it is content-blind (type=TIME,limit=1). So it is not a de-duplicator in the sense the name suggests, and this PR does not turn it into one: I ran the step twice against a live engine on a sentence carrying three hints, andtodayandthis afternoonwere still created afresh on the second encode. What did change is that one hint (this evening) was suppressed by an unrelatedTIMEneuron that happened to be inserted within an hour of that hint's midpoint.So: some
TIMEneurons will now be suppressed, on a criterion that is arguably not the intended one. That behaviour is pre-existing and orthogonal to this fix - the guard simply could not run before - but it becomes observable with it, and it may deserve its own look. I have deliberately not changed that logic here, to keep this to one change.The same mismatch exists on the recall side, and this fix activates it too.
retrieval.pyresolves a temporal hint withfind_neurons(type=TIME, time_range=(hint.absolute_start, hint.absolute_end), limit=5)- again matching the hint's referenced range against rows' insert time. Previously those anchors contributed nothing; now they contribute at full retriever weight. I checked what that does with twoTIMEneurons - one readingyesterday(inserted today) and one readingnext week(inserted yesterday) - and the hint for yesterday returnednext weekand notyesterday. So temporal anchors now participate in ranking on a criterion that does not mean what the hint means.Both of these are the pre-existing semantics of
_find_similar_time_neuronand the temporal retriever, and both match whatInMemoryStoragehas always done, so this PR restores parity rather than inventing behaviour - and returning the wrong-but-related anchor is arguably still better than the previous "always nothing". I am flagging them because they become live with this change and I would rather they were known than found later; whether the insert-time comparison should become a metadata-range comparison seems worth a separate issue, which I am happy to open.Test plan
pytest tests/ -m "not stress" -n autopasses locally: 7079 passed, 150 skipped, 1 xfailed. The skip delta againstmainis +4 - exactly the new tests, which skip whenSURREALDB_URLis unset.ruff check src/ tests/clean;ruff format --checkclean.mypy src/ --ignore-missing-imports-Success: no issues found in 353 source files.store.pyrestored to its state onmain, three of the four new tests fail - the in-window query returns nothing, the pipeline's TIME lookup returns nothing, and the counter returns2where1is correct. With the fix, all four pass.test_time_range_excludes_rows_outside_the_window) passes with or without the fix, because the broken filter also returned nothing. It is there to stop the fix over-correcting into false positives, not to demonstrate the bug. Its windows are one and two days out rather than decades, so a merely coarse comparison would not satisfy it.surrealdb:v3.2.4against a real schema, in a fresh process, with a fresh brain per run.utcnow()returns naive UTC and stored values are written from it; a naive bind is treated as UTC (a local-time interpretation would have returned 0 rows for the ±1h window) and an offset-aware bind is normalised correctly.Verified by
@RobertSigmundsson
Thanks for the pace on v3.7.0 and v3.8.0 - both out on 2026-08-27. This branch is rebased onto
ae8e8743, so the failure and the fix are measured against what you shipped, not against an olderbase. Applies cleanly; no overlap with #186's storage changes.