feat(memories): generated memory types — Tiers 1-3 and person throwback - #789
Open
Deeds67 wants to merge 19 commits into
Open
feat(memories): generated memory types — Tiers 1-3 and person throwback#789Deeds67 wants to merge 19 commits into
Deeds67 wants to merge 19 commits into
Conversation
github-actions
Bot
added
the
documentation
This was referenced Jul 16, 2026
Deeds67
force-pushed
the
feat/memory-types-tier1
branch
from
July 21, 2026 20:27
28f317c to
5dfcbb9
Compare
Deeds67
changed the title
Deeds67
added
the
rc
🧪 Release candidate buildLatest RC Images published
How to run this RCIn the directory containing your services:
immich-server:
image: ghcr.io/open-noodle/gallery-server:pr-789-rc.4Then pull and restart: docker compose pull immich-server
docker compose up -dEach push publishes a new numbered tag, so update the image line to move to a newer RC. To roll back, point it at an earlier Previous builds (4)
Last updated Wed, 05 Aug 2026 19:13:36 GMT — every push while the |
Deeds67
force-pushed
the
feat/memory-types-tier1
branch
from
July 30, 2026 07:22
392f88c to
7eab823
Compare
… query correctness)
Slice 1 of Tier 1 memory types. Pure, dependency-free helpers: - pickEvenlySpaced (ported from recent-trip.rule), sampleAssetsByTime, medianTime, dominantBy, recencyBonus - season.util: seasonOf, seasonYearOf (winter cross-year), seasonStartingOn, isSeasonStart, SEASON_MONTHS/SEASON_LABEL 26 unit tests, all green.
Slice 2 of Tier 1 memory types. One parametric query serving all four rules: filter by months (+ optional day, favoritesOnly), left-join exif for city/country, extract UTC year, Preview-file + Timeline + not-deleted guards, ordered by time. No flat LIMIT (avoids dropping year-groups). @GenerateSql snapshot regenerated. Added mock entry + 7 medium DB tests (month/multi-month/day/favorites/takenBefore/ geotag nulls/exclusions+ordering).
Slice 3 of Tier 1 memory types. MemoryRuleCandidate gains optional visibleForDays (default 1). createRuleMemories derives hideAt = startOfDay + (visibleForDays-1) days, endOf day, per candidate; showAt unchanged. Existing 1-day rules unaffected. 3 window tests (multi-day / absent / 1).
Slice 4 of Tier 1 memory types (end-to-end pattern-setter). MonthRecapMemoryRule: fires on the 1st, groups prior-year photos of this calendar month, emits top-3 years (>=10 photos) with a 7-day window. Registered in metadata + registry; admin toggle + en.json labels; metadata spec extended. 7 rule tests.
Slice 5 of Tier 1 memory types. FavoritesThrowbackMemoryRule: fires on the 15th, groups prior-year favorited photos of this calendar month, emits top-3 years (>=4 favorites, score-capped) with a 7-day window. Extracted shared monthName helper to curation.util (month_recap now reuses it). Registered + admin toggle + en.json; metadata spec extended. 6 rule tests + monthName util test.
Slice 6 of Tier 1 memory types. OnThisDayPlaceMemoryRule (fires daily): among a past year's on-this-day photos, when >=4 and >=60% cluster in one city, emits 'On this day in <city>' (single-day window, top-3 years). Registered + admin toggle + en.json; metadata spec extended. 11 rule tests (dominance boundary, tie note, ungeotagged, leap-day).
Slice 7 of Tier 1 memory types. SeasonRecapMemoryRule (fires on each meteorological season's first day): recaps a past season-year with >=15 photos (top-2, 10-day window). Winter groups Dec + following Jan/Feb into one season-year; drops the in-progress season. Registered + admin toggle + en.json; metadata spec extended. 9 rule tests.
Slice 8 of Tier 1 memory types. Mobile MemoryTypeEnum is {onThisDay, rule} and renders
rule memories generically from data.title/subtitle — new types need no mobile change and
mobile exposes no per-type toggle surface. Roadmap Status column updated to shipped.
CI caught hardcoded memory-type lists outside memory-rules/ that assumed the old 3-type set: preferences default map, server.service availableMemoryTypes (full + admin- disabled), web MemoriesSettings save payload, and the server e2e config snapshot. All now include month_recap, favorites_throwback, on_this_day_place, season_recap.
Correctness:
- dominantBy: fix empty-key tie-break sentinel (a real '' key now wins a tie correctly)
- on_this_day_place: treat a blank ('') EXIF city as absent, not a place
Coverage (fresh-eyes review found these gaps):
- add TWO end-to-end medium generation tests (month_recap 7-day window,
on_this_day_place 1-day window + dominant-city assets) — the new rules previously
had NO generation coverage, unlike birthday/recent_trip
- prove ungeotagged assets are excluded from the on_this_day_place dominance DENOMINATOR
- assert memoryAt / context / assetIds-order on the recap rules (persisted, user-visible)
- dominantBy empty-key tie test
Docs:
- correct spec 3.2: a single recap type can take BOTH daily slots for its window
(~monthly, not ~4x/yr) — flag the starvation trade-off + per-type-cap lever honestly
- document the 4 new types in features/memories.md and install/config-file.md
Security/RBAC review: no findings — query is structurally owner-scoped, matches
existing memory queries' visibility/trash filters, config schemas are open.
Resolves the slot-starvation trade-off surfaced in review. A rule whose candidate sets visibleForDays > 1 (the recaps) may now insert at most one memory per trigger day (insertedMultiDayRuleIds), so a recap type qualifying for several past years can no longer take both daily slots for its whole 7-10 day window and monthly starve the 1-day rules (on_this_day_place, recent_trip). 1-day rules are unchanged. Only the ~4x/yr season-starts still compound two DIFFERENT recap types. Test + spec 3.2/8 updated.
…nce) (#792) * docs(memories): Tier 2 people_together spec (pair co-occurrence memory) * docs(memories): harden people_together spec — TDD/BDD, full edge coverage, /impl-loop slices * feat(memories): pairCounts curation helper for people-together * feat(memories): getMemoryFacesForPeriod repository query * feat(memories): people-together memory type * chore(memories): people-together generation test + roadmap status * fix(memories): add people_together to e2e server-config fixture + docs * feat(memories): Tier 3 — trip anniversary, themed (smart search), video moments (#812) * docs(memories): spec tier 3 memory types (trip anniversary, themed, video moments) * docs(memories): fix 24 review defects in tier 3 spec Adversarial review found blockers verified against the code: - MemoryPeriodAsset new fields are required -> 4 fixture factories break tsc - themed firing daily with a multi-day window starves ALL rules (remainingSlots==0 returns early, memory.service.ts:130) - themed capped to 1 candidate makes older years permanently unreachable - searchSmart filters fileCreatedAt, not localDateTime (database.ts:725) - Luxon silently clamps Feb 29 -> Feb 28, breaking leap-year anniversaries - uncapped favoriteCount inverted the trip precedence invariant - recencyBonus maxes at 9 for past years, not 10 * feat(memories): return asset type and duration from getMemoryAssetsForPeriod Additive filter + two new required MemoryPeriodAsset fields, needed by the video_moments rule (slice 2). The fields are required deliberately so every construction site states the asset kind; the four existing rule-spec fixture factories are updated accordingly. TDD: 3 medium tests red (type/duration undefined, type filter not honored) -> 67/67 green. Unit suite 127/127 unchanged. SQL snapshot regenerated. * feat(memories): add video_moments memory type Videos filmed in this calendar month of a past year, triggered on day 8 with a 5-day window (staggered against days 1/15/20 so multi-day rules never contend for a trigger day). Memorability = a 3s-180s duration band, with favourites selected first and a capped favourite score bonus so it cannot outrank a trip anniversary. Registered at all 16 memory-type sites incl. the e2e server-config fixture that the server unit suite does not cover. TDD: spec red (module not found) -> 14/14 green; 217/217 server tests, tsc clean, eslint clean, web 3/3. Also corrects an unreachable '1 video' pluralization case in the spec: MIN_ASSETS=3 makes count===1 impossible for a fired candidate. * fix(mobile): force autoplay for videos in the memory viewer The memory card builds NativeVideoViewer with showControls: false, so a user who has viewer.autoPlayVideo disabled saw a frozen first frame with no way to start playback. Adds an opt-in forceAutoPlay flag (default false, so the ~40 other call sites keep honouring the global preference) and sets it in the memory card. Pre-existing since videos could already land in memories; user-visible now that video_moments surfaces them deliberately. Tests are construction-only by design: mounting DriftMemoryCard under flutter test throws two unavoidable environment exceptions (its remote image provider does not route through dart:io so MockHttpOverrides misses it, and NativeVideoViewer opens a platform video controller in initState). The risk here is the parameter contract and its default, which is what is covered; the single call-site literal is compile-checked by dart analyze. dart analyze --fatal-infos clean; 2/2 tests pass. * refactor(memories): extract shared trip detection and curation helpers Pulls the trip logic private to recent-trip.rule.ts into a pure, unit-tested trip.util.ts that trip_anniversary (slice 5) shares, and introduces the canonical placeKeyOf. placeKeyOf matters beyond tidiness: on_this_day_place built its place key as `${country ?? ''}:${city}` and recent_trip as `${country}:${city ?? ''}`. Slice 5 relies on both place-based rules emitting an IDENTICAL dedupe key so the engine's seenDedupeKeys collapses them; with the divergent formats that collapse would have silently never fired. recent_trip keeps its exact behaviour: curateTripAssets(assets, 10) preserves today's ceiling, and its spec passes byte-identical as the regression guard. findTripStartingOn is the one piece of genuinely new logic. TDD: spec red (module not found) -> 29/29 green; 171/171 memory-rules, tsc + eslint clean. * feat(memories): add trip_anniversary memory type Resurfaces a past multi-day trip on the anniversary of the day it began, detected fresh from location data so imported historical libraries are covered (recent_trip only ever fired for trips taken while already running Gallery). Cheap-probe-first: one on-this-day query prunes the common case before any cluster query runs. A trip is confirmed by starting the cluster window 5 days BEFORE the anniversary, so a cluster whose firstDate lands on the anniversary provably had no photos there beforehand -- a genuine arrival, not a mid-stay. Also makes on_this_day_place share the place_day: dedupe namespace so the engine's existing seenDedupeKeys collapses the two rules (they fire on the same signal and would otherwise take both daily slots with near-identical content). That collapse needs a TOTAL score precedence, so on_this_day_place's previously-unbounded count term is capped at 30: its max becomes 199 while trip_anniversary's floor is 275. Asserted as an invariant test derived from both rules' exported constants, not hardcoded numbers. Luxon silently clamps Feb 29 -> Feb 28 in non-leap years, so the rule skips years where the clamp changed the date rather than comparing the wrong day. TDD: spec red (module not found) -> 18/18 green, incl. mutation checks confirming each guard is load-bearing. 266/266 server, tsc + eslint clean, web 3/3. * feat(memories): add theme catalog and smart-search port for themed memories Themed memories ride the CLIP embeddings smart search already computes per asset -- no auto-classification dependency, no new ML model, no new infra. The rule (slice 7) depends only on a narrow ThemeSearchPort, so it unit-tests against a fake with no ML service and no DB. The adapter behind it memoizes each theme's embedding per (modelName, language), because encodeText is an ML HTTP call and createMemoryRules runs once per user per day -- encoding inside the rule would mean one round-trip per user per night. Theme text is user-independent, so it is encoded once and reused. searchAssetBuilder filters takenAfter/takenBefore on asset.fileCreatedAt (utils/database.ts:725-726), not localDateTime, which every memory rule buckets by. The adapter widens the window 2 days each side so no in-year asset is lost to that skew; the rule then filters precisely by localDateTime. resolveEmbedding never throws: smart search disabled or ML unreachable both yield null, which the rule treats as 'no candidates'. TDD: both specs red (module not found) -> 6/6 and 8/8 green; 260/260, tsc + eslint clean; SDK regenerated (themeMaxDistance only). * feat(memories): add themed memory type backed by smart search 'Sunsets from 2023' -- a curated 6-theme vocabulary matched against the CLIP embeddings smart search already computes. No auto-classification dependency. Fires on day 22 with one theme per MONTH. Both choices are load-bearing: a daily-firing rule with a multi-day window would hold both of the engine's 2 slots for a week at a time, and createRuleMemories returns early when remainingSlots is 0 -- meaning NO rule evaluates at all -- which could permanently starve trip_anniversary. Month-based rotation is also stable across year and leap boundaries, where day-of-year is not (365 % 6 != 0). Emits ALL qualifying years (capped at 3), not just the best: hasRuleMemory dedup happens in the engine after the rule returns, and recencyBonus always favours the newest year, so a single-candidate rule would make older years permanently unreachable once the newest year's memory existed. Re-filters results by localDateTime year because the port's bounds hit asset.fileCreatedAt, applying MIN_ASSETS after that filter. MemoryService memoizes the adapter behind an overridable createThemeSearchPort() seam, so a theme is encoded once per process rather than once per user per night, and slice 8's medium test can inject a stub without a live ML service. TDD: spec red (module not found) -> 13/13 green. Full server suite 4984 passed; memory.service.spec.ts untouched (its arg-agnostic spies still hold); tsc + eslint clean; web 3/3. * test(memories): end-to-end generation coverage for tier 3 memory types Eight medium tests against a real DB, each rule with a positive case and negatives that fail for a DIFFERENT reason than the positive passes: video_moments (wrong trigger day / out-of-band duration), trip_anniversary (single-day cluster fails MIN_TRIP_DAYS specifically, with home inference and asset count held fixed), themed (null embedding), plus a slot-budget test guarding the RULE_DAILY_LIMIT early-return that would otherwise let a multi-day rule starve every other rule. themed is exercised through the createThemeSearchPort seam with a stub, so no live ML service is needed. Also closes a pre-existing connection-pool leak in this file: every test's beforeEach opened a new Kysely pool via getKyselyDB() and none were closed. Harmless at 18 tests, it hit 'too many clients already' at 26. Adds an afterEach that destroys each pool. Documents themeMaxDistance in config-file.md and notes that Themes requires smart search. NOT DONE — threshold calibration (spec 4.2) needs an RC on the personal instance and is a human-run gate; 0.3 remains an unvalidated placeholder. * chore(open-api): regenerate Dart client for themeMaxDistance The TypeScript SDK was regenerated with the config field but the Dart client was not, so CI's OpenAPI Clients job failed on a dirty system_config_memories_dto.dart. make open-api generates BOTH clients; running only open-api-typescript leaves the Dart side stale. * feat(memories): person throwback — resurface a chapter with someone dormant 12+ months (#9) (#831) * docs(memories): spec for person_throwback memory type (#9) * docs(memories): fix trigger-day starvation, score band, and read-skew gaps in person_throwback spec * docs(memories): fix memoryAt type, trim dead fields, pin worked score, add impl-loop slices * feat(memories): add densestChapter window helper * docs(memories): correct i18n key shape and web spec sites; add slice plans * feat(memories): add dormant-person and chapter-window queries * feat(memories): add person_throwback memory rule * docs(memories): correct row-3 chapter year and chapterYear derivation * feat(memories): register person_throwback memory type * feat(web): expose person_throwback in memory settings * test(memories): end-to-end coverage for person_throwback + docs * docs: format slice plan files for docs prettier gate * fix(memories): make themed memories fire, and tune person_throwback dormancy `memories.themeMaxDistance` shipped at 0.3, which emits zero themed memories on a real library: a sweep over 65,685 embeddings returned nothing at 0.3 and 0.5, and genuine matches at 0.75. The default was calibrated on the scale of the image-to-image thresholds used elsewhere (duplicateDetection 0.01, facialRecognition 0.5), but this is a text-to-image distance, where CLIP's modality gap floors the value near ~0.6 even for a perfect match — so 0.3 is unreachable. Default is now 0.75, matching what the admin UI already recommends for `machineLearning.clip.maxDistance` (same metric, same embeddings). `updateConfig` persists only the diff against defaults, so installs that never overrode the value pick this up on upgrade. person_throwback's dormancy window was a hardcoded 12 months, which made the rule effectively unfireable: a 65k-asset library with 12 named people had a most-dormant gap of only 9 months. It is now `memories.personThrowbackDormancyMonths`, defaulting to 6 (range 1-120), so a wrong threshold is a settings change rather than a redeploy. Both knobs were previously config-file-only. They are now in Administration > Settings > Memories, indented directly under the memory type they affect and gated on that type being enabled, so it is unambiguous which type each knob tunes. Adds a test asserting all 12 server-registered types render a switch, guarding against the UI list drifting from the server registry.
New main bumps eslint-plugin-unicorn 64 -> 72 and typescript-eslint to 8.63, which add rules the Tier 1-3 memory files were written before: - unicorn/prefer-continue, prefer-iterator-to-array(-at-end), prefer-direct-iteration, prefer-includes-over-repeated-comparisons, consistent-conditional-object-spread - @typescript-eslint/require-array-sort-compare on the numeric `.toSorted()` in the medium specs All changes are behaviour-preserving. Two are small correctness improvements the rules surfaced: the person-id comparator in the people_together medium spec now returns 0 for equal ids instead of 1, and the ruleId cast is narrowed to a required string. `unicorn/prefer-simple-sort-comparator` would have suggested `a.id - b.id` for that string comparator (NaN); the explicit three-way form keeps codepoint ordering, matching pairCounts.
Deeds67
force-pushed
the
feat/memory-types-tier1
branch
from
July 30, 2026 08:32
7eab823 to
a28430f
Compare
`memories.types` is a `z.record` with an empty-object default, which makes it the one config key whose presence in the `/api/system-config` key diff depends on `getKeysDeep`'s `emptyObjectsAsLeaves` option (added in #783). The existing tests exercised it with a single-key map only, so a diff that truncated or filtered the record would still have passed. - system-config.service.spec.ts: round-trip the full registry map (alternating values) through save and reload against a stateful metadata store, so the assertion reads back what `updateConfig` actually persisted; cover overrides next to non-default sibling memory settings, the empty default not being persisted, accumulating a second toggle, and resetting back to the default. - system-config.service.spec.ts: the unknown-key warning must stay silent for a populated override map, while a genuinely off-schema `memories` key is still reported and the override keys inside the record are not. - misc.spec.ts: pin `memories.types` as the only empty-object default in the config tree, and that its siblings enumerate in both modes. - preferences.spec.ts: pin the load/save asymmetry for a memory-type key with no registry default.
…dows Live testing on a real library showed three curation limits set too tight: on_this_day_place attached 8 assets while its subtitle reported the full matched count, recent_trip vanished after a single day despite firing at most once per place per 30 days, and RULE_DAILY_LIMIT of 2 let four lingering recap windows crowd out the date-anchored rules. Also documents each type's generation day and visibility window, which users had no way to discover.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Generated memory types (Tiers 1–3 + person throwback)
Consolidates the whole memory-types stack into a single PR — previously split across #789 (Tier 1), #792 (Tier 2), #812 (Tier 3) and the person-throwback branch. Adds nine new memory types on top of the fork's rule engine, plus the shared foundation they run on.
Each new type is a
MemoryRulecompeting for the daily rule slots. The memory viewer is untouched — web and mobile render rule memories generically fromdata.title/subtitle.Specs:
docs/plans/2026-07-15-memory-types-tier1-spec.md,2026-07-16-memory-types-tier2-people-together-spec.md,2026-07-19-memory-types-tier3-spec.md,2026-07-22-memory-person-throwback-spec.md, roadmap in2026-07-15-memory-types-roadmap.md.New memory types
month_recapfavorites_throwbackon_this_day_placeseason_recappeople_togethervideo_momentstrip_anniversarythemedperson_throwbackAll nine default on, and are configurable at two independent layers (admin availability + per-user toggle).
Admin-tunable knobs
Two types have thresholds that depend on the library, so they are admin-settable in Administration → Settings → Memories and via the config file — rendered indented under the type they belong to, and disabled when that type is off.
memories.themeMaxDistance(default0.75) — max CLIP cosine distance forthemed.memories.personThrowbackDormancyMonths(default6, range 1–120) — how long someone must be absent beforeperson_throwbackresurfaces them.Calibration fixes found on a real library
Validated against a 65,685-asset library, which caught two defaults that made their rules unfireable in practice:
themedemitted zero memories.themeMaxDistanceshipped at0.3; a sweep returned nothing at0.3and0.5, and genuine matches at0.75. The default had been set on the scale of the image-to-image thresholds used elsewhere (duplicateDetection0.01,facialRecognition0.5), but this is a text-to-image distance where CLIP's modality gap floors the value near~0.6even for a perfect match — so0.3was unreachable. Now0.75, matching what the admin UI already recommends formachineLearning.clip.maxDistance(same metric, same embeddings).updateConfigpersists only the diff against defaults, so installs that never overrode it pick this up on upgrade.person_throwbackcould not fire. The dormancy window was a hardcoded 12 months; the test library's most-dormant named person was only 9 months out, putting the earliest natural fire ~16 months after install. Now configurable, defaulting to 6.Foundation
getMemoryAssetsForPeriod— one parametricAssetRepositoryquery serving the period-based rules (months, optional day, favoritesOnly, takenBefore; left-joins exif for city/country; extracts UTC year). No flatLIMIT, so year-groups are never dropped.@GenerateSqlsnapshots regenerated.curation.util—pickEvenlySpaced,sampleAssetsByTime,medianTime,dominantBy,recencyBonus,monthName.season.util— meteorological seasons with winter cross-year mapping.chapter.util—densestChapterover daily counts, so a heavy subject's full history is never fetched.trip.util— burst collapsing and home-dominance detection shared withrecent_trip.theme.catalog+ThemeSearchPort— rotating monthly theme vocabulary behind a port, with a CLIP-backed adapter that memoizes text embeddings per process.MemoryRuleCandidate.visibleForDays(default 1, so existing rules are unchanged);createRuleMemoriesderiveshideAtfrom it so recaps linger past their trigger day, and multi-day rules are capped to one memory per day so a recap cannot starve the 1-day rules.Tests
TDD throughout, built in numbered slices per spec (each slice an atomic commit).
Notes
themedrequires Smart Search. If it is disabled or the ML service is unavailable, the rule is skipped for that night rather than erroring.person_throwback(person.type = 'person'); the roadmap owns pets separately.MemoryTypeEnumis{onThisDay, rule}and rule memories render fromdata.title/subtitle.RULE_DAILY_LIMIT, and a per-group lateral cap for pathologically large libraries.