Skip to content

feat(memories): Tier 2 — people_together memory type (pair co-occurrence) #792

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Deeds67 merged 8 commits into from
Jul 26, 2026
Merged

feat(memories): Tier 2 — people_together memory type (pair co-occurrence) #792

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
ee4d1a9
docs(memories): Tier 2 people_together spec (pair co-occurrence memory)
Deeds67 Jul 16, 2026
dfb3996
docs(memories): harden people_together spec — TDD/BDD, full edge cove…
Deeds67 Jul 16, 2026
81204b1
feat(memories): pairCounts curation helper for people-together
Deeds67 Jul 16, 2026
881b153
feat(memories): getMemoryFacesForPeriod repository query
Deeds67 Jul 16, 2026
34c8a91
feat(memories): people-together memory type
Deeds67 Jul 16, 2026
306c06e
chore(memories): people-together generation test + roadmap status
Deeds67 Jul 16, 2026
e05a6f9
fix(memories): add people_together to e2e server-config fixture + docs
Deeds67 Jul 17, 2026
400d0bb
feat(memories): Tier 3 — trip anniversary, themed (smart search), vid…
Deeds67 Jul 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 21 additions & 9 deletions docs/docs/features/memories.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,18 +73,30 @@ A user receives a memory type only when it is **both** globally available **and*

The built-in types each have a stable key used in configuration:

| Type key | Setting label | Controls |
| --------------------- | ----------------------- | ----------------------------------------------------------------------- |
| `on_this_day` | On this day | "N years ago" photo memories |
| `birthday` | Birthdays | Birthday rule memories for named people |
| `recent_trip` | Recent trips | Recent trip rule memories |
| `month_recap` | This month | A past year's photos from this calendar month, shown early in the month |
| `favorites_throwback` | Favorite moments | Your favorite photos from this calendar month in a past year |
| `on_this_day_place` | On this day, in a place | A past year's on-this-day photos when they cluster in one place |
| `season_recap` | Season recap | A past meteorological season, shown when the new season begins |
| Type key | Setting label | Controls |
| --------------------- | ----------------------- | --------------------------------------------------------------------------- |
| `on_this_day` | On this day | "N years ago" photo memories |
| `birthday` | Birthdays | Birthday rule memories for named people |
| `recent_trip` | Recent trips | Recent trip rule memories |
| `month_recap` | This month | A past year's photos from this calendar month, shown early in the month |
| `favorites_throwback` | Favorite moments | Your favorite photos from this calendar month in a past year |
| `on_this_day_place` | On this day, in a place | A past year's on-this-day photos when they cluster in one place |
| `season_recap` | Season recap | A past meteorological season, shown when the new season begins |
| `people_together` | People together | Two people or pets often photographed together in a past year |
| `video_moments` | Video moments | Videos you filmed in this month of a past year |
| `trip_anniversary` | Trip anniversaries | A past trip resurfaced on the anniversary of the day it began |
| `themed` | Themes | Photo themes like sunsets, food, and beach days, found automatically |
| `person_throwback` | Times with someone | A warm chapter with someone who has not appeared in your photos for a while |

All default to **on**.

`themed` (Themes) additionally requires [Smart Search](/features/searching) to be enabled — it matches photos to a rotating monthly theme (sunsets, food, beach days, etc.) via CLIP embeddings. If smart search is disabled or the machine learning service is unavailable, Gallery simply skips the rule for that night; it does not surface an error.

Two of these types are tunable in **Administration → Settings → Memories**, or via the [config file](/install/config-file):

- **Theme match threshold** (`memories.themeMaxDistance`, default `0.75`) — how close a photo must be to the month's theme. This is a text-to-image CLIP distance, so it is much larger than a face-matching threshold; values under `0.5` usually yield no themed memories at all.
- **Person throwback dormancy** (`memories.personThrowbackDormancyMonths`, default `6`) — how long someone must be absent from your photos before `person_throwback` can resurface them.

### Per-user toggles

Each user manages their own memory types from **Account Settings → Features → Memories**. Below the master memory switch, a toggle appears for every memory type the admin has made available. Turning one off:
Expand Down
13 changes: 13 additions & 0 deletions docs/docs/install/config-file.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,8 +163,10 @@ The default configuration looks like this:
},
"memories": {
"birthday": true,
"personThrowbackDormancyMonths": 6,
"recentTrips": true,
"retentionDays": 365,
"themeMaxDistance": 0.75,
"types": {}
},
"metadata": {
Expand Down Expand Up @@ -325,6 +327,11 @@ The `memories` section configures generated memory retention and which memory ty
- `favorites_throwback` — your favorite photos from this calendar month in a past year
- `on_this_day_place` — a past year's on-this-day photos concentrated in one place
- `season_recap` — a recap of a past meteorological season
- `people_together` — two people or pets often photographed together in a past year
- `video_moments` — videos filmed in this calendar month in a past year
- `trip_anniversary` — a past trip resurfaced on the anniversary of the day it began
- `themed` — photo themes like sunsets, food, and beach days, found automatically via smart search
- `person_throwback` — a warm chapter with someone who has not appeared in your photos for a while

For example, to disable recent trips globally and leave the rest on:

Expand All @@ -336,6 +343,12 @@ For example, to disable recent trips globally and leave the rest on:
}
```

`themeMaxDistance` is the maximum CLIP cosine distance for the `themed` memory type (sunsets, food, beach days, etc. — found via smart search, not tags). It only takes effect for values `0 < x < 2`; the default is `0.75`. Setting it to `0` disables the quality gate entirely, so every smart-search result within a themed year is accepted regardless of similarity.

This is a **text-to-image** distance, so it sits far higher than the image-to-image thresholds used for duplicate detection (`0.01`) or facial recognition (`0.5`) — CLIP's modality gap means even a perfect textual match rarely scores below `~0.6`. Values under `0.5` will typically produce **no themed memories at all**. If themed memories stop appearing, raise this in small steps rather than lowering it. `themed` requires smart search to be enabled — see the [Memories docs](/features/memories).

`personThrowbackDormancyMonths` is how many months a person must be absent from your photos before the `person_throwback` memory type can resurface them. The default is `6`; valid values are `1`–`120`. Lower values surface more people — including some you still see regularly — while higher values concentrate the memory on people who have genuinely dropped out of your library. The gap itself is never shown in the memory and never affects ranking.

The config file only controls **global availability**. Within each available type, every user can still enable or disable it for themselves in their account settings. Disabling a type globally removes it from every user's settings and immediately hides existing unsaved memories of that type (saved memories are kept).

The per-type switches do not control whether the nightly task runs. To disable all generated memories, set `nightlyTasks.generateMemories` to `false`.
Expand Down
34 changes: 18 additions & 16 deletions docs/plans/2026-07-15-memory-types-roadmap.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,28 +51,30 @@ Spec: [`2026-07-15-memory-types-tier1-spec.md`](./2026-07-15-memory-types-tier1-

### 🟡 Tier 2 — Easy (planned)

| # | Idea | Surfaces | Effort | Impact | Notes |
| --- | ----------------------- | -------------------------------------------------------- | ------ | ---------- | ------------------------------------------------------------- |
| 5 | You & [person] | Two named people who co-occur often | 🟡 | High | Self-join `asset_face` on `assetId`, two `personId`s |
| 6 | Trip anniversary | A _past_ trip resurfaced on its anniversary | 🟡 | High | Reuse location-cluster logic anchored to the on-this-day date |
| 7 | Themed / classification | "Sunsets", "Food", "Beach days" from auto-classification | 🟡 | High | Query `tag_asset`; depends on classification being enabled |
| 8 | Shot on [camera/lens] | Gear nostalgia grouped by `make`/`model` | 🟡 | Low-medium | Niche; photographers only |
| # | Idea | Surfaces | Effort | Impact | Notes |
| --- | ----------------------- | -------------------------------------------------------- | ------ | ---------- | ------------------------------------------------------------------------------------ |
| 5 | You & [person] | Two named people who co-occur often | 🟡 | High | **Shipped** — `people_together` (reframed to a pair, month-anchored) |
| 6 | Trip anniversary | A _past_ trip resurfaced on its anniversary | 🟡 | High | **Shipped** — `trip_anniversary` |
| 7 | Themed / classification | "Sunsets", "Food", "Beach days" from auto-classification | 🟡 | High | **Shipped** — `themed` (reframed onto smart-search CLIP embeddings, not `tag_asset`) |
| 8 | Shot on [camera/lens] | Gear nostalgia grouped by `make`/`model` | 🟡 | Low-medium | Niche; photographers only |

Spec (#5): [`2026-07-16-memory-types-tier2-people-together-spec.md`](./2026-07-16-memory-types-tier2-people-together-spec.md)

### 🟠 Tier 3 — Medium (planned)

| # | Idea | Surfaces | Effort | Impact | Notes |
| --- | ------------------------ | -------------------------------------------------- | ------ | -------------- | --------------------------------------------------------- |
| 9 | Someone you haven't seen | A person whose most-recent photo is > N months old | 🟠 | High but risky | Sensitivity risk (deceased people); needs a careful frame |
| 10 | Your pet [name] | Leverages Gallery's pet detection | 🟠 | High | Fork differentiator; needs a look at how pets are stored |
| 11 | Video moments | Memorable videos, not just stills | 🟠 | Medium | Query easy; memory viewer must play video well |
| # | Idea | Surfaces | Effort | Impact | Notes |
| --- | ------------------------ | -------------------------------------------------- | ------ | -------------- | ---------------------------------------------------------------------------------- |
| 9 | Someone you haven't seen | A person whose most-recent photo is > N months old | 🟠 | High but risky | **Shipped** — `person_throwback` (reframed: gap is a silent selector, never shown) |
| 10 | Your pet [name] | Leverages Gallery's pet detection | 🟠 | High | Fork differentiator; needs a look at how pets are stored |
| 11 | Video moments | Memorable videos, not just stills | 🟠 | Medium | **Shipped** — `video_moments` |

### 🔴 Tier 4 — Hard (north star)

| # | Idea | Surfaces | Effort | Impact | Notes |
| --- | ------------------------ | ----------------------------------------------------------- | ------ | --------- | --------------------------------------------------------------- |
| 12 | Semantic themes (CLIP) | "Time in nature", "City lights" with no tag, via embeddings | 🔴 | Very high | The real Apple/Google magic; keep the rule interface plug-ready |
| 13 | "Best of" aesthetic rank | Auto-picks your most beautiful shots | 🔴 | High | Needs an aesthetic-scoring model (none today) |
| 14 | Named trip stories | Full trip recap with map + day-by-day route | 🔴 | High | A feature, not a rule |
| # | Idea | Surfaces | Effort | Impact | Notes |
| --- | ------------------------ | ----------------------------------------------------------- | ------ | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 12 | Semantic themes (CLIP) | "Time in nature", "City lights" with no tag, via embeddings | 🔴 | Very high | The real Apple/Google magic; keep the rule interface plug-ready. Note: `themed` (PR #812) already rides smart-search CLIP embeddings — the remaining work here is vocabulary breadth and `themeMaxDistance` calibration, not new infrastructure |
| 13 | "Best of" aesthetic rank | Auto-picks your most beautiful shots | 🔴 | High | Needs an aesthetic-scoring model (none today) |
| 14 | Named trip stories | Full trip recap with map + day-by-day route | 🔴 | High | A feature, not a rule |

## Sequencing

Expand Down
Loading
Loading