Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions docs/BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,15 @@
_(nothing active — pick the next batch from below)_

## Deferred / future refinements
- [ ] **P13c-2 — opt-in auto-tag-on-download.** The sibling of P13a-2/P13b-3: an opt-in (default-off) setting
to suggest tags for new downloads in the background. Unlike the passive summary/OCR enrichments, tags are
**user-curated** (they drive facets), so this should likely **suggest + post a "review tags" Activity
Inbox nudge** (or write into a marked "ai" namespace) rather than silently auto-apply LLM tags — decide
the apply-vs-review behaviour when planning it. *(From P13c.)*
- [ ] **AI tag casing/normalization.** `addTagToItem` trims but doesn't lowercase, so P13c's AI suggestions
are lowercased while manual/graph tags keep their case (e.g. `Live` vs `live` can coexist). Consider a
single normalization policy (case-fold on store, or a display-case + fold-key) if duplicate-case tags
become noisy. *(From P13c.)*
- [ ] **Translation — translate the summaries + cache + pack management.** P13b-2 translates the
**description + transcript** only (the derived AI/TextRank summaries stay in the source language) and
is **ephemeral** (re-translates each time; no DB cache). Future: also offer translated summaries, cache
Expand Down
9 changes: 9 additions & 0 deletions docs/VERIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -968,6 +968,15 @@ entries, or verify after P11c lands.)*
- [ ] **Default off:** with the toggle off, image downloads are not auto-scanned (on-demand "Scan text"
still works). A **video** download is never auto-OCR'd. The queue still drains normally.

### P13c — Smart auto-tagging *(install `app-arm64-v8a-debug.apk`; needs a capable device)*
- [ ] On a capable device with text generation enabled: open an item → **Edit info** → Tags → an **AI
suggestions** row with a **Suggest tags with AI** button. Tap it → sensible lowercase tag chips appear,
**fully offline** (airplane mode); tapping a chip **adds** the tag (it moves into the tag list and feeds
facets) and removes it from the suggestions.
- [ ] AI suggestions **exclude** tags already applied; tags are never added without a tap.
- [ ] On a **low-end** device the AI row is **absent** (the graph co-occurrence "Suggested" chips still work);
with generation **not enabled**, the button routes to **AI settings** (on-ramp).

### P13 (later subphases)
- [ ] **Transcription / summarization / translation / OCR** each work (capability-gated) and write
results back to the item.
Expand Down
12 changes: 11 additions & 1 deletion docs/design/P13-PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ target-language UX + GMS nuance). Measure APK-size impact in the first ML Kit bu
(c) quick wins — auto-transcribe skips image items, and `durationSec` is gated to non-image. The
unconditional `--write-thumbnail` and non-`mediaTypeForExt` image formats are logged in `BACKLOG.md`.

### `[ ]` P13c — Smart auto-tagging *(generation; APK)*
### `[~]` P13c — Smart auto-tagging *(generation; APK)*
LLM-suggested tags feeding the **existing** tag system — builds directly on the P13a generation patterns.
- Prompt the active model with the item's text for candidate tags; parse the **free-text** response (no
structured seam); present them through the **existing P10c-c-2 suggestion chips** in the metadata editor so
Expand All @@ -183,6 +183,16 @@ LLM-suggested tags feeding the **existing** tag system — builds directly on th
graph-co-occurrence tag suggestions remain).
- **Exit / review:** a capable device suggests sensible tags from an item's content offline; tapping a chip
persists the tag via the existing repository path; ineligible devices degrade to the co-occurrence chips.
- **Status:** implemented (CI-green) — **on-demand** (maintainer call): a separate, gated **"AI suggestions"**
row in the metadata editor (`_AiTagSuggestions`, below the graph `_Suggestions`), hidden when no generation
model fits the device. Pure `buildTagPrompt` + forgiving `parseTagSuggestions` (split/strip/lowercase/dedupe/
exclude-applied/cap); an autoDispose `itemAiTags(itemId)` controller (mirrors the P13b-2 translation
controller) builds the source from title + description/transcript/`ocrText`, generates, parses excluding
current tags; chips reuse `ActionChip` + `addTagToItem` (never auto-written). Reuses `aiSummaryAction` for
the on-ramp; one-shot `MetadataRepository.tagNamesForItem`/`mediaItemById`. **No deps/schema.** Tests:
prompt/parse units, controller (fake engine — excludes applied, lowercases, error path), editor gating
(low hides / high shows). **Pending APK spot-check** (generate + apply on a real item offline). The
generate→chips flow is APK-verified. Background **auto-tag-on-download is a deliberate follow-up (P13c-2)**.

### `[ ]` P13d — Local GraphRAG "Ask your library" *(flagship; split into 3 PRs)*
The headline differentiator — natural-language Q&A grounded in the private library, fully on-device
Expand Down
17 changes: 17 additions & 0 deletions lib/features/library/data/metadata_repository.dart
Original file line number Diff line number Diff line change
Expand Up @@ -630,6 +630,12 @@ class MetadataRepository {
_db.mediaMetadata,
)..where((t) => t.itemId.equals(itemId))).getSingleOrNull();

/// One-shot read of an item's row (P13c) — e.g. to read its title as a tag
/// signal without a reactive stream.
Future<MediaItem?> mediaItemById(String itemId) => (_db.select(
_db.mediaItems,
)..where((t) => t.id.equals(itemId))).getSingleOrNull();

Future<void> updateTitle(String itemId, String title) async {
await (_db.update(_db.mediaItems)..where((t) => t.id.equals(itemId))).write(
MediaItemsCompanion(title: Value(title.trim())),
Expand Down Expand Up @@ -701,6 +707,17 @@ class MetadataRepository {
return query.map((row) => row.readTable(_db.tags)).watch();
}

/// One-shot read of an item's current tag names (P13c) — for commands that
/// need the set once (e.g. excluding already-applied tags from AI suggestions)
/// rather than a reactive stream.
Future<List<String>> tagNamesForItem(String itemId) async {
final query = _db.select(_db.tags).join([
innerJoin(_db.mediaTags, _db.mediaTags.tagId.equalsExp(_db.tags.id)),
])..where(_db.mediaTags.itemId.equals(itemId));
final rows = await query.get();
return [for (final r in rows) r.readTable(_db.tags).name];
}

Future<void> addTagToItem(String itemId, String name) async {
final clean = name.trim();
if (clean.isEmpty) return;
Expand Down
92 changes: 92 additions & 0 deletions lib/features/library/presentation/item_ai_tags_provider.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
import 'package:grabbit/core/ai/generation_provider.dart';
import 'package:grabbit/core/ai/inference_error.dart';
import 'package:grabbit/features/library/data/metadata_repository.dart';
import 'package:grabbit/features/library/presentation/tag_suggestions.dart';
import 'package:riverpod_annotation/riverpod_annotation.dart';

part 'item_ai_tags_provider.g.dart';

/// Screen-scoped state for on-device LLM tag suggestions for one item (P13c).
/// Ephemeral — nothing is cached; the provider auto-disposes when the editor
/// closes.
class ItemAiTagsState {
const ItemAiTagsState({
this.suggestions = const [],
this.busy = false,
this.error,
});

/// Suggested tag names not yet applied.
final List<String> suggestions;
final bool busy;
final String? error;

ItemAiTagsState copyWith({
List<String>? suggestions,
bool? busy,
String? error,
}) => ItemAiTagsState(
suggestions: suggestions ?? this.suggestions,
busy: busy ?? this.busy,
error: error,
);
}

/// Per-item AI tag-suggestion controller (P13c). The metadata editor's AI row
/// calls [suggest]; tapping a chip applies the tag (via `addTagToItem`) and
/// [remove]s it from the local list.
@riverpod
class ItemAiTags extends _$ItemAiTags {
@override
ItemAiTagsState build(String itemId) => const ItemAiTagsState();

/// Generates tag suggestions from the item's title + description/transcript/
/// OCR text, excluding tags already applied. Assumes a generation model is
/// ready (the caller gates on that).
Future<void> suggest() async {
final repo = ref.read(metadataRepositoryProvider);
final item = await repo.mediaItemById(itemId);
final meta = await repo.metadataForItem(itemId);
final source = [
if (item != null) item.title,
?meta?.description,
?meta?.transcript,
?meta?.ocrText,
].where((s) => s.trim().isNotEmpty).join('\n');
if (source.trim().isEmpty) {
state = const ItemAiTagsState(error: 'Nothing to tag');
return;
}
final engine = ref.read(generationEngineProvider);
state = state.copyWith(busy: true, error: null);
try {
final existing = await repo.tagNamesForItem(itemId);
final p = buildTagPrompt(source);
final buffer = StringBuffer();
await for (final token in engine.generate(
p.prompt,
systemPrompt: p.systemPrompt,
)) {
buffer.write(token);
}
final tags = parseTagSuggestions(
buffer.toString(),
exclude: existing.toSet(),
);
state = ItemAiTagsState(
suggestions: tags,
error: tags.isEmpty ? 'No tags suggested' : null,
);
} on InferenceException catch (e) {
state = ItemAiTagsState(error: "Couldn't suggest tags — ${e.message}");
}
}

/// Drops [tag] from the suggestion list (after it's applied).
void remove(String tag) => state = state.copyWith(
suggestions: [
for (final t in state.suggestions)
if (t != tag) t,
],
);
}
125 changes: 125 additions & 0 deletions lib/features/library/presentation/metadata_edit_screen.dart
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import 'package:flutter/material.dart';
import 'package:flutter_riverpod/flutter_riverpod.dart';
import 'package:go_router/go_router.dart';
import 'package:grabbit/core/ai/generation_provider.dart';
import 'package:grabbit/core/theme/tokens.dart';
import 'package:grabbit/core/widgets/async_fade.dart';
import 'package:grabbit/core/widgets/content_bounds.dart';
Expand All @@ -8,8 +10,11 @@ import 'package:grabbit/core/widgets/error_view.dart';
import 'package:grabbit/core/widgets/section_header.dart';
import 'package:grabbit/core/widgets/skeleton.dart';
import 'package:grabbit/features/library/data/metadata_repository.dart';
import 'package:grabbit/features/library/presentation/ai_summary.dart';
import 'package:grabbit/features/library/presentation/graph_entity_providers.dart';
import 'package:grabbit/features/library/presentation/item_ai_tags_provider.dart';
import 'package:grabbit/features/library/presentation/library_controller.dart';
import 'package:grabbit/features/settings/presentation/settings_controller.dart';

class MetadataEditScreen extends ConsumerStatefulWidget {
const MetadataEditScreen({required this.itemId, super.key});
Expand Down Expand Up @@ -165,6 +170,7 @@ class _TagsEditor extends ConsumerWidget {
onSubmitted: (_) => _add(repo),
),
_Suggestions(itemId: itemId, repo: repo),
_AiTagSuggestions(itemId: itemId, repo: repo),
],
),
),
Expand Down Expand Up @@ -223,6 +229,125 @@ class _Suggestions extends ConsumerWidget {
}
}

/// On-device LLM tag suggestions (P13c) — a separate, gated "AI suggestions" row
/// below the graph suggestions. Hidden where no generation model fits the device
/// (the graph suggestions remain). A "Suggest tags with AI" button generates
/// chips that apply via the same `addTagToItem` path; tags are never auto-added.
class _AiTagSuggestions extends ConsumerWidget {
const _AiTagSuggestions({required this.itemId, required this.repo});
final String itemId;
final MetadataRepository repo;

Future<void> _onSuggest(BuildContext context, WidgetRef ref) async {
final messenger = ScaffoldMessenger.of(context);
final router = GoRouter.of(context);
final engine = ref.read(generationEngineProvider);
final enabled =
ref.read(settingsControllerProvider).asData?.value.generationEnabled ??
false;
final modelReady = enabled && await engine.ensureReady();
switch (aiSummaryAction(
eligible: ref.read(activeGenerationModelProvider) != null,
enabled: enabled,
modelReady: modelReady,
)) {
case AiSummaryAction.unavailable:
return;
case AiSummaryAction.offerSetup:
case AiSummaryAction.offerDownload:
messenger
..hideCurrentSnackBar()
..showSnackBar(
const SnackBar(
content: Text('Set up on-device text generation to suggest tags'),
),
);
await router.push('/settings/ai');
case AiSummaryAction.summarizeNow:
await ref.read(itemAiTagsProvider(itemId).notifier).suggest();
}
}

@override
Widget build(BuildContext context, WidgetRef ref) {
final theme = Theme.of(context);
final tokens = GrabBitTokens.of(context);
// No generation model fits this device → only the graph suggestions show.
if (ref.watch(activeGenerationModelProvider) == null) {
return const SizedBox.shrink();
}
final state = ref.watch(itemAiTagsProvider(itemId));
return Padding(
padding: EdgeInsets.only(top: tokens.spaceMd),
child: Column(
crossAxisAlignment: CrossAxisAlignment.start,
children: [
Row(
children: [
Icon(
Icons.auto_awesome_outlined,
size: 16,
color: theme.colorScheme.primary,
),
SizedBox(width: tokens.spaceXs),
Text(
'AI suggestions',
style: theme.textTheme.labelSmall?.copyWith(
color: theme.colorScheme.onSurfaceVariant,
),
),
const Spacer(),
if (state.busy)
const SizedBox(
width: 16,
height: 16,
child: CircularProgressIndicator(strokeWidth: 2),
)
else
TextButton(
onPressed: () => _onSuggest(context, ref),
child: Text(
state.suggestions.isEmpty
? 'Suggest tags with AI'
: 'Again',
),
),
],
),
if (state.suggestions.isNotEmpty) ...[
SizedBox(height: tokens.spaceXs),
Wrap(
spacing: tokens.spaceSm,
runSpacing: tokens.spaceXs,
children: [
for (final tag in state.suggestions)
ActionChip(
avatar: const Icon(Icons.add, size: 18),
label: Text(tag),
onPressed: () {
repo.addTagToItem(itemId, tag);
ref.read(itemAiTagsProvider(itemId).notifier).remove(tag);
},
),
],
),
],
if (state.error != null)
Padding(
padding: EdgeInsets.only(top: tokens.spaceXs),
child: Text(
state.error!,
style: theme.textTheme.bodySmall?.copyWith(
color: theme.colorScheme.error,
),
),
),
],
),
);
}
}

class _CollectionsEditor extends ConsumerWidget {
const _CollectionsEditor({required this.itemId});
final String itemId;
Expand Down
58 changes: 58 additions & 0 deletions lib/features/library/presentation/tag_suggestions.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
/// Pure, engine-free helpers for on-device LLM tag suggestions (P13c). Kept out
/// of the controller/widget so the prompt shape and the (forgiving) parser are
/// unit-testable in isolation (mirrors `ai_summary.dart`).
library;

/// System instruction for tag suggestions — biases toward a few short, topical
/// tags as a plain comma-separated list. On-device; nothing leaves the device.
const String kTagSystemPrompt =
'You suggest concise topical tags for a piece of media. Reply with 5–8 '
'short lowercase tags (1–2 words each) as a single comma-separated list, '
'and nothing else. Use only what the content is about; no hashtags, no '
'numbering, no sentences.';

/// Builds the (system, user) prompt pair for suggesting tags from [text].
/// The source is head-truncated to [maxChars] (small on-device models have a
/// limited context window).
({String systemPrompt, String prompt}) buildTagPrompt(
String text, {
int maxChars = 3000,
}) {
final trimmed = text.trim();
final source = trimmed.length > maxChars
? trimmed.substring(0, maxChars).trimRight()
: trimmed;
return (
systemPrompt: kTagSystemPrompt,
prompt: 'Suggest tags for the following:\n\n$source',
);
}

/// Parses a model's free-text reply into clean tag suggestions. Forgiving by
/// design: splits on commas/newlines/semicolons, strips `#`/quotes/bullets,
/// collapses whitespace, lowercases, drops empties and over-long entries,
/// de-duplicates (case-insensitive), removes anything in [exclude]
/// (case-insensitive), and caps to [max].
List<String> parseTagSuggestions(
String raw, {
Set<String> exclude = const {},
int max = 8,
int maxLen = 30,
}) {
final excluded = {for (final e in exclude) e.trim().toLowerCase()};
final seen = <String>{};
final out = <String>[];
for (final part in raw.split(RegExp(r'[,\n;]'))) {
var tag = part.trim().toLowerCase();
// Strip leading bullets/numbering/hashes and surrounding quotes.
tag = tag
.replaceAll(RegExp(r'''^[\s\-*#0-9.\)"'`]+'''), '')
.replaceAll(RegExp(r'''["'`]+$'''), '')
.trim();
if (tag.isEmpty || tag.length > maxLen) continue;
if (excluded.contains(tag) || !seen.add(tag)) continue;
out.add(tag);
if (out.length >= max) break;
}
return out;
}
Loading
Loading