fix: support a non-English base language in localize & generate (#11)

CHANGELOG.md

@@ -1,3 +1,8 @@
+## 0.6.0
+
+- **FIX**: `localize` skipped any sheet whose base language was not English — a `label | description | meta | ru | ...` header was rejected as "not a localization sheet". The fourth column is now accepted as the source language whenever it is a recognized language code (`ru`, `de`, `pt_BR`, …); non-language data columns (e.g. a `Family` reference table) are still skipped. The source language is carried through into the model prompt and JSON schema, so rows are translated *from* the actual base language instead of always "from English".
+- **FIX**: `generate` hard-coded the `flutter gen-l10n` template to `<prefix>_en.arb`, so a bucket without an English column failed to generate. The template is now chosen per bucket — English when present (unchanged behaviour), otherwise the first available ARB — so a non-English base language generates too.
+
 ## 0.5.0
 
 - **BREAKING**: `localize` now only writes to a sheet whose header matches `label | description | meta | en | <locale> ...`. A sheet whose fourth column is not the English source is skipped with a warning — reference tables kept in the same spreadsheet were previously "translated", overwriting their data.

CLAUDE.md

@@ -0,0 +1,48 @@
+# CLAUDE.md
+
+Guidance for Claude Code when working in this repository.
+
+## What this package is
+
+`sheety_localization` is a Dart CLI package that turns a Google Sheet into
+localization files. It ships two executables (see `pubspec.yaml`):
+
+- **`localize`** (`bin/localize.dart`, backed by `lib/src/localize/`) — fills in
+  missing translations in the sheet via the OpenAI API and writes them back.
+- **`generate`** (`bin/generate.dart`) — reads the sheet and generates ARB files
+  and Flutter localization classes (`flutter gen-l10n`).
+
+The two share the same sheet layout but are **separate code paths**. The column
+layout is `label | description | meta | <source> | <locale> ...`: the fourth
+column is the base/source language (not necessarily English) and every column
+after it is a target locale. When you change how a sheet is interpreted, check
+**both** `lib/src/localize/` **and** `bin/generate.dart`.
+
+## Release hygiene — required for every task/issue
+
+Every change that touches shippable code MUST leave the package publishable.
+Before finishing a task (and before opening a PR), you must:
+
+1. **Bump the version** in `pubspec.yaml` following semver
+   (`fix` → patch, new feature/relaxed constraint → minor, breaking → major).
+2. **Add a matching entry to `CHANGELOG.md`** under a new version heading, using
+   the existing `**FIX** / **ADDED** / **CHANGED** / **BREAKING**` style.
+3. **Verify the package still publishes** with:
+
+   ```sh
+   dart pub publish --dry-run
+   ```
+
+   It must report no errors (warnings that already existed on `master` are
+   acceptable, but do not introduce new ones).
+
+Skipping any of these makes the package impossible to publish, so treat them as
+part of "done", not optional follow-up.
+
+## Checks before opening a PR
+
+```sh
+dart analyze         # must be clean
+dart test            # must be green
+dart pub publish --dry-run
+```

README.md

@@ -421,7 +421,7 @@ dart pub global run sheety_localization:localize \
 
 ### How Localization Failures Are Handled
 
-> **Only localization sheets are written to.** A sheet is localized only when its header matches `label | description | meta | en | <locale> ...` — that is, when the fourth column is the English source. Reference tables and notes kept in the same spreadsheet have ordinary data in those columns, and translating them would overwrite it, so they are skipped with a warning. `--ignore` remains available for sheets that *do* match the layout but should be left alone anyway.
+> **Only localization sheets are written to.** A sheet is localized only when its header matches `label | description | meta | <source> | <locale> ...` — that is, when the fourth column is a recognized source language. It does not have to be English: `ru`, `de`, `pt_BR` and any other known language code work just as well, and the rest of the row is translated *from* that language. Reference tables and notes kept in the same spreadsheet have ordinary data in the fourth column (e.g. `Family`, `Region`), which is not a language, so they are skipped with a warning rather than overwritten with translations. `--ignore` remains available for sheets that *do* match the layout but should be left alone anyway.
 
 Language models are unreliable on ambiguous or rare locale codes, so `localize` defends against that:
 

bin/generate.dart

@@ -714,6 +714,24 @@ Future<List<io.File>> generateArbFiles({
   return files;
 }
 
+/// Choose the `flutter gen-l10n` template ARB for a single bucket.
+///
+/// gen-l10n needs one ARB as the template that defines every key and
+/// placeholder. Historically this was hard-coded to `<prefix>_en.arb`, which
+/// breaks a bucket whose base language is not English (no `en` column, so no
+/// `<prefix>_en.arb` is ever written). This prefers the English ARB when it
+/// exists — keeping the previous behaviour — and otherwise falls back to the
+/// first ARB by name, so a non-English base still generates.
+///
+/// [bucketArbs] are the ARB file paths (or names) belonging to one bucket.
+String selectTemplateArb(Iterable<String> bucketArbs, {String prefix = 'app'}) {
+  final names = bucketArbs.map(path.basename).toSet();
+  final english = '${prefix}_en.arb';
+  if (names.contains(english)) return english;
+  final sorted = names.toList()..sort();
+  return sorted.isEmpty ? english : sorted.first;
+}
+
 /// Generate Flutter localization files from the arb files
 /// flutter gen-l10n --no-synthetic-package \
 ///   --no-nullable-getter --template-arb-file=app_en.arb \
@@ -773,10 +791,21 @@ Future<Set<String>> generateFlutterLocalization({
 
   final localizations = <String>{};
 
-  final toGenerate = arbs.map((e) => e.parent.absolute.path).toSet();
+  // Group the generated ARB files by their bucket directory, so the gen-l10n
+  // template can be chosen from what each bucket actually has — the base
+  // language does not have to be English.
+  final arbsByDir = <String, List<io.File>>{};
+  for (final arb in arbs) {
+    arbsByDir.putIfAbsent(arb.parent.absolute.path, () => <io.File>[]).add(arb);
+  }
+  final toGenerate = arbsByDir.keys.toSet();
 
   for (final dir in toGenerate) {
     final bucket = path.basename(dir);
+    final template = selectTemplateArb(
+      arbsByDir[dir]!.map((f) => f.path),
+      prefix: prefix ?? 'app',
+    );
     final genPath = path.normalize(path.join(genDirectory.path, bucket));
     if (!genPath.startsWith(libDirectory.path)) {
       $err('Gen path is outside of the library directory: $genPath');
@@ -796,7 +825,7 @@ Future<Set<String>> generateFlutterLocalization({
         '--no-nullable-getter',
         // flutter config --explicit-package-dependencies
         //'--no-synthetic-package',
-        '--template-arb-file=${prefix ?? 'app'}_en.arb',
+        '--template-arb-file=$template',
         '--arb-dir=$dir',
         '--output-dir=${genDir.path}',
         '--output-localization-file=$outputFile',

lib/src/localize/localizer.dart

@@ -13,23 +13,29 @@ import 'validation.dart';
 
 /// Whether [header] describes a localization sheet.
 ///
-/// The expected layout is `label | description | meta | en | <locale> ...`, so
-/// the fourth column must be the English source. A spreadsheet usually holds
-/// other sheets too — reference tables, notes — whose columns are ordinary
-/// data. Localizing those would overwrite them with translations, so a sheet
-/// that does not match the layout is left alone.
+/// The expected layout is `label | description | meta | source | locale ...`,
+/// so the fourth column is the source language the rest are translated from.
+/// It does not have to be English: `ru`, `de`, `pt_BR` and any other
+/// recognized language code are equally valid sources.
+///
+/// A spreadsheet usually holds other sheets too — reference tables, notes —
+/// whose columns are ordinary data. Localizing those would overwrite them with
+/// translations, so a sheet whose fourth column is not a recognized language
+/// (e.g. a `Family` or `Region` data column) is left alone.
 bool isLocalizationHeader(List<Object?> header) {
   if (header.length < 5) return false;
-  final en = header[3];
-  if (en is! String) return false;
-  final code = normalizeLanguageCode(en);
-  return code == 'en' || code.startsWith('en_');
+  final source = header[3];
+  if (source is! String) return false;
+  return resolveLanguageName(source) != null;
 }
 
 /// Extract cells to be localized from the raw [values] of a sheet.
 ///
 /// [title] is the sheet title, used for diagnostics only.
-/// Column layout: `label | description | meta | en | <locale> ...`.
+/// Column layout: `label | description | meta | <source> | <locale> ...`.
+/// The fourth column is the source language (English by default, but any
+/// recognized language code is accepted); rows are translated from it into the
+/// locale columns that follow.
 /// A sheet whose header does not match that layout is skipped entirely.
 List<LocalizeRow> extractEmptyCells({
   required String title,
@@ -48,12 +54,18 @@ List<LocalizeRow> extractEmptyCells({
   if (!isLocalizationHeader(header)) {
     $err(
       'Sheet "$bucket" is not a localization sheet '
-      '(expected "label | description | meta | en | <locale> ..." header, '
+      '(expected "label | description | meta | <source> | <locale> ..." '
+      'header with a recognized source language in the fourth column, '
       'got ${header.take(4).toList()}), skipping sheet...',
     );
     return const [];
   }
 
+  // The fourth column is the source language every other column is translated
+  // from. `isLocalizationHeader` has already vouched for it being a String and
+  // a recognized language, so the sanitized code is safe to reuse below.
+  final sourceCode = sanitize(header[3] as String);
+
   // Fill locales
   final locales = List.filled(header.length, '', growable: false);
   final seen = <String>{};
@@ -102,8 +114,8 @@ List<LocalizeRow> extractEmptyCells({
         continue;
       }
 
-      // Extract label, description, and meta from the row
-      final [$label, $description, $meta, $english, ..._] = row;
+      // Extract label, description, meta and source text from the row
+      final [$label, $description, $meta, $source, ..._] = row;
       if ($label == null || $label is! String || $label.isEmpty) {
         $err(
           'Sheet "$bucket" has empty label in row #${i + 1}, '
@@ -137,6 +149,7 @@ List<LocalizeRow> extractEmptyCells({
         LocalizeRow(
           row: i,
           label: label,
+          sourceCode: sourceCode,
           description: switch ($description) {
             String text when text.isNotEmpty => text,
             num number => number.toString(),
@@ -147,7 +160,7 @@ List<LocalizeRow> extractEmptyCells({
             num number => number.toString(),
             _ => null,
           },
-          english: switch ($english) {
+          source: switch ($source) {
             String text when text.isNotEmpty => text,
             num number => number.toString(),
             _ => label,
@@ -191,7 +204,8 @@ Stream<LocalizeRow> localizeRows({
     try {
       final (:prompt, :schema) = buildLocalizationPrompt(
         label: row.label,
-        en: row.english,
+        source: row.source,
+        sourceCode: row.sourceCode,
         description: row.description,
         meta: row.meta,
         languages: languages,
@@ -215,7 +229,7 @@ Stream<LocalizeRow> localizeRows({
         _ => null,
       };
       final problem =
-          validateTranslation(source: row.english, translation: text);
+          validateTranslation(source: row.source, translation: text);
       if (problem != null) {
         $err('Rejected "$code" for "${row.label}": $problem');
         failed.add(code);

lib/src/localize/models.dart

@@ -31,8 +31,9 @@ class LocalizeRow {
     required this.label,
     required this.description,
     required this.meta,
-    required this.english,
+    required this.source,
     required this.cells,
+    this.sourceCode = 'en',
   });
 
   /// Zero-based row index in the sheet.
@@ -47,8 +48,14 @@ class LocalizeRow {
   /// Optional ICU/intl placeholder description.
   String? meta;
 
-  /// English source text.
-  String english;
+  /// Language code of the source text, e.g. `en`, `ru`, `pt_BR`.
+  ///
+  /// Taken from the fourth column of the sheet header; the whole row is
+  /// translated *from* this language into the locale columns.
+  String sourceCode;
+
+  /// Source text, written in the [sourceCode] language.
+  String source;
 
   /// Cells (locales) that have to be localized.
   List<LocalizeCell> cells;

lib/src/localize/prompt.dart

@@ -14,10 +14,16 @@ import 'language_names.dart';
 ///
 /// Every target language is spelled out by name, endonym and — for codes that
 /// models routinely misread, such as `uk` — an explicit disambiguation note.
+///
+/// [source] is the source text, written in the [sourceCode] language. The
+/// source language defaults to English but can be any recognized language
+/// code (`ru`, `de`, `pt_BR`, …); the prompt and schema name it explicitly so
+/// the model translates *from* the right language.
 ({String prompt, Map<String, Object?> schema}) buildLocalizationPrompt({
   required String label,
-  required String en,
+  required String source,
   required List<String> languages,
+  String sourceCode = 'en',
   String? description,
   String? meta, // keep as String? to avoid breaking callers; embed as-is
 }) {
@@ -36,13 +42,16 @@ import 'language_names.dart';
   // -- unpack & normalize ----------------------------------------------------
   final normLabel = safeStr(label);
   final normDesc = safeStr(description);
-  final normEn = safeStr(en);
+  final normSource = safeStr(source);
   final langs = uniqLangs(languages);
   final String? metaInline = safeStr(meta); // already a string; embed as-is
 
+  // Human-readable name of the source language, used to instruct the model.
+  final sourceName = resolveLanguageName(sourceCode) ?? sourceCode;
+
   // -- validation (explicit) -------------------------------------------------
   if (normLabel == null) throw ArgumentError('Missing label');
-  if (normEn == null) throw ArgumentError('Missing source English text');
+  if (normSource == null) throw ArgumentError('Missing source text');
   if (langs.isEmpty) throw ArgumentError('No target languages provided');
 
   // -- output skeleton (built once; used in prompt to fix structure) ---------
@@ -73,7 +82,8 @@ import 'language_names.dart';
     p.writeln('meta_placeholders (ICU / intl format): $metaInline');
   }
   p
-    ..writeln('en_source: $normEn')
+    ..writeln('source_language: ${describeLanguage(sourceCode)}')
+    ..writeln('source_text: $normSource')
     ..writeln('--- TARGET LANGUAGES ---')
     ..writeln('The JSON keys below are ISO 639-1 / BCP-47 LANGUAGE codes, '
         'never country codes. Translate into the named language, '
@@ -87,7 +97,8 @@ import 'language_names.dart';
   p
     ..writeln('Write each translation in the natural script of that language '
         '(e.g. Cyrillic for uk/ru/bg, Devanagari for hi, Arabic for ar).')
-    ..writeln('Never answer in English for a non-English language code.')
+    ..writeln('Never answer in $sourceName for a non-$sourceName '
+        'language code.')
     ..writeln('--- OUTPUT REQUIREMENTS ---')
     ..writeln(
         'Return ONLY valid minified JSON (no comments, no markdown fences).')
@@ -97,16 +108,16 @@ import 'language_names.dart';
         '(e.g., {name}, {version}, {count}).')
     ..writeln('Preserve HTML-like or XML-like tags verbatim if present.')
     ..writeln('Do not introduce new placeholders or variables.')
-    ..writeln('If translation would be identical to English, '
-        'repeat the English text.')
+    ..writeln('If translation would be identical to the source, '
+        'repeat the source text.')
     ..writeln('If a translation is infeasible or unclear, '
-        'copy the English text as fallback.')
+        'copy the source text as fallback.')
     ..writeln('Avoid adding periods if original does not have one; '
         'keep stylistic equivalence.')
     ..writeln('No leading/trailing spaces in values.')
     ..writeln('Each value MUST be culturally and medically appropriate, '
         'neutral and concise.')
-    ..writeln('Keep each translation roughly as long as the English source; '
+    ..writeln('Keep each translation roughly as long as the source text; '
         'never pad, repeat or explain.')
     ..writeln('No quotes escaping beyond standard JSON string escaping.')
     ..writeln('--- JSON SCHEMA (informal) ---')
@@ -146,7 +157,7 @@ import 'language_names.dart';
           'type': 'string',
           'minLength': 1,
           if (name != null)
-            'description': 'Translation of en_source into $name '
+            'description': 'Translation of source_text into $name '
                 '($code), written in the native script of that language.',
         },
       },

pubspec.yaml

@@ -9,7 +9,7 @@ description: >
   It uses the Google Sheets API to fetch translations and generates
   Dart localization files for use in Flutter applications.
 
-version: 0.5.0
+version: 0.6.0
 
 homepage: https://github.com/DoctorinaAI/sheety_localization
 repository: https://github.com/DoctorinaAI/sheety_localization

test/client_test.dart

@@ -150,7 +150,7 @@ void main() {
     Map<String, Object?> schemaFor(List<String> languages) =>
         buildLocalizationPrompt(
           label: 'greeting',
-          en: 'Hello',
+          source: 'Hello',
           languages: languages,
         ).schema;