Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added

- **Importers read from filesystem disks.** `laranail::settings.import --disk=<name>` reads an
export file, or the `toolkit` importer's JSON file, from any configured disk, with the source as
the path on it; local paths work as before. In PHP, the new `Contracts\DiskImporter` adds
`readFromDisk(?string $disk, ?string $path, StoreName $store)`, which `ToolkitJsonImporter`
implements through the injected filesystem factory. With no source, the `toolkit` importer reads
toolkit's own file from `laranail.toolkit.settings`' disk and path when that config is loaded,
otherwise `laranail/settings.json` on `local`; toolkit itself is not needed. The command's
`source` argument is now optional. The REST import takes a document, so it has no disk option.

### Fixed

- **Importers no longer overwrite colliding keys in silence.** Source keys that normalise to the
same settings key (differing by case, `-`, `_` or spaces, or toolkit's nested and dotted spelling
of one path) made the later one win with no report. The `spatie`, `toolkit` and `akaunting`
importers now fail with `InvalidImport` naming every colliding group
(`[Mail-Host, mail_host] -> mail_host`) before anything is written, in a dry run and when
applied. Input without collisions imports exactly as before. `ImportedKeys::values()` applies the
same check for a custom importer.

## [0.1.0] - 2026-11-11

The first public release. `Added` describes the package; `Changed`, `Removed`, `Fixed` and
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,7 @@ The hosted docs live at <https://opensource.simtabi.com/documentation/laranail/s
- [REST API quick start](docs/recipes/rest-api-quick-start.md) — enable the API, authenticate and make a first request.
- [Call the REST API with Sanctum](docs/recipes/calling-the-rest-api-with-sanctum.md) — a token with an ability ceiling, ETags and idempotency.
- [Migrate from spatie/laravel-settings](docs/recipes/migrating-from-spatie-laravel-settings.md) — import the values and keep typed classes.
- [Migrate from laranail/toolkit's settings store](docs/recipes/migrating-from-laranail-toolkit.md) — import toolkit's JSON file from a local path or a disk.
- [Build your own admin UI on the REST API](docs/recipes/building-an-admin-ui-on-the-rest-api.md) — forms from definitions, ETag saves, history and activity.
- [Read settings from an SPA or a mobile app](docs/recipes/consuming-from-an-spa-or-mobile-app.md) — cookies or tokens, ETags and safe retries.
- [Build a per-user preferences screen](docs/recipes/building-a-preferences-screen.md) — defaults, `many()`, `setMany()` and resetting.
Expand Down
42 changes: 42 additions & 0 deletions docs/recipes/migrating-from-laranail-toolkit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Migrate from laranail/toolkit's settings store

Move the values in toolkit's JSON `SettingsStore` file into this package, from a local path or any filesystem disk.

## 1. Dry run

```bash
php artisan laranail::settings.import --from=toolkit
```

With no source, the importer reads toolkit's own file: the disk and path in `laranail.toolkit.settings` (`LARANAIL_SETTINGS_DISK`, `LARANAIL_SETTINGS_PATH`) when that config is loaded, otherwise `laranail/settings.json` on the `local` disk. Toolkit does not have to be installed. To name the file yourself:

```bash
php artisan laranail::settings.import /var/www/storage/app/laranail/settings.json --from=toolkit # a local path
php artisan laranail::settings.import laranail/settings.json --from=toolkit --disk=s3 # a path on a disk
```

Nested objects become dot keys (`{"mail": {"host": …}}` is `mail.host`) and a list stays one value.

## 2. Fix collisions, if any

Keys are lower-cased and `-` and spaces become `_`. When two keys land on the same one, for example `Mail-Host` and `mail_host`, or a nested `mail.host` beside a literal `"mail.host"`, the import fails before writing anything and names every group:

```text
toolkit: 1 group of source keys collide after normalising; nothing was imported: [Mail-Host, mail_host] -> mail_host.
```

Keep one key in each group in toolkit's file, then run the dry run again.

## 3. Apply

```bash
php artisan laranail::settings.import --from=toolkit --apply
```

Each value goes through the write pipeline, in one transaction. Then replace `Toolkit::settings()->get('mail.host')` with `settings()->get('mail.host')`.

See [Importers](../tools/importers.md).

---

[← Docs index](../../README.md#documentation)
2 changes: 1 addition & 1 deletion docs/recipes/migrating-from-spatie-laravel-settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ php artisan laranail::settings.import :settings --from=spatie # a dry
php artisan laranail::settings.import :settings --from=spatie --apply --trusted
```

`:settings` is spatie's table on the default connection (`<connection>:<table>`). Each row becomes the key `<group>.<name>`; `--trusted` keeps spatie's locks. Values spatie stored encrypted arrive as spatie's ciphertext: define those keys `encrypted()` here and set them again.
`:settings` is spatie's table on the default connection (`<connection>:<table>`). Each row becomes the key `<group>.<name>`; `--trusted` keeps spatie's locks. Keys are lower-cased and `-` becomes `_`, so two rows such as `general.site_name` and `General.Site-Name` would land on one key: the import then fails, names both, and writes nothing. Rename one in spatie's table and import again. Values spatie stored encrypted arrive as spatie's ciphertext: define those keys `encrypted()` here and set them again.

## 3. Keep the typed classes

Expand Down
2 changes: 1 addition & 1 deletion docs/tools/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Nineteen Artisan commands, all named `laranail::settings.<command>`.
| `laranail::settings.clear-cache [--store=*]` | resets the cached settings of every store, or the named ones, for every tenant |
| `laranail::settings.prune-changes` | marks open change requests past `approvals.expire_after` as expired and deletes decided ones older than `approvals.keep_days` |
| `laranail::settings.export [path] [--store=] [--with-secrets] [--decrypt] [--confirm=]` | writes a store as a JSON document; secrets redacted unless asked for |
| `laranail::settings.import {source} [--store=] [--from=] [--apply] [--trusted]` | imports an export file, or another package's settings through an importer; a dry run without `--apply` |
| `laranail::settings.import {source?} [--store=] [--from=] [--disk=] [--apply] [--trusted]` | imports an export file, or another package's settings through an importer, from a local path or a filesystem disk; a dry run without `--apply` |
| `laranail::settings.diff {path} [--store=]` | compares a store with an export file, by identity only; exits 1 when they differ |
| `laranail::settings.reencrypt` | encrypts every stored secret again under the current key (values, history, change requests) |
| `laranail::settings.rescope {from} {to}` | renames an owner alias in stored values, history and change requests |
Expand Down
3 changes: 3 additions & 0 deletions docs/tools/import-export.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,11 @@ php artisan laranail::settings.diff storage/settings.json # exits 1 whe
php artisan laranail::settings.import storage/settings.json # a dry run
php artisan laranail::settings.import storage/settings.json --apply
php artisan laranail::settings.import storage/settings.json --apply --trusted
php artisan laranail::settings.import exports/settings.json --disk=s3 --apply # read the file from a disk
```

`--disk=<name>` reads the file from any configured filesystem disk, with the source as the path on that disk; without it the source is a local path, as before. The REST import takes the document in the request body, so it has no disk option.

The console command and `POST /stores/{store}/export` stream the document, reading 500 rows at a time, so a large store is never held in memory; the bytes are the ones `export()->toJson()` gives. In PHP, the `StreamExport` action (`handle($request, $stream, $actor)`) writes to any stream; `export()` still returns the whole `ExportDocument`.

An import creates missing categories first, by slug, parents before children, then writes each setting through the normal write pipeline as the actor: validated and cast under the destination's definition, encrypted, with history and one audit entry per key. A setting that needs approval opens a change request instead (the report lists them). Everything runs in one transaction, so one failed write leaves the store as it was. Redacted values, owners the store does not accept and global values for a dedicated store are skipped, each with its reason in the report.
Expand Down
30 changes: 28 additions & 2 deletions docs/tools/importers.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,43 @@ Three importers read another package's settings into an export document, which t
| Name | Reads | Source |
|---|---|---|
| `spatie` | spatie/laravel-settings' table (`group`, `name`, `payload`, `locked`): each row becomes the global key `<group>.<name>`, its JSON payload decoded, its lock kept for a trusted import | `<connection>` or `<connection>:<table>` (default table `settings`) |
| `toolkit` | laranail/toolkit's JSON `SettingsStore` file, flattened (`{"mail": {"host": …}}` is `mail.host`); a list stays one value | the file's absolute path |
| `toolkit` | laranail/toolkit's JSON `SettingsStore` file, flattened (`{"mail": {"host": …}}` is `mail.host`); a list stays one value | the file's absolute path; with `--disk`, the path on that disk; with no source, toolkit's own file |
| `akaunting` | akaunting/laravel-setting's database table (`key`, `value`); rows its extra columns scope to a model or company are skipped | `<connection>` or `<connection>:<table>` |

Keys are lower-cased and `-` and spaces become `_`; a key that still is not valid fails the import with `InvalidImport` naming it.

Two or more source keys that land on the same key (`Mail-Host` and `mail_host`, or toolkit's nested `{"mail": {"host": …}}` beside a literal `"mail.host"`) fail the whole import before anything is written, in a dry run and with `--apply` alike. The `InvalidImport` names every colliding group:

```text
toolkit: 2 groups of source keys collide after normalising; nothing was imported: [A, a] -> a; [b-c, b_c] -> b_c.
```

Rename or remove all but one key in each group at the source, then import again.

```bash
php artisan laranail::settings.import legacy:settings --from=spatie # a dry run
php artisan laranail::settings.import legacy:settings --from=spatie --apply --trusted
php artisan laranail::settings.import /var/www/storage/app/laranail/settings.json --from=toolkit --apply
php artisan laranail::settings.import backups/settings.json --from=toolkit --disk=s3 --apply
php artisan laranail::settings.import --from=toolkit --apply # toolkit's own disk and path
```

## Reading from a disk

The `toolkit` importer reads its file through Laravel's filesystem as well as from a local path. `--disk=<name>` reads the source as a path on that disk. With no source at all it reads toolkit's own file: the disk and path in `laranail.toolkit.settings` (`LARANAIL_SETTINGS_DISK`, `LARANAIL_SETTINGS_PATH`) when that config is loaded, otherwise toolkit's defaults, `laranail/settings.json` on `local`. Toolkit itself is not needed.

In PHP, an importer that reads files implements `Simtabi\Laranail\Settings\Contracts\DiskImporter`:

```php
$importer = app(Importers::class)->get('toolkit'); // a DiskImporter
$document = $importer->readFromDisk('s3', 'backups/settings.json', new StoreName('settings'));
$document = $importer->readFromDisk(null, null, new StoreName('settings')); // toolkit's own disk and path

settings()->import($document);
```

`spatie` and `akaunting` read a database, so they refuse `--disk`. A disk that is not configured, or a path that is not on it, fails with `InvalidImport`.

spatie/laravel-settings and this package both default to a table named `settings`. To read spatie's from the same database, point this package at other names (`tables.prefix` or `stores.settings.table`) before installing, or read spatie's table from another connection.

Values spatie stored encrypted arrive as spatie's ciphertext: define those keys `encrypted()` here and set them again.
Expand Down Expand Up @@ -48,7 +74,7 @@ final readonly class LegacyIniImporter implements Importer
],
```

An importer only reads. It is built from the container, so it can take its dependencies in its constructor.
An importer only reads. It is built from the container, so it can take its dependencies in its constructor. To accept `--disk`, implement `DiskImporter` and its `readFromDisk(?string $disk, ?string $path, StoreName $store)`, reading through an injected `Illuminate\Contracts\Filesystem\Factory`. `ImportedKeys::values()` normalises keys and refuses collisions the way the shipped importers do.

---

Expand Down
46 changes: 42 additions & 4 deletions src/Console/ImportCommand.php
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,13 @@

namespace Simtabi\Laranail\Settings\Console;

use InvalidArgumentException;
use Illuminate\Filesystem\Filesystem;
use Simtabi\Laranail\Settings\Data\Actor;
use Simtabi\Laranail\Settings\Data\StoreName;
use Simtabi\Laranail\Settings\Contracts\Importer;
use Simtabi\Laranail\Console\Tools\Commands\Command;
use Simtabi\Laranail\Settings\Contracts\DiskImporter;
use Simtabi\Laranail\Settings\Exceptions\InvalidImport;
use Simtabi\Laranail\Settings\Data\Transfer\ImportReport;
use Simtabi\Laranail\Settings\Data\Transfer\ImportRequest;
Expand All @@ -16,37 +19,44 @@
use Simtabi\Laranail\Settings\Data\Transfer\ExportDocument;
use Simtabi\Laranail\Settings\Exceptions\SettingsException;
use Simtabi\Laranail\Settings\Services\Stores\StoreRegistry;
use Illuminate\Contracts\Filesystem\Factory as FilesystemFactory;
use Simtabi\Laranail\Console\Tools\Commands\Concerns\SupportsNamespacedNames;

/**
* Import an export file, or another package's settings with `--from=<importer>`. A dry run unless
* `--apply`; `--trusted` lets the file set locks and system categories.
* `--apply`; `--trusted` lets the file set locks and system categories. `--disk` reads the file
* from a filesystem disk, for an export file or an importer that reads files.
*/
final class ImportCommand extends Command
{
use SupportsNamespacedNames;

protected $signature = 'laranail::settings.import
{source : An export file, or what the importer reads (a connection, connection:table or file)}
{source? : An export file, or what the importer reads (a connection, connection:table or file); a path on the disk with --disk}
{--store= : The store (default: the default store)}
{--from= : Read with an importer instead of an export file}
{--disk= : Read the file from this filesystem disk instead of a local path}
{--apply : Write the changes (without it, a dry run)}
{--trusted : Let the file set locks and system categories}';

protected $description = 'Import settings from an export file or another package (dry run by default)';

public function handle(ImportStore $import, StoreRegistry $stores, Importers $importers, Filesystem $files): int
public function handle(ImportStore $import, StoreRegistry $stores, Importers $importers, Filesystem $files, FilesystemFactory $disks): int
{
$source = $this->argument('source');
$source = is_string($source) ? $source : '';
$from = $this->option('from');
$disk = $this->option('disk');
$disk = is_string($disk) && $disk !== '' ? $disk : null;
$storeOption = $this->option('store');

try {
$store = is_string($storeOption) && $storeOption !== '' ? $stores->get(new StoreName($storeOption)) : $stores->default();

if (is_string($from) && $from !== '') {
$document = $importers->get($from)->read($source, $store->name);
$document = $this->readWith($importers->get($from), $from, $source, $disk, $store->name);
} elseif ($disk !== null) {
$document = ExportDocument::fromJson($this->readDisk($disks, $disk, $source));
} else {
if (! $files->isFile($source)) {
throw new InvalidImport(sprintf('[%s] is not a file.', $source));
Expand All @@ -73,6 +83,34 @@ public function handle(ImportStore $import, StoreRegistry $stores, Importers $im
return self::SUCCESS;
}

/** @throws InvalidImport when `--disk` is given for an importer that does not read files */
private function readWith(Importer $importer, string $name, string $source, ?string $disk, StoreName $store): ExportDocument
{
if ($disk === null) {
return $importer->read($source, $store);
}

if (! $importer instanceof DiskImporter) {
throw new InvalidImport(sprintf('%s: this importer does not read from a disk; drop --disk.', $name));
}

return $importer->readFromDisk($disk, $source === '' ? null : $source, $store);
}

/** @throws InvalidImport when the disk or the file on it cannot be read */
private function readDisk(FilesystemFactory $disks, string $disk, string $path): string
{
try {
$filesystem = $disks->disk($disk);
} catch (InvalidArgumentException $exception) {
throw new InvalidImport(sprintf('Disk [%s] cannot be used (%s).', $disk, $exception->getMessage()), 0, $exception);
}

$json = $path !== '' && $filesystem->exists($path) ? $filesystem->get($path) : null;

return $json ?? throw new InvalidImport(sprintf('[%s] does not exist on disk [%s].', $path, $disk));
}

private function report(ImportReport $report): void
{
$this->info(trans_choice('laranail/settings::settings.console.imported', count($report->categories), [
Expand Down
24 changes: 24 additions & 0 deletions src/Contracts/DiskImporter.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
<?php

declare(strict_types=1);

namespace Simtabi\Laranail\Settings\Contracts;

use Simtabi\Laranail\Settings\Data\StoreName;
use Simtabi\Laranail\Settings\Exceptions\InvalidImport;
use Simtabi\Laranail\Settings\Data\Transfer\ExportDocument;

/**
* An importer whose source is a file it can also read from a Laravel filesystem disk (`s3`, `sftp`,
* ...). `laranail::settings.import --from=<name> --disk=<disk>` calls it; an importer without it
* refuses `--disk`.
*/
interface DiskImporter extends Importer
{
/**
* `$disk` and `$path` may be null for the importer's own default, if it has one.
*
* @throws InvalidImport when the disk or file cannot be read or holds something unusable
*/
public function readFromDisk(?string $disk, ?string $path, StoreName $store): ExportDocument;
}
6 changes: 3 additions & 3 deletions src/Importers/AkauntingSettingsImporter.php
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ public function read(string $source, StoreName $store): ExportDocument
throw new InvalidImport(sprintf('akaunting: table [%s] on connection [%s] cannot be read (%s).', $table, $connection ?: 'default', $exception->getMessage()), 0, $exception);
}

$values = [];
$entries = [];

foreach ($rows as $row) {
$row = (array) $row;
Expand All @@ -47,9 +47,9 @@ public function read(string $source, StoreName $store): ExportDocument
continue;
}

$values[ImportedKeys::key(is_string($row['key']) ? $row['key'] : '', 'akaunting')->value] = [$row['value'], false];
$entries[] = [is_string($row['key']) ? $row['key'] : '', $row['value'], false];
}

return ImportedKeys::document($store, $values, 'akaunting');
return ImportedKeys::document($store, ImportedKeys::values($entries, 'akaunting'), 'akaunting');
}
}
Loading
Loading