Migrate appdata to ~/.studio/appdata.json with versioned migrations#2836
Migrate appdata to ~/.studio/appdata.json with versioned migrations#2836bcotrim wants to merge 20 commits intostu-1350-decoupled-config-devfrom
Conversation
| .middleware( async () => { | ||
| try { | ||
| const { migrateAppdata } = await import( 'cli/commands/_migrate' ); | ||
| await migrateAppdata(); | ||
| } catch ( error ) { | ||
| // Migration failure should not block CLI usage | ||
| console.error( 'Appdata migration failed:', error ); | ||
| } | ||
| } ) |
There was a problem hiding this comment.
The Studio app should be responsible for migrating the data (or at least pushing the trigger). If the standalone CLI does it automatically, then we cannot guarantee that the installed version of Studio can handle the current config setup. That's because theoretically, someone can have Studio 1.7.6 installed and install and run the standalone CLI. This code will make the standalone CLI run the migration on every run.
I think this logic should be a check rather than a migration. It should run for every command except _migrate. If the Studio app is installed and ~/.studio/shared.json is missing, it should return an error message prompting the user to update Studio. If Studio isn't installed, just proceed. If Studio is installed and the shared config exists, just proceed.
…nto stu-1350-decoupled-config-dev-v3 # Conflicts: # apps/cli/lib/appdata.ts # apps/cli/lib/tests/appdata.test.ts
fredrikekelund
left a comment
There was a problem hiding this comment.
The basics of the migration works, but we need to address the fact that we still expect the appdata file to contain a sites array in some places.
apps/studio/src/storage/paths.ts
Outdated
| return process.env.DEV_APP_DATA_PATH; | ||
| } | ||
| return path.join( getAppDataPath(), getAppName(), 'appdata-v1.json' ); | ||
| return path.join( getStudioHome(), 'appdata.json' ); |
There was a problem hiding this comment.
Do we want to keep calling it "appdata" or maybe just "app"? I don't really have a preference, but since we're calling the CLI config file "cli.json", I guess there's an argument for "app.json" being a good fit…
| @@ -28,7 +28,7 @@ export interface UserData { | |||
| } | |||
|
|
|||
| export interface PersistedUserData extends Omit< UserData, 'sites' > { | |||
There was a problem hiding this comment.
I commented about this on Slack, too, but appdata.json no longer contains any sites.
| function pickAppdataSiteFields( site: Record< string, unknown > ): Record< string, unknown > { | ||
| const result: Record< string, unknown > = {}; | ||
| for ( const key of Object.keys( site ) ) { | ||
| if ( ! excludedSiteFields.has( key ) ) { | ||
| result[ key ] = site[ key ]; | ||
| } | ||
| } | ||
| // Always keep id so we can correlate with cli.json sites | ||
| if ( site.id ) { | ||
| result.id = site.id; | ||
| } | ||
| return result; | ||
| } |
There was a problem hiding this comment.
I don't understand what this function is for
| if ( Array.isArray( oldData.sites ) ) { | ||
| const sitesWithDesktopFields = oldData.sites | ||
| .map( ( site: Record< string, unknown > ) => pickAppdataSiteFields( site ) ) | ||
| .filter( ( site: Record< string, unknown > ) => | ||
| Object.keys( site ).some( ( key ) => key !== 'id' && site[ key ] !== undefined ) | ||
| ); | ||
|
|
||
| if ( sitesWithDesktopFields.length > 0 ) { | ||
| config.sites = sitesWithDesktopFields; | ||
| } | ||
| } |
There was a problem hiding this comment.
The appdata file shouldn't contain any sites, though, right?
| function buildSharedConfig( oldData: Record< string, unknown > ): Record< string, unknown > { | ||
| return { version: 1, ...pick( oldData, sharedConfigFields ) }; | ||
| } |
There was a problem hiding this comment.
| function buildSharedConfig( oldData: Record< string, unknown > ): Record< string, unknown > { | |
| return { version: 1, ...pick( oldData, sharedConfigFields ) }; | |
| } | |
| function buildSharedConfig( oldData: Record< string, unknown > ): Record< string, unknown > { | |
| const schema = z.object( { ...sharedConfigSchema.omit( { version: true } ).shape } ); | |
| const parsed = schema.parse( oldData ); | |
| return { version: 1, ...parsed }; | |
| } |
I think it'd be a good idea to lean into zod parsing here, too. If oldData doesn't pass validation, then it would cause an error the next time the CLI or the app tries to read that file.
I had to play around with the schema definition a little, but this suggestion works. We want to get rid of the .loose() call from the original definition – that's why we jump through the extra hoops
| if ( Array.isArray( oldData.sites ) ) { | ||
| config.sites = oldData.sites.map( ( site: Record< string, unknown > ) => | ||
| pick( site, cliSiteFields ) | ||
| ); | ||
| } |
There was a problem hiding this comment.
| if ( Array.isArray( oldData.sites ) ) { | |
| config.sites = oldData.sites.map( ( site: Record< string, unknown > ) => | |
| pick( site, cliSiteFields ) | |
| ); | |
| } | |
| const cliSiteSchema = siteDetailsSchema.extend( { | |
| url: z.string().optional(), | |
| latestCliPid: z.number().optional(), | |
| } ); | |
| if ( Array.isArray( oldData.sites ) ) { | |
| config.sites = oldData.sites.map( ( site: Record< string, unknown > ) => | |
| cliSiteSchema.parse( site ) | |
| ); | |
| } |
Same thing here about leaning into zod parsing, because that's the schema we expect the file to conform to later, anyway.
| /** | ||
| * Fields from the old appdata that stay in the new appdata.json. | ||
| * Sites keep only Desktop-specific fields (themeDetails, sortOrder). | ||
| * | ||
| * We derive the "keep" set by excluding fields that moved to shared.json or cli.json. | ||
| */ | ||
| const movedTopLevelFields = new Set( [ ...sharedConfigFields, 'sites', 'snapshots', 'version' ] ); | ||
|
|
||
| // Per-site fields that stay in appdata.json: anything NOT in the CLI site schema, | ||
| // NOT in the snapshot schema, and NOT runtime state. | ||
| const excludedSiteFields = new Set( [ | ||
| ...cliSiteFields, | ||
| ...Object.keys( snapshotSchema.shape ), | ||
| 'running', | ||
| ] ); |
There was a problem hiding this comment.
I don't believe running was ever stored anywhere – it comes from the runtime type. As for themeDetails, we need to figure something out…
We might consider adding an array or a record of themeDetails entries to appdata.json. WDYT? It would take some plumbing to get right and to ensure we clean it up when a site is deleted, but it shouldn't be overly complex.
…ecord<id, AppdataSiteData>
…sing in migration
|
@fredrikekelund thanks for the review! |
|
@fredrikekelund E2E tests should be fixed now. I intentionally left the |
Related issues
Related to STU-1350
How AI was used in this PR
All code was generated via Claude Code (Opus + Haiku). Changes were reviewed and validated through iterative conversation with the user.
Proposed Changes
_migratecommand — Hidden command that copiesappdata-v1.jsonfrom platform-specific Electron path to~/.studio/appdata.json~/.studio/appdata.jsonwith CLI fallback to old platform-specific locationConfigMigratorintools/common/lib/that applies versioned migrations based on file version fieldTesting Instructions
npm run typecheck— Verify no type errorsnpm test— All 1314 tests pass (7 new tests for config migrator)appdata-v1.json→ Verify~/.studio/appdata.jsonis created and Desktop reads from it~/.studio/appdata.jsonis updatedstudio site list→ Verify reads from~/.studio/appdata.jsonPre-merge Checklist