Repository navigation
feat(config): declare AI providers and recommendations in the deployment #4668
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
d70a8b1
eb5d31c
c7622a0
37b1e6b
0955181
a353dad
0d2c685
5d3bc97
de5b0fd
640af99
461a39f
ab45d93
8982f78
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -937,11 +937,24 @@ func main() { | |
| // SILO_DB_MAX_CONNECTIONS sizes it. | ||
| isPrimaryNode := bc.Mode == "integrated" || bc.Mode == "api" || bc.Mode == "" | ||
| var envSettings config.EnvSettings | ||
| // The deployment document (SILO_CONFIG or SILO_CONFIG_FILE) is read with | ||
| // them: its AI providers are stored with the environment's settings, and | ||
| // its plugins section goes to the provisioning writer once plugins load. | ||
| var deploymentDocument *plugins.ProvisionDocument | ||
| var declaredAIProviders deploymentAIProviders | ||
| if isPrimaryNode && !*migrateOnly && !*migrateStatus && *migrateDownTo < 0 { | ||
| envSettings, err = config.ReadEnvSettings(os.Getenv) | ||
| if err != nil { | ||
| log.Fatalf("%v", err) | ||
| } | ||
| if deploymentDocument, err = plugins.ReadProvisionDocument(); err != nil { | ||
| log.Fatalf("read deployment configuration: %v", err) | ||
| } | ||
| providers, declared, err := deploymentDocument.DeclaredAIProviders() | ||
| if err != nil { | ||
| log.Fatalf("%s: %v", deploymentDocument.Variable(), err) | ||
| } | ||
| declaredAIProviders = deploymentAIProviders{declared: declared, providers: providers, variable: deploymentDocument.Variable()} | ||
| } | ||
|
|
||
| // Step 2: Connect to PostgreSQL (bootstrap pool sized by | ||
|
|
@@ -1067,7 +1080,7 @@ func main() { | |
| // The settings the environment sets win over the stored ones. Cluster-wide | ||
| // ones are stored so the admin UI and the other servers see the values in | ||
| // effect; per-process ones only overlay this process's settings. | ||
| envSettings, envChangedKeys, err := applyEnvSettings(ctx, settingsRepo, settings, envSettings, bc.RedisURL) | ||
| envSettings, envChangedKeys, err := applyEnvSettings(ctx, settingsRepo, settings, envSettings, declaredAIProviders, bc.RedisURL) | ||
| if err != nil { | ||
| log.Fatalf("%v", err) | ||
| } | ||
|
|
@@ -2077,12 +2090,17 @@ func main() { | |
| installer, | ||
| plugins.NewHostAdapter(pluginHost), | ||
| ) | ||
| provisionDocument, err := plugins.ReadProvisionDocument() | ||
| if err != nil { | ||
| log.Fatalf("read plugin provisioning: %v", err) | ||
| // A frontend server provisions plugins but stores no settings from | ||
| // its environment, so it reads the document here. | ||
| if bc.Mode == "frontend" { | ||
| if deploymentDocument, err = plugins.ReadProvisionDocument(); err != nil { | ||
| log.Fatalf("read deployment configuration: %v", err) | ||
| } | ||
| } | ||
| if provisionDocument != nil { | ||
| pluginProvisionWriter = plugins.NewProvisionWriter(deps.DB, repositoryStore, installationStore, installer, catalogService, runtimeConfigStore, pluginService, provisionDocument, | ||
| // A deployment document without a plugins section leaves plugin | ||
| // installations as they are, as no document does. | ||
| if deploymentDocument.DeclaresPlugins() { | ||
| pluginProvisionWriter = plugins.NewProvisionWriter(deps.DB, repositoryStore, installationStore, installer, catalogService, runtimeConfigStore, pluginService, deploymentDocument, | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. S3 · P2 — A rejected plugin declaration still applies the document's AI settings. Combine an AI provider or recommendation change with a plugin field that passes the document schema and fails its cached manifest schema. The AI settings commit at line 1077 and publish at line 1250.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not changed. On main, the environment settings are also stored and announced before the plugin declaration is checked against its manifest. The document's AI section follows the same order. Both are applied again on the next start, which by then has the corrected document, so the outcome is the one the operator asked for. |
||
| metadata.ApplyProvisionedMetadataPlacement) | ||
| runtimeConfigStore.SetProvisionWriter(pluginProvisionWriter) | ||
| deps.PluginProvisionStatus = runtimeConfigStore.ListProvisionStatus | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
S1 · P1 — Legacy conversion can make a valid deployment fail to start. Upgrade a pre-registry install with recommendations enabled and no explicit embedding URL, and declare
ollama-local: {type: ollama}withSILO_RECOMMENDATIONS_PROVIDER=ollama-local.main.go:1045runs legacy conversion first. It commits anOllamaprovider with a generated ID and advances the registry version. This merge at line 61 then rejects the declared provider's default name, even though the explicit binding would supersede the converted one. A retry fails on the same committed row. Plan the conversion and declaration together in the settings transaction. Honor explicit bindings, reserve declared names when generating legacy entries, and commit the conversion version only after validation succeeds.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not changed. The converted provider holds the name "Ollama", so a declared ollama provider without a
namewould take a name that's already used. Provider names have to be unique in the registry, and planning both steps in one transaction would still hit that. Startup stops with an error that names the converted provider's id and both fixes: give the declared provider a name of its own, or declare that id to take the converted provider over. The docs PR shows that error and both fixes.