Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -220,6 +220,11 @@ SILO_DATA_ROOT=/opt/silo
# SILO_TMDB_API_KEY=
# SILO_MDBLIST_API_KEY=
# SILO_MEILISEARCH_INDEX=silo_media_items
# Recommendations. SILO_RECOMMENDATIONS_PROVIDER names a connected AI provider
# that serves embeddings, by id, such as one the deployment document declares.
# SILO_RECOMMENDATIONS_ENABLED=true
# SILO_RECOMMENDATIONS_PROVIDER=
# SILO_RECOMMENDATIONS_MODEL=
# Storage: SILO_ARTWORK_STORAGE (auto, local or s3), SILO_ARTWORK_PATH, and
# SILO_S3_PUBLIC_* and SILO_S3_PRIVATE_* with ENDPOINT, REGION, BUCKET,
# KEY_PREFIX, PATH_STYLE, ACCESS_KEY and SECRET_KEY, plus
Expand All @@ -234,6 +239,13 @@ SILO_DATA_ROOT=/opt/silo
# SILO_SCANNER_WORKERS=
# SILO_MATCHER_WORKERS=

# Deployment document: JSON inline in SILO_CONFIG, or a JSON or YAML file named
# by SILO_CONFIG_FILE. It installs and configures official plugins (plugins:)
# and connects AI providers (ai_providers:), each keyed by its id, with secrets
# given as {fromFile: <path>} or {fromEnv: <VARIABLE>}. SILO_PLUGINS and
# SILO_PLUGINS_FILE name the same document; set only one of the four.
# SILO_CONFIG_FILE=/config/deployment.yaml

# Run from source / advanced overrides
# DATABASE_URL and SECRET_KEY are required when running Silo outside the default
# Docker Compose file. REDIS_URL is optional for integrated/API mode and
Expand Down
37 changes: 33 additions & 4 deletions cmd/silo/env_settings.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,30 +23,58 @@ func bootstrapPoolSize(env config.EnvSettings) int {
return 20
}

// deploymentAIProviders are the AI providers the deployment document
// declares. Without an ai_providers section (declared false) the stored
// registry is left as it is; an empty section releases every provider.
type deploymentAIProviders struct {
declared bool
providers []config.AIProvider
variable string
}

// applyEnvSettings checks the settings the environment sets the way an admin
// save checks them, stores the cluster-wide ones that differ from stored in
// one batch (through the encrypting repo, so secrets rest encrypted), and
// overlays every one on settings, the map startup builds its config from. It
// returns the checked values and the stored keys it changed.
//
// The AI providers the deployment document declares go into the registry in
// the same batch, checked with the environment's values, so a
// recommendations variable can name one of them and the recommendation
// engine starts with them on this start.
//
// The check and the write share one settings transaction, as an admin save's
// do, so the check sees the rows the write replaces: a storage transition
// another server commits while this one starts cannot slip between them.
func applyEnvSettings(ctx context.Context, repo *catalog.EncryptedSettingsRepo, settings map[string]string, env config.EnvSettings, redisBootstrapURL string) (config.EnvSettings, []string, error) {
if len(env) == 0 {
func applyEnvSettings(ctx context.Context, repo *catalog.EncryptedSettingsRepo, settings map[string]string, env config.EnvSettings, ai deploymentAIProviders, redisBootstrapURL string) (config.EnvSettings, []string, error) {
if len(env) == 0 && !ai.declared {
return env, nil, nil
}
normalizedRedis, redisErr := config.NormalizeRedisURL(redisBootstrapURL)
redisBootstrapAvailable := normalizedRedis != "" && redisErr == nil
var checked config.EnvSettings
var changed, snapshot map[string]string
var changed, snapshot, declared map[string]string
var checkErr error
err := repo.UpdateAtomic(ctx, func(current map[string]string) (map[string]string, error) {
if checked, checkErr = handlers.CheckEnvironmentSettings(current, env, redisBootstrapAvailable); checkErr != nil {
declared = nil
if ai.declared {
registry, err := config.ApplyDeclaredAIProviders(current[config.AIProvidersSettingKey], ai.providers, ai.variable)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

S1 · P1 — Legacy conversion can make a valid deployment fail to start. Upgrade a pre-registry install with recommendations enabled and no explicit embedding URL, and declare ollama-local: {type: ollama} with SILO_RECOMMENDATIONS_PROVIDER=ollama-local. main.go:1045 runs legacy conversion first. It commits an Ollama provider with a generated ID and advances the registry version. This merge at line 61 then rejects the declared provider's default name, even though the explicit binding would supersede the converted one. A retry fails on the same committed row. Plan the conversion and declaration together in the settings transaction. Honor explicit bindings, reserve declared names when generating legacy entries, and commit the conversion version only after validation succeeds.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed. The converted provider holds the name "Ollama", so a declared ollama provider without a name would take a name that's already used. Provider names have to be unique in the registry, and planning both steps in one transaction would still hit that. Startup stops with an error that names the converted provider's id and both fixes: give the declared provider a name of its own, or declare that id to take the converted provider over. The docs PR shows that error and both fixes.

if err != nil {
checkErr = fmt.Errorf("%s: ai_providers: %w", ai.variable, err)
return nil, checkErr
}
declared = map[string]string{config.AIProvidersSettingKey: registry}
}
if checked, checkErr = handlers.CheckDeploymentSettings(current, env, declared, ai.variable, redisBootstrapAvailable); checkErr != nil {
return nil, checkErr
}
snapshot = maps.Clone(current)
changed = map[string]string{}
for key, value := range declared {
if current[key] != value {
changed[key] = value
}
}
for key, value := range checked.Values(config.EnvScopeStored) {
if current[key] != value {
changed[key] = value
Expand All @@ -65,6 +93,7 @@ func applyEnvSettings(ctx context.Context, repo *catalog.EncryptedSettingsRepo,
// the earlier read, which another server may have changed since.
clear(settings)
maps.Copy(settings, snapshot)
maps.Copy(settings, declared)
maps.Copy(settings, checked.All())
return checked, slices.Sorted(maps.Keys(changed)), nil
}
Expand Down
30 changes: 24 additions & 6 deletions cmd/silo/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -937,11 +937,24 @@ func main() {
// SILO_DB_MAX_CONNECTIONS sizes it.
isPrimaryNode := bc.Mode == "integrated" || bc.Mode == "api" || bc.Mode == ""
var envSettings config.EnvSettings
// The deployment document (SILO_CONFIG or SILO_CONFIG_FILE) is read with
// them: its AI providers are stored with the environment's settings, and
// its plugins section goes to the provisioning writer once plugins load.
var deploymentDocument *plugins.ProvisionDocument
var declaredAIProviders deploymentAIProviders
if isPrimaryNode && !*migrateOnly && !*migrateStatus && *migrateDownTo < 0 {
envSettings, err = config.ReadEnvSettings(os.Getenv)
if err != nil {
log.Fatalf("%v", err)
}
if deploymentDocument, err = plugins.ReadProvisionDocument(); err != nil {
log.Fatalf("read deployment configuration: %v", err)
}
providers, declared, err := deploymentDocument.DeclaredAIProviders()
if err != nil {
log.Fatalf("%s: %v", deploymentDocument.Variable(), err)
}
declaredAIProviders = deploymentAIProviders{declared: declared, providers: providers, variable: deploymentDocument.Variable()}
}

// Step 2: Connect to PostgreSQL (bootstrap pool sized by
Expand Down Expand Up @@ -1067,7 +1080,7 @@ func main() {
// The settings the environment sets win over the stored ones. Cluster-wide
// ones are stored so the admin UI and the other servers see the values in
// effect; per-process ones only overlay this process's settings.
envSettings, envChangedKeys, err := applyEnvSettings(ctx, settingsRepo, settings, envSettings, bc.RedisURL)
envSettings, envChangedKeys, err := applyEnvSettings(ctx, settingsRepo, settings, envSettings, declaredAIProviders, bc.RedisURL)
if err != nil {
log.Fatalf("%v", err)
}
Expand Down Expand Up @@ -2077,12 +2090,17 @@ func main() {
installer,
plugins.NewHostAdapter(pluginHost),
)
provisionDocument, err := plugins.ReadProvisionDocument()
if err != nil {
log.Fatalf("read plugin provisioning: %v", err)
// A frontend server provisions plugins but stores no settings from
// its environment, so it reads the document here.
if bc.Mode == "frontend" {
if deploymentDocument, err = plugins.ReadProvisionDocument(); err != nil {
log.Fatalf("read deployment configuration: %v", err)
}
}
if provisionDocument != nil {
pluginProvisionWriter = plugins.NewProvisionWriter(deps.DB, repositoryStore, installationStore, installer, catalogService, runtimeConfigStore, pluginService, provisionDocument,
// A deployment document without a plugins section leaves plugin
// installations as they are, as no document does.
if deploymentDocument.DeclaresPlugins() {
pluginProvisionWriter = plugins.NewProvisionWriter(deps.DB, repositoryStore, installationStore, installer, catalogService, runtimeConfigStore, pluginService, deploymentDocument,

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

S3 · P2 — A rejected plugin declaration still applies the document's AI settings. Combine an AI provider or recommendation change with a plugin field that passes the document schema and fails its cached manifest schema. The AI settings commit at line 1077 and publish at line 1250. StartOffline can then return a local validation refusal and stop the process at line 2099. TestProvisionWriterRefusesInvalidSavedConfigDB covers that refusal. Other replicas can see the changed registry even though the rolling restart failed. Run the offline plugin validation that can abort startup before committing or announcing the shared AI settings.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed. On main, the environment settings are also stored and announced before the plugin declaration is checked against its manifest. The document's AI section follows the same order. Both are applied again on the next start, which by then has the corrected document, so the outcome is the one the operator asked for.

metadata.ApplyProvisionedMetadataPlacement)
runtimeConfigStore.SetProvisionWriter(pluginProvisionWriter)
deps.PluginProvisionStatus = runtimeConfigStore.ListProvisionStatus
Expand Down
Loading
Loading