Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 74 additions & 2 deletions docs/openapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -674,6 +674,8 @@ paths:

With `fromRelease` it recomposes that release: its stored plan supplies
the form layer, model and variant, and only what the request names moves.
With `reuseSource: true`, it instead reads the live LLMService plan and
merges arbitrary form overrides and optional edit values, without resolving a catalog.
operationId: composePlan
requestBody:
required: true
Expand Down Expand Up @@ -1805,7 +1807,18 @@ components:
`overrides`; the named fields below exist because their key path or
validation is swissd's business, not the form's.
properties:
fromRelease: { type: string, description: "Recompose this release; release, namespace, model and variant come from its plan." }
fromRelease: { type: string, description: "Recompose this release; release, namespace, model and variant come from its plan. With reuseSource, names the live LLMService in the explicitly supplied namespace." }
reuseSource:
type: boolean
default: false
description: |
Reuse the live LLMService plan named by `fromRelease` and `namespace`.
Both identity fields are required. Merge arbitrary `overridesYAML`
and `overrides` into its form layer; null clears a value. A nonempty
`editsYAML` replaces the edit layer. The source, chart and remaining
layers are reused without catalog resolution. Catalog and source
selection fields are unused in this mode. No Canary or adaptive
concurrency restrictions are enforced; false uses normal composition.
catalog: { type: string, description: "Deploy: the catalog to compose from. Upgrade: a different catalog moves the release (same HF model and engine only)." }
model: { type: string, description: "Required unless `fromRelease`." }
version: { type: string }
Expand Down Expand Up @@ -1861,7 +1874,7 @@ components:
profile: { type: string }
layers:
type: object
description: "One values document per layer, applied in order: `defaults`, `catalog`, `site`, `derived`, `form`, `edit`, `autoscale`. `autoscale` is applied last and wins `replicaCount`."
description: "One values document per layer, applied in order: `defaults`, `catalog`, `site`, `derived`, `form`, `edit`, `autoscale`, `canary-system`. Later layers override earlier ones."
properties:
defaults: { type: object }
catalog: { type: object }
Expand All @@ -1870,6 +1883,7 @@ components:
form: { type: object }
edit: { type: object }
autoscale: { type: object, description: "Only `replicaCount`. swissd writes it when the plan is disaggregated (`pd.enabled` true, `pd.colocate` not true) and the form layer has `sloRequirement.enabled` true, clamping the decision into [1, 64]." }
canary-system: { type: object, description: "Final values imposed by the Canary controller, preserved when reading a live LLMService plan and when reusing its source." }
createNamespace: { type: boolean, default: false, deprecated: true, description: "Use `applyFlags.createNamespace`." }
helmfile: { type: string, description: "The helmfile these values are applied through. Not hashed." }
warnings: { type: array, items: { type: string }, description: "Accepted but worth seeing. Not hashed." }
Expand Down Expand Up @@ -1917,10 +1931,64 @@ components:
status: { type: string, examples: [applied] }
statusError: { type: string, description: "The release is live but its status record was not updated." }

CanaryState:
type: string
enum: [standalone, referenced, owned]
description: |
LLMService relationship derived from Kubernetes metadata. Owned means
Canary ownership labels or an ownerReference are present; referenced
means an assignment annotation is present without ownership metadata;
standalone means neither is present. Ownership takes precedence.
Incomplete labels and malformed annotations retain their relationship
with verified=false in managedBy or assignedTo. Owned services follow
the Canary lifecycle through ownerReferences; referenced services are
retained when the experiment ends. Omitted on Helm releases or when
no LLMService exists. This field imposes no Swiss write restrictions.

CanaryOwner:
type: object
description: |
Canary ownership reported from LLMService owner references or labels.
Omitted when no ownership metadata exists. Incomplete or inconsistent
metadata is returned with verified=false and may contain empty identity
fields. This metadata does not impose Swiss write or SLO restrictions.
required: [apiVersion, kind, namespace, name, uid, verified]
properties:
apiVersion: { type: string, examples: [canary.modelsphere.dev/v1alpha1] }
kind: { type: string, examples: [LLMCanary] }
namespace: { type: string, description: "The owning experiment's namespace." }
name: { type: string, description: "The owning experiment's name." }
uid: { type: string, description: "The owning experiment's UID." }
verified: { type: boolean, description: "True when the controller owner reference has a name and UID and the Canary labels match it. Does not verify that the experiment still exists." }

CanaryAssignment:
type: object
description: |
The LLMService's canary.modelsphere.dev/assignment annotation.
Omitted when the annotation is absent. Malformed or incomplete metadata
is returned with verified=false and may contain empty identity fields
or unrecognized values. This metadata does not impose Swiss write or SLO restrictions.
required: [apiVersion, kind, namespace, name, uid, verified]
properties:
apiVersion: { type: string, examples: [canary.modelsphere.dev/v1alpha1] }
kind: { type: string, examples: [LLMCanary] }
namespace: { type: string, description: "The assigned experiment's namespace." }
name: { type: string, description: "The assigned experiment's name." }
uid: { type: string, description: "The assigned experiment's UID." }
role: { type: string, description: "production or arm for a verified assignment; omitted when empty.", examples: [production, arm] }
arm: { type: string, description: "Required for a verified arm assignment; empty and omitted for a verified production assignment." }
configDigest: { type: string, description: "Assigned experiment configuration digest, when recorded." }
configVersion: { type: string, description: "Assigned experiment configuration version, when recorded." }
verified: { type: boolean, description: "True when identity, role, arm and namespace fields pass assignment validation. Does not verify that the experiment still exists or that its configuration is current." }

Deployment:
type: object
required: [release, namespace, backend]
properties:
canaryState: { $ref: "#/components/schemas/CanaryState" }
uid: { type: string, description: "Kubernetes UID of the LLMService; omitted for Helm releases or when unavailable." }
managedBy: { $ref: "#/components/schemas/CanaryOwner" }
assignedTo: { $ref: "#/components/schemas/CanaryAssignment" }
release: { type: string }
namespace: { type: string }
chart: { type: string, description: "`name-version`." }
Expand Down Expand Up @@ -1972,6 +2040,10 @@ components:
type: object
required: [release, namespace, exists, revision, pods, ready, total, authPrefix]
properties:
canaryState: { $ref: "#/components/schemas/CanaryState" }
uid: { type: string, description: "Kubernetes UID of the LLMService; omitted for Helm releases or when unavailable." }
managedBy: { $ref: "#/components/schemas/CanaryOwner" }
assignedTo: { $ref: "#/components/schemas/CanaryAssignment" }
release: { type: string }
namespace: { type: string }
exists: { type: boolean }
Expand Down
133 changes: 133 additions & 0 deletions internal/llmsvc/canary.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
package llmsvc

import (
"encoding/json"
"strings"

"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
)

const (
CanaryPrefix = "canary.modelsphere.dev/"
AssignmentKey = CanaryPrefix + "assignment"
)

type CanaryOwner struct {
APIVersion string `json:"apiVersion"`
Kind string `json:"kind"`
Namespace string `json:"namespace"`
Name string `json:"name"`
UID string `json:"uid"`
Verified bool `json:"verified"`
}

type CanaryAssignment struct {
APIVersion string `json:"apiVersion"`
Kind string `json:"kind"`
Namespace string `json:"namespace"`
Name string `json:"name"`
UID string `json:"uid"`
Role string `json:"role,omitempty"`
Arm string `json:"arm,omitempty"`
ConfigDigest string `json:"configDigest,omitempty"`
ConfigVersion string `json:"configVersion,omitempty"`
Verified bool `json:"verified"`
}

// CanaryOwnership reports incomplete ownership metadata with Verified=false.
func CanaryOwnership(u *unstructured.Unstructured) *CanaryOwner {
if u == nil {
return nil
}
labels := u.GetLabels()
for _, ref := range u.GetOwnerReferences() {
if strings.HasPrefix(ref.APIVersion, CanaryPrefix) && ref.Kind == "LLMCanary" {
return &CanaryOwner{
APIVersion: ref.APIVersion, Kind: ref.Kind, Namespace: u.GetNamespace(), Name: ref.Name, UID: string(ref.UID),
Verified: ref.Controller != nil && *ref.Controller && ref.UID != "" && ref.Name != "" &&
labels[CanaryPrefix+"canary-name"] == ref.Name && labels[CanaryPrefix+"canary-namespace"] == u.GetNamespace() && labels[CanaryPrefix+"canary-uid"] == string(ref.UID),
}
}
}
for _, key := range []string{"canary-name", "canary-namespace", "canary-uid"} {
if _, ok := labels[CanaryPrefix+key]; ok {
ns := labels[CanaryPrefix+"canary-namespace"]
if ns == "" {
ns = u.GetNamespace()
}
return &CanaryOwner{APIVersion: CanaryPrefix + "v1alpha1", Kind: "LLMCanary", Namespace: ns, Name: labels[CanaryPrefix+"canary-name"], UID: labels[CanaryPrefix+"canary-uid"]}
}
}
return nil
}

// CanaryAssigned reports malformed or incomplete assignments with Verified=false.
func CanaryAssigned(u *unstructured.Unstructured) *CanaryAssignment {
if u == nil {
return nil
}
raw, ok := u.GetAnnotations()[AssignmentKey]
if !ok {
return nil
}
var a CanaryAssignment
if strings.TrimSpace(raw) == "" || json.Unmarshal([]byte(raw), &a) != nil {
return &CanaryAssignment{
APIVersion: CanaryPrefix + "v1alpha1",
Kind: "LLMCanary",
Namespace: u.GetNamespace(),
Verified: false,
}
}
a.APIVersion = strings.TrimSpace(a.APIVersion)
a.Kind = strings.TrimSpace(a.Kind)
a.Namespace = strings.TrimSpace(a.Namespace)
a.Name = strings.TrimSpace(a.Name)
a.UID = strings.TrimSpace(a.UID)
a.Role = strings.TrimSpace(a.Role)
a.Arm = strings.TrimSpace(a.Arm)
a.ConfigDigest = strings.TrimSpace(a.ConfigDigest)
a.ConfigVersion = strings.TrimSpace(a.ConfigVersion)

validRole := a.Role == "production" || a.Role == "arm"
validArm := (a.Role == "arm" && a.Arm != "") || (a.Role == "production" && a.Arm == "")
sameNS := a.Role == "production" || u.GetNamespace() == "" || a.Namespace == u.GetNamespace()
a.Verified = a.APIVersion == CanaryPrefix+"v1alpha1" &&
a.Kind == "LLMCanary" &&
a.Namespace != "" &&
a.Name != "" &&
a.UID != "" &&
validRole &&
validArm &&
sameNS
if a.APIVersion == "" {
a.APIVersion = CanaryPrefix + "v1alpha1"
}
if a.Kind == "" {
a.Kind = "LLMCanary"
}
if a.Namespace == "" {
a.Namespace = u.GetNamespace()
}
return &a
}

type CanaryState string

const (
CanaryStandalone CanaryState = "standalone"
CanaryReferenced CanaryState = "referenced"
CanaryOwned CanaryState = "owned"
)

func CanaryRelationship(u *unstructured.Unstructured) CanaryState {
if CanaryOwnership(u) != nil {
return CanaryOwned
}
if u != nil {
if _, ok := u.GetAnnotations()[AssignmentKey]; ok {
return CanaryReferenced
}
}
return CanaryStandalone
}
95 changes: 95 additions & 0 deletions internal/llmsvc/canary_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
package llmsvc

import (
"testing"

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
)

func TestCanaryOwnershipReportsIncompleteMetadata(t *testing.T) {
controller := true
ref := metav1.OwnerReference{APIVersion: "canary.modelsphere.dev/v1alpha1", Kind: "LLMCanary", Name: "demo", UID: "old-uid", Controller: &controller}
labels := map[string]string{CanaryPrefix + "canary-name": "demo", CanaryPrefix + "canary-namespace": "canary", CanaryPrefix + "canary-uid": "old-uid"}
for _, tc := range []struct {
name string
refs []metav1.OwnerReference
labels map[string]string
protected, verified bool
}{
{"ordinary", nil, nil, false, false},
{"matching", []metav1.OwnerReference{ref}, labels, true, true},
{"removed labels", []metav1.OwnerReference{ref}, nil, true, false},
{"removed reference", nil, labels, true, false},
{"different UID", []metav1.OwnerReference{ref}, map[string]string{CanaryPrefix + "canary-uid": "new-uid"}, true, false},
{"missing namespace label", nil, map[string]string{CanaryPrefix + "canary-name": "demo"}, true, false},
{"empty marker", nil, map[string]string{CanaryPrefix + "canary-name": ""}, true, false},
{"unrelated owner", []metav1.OwnerReference{{APIVersion: "apps/v1", Kind: "Deployment", Name: "demo", UID: "old-uid"}}, nil, false, false},
} {
t.Run(tc.name, func(t *testing.T) {
u := &unstructured.Unstructured{}
u.SetNamespace("canary")
u.SetOwnerReferences(tc.refs)
u.SetLabels(tc.labels)
owner := CanaryOwnership(u)
if owner != nil && owner.Namespace != "canary" {
t.Fatalf("owner namespace %q, want canary", owner.Namespace)
}
if (owner != nil) != tc.protected || (owner != nil && owner.Verified != tc.verified) {
t.Fatalf("%+v", owner)
}
})
}
}

func TestCanarySystemLayerIsEffectiveInReadPlan(t *testing.T) {
u, err := ToUnstructured(&LLMService{Spec: Spec{Chart: ChartSpec{Name: "vllm", Repo: "oci://example/charts", Version: "1.0.0"}, Layers: []Layer{
{Name: "form", Values: map[string]any{"replicaCount": 1, "scaler": map[string]any{"enabled": true}, "modelRoute": map[string]any{"nginx": map[string]any{"route": "arm-name"}}}},
{Name: "canary-system", Values: map[string]any{"scaler": map[string]any{"enabled": false}, "modelRoute": map[string]any{"nginx": map[string]any{"route": "production-route"}}}},
}}})
if err != nil {
t.Fatal(err)
}
p, err := ToPlan(u)
if err != nil {
t.Fatal(err)
}
v := p.Values()
if v["scaler"].(map[string]any)["enabled"] != false || v["modelRoute"].(map[string]any)["nginx"].(map[string]any)["route"] != "production-route" {
t.Fatalf("system policy ignored: %v", v)
}
if p.LayerOf("scaler.enabled") != "canary-system" {
t.Fatal("wrong provenance")
}
}

func TestCanaryAssignedReportsArmAndMalformedMetadata(t *testing.T) {
for _, tc := range []struct {
name string
raw string
present, verified bool
}{
{"none", "", false, false},
{"valid arm", `{"apiVersion":"canary.modelsphere.dev/v1alpha1","kind":"LLMCanary","namespace":"canary","name":"demo","uid":"parent-uid","role":"arm","arm":"baseline","configDigest":"sha256:abc","configVersion":"7"}`, true, true},
{"valid production", `{"apiVersion":"canary.modelsphere.dev/v1alpha1","kind":"LLMCanary","namespace":"canary","name":"demo","uid":"parent-uid","role":"production"}`, true, true},
{"empty annotation", " ", true, false},
{"malformed json", "{not-json", true, false},
{"missing uid", `{"apiVersion":"canary.modelsphere.dev/v1alpha1","kind":"LLMCanary","namespace":"canary","name":"demo","role":"arm","arm":"baseline"}`, true, false},
{"arm without arm name", `{"apiVersion":"canary.modelsphere.dev/v1alpha1","kind":"LLMCanary","namespace":"canary","name":"demo","uid":"parent-uid","role":"arm"}`, true, false},
{"production with arm name", `{"apiVersion":"canary.modelsphere.dev/v1alpha1","kind":"LLMCanary","namespace":"canary","name":"demo","uid":"parent-uid","role":"production","arm":"baseline"}`, true, false},
{"cross namespace production", `{"apiVersion":"canary.modelsphere.dev/v1alpha1","kind":"LLMCanary","namespace":"other","name":"demo","uid":"parent-uid","role":"production"}`, true, true},
{"cross namespace", `{"apiVersion":"canary.modelsphere.dev/v1alpha1","kind":"LLMCanary","namespace":"other","name":"demo","uid":"parent-uid","role":"arm","arm":"baseline"}`, true, false},
} {
t.Run(tc.name, func(t *testing.T) {
u := &unstructured.Unstructured{}
u.SetNamespace("canary")
if tc.present {
u.SetAnnotations(map[string]string{AssignmentKey: tc.raw})
}
got := CanaryAssigned(u)
if (got != nil) != tc.present || (got != nil && got.Verified != tc.verified) {
t.Fatalf("got %+v", got)
}
})
}
}
5 changes: 5 additions & 0 deletions internal/llmsvc/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,11 @@ func (c *Client) Delete(ctx context.Context, ns, name string) error {
return c.resource(ns).Delete(ctx, name, metav1.DeleteOptions{})
}

func (c *Client) DeleteObserved(ctx context.Context, u *unstructured.Unstructured) error {
uid, rv := u.GetUID(), u.GetResourceVersion()
return c.resource(u.GetNamespace()).Delete(ctx, u.GetName(), metav1.DeleteOptions{Preconditions: &metav1.Preconditions{UID: &uid, ResourceVersion: &rv}})
}

// Patch applies a patch and returns the updated object.
func (c *Client) Patch(ctx context.Context, ns, name string, pt types.PatchType, data []byte) (*unstructured.Unstructured, error) {
got, err := c.resource(ns).Patch(ctx, name, pt, data, metav1.PatchOptions{})
Expand Down
2 changes: 1 addition & 1 deletion internal/plan/plan.go
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,7 @@ func (p *Plan) YAML() ([]byte, error) {
}

// Layers are the values documents a release is applied through, in merge order.
var Layers = []string{values.LayerDefaults, values.LayerCatalog, values.LayerSite, values.LayerDerived, values.LayerForm, values.LayerEdit, values.LayerAutoscale}
var Layers = []string{values.LayerDefaults, values.LayerCatalog, values.LayerSite, values.LayerDerived, values.LayerForm, values.LayerEdit, values.LayerAutoscale, values.LayerCanarySystem}

// LayerValues splits the composed values by the layer that set each path, so a
// release is applied through one document per layer rather than one merged file.
Expand Down
Loading
Loading