[scotwells]

Summary

New projects and other resources could vanish from search right after they were created and stay missing until they changed again. In production, a newly created project was indexed and then deleted about 15ms later by a delete for the same document. This change sends deletes only to the indexes that could hold the resource, and makes sure an older upsert or delete can no longer override a newer one within an indexer replica. That shrinks the race and closes it inside one replica.

Why

• Delete churn. Every update to any resource sent a delete to every index whose policy it failed, including indexes for unrelated kinds. The Project index alone received about 1,550 delete tasks in 90 minutes, almost all deleting nothing, and every update anywhere fanned out to all nine indexes. Each of those deletes was a chance to race a newer upsert.
• Racing buffers. Upserts and deletes sat in separate buffers and went to Meilisearch as separate, parallel tasks. When an early update during creation queued a delete and a later one queued an upsert, both were sent, and whichever landed last won.

Changes

Scope deletes to the resource's type

• A non-matching update or patch deletes only from indexes whose policy targets the object's API group and kind. The version is ignored, since one object can be written through any served version.
• Terminating resources are scoped to their kind, read from the response object.
• Delete-verb events are scoped to their API group only. Their response is usually a Status and the audit object reference names only the plural resource, so the kind can't be determined reliably. Every policy in the group still gets the delete, so ghost-document cleanup is unchanged.
• Single-tenant handling of delete events is unchanged.

One latest operation per document

• The batcher buffers one pending operation per index and document, so the latest upsert or delete replaces any earlier one by construction.
• Upserts and deletes flush together as one batch on one flush slot. Each message is tracked once and acked once, only after the batch carrying its operations, or the ones that replaced them, succeeds.
• Each batch enqueues its Meilisearch tasks only after the batch taken before it has enqueued, so an older batch still sending can't land after a newer one. Waiting on tasks stays concurrent. A batch waits for its turn before taking a flush slot, so the in-flight cap can't deadlock. A batch abandoned at shutdown stays unacked for redelivery.
• Flush metrics gain a mixed type label for batches carrying both operation types.

Not covered

• Ordering across replicas. 3 to 10 indexer replicas share the consumer, and one resource's events can be split across them, so two replicas can still enqueue conflicting operations out of order.
• Nack redelivery. A failed batch is redelivered after newer operations may have landed, and can reapply a stale delete.
• Follow-up: a periodic reconcile of the indexes against Milo would repair documents that either gap leaves missing.

Test plan

• New consumer tests: an unrelated-kind update queues no delete; a same-kind non-matching update deletes from that index only, across versions; terminating resources are scoped to their kind; delete-verb events are scoped to their API group
• New batcher tests: delete then upsert keeps the document, upsert then delete removes it, a later batch waits for an earlier one to enqueue, and shutdown while waiting leaves the message unacked. Each asserts exactly-once acks
• go build ./..., go vet ./... and go test ./... pass, and go test ./internal/indexer/... -race passes three times, including the existing no-split, in-flight bound and ack-progress tests
• After release, confirm delete tasks on the Project index drop sharply and a newly created project stays searchable