You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Allow operators to disable Chat Completions persistence by setting the inference store reference to null in a run config:
storage:
stores:
inference: null
Semantics when disabled:
No InferenceStore is constructed, no inference_store table is created, and no background write workers are started — so no chat completion payload is ever persisted.
Streaming and non-streaming Chat Completions keep working.
The history endpoints (list, retrieve, messages) surface that persistence is not configured (HTTP 501) rather than returning empty data or a 404.
💡 Why is this needed? What if we don't build it?
We host OGX internally as a shared Chat Completions service. By default the inference store persists every request/response payload, so the database fills up and there's no way to disable it. On top of that, we need to guarantee zero data retention for our users — something always-on persistence makes impossible.
Other thoughts
Backward compatible by default: omitting the key keeps persistence enabled; only an explicit inference: null disables it. This follows the same optional-store pattern the Responses store already uses.
Other stores (responses, datasets, eval, files, prompts, vector_io) are unaffected.
Is null the right way to express "disable this store", or would a boolean (e.g. inference.persist: false) be preferable? Note the responses store is already default=None, so null matches existing config semantics.
Should disabling persistence also apply to the Responses API / conversations (openai_conversations) store?
Would there be interest in a TTL/retention policy as an alternative to all-or-nothing opt-out?
🚀 Describe the new functionality needed
Allow operators to disable Chat Completions persistence by setting the
inferencestore reference tonullin a run config:Semantics when disabled:
InferenceStoreis constructed, noinference_storetable is created, and no background write workers are started — so no chat completion payload is ever persisted.list,retrieve,messages) surface that persistence is not configured (HTTP 501) rather than returning empty data or a 404.💡 Why is this needed? What if we don't build it?
We host OGX internally as a shared Chat Completions service. By default the inference store persists every request/response payload, so the database fills up and there's no way to disable it. On top of that, we need to guarantee zero data retention for our users — something always-on persistence makes impossible.
Other thoughts
inference: nulldisables it. This follows the same optional-store pattern the Responses store already uses.responses,datasets,eval,files,prompts,vector_io) are unaffected.nullthe right way to express "disable this store", or would a boolean (e.g.inference.persist: false) be preferable? Note theresponsesstore is alreadydefault=None, sonullmatches existing config semantics.openai_conversations) store?inference_storethat this opt-out doesn't cover (cf. Don't write to inference_store when calling Responses API ogx-ai/ogx#2846)?Implementation exists on branch
disable-inference-store(PR: #1).