GPT-5 Responses taking a long time to complete

[richarddas]

I've dug in and implemented GPT-5 into my app, and although I understand it's a reasoning model, responses were taking 20s or more to complete.

I enabled streaming, and there is still a very long delay before the stream begins.

I am very familiar that GPT-5 is a reasoning model, and therefore takes longer (and burns 10× tokens compared to 4.1!!! 😳) but I'm just wondering if going via AIProxy is introducing any latency, or if there is anything in our control that can speed things up?

I've tried turning the reasoning effort down to .minimal (which throws an error as the OAI API doesn't recognise it, see my comment on the other ticket!) but even on .low time to first token is still very high.

I've opened this as a ticket in order to discuss, I understand it might not all be under AIP control, but I wanted to see if there's either something we can debug, or maybe just something to include in docs?

[lzell]

I've tried turning the reasoning effort down to .minimal (which throws an error as the OAI API doesn't recognise it, see my comment on the other ticket!)

Hi @richarddas, did you submit that through our dashboard support or through github? I'm not seeing the ticket

FWIW we introduce no new backend logic for gpt-5, we are just forwarding the request on exactly like we were for other models.

One way you can see this is by switching from the AIProxy.openAIService to AIProxy.directOpenAIService and use your own private key (obviously don't ship an app like this). If you do that, our cloud infra will be completely out of the mix, the requests will go straight to OpenAI

[richarddas]

Apologies 🤦🏻‍♂️ this should have been a support ticket not a GitHub issue 🤣

Thanks for the heads up, I wasn't aware of the AIProxy.directOpenAIService API, that would be useful for debugging.

For now I'm sticking to 4.1, but my understanding is that the ChatGPT product uses a router to direct requests to the appropriate models that the API does not have. That's why when switching my app to use the GPT-5 model, every request takes a longer time (comparatively) to complete, and burns 10× tokens even for simple requests.

[GeorgeKouk]

Just adding that I'm also experiencing 20s + message responses times for gpt-5-mini and it seems the reasoning_effort parameter is what controls the effort put into reasoning which by default is 'medium'

[lzell]

Yes, that is correct. You can adjust that as follows:

    let requestBody = OpenAICreateResponseRequestBody(
        input: .text("hello world"),
        model: "gpt-5",
        reasoning: .init(effort: .minimal, summary: .concise) // <-- Change to suit your needs
    )

[GeorgeKouk]

Thanks Lou, it seems .minimal is not a valid effort level (for gpt-5-mini at least). I could only get a response with .low, .minimal results in "Cannot initialise effort from Invalid String value minimal".

Perhaps its only available for larger models I am not sure, but thought I'd let you know

[lzell]

@GeorgeKouk interesting! Thanks for letting me know