test: add coverage for Cloudflare AI timeout fallback behavior

[whoabuddy]

Context

During review of PR #69 (#69), Copilot noted that the new timeout-fallback behavior and fallback_model response field are not covered by tests.

What needs testing

Add tests that cover the Cloudflare AI timeout fallback path:

1. Simulate timeout error: Mock c.env.AI.run to throw an error with code 3046 (timeout) on the primary model
2. Verify fallback retry: Confirm the handler retries the request with the fallback model after the primary model times out
3. Verify metadata in non-streaming responses: Assert that the fallback_model field is present and correct in the JSON response body when fallback is triggered
4. Verify metadata in streaming responses: Assert that the X-Fallback-Model header is set correctly when the fallback model is used in streaming mode
5. Verify normal path: Ensure that when the primary model succeeds, no fallback metadata is present

Acceptance criteria

• Test exists that triggers code 3046 timeout on primary model
• Test verifies fallback model is called after timeout
• Test checks fallback_model field in non-streaming JSON response
• Test checks X-Fallback-Model header in streaming response
• Test confirms no fallback metadata when primary model succeeds

[arc0btc]

Reviewed PR #69 and the CloudflareChat handler. Here's a concrete test plan with code.

Test structure

The existing tests/ files are integration tests that require live credentials and a running environment. The timeout fallback tests need to mock c.env.AI.run, so they'd fit best as unit tests using Bun's built-in test runner (bun test), separate from the integration suite.

Suggested file: tests/cloudflare-timeout-fallback.test.ts

import { describe, expect, test, mock, beforeEach } from "bun:test";

// --- Minimal test doubles ---

const mockLogger = {
  info: () => {},
  warn: () => {},
  error: () => {},
  debug: () => {},
};

const mockExecutionCtx = { waitUntil: (_p: Promise<unknown>) => {} };

const DEFAULT_CF_MODEL = "@cf/meta/llama-3.1-8b-instruct";
const FALLBACK_CF_MODEL = "@cf/meta/llama-3.3-70b-instruct-fp8-fast";

function makeSuccessResponse(model: string) {
  return { response: "Hello!", model };
}

function makeContext(aiRun: (model: string, opts: unknown) => unknown) {
  return {
    env: {
      AI: { run: aiRun },
      USAGE_DO: undefined,
    },
    var: { logger: mockLogger, requestId: "req-test", x402: undefined },
    executionCtx: mockExecutionCtx,
    req: {
      raw: { headers: new Headers() },
      json: async () => ({
        model: DEFAULT_CF_MODEL,
        messages: [{ role: "user", content: "Hello" }],
      }),
      query: (_k: string) => undefined,
    },
    json: (body: unknown) => new Response(JSON.stringify(body), { headers: { "Content-Type": "application/json" } }),
    header: (_k: string, _v: string) => {},
    status: (_s: number) => {},
  };
}

// --- Helper: parse JSON from Response ---
async function parseJson(r: Response) {
  return r.json();
}

// --- Tests ---

describe("CloudflareChat timeout fallback", () => {
  test("non-streaming: returns fallback_model in response when primary times out", async () => {
    let callCount = 0;
    const aiRun = mock((model: string, _opts: unknown) => {
      callCount++;
      if (model === DEFAULT_CF_MODEL) {
        throw new Error("3046: Request timeout");
      }
      return makeSuccessResponse(model);
    });

    const { CloudflareChat } = await import("../src/endpoints/inference/cloudflare/chat");
    const handler = new CloudflareChat();
    const ctx = makeContext(aiRun as unknown as (m: string, o: unknown) => unknown);
    const response = await handler.handle(ctx as unknown as AppContext);
    const body = await parseJson(response as Response);

    expect(callCount).toBe(2); // primary + fallback
    expect(body.ok).toBe(true);
    expect(body.fallback_model).toBe(FALLBACK_CF_MODEL);
    expect(body.model).toBe(DEFAULT_CF_MODEL); // original requested model preserved
  });

  test("streaming: sets X-Fallback-Model header when primary times out", async () => {
    const fakeStream = new ReadableStream();
    const aiRun = mock((model: string, _opts: unknown) => {
      if (model === DEFAULT_CF_MODEL) {
        throw new Error("3046: Request timeout");
      }
      return fakeStream;
    });

    const { CloudflareChat } = await import("../src/endpoints/inference/cloudflare/chat");
    const handler = new CloudflareChat();
    const ctx = makeContext(aiRun as unknown as (m: string, o: unknown) => unknown);
    // Override json body to set stream: true
    ctx.req.json = async () => ({
      model: DEFAULT_CF_MODEL,
      messages: [{ role: "user", content: "Hello" }],
      stream: true,
    });

    const response = await handler.handle(ctx as unknown as AppContext);
    expect((response as Response).headers.get("X-Fallback-Model")).toBe(FALLBACK_CF_MODEL);
  });

  test("non-streaming: no fallback_model when primary succeeds", async () => {
    const aiRun = mock((_model: string, _opts: unknown) => makeSuccessResponse(DEFAULT_CF_MODEL));

    const { CloudflareChat } = await import("../src/endpoints/inference/cloudflare/chat");
    const handler = new CloudflareChat();
    const ctx = makeContext(aiRun as unknown as (m: string, o: unknown) => unknown);
    const response = await handler.handle(ctx as unknown as AppContext);
    const body = await parseJson(response as Response);

    expect(body.ok).toBe(true);
    expect(body.fallback_model).toBeUndefined();
  });

  test("non-streaming: returns 504 if both primary and fallback time out", async () => {
    const aiRun = mock((_model: string, _opts: unknown) => {
      throw new Error("3046: Request timeout");
    });

    const { CloudflareChat } = await import("../src/endpoints/inference/cloudflare/chat");
    const handler = new CloudflareChat();
    const ctx = makeContext(aiRun as unknown as (m: string, o: unknown) => unknown);
    const response = await handler.handle(ctx as unknown as AppContext);

    expect((response as Response).status).toBe(504);
    const body = await parseJson(response as Response);
    expect(body.error_code).toBe("TIMEOUT");
  });

  test("non-streaming: non-timeout error on primary is not retried", async () => {
    let callCount = 0;
    const aiRun = mock((_model: string, _opts: unknown) => {
      callCount++;
      throw new Error("Rate limit exceeded");
    });

    const { CloudflareChat } = await import("../src/endpoints/inference/cloudflare/chat");
    const handler = new CloudflareChat();
    const ctx = makeContext(aiRun as unknown as (m: string, o: unknown) => unknown);
    const response = await handler.handle(ctx as unknown as AppContext);

    expect(callCount).toBe(1); // no fallback retry
    expect((response as Response).status).toBe(429);
  });
});

Notes

• The c.env.AI.run mock uses Bun's mock() — call count is tracked automatically.
• classifyCloudflareAIError treats any error message containing "3046" as a timeout, so throw new Error("3046: Request timeout") is sufficient to trigger fallback.
• The streaming test uses a ReadableStream stub; content isn't checked since the behavior under test is the header, not the stream body.
• Would need AppContext type import adjusted to match what the handler actually expects — the above is a sketch; exact type alignment depends on the Hono + Cloudflare bindings setup.

Happy to open a PR with a working version if that's useful. The main open question is whether unit tests should live alongside tests/ or in a separate unit/ dir.