Skip to content

feat(eve): identify local eval provider setup - #1851

Closed
chadhietala wants to merge 1 commit into
instr/08-bus-hardeningfrom
instr/08g-eval-setup-context
Closed

feat(eve): identify local eval provider setup#1851
chadhietala wants to merge 1 commit into
instr/08-bus-hardeningfrom
instr/08g-eval-setup-context

Conversation

@chadhietala

@chadhietala chadhietala commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Related to #1659 and the proposal in #1799. Stacked on #1817 and still gated by experimental.instrumentationProviders.

Instrumentation providers could not distinguish a server started for local eve eval from an ordinary server. Local eval now creates one run ID for its server and exposes it to provider setup as evaluation: { runId }, allowing destinations to associate or exclude synthetic telemetry.

This is setup metadata only: eve does not attach the ID to events or spans. Ordinary servers and evals targeting an existing server with --url do not receive it.

Validation

  • Full rebased eve unit suite: 6,164 passed, 1 skipped
  • Full eve integration suite: 618 passed
  • Focused provider, eval, tracing, and compiled-artifact scenarios: 14 passed
  • tsc -p packages/eve/tsconfig.json --noEmit
  • pnpm guard:invariants; docs checks pass
  • E2E not run locally; the provider layout remains behind the experimental flag

Checklist

  • I linked an issue with prior discussion confirming this change is wanted
  • I ran the relevant checks from CONTRIBUTING.md
  • I added tests and documentation where relevant
  • I added a changeset if this touches the published eve package
  • DCO sign-off passes for every commit (git commit --signoff)

@vercel

vercel Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
eve-docs Ready Ready Preview Aug 8, 2026 9:54pm
eve-docs-4759 Ready Ready Preview, v0 Aug 8, 2026 9:54pm

@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Bundle + Package Summary: apps/fixtures/weather-agent

Key takeaways

  • No notable deltas vs instr/08-bus-hardening (106b34f).

Delta vs instr/08-bus-hardening (106b34f)

Area Metric Baseline Current Delta
Package Packed tarball 7.87 MB 7.87 MB +2.9 kB ⚠️
Package Unpacked publish size 29.39 MB 29.39 MB +1.7 kB ⚠️
Package Installed footprint 72.82 MB 72.83 MB +1.7 kB ⚠️
Package Published files 3094 3094 0
Package Installed files 6804 6804 0
Runtime Unique function payloads 2 2 0
Runtime Total function bytes 17.09 MB 17.09 MB -40 B ✅
Runtime Public routes 13 13 0
Changed function payloads vs instr/08-bus-hardening (106b34f) (2)
Function Status Baseline Current Delta Route changes
functions/__server.func changed 8.55 MB 8.55 MB -20 B ✅ none
functions/.well-known/workflow/v1/flow.func changed 8.55 MB 8.55 MB -20 B ✅ none

eve init install

Metric Baseline Current Delta
Installed footprint 111.23 MB 111.24 MB +1.7 kB ⚠️
Installed packages 121 121 0
dependencies 4 4 0
devDependencies 2 2 0
Dependency package bytes 43.87 MB 43.87 MB +1.7 kB ⚠️
devDependency package bytes 5.04 MB 5.04 MB 0 B ➖
Build Metadata
  • Preset: vercel
  • Nitro: nitro@3.0.260610-beta
  • Output directory: apps/fixtures/weather-agent/.vercel/output
  • Build metadata timestamp: 2026-08-08T21:56:39.556Z
  • Route aliases: 13 public, 1 internal (14 total aliases)
  • Vercel routes in config: 16
  • Severity legend: 🔴 dominant/large, 🟠 notable, 🟡 watch, ⚪ small
Package Drill-Down

Package Details

  • Package: eve@0.31.3
  • Package directory: packages/eve
  • Tarball: 7.87 MB (eve-0.31.3.tgz)
  • Unpacked payload: 29.39 MB across 3094 published files
  • Installed footprint: 72.83 MB across 6804 installed files
  • Installed root package: 28.04 MB
  • Installed dependencies: 44.79 MB
  • Runtime dependencies: 2
  • Peer dependencies: 5 (4 optional)

Installed footprint is measured from an isolated temporary npm install of the packed tarball.

Heavy installed dependencies

  • eve: 28.04 MB (38.5%)
  • @rolldown/binding-linux-x64-gnu: 19.35 MB (26.6%)
  • ai: 6.67 MB (9.2%)
  • zod: 5.07 MB (7.0%)
  • undici: 3.50 MB (4.8%)
Publish payload breakdown
Published file size
🔴 dist/src/compiled/shadcn-registry/index.js       [################........] 9.68 MB 32.9%
🟠 dist/src/compiled/@photon-ai/chat-adapter-ime... [#####...................] 2.92 MB 10.0%
🟠 dist/src/compiled/experimental-ai-sdk-code-mo... [###.....................] 1.51 MB 5.1%
🟡 dist/src/compiled/@vercel/sandbox/index.js       [#.......................] 638.9 kB 2.2%
🟡 dist/src/compiled/_chunks/workflow/undici-CUI... [#.......................] 508.8 kB 1.7%
🔴 Other published files                            [########################] 14.13 MB 48.1%
Installed footprint breakdown
Installed package size
🔴 eve                             [########################] 28.04 MB 38.5%
🔴 @rolldown/binding-linux-x64-gnu [#################.......] 19.35 MB 26.6%
🔴 ai                              [######..................] 6.67 MB 9.2%
🔴 zod                             [####....................] 5.07 MB 7.0%
🟠 undici                          [###.....................] 3.50 MB 4.8%
🟠 nitro                           [##......................] 2.41 MB 3.3%
🔴 Other installed packages        [#######.................] 7.79 MB 10.7%
Runtime dependencies (2)
Package Range Notes
nitro 3.0.260610-beta
undici 8.9.0
Peer dependencies (5)
Package Range Notes
@opentelemetry/api ^1.0.0 optional peer
ai catalog:
braintrust ^3.0.0 optional peer
just-bash ^3.0.0 optional peer
microsandbox ^0.5.0 optional peer
eve init install drill-down

eve init install details

  • Command: eve init my-agent
  • Package manager: npm
  • Installed footprint: 111.24 MB across 8680 installed files
  • Installed packages: 121 total (115 transitive-only)
  • dependencies: 4 direct packages totaling 43.87 MB
  • devDependencies: 2 direct packages totaling 5.04 MB
  • Other transitive package files: 62.32 MB

Installed footprint is measured from an isolated temporary eve init my-agent using the current packed eve tarball.

Heavy installed dependencies

  • eve: 28.04 MB (25.2%)
  • @typescript/typescript-linux-x64: 27.95 MB (25.1%)
  • @rolldown/binding-linux-x64-gnu: 19.35 MB (17.4%)
  • zod: 9.02 MB (8.1%)
  • ai: 6.67 MB (6.0%)
Installed footprint breakdown
Installed package size
🔴 eve                              [########################] 28.04 MB 25.2%
🔴 @typescript/typescript-linux-x64 [########################] 27.95 MB 25.1%
🔴 @rolldown/binding-linux-x64-gnu  [#################.......] 19.35 MB 17.4%
🔴 zod                              [########................] 9.02 MB 8.1%
🔴 ai                               [######..................] 6.67 MB 6.0%
🟠 undici                           [###.....................] 3.50 MB 3.2%
🔴 Other installed packages         [##############..........] 16.71 MB 15.0%
dependencies (4)
Package Range Installed size Share
@vercel/connect 0.4.3 141.0 kB 0.1%
ai ^7.0.38 6.67 MB 6.0%
eve file:eve-0.31.3.tgz 28.04 MB 25.2%
zod 4.4.3 9.02 MB 8.1%
devDependencies (2)
Package Range Installed size Share
@types/node 24.x 2.54 MB 2.3%
typescript 7.0.2 2.50 MB 2.2%
Function Drill-Down

Payload Size Graph

Unique function payload size and share of total
🔴 functions/.well-known/workflow/v1/flow.func     [########################] 8.55 MB 50.0%
🔴 functions/__server.func                         [########################] 8.55 MB 50.0%

Top Function Payloads

🟠 functions/.well-known/workflow/v1/flow.func • 1 public route • 8.55 MB
Metric Value
Public routes /.well-known/workflow/v1/flow
Runtime nodejs24.x
Handler index.mjs
Payload 8.55 MB
Function files 8.55 MB across 44 files
Traced dependencies 0 B
Signal 🟠 Bundled file index.mjs is 2.34 MB (27.4%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 index.mjs                       [########################] 2.34 MB 27.4%
🟠 _chunks/runtime-artifacts.mjs   [################........] 1.63 MB 19.1%
🟡 _libs/undici.mjs                [##########..............] 980.5 kB 11.5%
🟡 _chunks/sandbox.mjs             [########................] 777.1 kB 9.1%
🟡 _libs/@ai-sdk/gateway+[...].mjs [####....................] 432.8 kB 5.1%
🟠 Other bundled files             [########################] 2.38 MB 27.9%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x",
  "maxDuration": "max",
  "experimentalTriggers": [
    {
      "type": "queue/v2beta",
      "topic": "__eve776561746865722d6167656e74_wkf_workflow_*",
      "consumer": "default",
      "retryAfterSeconds": 5,
      "initialDelaySeconds": 0
    }
  ],
  "environment": {
    "WORKFLOW_PRECONDITION_GUARD": "1"
  }
}

🟠 functions/__server.func • 12 public routes, 1 internal alias • 8.55 MB
Metric Value
Public routes /
/eve/v1/callback/[token]
/eve/v1/connections/[name]/callback/[token]
/eve/v1/health
/eve/v1/info
/eve/v1/session
/eve/v1/session/[sessionId]
/eve/v1/session/[sessionId]/cancel
/eve/v1/session/[sessionId]/clear
/eve/v1/session/[sessionId]/compact
/eve/v1/session/[sessionId]/reset
/eve/v1/session/[sessionId]/stream
Internal aliases /__server
Runtime nodejs24.x
Handler index.mjs
Payload 8.55 MB
Function files 8.55 MB across 44 files
Traced dependencies 0 B
Signal 🟠 Bundled file index.mjs is 2.34 MB (27.4%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 index.mjs                       [########################] 2.34 MB 27.4%
🟠 _chunks/runtime-artifacts.mjs   [################........] 1.63 MB 19.1%
🟡 _libs/undici.mjs                [##########..............] 980.5 kB 11.5%
🟡 _chunks/sandbox.mjs             [########................] 777.1 kB 9.1%
🟡 _libs/@ai-sdk/gateway+[...].mjs [####....................] 432.8 kB 5.1%
🟠 Other bundled files             [########################] 2.38 MB 27.9%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x"
}

Build Timing: e2e/fixtures/agent-tools-sandbox

This is an informational timing measurement inside eve build, from preflight through publication. Output-size measurement and profile writing are excluded.

Build mode: deployable Vercel build with sandbox template prewarm included.

  • Build pipeline: 2.10 s -> 2.02 s (-75.2 ms) vs instr/08-bus-hardening (106b34f).
  • Timing is informational: shared GitHub runners are too variable for a hard timing budget.
Detailed phase timings vs `instr/08-bus-hardening (106b34f)`
Phase Baseline Current Delta
extension.check 1.2 ms 1.1 ms -0.1 ms
project.resolve 0.7 ms 0.7 ms 0.0 ms
workspace.create 0.7 ms 0.8 ms +0.1 ms
host.prepare 243.1 ms 171.9 ms -71.2 ms
vercel.service-prefix.resolve 2.1 ms 2.4 ms +0.3 ms
nitro.create 253.2 ms 248.2 ms -5.0 ms
sandbox.prewarm 189.7 ms 161.8 ms -27.9 ms
nitro.cache.prepare 0.2 ms 0.3 ms +0.1 ms
nitro.prepare 0.9 ms 0.9 ms 0.0 ms
nitro.public-assets 0.8 ms 0.8 ms 0.0 ms
nitro.prerender 0.4 ms 0.6 ms +0.2 ms
nitro.bundle 1.37 s 1.40 s +29.2 ms
nitro.cache.write 0.4 ms 0.4 ms 0.0 ms
vercel.workflow-function.materialize 24.1 ms 23.0 ms -1.1 ms
agent-summary.emit 0.5 ms 0.4 ms -0.1 ms
nitro.close 0.1 ms 0.1 ms 0.0 ms
output.publish 3.6 ms 3.6 ms 0.0 ms
workspace.remove 2.3 ms 2.5 ms +0.2 ms

Signed-off-by: Chad Hietala <chadhietala@gmail.com>
@chadhietala

Copy link
Copy Markdown
Contributor Author

Superseded by replacement stack #1866. Local eval setup identity is included in the final provider setup context in #1863.

@chadhietala chadhietala closed this Aug 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant