feat: Phase 3 Dashboard and Shared DB - #5
Open
0xadityaa wants to merge 7 commits into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
Implements Phase 3 by extracting a shared Prisma DB workspace (@time-box/db), adding hourly aggregated metrics storage, and scaffolding a Next.js dashboard that reads traces/metrics and can fetch raw payload blobs from MinIO.
Changes:
- Added
packages/dbworkspace exporting a shared Prisma client and updated Prisma schema (newHourlyMetrics, newSpan.status). - Updated collector to write span status, use shared DB package, and run an hourly aggregation job that upserts metrics.
- Added dashboard pages/components for overview + traces, including MinIO payload viewing.
Reviewed changes
Copilot reviewed 22 out of 23 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/db/tsconfig.json | TypeScript config for the new shared DB package build. |
| packages/db/src/index.ts | Exposes a shared Prisma client instance and re-exports Prisma types. |
| packages/db/prisma/schema.prisma | Adds Span.status and the new HourlyMetrics model. |
| packages/db/package.json | Defines @time-box/db package build/generate scripts and deps. |
| docs/runs/phase-3.md | Run document describing Phase 3 architecture and implementation notes. |
| bun.lock | Locks new workspace package references and new dashboard deps. |
| apps/dashboard/src/lib/s3.ts | MinIO/S3 client + helper to fetch payload blobs. |
| apps/dashboard/src/components/ui/table.tsx | UI table component (shadcn-style). |
| apps/dashboard/src/components/ui/chart.tsx | UI chart wrapper components (Recharts helpers). |
| apps/dashboard/src/components/ui/card.tsx | UI card component (shadcn-style). |
| apps/dashboard/src/components/ui/badge.tsx | UI badge component. |
| apps/dashboard/src/components/payload-viewer.tsx | Client component to expand/render JSON payloads. |
| apps/dashboard/src/components/layout/sidebar.tsx | Dashboard sidebar navigation. |
| apps/dashboard/src/app/traces/page.tsx | Traces list page reading traces/spans from Prisma. |
| apps/dashboard/src/app/traces/[id]/page.tsx | Trace detail page that loads spans + payloads. |
| apps/dashboard/src/app/page.tsx | Overview page reading hourly metrics from Prisma. |
| apps/dashboard/src/app/layout.tsx | Root layout integrating the sidebar and global dark styling. |
| apps/dashboard/package.json | Adds dashboard dependencies (@time-box/db, S3 SDK, recharts, etc.). |
| apps/collector/src/processor.ts | Uses shared Prisma client and persists Span.status. |
| apps/collector/src/processor.test.ts | Updates Prisma mocking to mock @time-box/db. |
| apps/collector/src/index.ts | Starts the hourly aggregator alongside the queue. |
| apps/collector/src/aggregator.ts | New hourly aggregation job that upserts HourlyMetrics. |
| apps/collector/package.json | Switches collector Prisma dependency to @time-box/db. |
Comments suppressed due to low confidence (2)
packages/db/prisma/schema.prisma:32
statusis stored as a free-formString?but the code relies on a small fixed set of values (OK/ERROR/UNSET). Consider modeling this as a Prismaenum(and ideally making it non-null with a default) to prevent inconsistent strings and simplify querying.
packages/db/prisma/schema.prisma:54- This schema change adds a new table (
HourlyMetrics) and a new column (Span.status). Ensure a corresponding migration/update step is included in your rollout (e.g. prisma migrate/db push in CI/deploy) so the collector/dashboard don’t crash against older schemas.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @@ -0,0 +1,10 @@ | |||
| import { PrismaClient } from '@prisma/client'; | |||
|
|
|||
| const globalForPrisma = globalThis as unknown as { prisma: PrismaClient }; | |||
| 1. **Shared `@time-box/db` Workspace**: Extracted Prisma out of the collector into a shared library. This ensures strong typing across both applications and allows Next.js to leverage server components to securely query Postgres. | ||
| 2. **Background Metrics Aggregation**: The collector now runs an hourly interval job (`aggregator.ts`) to calculate P50/P90/P99 latency percentiles and system token usage, storing them in a new `HourlyMetrics` table. This prevents heavy aggregate queries from crippling the DB. | ||
| 3. **Next.js Dashboard Scaffolding**: | ||
| - Initialized Next.js 14 app router, Shadcn UI, and Tailwind. |
Comment on lines
+6
to
+14
| s3Client = new S3Client({ | ||
| region: 'us-east-1', | ||
| endpoint: process.env.MINIO_ENDPOINT || 'http://localhost:9000', | ||
| credentials: { | ||
| accessKeyId: process.env.MINIO_ACCESS_KEY || 'minioadmin', | ||
| secretAccessKey: process.env.MINIO_SECRET_KEY || 'minioadmin', | ||
| }, | ||
| forcePathStyle: true, | ||
| }); |
Comment on lines
+18
to
+29
| export async function fetchPayloadFromS3(uri: string): Promise<string | null> { | ||
| if (!uri.startsWith('minio://')) return null; | ||
| const path = uri.replace('minio://', ''); | ||
| const [bucket, ...rest] = path.split('/'); | ||
| const key = rest.join('/'); | ||
|
|
||
| const client = getS3Client(); | ||
| try { | ||
| const response = await client.send(new GetObjectCommand({ | ||
| Bucket: bucket, | ||
| Key: key, | ||
| })); |
Comment on lines
+10
to
+14
| // Await the params object itself per Next.js 15+ constraints if this was upgraded, but in 14 it's technically sync. Wait, Next.js 16 requires awaiting params. | ||
| const resolvedParams = await params; | ||
|
|
||
| const trace = await prisma.trace.findUnique({ | ||
| where: { id: resolvedParams.id }, |
Comment on lines
+10
to
+18
| const traces = await prisma.trace.findMany({ | ||
| orderBy: { startTime: 'desc' }, | ||
| take: 50, | ||
| include: { | ||
| spans: { | ||
| select: { id: true, name: true, latencyMs: true, status: true, inputTokens: true, outputTokens: true } | ||
| } | ||
| } | ||
| }); |
Comment on lines
+4
to
+10
| const now = new Date(); | ||
| // Truncate to the start of the current hour | ||
| const currentHour = new Date(now.getFullYear(), now.getMonth(), now.getDate(), now.getHours(), 0, 0, 0); | ||
|
|
||
| // We process the previous hour | ||
| const targetHour = new Date(currentHour.getTime() - 60 * 60 * 1000); | ||
| const targetEnd = currentHour; |
Comment on lines
+114
to
+121
| // Run aggregation every hour | ||
| export function startAggregator() { | ||
| // Run once on startup for the previous hour just in case | ||
| aggregateHourlyMetrics(); | ||
|
|
||
| // 60 minutes * 60 seconds * 1000 ms | ||
| setInterval(aggregateHourlyMetrics, 60 * 60 * 1000); | ||
| } |
| import Image from "next/image"; | ||
| import { prisma } from '@time-box/db'; | ||
| import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'; | ||
| import { Badge } from '@/components/ui/badge'; |
Comment on lines
+3
to
+10
| export async function aggregateHourlyMetrics() { | ||
| const now = new Date(); | ||
| // Truncate to the start of the current hour | ||
| const currentHour = new Date(now.getFullYear(), now.getMonth(), now.getDate(), now.getHours(), 0, 0, 0); | ||
|
|
||
| // We process the previous hour | ||
| const targetHour = new Date(currentHour.getTime() - 60 * 60 * 1000); | ||
| const targetEnd = currentHour; |
| @@ -0,0 +1 @@ | |||
| DATABASE_URL="postgresql://timebox:password@localhost:5432/timebox?schema=public" | |||
Comment on lines
44
to
46
| try { | ||
| const captureContent = process.env.OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT === 'true'; | ||
| await processOtlpPayloadBatch(batch, captureContent); | ||
| await processOtlpPayloadBatch(batch); | ||
| } catch (err) { |
Comment on lines
8
to
11
| traceQueue.start(); | ||
| startAggregator(); | ||
| startSettingsPoller(); | ||
|
|
Comment on lines
+6
to
+13
| export async function updateSystemSettings(formData: FormData) { | ||
| const capturePayloads = formData.get('capturePayloads') === 'on'; | ||
|
|
||
| await prisma.systemSettings.upsert({ | ||
| where: { id: 'global' }, | ||
| update: { capturePayloads }, | ||
| create: { id: 'global', capturePayloads, retentionDays: 30 } | ||
| }); |
Comment on lines
+10
to
+14
| // Await the params object itself per Next.js 15+ constraints if this was upgraded, but in 14 it's technically sync. Wait, Next.js 16 requires awaiting params. | ||
| const resolvedParams = await params; | ||
|
|
||
| const trace = await prisma.trace.findUnique({ | ||
| where: { id: resolvedParams.id }, |
Comment on lines
+26
to
+33
| // Pre-fetch payloads for GenAI spans | ||
| const spansWithPayloads = await Promise.all(trace.spans.map(async (span) => { | ||
| let payloadContent = null; | ||
| if (span.payloadUri) { | ||
| payloadContent = await fetchPayloadFromS3(span.payloadUri); | ||
| } | ||
| return { ...span, payloadContent }; | ||
| })); |
Comment on lines
+93
to
+113
| return ( | ||
| <style | ||
| dangerouslySetInnerHTML={{ | ||
| __html: Object.entries(THEMES) | ||
| .map( | ||
| ([theme, prefix]) => ` | ||
| ${prefix} [data-chart=${id}] { | ||
| ${colorConfig | ||
| .map(([key, itemConfig]) => { | ||
| const color = | ||
| itemConfig.theme?.[theme as keyof typeof itemConfig.theme] ?? | ||
| itemConfig.color | ||
| return color ? ` --color-${key}: ${color};` : null | ||
| }) | ||
| .join("\n")} | ||
| } | ||
| ` | ||
| ) | ||
| .join("\n"), | ||
| }} | ||
| /> |
Comment on lines
+5
to
+23
| export async function fetchSystemSettings() { | ||
| try { | ||
| const settings = await prisma.systemSettings.findUnique({ | ||
| where: { id: 'global' }, | ||
| }); | ||
|
|
||
| if (settings) { | ||
| globalCapturePayloads = settings.capturePayloads; | ||
| } else { | ||
| // Create default settings if they don't exist | ||
| const defaultSettings = await prisma.systemSettings.create({ | ||
| data: { | ||
| id: 'global', | ||
| capturePayloads: false, // Privacy by default | ||
| retentionDays: 30, | ||
| }, | ||
| }); | ||
| globalCapturePayloads = defaultSettings.capturePayloads; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implementation of Phase 3, featuring Next.js dashboard, Shadcn UI, and packages/db extraction.