[2/7] Telemetry Infrastructure: CircuitBreaker and FeatureFlagCache by samikshya-db · Pull Request #325 · databricks/databricks-sql-nodejs

samikshya-db · 2026-01-28T13:10:26Z

Part 2 of 7-part Telemetry Implementation Stack

This layer adds infrastructure components for resilience and feature flag management.

Summary

Implements CircuitBreaker for per-host endpoint protection and FeatureFlagCache for efficient feature flag management with reference counting.

Components

CircuitBreaker (`lib/telemetry/CircuitBreaker.ts`)

State machine for endpoint protection:

CLOSED (normal operation) → OPEN (failing) after threshold failures
OPEN → HALF_OPEN (testing) after timeout period
HALF_OPEN → CLOSED (recovered) after consecutive successes
HALF_OPEN → OPEN (still failing) on any failure

Configuration:

Failure threshold: 5 (default)
Timeout: 60s (default)
Success threshold: 2 (default)

CircuitBreakerRegistry

Manages circuit breakers per host for isolation:

Each host gets independent circuit breaker
Prevents cascade failures across hosts
Thread-safe per-host state management

FeatureFlagCache (`lib/telemetry/FeatureFlagCache.ts`)

Per-host feature flag caching with lifecycle management:

15-minute TTL with automatic refresh
Reference counting tracks connection lifecycle
Context removed when refCount reaches zero
Reduces API calls for feature flag checks

Testing

32 unit tests for CircuitBreaker (100% function coverage)
29 unit tests for FeatureFlagCache (100% function coverage)
CircuitBreakerStub for testing dependent components
All state transitions and edge cases covered

Next Steps

This PR is followed by:

[3/7] Client Management: TelemetryClient and Provider
[4/7] Event & Aggregation: EventEmitter and MetricsAggregator
[5/7] Export: DatabricksTelemetryExporter
[6/7] Integration: Wire into DBSQLClient
[7/7] Testing & Documentation

Dependencies

Depends on: [1/7] Telemetry Foundation (#324)

This is part 2 of 7 in the telemetry implementation stack. Components: - CircuitBreaker: Per-host endpoint protection with state management - FeatureFlagCache: Per-host feature flag caching with reference counting - CircuitBreakerRegistry: Manages circuit breakers per host Circuit Breaker: - States: CLOSED (normal), OPEN (failing), HALF_OPEN (testing recovery) - Default: 5 failures trigger OPEN, 60s timeout, 2 successes to CLOSE - Per-host isolation prevents cascade failures - All state transitions logged at debug level Feature Flag Cache: - Per-host caching with 15-minute TTL - Reference counting for connection lifecycle management - Automatic cache expiration and refetch - Context removed when refCount reaches zero Testing: - 32 comprehensive unit tests for CircuitBreaker - 29 comprehensive unit tests for FeatureFlagCache - 100% function coverage, >80% line/branch coverage - CircuitBreakerStub for testing other components Dependencies: - Builds on [1/7] Types and Exception Classifier

Implements getAuthHeaders() method for authenticated REST API requests: - Added getAuthHeaders() to IClientContext interface - Implemented in DBSQLClient using authProvider.authenticate() - Updated FeatureFlagCache to fetch from connector-service API with auth - Added driver version support for version-specific feature flags - Replaced placeholder implementation with actual REST API calls Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

- Change feature flag endpoint to use NODEJS client type - Fix telemetry endpoints to /telemetry-ext and /telemetry-unauth - Update payload to match proto with system_configuration - Add shared buildUrl utility for protocol handling

- Change payload structure to match JDBC: uploadTime, items, protoLogs - protoLogs contains JSON-stringified TelemetryFrontendLog objects - Remove workspace_id (JDBC doesn't populate it) - Remove debug logs added during testing

- Fix import order in FeatureFlagCache - Replace require() with import for driverVersion - Fix variable shadowing - Disable prefer-default-export for urlUtils

Fix TypeScript compilation error by implementing getAuthHeaders method required by IClientContext interface.

Added osArch, runtimeVendor, localeName, charSetEncoding, and processName fields to DriverConfiguration to match JDBC implementation.

…n interface

jadewang-db · 2026-02-02T23:08:06Z

lib/telemetry/CircuitBreaker.ts

@@ -0,0 +1,235 @@
+/**


try use some existing library for circuitbreaker

jadewang-db · 2026-02-03T04:10:58Z

lib/telemetry/DatabricksTelemetryExporter.ts

+    const authHeaders = authenticatedExport ? await this.context.getAuthHeaders() : {};
+
+    // Make HTTP POST request with authentication
+    const response: Response = await this.fetchFn(endpoint, {


does this support proxy?

jadewang-db · 2026-02-03T04:15:34Z

lib/telemetry/TelemetryEventEmitter.ts

+      this.emit(TelemetryEventType.CLOUDFETCH_CHUNK, event);
+    } catch (error: any) {
+      // Swallow all exceptions - log at debug level only
+      logger.log(LogLevel.debug, `Error emitting cloudfetch chunk: ${error.message}`);


should we make this log error or debug

sreekanth-db · 2026-02-03T13:36:36Z

lib/telemetry/DatabricksTelemetryExporter.ts

+interface DatabricksTelemetryPayload {
+  uploadTime: number;
+  items: string[]; // Always empty - required field
+  protoLogs: string[]; // JSON-stringified TelemetryFrontendLog objects


TelemetryFrontendLog - do you mean DatabricksTelemetryLog ?

sreekanth-db · 2026-02-03T13:42:58Z

lib/telemetry/DatabricksTelemetryExporter.ts

+  private getDriverVersion(): string {
+    try {
+      // In production, this would read from package.json
+      return '1.0.0';


is this logic updated in the next PRs ?

sreekanth-db · 2026-02-03T14:55:04Z

lib/telemetry/MetricsAggregator.ts

+      logger.log(LogLevel.debug, `Flushing ${metricsToExport.length} telemetry metrics`);
+
+      // Export metrics (exporter.export never throws)
+      this.exporter.export(metricsToExport);


should we reset the flush timer here by calling startFlushTimer ?

Example scenario:
Flush timer duration: 30 sec

flush timer starts at 0

lets say pendingMetrics reaches its limit at 25th sec - export the data to backend

timer reaches its limit at 30th sec - again tries to export the data within 5 seconds (we might hit rate limits on telemetry with large loads)

sreekanth-db

Added few comments, please take a look

samikshya-db requested review from jadewang-db and sreekanth-db January 28, 2026 23:22

Base automatically changed from telemetry-1-types to main January 29, 2026 20:19

samikshya-db and others added 5 commits January 29, 2026 20:20

Fix feature flag and telemetry export endpoints

c3779af

- Change feature flag endpoint to use NODEJS client type - Fix telemetry endpoints to /telemetry-ext and /telemetry-unauth - Update payload to match proto with system_configuration - Add shared buildUrl utility for protocol handling

Match JDBC telemetry payload format

a105777

- Change payload structure to match JDBC: uploadTime, items, protoLogs - protoLogs contains JSON-stringified TelemetryFrontendLog objects - Remove workspace_id (JDBC doesn't populate it) - Remove debug logs added during testing

Fix lint errors

1cdb716

- Fix import order in FeatureFlagCache - Replace require() with import for driverVersion - Fix variable shadowing - Disable prefer-default-export for urlUtils

samikshya-db force-pushed the telemetry-2-infrastructure branch from 06b1f7a to 1cdb716 Compare January 29, 2026 20:20

samikshya-db temporarily deployed to azure-prod January 29, 2026 20:20 — with GitHub Actions Inactive

samikshya-db temporarily deployed to azure-prod January 29, 2026 20:26 — with GitHub Actions Inactive

Add missing getAuthHeaders method to ClientContextStub

689e561

Fix TypeScript compilation error by implementing getAuthHeaders method required by IClientContext interface.

samikshya-db force-pushed the telemetry-2-infrastructure branch from 7ec20b2 to 689e561 Compare January 29, 2026 20:27

samikshya-db temporarily deployed to azure-prod January 29, 2026 20:27 — with GitHub Actions Inactive

Fix prettier formatting

2d41e2d

samikshya-db temporarily deployed to azure-prod January 29, 2026 20:30 — with GitHub Actions Inactive

Add DRIVER_NAME constant for nodejs-sql-driver

e474256

samikshya-db temporarily deployed to azure-prod January 30, 2026 05:52 — with GitHub Actions Inactive

Add missing telemetry fields to match JDBC

a3c9042

Added osArch, runtimeVendor, localeName, charSetEncoding, and processName fields to DriverConfiguration to match JDBC implementation.

samikshya-db had a problem deploying to azure-prod January 30, 2026 05:59 — with GitHub Actions Failure

Fix TypeScript compilation: add missing fields to system_configuratio…

6110797

…n interface

samikshya-db temporarily deployed to azure-prod January 30, 2026 06:14 — with GitHub Actions Inactive

jadewang-db reviewed Feb 3, 2026

View reviewed changes

sreekanth-db reviewed Feb 3, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[2/7] Telemetry Infrastructure: CircuitBreaker and FeatureFlagCache#325

[2/7] Telemetry Infrastructure: CircuitBreaker and FeatureFlagCache#325
samikshya-db wants to merge 10 commits intomainfrom
telemetry-2-infrastructure

samikshya-db commented Jan 28, 2026

Uh oh!

jadewang-db Feb 2, 2026

Uh oh!

jadewang-db Feb 3, 2026

Uh oh!

jadewang-db Feb 3, 2026

Uh oh!

sreekanth-db Feb 3, 2026

Uh oh!

sreekanth-db Feb 3, 2026

Uh oh!

sreekanth-db Feb 3, 2026 •

edited

Loading

Uh oh!

sreekanth-db left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

samikshya-db commented Jan 28, 2026

Part 2 of 7-part Telemetry Implementation Stack

Summary

Components

CircuitBreaker (lib/telemetry/CircuitBreaker.ts)

CircuitBreakerRegistry

FeatureFlagCache (lib/telemetry/FeatureFlagCache.ts)

Testing

Next Steps

Dependencies

Uh oh!

jadewang-db Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

jadewang-db Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

jadewang-db Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

sreekanth-db Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

sreekanth-db Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

sreekanth-db Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sreekanth-db left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

CircuitBreaker (`lib/telemetry/CircuitBreaker.ts`)

FeatureFlagCache (`lib/telemetry/FeatureFlagCache.ts`)

sreekanth-db Feb 3, 2026 •

edited

Loading