1.2.0

Infrastructure & Workload Isolation Update

This release introduces an enterprise-grade Kubernetes architecture redesign, focusing heavily on workload isolation and enhancing the local developer experience (DX).

Key Features & Architecture changes

• Multi-Node Kubernetes Provisioning: Transitioned from a single-node setup to a 3-node architecture (1 Control Plane, 2 Workers) via local Kind configurations.
• Workload Isolation: Implemented strict Taints, Tolerations, and Node Affinity rules. Heavy AI inference workloads (Python Consumer) are now completely isolated from stateful storage components (PostgreSQL, Redis, Redpanda) to eliminate "Noisy Neighbor" CPU contention.
• Optimized Packing: Lightweight stateless services (like the Go API Gateway) are automatically packed into available idle resources, maximizing hardware utilization.

Developer Experience (DX) & Automation

• Modular Taskfile Pipelines: Refactored the local orchestration commands.
• New Teardown Workflows: Introduced task docker:off and task k8s:off for sequential teardown processes.
• Added task k8s:clean to completely destroy local cluster data and free up resources with a single click.

Documentation

• Comprehensive README Overhaul: Completely rewrote the Kubernetes deployment section to reflect the new automated multi-node setup.
• Refined Sanity Checks: Simplified the End-to-End verification steps using universal curl commands and explicit PowerShell alternatives.
• Troubleshooting Guide: Updated dependency installation guides for task, oha, and kind across all major operating systems.

1.1.0

Goodbye 700MB Web Bloat, Hello Blazing-Fast Go TUI!

This release introduces a complete overhaul of the Sentinel Command Center. We have officially deprecated the legacy Python/Streamlit dashboard and rewritten our entire monitoring UI from scratch in pure Go, leveraging the powerful Charm CLI (bubbletea, lipgloss) ecosystem.

By going back to the terminal, we eliminated Python's GIL limitations for metric polling, decoupled our frontend from production cluster runtimes, and reduced our deployment footprint by approximately ~700MB.

Key Enhancements

• Real-Time Metrics: Live tracking of RPS and Average Latency with lightweight Unicode sparklines for immediate traffic trend visualization.
• Infra Heartbeat: Native, zero-overhead health checks for the Go API Gateway, Redis Cache, and Redpanda (Kafka) Broker.
• K8s Pod Observability: Light-speed, direct kubectl integration to monitor pod availability status instantly from the shell without web wrappers.

Architectural Impact

• Container Optimization: Shed ~700MB of bloated web dependencies.
• Resource Isolation: Isolated the monitoring layer completely from production K8s microservice state.
• Concurrency: Fully bypassed python GIL to ensure synchronous, non-blocking telemetry collection.

1.0.0

The Go Gateway & Microservices Evolution

This major release marks a complete architectural evolution of the Sentinel API Gateway. The primary ingestion layer has been rewritten from Python (FastAPI) to Go (Gin), transforming the system into a highly concurrent, ultra-lightweight microservice.

Performance Benchmarks (OHA Load Test)

The migration to Go's Goroutine architecture yielded massive performance gains under extreme concurrency (10,000 requests, 200 concurrent connections):

• Throughput: Increased by 6.3x (from 1,126 RPS to 7,167 RPS).
• Average Latency: Reduced by 84% (from 177ms to 27.6ms).
• Minimum Latency: Achieved sub-millisecond response times (0.5ms).

Docker & Infrastructure

• Multi-Stage Build: Implemented a 2-stage Dockerfile utilizing an ultra-lightweight Alpine runtime.
• Image Size Reduction: The API gateway image size was slashed by ~98%, dropping from 2.45GB to an astonishing 49MB. This ensures lightning-fast deployments and minimal container startup times.
• Memory Footprint: Container memory limits were reduced from 1GB to 128MB, maintaining extreme efficiency.

Core Engineering Upgrades

• Graceful Shutdown: Replaced Python's lifespan contexts with native Go os.Signal handling. This ensures zero dropped requests during server termination and guarantees clean closure of Redis and Redpanda client connections.
• Native Deterministic Hashing: Replaced the external orjson.OPT_SORT_KEYS dependency with Go's native json.Marshal, which inherently sorts map keys. This guarantees our O(1) Redis idempotency checks remain flawless and blazingly fast without third-party overhead.
• Advanced Profiling: Integrated gin-contrib/pprof for real-time, millisecond-level CPU, memory, and Goroutine profiling, accessible directly via the /debug/pprof/ endpoint.

0.9.0

The Performance and Async I/O Update

This release focuses on eliminating I/O bottlenecks and maximizing the throughput of the inference pipeline. By migrating to a fully asynchronous event loop and implementing O(1) database lookups, the system is now capable of sustaining massive event streams without blocking.

Performance & Profiling

• Integrated PyInstrument middleware into the FastAPI gateway for precise bottleneck analysis.
• Confirmed CPU-bound execution time is highly optimized (approx. 4-5ms per request).
• Validated that the previous 48ms latency observed in soak tests was primarily Docker networking overhead, proving the core logic is production-ready.

Asynchronous Inference Pipeline

• Refactored the consumer pipeline from synchronous to asynchronous I/O.
• Replaced confluent_kafka with aiokafka and psycopg2 with psycopg3 (AsyncConnection).
• Implemented non-blocking bulk inserts (batches of 500 records). This allows continuous data ingestion from Redpanda while PostgreSQL handles transactions in the background, significantly increasing throughput.

Database Optimizations

• Implemented an O(1) Hash Index on the transaction_id column in PostgreSQL, replacing the default B-Tree. This guarantees constant-time exact-match lookups regardless of database scale.
• Pinned the pgvector Docker image version to ensure reproducible infrastructure deployments.

Developer Experience (DX)

• Optimized the docker/backend/requirements.txt generation.
• Removed SHA256 hashes using the uv export --no-hashes flag. This reduces file size from 2000+ lines to a concise list, improving readability for code reviews while maintaining strict version pinning.

0.8.0

End-to-End Data Pipeline & Persistence

This major release establishes the permanent storage layer for the architecture, completing the end-to-end data ingestion, inference, and persistence pipeline. By integrating PostgreSQL with the pgvector extension, the system is now fully equipped to handle both relational transaction data and machine learning embeddings.

Key Architectural Milestones

• Fail-Fast Docker Validation: Implemented and validated the database schema, user roles, and the pgvector extension using a Docker-first approach via an automated init.sql script before scaling to the orchestration layer.
• Kubernetes Native State Management: Transitioned the PostgreSQL deployment into the cluster utilizing production-ready paradigms:
  • StatefulSet and PersistentVolumeClaim (PVC) for robust data persistence.
  • ConfigMap for injecting initialization scripts.
  • Secret for secure database credentials injection via envFrom.
• High-Throughput Persistence (Consumer): Upgraded the AI inference consumer unit with psycopg2-binary. It now utilizes execute_values for highly optimized bulk insertions, preventing database locking under heavy load, and includes ON CONFLICT clauses for strict idempotency.
• Resilient Startup Sequences: Expanded the initContainers architecture across API and Consumer pods to strictly await postgres:5432, eliminating CrashLoopBackOff errors and ensuring zero-restart deployments.
• End-to-End System Validation: Successfully completed a full pipeline integration test. A raw HTTP transaction successfully passed FastAPI Pydantic validation, was deduped by Redis, buffered by Redpanda, processed by the ONNX ML model, and durably committed to the PostgreSQL database with its calculated risk score.

Under the Hood

• Synchronized local and containerized environments by resolving psycopg2 dependency mismatches in the Docker build process.
• Delegated data state entirely to the infrastructure layer, maintaining the stateless and scalable nature of the API gateways.

0.7.0

Elasticity & Autoscaling

This release introduces dynamic elasticity to the Sentinel architecture.

Features

• Horizontal Pod Autoscaling (HPA): Implemented K8s native autoscaling for API and UI layers based on CPU/Memory utilization.
• Worker Optimization: Delegated process management entirely to Kubernetes by reducing Uvicorn internal workers to 1, aligning with cloud-native scale-out best practices.
• Load Tested: Architecture successfully benchmarked to handle massive concurrency (1500+ req/sec) with zero dropped connections and seamless pod replication.

0.6.0

Kubernetes Orchestration & Autonomous Startup

This major release marks the transition of the Sentinel architecture from a single-node Docker Compose environment to a fully orchestrated Kubernetes (K8s) ecosystem. The focus of v0.6.0 is infrastructure resilience, zero-restart deployments, and a streamlined developer experience (DX).

Major Architectural Milestones

1. Robust Kubernetes Migration

The entire Sentinel stack (Redis, Redpanda Broker, Redpanda Console, FastAPI Gateway, and Streamlit UI) has been successfully mapped to native Kubernetes Deployments and Services. The manifests are strictly modularized into infrastructure and apps layers for isolated management and scaling.

2. Cascading Dependency Architecture (Zero-Restart)

To eliminate the common CrashLoopBackOff errors prevalent in distributed microservice startups, we implemented a strict "Cascading Dependency" pattern using K8s initContainers.

• Infrastructure First: The API layer waits securely until both the Redis TCP port and Redpanda Broker are fully responsive.
• UI Orchestration: The Streamlit UI remains in a dormant initialization state until the FastAPI gateway successfully exposes its endpoint.
• Result: The entire cluster boots sequentially with a 100% success rate and zero restart noise.

3. Advanced Taskfile Automation

Local Kubernetes management (via Kind) has been heavily abstracted using advanced Taskfile.yml configurations, bypassing the need for complex kubectl management.

• Utilized Taskfile dependencies (deps) for parallel background execution.
• Introduced a unified task k8s:forward command that autonomously binds all required K8s service ports (API, UI, Console) to the local machine concurrently in a single terminal instance.

Technical Details & Fixes

• Manifest Organization: Split manifests atomically (e.g., separating redpanda-broker and redpanda-console) to ensure safer, isolated rollouts.
• K8s Env Override: Fixed Uvicorn startup crashes by explicitly overriding K8s-injected service environment variables (resolving an integer parsing conflict with REDIS_PORT).
• CI/CD Optimization: Updated .pre-commit-config.yaml to parse multi-document YAMLs standard in Kubernetes manifests.
• Documentation: Restructured README.md to cleanly separate Docker Compose and Kubernetes deployment paths while adding OS-specific virtual environment instructions.

0.5.0

The MLOps & Orchestration Update

Sentinel has officially evolved from a standalone API into a fully automated, self-orchestrating MLOps platform powered by Prefect 3.x.

Key Highlights

• Prefect Orchestration: Integrated a local Prefect Server and a centralized master_deploy.py to manage all deployments natively within Docker using the .serve() method.
• Automated Pipelines:
  • Health Check (daily-health-monitor): Automated daily API uptime and version monitoring.
  • Data Ingestion (hourly-data-ingestion): Scheduled ETL pipeline for extracting, transforming, and loading data into Redpanda.
  • Model Training (weekly-model-training): Weekly XGBoost model retraining with integrated ONNX export.
• Infrastructure Upgrades: Added a dedicated orchestrator container with ML dependencies, enforced strict CPU/Memory limits across all services for a lightweight DX, and implemented self-healing worker policies.
• Bug Fixes: Resolved Docker bridge network routing issues (127.0.0.1 -> api:8000) and polished Streamlit UI JSON parsing.

0.4.0

Infrastructure & Deployment Status

The project architecture has been streamlined and is currently fully containerized and orchestrated using Docker Compose. This transition ensures a lightweight, highly reproducible, and efficient deployment environment, prioritizing localized stability without the immediate overhead of Kubernetes cluster management.

Fully Operational Services:

• FastAPI Gateway (api) - Handles incoming traffic, validates payloads, and routes events for processing.
• AI Inference Engine (consumer) - Micro-batching Kafka consumer executing real-time ONNX model predictions.
• Streamlit Dashboard (ui) - Interactive user interface for real-time monitoring and analytics.
• Redis Cache (redis) - High-speed, atomic in-memory data store ensuring strict transaction idempotency.
• Redpanda (Kafka) (redpanda, console) - High-throughput event streaming broker with an integrated management console.

Next Steps & Roadmap:

• Conduct intensive load testing (e.g., via Locust) to benchmark the system against high-throughput targets.
• Finalize end-to-end integration and stress testing for the event-driven microservices pipeline.
• Re-evaluate production-grade orchestrators (e.g., Kubernetes/Helm) strictly when horizontal scaling limits of the current architecture are reached.

0.3.0

Overview

This release transforms the event-driven architecture into a fully interactive web service. We introduced a modern API Gateway to handle external traffic, implemented strict data validation, and resolved a critical MLOps challenge (Training-Serving Skew) by decoupling the feature scaling logic from the model training pipeline.

Key Architectural Implementations

• FastAPI Ingress: Built an asynchronous API gateway acting as the official entry point for external systems (POS, Web Apps) to submit transactions.
• Pydantic Validation: Enforced strict data contracts at the API level, ensuring the inference engine is protected from malformed payloads.
• Skew Resolution (Decoupled Preprocessing): Extracted RobustScaler artifacts (.joblib) from the training phase and integrated them directly into the Kafka Consumer. This allows the system to accept raw Time and Amount values via the API and transform them on the fly, matching the exact state the ONNX model expects.
• End-to-End Real-Time Flow: Successfully tested the complete pipeline: HTTP POST -> Pydantic Validation -> Redpanda Queue -> Real-Time Scaling -> Sub-millisecond ONNX Inference -> Log Output.