Real-Time Analytics Dashboard
A low-latency dashboard visualizing millions of events per minute with interactive drill-down capabilities.
- React
- TypeScript
- D3.js
- WebSockets
- ClickHouse
- Go
Overview
Designed and built a real-time analytics dashboard that ingests, aggregates, and visualizes high-volume event streams. The interface supports interactive filtering, time-range selection, and drill-down into individual event traces — all updating live without page refreshes.
The Challenge
The existing analytics solution relied on batch processing with 15-minute reporting delays. Product and engineering teams needed sub-second visibility into system health, user behavior, and business metrics to respond to incidents and opportunities in real time.
Approach
Replaced the batch ETL pipeline with a streaming architecture. Events flow through a Go ingestion service into ClickHouse for columnar storage optimized for aggregation queries. The frontend connects via WebSockets to receive push updates, with D3.js rendering responsive, interactive charts that handle 60fps updates on commodity hardware.
Results
Reduced reporting latency from 15 minutes to under 2 seconds. The dashboard handles 3M+ events per minute at peak load. Incident detection time dropped by 85%, and the product team uses real-time funnel analysis to run experiments that previously required overnight batch jobs.
Key Features
- Live-updating charts with 60fps rendering performance
- Interactive time-range selection and drill-down filtering
- Streaming ingestion pipeline handling 3M+ events per minute
- Columnar storage optimized for sub-second aggregation queries
- Alerting system with configurable thresholds and Slack integration
Lessons Learned
Rendering performance was the hardest problem. Naive approaches to updating D3 charts with high-frequency data caused jank and memory leaks. The solution was a virtualized rendering layer that batches updates into animation frames and only redraws visible data points, treating the chart more like a game engine than a traditional DOM application.