Horizon
Distributed Logging Infrastructure
Horizon is a high-performance, distributed logging infrastructure designed to solve the overhead of traditional ELK stacks. By leveraging ClickHouse’s columnar storage, I created a system that provides sub-second query speeds on millions of logs while maintaining a lightweight footprint that can be self-hosted or scaled as a SaaS.
Technical Deep Dive
I wrote a comprehensive breakdown of the architecture, engineering challenges, and performance optimizations for Horizon.
🎯The Problem
Traditional logging stacks (ELK) are resource-heavy, complex to maintain, and often too expensive for smaller teams or high-volume streams.
💡The Solution
Architected a non-blocking ingestion engine using Fastify and ClickHouse with a batched write strategy to handle massive log volumes with minimal infrastructure overhead.
Tech Stack
Key Features
My Responsibilities
Architected the entire distributed logging system
Implemented high-volume ingestion logic with ClickHouse
Developed multi-tenant SaaS infrastructure using Clerk and Prisma
Built an NPM-distributed SDK for seamless integration
Designed and developed the Next.js analytics dashboard
DevOps and infrastructure management using Coolify and Docker
Outcome & Impact
Built a production-ready logging engine capable of handling millions of logs with sub-second query latency.
Sub-second query speeds on millions of logs
Non-blocking ingestion API
Low-latency dashboard with live-tailing
Usage-based quota enforcement