throttleGate — Production-Grade API Gateway

API gateways concentrate the hardest edge concerns: rate limiting must be accurate under concurrency (hence atomic Lua in Redis), circuit breaking must absorb upstream failures without cascading, and service discovery must track health in real time. throttleGate demonstrates each with measurable rigor — 4,500 req/s max throughput with p50 8ms, p95 45ms, p99 120ms.

A 16-step request pipeline: size limiter → slow-request protection → IP filter (CIDR) → input sanitizer → CORS → tracing → health/metrics → body parse → registration → stacked Lua rate limiter → auth (API key/JWT) → request coalescer → circuit breaker → route match → load balancer → proxy with retry/backoff. Hot-reload rebuilds the router stack on config change.

Config-driven: a Zod schema with 63+ fields covering limits, tiers (Free/Pro/Enterprise), circuit breaker thresholds, health checks, and routing rules. Service registry entries live in Redis with active + passive health checks.

Writing correct rate-limiting Lua was the core challenge — refill math, atomicity, TTL management, and fail-open behavior under Redis outages. The request coalescer dedupes identical GET/HEAD requests so 100 concurrent clients share one upstream call. Circuit breaker state is inherently local, so it runs in-process per route.

Stateless gateway instances scale horizontally behind Kubernetes (HPA 2-10, PDB min 2). Redis holds shared state — rate limits, registry, config. Weighted round-robin + least-connections load balancing distributes traffic. Bulkheading isolates connection pools per upstream.

Fail-open mode when Redis is down (configurable), retry with exponential backoff + jitter for idempotent methods only, dead-letter circuit breaker transitions, and graceful shutdown on SIGTERM/SIGINT.

18 Prometheus metrics across 6 groups (proxy, rate limit, circuit breaker, security, service discovery, coalescer) plus OpenTelemetry spans, Pino logs, and a React dashboard (6 pages, 15 Radix components, Recharts).

Custom Lua over npm rate-limit packages for atomicity and performance. In-process circuit breakers over centralized state. Sliding-window-log over fixed-window (no boundary spikes). No retry for POST by default (idempotency only). Express over Fastify for ecosystem maturity.