Enterprise AI Platform
Multi-model LLM inference with routing, guardrails, and usage metering
Enterprise AI Platform
Four repositories: a multi-model inference API with intelligent routing and three-layer guardrails, a content ingestion and RAG pipeline processing documents and media at scale, a dashboard for AI-powered content discovery, and a metering library tracking API usage for billing. The full-stack versatility came from necessity -- cross-functionality was survival in a fast-moving engineering org.
Web + API
Platform
Full-Stack Engineer
Role
From Architecture to Production
Cross-Stack Delivery
Built the native iOS chat client first, then moved to web and backend. Picking up an unfamiliar part of the stack and contributing to it quickly turned out to be the skill that mattered most here.
Multi-Model Inference & Safety
The core API routes requests across multiple LLM providers with tier-based classification. Three layers of content safety run before every response: fuzzy text matching, vector similarity against curated examples in Weaviate, and a final LLM-based check. Each layer short-circuits on the first match.
RAG Pipeline & Content Ingestion
Distributed workers process documents and media through vector search pipelines. Background jobs run with automatic retries. This was the unglamorous part of the project, and also the most important.
Dashboard & Metering
Next.js SaaS dashboard for content discovery, user management, and embedded widgets. I also created a standalone metering library that instruments endpoints with a single decorator. Events fire asynchronously in fire-and-forget mode, so zero latency impact on API responses.
Scale Meets Safety
Intelligent Model Routing
Every inference request gets classified into complexity tiers. Simple factual queries go to fast, cheap models. Nuanced responses hit mid-range. Complex reasoning reaches the heavy hitters. If a provider goes down, requests reroute transparently. Users never notice.
Three-Layer Safety Pipeline
Layer 1: fuzzy matching against known patterns. Layer 2: vector similarity against curated examples. Layer 3: LLM-based contextual check. Most requests clear the first layer in under 50ms and never need the expensive stuff.
Zero-Latency API Metering
One decorator on an endpoint and it's metered. Events fire asynchronously, fire-and-forget. No blocking, no retries on the hot path. JWT subject extraction ties usage to billing accounts, so the billing team gets what they need without slowing anyone down.
Under the Hood
Key Technical Decisions
Provider Resiliency Pattern
Each LLM provider gets a health-checking wrapper with circuit breaker logic. Errors mark a provider as degraded, and traffic reroutes automatically. Recovery is automatic too. Token pricing is pre-computed per provider so cost tracking happens in real time without external calls.
Vector-Augmented Guardrails
Keyword-based content filtering misses semantic variations, and it's not close. We embedded thousands of curated examples and classify new inputs by vector similarity. Catches paraphrased harmful content that keyword matching would miss, false positive rate stays low.
Content Processing Pipeline
The ingestion pipeline uses distributed workers with at-least-once delivery and automatic dead-letter queuing. Each content type runs its own task chain, so a failure in one pipeline doesn't take down the others.
What I Learned
Learning 1
Things break constantly and context does not arrive pre-packaged. I went from iOS to web to backend because that is what the work required. The ability to pick up any layer of the stack and contribute to it immediately became the most valuable skill I have.
The original safety pipeline killed performance
Four layers added 400ms to every request. Nobody was going to ship that. We cut it to three layers with early short-circuiting, and most requests clear the first layer in under 10ms. Invisible safety stays on. Slow safety gets disabled.
Scattered billing code is a nightmare
We had metering logic copy-pasted across route handlers until I pulled it into a standalone decorator library. One line per endpoint. The library is reused across three services now.
Enterprise AI Chat Client
Modular iOS chat client across seven Swift packages