Enterprise AI Platform
Production AI platform routing across 6 LLM providers with built-in safety
Enterprise AI Platform
Started on this project as an iOS engineer building the native chat client, then moved to web and backend after the chat app was sunsetted. Ended up shipping across four repositories: a multi-model inference API with intelligent routing and three-layer guardrails, a content ingestion and RAG pipeline processing documents and media at scale, a SaaS dashboard for AI-powered content discovery, and a metering library tracking API usage for billing. The full-stack versatility came from necessity -- cross-functionality was survival in a fast-moving engineering org.
Web + API
Platform
Full-Stack Engineer
Role
From Architecture to Production
iOS Origins & Platform Shift
Joined as the iOS engineer building the native chat client. After that app was sunsetted, I moved to web and backend -- a transition that happened organically in a fast-moving org where cross-functionality wasn't optional. That versatility shaped everything that came after.
Multi-Model Inference & Safety
The core API routes requests across multiple LLM providers with tier-based classification. Three layers of content safety run before every response: fuzzy text matching, vector similarity against curated examples in Weaviate, and a final LLM-based check. Each layer short-circuits on the first match.
RAG Pipeline & Content Ingestion
Distributed workers process documents and media through vector search pipelines. Background jobs run with automatic retries. This was the unglamorous part of the project, and also the most important.
Dashboard & Metering
Next.js SaaS dashboard for content discovery, user management, and embedded widgets. I also created a standalone metering library that instruments endpoints with a single decorator. Events fire asynchronously in fire-and-forget mode, so zero latency impact on API responses.
Scale Meets Safety
Intelligent Model Routing
Every inference request gets classified into complexity tiers. Simple factual queries go to fast, cheap models. Nuanced responses hit mid-range. Complex reasoning reaches the heavy hitters. If a provider goes down, requests reroute transparently. Users never notice.
Three-Layer Safety Pipeline
Layer 1: fuzzy matching against known patterns. Layer 2: vector similarity against curated examples. Layer 3: LLM-based contextual check. Most requests clear the first layer in under 50ms and never need the expensive stuff.
Zero-Latency API Metering
One decorator on an endpoint and it's metered. Events fire asynchronously, fire-and-forget. No blocking, no retries on the hot path. JWT subject extraction ties usage to billing accounts, so the billing team gets what they need without slowing anyone down.
Under the Hood
Key Technical Decisions
Provider Resiliency Pattern
Each LLM provider gets a health-checking wrapper with circuit breaker logic. Errors mark a provider as degraded, and traffic reroutes automatically. Recovery is automatic too. Token pricing is pre-computed per provider so cost tracking happens in real time without external calls.
Vector-Augmented Guardrails
Keyword-based content filtering misses semantic variations, and it's not close. We embedded thousands of curated examples and classify new inputs by vector similarity. Catches paraphrased harmful content that keyword matching would miss, false positive rate stays low.
Content Processing Pipeline
The ingestion pipeline uses distributed workers with at-least-once delivery and automatic dead-letter queuing. Each content type runs its own task chain, so a failure in one pipeline doesn't take down the others.
What I Learned
Working across a large engineering team taught me more than any solo project
Things break constantly, comms move fast, and you can't wait for someone to explain the context. I went from iOS to web to backend because that's what the team needed. The ability to pick up any part of the stack and contribute immediately became my most valuable skill.
The original safety pipeline killed performance
Four layers added 400ms to every request. Nobody was going to ship that. We cut it to three layers with early short-circuiting, and most requests clear the first layer in under 10ms. Invisible safety stays on. Slow safety gets disabled.
Scattered billing code is a nightmare
We had metering logic copy-pasted across route handlers until I pulled it into a standalone decorator library. One line per endpoint. The library is reused across three services now.
Enterprise AI Chat Client
iOS chat client for an enterprise AI platform -- shipped to the App Store