Building Yaldi: Why AWS Lambda + API Gateway Was the Right Call

The easy part of building Yaldi was storing metadata. The hard part is everything around it: authentication, abuse resistance, performance, cost control, and scaling without turning a personal project into a second job.

Early on, I made a deliberate decision: the Yaldi backend should be stateless, elastic, and cheap at idle. That led to a fairly straightforward conclusion: AWS Lambda for compute, fronted by Amazon API Gateway for HTTP routing, throttling, auth boundaries, and operational controls.

What Yaldi Needed From a Backend

Before picking services, I defined what “good” looked like:

  • Spiky traffic tolerance (sudden stream event, chat accelerates, extension clients connect)
  • Low idle cost (most of the day, traffic is modest)
  • Clear boundaries between public requests and authenticated operations
  • Rate limiting and abuse resistance from day one
  • Simple deploys with minimal ops overhead
  • Strong observability (logs, metrics, request tracing)

And I wanted to avoid the common trap: “just run a server,” then gradually accumulate load balancers, process managers, queues, and fire drills.

Why Lambda Was a Strong Fit

1) Scale without asking permission

Lambda scales horizontally by default. For Yaldi, that matters because demand is not smooth. A larger channel going live can create a step-function increase in requests. Lambda absorbs those spikes without me resizing instances or babysitting autoscaling rules.

2) Pay for usage, not uptime

Servers charge you for being alive. Lambda mostly charges you for doing work. For a project that might have bursty usage (and a long tail of quiet time), that pricing model matches reality.

3) Stateless architecture becomes the default

Lambda encourages a clean separation: the function handles a request, does the work, returns a response. State belongs in the database or the client. That lines up with how I wanted Yaldi to function anyway.

4) Security posture improves when there is less surface area

No SSH. No open ports. No long-lived processes. No “what’s running on that box?” Lambda reduces entire categories of operational security issues.

Why API Gateway (Not “Just Expose Lambda”)

Lambda alone is compute. The real question is how you expose it to the internet and keep it controlled. API Gateway is where a lot of the “boring but necessary” platform features live.

1) Request routing and API structure

API Gateway provides a clean way to define endpoints and route requests to the right Lambda handler(s). You can evolve an API surface deliberately, add resources, version routes, and keep your API shape consistent.

2) Throttling and rate limiting as a first-class control

If you’re hosting user-generated content and client-side integrations (like a browser extension), assume some portion of traffic will be abusive or accidental.

API Gateway gives you practical controls like throttling limits and usage plans. Even if you don’t use every feature on day one, the point is this: the guardrails exist.

3) Authentication boundary (where it belongs)

You can enforce authentication at the edge of the API, not scattered across every handler. API Gateway supports multiple auth strategies (depending on your design), and even when auth logic is implemented in Lambda, API Gateway is still the choke point where you can enforce consistent behavior.

4) Operational tooling: stages, deployments, and controlled rollouts

API Gateway stages make it easier to have a dev/staging/prod mentality without reinventing deployment patterns. If you want to change an endpoint, you can treat it like an API change, not a “random server update.”

5) Logging and tracing alignment

API Gateway + CloudWatch gives you structured request logs and metrics. When something breaks, you want to know:

  • Was the request received?
  • Was it rejected (auth/throttle)?
  • Did Lambda execute?
  • Did it time out?
  • What status code did we return?

Having a consistent front door makes debugging dramatically easier.

The Alternatives (And When They’re Better)

Lambda + API Gateway is not “the best.” It’s “the best for this set of constraints.” Here are the main alternatives I considered, and why they didn’t win for Yaldi.

1) A traditional server (EC2 / VPS)

Why people like it: familiar, flexible, no platform constraints, can be cheapest at very high sustained traffic.

Why I didn’t: you inherit operations immediately—patching, uptime, monitoring, scaling, process management, and the constant temptation to let the box become a junk drawer. For Yaldi, the goal was to keep backend ops close to zero.

2) Containers (ECS/Fargate, Kubernetes)

Why people like it: predictable runtime, easier local parity, long-running workloads, good for complex services.

Why I didn’t: it’s operationally heavier. Even with managed offerings, you’re managing services, deployments, autoscaling, networking, and costs that exist even when usage is low. Yaldi’s backend is mostly request/response and does not need a long-running process.

3) Cloudflare Workers

Why people like it: fast edge execution, excellent for lightweight request handling, tight CDN integration.

Why I didn’t: Workers are great for certain shapes of compute, but Yaldi’s API needs deep integration with AWS services and a straightforward Python runtime and ecosystem. Also, when the system grows, you want fewer “split brain” operational domains unless the edge requirements demand it.

4) Lambda Function URLs (instead of API Gateway)

Why people like it: simpler, cheaper, fewer moving parts.

Why I didn’t (for Yaldi): API Gateway provides stronger “front door” controls and a more structured API surface for growth: throttling models, stage deployments, request/response transformation patterns, and a mature operational story. Function URLs can be valid, but API Gateway better matches the “platform” direction.

Where Lambda + API Gateway Can Hurt

This combination isn’t free of trade-offs. The main downsides:

1) Cold starts and latency variance

Lambda can introduce cold starts depending on runtime and traffic patterns. For Yaldi, this is mitigated by keeping handlers lean, using efficient database connection strategies, and designing the system so most user-facing “fast paths” are CDN-served rather than API-served.

2) Platform constraints

Timeout limits, payload limits, and execution model constraints are real. If you need long-running jobs, heavy CPU work, or streaming responses, Lambda may not be the right home.

3) Complexity can creep in if you treat it like a monolith

If you jam too many responsibilities into a single Lambda and call it a day, you will eventually recreate the same complexity you were trying to avoid. The fix is architectural discipline: keep endpoints narrow, separate concerns, and push media delivery to the CDN.

4) At very high sustained traffic, servers can become cheaper

Serverless shines for bursty workloads and low idle costs. If you have constant heavy load, dedicated compute can be more economical. Yaldi’s workload is not constant heavy load, and the CDN carries the heaviest bytes anyway.