# ShipReal: From Vibe Coder to Real Engineer (full course guide)

> Your AI wrote the app. Nobody wrote the part that keeps it up. This 20 hour course closes that gap.

Last verified: 2026-08-30. Formats: Free on YouTube (https://www.youtube.com/watch?v=fyHbx9de3GI&list=PLFEcqBh9-Xwo), Complete on Teachable (https://shipreal.teachable.com/courses/course/lectures/66619411). Teams: michael@shipreal.dev.

## Introduction

Your map of the whole course: what every deep-dive module delivers and how the eleven parts build on each other.

### Introduction: The Road Ahead

The map on the table: all 38 modules, what every deep dive delivers, and which module to jump to when something breaks.

Chapters: You Ship. Now Let's Make It Real.; Part 1: The Mental Shift (Modules 1 through 3); Part 2: The Developer's Foundation (Modules 4 through 9); Part 3: Data (Modules 10 through 12); Part 4: Networks and APIs (Modules 13 through 15); Part 5: Fortification (Modules 16 through 19); Part 6: Infrastructure (Modules 20 through 22); Part 7: Architecture and Performance (Modules 23 through 27); Part 8: The Business Layer (Modules 28 through 30); Part 9: AI and ML Products (Modules 31 through 33); Parts 10-11: Debugging, Production Readiness, and Mobile (Modules 34 through 38); The Anatomy of Every Module; Your Project Is the Patient

You ship code. You've been shipping code. You open Cursor or Copilot, describe what you want, the code appears, you push it, users see it. That skill is real. I'm not here to take it away from you.

But you've had that moment. 2 AM. Something broke. You stared at a stack trace that might as well have been in Aramaic. You poked at it, asked the AI to fix it, deployed the fix, and honestly? You're still not sure what went wrong. You just know it stopped screaming.

That gap, between shipping and understanding, is the entire reason this course exists. You can produce code. What you can't yet do is reason about code. You can't predict where it'll fail, why it's slow, or what happens when ten thousand people hit it at once instead of ten. This course closes that gap. Thirty-eight modules. Let me walk you through the map.

Part 1 rewires how you think. You'll trace what actually happens when a user clicks a button: DNS lookup, TCP handshake, TLS negotiation, HTTP request, routing, database query, response, all the way back to pixels on screen. Your laptop lies to you. Zero latency to itself, a warm database, no other users. Production has none of those luxuries. You'll also learn state (any data your system remembers between requests) and why it's where bugs hide, because two things can try to change it at the same time. One concept from this section, idempotency, prevents an entire class of payment bugs where customers get charged twice.

Part 2 builds the foundation you skipped. Git beyond commit-and-push. Prompting AI like you're writing a spec, then reviewing its output with suspicion, because the AI doesn't know your system's constraints. You do. You'll learn how code actually runs: the Node.js event loop, Python's GIL. These aren't trivia. They determine whether your app handles 100 users or 10,000. You'll get Big-O intuition so you can spot when the AI wrote you a nested loop that'll bring your server to its knees at scale.

Part 3 is data, and this is where most vibe-coded apps are quietly rotting. Can you read a query plan? Do you know what an index is? When you see "Seq Scan" on a table with a million rows, that means PostgreSQL is reading every single row. An index turns that into a handful of lookups. You'll learn schema migrations that don't lock your production database, connection pooling so your deployment doesn't overwhelm PostgreSQL's hard connection limit, and when to start worrying about scale (later than you think, but you need the vocabulary now).

Part 4 covers APIs and networking. An API is a contract. Change it carelessly and you break every client depending on it. You'll learn DNS, TCP, TLS, HTTP, and how to use curl and browser DevTools to figure out where slowness actually lives. Real-time patterns too: WebSockets versus polling. Polling wins more often than you'd expect. […]

## Part 1: The Mental Shift

Rewire how you think about working software - systems, state, and trade-offs. The judgment layer AI can't supply.

### The Illusion of 'It Works'

The request lifecycle, environments, and the production iceberg - why 'it runs on my laptop' means almost nothing.

Chapters: The Line You Can't See; The Life of a Request; The Iceberg Beneath Your Features; Why Your Laptop Lies to You; The Environment Ladder; How Production Fails: Performance and Data; How Production Fails: Security and Cost; The Gap Has a Price Tag; What Engineers Actually Do; Software Is a Restaurant, Not a Document; Your Production Degradation Timeline; The Checklist Before You Close This Module

Your code runs. Users can reach it. You shipped, and that's more than most people ever do. But here's the sentence that changes everything: running code is a necessary condition for production software. It is nowhere close to a sufficient one.

Production means real humans who didn't build your app are using it in ways that would make you question their sanity. It means you're holding data you're legally obligated to protect. It means your infrastructure costs money every second, whether anyone's online or not. And it means bots found your URL about twelve minutes after you deployed.

A demo needs to show features. A production system needs six things simultaneously. Availability, it's up when users need it. Reliability, it does what it's supposed to do every time. Durability, data doesn't vanish. Security, unauthorized access is blocked. Performance, responses come back fast enough. And economy, it doesn't cost more to run than the value it creates. Your vibe-coded app probably nails zero of those six. Not because you're bad at this, but because nobody told you these were the requirements.

Now zoom into what happens when a user taps a button. DNS resolution. TCP handshake. TLS negotiation. HTTP construction. Internet routing. Load balancer. Application routing. Your code. Database connection. Database query. Response assembly. Return trip. That's twelve distinct points where something can fail. Your code is one of twelve. And when your app breaks in production, the bug is more often in layers you didn't know existed than in anything you wrote. An expired TLS certificate. A misconfigured load balancer. Database connection pool exhaustion. You can't debug what you can't see.

Zoom out further. Picture an iceberg. Above the waterline: your features, your UI. Below it: monitoring, logging, backups, security patching, error handling, scaling, cost management, incident response. Every item below the waterline is work someone has to do, and for a solo project, that someone is you. The AI that helped you write features has no opinion about your backup strategy. It doesn't know your TLS cert expires next Tuesday.

Here's a number that should hit you. In mature engineering teams, new feature development accounts for roughly 20 to 40 percent of total effort. The other 60 to 80 percent goes to maintenance, operations, infrastructure, testing, security, and performance. IEEE software engineering studies have shown this for decades. Vibe coders spend close to 100% on features and 0% on everything else. That inversion is a ticking clock.

Your laptop makes this worse because it lies to you constantly. One user (you) versus thousands of concurrent users competing for the same memory, CPU, and database connections. Forty-seven test rows versus 4.7 million, where that 2-millisecond query becomes a 14-second full table scan. No attackers locally versus bots probing every endpoint in production. Race conditions, where two requests hit the same resource simultaneously, literally cannot appear on your machine because you're one human with one browser. "I tested it locally" is not a meaningful statement about production readiness. […]

### Thinking in Systems & State

State, side effects, and idempotency - why 'just run it again' sometimes makes things worse.

Chapters: You Build Features. Systems Build Bugs.; The Cast of Characters: What Systems Are Made Of; Following a Request Through the System; State: The Root of All Your Bugs; Where State Lives: The Decision That Makes or Breaks You; How State Moves: Data Flow Patterns; Rendering Models: Where Your UI Gets Built Changes Everything; Hydration: The Bug Factory You Didn't Know Existed; Stateless vs. Stateful: The Distinction That Governs Scaling; The Statelessness Checklist; Reading Architecture Diagrams; Putting It Together: A System Thinking Walkthrough

You build features. Systems build bugs. You prompt your AI for a login page, a dashboard, a checkout flow. Each one works perfectly. Then a user logs in on their phone while already logged in on their laptop, and your app shows two different shopping carts. Nobody wrote that bug. It emerged from features that were never designed to coexist.

That's the core shift in this module. A feature is a single unit of functionality. A system is the interconnected set of components keeping all those features alive and correct at the same time. Systems produce emergent behavior, things none of the individual parts would do alone. And the math is scary: two features have one interaction. Five features have ten. Twenty features have a hundred and ninety. Every feature you add multiplies the ways things can break.

Two terms you need right now. Coupling is how much one component depends on another. If changing your user table breaks checkout, those are tightly coupled. Cohesion is how well a component's internals relate to each other. You want low coupling between components, high cohesion within them.

So what are these components? Almost every production web app has the same cast. Clients that send requests. Servers holding application logic. Databases for persistent storage. Caches for fast temporary reads. Queues for deferred work like sending emails. External services like payment processors. Load balancers directing traffic. But here's what matters more than any of those boxes: the arrows between them. Every boundary is a place where data gets serialized, transmitted, and deserialized. Every boundary adds latency. Every boundary can fail. A principal engineer sees boundaries first, boxes second. And every interaction across a boundary is a contract, whether you've documented it or not. Every implicit contract is a landmine.

When a user clicks "Place Order," some operations must happen in sequence. You can't calculate the total before you have prices. You can't return success before writing to the database. But the confirmation email? Push that to a background queue. The user doesn't wait. This is your hot path versus your cold path. The hot path is everything the user stares at a spinner for. The cold path is everything else. Your default question for every operation: does the user need to wait for this? If not, move it off the hot path.

Now, state. It sounds simple: data that exists at a given moment. The user's name, the items in their cart, whether they're logged in. It is the source of most bugs you've ever hit. State can be stale (your cache shows a price that changed five minutes ago). State can be duplicated (the same data in your database, your cache, a JavaScript variable, and a URL parameter, all disagreeing). State can be lost (a shopping cart in a JS variable vanishes on page refresh). Every bug, if you trace it deep enough, is a state bug. […]

### Decisions, Complexity & Abstraction

Trade-offs, YAGNI, and when an abstraction pays rent - the judgment calls AI can't make for you.

Chapters: Why "It Depends" Isn't a Dodge; Satisficing: The Art of Good Enough; Build vs Buy vs Glue; The 30-Minute Technology Litmus Test; Complexity: The Thing That Actually Kills Software; The Complexity Budget; Abstraction: Your Most Dangerous Weapon; Over-Engineering: Friendly Fire; Technical Debt Is a Strategy, Not an Insult; The Twelve-Factor App: Principles That Survived 15 Years; Reading Systems You Didn't Build; Reading Documentation Like an Engineer; Code Smells: Diagnostic Signals in Unfamiliar Code; Pulling It Together: Your Engineering Decision Instinct

You've been making technical decisions every day. Framework, database, hosting. You ask the AI, it gives an answer, you ship it. But you've been accepting answers without understanding the constraints behind them. That changes now.

Every technical choice runs on three axes. First, constraints: what's non-negotiable. Budget, timeline, team size, existing infrastructure. Second, tradeoffs: what you gain and what you sacrifice. Every choice has a cost, and if you can't name it, you don't understand the choice. Third, reversibility: how painful is it to change your mind later?

Reversibility is the one most people skip, and it matters the most. Jeff Bezos calls these one-way doors and two-way doors. Your database engine, your primary language, your core architecture? One-way doors. Spend days on those. Write things down. Your CSS framework or logging format? Two-way doors. Spend 30 minutes, pick good enough, ship it.

That phrase, "good enough," has a name. Herbert Simon, Nobel Prize-winning economist, coined "satisficing" in the 1950s. Satisfy plus suffice. You're not hunting for the optimal answer. You're looking for the first option that meets your minimum criteria, then you stop searching and execute. You need a form validation library? Your criteria: works with React, handles async validation, has TypeScript types, 5,000-plus GitHub stars. You find one that checks all four? Done. Don't spend another hour reading comparisons.

For irreversible decisions, write a decision document. Five sections: problem statement, options considered, evaluation criteria ranked, your recommendation, and the conditions under which you'd revisit. That last section is the one everyone skips. Writing "revisit if we exceed 10,000 concurrent users" turns a decision into a living contract instead of forgotten tribal knowledge.

Now, the highest-leverage decision you'll face repeatedly: build versus buy versus glue. Build when the capability is your core differentiator. If you're a payments company, build your payment engine. If you're a recipe app, buy payments. Buy when the problem is well-solved by others and isn't your competitive advantage. Glue is the dirty secret: stitching tools together with thin integration code. Most vibe-coded apps are glue architectures whether the builder knows it or not. Low upfront cost, but every integration point breaks. And the moment glue code gets its own bug tracker, it's become a build whether you planned it or not.

Before you commit to any technology, run a 30-minute litmus test. Five checks, six minutes each. Maintenance pulse: when was the last commit, how fast do maintainers respond to issues? Documentation quality: does the getting-started guide actually work? Community trajectory: are npm downloads growing or declining? API coherence: does the API feel designed or bolted together? And a proof-of-concept against your use case, not the tutorial's. If a technology fails any two of these five, skip it.

Here's the idea that ties everything together. Complexity kills software. Not databases, not algorithms. Complexity. Every line of code is a liability. It must be understood, maintained, tested, and debugged at 2 AM. […]

## Part 2: Developer's Foundation

The daily craft: Git as a thinking tool, an AI-native workflow, runtimes, algorithms, frontend architecture, and the dependency minefield.

### Version Control as a Thinking Tool

Git beyond commit and push: branching strategy, bisect, and blame as archaeology.

Chapters: The Most Misunderstood Tool You Use Every Day; Snapshots, Hashes, and the Graph You Never Knew You Were Building; The Three Trees: Your Surgical Precision Toolkit; Atomic Commits: Write History Like a Story; What Goes In, What Stays Out, and What Can Never Be Deleted; Branching Strategies: Picking the Right Model for Your Team; Merges, Rebases, and the Golden Rule; Conflict Resolution: The Skill Nobody Teaches; Pull Requests Are Design Documents, Not Merge Buttons; Time Travel: Revert, Reset, Cherry-Pick, and Bisect; The Reflog: Your Hidden Safety Net; Monorepo vs. Polyrepo: The Repository Architecture Decision; Putting It All Together: Your Version Control Habits

Your Git repository is a decision log, not a save button. Every commit is a message to your future self, your teammates, and the automated systems that build and deploy your code. Every branch is a parallel experiment. Every merge is a negotiation between two versions of reality. Once you see it that way, everything about how you use Git changes.

Most vibe coders treat Git like a backup drive. You run git add dot, commit with the message "changes," push once a day. That's using a surgical scalpel to butter toast. It works, but you're throwing away the entire point.

So how does Git actually work under the hood? Most people think it stores diffs. It doesn't. Git stores snapshots. Every commit is a complete picture of your entire project at that moment. If a file hasn't changed, Git stores a pointer instead of a duplicate, but conceptually each commit says "here's what everything looked like right now." Each snapshot gets a SHA-1 hash, a 40-character fingerprint. Change one character in one file and you get a completely different hash.

Commits point to their parents, forming a directed acyclic graph. A branch is just a pointer, a sticky note on one node in that graph. Creating a branch costs almost nothing because it's literally writing 40 characters to a file. HEAD points to whichever branch you're on. Merging creates a commit with two parents. Rebase replays commits onto a different base. These aren't magic incantations. They're graph manipulations. When a Git command confuses you, draw the graph on paper. Boxes and arrows. The answer becomes obvious.

Now, Git gives you three places where code lives, not two. First, your working directory, the actual files on disk. Second, the staging area, which is your proposed next commit. Third, the repository itself, the committed history. The staging area is what gives you surgical control. Say you changed eight files today. Five fix a login bug, three refactor a utility. With git add, you stage just the five bug-fix files, commit with a clear message, then stage the other three and commit separately. You can even stage individual lines within a file using git add -p. Two logical changes in one file become two separate commits.

This connects directly to atomic commits. Each commit should represent one logical change, not everything you did today. This matters especially when your AI assistant generates fifty lines in one shot. Don't commit the blob. Break it apart. The database migration gets its own commit. The API endpoint gets its own. Each one tells a piece of the story. Write imperative subject lines under 50 characters: "Fix session timeout," not "Fixed" or "Fixing." If you need to explain why, add a body paragraph after a blank line.

Your local history can be messy while you work. That's fine. Use git rebase -i to squash and reword before you push. But here's the golden rule: never rebase commits that have been shared. Rebase rewrites hashes. If someone already pulled your commits, you'll create a mess that takes real time to untangle. […]

### The AI-Native Engineering Workflow

Prompting as spec-writing, reviewing code you didn't write, and the guardrails that keep AI speed safe.

Chapters: The Slot Machine Problem; The Division of Labor; Specification-Driven Development; Effective Prompting (Without the Magic); Context Management: What the AI Can See; Reviewing AI-Generated Code; Calibrating Your Trust; Naming as a Design Act; Code Organization and the DRY Trap; Type Systems: Your Best AI Guardrail; Automated Quality Enforcement; Documentation That Actually Gets Read; Putting It All Together

Your AI coding assistant is a slot machine, and you've been pulling the lever without knowing it. You prompt, you get code, you paste it in. If it runs, you move on. If it doesn't, you tweak and try again. That feels productive. A METR randomized controlled trial from July 2025 measured experienced developers on real tasks and found they were actually 19% slower with AI, while believing they were 20% faster. A 43-point gap between perception and reality. The tool felt like progress while making them worse.

Why? Because generating code isn't building. It's accumulating. GitClear's research across 211 million lines of code showed code duplication rose eightfold in 2024. Refactored lines dropped from 25% of changes to under 10%. For the first time ever, copy-pasted code exceeded refactored code. AI makes generation so cheap that nobody cleans up.

So here's the workflow that replaces the slot machine. It has a clear division of labor. You own the what and the why: requirements, architecture, constraints, edge cases, the decision about whether the output actually solves the problem. AI owns the how at the syntax level. Picture briefing a brilliant but context-blind contractor. They'll execute your blueprint with impressive speed and zero initiative to question whether the blueprint makes sense. Hand them a bad one, they'll build a bad building. Fast. And the most dangerous trait: AI never says "I don't know." It generates confident-looking output regardless of whether it has adequate information.

Default rule: if the task is a well-known pattern with clear inputs and outputs, let AI draft it. If it requires judgment about your system or your users, you draft and AI assists.

The single highest-leverage practice in this entire module is specification-driven development. Write down what you want before you ask AI to build it. "Build me a login page" is a coin flip. A structured spec describing the authentication flow, edge cases like expired tokens and account lockout after five failed attempts, error states, and validation rules produces dramatically better output. Often on the first attempt.

A spec answers five questions. What does this feature do? What are the inputs and outputs? What are the edge cases? What are the constraints? What does done look like? Thirty minutes writing a spec saves you two hours debugging AI output that solved the wrong problem. No spec, no code. Treat it like a prescription.

Once you have a spec, manage what the AI can see. Every model has a finite context window. Bigger doesn't mean better. Feed it your entire codebase and it drowns in noise. Feed it the three files relevant to your current task, with clear type definitions, and you get better output with a fraction of the context. Your file structure is implicit context. A file named stripe-webhook-handler.ts tells the AI more than handler.ts before it reads a single line. […]

### How Code Runs: The Runtime Layer

Processes, memory, and the event loop - why Node blocks and why Python's GIL matters.

Chapters: The Gap You Don't Know You Have; From Text File to Running Instructions; Processes and the OS Contract; Memory: Stack and Heap; Memory Leaks in Garbage-Collected Languages; Garbage Collection: How It Actually Works; Execution Models: Threads, Event Loops, Coroutines; Blocking vs Non-Blocking I/O; Concurrency vs Parallelism; Async/Await: What It Actually Does; Cold Starts and Connection Costs; Resource Limits and Production Failure Modes; Putting It All Together: The Runtime Mental Model

Between the code you write and the thing your computer does, there's an entire layer of machinery you've never inspected. That layer is the runtime. It controls how fast your app responds, how much memory it burns, why it freezes randomly on a Tuesday, and why it dies after six days with nothing but "OOM killed" in a log file nobody checked. This is the physics of software. And your AI assistant will never warn you about any of it.

Your source code is a text file. CPUs can't read text files. Something has to translate. Three strategies exist. First, ahead-of-time compilation: languages like Go and Rust translate everything to machine instructions before the program runs. You get a binary. Fast execution, but you compile per platform. Second, interpretation: Python and Ruby use an interpreter that reads and executes your code line by line. Quick iteration, slower execution. Third, JIT compilation: JavaScript's V8 engine and Java's JVM start by interpreting, then watch which functions run most and compile those hot paths to optimized machine code while your program is live. Your code literally gets faster the longer it runs.

Here's what changes your decisions: the same source code behaves differently depending on which runtime executes it. Python runs dramatically faster under PyPy than under CPython. A JavaScript function performs differently in V8 in the browser versus Node.js on a server. Your default: always know which runtime and version your production code runs on. Pin it. Don't assume.

When you run your program, the OS creates a process with its own memory space, its own file handles, its own identity. Your CPU has maybe 8 cores but dozens of processes running. The kernel handles this through time slicing, giving each process a few milliseconds of CPU time, then switching. Looks simultaneous. Isn't.

Processes die two ways. A graceful shutdown sends SIGTERM, and your code can catch it, finish that database write, close connections. A hard kill, SIGKILL, vaporizes your process instantly. No cleanup. The OOM killer uses SIGKILL. When Linux runs out of memory, it picks the hungriest process and terminates it without warning. You find "Killed" in your logs hours later if you're lucky.

Memory lives in two places. The stack is fast and automatic: function calls push frames on, returns pop them off. The heap is large and messy: objects, arrays, anything with a dynamic lifetime goes there. Value types like numbers live on the stack. Reference types like objects live on the heap with a pointer on the stack. Pass an object to a function and you're passing the pointer. Two variables pointing at the same heap object, mutate through one, the other sees it. Your AI will generate this bug constantly. […]

### Algorithmic Thinking: Complexity & Data Structures

Big-O intuition and arrays vs maps vs trees - when N-squared quietly kills you at 10,000 users.

Chapters: "It Works Fine" Is Not an Engineering Statement; Big-O as a Feeling, Not a Formula; Pricing Your Everyday Operations; Arrays: What They're Good At, What They're Terrible At; Hash Maps: The Single Most Useful Performance Upgrade; Sets: Membership and Deduplication in One Line; Stacks and Queues: Order as a Constraint; Trees and Heaps: Why Your Database Is Fast; Graphs: Your Data Is Already a Graph; Choosing the Right Structure: A Decision Checklist; Reading AI-Generated Code Like an Engineer; Talking to the AI Better: Complexity as a Prompt Tool; Installing the Smoke Detector

Stop asking "is this fast?" and start asking "is this fast as the data grows?" Those are different questions with different answers. Only the second one matters in production.

Ask an AI to find duplicate users. It'll hand you a nested loop that compares everyone to everyone. Works great on 50 test users. Hangs for 40 seconds on 50,000 real ones. The AI isn't wrong about what the code does. It's indifferent to what it costs, because cost only shows up at scale. That judgment is yours to provide.

Big-O notation gives you the vocabulary. Five growth rates, that's all you need. O(1), constant time: a hash map lookup takes the same time whether you have ten entries or ten million. Like knowing your locker number. O(log n), logarithmic: doubling your data adds one extra step. A billion database rows? About 30 comparisons. O(n), linear: double the data, double the time. Reading every item once. Predictable, honest, usually fine. O(n log n): the cost of sorting. Rarely your bottleneck. And O(n squared), quadratic: double the data, quadruple the time. A loop inside a loop over the same collection.

Let me make that last one visceral. Take a million items. O(n) is a million operations, your CPU handles that in milliseconds. O(n squared) on the same data? A trillion operations. Over sixteen minutes. Same data. Same machine. Different code shape.

Here's the damn thing that catches people. Quadratic complexity hides. An array.includes() call is itself a loop, scanning from start to finish. Put it inside a .filter() or a for loop and you've built an O(n squared) machine without writing the word "for" twice. Same goes for .find() and .indexOf(). This is the number one performance smell in AI-generated code.

Two data structures fix 90% of the problems you'll hit. Arrays and hash maps.

Arrays are perfect when your job is "a sequence of things I'll process in order." Rendering a list of messages. Mapping over products. Where arrays fall apart: finding a specific item by some property. "Give me the user with ID 4872." The array has to scan from the start, checking each item. That's O(n). Do that search inside a loop and you're paying quadratic cost.

The fix is a hash map. Dictionary in Python, object or Map in JavaScript. You give it a key, it gives you the value, in O(1). It doesn't scan. It computes where the value lives and jumps straight there. The single most common performance upgrade in real code: you notice code repeatedly searching an array by some ID, and you convert that array to a hash map keyed by that ID. That one change turns O(n squared) into O(n). I've applied this probably hundreds of times.

A Set is a hash map that only stores keys, no values. It answers "is this thing in the collection?" in O(1). Need unique values from an array? new Set(array). Done. One line, correct by construction. The "oops, I double-counted" bug can't happen because the container makes it structurally impossible. […]

### Frontend Architecture: What AI Builds for You (And What It Gets Wrong)

Rendering models, state management, and the parts of your frontend AI scaffolds wrong.

Chapters: The Screenshot Lies; Components That Have One Reason to Change; The Tree, Props Down, Events Up; Five Kinds of State and Where Each One Lives; Derived State and the Server Cache You Keep Rebuilding; Re-renders, Keys, and Why Memo Isn't the Fix; The URL Is Your Most Important State Container; Forms: Every Unhappy Path AI Skipped; Accessibility Is Load-Bearing; CSS Architecture and the Fifty Shades of Gray Problem; Design Systems: Buy the Behavior, Own the Paint; Mobile-First for Real, Not as a Slogan; Every Byte You Ship Is a Choice

Frontend failures don't crash. That's the whole problem.

A missing database index throws a timeout and wakes you up at two in the morning. A missing aria-label throws nothing. It quietly removes a category of humans from your product, and nobody files a bug, because the people affected just leave. A useState in the wrong component doesn't error either. It just makes your app feel cheap in a way nobody can name, and the cost of fixing it climbs every week.

AI is tuned for exactly one thing: code that looks correct in a screenshot. That's maybe a third of frontend engineering. The rest is invisible to a camera. Where state lives. Which direction data flows. What HTML element you actually produced. What the browser downloads before the first pixel.

So start with a habit. When AI hands you a component, don't read the markup. Read two things: every useState in the file, and every element carrying a click handler. State placement tells you the architecture. Click handlers tell you whether you just built a keyboard trap. Thirty seconds, and it catches most of what follows.

State placement is the game. There are five kinds and they are not interchangeable. Local state, like a dropdown's open flag. Lifted state, the same thing moved up because two siblings need it. Global state: current user, theme, permissions. Server state, which is data that lives in your database and is only cached in your browser. And URL state: the route, the active filter, the search query.

Ask AI for a dashboard and you get a store with all five jammed into it. Selected tab, modal flag, form draft, fetched rows, one global object. Looks tidy. It's a trap. That's how you get a store where updating a user's avatar re-renders the product catalog and no single line of code explains why.

Colocate instead. Start local, always. Move state up only when a second component provably needs it. Most apps have fewer than five things that genuinely belong in a global store, and three of them are the current user, the theme, and an auth token. My test: if you deleted this feature tomorrow, would this state get deleted with it? A checkout wizard's step number isn't app state. It's checkout state. The real cost of getting this wrong isn't slow rendering. It's that six months in, nobody can delete anything.

Two special cases cause more bugs than everything else combined.

Derived state first. AI writes tasks, filter, and filteredTasks, plus an effect keeping the third in sync. Three sources of truth for one fact. They drift. Rule: if you can compute it, compute it during render. Your filtered list is a const, not a useState. Filtering five hundred objects takes under a millisecond, so skip the useMemo. […]

### Dependency Management: The Hidden Minefield

Lockfiles, semver lies, and supply-chain attacks - the left-pad story and how not to star in the sequel.

Chapters: You're Shipping 300 Megabytes of Strangers' Code; What Install Actually Does; Semver Is a Promise, Not a Guarantee; Lock Files: What You Actually Got; Diamonds, Duplicates and Peer Dependencies; The Attack Surface of Install; Should This Package Exist In Your Project; Tree Shaking and What Actually Ships; Staying Current Without Losing a Week; Your Own Code as a Dependency; Triaging Audit Noise; Write the Policy Down

Your package.json has eight lines. Your node_modules folder has 300 megabytes and several hundred packages in it. That gap is the whole lesson.

Start with the idea that should change how you work today: your dependency graph is your architecture. Not the folder structure your AI assistant generated. Not the boxes in your README. The set of packages that actually load into memory when your process boots - that's the system you're on call for.

Two words, fast. A direct dependency is something you asked for by name. A transitive dependency is something your dependency asked for. You never typed its name. You never approved it. And it runs with exactly your permissions: your file system, your environment variables, your network.

A fresh Vite React TypeScript app lands around 200 packages. Add ESLint, Tailwind, a test runner, a component library and an ORM, and you're past 700 without doing anything weird. Twelve names you chose. Seven hundred you inherited. Nobody reads 700 packages. I've never met an engineer who has, and I don't believe anyone who says otherwise. The goal isn't paranoia. It's moving from install-as-reflex to install-as-decision.

So do this first: write two numbers at the top of your README. Direct dependency count. Total installed package count. Then watch the delta on every pull request. When a one-line feature adds 40 packages, that's a design decision somebody made without telling you.

Now, versions. Semver is three numbers and a promise from a stranger, and the only enforcement is embarrassment. Caret two point three point zero means anything below 3.0. But below version one, caret changes meaning: caret zero point two point three does not allow 0.3.0, because pre-1.0 authors reserve the right to break you on any release. That catches engineers with five years in. And the promise breaks even when honored - TypeScript states plainly that it doesn't follow semver, because minor releases add compile errors to code that used to build.

The default a principal engineer picks: in an application, save exact versions. Set save-exact true and let a bot propose upgrades as reviewable pull requests, instead of letting a resolver decide in the dark at 3am. In a library you publish, use caret ranges, because pinning exact versions forces duplicate copies into every consumer's tree. The range you write is your blast radius.

Your manifest is what you want. Your lock file is what you got. It records every resolved version plus a sha512 integrity hash, so the tarball CI downloads today is byte-for-byte what resolved on your laptop three weeks ago. Commit it. And then respect it: npm install treats the lock file as a suggestion, npm ci treats it as law and fails when the manifest disagrees. CI must use npm ci, or a frozen lock file in pnpm, or an immutable install in Yarn. When something breaks, do not delete the lock file. Deleting it re-rolls the dice on hundreds of packages and hands you a brand new set of bugs while you're already debugging. […]

## Part 3: The Data Layer

Databases as mental models: tables and indexes before queries, SQL without fear, and migrations that don't take production down.

### Database Fundamentals: Thinking in Data

Thinking in tables, indexes, and constraints before you write a single query.

Chapters: Your App Is a Data Model Wearing a UI; Entities and Attributes: Which Nouns Earn a Table; Relationships, Cardinality, and the Junction Table Nobody Respects; Modeling a Real Domain: What Is vs What Happened; The Relational Model and the Keys That Hold It Together; Constraints: Business Rules the Database Actually Enforces; Normalization Without the Textbook; Denormalization as a Calculated Bet; Beyond Relational: Picking the Right Shape; Schemas Are Contracts With Your Future Self; ORMs and Query Builders: The Leaky Layer You Use Daily; Where the Database Runs: Managed, Self-Hosted, Embedded; Choosing a Database Without Lying to Yourself

Strip the CSS off any app you shipped this year. Strip the animations, the routes, the toast notifications. What's left is a dozen tables and the lines between them. That's the product. Everything else is paint.

Vibe coders think in screens. Engineers think in data. And your data model decides what's easy, what's hard, and what's impossible for the next three years.

Here's how that plays out. You build a chat app. The AI writes a messages table with sender_id and recipient_id. Ships in an afternoon. Four months later the PM wants group chats. That recipient_id column cannot express "three people." Now you need a conversations table, a participants table, a backfill of four hundred thousand rows, new indexes, a rewritten read path, and every API response changes shape. A two-day feature becomes a three-week migration with a data integrity risk stapled to it. Nothing about the code was wrong. The model was too small for the domain.

Code is cheap. You can rewrite a service in a weekend. Data has gravity, and it gets heavier every day you're in production.

So steal this habit. Before you prompt for a single line of code, open a text file and write the tables and relationships as plain sentences. "A user has many orders. An order has many line items. A line item points at one product." If you can't write those sentences, you don't understand the feature yet, and neither does the model you're about to prompt.

Which nouns earn a table? Run three questions on every candidate. First, does it have identity that outlives this request? Second, does anything else need to point at it? And third, does it have its own lifecycle - created, updated, deleted? A shopping cart passes all three. A "dashboard" fails all three. It's a view, not a thing.

For attributes, don't store what you can derive. Store birth_date, never age, because age is wrong the second you write it down. But here's the exception people miss: when the source of a fact can change and history must not, you copy the value. An order line stores unit_price_at_purchase, because the product's price changes next Tuesday and your 2023 invoices must still say what the customer actually paid. An invoice that recalculates itself is a lawsuit.

Relationships. One-to-many is the workhorse, and the foreign key lives on the many side. Orders.user_id points at users.id. Say it out loud, because AI-generated schemas love to do the opposite and jam an array of order IDs onto the user row. That can't be indexed, can't be constrained, and rots the first time a write fails halfway.

Many-to-many needs a third table, and this is the one nobody respects. Students and courses give you enrollments. People call it a junction table and treat it as plumbing. It isn't. Junction tables always grow attributes: enrolled_at, then grade, then status, then dropped_at. Name it Enrollment, treat it as a real entity from day one, and put a unique constraint on the pair. […]

### SQL & Query Thinking

Joins without fear, query plans you can actually read, and the queries that melt at scale.

Chapters: Declarative Is the Whole Trick; Sets, Not Loops; The Query Runs in a Different Order Than You Wrote It; JOINs and the Rows That Multiply; Counting Things Correctly; Indexes: The One Performance Concept That Pays for Itself; Stop Guessing. Ask the Planner.; Transactions: What You're Actually Promised; Isolation Levels and the Double Booking; N+1: The Silent Epidemic; CTEs, Subqueries, and the NOT IN Landmine; Window Functions: Analytics Without Leaving the Database; Views, Materialized Views, and What to Do Monday

Two queries can return the exact same rows, where one finishes in three milliseconds and the other takes ninety seconds. Same answer. Different plan. Understanding why is the whole skill.

SQL is declarative. Every other language you've written is imperative: you spell out the steps, the machine follows. In SQL you describe a set of rows and hand it over. A component called the query planner decides how to get it - which table to read first, which index to use, whether to sort or hash. That trade is why a query somebody wrote in 1985 still runs today, only a thousand times faster, because the planner got smarter and the code never changed. It's also why the NoSQL wave that was supposed to bury SQL around 2010 ended in surrender. Cassandra shipped CQL. DynamoDB added PartiQL. Spark's main interface is Spark SQL.

The catch is that you don't control execution. So your job changes. Stop writing steps. Start describing results clearly and giving the planner what it needs.

Which brings the first rewire: sets, not loops. You need to give two hundred thousand users in Israel a fifty shekel credit. Your instinct is to select them and loop. That's two hundred thousand round trips, each paying network latency and transaction overhead. At half a millisecond each, that's a hundred seconds of your life. The set version is one UPDATE with a WHERE clause. One round trip. Sub-second. And atomic, so a crash halfway doesn't leave half your users angry. The speed is the obvious win. The deleted bugs are the bigger one - no iteration order, no accumulator to forget, no off-by-one in the pagination. Tattoo this: a database query inside a loop is a bug until proven otherwise. Not slow code. A bug.

Now the trap that makes dashboards lie. A join to a one-to-many relationship multiplies rows. Join gigs to bookings and a gig with twelve bookings appears twelve times. Add a second join to reviews, five of them, and that gig is now sixty rows. Somebody sums the booking amount and reports revenue five times too high. The query doesn't error. Finance believes it for a quarter. Fix: aggregate each one-to-many side in its own CTE, then join the already-collapsed results. And after any multi-table query, run COUNT star against the count of your driving table. If they differ and you didn't mean it, you've got a bug that returns results instead of an error. Worst kind there is.

One more join edge, because it fools mid-level engineers. Put a filter on the right-hand table in the WHERE clause of a LEFT JOIN and you've silently turned it into an INNER JOIN. WHERE runs after the join, so your NULL rows fail the condition. Filter belongs in ON if it describes what should match, in WHERE if it describes which final rows you want. […]

### Data at Scale: Mutations, Migrations & Beyond

Mutations, migrations, and backfills that don't take production down with them.

Chapters: Success Is the Problem; Why Your Database Says No; Transaction Mode, Session Mode, and Serverless; Expand and Contract: Changing the Plane While Flying; The DDL That Locks Your Table; Deletes, Bloat and Retention; Two Writers, One Row; Replication and the Read You Can't Trust; Failover Is a Loaded Weapon; Partitioning, Sharding, and the Ladder You Climb First; When LIKE Takes Nine Seconds; The Backup You Never Restored; CAP: The Questions, Not the Triangle

Every database technique you're about to learn buys you something and charges you something. Read replicas buy read throughput and charge you stale reads. Soft deletes buy an audit trail and charge you table bloat forever. Sharding buys write throughput and charges you your entire query model. So the skill isn't knowing the techniques. It's knowing which pain you actually have right now.

Which means before you adopt anything, write down the number that made you need it. Connections in use versus your ceiling. Rows in your largest table. Replication lag at peak. If you can't write the number down, you're not scaling. You're decorating.

Start where the pain starts, because it's never where people look. A Postgres connection isn't a channel. It's a whole operating system process, forked at connect time, holding several megabytes before you run a single query. That's why your managed database caps you around 100. Raising that to 2000 doesn't buy you 2000 workers. It buys you 2000 processes fighting over eight cores, and throughput drops while latency climbs. An oversized pool is slower than a right-sized one.

Sizing comes from the HikariCP formula that's held up for twenty years: cores times two, plus your disk concurrency. Eight cores, about eighteen connections. Total. Across every pod. Teams set a pool of twenty, feel responsible, then autoscaling runs forty pods and asks the database for eight hundred connections against a ceiling of a hundred. Your pool size is your ceiling divided by your max instance count, minus headroom.

Two things to change today. Set an acquire timeout of two to five seconds so connection famine shows up as a fast clear error instead of hanging requests. And export pool wait time as a metric. The moment average wait climbs above zero, you're queuing, and that's your warning weeks before the outage.

Beyond that, run PgBouncer in transaction mode as your app's endpoint. It hands out a backend connection when a transaction starts and takes it back at COMMIT, which is how four thousand clients ride on twenty backends. The price is that anything outside a transaction breaks: session SET commands leak into other requests, advisory locks silently release, LISTEN and NOTIFY dies. So run a second tiny pool in session mode on a different port for migrations and psql. App points at transaction mode. Tools point at session mode.

Now schema changes. Kill one belief immediately: the down migration. In production you'll almost never run it, because by the time you notice the problem, new rows exist in the new shape and rolling back destroys them. You roll forward.

The pattern that keeps you alive is expand and contract. You never change a thing, you add the new thing, move traffic, then remove the old one. Renaming a column is five deploys, not one. Add the nullable column. Deploy code writing both. Backfill in batches of five to ten thousand rows watching replication lag. Deploy code reading the new one. Drop the old. Slow, yes. Also the difference between a boring week and a rollback at midnight. […]

## Part 4: Networks & Contracts

How systems talk to each other: APIs as contracts, the DNS-to-HTTP chain every request lives on, and real-time patterns.

### API Design as Contracts

REST vs RPC thinking, versioning, and evolving APIs without breaking the people who depend on them.

Chapters: The Eight-Second API and the Ten-Year Promise; Contract-First and Consumer-Driven Design; REST: The Constraints That Actually Pay Rent; HTTP Methods and the Idempotency Contract; Status Codes as a Communication System; URLs, Resources, and Actions That Don't Fit CRUD; Payloads, Nulls, and the PATCH Everyone Botches; Pagination: The Decision You Can't Take Back; Error Design as a First-Class Contract; GraphQL: What It Buys and What It Costs; gRPC and tRPC: Contracts Between Services; OpenAPI, Docs, and Specs That Don't Drift; Versioning, Deprecation, and the Enum Trap

Your assistant wrote a working CRUD API in about eight seconds. Routes, controllers, serializers. It ran, you shipped it. What those eight seconds didn't give you is a promise you can keep.

That's the shift this module is about. An API isn't an interface. It's a contract. Promises about how to ask, what shape comes back, what errors look like, and how the thing behaves when a caller does something stupid.

Internal code is different in exactly one way. Rename a function and the compiler finds every caller. You fix them in one commit. Your API's callers live on someone else's laptop, inside a mobile app nobody's updated since 2022. You can't grep for them. You can't fix them. You can only break them.

Hyrum Wright at Google named the rule. With enough users, every observable behavior of your system gets depended on by somebody. Not the documented behavior. The observable one. Your IDs being sequential integers, which some analyst is quietly using to estimate your monthly order volume. Switch to UUIDs and you break a business report you've never heard of.

So tier your routes before you design them. First, internal - your own services, break freely, coordinate in the repo. Second, partner - integrators whose email addresses you have, thirty days notice. Third, public - strangers, versioned, never broken. Write it in a file called API-POLICY.md. Ten minutes of typing, two years of arguments avoided.

Now the ordering problem. Most people build the endpoint first and generate docs from decorators. Backwards, and you can see the damage. Implementation-first turns your API into a mirror of your database. Your users table has forty columns including password_hash and internal_risk_flags, the serializer dumps the row, and now every one of those is a promise.

Contract-first means you write the response before the handler. Literal JSON, real values, a document a human can read. Then go further: start from the job the caller is doing. A mobile list screen needs an ID, a title, a thumbnail, and an unread count. Four fields. Not your forty-column entity.

Here's the test. Hand that example payload to the person building the client before any controller exists and ask them to sketch the render. If they start flattening arrays or deriving a boolean from three fields, your shape is wrong. Fifteen minutes arguing over a JSON blob beats three weeks of them building a compatibility layer around your mistake. And that layer, once written, is permanent.

There's a bonus that fits how you already work. Coding assistants are great at generating handlers from a spec and mediocre at inventing a good contract. Flip the direction. Write the payload, write the OpenAPI, hand the model the spec, ask for handlers and tests. Better code, because you made the decisions instead of outsourcing them. […]

### Networking Fundamentals

DNS, TCP, TLS, HTTP - the chain every request lives or dies on.

Chapters: The Pipe Is Not Magic; Addresses, Ports and the Socket Nobody Explained; TCP, UDP and the Cost of Being Reliable; DNS: Caches Expiring, Not Propagating; HTTP, Four Generations Deep; TLS Without the Hand-Waving; CORS Is Not a Feature You Enable; Everything Between the User and Your Server; Drawing the Blast Radius; Physics Doesn't Negotiate; Timeouts, Retries and the Stampede You Caused; The Triage Playbook

Your dashboard says the server answered in 40 milliseconds. Your user says the page took four seconds. Both are telling the truth. The missing 3.96 seconds happened inside the pipe, and if you don't have a model of the pipe, you'll spend the afternoon adding a database index to fix a problem that lives in a TLS handshake.

So build the model. Four layers, bottom to top. The link layer moves bits to the next box on the wire. IP gets a packet across the planet and promises nothing about arrival. TCP or UDP turns "maybe" into "reliably, in order" - or deliberately refuses to. On top sits HTTP, DNS, your API. Each layer wraps the one above it in its own header. A letter, in an envelope, in a mailbag, on a truck. Routers only open the outer envelopes, which is why a firewall filtering on port numbers has no clue what JSON you sent. Forget the seven-layer diagram you saw once. I've never fixed an outage by knowing the presentation layer exists.

What the model buys you is triage. Emergency rooms check airway, breathing, circulation, in that order, because the order kills whole categories fast. Same idea here. When someone says "the API is slow," you don't open the code. You run one curl command with a timing format string that prints DNS lookup, TCP connect, TLS handshake, time to first byte, and total. Five numbers. The fat one names your suspect. Put it in your shell as an alias today.

Now the currency you're actually spending: round trips. Light in fiber moves about 200,000 kilometers per second. New York to London round trip has a hard floor near 56 milliseconds, and real cable routes bend, so you'll measure 70 to 80. Count a cold HTTPS request. DNS is one. TCP handshake, two. TLS, three. Your actual request, four. That's roughly 300 milliseconds of pure geography before your database is even consulted. No language rewrite touches that number.

Which is why the highest-value performance work is deleting round trips, not shaving microseconds off a handler. Reuse connections. A pooled keep-alive connection skips the handshake and keeps its congestion window warm, and that matters because TCP doesn't start at full speed - it starts around ten packets, roughly fourteen kilobytes, and doubles each round trip. A 200 kilobyte bundle on a fresh connection burns four or five round trips just ramping up. In Node, that's an agent with keepAlive on. In Python, a Session, not a bare get inside a loop.

Three defaults a principal engineer picks without debating. First, set explicit timeouts on every outbound client, because Linux TCP keepalive won't fire for two hours and the network can't tell "slow" from "dead" - only your timeout can. Second, retry only what's safe to repeat, with exponential backoff and full jitter, capped by a budget. AWS's client libraries use a token bucket so retries stay near ten percent of traffic. Without that, a struggling service gets tripled load from polite clients and dies properly. Third, remember a 504 is not a rollback. The upstream may have completed the work. If that call moved money, you now have a payment in an unknown state. […]

### Real-Time & Event-Driven Patterns

WebSockets, server-sent events, and event-driven flows - for when polling stops being enough.

Chapters: The Pull Ceiling; Four Transports, One Decision; WebSockets: The Upgrade And The Hole In It; Staying Connected: Heartbeats, Backoff, And Gaps; Server-Sent Events: The One You're Skipping; Long Polling: Ugly, Useful, Everywhere; Webhooks: Someone Else's Push, Your Problem; Commands And Facts; Event Sourcing: Keep The Ledger, Derive The Balance; CQRS Without The Religion; Pub/Sub: The Wiring Behind Events; Scaling, Serialization, And When To Buy Instead

That little green LIVE pill on your dashboard is lying to you. Behind it is a timer firing every five seconds, asking the server the same question and getting the same answer. Nothing new. Nothing new. Nothing new.

HTTP is a pull model. The client asks, the server answers, the exchange ends. The server never speaks first because it has no idea where you are.

Do the arithmetic, because arithmetic is what wins design reviews. Ten thousand users polling every five seconds is two thousand requests per second. In a normal app, ninety five percent come back as an empty array, and each one still costs you auth middleware, a session lookup, and a database query. You're paying retail for the word "no." And you still lose - your data is five seconds stale. Tighten to one second and you multiply load by five to buy back four seconds. That's the ceiling of pull. Freshness costs load, linearly, forever.

Push flips who starts talking. The server holds a connection open and writes the instant something happens. Zero wasted requests. Here's the bill: an open connection is memory that stays allocated, a file descriptor that stays claimed, and state pinned to one server your load balancer can't shrug off. You just turned a stateless web tier into stateful infrastructure.

So before you pick a technology, write down three numbers. First, which direction does data actually flow. Second, your freshness budget in seconds, honestly, not aspirationally. Third, peak concurrent connections. Those three numbers pick your protocol. Not a blog post, and definitely not whatever your AI assistant scaffolded last week.

Now the defaults. Freshness budget over thirty seconds? Poll. Seriously. A notification bell allowed to be a minute stale doesn't need persistent connections, and you'll thank yourself at three in the morning. Data flows one way, server to client? Server-Sent Events. Both sides sending frequently with latency that matters - chat, multiplayer, trading? WebSockets.

Server-Sent Events are the most underused thing in web engineering, and it's a branding problem. The name sounds like a 2011 jQuery plugin. The mechanism is one sentence: your server responds with Content-Type text/event-stream and never closes the response. It's an HTTP response that takes forever on purpose. What you get free is enormous. The browser's EventSource reconnects automatically, and it sends back a Last-Event-ID header so your server resumes exactly where it stopped. That's why every serious language model API streams tokens over SSE, not WebSockets. Two traps: Nginx buffers proxied responses by default, so send X-Accel-Buffering: no or your "real-time" stream arrives in clumps. And over HTTP/1.1 browsers allow six connections per domain total, so run SSE over HTTP/2.

The mistake I see constantly in AI-scaffolded projects is WebSockets everywhere because the feature "feels real-time." You end up with a full-duplex socket carrying one message every four minutes, plus sticky sessions, plus a Redis backplane, plus a reconnect bug you'll debug for a week. […]

## Part 5: Fortification

Making it survive contact with reality: security by default, resilient error handling, tests that buy confidence, and observability that answers 'what broke at 3 AM'.

### Security by Default

OWASP's top risks, injection, and auth failures - the attacks that find you, not the other way around.

Chapters: The Code Works. That's Not the Same as Safe.; Threat Modeling in Thirty Minutes; The OWASP Top Ten as a Map, Not a Quiz; Injection: Data That Becomes Instructions; XSS and the Browser's Only Real Rule; CSRF, SameSite, and the CORS Misunderstanding; Authentication: Buy It, Don't Build It; Authorization: The Check AI Always Skips; Passwords and the Crypto You're Allowed to Touch; Sessions, Cookies, and the JWT You Can't Log Out Of; Secrets, Rotation, and Least Privilege; Defense in Depth: The Outer Rings

Working code and safe code are graded by two completely different tests.

You grade working code by clicking your own buttons, in the order you designed them. Safe code gets graded by a stranger who never loads your page at all. They read your JavaScript bundle, find your API routes, and hit them with curl at three in the morning using values you never imagined.

So here's the real definition. Security is the property that your system still behaves correctly when the person using it wants it to behave incorrectly. Not a scanner. Not a checklist. A property.

And this is where AI-generated code is weak structurally, not by accident. You asked for an endpoint that returns an invoice by id. The model returned the invoice by id. It never asked whether the person requesting invoice 4,112 is allowed to see invoice 4,112, because you didn't ask, and the tutorials it learned from skip it too.

Adopt this habit today. For every endpoint you ship, write one sentence describing what happens when the caller isn't your app. Not your React component. A raw HTTP request from someone with a valid login and bad intentions. Can't answer it? You haven't finished the endpoint.

Now, threat modeling. It has a reputation problem - people picture a two-day workshop with sticky notes. It's four questions, and Adam Shostack refined them at Microsoft. What are you building. What can go wrong. What are you going to do about it. Did the fix work.

Draw your app as boxes and arrows. Browser, server, database, payment provider, webhook receiver, background worker. Then draw a dotted line everywhere data crosses from somewhere you don't control into somewhere you do. That line is a trust boundary, and it's the only part of the diagram that matters. Everything inside your server is trusted. Everything arriving at it is a claim, not a fact.

Walk each arrow and ask six questions, the STRIDE list. Spoofing: can someone pretend to be another user? Tampering: can they change data in a hidden field or a JWT payload? Repudiation: can you prove who did it? Information disclosure: does that stack trace leak more than you meant? Denial of service: what does one attacker with a loop do to your bill? And elevation of privilege: can a normal user become an admin? Twenty minutes, one page of findings. Do it before you write the feature.

Which brings us to the check AI always skips. Authorization failures are the single most common serious flaw in AI-built applications, and it isn't close. OWASP refreshed their Top Ten in November 2025, and broken access control stayed at number one, now explicitly covering API authorization - the endpoint that checks you're logged in and never checks whose data you asked for.

Authentication is a bouncer at the front door of the building. It says nothing about which apartments you can enter. Change the id in the URL from 4,112 to 4,113 and you're reading someone else's records. In 2022, Optus in Australia had a customer API reachable with no authentication at all. Roughly ten million people's records walked out. […]

### Error Handling & Resilience

Timeouts, retries with backoff, and circuit breakers - failing gracefully instead of loudly.

Chapters: The Unhappy Path Is the Job; Why Generated Code Is Optimistic; Classify Before You Handle; Partial Failure: The Charge With No Order; Handle, Propagate, or Swallow; Three Audiences, One Failure; Timeouts: The One Pattern to Take; Retries Without the Storm; Circuit Breakers and Bulkheads; Degrade on Purpose; How One Slow Thing Kills Everything; Finding Out, and Closing the Loop

Open the last endpoint you shipped. Count the lines that run when everything works. Now count the lines that run when the payment provider takes nine seconds to answer instead of two hundred milliseconds. If that second number is zero, you didn't ship a feature. You shipped a demo that happens to have users.

The happy path is one branch. Production is a tree. The request arrives twice. The token expired four seconds ago. The third party API returns an HTML login page instead of JSON because someone rotated a key at 4 PM on a Friday. That's not exotic. That's a Tuesday.

So design backwards. Before you write the success case, write the failure list. For every function that leaves your process - network, disk, another service - write three lines first. Slow. Down. Wrong. One sentence each. If you can't answer "down," you don't understand the feature yet, and neither does the assistant writing it for you.

Which brings up why generated code is so optimistic. Ask an AI for a function that charges a card and you get clean async code, tidy naming, maybe a try block with a console.error inside. That's not a knock on the model. Blog posts, README snippets and course examples strip error handling out on purpose, because it distracts from the lesson. The model is a mirror of people who were explaining, not operating. So hunt one pattern in review: the catch block that only logs. Grep for catch blocks shorter than three lines. Most of them are lies. You logged the error, kept going, and now you've got a half-finished operation and a 200 response.

Before you handle anything, classify it. Three axes. First, expected versus unexpected. Insufficient funds is a business rule and should never wake anyone up. Reading a property of undefined is a bug and belongs in your error tracker. Second, transient versus permanent. A 503 might work in two seconds; a 404 never will. Third, whose fault - a spike in 400s means someone shipped a bad client, a spike in 500s means you shipped a bad you. Put "retryable" and "expected" on the error object itself, so your retry layer, logger and alerting read the same flag instead of pattern matching on the string "ECONNRESET" in nine different files. And a strong opinion: return your domain failures, don't throw them. A return value forces the caller to look. An exception lets them pretend it doesn't exist.

Now the expensive one. Your checkout charges the card, writes the order, decrements inventory. Step one succeeds, step two throws. Customer has a charge and no order, and support finds out before monitoring does. That's partial failure, and it doesn't look broken - there's no red graph, just wrong data someone discovers three weeks later. Two rules. Order operations so the reversible ones happen first and the irreversible one happens last: pending order, then charge, then mark paid. And never hold a database transaction open across a network call. You can't roll back Stripe, and you're holding row locks for two and a half seconds while your connection pool builds a queue it'll never drain. […]

### Testing: Confidence, Not Coverage

Tests that buy confidence instead of coverage numbers - what to test, what to skip, and why.

Chapters: Tests Are a Change Detector, Not a Proof; The Pyramid Is an Economics Argument; Unit Tests: Test the Behavior, Not the Function; Integration Tests: The Seams Are Where Bugs Live; End-to-End: Five Flows, Ten Minutes, No Sleeps; Test Doubles: Fake the Boundary, Nothing Else; Property-Based Testing: Let the Machine Find It; Contract Testing: Stop Finding Out From Customers; TDD as a Design Tool, Used Selectively; Reviewing AI-Generated Tests; Determinism, Data, and the Flaky Test Policy; Coverage Lies. Mutation Testing Doesn't.

A test suite is not proof your code works. It can't be. Proving correctness for anything real is a math problem nobody solves on a Tuesday afternoon. What your tests actually are is a change detector. The monitor beeping next to the hospital bed. It doesn't cure anything. It tells you, in seconds, that something which used to be true stopped being true.

That reframe changes what you write. You stop asking "did I cover this file" and start asking "what change would scare me here, and would anything scream if I got it wrong?"

This matters more for you than for someone who learned testing in 2011. You generate code faster than you build understanding of it. That's the deal you made with the AI, and it's mostly a good deal. But it means the amount of code in your repo that nobody on earth fully understands grows every week. Untested code isn't faster. It's borrowing speed at a brutal rate: save two hours today, pay four hours a month for a year in manual clicking, reverts, and hesitation before every merge. I've watched three-month-old products hit the ceiling where changing one thing breaks two things you forgot existed.

So the operating rule. Before you write a test, write the sentence it protects, in the words a user would use. "An expired token is rejected." "Archiving a task removes it from the dashboard count." If you can't write that sentence, you're about to test code shape instead of behavior, and that test will cost you more than it gives.

Now, what kind of test. The pyramid isn't handed down from heaven, it's an economics argument. A unit test runs in single-digit milliseconds and points at exactly one function when it fails. An integration test against a real database takes tens to hundreds of milliseconds and gives you two or three suspects. An end-to-end test driving a browser takes five to sixty seconds, and when it fails you're watching a video replay wondering whether your code broke or the test just clicked too early. Multiply by how often each runs. Unit on every keystroke, end-to-end on merge. That ratio is the whole argument.

Here's the default I'd pick. If your complexity lives in domain rules - pricing, permissions, scheduling, state machines - weight the base with unit tests. If your complexity lives in wiring, request in, database out, third party in the middle, weight the middle with integration tests. Most products you'll build are the second kind. Most production bugs don't live inside functions. They live at the seams. The query that passes every unit test and deadlocks against real Postgres. The vendor who returns a number as a string. Your unit tests can't see any of that, by construction, because you deleted the thing that breaks.

So adopt this: every endpoint that writes data gets one integration test hitting the real handler and a disposable database container, asserting on what's actually in the row, not what the response body claims. Those two disagree more than you'd think. Cap end-to-end at five to seven flows under ten minutes, and if a flow fails twice in a month for something that isn't a real bug, delete it and push that behavior down a layer. […]

### Observability: Logs, Metrics & Traces

Logs, metrics, and traces that answer 'what broke at 3 AM' before your users do.

Chapters: Flying Blind: What Observability Actually Means; Cardinality: The Number That Decides Your Bill; Structured Logs: Events, Not Sentences; Levels Are a Contract, and Secrets Are Forever; Correlation IDs: Stitching One Request Together; Aggregation and Retention: Where the Money Goes; Metrics: Counters, Gauges, and Why Averages Lie; The Four Golden Signals and Their Sharp Edges; Distributed Tracing and the Sampling Decision; OpenTelemetry: Instrument Once, Export Anywhere; Dashboards That Answer Questions; Alerting Without Burning Your On-Call; SLIs, SLOs and the Error Budget; Real Users and Robots: RUM and Synthetics; The Economics of Watching Your Own System

Monitoring answers questions you wrote down in advance. Is CPU over eighty percent. Is the error rate over one percent. Observability lets you ask a question nobody anticipated, at 3 AM, without shipping code. Why are signups from Brazil failing only for users paying with a saved card, and only since Tuesday.

That's the test. If answering a new question means adding a log line, deploying, and waiting for the bug to happen again, you don't have observability. You have a printf habit with extra steps.

This matters more for you than for most engineers, because AI writes code that runs. It doesn't write code that explains itself. I've reviewed a pile of AI-scaffolded services and I have never once seen a model add a latency histogram unprompted. So build the habit: when you write a feature, write down the question you'd ask if it broke at 3 AM for one customer in one region. Check whether a query could answer it today. If it can't, add the field before you merge. Instrumentation is part of the feature, same as the error handling from Module 17.

Now the number that decides whether your setup scales or bankrupts you: cardinality. That's how many unique values a dimension can take. A metrics backend stores one time series per unique combination of labels. Status code has six values. Endpoint has forty. That's 240 series, fine. Add user ID with two hundred thousand users and you've asked your database for forty eight million series. Prometheus eats your RAM and dies. A hosted vendor bills you per series, which is the more expensive kind of dying. The classic trip-up is labeling a metric with the raw request path, slash orders slash 8832, one unique value per order in your database. Use the route template. Your AI assistant will happily write the bad version.

So the rule: every metric label gets a bounded set of values, and you should be able to say the bound out loud. Status, method, route template, region, environment. If you can't name the maximum, it doesn't go on a metric. User IDs, order IDs, full URLs, email addresses - those go on log lines and span attributes, because events don't multiply. That's the real dividing line between the pillars, and almost nobody explains it that way.

Which brings us to logs. Stop writing sentences. "Error processing payment" is a clue. An event named payment dot failed with user ID, amount as a number, currency, provider, error code, request ID, and duration in milliseconds is an answer. Here's your test: if answering an operational question needs a regex against your log text, the thing you regexed should have been a field. Every string interpolation destroys data. Forty nine ninety nine baked into a sentence can't be summed.

Two habits that pay forever. Name events like a schema, not like prose, so a refactor doesn't break every saved query. And log the duration on anything that crosses a network or touches disk. Eight bytes, and it's the field you'll use most. […]

## Part 6: Infrastructure & Deployment

What your code stands on: what containers actually solve, cloud concepts that outlive any console redesign, and pipelines that block bad deploys.

### Containers: Why, Not How

What Docker actually solves: images vs containers, and why 'works in the container' finally means something.

Chapters: The Dockerfile You Copy-Pasted; It's a Process, Not a Machine; Layers and Why Your Build Takes Eleven Minutes; Images That Belong in Production; Networking: localhost Is Lying to You; Ephemeral by Design, and Where Your Data Goes; Compose: The Whole Stack as One File; Registries, Tags, and Why latest Is a Loaded Gun; Limits, Throttles, and the OOM Killer; The Thin Wall: Container Security; When Not to Containerize; Outgrowing One Host

A container freezes your userland. Nothing else. That one sentence explains most of the container pain you've had and most of the pain coming.

Here's what a container is. Your app, its runtime, its system libraries, its config, packaged as one artifact that runs the same on any machine with a container runtime. Everything above the kernel travels with your code.

Which means everything below it doesn't. DNS resolvers differ between your laptop and your VPC. Cloud IAM roles exist in staging and not in production. Your managed database is on a different major version. And CPU architecture. You build on an M-series Mac, you get an arm64 image, you push it, and production, which is amd64, answers with exec format error. Your image was perfectly reproducible. It was reproducibly wrong. Set your build platform explicitly today and that trap never gets you.

Now kill a sentence you've probably said: a container is a lightweight VM. It isn't. A VM virtualizes hardware and boots its own kernel. A container is a normal Linux process on the host's kernel wearing a costume, stitched from namespaces for its own view of the world, cgroups to cap what it consumes, and a union filesystem so fifty containers share one copy of the same base.

That's not trivia. It decides how you debug. Your app runs as PID 1, and in Linux, PID 1 doesn't get the default signal handlers. If your process doesn't explicitly handle SIGTERM, it ignores it. Docker stop sends SIGTERM, nothing happens, and ten seconds later the runtime sends SIGKILL. Every graceful shutdown you wrote, draining connections, flushing buffers, never runs. Go time a docker stop with a stopwatch. Full ten seconds means your shutdown path has never once executed. Fix it two ways: exec form of CMD, so your binary is actually PID 1, and run with an init that forwards signals.

Layers next, because that's where your eleven minute build lives. An image is an ordered stack of filesystem diffs. The builder computes a cache key per instruction, and if one key misses, every layer after it rebuilds. Not some. All of them. So copy your dependency manifest first, install, then copy source. Do it backwards and every commit invalidates your install layer. Reversing three lines gets those minutes back.

Two sharp edges there. For RUN, the cache key is just the command string, so apt-get update caches forever against an index that went stale six months ago. That's why update and install always live in one RUN. And deleting a file in a later layer never removes its bytes. The password you added then deleted is still sitting there, readable by anyone with docker history and tar. […]

### Cloud Infrastructure: Concepts Over Console Clicks

The managed-services decision tree - cloud concepts that outlive any console redesign.

Chapters: The Cloud Is a Landlord, Not Magic; The Compute Spectrum: VMs, Containers, Functions; Storage: Block, Object, and the One You Should Default To; VPCs and Cloud Networking: Why Your Deploy Times Out; IAM: The Blast Radius You Choose in Advance; Regions and Availability Zones: Geography Is Latency; Managed Services: Renting Someone Else's On-Call; Serverless, Honestly: Limits, Cold Starts, and the Memory Dial; Edge Computing: The Thin Layer at the Perimeter; Cloud Cost: Where the Bill Actually Comes From; Infrastructure as Code: Clicking Is Debt With Interest; The Defaults, and What Every Abstraction Costs You

Every abstraction you buy in the cloud takes something from you. If you can't name what it took, you're not making an engineering decision. You're following a tutorial and hoping.

Start with what the cloud actually is. It's a rental business. Somebody bought thousands of servers, racked them in a warehouse with industrial cooling and diesel generators, and sells you slices through an API. A user taps your app in Tel Aviv, that packet crosses fiber and probably an undersea cable, lands in a building with a street address, hits a rack of forty machines, and one of those machines runs a hypervisor that carves it into slices. Your code shares a CPU with strangers. Every layer is maintained by a human and every layer can fail.

The one sentence this whole module hangs on: the shared responsibility model. The provider secures the infrastructure of the cloud. Buildings, hypervisor, disks, network fabric. You secure everything you build in the cloud. Your packages, your bucket policy, your permissions, your keys. And that line moves with every service. On a virtual machine you patch the kernel. On managed Postgres they patch the engine and you still own the credentials. On object storage they own everything except one checkbox that decides who on earth can read your files. For every service you run, answer one question: if this breaks at three in the morning, is it my pager or theirs?

Now compute. Three rungs. First, virtual machines: full OS, SSH key, you patch it, you pay per hour whether the CPU is pinned or asleep. Second, containers on a managed runtime: you hand over an image, they do placement, health checks, rolling restarts. Third, serverless: you hand over a function, they own everything, you pay per millisecond.

Here's the arithmetic nobody does. A three hundred millisecond function at ten million requests a month costs about twenty-seven dollars. Beautiful. Same function at five hundred million requests is around thirteen hundred fifty. Two mid-sized containers carrying that load around the clock cost maybe a hundred fifty. The crossover happened somewhere in between and nobody emailed you about it. So pick from your utilization curve, not your feelings. Spiky and event-driven goes serverless. Steady traffic above roughly forty percent utilization goes containers. Raw VMs only for GPUs, kernel modules, or licensed software. My default is the middle rung, because a container image runs on your laptop, on a managed service, on Kubernetes, on a box in a closet. When you outgrow serverless, you rewrite.

Storage. Block storage is a virtual hard drive bolted to one machine in one availability zone. Object storage is a flat namespace, key to blob, over HTTP, and those folders in the console are a fiction rendered from slashes in your key names. Object storage is your default for uploads, backups, logs, anything static. Store the key in your database, never the bytes. Use presigned URLs so the browser uploads straight to the bucket and your server never touches the file. Turn on versioning, because eleven nines of durability protects you from a dying disk, not from your own delete call. […]

### CI/CD: The Pipeline That Protects You

The pipeline that blocks bad deploys before your users find them.

Chapters: Integration Hell and the Tax You're Already Paying; Delivery vs Deployment: Where Your Team Actually Belongs; Anatomy of a Pipeline: Build Once, Promote Everywhere; Runners: The Machines Your Pipeline Actually Runs On; Environments, Promotion Gates, and Why Staging Lies; Preview Environments: Every Pull Request Gets a World; Deployment Strategies: Rolling, Blue Green, Canary; Feature Flags: Deploy Is Not Release; Zero Downtime: The Four Mechanics; Rollbacks: Your Most Important Deployment; Pipeline Security: Secrets, Scanning, Supply Chain; Speed Is a Feature

Continuous integration was never invented to save you time. It was invented because merging hurt. In the nineties, teams worked on separate branches for weeks and then spent an entire week merging them back. "Integration engineer" was a real job title.

The math is the part people miss. The cost of merging doesn't scale with the size of your change. It scales with your change multiplied by everything that moved on main while you were gone. Two days of divergence is one conflict in a config file. Three weeks is archaeology. And the conflicts git shows you are the easy ones. The scary conflicts are semantic: your code compiles, their code compiles, both merge clean, and the behavior is wrong because you both changed assumptions about the same function. Git has no opinion about that. Your tests probably don't either.

So there's a gap between doing CI and practicing CI. Running tests in a pipeline is doing CI. If your branch has been alive three weeks with a green checkmark on it, you've integrated nothing. You automated the wrong half. The other half is cultural: a red main branch stops the line. Like the cord in a factory that anyone can pull. Nobody merges on top of red, and whoever broke it reverts within ten minutes. Reverting isn't an insult. It's the default move.

Your rule: branch lifetime under twenty four hours. A two week feature merges eight times behind a feature flag, dark, flag off. Which is the whole point of flags. Deploying code and releasing a feature are two separate events. Merge unfinished work daily, nobody sees it, then turn release into a dial: staff, one percent, ten percent, watching error rates. Something's wrong at one percent, you flip it off. Seconds, no rebuild, no pipeline. Just give every temporary flag an owner and an expiry date, and fail the build when it expires. Ten live flags is a thousand combinations nobody will ever test, and stale flags cause outages.

Now the pipeline itself. Lint, test, build, scan, deploy, verify. That order is sorted by how fast a stage fails and how often it does, because nobody should wait twelve minutes for an integration suite to find an unused import. And the rule that separates real pipelines from tutorial ones: build the artifact once, promote the same bytes everywhere. One container image, identified by its content digest, not a tag like latest that anyone can move. Rebuild per environment and you've invented a new bug class where the thing you tested isn't the thing you shipped. A dependency published a patch between builds. Staging is green, production's on fire, and the git commit is identical.

Add a verify stage after deploy. Most pipelines end at "deployment succeeded," which only means the orchestrator accepted your request. Hit four real endpoints and fail loudly. That's the difference between deployed and working. […]

## Part 7: Architecture & Scale

Designing for growth: monolith-first architecture, queues, caching, profiling before optimizing, and distributed-systems intuition.

### Architecture Patterns: Monoliths, Services & the Middle Ground

Monolith-first thinking, when services earn their complexity, and the middle ground nobody markets.

Chapters: Your Architecture Is Your Org Chart; Four Forces and the Door You Can't Walk Back Through; The Monolith Deserves More Respect Than You Give It; What a Good Monolith Looks Like Inside; When the Monolith Actually Cracks; Microservices Buy Autonomy, Not Speed; The Distributed Systems Tax; The Middle Ground Where Most Teams Actually Live; Domain-Driven Design: Where To Actually Cut; Who Owns the Data; Multi-Tenancy: One System, Many Customers; The Front Door: API Gateway and Backend-for-Frontend; The Strangler Fig: Migrating Without Betting the Company

The strongest force shaping your architecture isn't your language, your cloud, or your framework. It's your org chart.

Melvin Conway wrote this down in 1967: organizations design systems that copy their own communication structure. People quote it like a fortune cookie. Treat it like gravity. Four teams, four things, and the seams land exactly where humans stop talking to each other. Watch it happen in slow motion: two teams share a checkout module, every change needs the other team's review, so someone adds a config flag, then a plugin hook, then a whole separate code path. They built an interface at a team boundary and never called it architecture.

Run it backwards and it becomes a lever. Pick the architecture you want, then shape teams to match. Amazon didn't get service-oriented architecture from diagrams. They got it by making small teams own things end to end and forbidding them from touching each other's data.

So before you draw a single box, draw the humans. Who owns what, who carries the pager, who has to be in the room for a release. Any service owned by two teams is the worst of both worlds - you pay the network cost and you still need a meeting to ship. Move the ownership first. The boundary gets obvious.

Now your default. A monolith is one deployable unit containing all your functionality. That's the whole definition, no shame attached. The internet spent a decade calling it training wheels, and that framing has burned more engineering years than any other idea in our field.

Look at the numbers instead. An in-process function call costs single-digit nanoseconds. The same call over the network, same data center, with serialization and TLS, costs half a millisecond to two milliseconds when everything's healthy. That's a hundred thousand times slower. You turned a free operation into a budget item. In a monolith, updating an order, decrementing inventory and writing a payment record commit together or not at all. Split them and you're writing a saga with compensating actions, plus a Tuesday afternoon where someone got charged for an item that no longer exists.

And here's what surprises people: monolith describes your deployment unit, not your capacity. Run forty copies behind a load balancer. Stack Overflow served the programming questions of the entire internet from a handful of web servers running one .NET app. Shopify runs one of the biggest Rails codebases alive as a modular monolith, deliberately. Scaling out and splitting up are different decisions. Conflating them is the most common mistake I see in architecture reviews.

A good monolith has structure inside. One folder per business domain, each with exactly one public entry point, everything else private. Dependencies point one direction - infrastructure depends on domain, never the reverse. And the move nobody makes: enforce it at the database. Give each module its own schema and its own database role that can only touch its own tables. Now module A physically cannot select from module B's tables. That's what makes extraction cheap two years later, because untangling queries is the hard part, not moving code. […]

### Asynchronous Processing & Queues

Message queues - why every real system has one, and what happens when consumers fall behind.

Chapters: Four Seconds of Spinner: Why Sync Breaks; What Async Actually Costs You; Queues and Streams Are Not the Same Thing; Producers, Brokers, Consumers: The Moving Parts; Delivery Guarantees and the Exactly-Once Lie; Idempotency: The Concept That Makes Retries Safe; Background Jobs and Worker Design; Email Is a Pipeline, Not an API Call; File Pipelines and Notification Fanout; Scheduled Jobs Versus Event-Driven Work; Sagas: Transactions Across Services; The Transactional Outbox and the Dual Write Problem; Poison Messages and Dead Letter Queues; Backpressure, Consumer Lag, and Shedding Load

Here's the test that decides whether work belongs on the request path. If the user can't see the result on the next screen, and it doesn't change what they're allowed to do next, it comes off the request path. That's it. That one sentence rewrites most checkout handlers you've ever written.

Picture a Buy button that charges the card, writes the order, sends a receipt, renders a PDF, calls the warehouse, updates the search index, and awards loyalty points. Seven steps, four seconds of spinner. Slow is the least of your problems. When your mail vendor returns a 503 at three in the morning, your handler throws, the transaction rolls back, and now the card is charged and the order doesn't exist. You tied the correctness of a payment to the uptime of an email provider. That's insane, and it's the default shape of every app that grew organically.

So sort every operation into three buckets. First, must-be-synchronous: charge the card, check inventory, validate the coupon. Second, deferrable-but-soon: the receipt, the analytics event, the index update. Third, fully background: nightly reindex, monthly report. Move work off when it's slow, over roughly 200 milliseconds, when it talks to a system you don't control, or when it can retry without the user watching. Everything else stays inline where a stack trace still tells the whole story.

Because async costs you. Users now see windows where the screen and the database disagree, so show a processing state instead of pretending. Debugging gets worse, so generate a correlation ID at the edge, put it in the message envelope, and log it in every worker. Without it you're grepping timestamps like it's 2009. And start with two queues, not twelve. Urgent and bulk. Twelve queues on day one becomes a config file nobody understands by March, with four empty and one silently backed up.

Now, queues and streams are different things, and picking wrong costs you a rewrite. A queue is point to point: one consumer takes the message, acknowledges, and it's gone. A stream is an append-only log; consumers each track their own position, and five teams can read the same event. Queues forget. Streams remember. The naming test decides it. Imperative commands like SendReceiptEmail or ChargeCard go on a queue. Past-tense facts like OrderPlaced go on a stream. Publish a command dressed up as an event and someone wires a marketing tool to your signup event, and new users get two welcome emails for a month before anyone notices.

Default to a queue. Honestly, under a few thousand jobs a minute, a Postgres table with SELECT FOR UPDATE SKIP LOCKED beats running Kafka, and nobody gets paged for a broker. I've watched three-person teams operate a Kafka cluster for one topic. Don't be that team. […]

### Caching Strategies

Redis, TTLs, and cache invalidation - famously one of the two hard problems in computer science.

Chapters: The Bargain: You're Trading Truth for Speed; Where Caches Actually Live; HTTP Caching, Part One: The Free Infrastructure; HTTP Caching, Part Two: ETags, Vary, and Cache Busting; Application Patterns: Cache-Aside and Its Cousins; Eviction: What to Forget, and How Big to Get; Invalidation: Make the Lie Expire; Redis Does More Than get and set; The Four Ways Your Cache Kills Your Database; Cold Starts and Multi-Tier Caching; What You Must Never Cache; The Decision Framework You'll Actually Use

Every cached value is a copy, and every copy can become a lie. That's the trade you're making when someone says "just add Redis." You're swapping correctness for speed, and the engineering is in making the lies small, short, and survivable.

A cache is a copy of a result, stored closer to whoever needs it, so you don't pay to compute it twice. Your CPU has done this your whole career. L1 read, about a nanosecond. Main memory, a hundred. A database query across the network, a millisecond or three on a good day.

Now the arithmetic that decides whether your cache earns its keep. A hundred thousand requests a second. At a ninety-nine percent hit rate, a thousand reach your database. At ninety-five percent, five thousand do. Same cache, same code, five times the load on the hardest thing you own to scale. That's why hit ratio goes on the dashboard first, not last. You'll never feel that difference in staging, because staging has four users and one of them is you.

So before any cache ships, answer four things in writing. What's the origin of truth. What's the TTL. What invalidates the entry early. And what happens when the cache is empty or completely down. Can't answer all four? You're not adding a cache. You're adding an outage with a better p50.

Ask a vibe coder where the cache lives and they point at Redis. Ask a request, and it passed through five before it reached your code: the browser, the CDN, a reverse proxy, your app memory plus Redis, and the database itself, which caches whether you asked or not. Each layer removes a different cost, and that's what changes your decisions. Browser caching removes the request entirely. CDN caching removes distance, and distance is physics. You can't code your way out of a hundred and forty millisecond round trip to Sydney, so a bigger Redis does exactly nothing for that user.

Fix the layer closest to the user first. It's the cheapest and it removes work from every layer behind it. The free version lives in HTTP, in one header, and most people shipping with AI have never set it on purpose. Content-hashed assets like app dot a3f9c2 dot js get public, max-age one year, immutable. HTML gets no-cache, which does not mean don't cache, it means store it and revalidate. The one that means don't store is no-store. Public API responses get a short s-maxage plus a generous stale-while-revalidate, so users never wait on a miss.

Inside your code, default to cache-aside. Check the cache, miss, hit the database, write it back with a TTL. And on write, delete the entry, never update it. Updating looks smarter and creates a race where two writes land out of order and your cache holds a value the database never had. Deletion just makes the next reader go get the truth.

Your keys are a contract. Namespace, entity, id, and a schema version: prod colon user colon forty-two colon v3. The day you add a field, old entries deserialize into garbage, and rolling back your code doesn't fix it because the poison is in the cache. Bumping the version retires every stale shape instantly. […]

### Performance, Profiling & Scaling

Profile before you optimize: finding the real bottleneck instead of the one you guessed.

Chapters: You're Guessing, And Guessing Is Expensive; The Four Kinds Of Slow; Percentiles, Budgets, And The Loop; Reading A Flame Graph Without Pretending; Memory Leaks, GC Pressure, And The Sawtooth; Where The Milliseconds Actually Go; The Database Is The Bottleneck (It Usually Is); Frontend: Measuring What Users Feel; Load Testing: Breaking It On Purpose; Bigger Machine Or More Machines; Auto-scaling That Actually Arrives In Time; Rate Limiting, Throttling, And Load Shedding; Global Distribution And The Speed Of Light; Code, Hardware, Or Architecture: Pick The Lever

At any given moment, exactly one thing is making your system slow. Not five things. One. Everything else on your performance to-do list is noise wearing a costume.

That's Amdahl's law in plain language. The speedup you get from fixing one piece is capped by how much of the total time that piece takes. If that ugly serialization function you've been meaning to rewrite since March is three percent of your response time, deleting it entirely buys you three percent. Two days of work for a rounding error. I've done exactly this. Guessing feels like progress and measuring feels like procrastination, so everybody guesses.

Flip it around and the same law pays you. If seventy three percent of your request is database time, cutting that in half is a thirty six percent win from one afternoon. Same effort. Wildly different return. The only difference is that you looked first.

So here's the cheapest rule in this module: no performance change gets merged without two numbers in the description. What it was before, what it is after, measured the same way under the same load. No two numbers, no fix. You have a hypothesis with a confidence problem.

Before you open any tool, classify the slowness. There are four kinds, and they look nothing alike. First, CPU-bound: cores pinned, latency climbing with traffic. On July second, 2019, one bad regular expression in a Cloudflare firewall rule pinned the CPU on every core handling HTTP traffic worldwide and took them down for twenty seven minutes. One line of code. Second, memory-bound: resident memory climbing, garbage collection eating more and more wall clock, then the kernel kills you. Third, I/O-bound, which is most web services. CPU at twelve percent while requests take eight hundred milliseconds, because your code spent seven hundred and forty of them asleep, waiting on a database or somebody else's API. And fourth, contention-bound: requests waiting on each other. A lock, a hot row, a connection pool with nobody free.

Contention is the one that fools people, because on every dashboard it looks identical to I/O. Low CPU, fine memory, terrible latency. The free test is doubling the load. CPU-bound, utilization rises with latency. I/O-bound, latency stays flat until the downstream saturates. Contention-bound, latency doubles while every resource graph stays boring, and adding servers makes it worse. Sixty seconds of looking at utilization per resource saves you a week of using the wrong tool.

Now stop reading averages. A service averaging a hundred and twenty milliseconds can have a p99 of four seconds. And do this arithmetic once: if your page makes thirty backend calls and each has a one percent chance of being slow, roughly twenty six percent of page loads hit a slow call. Your p99 isn't an edge case. At real fan-out, your p99 is your product.

Write a budget with actual numbers. P95 under two hundred milliseconds for reads. Largest Contentful Paint under two and a half seconds at the seventy fifth percentile of real users. A budget nobody enforces is a wish with a spreadsheet, so it has to fail a build, not decorate a dashboard. […]

### Distributed Systems Intuition

The fallacies of distributed computing - and the intuition to spot them in your own design.

Chapters: One Machine Was Lying To You; The Eight Fallacies, Used As A Checklist; Partial Failure: Nobody Crashed, Everything Is Wrong; Clocks Lie, And Nobody Logs It; Causality: Ordering Without Trusting Time; FLP, CAP, And The Version Of CAP You Should Actually Use; Consensus, Quorums, And Fencing Tokens; The Consistency Spectrum And What Users Actually Notice; Replication: Three Architectures, One Default; Partitions And Split Brain; Telling Slow From Dead; The Reflex: Reading Any Architecture Diagram

A timeout does not mean the call failed. It means you don't know. That single sentence is most of what separates an engineer who can design across machines from one who can't.

Think about what happens when you send a request and nothing comes back. Four different things could have happened. The request never arrived. It arrived and the server is still grinding. The server crashed halfway. Or the server did the work perfectly and the reply got lost coming home. Four worlds, one observation: silence. Sit with that last one, because that's why your retry charged the customer twice.

On one machine you got four gifts for free. Shared memory, so a variable is the same variable everywhere. One clock, so this happened before that, no argument. Total failure, where the process is up or it's dead, all of it. And calls that cost nothing. Cross a network and you lose all four at once. Every node has its own memory, its own clock, its own picture of the world, and the only way two nodes learn anything about each other is a message the network is free to delay, reorder, duplicate, or silently drop. No apology. No log line.

So the operating rule that carries this whole module: for every outbound call in your codebase, you owe a written answer to one question. What do you do when it times out? The honest answer is never "it failed." It's "I don't know," and the only way to make "I don't know" survivable is to make the operation safe to repeat. That's what idempotency keys back in Module 12 were buying you.

Now clocks. Ask your laptop the time and it answers instantly and confidently, and that confidence is misplaced. Crystals drift by tens of parts per million. NTP corrections can jump time forward, and they can jump it backward. Two cloud machines disagreeing by tens of milliseconds is routine. Which means two things you should change tomorrow. First, never subtract two wall clock readings to measure elapsed time - an NTP correction in the middle hands you a request that took minus three hundred milliseconds. Use the monotonic clock your language already ships: time dot monotonic, performance dot now. That's not a tradeoff, it's just correct. Second, last-write-wins conflict resolution keyed on timestamps means the server with the fastest clock always wins and the other write is deleted. Not conflicted. Deleted, with no error anywhere. Google solved this honestly with Spanner, and the price tells you everything: GPS receivers and atomic clocks in every datacenter, plus a deliberate wait on every commit. They bought hardware and then paid again in latency to make time trustworthy. […]

## Part 8: The Business Layer

The code that pays the bills: payments and compliance, metrics that actually predict revenue, and attribution without lying to yourself.

### Payments, Subscriptions & Compliance

Payment webhooks that survive retries, subscription state machines, and the compliance basics you can't skip.

Chapters: Money Doesn't Retry; Authorize, Capture, Settle; Card Data Is Radioactive; Picking a Provider Without Marrying One; The Subscription State Machine; Dunning, Proration and Usage; Marketplaces and Other People's Money; Fraud Is a Dial, Not a Switch; Chargebacks Cost More Than the Sale; Currency, Local Methods and Tax; Webhooks Without Duplicate Charges; Reconciliation, or How You Find Out You're Wrong

A dropped analytics event costs you nothing. A dropped charge costs you a customer staring at their banking app wondering where their money went. That's the whole module in one sentence: money has no undo button, so every design choice you make around it has to assume you'll be wrong at least once.

Start with the thing most code gets wrong on day one. A card payment isn't one event. It's three. Authorization asks the issuing bank to reserve an amount, and no money moves. Capture claims that reserved money, and now it's real debt. Settlement is the overnight batch that actually lands cash in your bank, two days later if you're lucky, seven if you're new or in a risky category.

So your database must never have a boolean called paid. It needs a status field with explicit states - requires action, authorized, captured, settled, refunded, disputed - and a timestamp on every transition. At 2 AM the only question anybody asks is "where in the lifecycle is this transaction," and a boolean can't answer that.

Two decisions fall straight out of that. Digital goods, SaaS, anything instant: authorize and capture in one call. Physical goods you ship later: authorize at checkout, capture when the package leaves. And know the difference between a void and a refund. A void cancels an authorization before settlement and it's free. A refund reverses a settled charge, and you usually eat the original processing fee on top of giving the money back. Customer cancels four hours in and you haven't captured? Void. That's free money you're leaving on the table otherwise.

Next: card data is radioactive. Never let a raw card number touch your servers, your logs, or your error tracker. PCI compliance isn't really about the document, it's about scope - how much of your infrastructure gets audited - and scope is decided by how the data flows, not by how careful you feel. The default a principal engineer picks is provider-hosted fields inside an iframe. Your CSS, their input, the number goes browser-to-provider, you get back a token. Taking raw cards through your own API means quarterly scans, penetration tests, network segmentation, and an on-site assessor. I've seen two companies where that was justified and both had thirty people on payments.

And be paranoid about the page around the form. British Airways in 2018 had a compliant card field. Attackers injected a script into the surrounding page, skimmed more than four hundred thousand customers, and the UK regulator fined them twenty million pounds. Put a content security policy on your checkout that whitelists your provider's domain and nothing else.

Now subscriptions, and the mistake in almost every AI-generated billing implementation I've read. The code checks whether status equals active to decide access. Wrong. Entitlement is a timestamp, not a status string. Use one column, access_until. Payment succeeds, you push it forward. Customer cancels, you leave it alone so they keep what they paid for. Your feature gate compares one field to now. When your provider has a bad hour and webhooks stall, nobody gets locked out of software they already bought. […]

### Business Metrics & Product Analytics

The metrics that actually predict revenue - and the vanity ones that don't.

Chapters: Your Code Is the Measuring Instrument; The Metrics, Rewritten as Questions Your System Must Answer; Designing the Event Schema Before You Write a Single track() Call; Client or Server: Where the Event Is Born; Identity Resolution: One Human, Five User IDs; The Stack: Hosted, Product Analytics, or Your Own Warehouse; Pipelines: Getting Events There Without Losing or Cloning Them; Sessions and the Shape of a Retention Curve; Activation, Behavioral Segments, and Feature Adoption; Experiment Infrastructure: Assignment, Exposure, Sample Size; How Your Experiment Lies to You; Data Quality: Tests, Contracts, and One Definition of MAU; Privacy-Compliant Analytics: Consent as a System

Somebody at your company is looking at a chart right now, deciding whether to hire two more engineers. An analyst didn't draw that chart. You did, the moment you picked which line of code fires the tracking call.

That's the whole idea of this module. Measurement isn't a layer you paint on top of a working system. It's a property of the system, same as latency. The old split where engineers build and product people measure is exactly why so many dashboards are fiction.

Start with the failure mode you've probably already shipped. A signup event fired inside a React effect. Effect runs, event fires. Then React 18 mounts the component twice in development and fires it again. Or the user hits back, then forward, and it remounts. Your signup count is inflated and nobody can say by how much. Add ad blockers on top: on a developer-facing product, expect a quarter to a third of browser events to never arrive. No error thrown. The number is just quietly low. Forever.

So here's the rule. When someone asks you for a metric, don't ask what chart they want. Ask which row in which table proves it, which code path writes that row, and what happens to that row when the action fails halfway through. Can't answer those three? The metric doesn't exist yet. You've just agreed to draw a line.

Which sets up the first real decision: where is the event born, browser or backend? They're not interchangeable. The browser knows things your server never will - hovers, scroll depth, the four visits to pricing before signup - and it also loses events to closed tabs and tunnels and blockers. Your server knows what actually happened. The row committed. The charge cleared.

The operating rule: if a number could show up in a finance conversation, it fires server-side, after the source of truth commits. Purchases, upgrades, refunds, anything with a currency symbol. Optimistic UI makes this sharp. Showing "you're on Pro now" before the charge clears is good product design. Firing subscription_created at that moment is a lie, because expired cards and abandoned 3D Secure challenges end with a happy success screen and no money.

And "fire it server-side" hides a real problem. Your API can commit the transaction, then crash before the analytics call goes out. Use an outbox. Same database transaction that writes the subscription row also inserts a row into an analytics_outbox table. A worker drains it. Committed together or not at all. Each row carries a unique event id, so retries cost you bandwidth and nothing else.

Now, schema, and do this before the first track() call. Six months into a sloppy implementation you'll find button_clicked, Button_Clicked, and btn_click, and nobody alive knows if they're the same thing. That's data debt, and it's worse than code debt, because you can refactor code but you cannot go back and re-collect last quarter. Object first, action second, past tense, snake case. checkout_completed. invite_sent. Write it in a tracking plan that lives in your repo, generate a typed client from it, and fail CI on unregistered events. Thirty to fifty well-defined events beat four hundred auto-captured ones. […]

### User Acquisition, Ads & Attribution

UTMs, attribution windows, and ad math without lying to yourself.

Chapters: The Funnel Is a Data Pipeline, Not a Slide; Ad Auctions: You're Training Someone Else's Model; Attribution Models Are Queries, and Last Touch Lies; Incrementality: The Only Number That Survives an Audit; Click IDs, UTMs, and Capturing Them Before They Vanish; Server-Side Conversions, Hashing, and Deduplication; Signal Loss: Build for What You Own; Deep Links and Identity Resolution; SEO Is a Rendering Problem; Owned Channels: Deliverability Is Engineering; Landing Pages and the State Machine Behind Signup; Referrals, K-Factor, and Cycle Time; Ad Fraud and the Reconciliation Job

Marketing tells you the paid channel is dead. Spend went up, signups went flat, kill it. Nine times out of ten I've opened that up and the ads were fine. A redirect ate the query string. A consent banner blocked the tag. Acquisition problems wear a business hat, but underneath, most of them are pipeline problems, and you're the one who can fix them.

Start with the model. Your funnel isn't a triangle on a slide. It's a distributed data pipeline with six handoffs: impression, click, landing, signup, activation, payment. Different owners, different protocols, and nobody owns the seams.

The economics decide where you spend your week. Ten thousand dollars buys five hundred thousand impressions. One percent click through gives you five thousand clicks at two dollars. Twenty percent become signups, ten percent of those pay. A hundred customers, a hundred dollars each. Now push click through from one percent to one point two. Cost per acquisition drops to eighty-three dollars. Grind out the same relative lift at checkout instead, ten to eleven percent, and you only get to ninety-one. Harder work, less money, because every stage multiplies and a gain at the top gets multiplied by everything below it. Go find the leaks at the top of the pipe before you polish the checkout button.

Here's the habit. Every transition gets a counter on both sides. Clicks the platform reports versus landings in your server logs. Signups attempted versus signups committed. Disagreement over about five percent is a leak, not a marketing trend. Write that query before the campaign launches, because retrofitting instrumentation means throwing away your first month of data.

Now the thing that changes how you think about ad platforms. You are not buying impressions. You're training somebody else's model, and your conversion events are the training labels. Automated bidding means you don't pick who sees the ad. You send conversions, the model learns what those people look like, and hunts for more. Noisy labels, noisy audiences. No creative work fixes a poisoned label stream. So send the deepest event that still has volume, ideally "first payment succeeded," and pass the actual revenue value with it. An event worth four dollars and one worth four hundred should not train the model identically. But these models need roughly fifty conversions per ad set per week to leave their learning phase. Below that they're guessing, so send a mid-funnel proxy with revenue attached instead.

Attribution. Monday she sees your ad on her phone. Tuesday she clicks a post on her laptop. Wednesday she types your name in and buys. Who gets credit? There's no true answer. Attribution is a credit assignment rule you choose, not a fact you discover.

Last touch is the default in nearly every tool and it systematically over-credits whatever sits closest to the purchase. Branded search always looks incredible, because it intercepts demand something else created. I watched a team triple their branded search budget off a twelve dollar CPA. They were paying to buy back people already typing their name. […]

## Part 9: AI-Powered Products

Shipping intelligence responsibly: how models learn, embeddings and RAG, and production AI features with evals and cost budgets.

### Machine Learning Fundamentals: How Models Learn

How models actually learn - enough to not be fooled by demos or vendors.

Chapters: Rules In, Answers Out - and the Inversion; Three Kinds of Examples: Supervised, Unsupervised, Reinforcement; Features, Labels, and Why the Data Is the Project; Leakage: The Mistake That Looks Like Success; Train, Validation, Test - the Split That Keeps You Honest; Overfitting and Underfitting in Thirty Seconds; Bias, Variance, and Why Bigger Isn't Better; The Model Families and Your Default Stack; Accuracy Is Lying to You; The Rest of the Report Card - and What Each Number Hides; When Not to Use Machine Learning; Five Questions That Make You the Adult in the Room

Every line of code you've written follows one contract. You write the rule, the machine obeys. Income over eighty thousand, debt ratio under zero point four, approve the loan. Something breaks, you open the file, you find the line, you fix it. That contract is the only reason debugging works.

Machine learning rips it up. You hand a program ten thousand old loan applications with the outcome attached, paid back or defaulted, and it works out the rule itself. What comes back isn't code. It's a file full of numbers.

So kill the mysticism now. There's no thinking in there. Learning is a marketing word for fitting. The program starts with random numbers, guesses, measures how wrong it was, nudges every number toward less wrong, and does that a few million times. Warmer, colder, warmer. That's it, from a straight line through five points to the thing autocompleting your React at three in the morning.

Picture a cook who's never read a recipe but has tasted ten thousand bowls of soup and can reproduce the taste exactly. Ask what's in it, he shrugs. He can only make the soup.

Here's the sentence everything hangs on: a model is only as good as the examples you showed it. It has no judgment. It doesn't know your data collection was sloppy or that one column got backfilled last March. It finds whatever separates your examples with the least effort, and it commits, with total confidence.

Every disaster in machine learning is a footnote to that sentence.

Start with the shapes. Supervised learning means examples where somebody already knows the answer. Spam or not spam. Houses with the price they sold for. That's the workhorse; if a company makes money off machine learning and it isn't a chatbot, it's almost certainly this. Unsupervised learning is a pile of data with no answer key, and because there's no right answer, there's no honest way to be wrong. I watched a team present six beautifully colored customer segments to a board, and not one person could say what would have made those segments incorrect. Clustering gives you hypotheses, not conclusions. Reinforcement learning needs a simulator or cheap failures, and most products have neither. The famous flop is a boat racing agent that learned to spin in circles collecting pickups instead of finishing the race. If you write the reward, mean it literally.

Now the expensive part. Data leakage is when a feature contains information you wouldn't actually have at the moment of prediction. It never looks like a mistake. It looks like success. You're predicting fraud and one feature is chargeback_flag, which the fraud team sets weeks later, after the fraud. Your model reads the answer off the back of the card. Hospitals hit the same thing predicting sepsis using "patient is on antibiotics," prescribed because a doctor already suspected sepsis. Shuffle a year of transactions randomly and your model trains on February and gets tested on January. It has seen the future. […]

### Adding Intelligence to Your App

Embeddings, RAG, and picking the right model for the job instead of the loudest one.

Chapters: You're Renting a Model, Not Hiring One; The Cheapest AI Feature Is the One You Don't Build; Five Rungs: API Call to Your Own GPUs; Choosing a Model Without Reading a Leaderboard; A Prompt Is an Interface Contract; Prompt Injection: Permissions, Not Persuasion; Structured Outputs and the Trust Boundary; Embeddings: Meaning as Coordinates; Chunking Decides Your Answer Quality; The RAG Pipeline, Stage by Stage; Vision, Audio, and Documents That Have Layout; Tokens Are Money and Users Control the Spend; Evaluation: The File That Outlives Your Prompt

You can't fix a hosted model. You can only fix what you send it, and what you do with what comes back. Every decision in this module falls out of that one sentence.

In Module 31 you trained small models on your own data. Now you're renting one somebody else built on the whole internet, that bills you per call. Here's what it actually is: a compressed statistical summary of its training data, predicting the next token, over and over. A token is roughly three quarters of a word. No plan, no beliefs, nobody inside checking whether the answer is true.

Keep two words separate forever. Training happened once, cost tens of millions in GPU time, and is frozen. Inference is what happens when you call the API, and that bill is yours, on every request, forever.

Pattern completion is superb at language and terrible at four things. First, arithmetic, because digits are just tokens. Second, exact recall, because it approximates facts instead of looking them up. Third, long chains of logic, where an error in step two survives happily to step nine. Fourth, anything outside its training cutoff, which includes every row in your database.

And it fails in the same confident tone it uses when it's right. Your compiler screams at you. A model smiles at you.

So your operating rule: if a human can't tell a wrong answer from a right one just by reading it, the model doesn't answer alone. It gets a tool, a source document, or a validator sitting between it and your user. Language to the model. Math and facts to your code.

Before you write a prompt, though, ask whether you need one. Start with rules. A boolean, a regex, a lookup table. Then classical code: a SQL query, full text search, a scoring formula somebody can read. Pulling an invoice number out of an email? If the format's stable, a regex is free, instant, and never invents a digit. "People who bought this also bought that" is a SQL join, and it beats an embedding model more often than anyone in AI admits.

A model earns the slot when input is genuinely messy language, images, or audio. Users write badly and upload photos sideways. Deterministic code shatters on that. A model bends. What it costs you is a new failure category: not a crash, not a five hundred, a plausible well-formatted wrong answer your monitoring records as a success.

Now, how deep do you integrate? Five rungs. Rung one is a direct API call, and most features should live there forever. Rung two is that same API, engineered: designed system prompt, worked examples, your own data retrieved and injected at request time. That's a Tuesday of work, and it's what people mistake for needing a custom model. Rung three is fine-tuning, which teaches format and tone. It does not reliably teach facts. Facts go in the context window, not the weights. Rung four is distillation. Rung five is your own GPUs, and the math is brutal: two to three dollars an hour, two for redundancy, plus the engineer feeding them. Call it fifty to eighty thousand a year before you serve one token. Idle GPUs bill you at three in the morning. Idle APIs don't. […]

### Building Production AI Features

Evals, cost and latency budgets, and LLM features that survive real users.

Chapters: It Worked in the Demo; Pick the Pattern Before You Pick the Model; Streaming: Buying Patience by the Token; Prompts Are Code, Treat Them That Way; Context Budgets and the Memory That Isn't; Prompt Injection: The Unsolved One; Output Validation: Nothing Ships Unchecked; Grounding, Citations, and Earning the Right to Say I Don't Know; Agents: Bounding the Loop; Humans in the Loop Without Burning Them Out; Observability: Watching Quality Move; Latency, Cache, Cascade, Fallback

In normal code, correctness is a yes or no. You prove it once in CI and move on. In an AI feature, correctness is a distribution. Your test doesn't pass. It passes with probability 0.94, and you never knew, because you ran it once.

That's the whole shift. Deterministic software either works or crashes, and when it crashes you get a stack trace with a line number. AI features don't crash. They degrade. The wrong answer arrives in the same confident prose as the right one, no exception fires, and the fluency is the bug.

Air Canada learned this in front of a British Columbia tribunal in February 2024. Their support chatbot invented a bereavement refund policy that didn't exist. A customer relied on it. The tribunal ordered the airline to pay him around eight hundred Canadian dollars, and rejected the argument that the chatbot was somehow its own legal entity. Whatever your model says, you said.

So before you write another line of feature code, write down three things and commit them to the repo. First, the worst thing this feature can do to a user if the model is completely wrong. Second, your latency budget, split into time to first token and time to done. And third, the exact sentence the feature says when it doesn't know, word for word. Can't fill in all three? You don't have a feature. You have a demo.

Now, the pattern. Pick your interaction pattern before you pick your model, because it silently sets your latency budget and your blast radius. Autocomplete has to land under two hundred milliseconds and being wrong is free. Copilot drafts, a human accepts or edits, and one to three seconds is fine. Chat is the default choice and usually the lazy one, because a blank box makes the user guess what your system can even do. Agents take actions, and the cost of wrong goes from "ignore it" to "unsend that email." Ambient features, sorting the inbox, tagging the ticket, writing alt text, are the best value per dollar I've seen, because nobody has to believe in AI to enjoy better sorting.

The axis that matters isn't capability. It's reversibility. Map every action to how hard it is to undo, and if the user can't undo it themselves in one click, a human approves first. Default for anything new: start at copilot, measure how often humans change the draft, and grant autonomy per category only when the edit rate is already near zero. Never per product.

Your prompts are code, so store them like code. A directory of files in git, each with a version and a content hash, loaded by ID, never interpolated inline across twelve modules. Stamp that hash on every logged response. That one field answers "did this get worse on Tuesday" in ninety seconds instead of a day of archaeology. Wire the golden set you built in Module 32 into CI so a prompt change prints a behavior diff on the pull request. And cap yourself around a dozen rules in a system prompt. Every rule you add makes known cases better and unknown cases worse. Want precision? Add two or three examples instead. Examples generalize. Rules just pile up. […]

## Part 10: Craft & Production Mastery

The senior habits: hypothesis-driven debugging and the go-live checklist that separates launches from gambles.

### Debugging as a Discipline

Hypothesis-driven debugging: reproduce, isolate, prove - not print statements and prayer.

Chapters: Slot Machine Engineering (And Why It Fails); The Debugging Journal and the Forty-Five Minute Rule; The Loop: Reproduce, Isolate, Identify, Fix, Verify, Prevent; Reading the Announcement: Traces, Errors, Logs; Cut It In Half: Bisect, Wolf Fence, Delta Debugging; Client and Network: The Tab You Never Opened; The Data Layer: Pools, Plans, Locks, Corruption; Containers: The Four Things It Can't Package; Production: Debugging Without a Debugger; Leaks, Descriptors, and Flame Graphs; Concurrency: Widen the Window, Then Design It Away; Ducks, Fresh Eyes, and Saying I Don't Know

Debugging is the scientific method applied to a system that's lying to you. That's the whole definition. You observe a symptom, you write down a hypothesis, you design the cheapest experiment that would prove your hypothesis wrong, and you run it. Wrong, not right. That's the step everybody skips.

What most people do instead is paste the error into a chat window, apply the suggestion, refresh, repeat. Forty minutes later there are six changes in your working tree, you don't know which one helped, and the bug is still there. That's a slot machine. You're pulling a lever.

Marc Eisenstadt collected around fifty war stories from working programmers in 1997 about their hardest bugs. Roughly half the difficulty came from the cause being physically far from the symptom, and from tools that couldn't show what was happening. Not from clever bugs. From distance and blindness.

So here's the rule that costs you nothing. Before you touch a line of code, finish this sentence: "I believe X is happening, and if I'm right, then Y should be true." Then go check Y. If you can't finish it, you don't have a hypothesis. You have a feeling, and feelings don't survive contact with a distributed system.

Three biases eat your time. Confirmation bias: you guess in the first thirty seconds and every log line after that gets read as support. I watched an engineer spend a full day on a caching theory while the stack trace on his second monitor said the request never reached the cache. Anchoring: the first weird thing you see owns your brain, usually a deprecation warning that's been harmless for eight months. And recency bias, the "it must be my last change" reflex. Plenty of bugs sit dormant for a year until data volume wakes them up. Your change was the alarm clock, not the burglar.

Open a text file. Call it scratch dot md, keep it out of the repo. Three kinds of lines go in it: what you observed with a timestamp, what you tried with the result, and what you've ruled out with the evidence that ruled it out. That third one pays for the whole habit, because without it you re-test the same theory twice and the second run gives a slightly different result and costs you another hour. Your working memory isn't up to this. Nelson Cowan's work puts the real figure around four unrelated chunks. A moderately annoying production bug has twelve moving pieces before lunch.

And the hard rule: forty-five minutes with no new information, you stop. Not a quick break. Stop. When you notice you're re-running the same command with slightly different flags, frustration has already collapsed your search space.

The loop is reproduce, isolate, identify, fix, verify, prevent. The step that deserves your time is isolation, and it's the one you're giving nothing to. Bypass the cache. Remove the middleware. Hardcode the input. Call the service directly instead of through the queue. You're building the smallest system that still misbehaves. Beginners spend almost nothing here and burn the session trying fixes. Flip that ratio this week and everything else gets easier. […]

### Production Readiness & Ongoing Operations

The go-live checklist, incident response, and the operations rhythm that starts after launch.

Chapters: Deployed Is Not Ready; Probes That Lie, Shutdowns That Drop Requests; Migrations With No Undo Button; Rollback Is A Feature You Ship; Alerts People Actually Answer; Stop The Bleeding First; Postmortems That Change Something; Runbooks And ADRs: Memory Outside Your Head; Break It On Purpose; Compliance As Engineering Work; The World Is Not Your Locale; Rot: Dependencies, Vacuum, Certs, Dead Features; The Bill And The Ceiling; Backups You Have Actually Restored

Deployed is not ready. Deployed means the code is running. Ready means someone who didn't write it can operate it at 3 AM without calling you. That gap is where most production pain lives, and you close it with writing, not vibes.

Start with four questions, answered in a file in your repo, with links as proof. First, can you tell if it's healthy - not "the server responds," but "checkout actually succeeds"? Second, can you tell if it's slow, at the p99, per endpoint, without SSH-ing anywhere? Third, can you tell who's affected, so an error spike says three users or thirty thousand? And fourth, can you undo the last deployment in under ten minutes? Any answer starting with "not really" is a finding, and findings are sprint work. Add one line to that file after every incident. That's how it stays true to your system instead of somebody else's blog post.

Now the mistake sitting in half the codebases I review. Your health endpoint checks the database, and it's wired to the liveness probe. The database blips for ten seconds. Every pod fails liveness at once, restarts, comes back cold, slams the database with a reconnect storm, fails again. You just turned a ten second degradation into a twenty minute outage, and the restarts wiped the memory state you needed to debug it. So: liveness is a cheap in-process check that touches nothing external. Is the process alive? Return 200. That's the whole endpoint. Readiness may check dependencies, but if a dependency only powers one feature, stay ready and fail that feature instead of removing every replica from rotation.

Shutdown is where your mystery 502s come from. When a pod terminates, your container gets SIGTERM at the same moment the load balancer starts removing your endpoint - and that removal takes seconds to propagate. Exit instantly and traffic is arriving at a corpse. Two moves fix it. A preStop hook that sleeps five to fifteen seconds and does nothing else. Then on SIGTERM: stop accepting new work, finish what's in flight, then close the database pool, then flush logs, then exit. Reverse those last two and you kill the exact requests you were draining. Kubernetes gives thirty seconds of grace by default. Go look at your p99.9. If a request can run forty-five seconds, you're cutting off paying customers on every single deploy.

Schema changes are the one step with no clean undo. Expand and contract, always, minimum two deployments. Add the new column, write to both, backfill, read from the new, and only drop the old thing weeks later. Renaming a column is five steps. Never one. And here's the part that surprises people: in PostgreSQL 11 and up, adding a column with a default is a metadata change, milliseconds. So why do ALTERs take sites down? The lock queue. Your ALTER waits behind some four minute analytics query, and every request arriving after it waits too. The table is down because your migration is standing politely in line, holding the door shut. One line fixes it: set lock_timeout to three seconds. Fail fast, retry. […]

## Part 11: Mobile Expansion

Everything changes on a phone: offline-first data, networks that vanish mid-request, and app-store release trains.

### Mobile-Specific Mental Models

Why mobile isn't a small website: lifecycles, permissions, and the OS as a strict landlord.

Chapters: A Phone Is Not a Small Laptop; Native, Cross-Platform, or WebView; The Lifecycle: Your Code Runs on Borrowed Time; Design for Death: State Restoration; The Sandbox and the Permission Model; Talking to the Outside World; Offline-First Is a Product Decision; Touch, Conventions, and Accessibility; Push Notifications: Plumbing and Restraint; Deep Links: Giving Screens an Address; PWAs: When Native Isn't Worth It; The Five Numbers of Mobile Performance; You Cannot Hotfix a Phone

A phone is not a small laptop. Plug a laptop in and it stays plugged in. Power is free, the network is there, your JavaScript runs until the tab closes. Every one of those assumptions is baked into the web code you've been shipping with AI, and none of them survive contact with a phone.

Mobile is a different operating contract, and five constraints apply at the same time. Every feature gets graded against all five at once.

First, battery. Both iOS and Android show a per-app battery breakdown in Settings. Land near the top of that list and you don't get a bug report, you get an uninstall.

Second, connectivity, which is not a boolean. Wi-Fi, LTE, two bars in a parking garage, nothing in a tunnel, all inside one three minute session. The worst state isn't offline. It's connected but dead: an airport captive portal returning a login page with a 200 status, or a socket that hangs for seventy five seconds before your default timeout fires. Offline you can detect. Half-alive you have to design for.

Third, lifecycle. The operating system owns your process. It suspends you, freezes you, kills you, and never asks.

Fourth, permissions. Camera, location, contacts, photos. Each is a dialog the user can refuse, and revoke six months later while your app is running.

And fifth, the gatekeeper. You don't deploy to users. You submit a binary to a reviewer who's never heard of you.

Here's the habit that costs ninety seconds per ticket. Before you build anything, write one line for each: battery cost, no-network behavior, what happens if the process dies mid-flow, which permission it needs, and how much it grows the binary. Can't answer all five? You don't understand the feature yet.

Now the one that breaks web brains. Design for death. Your process will be killed. That's not a failure, that's the normal exit path. The user opens the camera, then Maps, then a video call, memory gets tight, and you're the least recently used thing on the list. Gone. Forty minutes later they come back expecting their half-filled form and their scroll position exactly where they left them.

So: in-memory state is a cache, persisted state is the truth. Save intent and unsubmitted input - draft text, form fields, the cart, the navigation stack as a list of IDs. Discard anything you can refetch. Save on transitions, not on a timer, because writing to flash on every keystroke burns battery. And no tokens or card numbers in a restoration bundle; that's disk, not a keychain.

Then test it, because nobody does. On Android, flip on Don't Keep Activities in developer options and use your app for a day. On iOS, background it, kill it from Xcode, relaunch. Ten minutes of that finds more real bugs than a week of unit tests, because these never reproduce on your machine where the debugger holds the app alive. […]

### Mobile Data & Networking

Offline-first data, sync conflicts, and networks that vanish mid-request.

Chapters: The Radio Is Asleep: Mobile Networking as a Physical Problem; Four Storage Tiers, and Putting Data in the Wrong One; Local-First: The Network Is Not in the Read Path; The Operation Queue: Where Money Gets Lost; Delta Sync: Only Ask for What Changed; Conflict Resolution: Deciding Whose Edit Dies; API Shapes That Survive a Cold Radio; Serialization: When JSON Stops Being Free; Images: The 200KB File That Costs You 12MB of RAM; Background Work: You Don't Get to Decide When; Adapting in Real Time: Reachability Lies; Security on a Device You Don't Own

The radio in a phone is asleep most of the time. That one physical fact rewrites everything you learned about networking on a laptop.

A modem that transmits nonstop drains the battery in a few hours, so it dozes. When your code fires an HTTP request, the request-and-response protocol your app speaks to the server, the modem often has to climb back onto the tower before a single byte moves. On LTE that wake-up costs 200 to 500 milliseconds. Once it's awake, round trips settle to 30 or 70. So your first call is slow and the next four are fast. Backwards from what your laptop taught you.

And the tax is per wake, not per request. Five calls fired together pay it once. Five calls dribbled out four seconds apart pay it five times, and each wake holds the radio in a high-power state for seconds afterward, burning battery doing nothing. That's why a polling loop every thirty seconds is a battery bug wearing a feature costume.

Two more things about the pipe. Uplink on cellular routinely runs five to ten times slower than downlink, so your 4 megabyte photo upload is slower than a 16 megabyte download. And failure is rarely clean. DNS resolves, the connection opens, and then nothing comes back. No error. Silence. Captive portals, carrier middleboxes killing idle connections, a tower handoff mid-response. Every mobile engineer has lost a day to lie-fi. So set an explicit deadline on every request. My default: ten seconds to connect, thirty second total budget for anything a user is staring at. Never infinite. And treat a request that never answers as a normal condition with a designed screen, not an exception you log and forget.

Now storage. You get four tiers and picking wrong is a real bug. First, key-value stores, UserDefaults and DataStore, for flags and scalars. A few kilobytes. Nothing you'd want to filter. Second, SQLite, running under nearly everything on both platforms. Turn on write-ahead logging so readers never block behind a writer, and test your migration ladder in CI with a real version-2 database file checked in, because somebody will install version 2, ignore your app for a year, and open version 11 on a cold morning at 3% battery. Third, the file system, where the directory matters more than the file. Cache directories get purged by the OS without asking, which is correct, so never put anything there you can't re-download. Documents directories get backed up to iCloud, which is how a 200 megabyte image cache earns you a one-star review. Fourth, Keychain and Keystore. Hardware-backed. Tokens and encryption keys live there and nowhere else.

Here's the decision that reorganizes your whole app: no screen may await a network call to render its primary content. The user taps save, you write to SQLite in a transaction, a reactive query fires, the screen updates in eight milliseconds. Separately, invisibly, a sync engine pushes that work whenever the radio cooperates. If a view controller anywhere renders straight from an HTTP response body, you don't have offline-first. You have a cache with extra steps. […]

### Mobile Release & Distribution

App-store review, release trains, and why you can't hotfix a binary.

Chapters: The Distribution Wall; Code Signing and the Trust Chain; Variants, Versioning and the Update Policy; Beta Testing That Isn't Theater; The Review Gauntlet; The Pipeline: CI/CD With a Human at the End; Staged Rollouts and the Statistics Trap; Over the Air: A Scalpel, Not a Sword; Crash Reporting and Release Health; Remote Config, Experiments and the Kill Switch; ASO: The Four Seconds Before Install; The Runbook You Write Before You Need It

On the web, there's exactly one version of your app running: yours, right now. Broken? Push again. Eleven minutes, worst case.

Mobile takes that reflex away on day one. There is no single live version. There's the build you uploaded this morning, the one from last month that most people are actually on, and a fourteen-month-old binary sitting on a phone that's been in airplane mode since somebody's vacation. Open your analytics and you'll see users spread across a dozen versions at once. You don't get to end that. You only get to make it survivable.

So start here. Pull your version distribution today and find the oldest version with more than one percent of your users. That number, not your latest tag, is the contract your backend actually has to keep. Old binaries can't be patched. They can only be outlived.

Which forces the first big shift: shipping code and shipping a feature become two separate acts. The binary leaves weeks before you want the feature visible, so the switch lives on your server. Every meaningful feature merges wrapped in a remote flag with a default compiled into the app, so a config outage is a normal path instead of a frozen screen. That flag is a kill switch. When the new checkout starts failing at two in the morning, a kill switch turns a three-day review cycle into a thirty-second dashboard change. Nothing else in mobile gives you that.

Second shift: rollback doesn't exist. The bits are already on the phone. All you can do is stop giving them to more people and race a fix through review. That's why staged rollout is your primary safety mechanism, not a nicety. Apple runs a fixed seven-day phased release, roughly one percent, then two, five, ten, twenty, fifty, a hundred. Google Play lets you pick your own percentages and halt manually. Halting stops distribution. It does not un-install the bad build from anyone who already got it.

And here's the trap nobody warns you about. At one percent, your statistics are garbage. Forty thousand daily users means one percent is four hundred sessions, and a crash hitting one in two hundred users shows up as two crashes. Two. That's indistinguishable from noise. So at small percentages you're only hunting catastrophic signals - launch crashes, checkout failures. Subtle regressions don't become visible until twenty percent and up. Soak each stage a minimum of twenty-four hours, because your users in other timezones haven't woken up yet. Name a release owner with the authority to halt without asking permission. Write halt criteria as numbers before you ship, because during an incident nobody can agree what "bad" means. […]

## Part 12: The Send-Off

The closing bookend: what changed in how you think, and the one thing to go do while it is still fresh.

### The Send-Off: Now Go Ship Something That Holds

What changed in how you think, the one thing to do this week, and how to keep the judgment you just built.

Chapters: You Finished; The Expensive Way; Think About What Changed; The Questions That Fire Automatically Now; The Modules That Felt Abstract; Do One Thing This Week; Now Go Ship Something That Holds

Expertise isn't a pile of facts. I've met engineers who could recite the entire HTTP spec from memory and still design a service that fell over the first Monday it saw real traffic. Expertise is a set of questions that fire before the code gets written. Automatically. Slightly annoyingly, because they show up in code review when everyone else just wants to merge.

You have those questions now. That's what thirty-eight modules bought you, and it's worth naming out loud, because skill is invisible from the inside. There was no morning you woke up and noticed you'd become someone who checks the query plan. You just stopped making a category of mistake, then forgot you used to make it.

Here are the six questions. Screenshot them.

First: what happens if this runs twice? Retries exist, networks lie, users double-click.

Second: where does this state actually live? Memory, database, cache, browser tab, some other team's service, or four of those at once and out of sync.

Third: what does this look like at a hundred times the traffic? Not infinity. A hundred times, which is a Tuesday after a good marketing campaign.

Fourth: how does this fail, and who finds out first? You, or your customer. Those are the only two options, and one of them is much more expensive.

Fifth: what does this cost per request? In milliseconds, in database connections, in actual money.

Sixth: if this breaks at 3 AM, what tells me? A page, a dashboard nobody's looking at, or a support ticket at nine the next morning.

Notice what none of them ask. None of them ask whether the code is correct. Correct is the floor. Every one of those questions is about what happens after the code is correct and the world starts pushing on it. That's the gap between writing software and running software, and you're on the other side of it.

Here's why this course exists at all, and it wasn't a business decision. Twenty years ago I ran a schema migration by hand against a live database at eleven at night, because the deploy pipeline was blocked and I was impatient and I was sure I understood the table. I didn't. The lock held for forty minutes. Every write in the product queued behind it. No rollback plan. I sat there watching a terminal with my ears going hot. Nobody had ever sat me down and explained what a lock does under load, or why you don't operate on a live patient because the scheduling system is annoying.

Everything I taught you came to me that way. One outage at a time. Each lesson arrived with an invoice attached, and the invoice was always somebody's weekend. You got the shortcut. That's not a lesser education. It's the version I'd have wanted.

Now, some modules landed flat. I know they did. You followed every sentence and felt nothing. Probably the queue durability one, or backpressure, or a failure mode that's never once happened to you. That's not a gap in you. Queue durability is abstract right up until the afternoon your jobs start vanishing, the retry count is climbing, and nobody can tell you where the messages went. Same words, different day, completely different content, because now you have the problem those words were describing. […]