Seeking Wisdom · AI/ML Engineering · Investing · Books · Projects · About ·

A series in fourteen parts

Kubernetes by Example

Everything you need to run something real, taught in order, against one project small enough to fit on a laptop and complete enough to break properly.

Most Kubernetes tutorials put nginx behind a Service and call it a day. That gets you through the first afternoon and then abandons you, because the questions that actually come up — why is this one a DaemonSet, where did my shared memory go, why won't this pod schedule, who keeps resurrecting the thing I deleted — never arise when the example has one container and nothing at stake.

So this series uses one project the whole way through. It starts in Part 1 as a single pod doing almost nothing, and finishes in Part 14 as a fleet run by a controller you wrote yourself. Nothing gets introduced before you've felt the problem it solves.

starts from zeroruns on a laptopGPU optionalevery part has a break-it step

The project: airwaves

Transcribe a fleet of internet radio stations, live. One pod per station pulls the audio down. A shared decoder on each machine turns compressed audio into raw samples. A speech model turns samples into words. The words go to a queue, into a database, and onto a small web page you can watch scroll past.

It was chosen for a boring but important reason: audio is about a hundred times smaller than video. Thirty stations run comfortably inside a pretend cluster on a laptop, while keeping the exact shape a serious fleet has — many identical workers, a couple of shared services they all depend on, and a control plane deciding who exists. The speech model is tiny and runs on a CPU, so every GPU section has a fallback you can actually try.

one machine · your laptop, pretending 30 × tuner — one pod per station pulls a stream · publishes what was said decoder one per machine · audio → samples asr one per machine · samples → words sweeper · clears up after the dead shared memory on the machine itself one ring per station — audio never travels down a socket web a page showing what is being said, live conductor reads the station list · makes the pods busa queue of things that were said small and valuable goes here scribe queue → database rewind 0 → many, on demand catalogue which stations exist prune nightly tidy-up
Hover a part below to see which pieces it covers.

The parts

Four tiers, in dependency order — each one assumes the ones above it and nothing else.

Tier 0 · The model

  1. 01The object modelWhat Kubernetes actually is. Why apply saying “created” tells you nothing about whether anything is running.
  2. 02Pods and containersThe smallest unit, read line by line. Images, commands, environment, and the containers that run before the real one.
  3. 03Labels, selectors, namespacesHow thirty identical pods stay distinguishable, and how everything else in the cluster finds the ones it wants.

Tier 1 · Workload kinds

  1. 04Deployments and rolloutsRolling out a new version without dropping what's in flight — and what maxSurge actually costs you.
  2. 05DaemonSetsOne copy on every machine. The three questions that tell you whether you need this instead of a Deployment.
  3. 06StatefulSets, Jobs, CronJobsThings with names that survive restarts, things that run once, and things that run at four in the morning.

Tier 2 · Wiring

  1. 07Services and IngressHow anything reaches anything else — and the case for deliberately using none of it.
  2. 08Config and SecretsWhere settings live, how a pod learns its own name, and what Secrets honestly do not protect you from.
  3. 09StorageDisks that vanish, disks that persist, and the one that lets two pods look at the same memory.

Tier 3 · Scheduling

  1. 10Requests, limits, QoSThree completely different numbers for the same CPU, and which one the scheduler is actually reading.
  2. 11Placement and devicesGetting a pod onto the machine you meant, and getting a GPU to the pod that needs one.

Tier 4 · Production

  1. 12Staying upProbes that help, probes that cause outages, and shutting down without losing work.
  2. 13Scaling and permissionsGrowing on demand, shrinking to nothing, and why a pod that asks the cluster questions needs permission.
  3. 14Helm and custom resourcesPackaging all of it, then teaching Kubernetes a brand new kind of object that it manages for you.

How every part is built

The same seven moves each time. Knowing the shape in advance means you can skip to the bit you came for.

MoveWhat it is
The symptomSomething has gone wrong, or is about to. Usually something that happened to me.
The ideaThe concept in plain words, with a diagram you step through one claim at a time.
AnatomyThe relevant commands pulled apart piece by piece, so you can build your own instead of copying mine.
Build itThe smallest thing that works. Runnable, not illustrative.
Break itReproduce the failure deliberately. This is the part people actually remember.
What followsThe consequences, including the ones that are inconvenient.
Cheat sheetEight or nine lines worth keeping.