A Sandbox for Demos You Throw Away

Every demo I need to show someone lives on one machine, and none of it is allowed anywhere near the hosts that matter. That box runs a small platform-as-a-service alongside a reverse proxy, and each demo is a compose project with its own hostname, its own database where it needs one, and its own lifetime.

The layout is deliberately boring: application source in one tree, compose files in another. A demo is a directory. Deleting a demo is deleting a directory. That reversibility is the whole point of having a sandbox rather than "a spare folder on the good server."

Three things worth stealing

A permanently-running canary. There is a trivial hello-world service on the same proxy and the same network as every demo. When a demo stops answering, the first question is whether the demo broke or the routing broke, and the canary answers it in one request. It costs nothing and it has saved me from debugging an application that was fine at least twice.

Observability attached by default, not per-project. Container metrics, a log shipper, and a lightweight agent run on the box itself, so a demo inherits monitoring by existing. Anything you have to remember to add to each new project will eventually be missing from the project you most needed it on.

Demos are stateful enough to be honest. Each one gets a real database and a real queue where the design calls for it, rather than a mocked backend. A demo that cannot fail the way the real thing fails is a slide, not a demo.

The bug: four running services reported as missing

My homelab keeps a relationship graph, built by collectors that observe what is actually running. It reported four of these demos as documented but not observed — the flag it raises for a service that exists on paper and has vanished in reality.

All four were running. I checked.

The cause is a naming problem with three axes:

• my inventory names the logical service
• the collector observes the container name
• the uptime monitoring uses a display name

Nothing reconciled the three, so a service whose container name differed from its inventory name looked, to the graph, like a service nobody had ever seen. The fix was an alias map, plus a deliberate decision not to merge sidecars — the database and worker containers stay distinct rather than being absorbed into the service they support, because collapsing them would hide a dead worker behind a healthy web container.

The part I did not fix

The alias map is flat: service name to container names, with no notion of which host. And two of my machines run containers with identical names — the sandbox demo and the real GPU stack behind it.

Today that is inert, because only one of those hosts has a container collector. If the other ever gains one, the alias would quietly absorb the production stack into the demo's identity.

I wrote that down in the commit rather than fixing it. Not because it was too hard — host-scoped aliases are the obvious answer — but because a latent problem that is recorded is a known risk with a stated trigger, while a latent problem that is fixed under time pressure with no test for it is just a different bug you have not met yet. The trigger is written down. When that host gets a collector, the note is waiting.

"Not observed" is a statement about the observer. It took four healthy services reporting as missing for me to internalise that one.