Docs that screenshot themselves

There's a small, universal indignity to shipping a product quickly: you rename a button, and three weeks later a customer emails to say your documentation shows a button that doesn't exist. You go looking, and it turns out the screenshot was taken on someone's laptop, at some point, at whatever window size they happened to have open, and then dragged into a folder. Nobody knows how to retake it. So it sits there, quietly lying, until someone gets annoyed enough to artisanally crop a new one by hand.

Screenshots age faster than prose does, because prose is vague enough to survive a redesign and a screenshot isn't. "Open the test menu and choose Duplicate" is still true after you move the menu, but a picture of the old menu is (sadly) wrong.

The usual fix is discipline: a checklist item, a reminder in GitHub, someone whose job it is to notice. That works right up until the week you ship a lot, which is... every week.

Make the screenshot a build artefact

We recently launched our shiny new docs site, and wanted to take a different route. It might be the first place that someone hears about Semaloop, and so it's important to us that it's cared for, and up to date. No screenshot in there is a file that a human produced. Each one is declared, and then generated.

We do this using a single manifest, shots.yml. Each entry says where to point a browser, what to crop to, and what to do first (like opening a modal) if the thing you want isn't on screen yet:

- output: src/assets/screenshots/add-test-dialog.png
  url: ${BASE_URL}/app/${APP_ID}/tests
  width: 1280
  height: 900
  wait_for: *app_loaded
  javascript: '(${CLICK_BY_TEXT})("Add test")'
  selector: "[role='dialog']"
  padding: 16

Then, pnpm shots drives a headless browser through the whole manifest — 28 images and counting — and writes every image afresh. The heavy lifting is done by Simon Willison's shot-scraper, which conveniently turns "screenshot this page" into a line of YAML rather than a Playwright script.

This means a screenshot is no longer an asset someone owns. It's derived from the app, the same way a generated API client is, and the honest question about any of them stops being "is this still right?" and becomes "when did we last run the command?"

Fiddly bits

Two things bit us. Neither of them was the screenshotting.

The first: knowing when the page is actually ready. Our app is a React SPA, so "the page loaded" isn't a useful signal. Say "wait for #root to have children", and you'll faithfully capture the splash screen. Worse, nothing fails: you just get 28 pictures of a centred logo. Each of our screenshots now waits for the app shell and for any loading spinners to disappear, and any screenshot whose subject is a list waits for a row of that list, because a page that renders an empty array while its query is in flight looks exactly like a page with nothing in it.

The second: retina displays. Everything is captured at 2x, which means every image's intrinsic size is double what it should display at. Embed it naively and it renders twice as large as intended, and (more annoyingly) differently-cropped shots stop looking like they're at the same scale. Our pages don't embed images directly: they use a small <Screenshot> component that reads the file's own dimensions, halves them, and lets the browser serve the crisp 2x pixels to screens that want them.

One deliberate constraint: screenshots always come from our pre-production environment and one fixed internal demo app, meaning we can curate the state of that account so that the examples are just what we want. This also means two runs a month apart are actually comparable — if a screenshot changes, it changed because the product did.

Wrapping up

We're not fully there yet: today pnpm shots is a command someone runs, not something that happens on every release. But that's a scheduling problem, and scheduling problems have obvious answers. The hard part — making "the screenshots are out of date" into a thing you can fix with one command instead of an afternoon — is done.

I can't leave without squeezing in a pitch... There's a nice symmetry here with Semaloop. A test is a sentence you write once — "subscribe to premium" — and the agent works out the taps when it runs, because the taps are what goes stale. A screenshot is that same bargain: write down which dialog on which page, and let something else produce the pixels.

If your docs are full of screenshots of a product that no longer exists, the fix probably isn't more discipline. Find out more about what we're building at semaloop.com.