Docs that listen

Every customer of ours gets a shared Slack channel, and we're on regular calls with all of them. Those two places are where the real questions turn up: how to run only some tests from CI, what gets reset between runs, whether deleting a test is permanent. When we answer, we do it in that thread or on that call, and then the answer disappears.

So we've started treating those conversations as the to-do list for our docs.

How it works

Every Monday morning an agent reads the last week of our customer and trial Slack channels, along with the notes from our customer and sales calls. It keeps only the questions another customer could plausibly ask, so one-off account admin, bug reports, pricing conversations and feature requests are left out. It then checks each one against the whole docs site (we publish it as a single llms-full.txt, which turns out to be just as handy for our own agents as for everyone else's) and drops anything the docs already answer clearly.

What's left comes back to me as a short list of suggested changes: the page to change, what to add, who asked, and anything it couldn't confirm from the code or the conversation, marked as needing my input. I review them, and the agent makes the approved edits on a branch and opens a pull request. Nothing gets published without a person reading it first, which matters, because a model summarising a call can confidently remember something we never said.

What the first run found

Before switching it on we pointed it at the last few months of conversations. It came back with a long list of gaps, from the obvious (what counts as a test execution) to the ones we'd stopped noticing (which variables a custom tool can use, why a request to a pinned host can go missing from the network log).

Why bother

None of this is clever. It's an agent, a docs site that's easy to read in one go, and a person who approves the changes. But it turns support from something we repeat into something that compounds: every question answered in Slack now has a decent chance of being the last time anyone needs to ask it.

If you want to see where those answers end up, the docs are the place, and if you'd like to see what we're building, we're at semaloop.com.