Why we started Semaloop

Charlie·Co-founder & CEONov 17

Before Semaloop I was an early hire at incident.io, and before that I spent years at Palantir shipping software into enterprises where a bug isn't just an inconvenience. Two very different worlds, but they taught me the same lesson: the gap between "we shipped it" and "we know it works" is where companies quietly lose their customers.

Shipped isn't the same as working

Shipping is the part everyone sees. You merge, the release goes out, and it feels like the work is done. Knowing it works is quieter and harder: it means someone has checked that the thing does what it should, for the people using it, in the conditions they're actually in.

When that second part is missing, customers rarely tell you. They find that something doesn't work, decide the product isn't for them, and leave. The bug never makes it into a ticket, and the churn never gets traced back to it.

Mobile is where the gap is widest

Mobile is where that gap is widest. You ship to a device you don't control, running an OS version you didn't choose, in conditions you can't reproduce, and the tools we've all been handed to test any of this were built for a web page that sits still.

On the web, you deploy and everyone gets the new version. On mobile, a release goes out to phones of every age and screen size, some updated and some not, on patchy connections, with permissions granted or denied, and you find out how it behaves from whoever happens to be using it. If something is broken, you can't quietly roll it back in five minutes.

The testing tools haven't caught up. Most mobile automation borrows its ideas from the web, finding an element and tapping it, and the tests it produces break whenever the screen changes underneath them. Teams end up spending as much time maintaining their tests as learning anything from them, which is a big part of why flaky tests should embarrass us.

What teams do instead

So teams do the rational thing: test less, ship on vibes, and hope the crash reports stay quiet. When testing a release properly means a day of manual clicking or a suite nobody trusts, skipping it is a reasonable bet.

Why we started

This felt wrong to me and to Rory, now our CTO, who had seen the problem first hand at Apple. So we quit and started building, beginning with a handful of iPhones in my shed.

The idea was simple to say and hard to build: agents that test apps the way people do, by looking at the screen and using it, with tests described in natural language rather than scripted tap by tap.

Where we are now

A year later, we're making it possible for mobile teams to ship every release with confidence. Semaloop runs every test against every new build on real devices, now on Android as well as iOS. While it does, it quietly picks up signals from each run and groups them into issues, finding the things you didn't think to test for, and with Ask it can answer questions about your app's whole testing history.

That points at where we're going. We want Semaloop to go beyond the tests you wrote: exploring your app on its own, and testing it adversarially, the way a determined user would try to break it.

We've also gone from two people in a shed to a team in an office in Old Street. Dani has written about how we work and what the first month here was like, Sam about week one as a founding engineer, and you can meet Andrew, along with the rest of us, in our team intros. If this sounds like a problem you'd like to work on, our careers page is the place to start.


Charlie·Co-founder & CEO·Nov 17