How Semaloop works with your existing tests

"We already use Maestro."

We hear this most weeks, and the honest answer surprises people: a lot of you should keep it. The question of whether Semaloop replaces your existing test suite is the wrong question, and it is one we have probably encouraged by answering it badly for a long time.

Three places to put the intelligence

Every test needs someone or something to decide two things: what to do next, and whether what just happened was right. The real difference between testing approaches is when that decision gets made and who makes it, not the syntax.

Script-based tools decide at authoring time. With Maestro, XCUITest, Appium or Detox, a person works out the exact sequence of taps and the exact assertions, encodes them once, and the machine repeats them precisely on every run. The decision is made once and then frozen until the next update. That's what makes those runs fast, cheap and perfectly repeatable, and it is also why the knowledge inside them starts going stale the moment your app changes.

Manual testing decides at run time. A person works out what to do next as they go, notices when something looks wrong even though nobody thought to check for it, and adapts to whatever the app puts in front of them. Endlessly flexible, and you pay the full cost again on every single run.

Semaloop decides at run time without a person. Our agent reads the screen, works out the path to the goal you described, and judges whether the outcome was right. It is why our tests survive renamed buttons, reordered onboarding and ongoing A/B tests. It is also why a run takes minutes rather than milliseconds, but that time is saved when it comes to not needing to maintain the suite.

None of those three is outright better than the others. They are good at different things, and the teams getting the most out of us are the ones who have worked out the split rather than picking a single winner.

Who should own what

Where Maestro earns its place

We are not going to pretend Maestro is a bad tool, partly because it is not, and partly because the person reading this is often the one who introduced it after a long fight to get off something way worse.

The things it is genuinely good at are real properties that Semaloop does not have:

• YAML that lives in your repo and gets reviewed in a pull request.
• Runs measured in seconds, which is what you want when the check is blocking a merge.
• Complete determinism, so a failure means one thing and only one thing.
• Open source, so you have complete control.

Where it stops paying is a different set of things:

• Flows that change every sprint, so the test changes every sprint too.
• Pop-ups and interstitials owned by another team that break everyone else's suite the week they ship.
• Suites where somebody has quietly started adding conditions like "if this is an E2E run, do not show the modal", which is the point at which the test has stopped describing your product.

If you want a rule for the triage as you move over to a tool like Semaloop, we think this is a pretty good one: keep the scripted tests you have not edited in three months and delete the ones you've had to fix more than twice, rebuilding those flows in Semaloop and see whether you miss them.

Couldn't I just get an AI to write my Maestro tests?

You can, and the tooling for it is getting good. What it changes is the cost of writing tests, which was rarely the expensive part. What comes out of the other end is still a deterministic script that encodes an assumption about a UI that is about to change, so you have made brittleness cheaper to produce rather than making it go away. It also does nothing at run time, which is where flakes actually happen, when the network is slow or a sign-in prompt appears in the wrong order.

The distinction is authoring-time intelligence versus run-time intelligence. Generating YAML with an agent is the former. Semaloop is the latter, which is why the spec stays a description of what the user is trying to do and the adaptation happens live, on the device, on that run.

Where your manual testers fit

This is not a headcount argument. We are not pitching you a way to shrink your QA team, and the teams we work with have not shrunk theirs.

Across the teams we speak to, the pattern is remarkably consistent: a regression suite of a couple of hundred manual checks that eats three or four hours of the team every time they want to ship, usually on a Friday, usually with someone watching the clock. That work is repetitive, and it is exactly what an agent should be doing. The rough shape we see is that the routine sixty to seventy per cent gets automated and the testers move onto new features, exploratory work and hunting the edge cases nobody wrote down.

There is a second-order effect worth mentioning. Because our tests are written in plain language, manual testers are usually the best people in the building to write them. They already produce exactly this artefact, a clear description of what should happen, so the tool becomes leverage for the QA team rather than something being done to it.

Where we are not the right answer

We can report test status as checks on your pull requests, but a Semaloop run takes minutes, so if what you need is a binary answer in under a minute before a merge, keep a small scripted smoke suite for that job. If you need to assert an exact value in an exact place, a script does it better and cheaper. If your team's hard requirement is that every test lives in the repo as code, we are not that today. And if you are running a production canary every fifteen minutes that you want to end-to-end test, the economics favour a script.

What a sensible setup looks like

For most of the teams we work with, the end state looks something like this:

• A handful of scripted smoke tests that run on every commit and stay untouched for months.
• Semaloop running the full regression suite on every build, on real devices, adapting as the app changes.
• Your people spending their time on new features and on the exploratory work that finds the bugs nobody thought to specify.

That's what most of our customers running Maestro do, and the split usually becomes obvious well before their two week trial is up.