r/iOSProgramming • • 4d ago

Article Free iOS Simulators on GitHub Actions for exploratory testing: a template that gives each one to an AI agent

Post image

Public repos get free macOS runners (5 at a time on the Free plan). The workflow builds your app (Swift, Flutter, React Native, Expo, KMP...), boots one Simulator per persona with its own model, iOS version, Dynamic Type size, language and dark mode, and gives it to a Claude Code agent that uses the app like a new user.

gh repo create my-testers --public --template pierridotite/mobile-agent-testers --clone
cd my-testers && ./setup.sh
gh workflow run agent-testers.yml

On LocalSend, the iPhone SE at the largest accessibility size couldn't send text at all: the dialog overflows by 149 px. Hosted runners are meant for the repo's own project, so for a closed-source app, run it in a private repo or on your own Mac.

https://github.com/pierridotite/mobile-agent-testers

5 Upvotes

12 comments sorted by

2

u/dawedev 4d ago

The persona matrix is the useful part here. An iPhone SE at the largest Dynamic Type size is exactly the case most of us never check by hand, and the 149 px overflow in LocalSend shows why. How long does one persona take end to end, including the simulator boot? Cold boots on hosted runners can be slow, so I'm curious whether you cache anything between runs or just accept the time since the minutes are free.

1

u/Cylicium 4d ago

On the run that I made on LocalSend, Android is fast, 4-5 min per persona including the cold boot (arround 2 min). iOS is the slow one, 3-5 min just to boot the Simulator and 10-35 min total.

I don’t snapshot anything, every job boots fresh. Only the build is cached (Gradle/npm/Pods) and it’s built once per run. Honestly boot time isn’t the bottleneck, the 5 macOS jobs at a time on the free plan is.

1

u/dawedev 4d ago

So the queue is the real limit, not the boot. With 5 macOS jobs at a time and up to 35 min per iOS persona, a full matrix adds up fast. Running Android first for quick feedback and saving the iOS slots for the personas that turned something up could stretch the free minutes a lot further.

1

u/Cylicium 4d ago

Exactly 👍 that’s why I think this method is a game changer !

1

u/bramantec 3d ago

The persona matrix is the part that caught my attention. My app ships in five languages and the combination I'd want to check first is the longest one at the largest Dynamic Type size on the smallest iPhone, which by hand I'd never do for every screen.

It's a closed source Flutter app. That means a private repo or my own Mac. With 10 to 35 minutes per iOS persona, would you run the iOS side locally for a private project?

1

u/Cylicium 3d ago

Yes it could work bc u have every month a large amount of free minutes on a private repo on GitHub

1

u/Cylicium 3d ago

Moreover u can also create a public repo just to build use the bumble of ur app and test it in an unlimited way !

1

u/bramantec 3d ago

Good to know about the free minutes on private repos. A public repo with only the build is clever, but anyone could download the bundle, which I'd rather avoid with a closed source app.

1

u/Cylicium 3d ago

No necessary u can create a cloud server to store ur bumble and the download of it is protected by a key that u store in the private environnement variable of ur public repo !

1

u/bramantec 3d ago

Neat workaround, thanks. Would the screenshots and the agent's notes still end up in the public run logs?

2

u/Cylicium 3d ago

Exactly, u can do the same process with log