Claude has been very good this week so I decided to let it have some screen time playing GTA 6, limited to one hour.
With GTA 6 came out on Thursday and it's only on consoles, this took some setting up. Claude can only control my PC with screenshots, clicks and key presses. Each move takes a few seconds, which is nowhere near fast enough for real-time play.
What we did:
Streamed the PS5 to the PC with chiaki-ng, an open-source Remote Play client. It lets you map the controller to the keyboard, whereas Sony's own app wants a physical pad. Windows Defender didn't like chiaki at first, so that was a good start.
Mapped everything to keys: WASD for the left stick, arrow keys for the camera, E/Q for R2/L2 and Escape to pause.
Made it turn-based. Claude unpauses, holds a key for about a second, pauses again, takes a screenshot, decides what to do next and repeats. The Cradle research project used the same trick on Red Dead 2.
Changed some game settings to make it survivable: lock-on targeting, press once instead of hold for aiming and interacting, the largest subtitles, and the minimap always on.
One workaround needed Shift, so sprint moved to T. Pressing two buttons at once turned out to be one of the harder problems.
OBS recorded the whole session, about 45 minutes. Afterwards Claude cut it down with ffmpeg on my PC into a 12-minute video and a few vertical Shorts.
How it went: it crashed the car four times, but it didn't die. The package from the mission is still "pending".
To be honest, it isn't really playing GTA, and it's very slow. Still, watching it reason its way through a stick-up one screenshot at a time is oddly gripping.
I've now told it to look at how it did so it can improve things both in terms of how it plays and whether it is happy with the current setup (it can't be!).
It will be allowed to implement its changes later on today, then have another hour of gameplay.
You can see its (literal lack of) progress here, lets see what it gets up to tonight!
https://youtu.be/N6jT47XGUvY