Piano Tiles 2 is a touch-based mobile game that’s lots of fun, and also maddeningly hard. A normal person would play for a few weeks, get frustrated, and uninstall it like so many similar games. Thankfully, YouTuber DenverFinn is not normal.
For him, the obvious solution was to build a robot that slips over an iPad and acts like human hands, only better. An iPhone overhead films the game at 120fps and sends visual input to a controller which knows to seek out the black keys and avoid the white spaces. When a black key aligns with one of the the tiny mallet-like “fingers,” the controller fires a tiny motor to make the appropriate finger tap the screen. It might not have the interpretive gifts of a concert pianist, but the finger bot managed to hit a record of 21.079 keys per second. In case you’re wondering, that’s blisteringly fast. Near then end, it’s borderline impossible to follow the game with your eyes.
Having conquered Piano Tiles, its easy to imagine repurposing this bot for other tap- or swipe-based games, similar to this spooky bot that lives to beat Facebook Messenger’s basketball minigame.