The Google I/O2024 keynote was pretty heavy on AI, but more announcements are expected from the rest of the conference. 1 thing Google has already announced is "Project Gameface," an open source feature that allows you to control Android's "gaming mouse" using your face.

At the moment, Project Gameface is an accessibility feature only available to developers. So it's entirely up to them whether this feature is actually integrated into Android mobile games. But just because this focuses on accessibility doesn't mean that not everyone has been able to benefit from such a feature.

We've found that smartphone games are always very frustrating because they don't mix well with many traditional games that require the push of a button. I've been trying to play the classic Sonic the Hedgehog game recently, and while readily available on Android, I'm all for it if I can add extra control options that let them control facial expressions without using a controller.

The idea behind Project Gameface is to utilize the device's camera and a set of facial expressions supplied from MediaPipe's Face Landmarks Detection API. These expressions are used to control the "cursor" (or anything that passes it on an Android touch screen) by converting facial and head movements into actual commands.

Developers can also set and customize how these commands actually play, including expressions, gesture size, and more. This feature is not dedicated to games, and google has partnered with Incluzza to offer Project Gameface elsewhere, such as work, school or social situations.

Hopefully, developers will take this seriously and provide this kind of gesture control in their own app. The last thing we want is that Project Gameface will end up ignoring and ignoring Google's Soli gesture control dedicated to pixel4. Since Gameface does not require any specific hardware and uses open source software for its implementation, that should not happen.