Sony's New AI Patent Predicts Button Presses Using Finger-Camera Tech

Sony has recently filed a new patent, WO2025010132, titled “TIMED INPUT/ACTION RELEASE,” which aims to significantly reduce latency in future gaming hardware. This innovation was first spotted by Tech4gamers and focuses on streamlining the "timed release of user commands" by predicting the player's next button press. This approach could be a game-changer for PlayStation, enhancing the responsiveness of games, particularly in genres like twitch shooters that demand both high framerates and minimal latency.

With the introduction of PlayStation Spectral Super Resolution (PSSR) on the PlayStation 5 Pro, Sony has already demonstrated its commitment to enhancing visual quality through upscaling to 4K. However, newer graphics technologies such as frame generation can introduce additional latency, which may compromise the responsiveness of gameplay. Recognizing this issue, Sony's patent proposes a solution that involves a machine-learning AI model and additional sensors to anticipate user inputs.

According to the patent, the system uses an AI model to predict the next input, potentially aided by external sensors like a camera focused on the controller to detect which button the player is about to press. Sony explains, "there can be latency between the user's input action and the system's subsequent processing and execution of the command. This in turn results in delayed execution of the command and unintended consequences in the game itself." The proposed method includes using camera input as an input to the machine learning model to indicate the first user command.

Additionally, Sony considers the use of controller buttons as sensors, leveraging their past innovations with analog buttons in next-generation controllers. While the exact implementation of this technology in future consoles like the PlayStation 6 remains uncertain, the patent clearly reflects Sony's ongoing efforts to mitigate latency issues, especially in light of popular rendering technologies like FSR 3 and DLSS 3, which can add frame latency.

This patent underscores Sony's dedication to enhancing the gaming experience by ensuring games remain responsive even as visual and performance technologies continue to evolve. Whether this specific solution makes it into future hardware is yet to be seen, but it's evident that Sony is actively exploring ways to keep latency at bay.