#HTE

Sony System for Using Any Object, Including a Banana, as a Game Controller

While Sony’s PS5 comes with controllers of their own design, they’re speculating that one day users might like to create their own. Their U.S. Patent #1910975.0, published last month, calls for “a system for generating video game inputs” using any old “passive non-luminous object being held by a user.” To the company’s thinking, anything that isn’t shiny or emitting light could be used. The example cited in the patent is a banana.

So how would this work? Via a camera and projector (mounting location TBD), and software. The camera detects the object and its contours:

The software detects the user’s fingers and where they obscure the object, while using machine learning to work out a rough centerline of the object to help place it in space:

Machine learning also helps to determine the object’s centerpoint:

The projector shines virtual buttons onto the object’s surface, tracking with the object as it’s moved:

When users press the buttons, the camera and software register this as a click; when the user tilts, turns or rotates the object, that movement in relation to the object’s centerline and centerpoint is translated into motive instructions for whatever the on-screen action is.

It’s an intriguing idea and I’d love to see a bunch of people carving blue foam to create their personally ideal ergonomic shapes. Someone would open an Etsy shop for sure.