This Necklace Camera Can Hear Silent Voice Commands | Screen Rant – Screen Rant

In what could prove to be a major accessibility breakthrough, a Cornell assistant professor has created a necklace-inspired wearable camera that can track the movement of skin on the neck and face without any actual sound input. The device is actually a third-generation innovation that builds upon two other remarkable projects. The first one was C-Face, an earphone that studies cheek contours to track facial expressions, and translates them into 3D emojis and speech commands. The 2020 showcase of C-Face as a minimally obtrusive facial expression tracking device was followed by another cool invention called NeckFace in August last year.

NeckFace was a necklace-type wearable sensing device that deployed IR cameras to capture an image of the chin and face from beneath the neck. The idea was to deploy its core tech for applications such as attending videos without a front-facing camera, silent speech recognition, and facial expression tracking in virtual reality. The idea sounds similar to Apple’s rumored AR/VR headset that will reportedly let users attend FaceTime calls via their Memoji avatar. Now, the mind behind C-Face and NeckFace is back with another promising device.

SpeeChin is a speech-recognition device that takes the voice aspect out of voice commands. The brainchild of Cheng Zhang, assistant professor of information science in the Cornell Ann S. Bowers College of Computing and Information Science, SpeeChin adopts the necklace design and relies on an IR camera to study the deformations that appear on the face and neck when an individual speaks. Apple also employs an infrared camera in its TrueDepth system on iPhones to generate a 3D face map for authentication and creation of Animojis.

Understated, But Meaningful

A key objective of SpeeChin is that it allows a person with vocal impairment to just mimic the task of uttering a voice command, and the device will actually track the movements to understand what was said without an audible input. “This device has the potential to learn a person’s speech patterns, even with silent speech,” Zhang notes. Another use case scenario is when an individual wants to summon Siri to perform a task, but the surroundings are not fit for talking loudly to one’s phone or speaker. What’s truly impressive is that Zhang built the device on his own at home while handling academic duties remotely. It’s remarkable to see that a promising tech like this can be so simple in its engineering, and potentially more affordable, compared to the thousands of dollars that gadgets like Facebook’s Project Cambria headset or Apple’s AR/VR headset are reportedly going to charge.

SpeeChin comes with an IR camera that’s glued to a 3D-printed necklace platform, and a coin has been attached to the base for added stability. Another promising aspect of SpeeChin’s design is privacy. Unlike Snap’s Spectacles or Facebook’s Ray-Ban Stories glasses that record everything (and every person) in the camera view, SpeeChin’s hanging orientation is fit only for recording the facial expressions of the person wearing the smart necklace. In tests, SpeeChin recognized English and Mandarin commands with an accuracy of 90.5% and 91.6%, although the accuracy dipped when the subject was walking. There is no word on whether Zhang has found a partner to commercialize the idea, or if the project is going to be open-sourced, but the camera-based device is ingenious and would benefit from a wide release.

Next: Watch A Cyborg Fish Powered By Human Heart Cells Swim

Source: Cornell

Shoresy Trailer Gives First Look At Letterkenny Spinoff

About The Author