Ever since the global explosion of generative artificial intelligence in 2022, the technology has become the talk of the town. It is not only the large language model (LLM)-based AI but all the different branches of the technology have seen exponential growth in both development and application. AI has also left the digital realm and has entered the real world, helping people improve their lives. From language translation tools to early detection of diseases, it has impacted all spheres of society.
Glovatrix, an Indian startup based out of Pune, is also aiming to solve a similar pain point for those who suffer from speech and hearing impairments. As per a report by the World Health Organisation (WHO), there are 1.5 billion people in the world who suffer from some level of hearing impairment. While the organisation does not keep such data for speech impairments, it is likely in the millions as well. In many cases, these people are dependent on either written communication or sign language to communicate. While the former can make communication very slow, the latter requires an interpreter at the listener’s disposal to translate the sign language into spoken words.
This situation can often result in difficulties in securing a job, especially one that is customer-facing or requires one to communicate a lot. But this is where companies like Glovatrix come in. The Indian startup is working on an AI-powered device called the Fifth Sense that can convert gestures from sign language to speech in almost real-time and help people suffering from verbal impairments communicate effectively and without assistance. We, at Gadgets 360, spoke with the cofounders of the startup to know more about the product and the technology behind it.
Glovatrix, an Indian AI startup
Glovatrix was founded in 2021 by co-founders Aishwarya Karnataki and Parikshit Sohoni, and both of them serve as the joint CEO of the company. Karnataki met a child with hearing impairment in 2009, which inspired her to learn sign language and their interactions planted the seed of making a difference for those who suffer a similar fate. The seed bloomed into Fifth Sense when she met the other co-founder Sohoni, a data scientist with extensive experience in predictive analysis, who experienced these struggles within his own family and was able to resonate with the cause instantly.
The duo have been working out of a shared office space in Baner, Pune, along with an engineer, to make a team size of three. Explaining her vision, Karnataki said, “Our vision is to facilitate seamless communication between people of all abilities and give every deaf and speech impaired person the ability to express and be heard in their natural language – sign language.”
Fifth Sense, the AI-powered gloves
Photo Credit: Glovatrix
Fifth Sense, AI-powered gloves for the speech-impaired
From a form-factor perspective, the AI-powered device looks like a glove with a smartwatch placed on top. Sohoni told us that the AI gloves were made using a lightweight fabric that can be worn for 6-8 hours at a stretch without discomfort. There is an opening at the top of the gloves where the fingers can poke out to enable the user to use a smartphone or any other tasks that require a better grip. The fabric comes equipped with a smartwatch and multiple sensors to capture any gestures made. The fabric itself is detachable and can be washed separately as well.
Coming to the hardware, Sohoni explained that the company sources the components used in the device from different countries, and then gets it manufactured separately based on the company’s in-house design. This is a standard model adopted by most of the wearable companies that operate in India.
But it is the software, where Glovatrix has brought forth its innovation. There are two parts in this system that enable seamless two-way communication. The first is the device itself which is powered by AI, and the second is a companion app. Sohoni said, “The AI architecture was completely developed in-house as there was no reference for us to look into.” Interestingly, Glovatrix does not use generative AI and instead uses a mix of machine learning and different types of analysis algorithms for its gesture-to-speech interface.
When a gesture is made, the companion app converts it into audio and plays it out for the listener. It also acts as a receiver when a hearing-impaired person has to listen to the speaker without sign language. The app listens to sound and then converts it into text for the user to read. Interestingly, the app not only picks up spoken words but also other sounds such as the doorbell ringing. The company also claims that the device works almost in real-time, enabling fluent communication.
How Glovatrix is solving the privacy and connectivity challenges
The common problems that are seen with smart devices like these are connectivity and privacy. Most of the smart devices, especially those that use AI, compute and process on the server. This means a fast working internet is crucial to the lag-free experience. Similarly, smart devices are required to collect a lot of user data to offer their functionalities. Keeping this data on the server can also raise privacy concerns in case of a breach.
Glovatrix has found solutions for both problems. The entire receiver part of the app is done on-device, meaning any user-side audio collected to convert to text never leaves the device. This part is also lag-free as it does not require active internet connectivity. On the gesture-to-speech side, Sohoni said that some of the important words and individual alphabets will also be added to the app itself to eliminate the problem of connectivity. However, since AI models do require powerful computational processing, the rest will occur on the cloud which will require a steady internet connection. Notably, the company’s cloud is also built natively and should help in optimizing the server-device connectivity for Indian users.
A thing to note here is that while the AI has been trained in Indian sign language and converts the text in Hindi and Marathi, a text translation tool within the app will be able to generate audio in English and most Indian regional languages.
Finding the product-market fit
Fifth Sense is currently a prototype, and Sohoni revealed that the company will soon begin its first pilot test. The startup also remains confident that it will be able to find the product-market fit within the next six months.
Even as the product is not market-ready, Glovatrix has already seen the impact of its device. It claims that a speech-impaired individual was able to land a job after using Fifth Sense to communicate with his interviewer.
And just how expensive can the product be? Sohoni said that the vision with the AI gloves is to keep the price point competitive so that it can reach the masses. While he did not disclose a particular price point, he said it could cost as much as a mid-range smartphone. Further, to bring down the price point, Glovatrix is also looking into a subscription-based revenue model which can further reduce the price burden on the end consumer.