Ever have an experience where you just couldn’t come up with the name of an everyday object that was sitting right in front of you? This common phenomenon—known as “tip of the tongue”—affects many people, particularly those learning a new language and trying to remember unfamiliar names for familiar things. To help these struggling language students, one developer has devised an Android app that leverages object recognition techniques to present items’ names in multiple languages.
The “What the Thing” app allows users to take photos with their smartphones are receive an instant translation in various tongues.
For example, if the camera photographs a pair of glasses, the app immediately executes an object-recognition algorithm to identify them. What the Thing then shows the English name for eyeglasses and presents a menu of multiple language choices for translation. If the user selects the Czech language option, What the Thing presents the Czech translation for eyeglasses: “brýle.”
What the Thing, which was developed by a Reddit user known as /u/vigzmv, is available as a web app that demonstrates its capabilities. The developer is also offering What the Thing as an open-source app, with the code available for others to use in their software.
What The Thing makes use of Google’s Cloud Vision and Translate applications programming interfaces (APIs.) Cloud Vision supports object-recognition capabilities that allow it to detect items and faces and read words contained within images.
The What the Thing Android app is based on a web app called Thing Translator web app that has the same functionality, according to the XDA Developers website. Thing Translator was developed as part of Google’s AI Experiments project, a showcase designed to allow any coder to tinker with machine-language technology. AI Experiments gives coders at all skills levels access to various Google artificial intelligence technologies, including Google Photos, which employs object recognition to identify people, animals, and things.
AI Experiments is focused on encouraging the creativity of coders who don’t have any experience with machine language. For example, the project offers the Google Cloud Machine Learning Engine, a managed service that allows designers to easily construct machine-learning tools using Google APIs.
The project also includes The Wekinator, a development program that allows users to build new interactive systems by observing human actions, rather than writing code.
With the arrival of apps like What the Thing and Thing Translator to help people find elusive words, terms like “thingamabob” and “whatchamacallit” may soon be relics of the past.
Tyler Schulze is vice president, strategy & development at Veritone. He serves as general manager for developer partnerships, cognitive engine ecosystem, and media ingestion for the Veritone Platform. Learn more about our platform and join the Veritone developer ecosystem today.