Chatbots Learn Words as They Dialogue

Japanese university researchers have devised an artificial intelligence system that can learn new words by interacting with users, potentially enhancing speech-oriented applications like chatbots, according to the Digital Journal.

Scientists at Osaka University have developed a technology called “lexical acquisition through implicit confirmation.” This method allows computers to infer the definition of a new, unrecognized word based on multiple conversations with users.

The AI system then confirms the definition of the new word by engaging in dialog. This confirmation is conducted implicitly, working the definition checks into the talks with users.

While such systems exist today, they often engage in laborious question and answer sessions, requiring users to respond to multiple requests to define words. This process can make these systems less desirable to use.

“…repeating such explicit confirmation requests can degrade the user experience in chat-oriented dialogues,” the Osaka University scientists wrote in a research paper. “We, therefore, need to find a way to enable chatbots to interact with the user naturally and acquire lexical information. To solve this dilemma, we proposed an approach using implicit confirmation where the system makes a confirmation request about the predicted category and uses the user’s response to decide if the category is correct or not.”

Essentially, the system engages in conversation with users to test whether its predicted definition of a word is correct. The system works the words into dialogs and evaluates the accuracy of its interpretation based on user response.

The technology could allow conversational systems like chatbots, robots and voice assistants to become vastly more useful by expanding their lexicon in the field. Current systems often are limited to the vocabulary of words that have been pre-programmed within them.

Driven by demand from the customer-service sector, the global market for chatbots is expected to nearly double over a period of three years. Worldwide chatbot revenue will rise to $1.86 billion in 2020, up from about $1 billion in 2017, according to the consulting firm Accenture. Revenue then is set to triple during the coming decade.

“We know that nine out of 10 consumers globally want to use messaging to interact with businesses,” Accenture stated. “And in customer support, as the telephone loses its primacy as the preferred channel of communication, mobile apps, social media and web chats are taking over. This shift points to an explosion in the global market for chatbots.”

Accenture predicts conversational learning techniques will play an important role in expanding the chatbot market.

“Using machine learning, we’ll see more sophisticated chatbots developed that can learn about users, based on existing conversations and all the other data held on that individual within the business,” Accenture wrote. “An airline customer, for example, might ask about available flights to Madrid three months ahead. Nearer the likely travel date, the chatbots would use that information to initiate a relevant conversation and prompt a booking.”

Tyler Schulze is vice president, strategy & development at Veritone.  He serves as general manager for developer partnerships, cognitive engine ecosystem, and media ingestion for the Veritone platform. Learn more about our platform and join the Veritone developer ecosystem today.