Distinguishing between the relevant and irrelevant bits of a conversation is a good life skill in general, but for voice assistants like Amazon’s Alexa, it’s indispensable. In order to respond appropriately to what’s being said — about anything from the weather to a nearby restaurant or a package in transit — they need to know whether the subject at hand is beyond their knowledge scope.

Researchers at Amazon tackled the problem with a natural language understanding (NLU) system that simultaneously recognizes in-domain (known) and out-of-domain (unknown) topics. The results will be presented at this year’s Interspeech conference in Hyderabad, India in early September.

“Sometimes … an Alexa customer might say something that doesn’t fit into any domain,” Yong-Bum Kim, a scientist within Amazon’s Alexa team and a lead author on the paper, wrote in a blog post. “It may be an honest request for a service that doesn’t exist yet, or it might be a case of the customer’s thinking out loud: ‘Oh wait, that’s not what I wanted.’ If a natural-language-understanding (NLU) system tries to assign a domain to an out-of-domain utterance, the result is likely to be a nonsensical response.”

The team began by assembling two datasets comprising utterances (i.e., voice commands): one covering 21 different domains and the other sampled from 1,500 frequently used Alexa skills.

When it came to choosing a model, they settled on a bidirectional long short-term memory (Bi-LSTM) architecture that (1) factored in the order in which the utterances were received and (2) considered the data…