Fortune just posted an article about a trivia war between the Google Home and the Amazon Echo (or really “Google Assistant vs Alexa”). The report claims that of 3,000 questions, the Google Home was six times more likely to provide a correct response than the Echo.
The real test would be not to check questions like “What is the capital of Somalia” but what percentage of true user queries are correctly answered. That is more of a gauge of the device’s abilities to enchant customers. For a virtual assistant, domain knowledge is much more important than general knowledge.
While there is a long tail of queries that can be asked of an assistant, the distribution has a very narrow head that probably accounts for much more than 80% of queries. Being perfect at handling the top 20 queries is worth much more as an investment compared to adding answers to another 3,000 questions. What will become even more important is the elements of personalization that can surround a response.
In that respect, we might need to be more specific and look at artificial emotional intelligence vs. straight AI.