Voice UX: Ideas For A Voice App Quality Metrics Framework

Background

In the midst of the ongoing hype around emerging voice-based Virtual Assistants (including Virtual Advisors, Virtual Companions, etc.), there is a crucial aspect that has been somehow pushed to the backburner and largely overlooked by the recent media analysis. That is indeed the overall voice UX quality, and particularly the usability aspects of the 1000s of skills, actions and voice apps already released or being offered to the public. Compared to the more traditional user interface paradigms such as CLI, GUI, etc., the lack of attention to the qualitative aspects seems to oddly suggest the naive belief that the voice user interface by itself could and should be considered enough to maximize automagically the quality of the user experience of any given application or product.

 We all had our share of bitter experiences using the clumsy IVR (Interactive Voice Response) systems. Some of us would recall that the level of frustration has started dampening with the increasing improvement and adoption of NLP (Natural Language Processing) by service providers. With Apple Siri first and then with the new breed of wannabe CUI (Conversational User Interface) solutions by Google, Amazon, Samsung, Microsoft, etc., we discovered the actual and potential virtues of the overall voice UX. However, the expected hype of the marketers by one hand and the genuine curiosity and easy enthusiasm of the early adopters by the other have generated a thick smokescreen that has mostly masked the slithering consumers' dissatisfaction.

 There are still none or not enough trustworthy analytics released to the public and openly supported by the major market players in regards to the actual consumers use of the thousands of Alexa skills, Google Home conversational actions, etc.. However, both our usability test and field interviews in the Silicon Valley area offer a number of clues that can be traced back to discoverable diffused consumers' frustration. 

Our Goal

The main objective of this quick report is not to compile a list of consumers' grievances but to suggest an initial framework for establishing measurable criteria that can allow for a more detailed evaluation of skills, actions, and other similar voice apps released or to be released soon based on emerging platforms.

 The core of the voice UX is to make sure users find actual value in what is offered them. Based on an adaptation of the renowned Peter Morville's User UX Honeycomb model, we can establish -- and propose for a wider discussion -- a tentative voice app (v-app) quality metrics framework as follows:
  1. Is the v-app useful? This question tries to establish whether the content is original and satisfies a user's genuine need. In other words, the v-app design should actually present some innovation in functionality versus other comparable products. It should enable the user to achieve practical goals in a better way compared to what the other existing solutions would allow. 
  2. Is the v-app usable? Here we try to measure the overall ease of use of a given v-app. We obviously assume that v-app should work as claimed, without malfunctions.
  3. Is the v-app desirable? This question relates to those properties of a design that are deemed to trigger positive emotions and enjoyment on the user's end. In other words, a user should like the way a given v-app works in comparison to the existing solutions. 
  4. Does the v-app offer easily navigable content? This is all about being intuitive and natural, that is, as much as possible close to the user's spontaneous conversational expectations. 
  5. Is the v-app accessible for the people with disability? This is also a crucial aspect of a v-app. It is about a design that increments all the other qualities without making steeper the learning curve for people with some level of physical and/or cognitive challenge. 
  6. Is the v-app credible? This question relates to both which content a given v-app offers and the way the content is presented to the user. Credibility becomes of enormous relevance particularly when a content is used to support decision making in a number of critical domains such as health, diet, finances, legal issues, etc. . 
 For all the above mentioned 6 measurement criteria, we propose to adopt a standard rating scale from 1 (minimum) to 10 (maximum). Such a wide scale might appear to complicate the overall evaluation processes. However, we think that it is worth the effort because it allows to better capture nuances that down the road could generate unexpected dynamics in the users behavior.

 We invite all the interested parties to intervene and help to test and refine as much as possible this voice UX quality metrics framework (QMF). We hope and believe that such initiative will assist users, voice UX designers and developers to better approach this emerging human-machine relationship experience as it expands increasingly into many domains and their related products and services. 

Comments

Post a Comment

Popular posts from this blog

Hitler’s Circle Of Evil: A Great Documentary to Watch

If you still think that Nazism was a bolt from the blue and that Hitler and his gang were parachuted from another planet, think again. Watch Hitler’s Circle Of Evil to learn how Nazism has grown deep through the German democratic institutions with significant financial support by the usual "hidden hands" to become one of the major nightmares of the XX century that left behind 55 million dead. This documentary is a crucial wake-up call for those wishful thinkers who still dream that the US, and more in general the Western democracies, are vaccinated against such a risk. Just carefully watch the documentary then try to put in perspective many US administrations since the end of World War II, in particular the current Trump's administration. You would be damn surprised to find out that Germany indeed lost the war but Nazism as ideology in its deepest criminal plans continued to our days...underneath... and that it can reappear publicly on the political scene anytime. The Cor...
Read more

Biden-Harris vs Trump-Pence: Where Are We Headed?

Image
I believe that we must distinguish between two analytical levels: Reporting on the facts and tactics used by the parties involved, on the one hand, and lasting and strategic consequences for the US political system, on the other. Regarding tactics, the situation is quite dynamic, and legal specialists are essentially divided on what the American Constitution would allow, especially regarding the legal loopholes that Trump's supporters would like or simply could use to try to reverse the situation on the ground. A likely and highly risky scenario for the Biden-Harris ticket is that Trump's campaign operatives could be able to force the adoption of what is established by the Amendment 12 of the US Constitution whereby the states through their members in Congress will choose the president and vice-president based on one vote per state. Most probably all of this would result inevitably in a victory for the Republican side as they control the majority of St ate delegations in Congre...
Read more

California Wildfires: Is this all about climate change?

Image
Golden State Burning The short answer is a big NO! A longer one needs a few more words. There is currently an overall consensus that the electrical lines running above ground are the main culprit of several of the ongoing disastrous wildfires across the Golden State. According to Wikipedia, the state of California has 26,000 miles of high voltage transmission lines, and 240,000 miles of distribution lines. Distribution lines bring electricity directly to homes: Around 66% of them statewide are above ground. That is a dramatic picture indeed. 66% of California's statewide distribution lines are above ground. For transmission lines, the cost of under-grounding is about $80 million per mile while for distribution lines, the same expense is about $3 million per mile. All that compared to overhead lines at about $800,000 per mile. The state's largest utility, PG&E, has 107,000 miles of distribution lines, 81,000 miles of which are overhead. The cost to convert all of PG&E...
Read more