ideas worth stealingThe right time for usability testing
Part of believing that people adapt to technology is realizing that we also need to understand how people adapt to technology. So when something new comes up, or when we come up with something new ourselves, there’s a lot of value in running usability tests to determine how easily people will adapt to it, if they will adapt at all. There’s value in finding out if it’s desirable. And there’s also value in the things you can learn from usability tests that you never intended to learn, like the fact that they keep ignoring the giant Help button.
But some companies test almost religiously. Once they decide to start caring about user experiences, they start running usability tests constantly because they think it’s the only way to learn anything and be sure about it.
What they typically find out, however, are the exact things a good designer could have told them without pestering eight of the company’s most valued customers. A good designer should be able to tell you most of what’s wrong with an application once its built as well as how to fix it, and should be able to design something that prevents most of the issues in the first place.
Instead of spinning your wheels researching things your designer should already know, save your strength for the strange things. The things no one else has done before. Test those.
Spend the rest of your time finding a good designer and keeping him happy.
Posted by Robert on January 16th, 2007
2 comments
I wonder if overtesting is another example of decision paralysis, risk aversion and CYA in bloated organizations. Nobody is going to get fired or sued for introducing a new feature that flops but “tested well” but it comes at the price of a slow development cycle. It seems sort of the opposite extreme from consensus-based “design”-by-committee where nothing is tested but as long as everyone agrees on it nobody is responsible when it flops.
Incidentally, have you come across any research about the real incremental value of formal user testing over expert review (i.e. a designer critically looking at it)? All the reviews I have seen show RoI for usability but it’s never compared to expert review.
There is also the danger that poorly-done usability tests are worse than expert reviews in terms of ending up with really elegant solutions due to the Blink effect and removal of context. A further caution is treating usable=perfect, whereby decision-makers might think “This is easy to use, therefore users will use it and like it” which is not true.
A lot of testing, from what I\’ve seen, is driven by politics. Employers who may *say* they trust their deign team, for example, may still have testing done to prove the designers know what they\’re talking about. But yes, many people don\’t realize that usable does not necessarily equal good, elegant, or desirable.
And I definitely agree that in many cases, expert reviews can be more effective than a testing session. Hence, my suggestion that testing should be reserved for atypical cases.