I hate to quote so much from one article, but Lane Becker really nails some things in his article. He wrote:
Usability Testing Is Design, Not QA
Changing the approach to usability testing also changes what you should expect to get out of it. Rather than a validation done once before completing a product, this internal, qualitative usability testing is done earlier, more frequently, and as part of the design processónot separate from it.
When viewed as a sort of quality assurance, as it classically is, usability testing becomes a late-stage ďnice-to-have,Ē less important than getting the newest version of the Web site out the door. It usually results in a thick document that outlines everything thatís wrong with a Web application, including fundamental design issues that canít be fixed in the few weeks left before launch. This isnít the best way to effect positive change.
When viewed as part of the design process, the outcome is very different. Some of the most inspired work Iíve seen has happened on whiteboards in the observation room while testing is going on several feet away. In some situations weíve even made changes and applied them between rounds of testing in order to gauge the next userís response to the new solution. From a statistical point of view, this approach is a disaster; from a design point of view, itís a boon.
I can't explain how relieved I am to see someone write the statement, "Usability Testing Is Design, Not QA", and then describe the difference between classic QA and usability testing.
My swift kick into usability testing was while I was employed as a QA Engineer, testing web site applications. First, I learned the disciplines of QA, then I was told to "test for usability". But, nobody had a method for me to do this. In fact, the user interface folks and programmers were apprehensive about what I'd be doing. They thought I'd be critical of their work.
I never approached it that way. A Human Factors Phd mentor opened up my love for usability, but my admiration for the soul work behind design and code helped me provide gentle guidance, or sometimes it was best to shut up and listen to other ideas. For example, I could walk a web page mock up over to a programmer, and together, while he'd be keying in code, we'd discuss how a user might want to get from point A to point B, and ways his programming could make it more efficient.
Was it the way the user interface person imagined it? Sure. That was one way. But, the programmer had ideas too. So did the marketing department, and CEO of the company. Everyone had a stake in the final outcome. Everyone felt they knew what usability was. Eventually, the company learned to invite the usability person to meetings before the design was even started. I was always interested in what everyone had to say because people like them would be using their product later.
So, I devised test plans for usability that permitted flexibility. I studied Cooper's user personas, and their use in testing during planning stages, to get an idea of WHY we should build for the people who will use a web site or application.
I finally came to disregard measuring results, at least the way Human Factors people would approach it for software. I could do defect tracking and risk assessment, but task analysis was harder to define. People do things in a variety of ways. How do you score human preferences?
I could assign metrics for broken links, and write test cases to make sure every mock up, and every click path expectation was fulfilled. With a set of business requirements, I could pass or fail whether they were met. But, I stressed over how people were going to move from page to page.
Still, usability testing just before launch and especially afterwards is an excellent way to check the health of a web site or application. This how we suggest user enhancements or repair things like page abandonment or low newsletter signups.
I even wrote test plans for search engine optimization - something nobody else had even considered doing.
I've been the co-author of those documents "that outlines everything thatís wrong with a Web application, including fundamental design issues that canít be fixed in the few weeks left before launch. This isnít the best way to effect positive change." I know what it's like to have project managers stop talking to you because you found so much wrong with a design that the project was halted, or trashed altogether.
Better I be hated, then the entire company embarrassed for launching something that doesn't work.
Someone like Sophie, and other designers I know of, spend incredible time with the foundation aspects of design. These are the folks to hire. Chances are good these people are usability testing as they create and build.
Long live the whiteboard!