MYP Design Usability Testing Notes

Usability Testing Means Watching, Not Asking

1. A usability test hands one real user a task on your design and records what actually happens while you say as little as possible.
2. Asking "do you like it" gets you politeness, while setting a task gets you behaviour you can count.
3. Testers should match your persona, so a product aimed at year 7 students cannot be tested only on your design teacher and your best friend.
4. Three tasks in about fifteen minutes is a sensible session, since eight tasks exhausts everybody and blurs the results.
5. Test whatever exists, because paper, a clickable prototype and a finished build all surface real problems.

Write Tasks, Not Questions

1. A task is a goal with no instructions hidden inside it, such as finding out whether the library has a particular book and when it comes back.
2. Never name a button in the task, because "tap Search and type the title" tests whether they can follow orders.
3. Give the task a reason so it feels real, such as needing the book for tomorrow's lesson.
4. Write the success condition before the session, so you are not inventing the pass mark once you have seen the result.
5. Order the tasks so an early one does not teach the answer to a later one.

Think Aloud Turns Silence Into Data

1. In the think aloud protocol you ask the tester to keep saying what they are looking for, expecting and thinking as they go.
2. It exposes the reasoning behind a wrong tap, for instance "I thought the star meant save it, not rate it".
3. People go silent when they concentrate, so prompt with "what are you thinking now" rather than with a hint.
4. Answer no questions during the task, and use the standard reply: "what would you do if I was not sitting here".
5. Save your own questions for a debrief at the end, when asking them can no longer contaminate the test.

Measure Four Things Every Session

1. Task completion is the headline result, recorded as finished, finished with difficulty, or gave up.
2. Time on task runs on a stopwatch from the moment you stop reading the task out to the moment they succeed or quit.
3. Errors are wrong taps, wrong paths and text typed in the wrong box, counted as a tally rather than judged.
4. Hesitations are pauses longer than about three seconds, and where they land is usually the most useful thing you take away.
5. Note the first click as well, because the first tap reveals what the tester thought the screen was offering.
6. Ask for a 1 to 5 difficulty rating straight after each task, which gives you one comparable number per task per person.

Five Users Find Most of the Problems

1. Testing with about five users uncovers the large majority of usability problems in a design.
2. The reason is overlap, since the sixth and seventh testers keep walking into the same walls the first five already found.
3. Three rounds of five beats one round of fifteen, because the rounds give you space to fix things in between.
4. It is five per user group, so a product with two genuinely different personas needs two sets of five.
5. Five people is enough to find problems and nowhere near enough to prove a preference, so never report percentages off five testers.

Findings Are Worthless Until Something Changes

1. Write the session up within an hour, because the detail you did not note down disappears quickly.
2. Combine findings from all testers into one list and put a number next to each problem for how many people hit it.
3. Sort by severity: blocks the task, slows the task, or merely irritates.
4. Fix the blockers first even though the small irritations are easier and far more tempting to tick off.
5. Write each change in three parts, finding, change and reason, and keep the before and after screenshots side by side.
6. Retest the changed screens with fresh testers, since anyone who struggled the first time now knows where the button is.