In another session at the Penn State Web conference with Steve Krug, focused more specifically on usability testing.
Here's what works (in his opinion):
--simple, frequent, iterative user testing (i.e. watch people try to use what you build)
Reinforces his idea of effective testing:
--A morning a month
--Three or four users is enough if you iterate (references Nielsen--says it depends on what you're testing--but you can get enough serious problems to work on for a month if you only use 3 or 4 testers.)
--Get fresh users every time. Can people who know almost nothing about this figure this out? This means we do not need to worry about usability testing the expert users.
--We tend to think that people from our target audience know more than they do. You must test users who bring a fresh perspective to our site.
--No lab; no mirrors; recruit loosely
--Recommends recording with Camtasia, Morae or CamStudio
--No big report--just watch the sessions live---and talk about it over lunch. Get stakeholders to come--anyone who is interested.
--Use Camtasia recordings during debrief and discussion.
Testing specifics
3 or 4 people; one hour each; have them look at your home page and tell you verbally what they make of it
--Have them try to do several pre-specified tasks
--Do think aloud protocol (hello, Eric N.!) (Remember this is qualitiative testing where you are in it for the insights.)
--Have them do the same process with one or two competitors' sites. (Great idea!)
Live test
(Steve tests the prospective students portion of the Penn State web site)
Here's the script he used during this test.
Started off by having the test subject give her thoughts on the top-level Penn State home page. (She was too kind.)
I'm now flashing back to my CAT usability testing and remembering why this stuff is like watching paint dry.
(This is actually is sort of funny. The test subject is trying to find Penn State University Park--and UP is not listed on the campuses page as a campus!)
Post-Test
Steve recommends using a USB microphone for testing.
He also recommends:
--Writing tasks that avoid words that are already used on the screen in question.
(Bad: Customize your LAUNCHcast station. Better: Choose the kind of music you want to listen to.)
When fixing usability problems, your motto should be:
"What's the smallest change we can make that might solve the problem?"
Steve recommends Letting Go of Words and a Step by Step Guide for Usability Testing.
Here's what works (in his opinion):
--simple, frequent, iterative user testing (i.e. watch people try to use what you build)
Reinforces his idea of effective testing:
--A morning a month
--Three or four users is enough if you iterate (references Nielsen--says it depends on what you're testing--but you can get enough serious problems to work on for a month if you only use 3 or 4 testers.)
--Get fresh users every time. Can people who know almost nothing about this figure this out? This means we do not need to worry about usability testing the expert users.
--We tend to think that people from our target audience know more than they do. You must test users who bring a fresh perspective to our site.
--No lab; no mirrors; recruit loosely
--Recommends recording with Camtasia, Morae or CamStudio
--No big report--just watch the sessions live---and talk about it over lunch. Get stakeholders to come--anyone who is interested.
--Use Camtasia recordings during debrief and discussion.
Testing specifics
3 or 4 people; one hour each; have them look at your home page and tell you verbally what they make of it
--Have them try to do several pre-specified tasks
--Do think aloud protocol (hello, Eric N.!) (Remember this is qualitiative testing where you are in it for the insights.)
--Have them do the same process with one or two competitors' sites. (Great idea!)
Live test
(Steve tests the prospective students portion of the Penn State web site)
Here's the script he used during this test.
Started off by having the test subject give her thoughts on the top-level Penn State home page. (She was too kind.)
I'm now flashing back to my CAT usability testing and remembering why this stuff is like watching paint dry.
(This is actually is sort of funny. The test subject is trying to find Penn State University Park--and UP is not listed on the campuses page as a campus!)
Post-Test
Steve recommends using a USB microphone for testing.
He also recommends:
--Writing tasks that avoid words that are already used on the screen in question.
(Bad: Customize your LAUNCHcast station. Better: Choose the kind of music you want to listen to.)
When fixing usability problems, your motto should be:
"What's the smallest change we can make that might solve the problem?"
Steve recommends Letting Go of Words and a Step by Step Guide for Usability Testing.
Leave a comment