Lesson 05: Usability and Evaluation

Lesson Objectives

Usability testing and evaluation is a vital discipline. Testing should be done early and often in the design process. It can providing the designers with feedback on user satisfaction and find errors in the systems design in a timely manner.

After reading this lesson, you should be able to:

• Identify the attributes of usability.
• Describe approaches to usability testing and evaluation.

Usability

An important part of the user centered design process is the incorporation of usability testing during both the design and evaluation stage of the systems or interface development. Usability refers to how easy an information technology is to use. Useful information technologies must be functional, useful, and learnable by humans.

Usability testing is important for:

1. Demonstrating the strengths and weaknesses of a design process and product. Usability testing includes collecting data that can be used to improve and redesign the interface.
2. Evaluating the overall design and specific system features. For example, HCI workers may test if users prefer a command line or menu interface.
3. Assessing the functionality of the system for a particular organization or set of users.
4. Validating the effectiveness and efficiency of the system, including potential productivity gains.
5. Providing the system designers with feedback on user satisfaction.
6. Identifying errors or mistakes in the systems design.

Usability Testing

Usability workers systematically and iteratively test each aspect of the system to improve systems design. Usability testing is a key part of iterative design techniques.

Systems are tested to see whether they fulfill the user's goals and provide feedback to the user on their actions taken and results. Every system or interface developed by industry involves some level of usability testing. Many industry accidents occur due to poorly designed information technologies.

The goal of usability testing is also to identify users' problems with the system, enable the users to provide the systems designers with feedback, and evaluate the performance of the system.

Formative and Summative Evaluation

Usability testing can occur at any time during the design process. User testing should be done early and often with real users.

There are different approaches and terms relating to user testing:

Formative evaluation: This occurs in order to assist IT designers in forming and refining their designs. Specific problems are identified during the design process. This is part of the iterative design process. The formative evaluation stage could include a think-aloud session where the user verbalizes their thoughts, choices, and questions to the evaluator. Evaluation is more likely to be done in person with direct observation. Video or audio recording of the user's interactions may be done.

Summative evaluation: This follows usability testing. The overall effectiveness and impact of the system is summarized. This may include a test between two or more alternatives. Statistical differences between features may be summarized and compared. Evaluation may be done remotely.

Alpha Testing: This usually is internal testing. The prototype that is developed is evaluated by internal users.

Beta Testing: This is usually available to external users. The prototype is made available to be evaluated by external users.

Usability Testing Tasks

Usability testing tasks include:

1. Analyzing users' interaction with the system, for example, users' keystrokes and interaction history, eye movements, and patterns of use.
2. Conducting experiments in which users perform system interaction tasks and think aloud (talk out loud about their actions and tasks) as they interact with the system.
3. Measuring the time users take per task, their error rates, and their level of satisfaction with the system.
4. Recording users' interaction with the system via paper forms, audio taping, or video taping to examine users' problems, errors, or interaction effectiveness.
5. Surveying users using a questionnaire or interview regarding their satisfaction with the system.

Usability testing includes the ethical concerns of respecting a user's mental and physical well-being and privacy. Evaluation workers must obtain the informed consent of volunteer participants before beginning usability testing. Usability workers must also be concerned not to bias users when conducting evaluation testing.

Lesson Wrap-Up

As you now know, usability testing is performed to answer questions and find problems before the consequences are significant. The investment of time and resources to usability testing and evaluation is well worth it.

Now that you have completed this lesson, you should be able to:

• Identify the attributes of usability.
• Describe approaches to usability testing and evaluation.

Usability Additional Readings:

Krug, S. (2000). Don't make me think: A common sense approach to web usability. Indianapolis: New Riders.

Neilsen, J. (2000). Designing web usability. Indianapolis: New Riders.