Usability Evaluations

"Every CEO, manager, designer and developer should observe REAL PEOPLE using their software product."

Your software products today cannot only be functionally complete - today's products must also provide an intuitive, usable, and enjoyable user experience for your customers and site visitors. PC GUI applications, Websites and Web applications can be evaluated using existing users, new/potential users and other representative users. Usability evaluations often unearth many usability (and functional) issues that may not yet have been uncovered with other functional or quality assurance tests and evaluations.

Usability evaluations can be conducted remotely, at specified locations world-wide, or at a client's preferred location. Evaluations can be very informal and take only a few minutes or hours, or they can be quite extensive and take many hours or days.

The number of participants can range from 5-10 or as many as 20-30, depending on the user demographics, tasks, and the degree of statistical reliability needed. Most evaluations are structured so that each participant works individually on a specific set of product or site tasks.

Request a usability evaluation proposal now!

Usability Evaluation Methodology

A usability evaluation is a systematic design assessment conducted in a controlled setting. This evaluation approach provides participants with an environment similar to what they use for the product in their home or office. Each participant was asked to perform a predetermined set of tasks during a specified time period.

A usability evaluation often follows a “think out loud” usability methodology. While performing typical user tasks, participants are encouraged to talk about what they are thinking, their feelings and opinions about how they use the system, how the system works, and whether the system performs as expected.

An administrator logs subjective user comments about the system, along with task begin and end times, completion status (successful or not), and assistance needed to enable the participant to continue or complete a task. The participants’ screen activities, physical activities, and speech are usually recorded on audio- or videotape. All of these records allow the administrator to collect and later analyze evaluation data that accurately reflects each participants’ performance and attitude.

Usability evaluations typically measure these characteristics:

• Task completion (success or failure)
• Task time
• User errors (frequency and severity)
• Assistance needed
• Subjective satisfaction

Some of the key activities conducted by Theo Mandel in product usability evaluations include:

• Define business and product goals and objectives
• Define user requirements, goals, and objectives
• Incorporate iterative usability testing into the software development cycle
• Determine usability test iterations and task-based user scenarios
• Plan, conduct, analyze and report usability tests of product designs, prototypes and products under development
• Conduct usability evaluations and reviews of competitive products on a regular or as-needed basis

Usability Labs

Many usability evaluations are conducted at in-house or commercial usability labs. Evaluations can also be conducted with portable or rented equipment in accessible areas such as large conference rooms. Dr. Mandel has built usability labs at client locations and has built portable usability labs for clients.

The diagram below shows a common floor plan layout for a usability lab. On the far right, the participant area allows users to work in a comfortable environment similar to their home or office. This room has a separate entrance. The central control room is separated from the participant area by a one-way sound-proof glass mirror. Here the evaluation administrator, data logger, and equipment supervisor can observe and control the usability evaluation. Workers in the control room can hear and see what is being said and done in the participant area using microphones and cameras. A second computer monitor or TV with scan converter allows the secondary display of the participant's monitor screen. An intercom allows communication with the participant. There is usually an additional observer area for product managers and developers to observe the usability evaluation session. This room has TV monitors or mirrors that allow visitors to hear and see what is going on in the participant area.