Project Overview
Choosing
a Good Site
Deliverables
User Testing
Design Plan Report
Progress Report
Final Report
Group Evaluation
Download
Memo Template
|
This page presents the background
information on usability and the user testing process that you will need
to complete your Design Plan Report memo.
It includes several links to more information and some sample user test data
sheets. At the end is a more detailed list of online resources on usability
testing and Web site design.
What
is user testing?
"Usability" generally refers to how well a tool or technology helps someone
perform a task. Web site usability deals with effective user-interface design.
The Usability.gov site defines Web site user testing the following way:
- Usability testing encompasses a range
of methods for identifying how users actually interact with a prototype
or a complete site. In a typical approach, users — one at a time or two working
together — use the Web site to perform tasks, while one or more people watch,
listen, and take notes (http://usability.gov/basics/index.html).
The following online resources provide a
good introduction to usability testing basics:
What is the
process for user testing?
Usability experts generally agree on a 5 to 6 step process for usability
testing:
- Determine the goals of the study
- Develop a profile of typical users
- Write the user tasks
- Conduct user tests
- Evaluate the data
- Implement or recommend changes
Step 1: Determine the goals of the study
Your evaluation should target common tasks that users are expected to
perform on the Web site. Identify the purposes or aims of the site. What
are the intended uses of the site? Why do people come to the site? What
tasks might they attempt to perform? Break these uses down into primary
and secondary tasks. For example, a travel Web site’s primary function is
airline booking, a secondary task is finding information about travel destinations.
Step 2: Determine the user profile
The goal is to match the people you user test with the target user profile.
Identify typical users’ attributes: Who will come to the Web site? What
is their age, gender, education level, etc.? What will be their knowledge
level about the Web site and its purposes? What will be their knowledge level
about the Internet in general? What other Web sites are related to this
and will the users be familiar with them? If you’re using 404 students as
your user group, justify how they fit with the user profile.
Step 3: Write the user tests
User testing involves observing actual members of the target group using
the Web site and interviewing or surveying them at key points (before, during,
and after they use the site) to gather their feedback. Most experts agree
that a well designed test of only a small group, between 5 to 10 people,
reveals any major usability issues with a Web site (see Jakob Nielsen’s
“Why You only Need to Test With 5 Users” http://www.useit.com/alertbox/20000319.html)
Once you’ve brainstormed a list of primary and secondary uses, translate
them into a list of tasks or scenarios you will ask your users to perform
during the tests. There are two basic types of user tasks:
- Directed tasks. These are specific,
close-ended tasks related to primary or secondary purposes. For example,
to study the usability of an airline’s Web site, you might ask the user to
“Book a flight from Cleveland to Phoenix for your Christmas vacation. You
want to leave on X date and return on Y date.”
- Examples:
- Undirected tasks. These are
good for general evaluations/assessments (a good choice if you notice lots
of problems with a Web site). Undirected tasks are more open-ended. They
are scenarios designed to mirror how intended users might typically use the
site, such as: “You’re at home, browsing the internet, interested in finding
the latest news about Afghanistan. You decide to use the latest search engine
being promoted by your ISP.” Once the user is finished with the task, you
can collect their opinion about general and specific issues (see the examples
below for general Web site evaluation criteria)
- Examples:
After you have developed your user tasks/scenarios,
decide on a method for collecting user feedback. Choose either a written
protocol (distributed to user who writes own feedback) or an interview
protocol (answers recorded by researcher) or a combination of
both.
Make sure you plan on asking enough of the right questions before, during,
and after your user tests:
- Pre-test questions: Used to
collect basic demographic information and possibly information related to
purposes of your Web site (e.g., if it's an auto manufacturer's Web site,
ask users if they have ever visited such a site before). Example pre-test
questions:
-
- Age?
- Gender?
- Previous Computer Experience?
- How often do you access
the World Wide Web?
- For what length of time
do you typically use the WWW?
|
- For what purposes do you
use the WWW?
- Have you ever used to the
WWW to investigate products you wish to buy?
- Have you ever purchased
anything over the WWW?
- Have you ever purchased
anything you saw advertised on the WWW?
|
- In-test questions: Can be used
to prompt users' reactions to specific tasks:
-
- What are you doing now?
Why?
- What is the first thing
you’re attracted to?
- What is [particular feature,
e.g., "background color"] like?
|
- NOTE: some of the post-test
questions below might work better as in-test questions, given during or
immediately after specific tasks. See also D’Hertefelt’s article on “obtrusive
observation” techniques below
|
- Post-test questions: Prompt
users' overall reactions to the test and Web site:
-
- What was your general impression
of each page?
- What problems did you have
using the page and why?
- What do you believe caused
this problem?
- What, if anything, helped
you find the information you were looking for?
|
- What, if anything, made
finding the information more difficult?
- What information would
you like to see on the page that was missing?
- Was the page easy to use?
- Rate the pages you saw--which
was best? Give three reasons
|
Lastly, check to see if you want to use
any other methods for collecting data:
- Think-aloud protocol. Records
what users say while performing tasks and verbalizing decision making process
involved in performing a certain directed or undirected task.
- Observation. Records users
actions, e.g., where user clicks, where they move mouse, what their page-to-page
navigation is (can use images of site/page to diagram). You can also record
how much time users take on each task and the whole test.
The following are samples of user test protocols
(or plans for conducting user tests) used by students and professionals:
Step 4: Conduct the tests
This is where you get to actually see how users perform your tasks.
Keep Keith Instone’s advice in mind as you conduct your tests: “watch and
learn” (see http://www.webreview.com/1997/04_25/strategists/04_25_97_1.shtml).
That is, don’t interfere with the users as they attempt to perform the tasks.
Avoid giving verbal or visual cues that might influence the user’s interaction
with the Web site. Do not give them any background information or help
them if they get stuck. That’s one thing you’re looking for, if they get
stuck. Take notes—write down what they say, what they get stuck on, what
links they use, how much time it takes to perform tasks, etc. Ask follow
up questions about what they had trouble with, their overall opinion of
the site, etc. Use the questions from the protocol you developed beforehand.
At the end of the test, you can demonstrate for users what they had trouble
figuring out.
Step 5: Evaluate the Data
The most important skill of the usability consultant is the ability
to recognize patterns (see Jakob Nielsen’s “Becoming a Usability Professional”
http://useit.com/alertbox/20020722.html).
As researchers and usability analysts, it’s your team’s task to make sense
of the results of your user testing. You should first compile your findings
(one of the sections of your final report will be a “findings” section).
You can group your findings by tasks and by answers to protocol questions.
Once you compile the data, you’ll want to look for significant patterns:
what did most users have the most difficulty with? How severe were their
difficulties? How satisfied were they with aspects you were testing? Once
you start to ask questions like this, then you can begin to brainstorm solutions
or fixes (your final recommendations). As you start to discover patterns
in the results of your user tests, consult Web site design principles for
expert opinions about how to improve these elements.
Keith Instone’s “First User Test” article is an excellent discussion
of the findings and recommendations of his evaluation of the Internet Travel
Network Web site http://www.webreview.com/1997/05_30/strategists/05_30_97_8.shtml.
Step 6: Implement or Recommend Changes
Most usability consultants can implement recommendations firsthand.
Your team will have to instead write up your recommendations in a credible
and persuasive technical report. Again, Keith Instone’s “First User Test”
article is an excellent example of how detailed your recommendations should
be (notice that Instone makes three recommendations, all of which are linked
to a more detailed discussion http://www.webreview.com/1997/05_30/strategists/05_30_97_8.shtml).
Online
resources
Usability Testing
Web Design
home | course
info | projects | schedule | resources
| jablonski
<http://complabs.nevada.edu/~jeff_jablonski/404/procedures_project.html>
|
