Conducting a rapid usability test

Last updated on 2025-07-25 | Edit this page

Estimated time: 32 minutes

Overview

Questions

  • What final set up is needed?
  • How do I find out what a participant is thinking while doing their assigned task?
  • How can I maximize insights from my small study population?

Objectives

  • Prompt participants to think-aloud
  • Assist participants with error recovery
  • Avoid giving users hints while they are completing a task

Ensuring smooth data collection

Having determined your tasks and written a script for your study sessions, prepared your participant tracker, and recruited participants, most of your work is already done!

Before each study session, send a reminder to your participants and, if appropriate, include the link to the video conferencing tool you will be using (e.g. the link to the Zoom room). In your participant tracker, mark down that you sent the reminder.

Before the session begins, make a unique copy of your script and include the anonymous ID you’ve assigned to the session participant in the title. Have this copy open so you can read along and take notes. We recommend making your notes another color like blue so you can easily distinguish what you should say aloud from your notes. Link to this copy in your tracker.

During the session, participants will need to share their screen. Encourage them to only share the needed application (e.g. a browser window if the product being evaluated is online) and when sending survey questions, tell the participant they can open the survey in a different browser window so that they have privacy when responding. Make sure these suggestions are included in your study script so you don’t forget to say them.

With small studies, getting more qualitative data (i.e. non-numerical data) can help ensure you gather rich insights and don’t rely too heavily on interpreting trends from small sample sizes. To gather this data, when orienting your participants, tell them you would like them to think-aloud while they work on tasks. This is somewhat unnatural and might slow participants down—if you are using time as a metric, you might reconsider this approach. However, if you want to get more information on participants’ impressions, it can be quite helpful. You might use prompts like, “What are you thinking?” or “Tell me what’s going through your head,” if they need reminding to think aloud.

Sometimes participants might quit a task without finishing it or may get quite lost while attempting to carry out the task. Assure them that this is okay and you are learning a lot from them. It is very important to not give any hints—you won’t be there to do that for real users. Have links and any needed information on hand to send to participants so that when they begin a new task, they can do so regardless of how successfully they carried out the previous task.

It is tempting to want to fill gaps in conversation. However, intentional silence can be a powerful technique for moderation and learning more from participants. Try to hold silences past your comfort point so that your participant fills the space and not you. Wait with a relaxed manner. If your participant asks a question, try offering a noncommittal, “Hmmm,” or, if this is during a follow up interview, you can encourage them to go on by saying, “Tell me more about that.”

Include multiple researchers

If you are a part of a team with the capacity for more than one person to help with rapid usability testing, it can be helpful to plan for different roles and responsibilities during a session. For example, you might have a moderator verbally facilitate the session, a notetaker make and record real-time observations, and a tech role to manage recording, pasting in the meeting chat, and any other troubleshooting. Allowing for multiple roles–especially if a recording is not possible–can help each individual stay more focused and present in their role.

Exercise 6: Thinking aloud

In breakout rooms, you and a partner will act as study facilitator and participant in a rapid usability test. To save time, in this scenario, we will skip some other portions of orienting participants and the usual warm up questions.

Choose your roles, have the participant share a browser window in Zoom, then have the facilitator read the script aloud and the participant engage in the assigned task. The facilitator can take notes in the place of their choice. Swap roles if you have time. We’ll discuss the exercise as a group when you’re done.

SCRIPT

Facilitator: “While you work, please think-aloud as much as you can. That just means narrating out loud what you are doing, what you’re looking for, what you’re thinking. That will help me understand your thought process. If you fall silent for a little while, I’ll remind you to think-aloud. Otherwise I’m going to stay as quiet as possible until we reach the interview portion. Any questions?”

Facilitator: “Ok. This is a link to the US-RSE website.” Send link via chat: https://us-rse.org/

Facilitator: “Your first task is to go there and find information on how you could receive a financial award from US-RSE to support your work. Let me know when you’ve found the information.” Remind participants to think-aloud if they fall silent too long while working. Take notes.

Ask the group to talk about their experiences. What went well? What were they uncertain about? Did they actually follow the script? Could they take notes while listening? What questions do they have?

After the session is over, store the recording into a private folder for raw data. Zoom will produce a transcript but services like Dovetail and Otter.ai also provide helpful audio transcription and sync the recording to the transcript itself. Review the transcript and anonymize it, removing names and identifiable information. Keep in mind that specific details about research projects can be very identifying. The Qualitative Data Repository has useful information on how to anonymize transcripts, as does this article from the UK Data Service. Save this anonymized version in a separate folder from the raw data. Link in your tracking spreadsheet to the anonymized version.

It is also recommended to go through the recording on your own to note when there are long pauses or where a user’s mouse was hovering when they were looking for a particular feature. This non-verbal information can be helpful in understanding your user’s mindset and assumptions.

Your institution may have an IRB that has additional guidance or policies on anonymization or data retention. If you are collecting data in the EU or European Economic Area or are otherwise subject to the General Data Protection Regulation (GDPR), note that after anonymization, you should remove any records that would allow you to re-identify data (e.g., identifying info in your participant tracker). UCSF has helpful information on understanding GDPR policies.

Key Points

  • To protect their privacy, participants sharing their screen should be able to share only what is needed for the study.
  • Having participants think-aloud is a good way to learn more about their reactions and opinions. It can slow them down, however, so reconsider this approach if you are using time as an evaluation metric.
  • Ensure errors from one task don’t propagate to the next by sending participants new links at the start of each task.
  • As you collect data, anonymize it and link to that data in your tracking spreadsheet.