# How to Run a Think-Aloud Choosing tasks and running a think-aloud study of your prototype cse482a: Fall 2026; Jennifer Mankoff (Last Edited: 2026-11-02). <a href="/courses/cse482a/26au/slides/think-aloud.html">Live View: /slides/think-aloud.html</a> Important Reminder: check zoom & captioning <a href="/courses/cse482a/26au/slides/think-aloud.html"><img src="https://api.qrserver.com/v1/create-qr-code/?data=/slides/think-aloud.html&size=100x100" alt="QR for /courses/cse482a/26au/slides/think-aloud.html" title="" /></a> --- <div class="slide-outline" style="margin: 1em 0; padding: 1em; background: #f5f5f5; border-radius: 5px;"> <h2 style="margin-top: 0;"> Outline </h2> <ul style="list-style-type: none; padding-left: 0;"> <li><a href="#3">Milestone 3</a> (Slide 3)</li> <li><a href="#10">How to Run a Think-Aloud</a> (Slide 10)</li> </ul> </div> --- class: center, middle, inverse # Milestone 3 Study & Accessibility Findings Starts **Thu 11/5**. Due **Wed 11/18** (Demo Day in class). --- ## The big picture  Pink: Canvas. Blue: your repo. Green: your public site. Gray: in class. --- ## What you will do 1. **Plan a small formative study**: a protocol and an adapted consent form (in `LOG.md`) 2. **Recruit and consent 2-3 participants**, then submit signed consent forms in **Milestone 3 Study Consent Forms** (Canvas) 3. **Run the sessions** (think-aloud, about 30 minutes) in week 7. Keep anonymized notes 4. **Report your findings** in `study.html`: method, what you learned, what you will change 5. **Implement more of your prototype**, and say what is still missing 6. **Audit what you implemented** (keyboard, focus order, focus visible, headings) and add rows to your VPAT 7. **Demo Day:** a 5-minute readout of your study findings --- ## Checklist: what "complete" looks like - `study.html` describes your study and what you learned from it - `LOG.md` has a protocol and anonymized notes for each session (2-3 participants) - Signed study consent forms were submitted in Milestone 3 Study Consent Forms - Your prototype implements more than at Milestone 2, and says what is missing - `vpat.html` has rows for keyboard, focus order, focus visible and headings, and says what was **not tested** - `LOG.md#build` has an entry from every teammate --- ## How it is graded - **Inclusive Experiments**: how inclusive your study design is: access, consent, who you recruited - **Accessible Implementation**: the prototype and your audit - **Accessible Communication**: how clearly and accessibly you report Consent for your Milestone 1 interviews does **not** cover the study. Get consent *before* each session. Handin: tag your repo (`git tag milestone-3`), then submit the compare link `milestone-2...milestone-3`, and upload your consent forms. --- ## Class time until it is due | Day | What | |---|---| | Mon 11/9 | Accessibility stack, then **check-ins**: bring your protocol and consent form | | Wed 11/11 | No class (Veterans Day). **Run your sessions this week** | | Mon 11/16 | Last WCAG criteria, then check-ins | | Wed 11/18 | **Demo Day:** study readout | Then: **audit walkthroughs with TAs on Mon 11/23**. Teamwork reflection (due Mon 11/23). --- ## Common problems - Recruiting too late: start the week of 11/5 - Consent forms still written for interviews - A protocol with tasks that need more than 5 steps - An audit that says what you found but not what you **did not test** - Study findings that list problems but not what you will change Bring your protocol to the 11/9 check-in. --- class: center, middle, inverse # How to Run a Think-Aloud Your Milestone 3 study --- ## Plan for today | Time | What | |---|---| | 5 min | What your study needs to answer | | 15 min | Choose your tasks | | 10 min | How a think-aloud works, and what to say | | 20 min | **Practice in pairs** | | 5 min | Access, consent and next steps | | rest | Build time for Wednesday's Demo Day | You will use this in [Milestone 3](/courses/cse482a/26au/projects/milestone-3-study-and-audit/): 2-3 participants, about 30 minutes each, on your prototype. --- ## Basic question: what do you want to test? Your tasks need to be: - **Concrete and actionable** ("Enter a review for the University of Washington") - **Do-able in under 5 steps** - **Do-able using your prototype** (it helps to design with tasks in mind) --- ## Example Task: *"Enter a review for the University of Washington"* 1. Search for the University of Washington 2. Click add review 3. Select a star rating 4. Write the review 5. Assign metadata about identity or topic Notice that the task says **what to accomplish**, not which buttons to press. --- ## Next question: how do you plan to test it? - What actions will your prototype allow? - What does the **facilitator** need to be prepared to do (for example, play the part of a missing feature)? - What **artificial constraints** might you be putting on the participant? --- ## Exercise: choose your tasks (15 min) In your team: 1. Pick **one or two** of your three tasks for the study 2. Write each as a concrete task a participant could do, in **5 steps or fewer** 3. For each: what does your prototype let them do? What will the facilitator do or say if it doesn't? 4. Write the tasks in the protocol in `LOG.md` (template in [Milestone 3](/courses/cse482a/26au/projects/milestone-3-study-and-audit/)) --- ## When you run the test, use "think aloud" - Tell your participant to accomplish a **specific task** - Ask them to **say out loud what they are thinking** as they do it - **Encourage** them to keep speaking if they stop Works with "real" interfaces too, not just a lofi prototype. --- ## You want to remain neutral. Things you can say (1/2) | When this happens | Say this | |---|---| | You're not sure what the participant is thinking | "What are you thinking?", or "What are you looking at?", or "What are you doing now?" | | They seem surprised or say "hmm" | "Is that what you expected to happen?" | --- ## You want to remain neutral. Things you can say (2/2) | When this happens | Say this | |---|---| | They ask you what to do | "What would you do if you were at home?" then "then try that". Or "What would you do if I was not here?" Or "Please do whatever you would normally do" | **Don't** explain the interface, defend a design choice, or hint at the "right" answer. You are there to learn, not to teach. --- ## Practice in pairs (20 min) Pair with someone from **another team**. 1. **Facilitator:** read your partner's task aloud. Use *only* the neutral phrases 2. **Participant:** try the task on the prototype (or a sketch, or a screenshot) and think aloud 3. Facilitator: take notes. What did they do? What did they say? Where did they hesitate? 4. **Swap roles** (10 min each) Afterward, tell each other: which prompt felt hardest to hold back from? ??? Circulate. Common mistakes: explaining the interface; answering "what should I do?"; asking leading questions ("Isn't that button clear?"). --- ## What access needs does your session need to address? In your prototype, focus on the specific access needs your **participants** will have. - Ask ahead of time (not at the door) what they need to take part - How will you make the session itself accessible: location, time, materials, how you communicate? - Will recording itself be a barrier for anyone? Your implementation will go further than your prototype, within reason. --- ## Next steps - Draft your **protocol** (goal, participants, tasks, access, what you record, consent) in `LOG.md` - Adapt your **consent form**; consent must come *before* each session, and signed forms are submitted in Milestone 3 Study Consent Forms - Start **recruiting** (2-3 participants). Run sessions in **week 7** - Bring your protocol and consent form to **Monday 11/9** check-ins Wednesday: **Milestone 2 Demo Day**.