Moderated Usability Testing Protocol

Template Overview

Observing real human beings interact with your software is the most humbling—and enlightening—experience in product design. Designers and engineers spend months perfecting interfaces, only to watch real users stumble over confusing button labels, miss critical controls, and struggle to complete basic tasks.

A Moderated Usability Testing Protocol establishes an objective testing discipline: defining realistic user scenarios, timing task execution, recording think-aloud observations, and scoring usability without leading the participant.

This template provides an end-to-end usability testing framework: test briefing scripts, task scenarios, completion scorecards, qualitative friction logs, and System Usability Scale (SUS) calculations.

Markdown Template Structure

Copy the block below or click Download .md File to add this template directly to your personal vault:

---
title: "Usability Test: [Participant ID] — [Test Round / Feature]"
date: [YYYY-MM-DD]
participant_id: "UT-08"
facilitator: "[[Lead UX Researcher]]"
observer_scribe: "[[Product Designer]]"
tested_feature: "Markdown Template Gallery & One-Click Download"
prototype_build: "v1.5.0-rc2"
overall_task_success_rate: "83% (5 of 6 Tasks)"
system_usability_score: "82.5 / 100"
tags: [usability, ux-testing, protocol, task-success, sus]
---

# Usability Test: [Participant ID] — [Test Round / Feature] — [YYYY-MM-DD]

## 1. Facilitator Briefing & Think-Aloud Framing
- **Introduction:** "Thank you for joining. Today we are testing our new Markdown template repository. We are testing the software, not you. There are no wrong answers. Please think aloud as you navigate."

## 2. Task Scenarios & Performance Scorecard
| Task # | Scenario / Prompt | Expected Path | Observed Outcome | Completion Time | Score |
| :--- | :--- | :--- | :--- | :--- | :--- |
| **Task 1** | "Find a template for an Executive Board Meeting." | Click 'Meetings' category -> Select Board Meeting | Direct success; navigated via category pill | 18 seconds | 🟢 Direct Success |
| **Task 2** | "Copy the plain Markdown template to your clipboard." | Click 'Copy Markdown Template' button | Direct success; button feedback clear | 6 seconds | 🟢 Direct Success |
| **Task 3** | "Download the template as a `.md` file to your Mac." | Click 'Download .md File' button | Direct success; file saved immediately | 8 seconds | 🟢 Direct Success |
| **Task 4** | "Find the AI prompt used to fill this template." | Scroll to 'On-Device AI Integration' section | Indirect success; scrolled past heading twice | 45 seconds | 🟡 Indirect Success |
| **Task 5** | "Navigate to a related sprint retrospective template." | Use 'Related Templates' sidebar links | Direct success; sidebar link worked well | 12 seconds | 🟢 Direct Success |
| **Task 6** | "Filter templates by '1-on-1s' using breadcrumb trail." | Click 'Templates' in breadcrumb | Direct success; breadcrumbs intuitive | 10 seconds | 🟢 Direct Success |

## 3. Think-Aloud Observations & Friction Points
- **Observation on Task 4:** The user scrolled looking for the word "Prompt" and initially overlooked the "On-Device AI Integration" heading. 
  - *Recommendation:* Update section title to "AI Prompt & Local Workflow" for faster scanning.
- **Positive Delight Moment:** The participant audibly smiled when the copy button displayed a checkmark and the text "Copied to Clipboard!".

## 4. System Usability Scale (SUS) Scoring
- **SUS Score Calculated:** 82.5 / 100 (Grade A / Excellent usability threshold).
- Participant commented: "Clean, fast, and completely distraction-free. No unnecessary popups."

## 5. Prioritized Design Action Items
- [ ] Rename 'On-Device AI Integration' heading to 'AI Prompt & On-Device Workflow'.
- [ ] Add explicit keyboard shortcut hint (`Cmd+C`) to copy button tooltip.

Filled Real-World Example

Here is a realistic example demonstrating how this template captures actual deliberations:

---
title: "Usability Test: UT-08 Template Gallery"
date: 2026-10-07
participant_id: "UT-08"
system_usability_score: "82.5"
tags: [usability, ux-testing]
---

# Usability Test: UT-08 Template Gallery — October 7, 2026

## 1. Summary
Participant completed 5 of 6 tasks with direct success. Copy and download flows functioned flawlessly.

## 2. Recommendation
Clarify AI prompt section heading to improve scanability.

When to Use and When to Avoid

When to Use

• Use during moderated usability testing rounds for new features, prototypes, or user flows.
• Use to measure task success rates, time-on-task, and standardized SUS scores.
• Use to identify friction points and usability bottlenecks before deploying to production.

When to Avoid

• Avoid using for exploratory, open-ended generative user discovery (use [[user-research-interview]]).
• Avoid using for unmoderated mass analytics tracking.

On-Device AI Integration

Capture moderated usability testing sessions with Oyma on macOS:

Analyze the usability testing session audio transcript:
1. Synthesize participant background and the tested prototype feature build.
2. Build the task scenario performance table recording expected paths, outcomes, times, and completion scores.
3. Extract qualitative think-aloud quotes and specific user interaction friction points observed.
4. Calculate or record the System Usability Scale (SUS) score and prioritized design action items.
Output in structured CommonMark format.

Your usability session recordings and participant interaction logs stay strictly on your private Mac.