UX Benchmarking Scorecard

Template Overview

Rigorous product design requires objective, quantitative benchmarks to track usability progress across major product releases. The UX Benchmarking Scorecard provides a standardized framework to record task completion rates, Single Ease Question (SEQ) ratings, System Usability Scale (SUS) distributions, and task completion durations. By establishing historical baselines, product teams can prove whether redesigns objectively reduce cognitive load and friction.

Markdown Template Structure

Copy the block below or click Download .md File to add this template directly to your personal vault:

---
title: "UX Benchmark: [Product / Release Version]"
date: [YYYY-MM-DD]
benchmarking_cycle: "[e.g., Q4 Baseline / Post-Redesign]"
researcher: "[[Senior UX Researcher]]"
sample_size: [Number of Participants]
target_sus_score: 80.0
tags: [ux-research, benchmarking, sus, seq, quantitative-ux, metrics]
---

# UX Benchmarking Scorecard — [Product / Version]

## 1. Executive Summary & Benchmark Goals
- **Objective:** Evaluate core task usability metrics following the version release to determine if usability meets industry top-quartile standards (SUS ≥ 80).
- **Core Tested Scenarios:** Task 1 (Account Setup), Task 2 (Project Creation), Task 3 (Data Export).
- **Overall System Usability Scale (SUS):** [XX.X] / 100 (Industry Average: 68.0).

## 2. Standardized Task Benchmark Table
| Task # | Task Scenario Description | Completion Rate (%) | Mean Time on Task (s) | Single Ease Question (SEQ 1-7) | Error Rate per User |
|---|---|---|---|---|---|
| T-01 | Create and configure new workspace | [XX]% | [XX]s | [X.X] / 7 | [X.X] |
| T-02 | Import 100 markdown records | [XX]% | [XX]s | [X.X] / 7 | [X.X] |
| T-03 | Configure local sync encryption key | [XX]% | [XX]s | [X.X] / 7 | [X.X] |
| T-04 | Execute complex full-text search | [XX]% | [XX]s | [X.X] / 7 | [X.X] |

## 3. System Usability Scale (SUS) Breakdown
*Scale 1 (Strongly Disagree) to 5 (Strongly Agree)*
1. I think that I would like to use this system frequently: `[Score]`
2. I found the system unnecessarily complex: `[Score]`
3. I thought the system was easy to use: `[Score]`
4. I think that I would need the support of a technical person: `[Score]`
5. I found the various functions were well integrated: `[Score]`
6. I thought there was too much inconsistency: `[Score]`
7. Most people would learn to use this system very quickly: `[Score]`
8. I found the system very cumbersome to use: `[Score]`
9. I felt very confident using the system: `[Score]`
10. I needed to learn a lot before I could get going: `[Score]`
- **Calculated Composite SUS:** `[XX.X]` (Grade: `[A / B / C / D]`)

## 4. Longitudinal Benchmark Comparison
| Metric Tracked | Previous Baseline (v1.0) | Current Release (v2.0) | Relative Delta (%) | Target Status |
|---|---|---|---|---|
| Composite SUS Score | 68.4 | [XX.X] | +[XX]% | [Met / In Progress] |
| Mean Task Duration | 142s | [XX]s | -[XX]% | [Met / In Progress] |
| Unassisted Task Success | 74% | [XX]% | +[XX]% | [Met / In Progress] |

## 5. Prioritized UX Interventions
- [ ] Streamline configuration step in Task 3 where SEQ score dipped below 5.0.
- [ ] Enhance inline error recovery messaging during bulk markdown data imports.
- [ ] Conduct targeted micro-copy audit on cryptic encryption settings terminology.

Filled Real-World Example

Here is a realistic example demonstrating how this template captures actual deliberations:

---
title: "UX Benchmark: Oyma Desktop v2.4 Release"
date: 2026-10-09
benchmarking_cycle: "v2.4 Post-Launch Benchmark"
researcher: "[[Dr. Henrik Lindqvist]]"
sample_size: 20
target_sus_score: 82.0
tags: [ux-research, benchmarking, sus, seq, quantitative-ux, metrics]
---

# UX Benchmarking Scorecard — Oyma Desktop v2.4 Release

## 1. Executive Summary & Benchmark Goals
- **Objective:** Quantify usability improvements following the on-device transcription overhaul in release v2.4.
- **Core Scenarios:** Audio recording initiation, instant transcription generation, and local Markdown export.
- **Overall System Usability Scale (SUS):** 84.5 / 100 (Grade A, Top 10th percentile).

## 2. Standardized Task Benchmark Table
| Task # | Task Scenario Description | Completion Rate (%) | Mean Time on Task (s) | Single Ease Question (SEQ 1-7) | Error Rate per User |
|---|---|---|---|---|---|
| T-01 | Start one-click microphone recording | 100% | 3.2s | 6.8 / 7 | 0.05 |
| T-02 | Select local Whisper model weight | 95% | 12.4s | 6.1 / 7 | 0.15 |
| T-03 | Export structured Markdown to Obsidian vault | 90% | 8.8s | 6.4 / 7 | 0.20 |
| T-04 | Search historical transcripts offline | 100% | 4.1s | 6.7 / 7 | 0.00 |

## 3. System Usability Scale (SUS) Breakdown
- Positive Items Composite: 4.4 / 5.0 average across ease of use and rapid onboarding.
- Negative Items Composite: 1.3 / 5.0 average across complexity and technical assistance requirements.
- **Calculated Composite SUS:** 84.5 (Percentile Rank: 94th percentile).

## 4. Longitudinal Benchmark Comparison
| Metric Tracked | Previous Baseline (v2.0) | Current Release (v2.4) | Relative Delta (%) | Target Status |
|---|---|---|---|---|
| Composite SUS Score | 73.2 | 84.5 | +15.4% | Met |
| Mean Task Duration | 48s | 28s | -41.6% | Met |
| Unassisted Task Success | 81% | 96% | +18.5% | Met |

## 5. Prioritized UX Interventions
- [ ] Clarify model weight download status indicator in settings panel (Task 2).
- [ ] Add drag-and-drop vault destination selector to shave 3 seconds off export setup.
- [ ] Schedule Q1 2027 benchmarking study for mobile companion app release.

When to Use and When to Avoid

When to Use

• Use at recurring milestones (e.g., quarterly or major version releases) to measure longitudinal design progress.
• Use when reporting objective UX health metrics to executive leadership and product stakeholders.
• Use to compare the usability of redesigned workflows against historical product baselines.

When to Avoid

• Avoid using in early exploratory phases where task workflows and UI concepts are still fluid.
• Avoid using when sample sizes are under 8 participants (insufficient for statistical benchmark rigor).

On-Device AI Integration

Summarize your recording in Oyma with this local instruction:

Analyze the UX benchmarking scorecard:
1. Calculate the composite SUS score using standard odd/even item formula.
2. Flag tasks where Single Ease Question (SEQ) falls below 5.5 or error rate exceeds 0.3.
3. Compare relative deltas against prior baselines and summarize key return on investment (ROI).
Output in clean Markdown.

Your data remains securely on your Mac with zero cloud exposure.