Pilot Evaluation Criteria Log

Template Overview

Enterprise pilots often languish without clear decision milestones, turning into indefinite free trials. The Pilot Evaluation Criteria Log establishes mutual commitment by defining upfront technical criteria, tracking cohort testing milestones, and securing written agreements to purchase upon criteria satisfaction.

Markdown Template Structure

Copy the block below or click Download .md File to add this template directly to your personal vault:

---
title: "Pilot Evaluation Plan: [Client Organization Name]"
date: [YYYY-MM-DD]
sales_engineer: "[[Sales Engineer]]"
account_executive: "[[Account Executive]]"
client_lead: "[Client IT Architect / Champion]"
evaluation_period: "14-Day Evaluated Proof of Concept"
pilot_seats: 30
target_contract_value: "$54,000 / year"
pilot_status: "Active Testing / Day 6"
tags: [poc, pilot, evaluation-criteria, sales-engineering, enterprise]
---

# Pilot Evaluation Criteria Log — [Client Organization Name]

## 1. Mutual Success Agreement & Commercial Trigger
> [!IMPORTANT] Commercial Conversion Agreement
> Upon successful validation of the 4 technical criteria outlined below, the client agrees to execute the enterprise order form for 350 seats within 10 business days of pilot completion.

## 2. Agreed Technical & Functional Evaluation Criteria
| # | Evaluation Criterion | Measurement Method | Verification Result | Sign-Off |
|---|---|---|---|---|
| **C1** | Zero cloud audio egress during meetings | Little Snitch / Wireshark packet trace | **Verified Pass** (0 bytes leaked) | [[Client SecLead]] |
| **C2** | Sub-60s summary generation on M-series Mac | Local benchmark across 45-min recording | **Verified Pass** (Avg 34s) | [[Champion]] |
| **C3** | Direct open in Obsidian without conversion | Open generated `.md` note in Obsidian | **Verified Pass** (Syntax clean) | [[Champion]] |
| **C4** | Silent Jamf Pro MDM package deployment | Push installer to 5 pilot laptops | **In Progress** (Scheduled Thu) | [[Mac Admin]] |

## 3. Pilot Cohort Health & Engagement Tracking
- **Active Cohort Size:** 30 users (20 software engineers, 5 product managers, 5 executive leads).
- **Cohort Meetings Recorded:** 74 meetings recorded during the first 6 days of evaluation.
- **Top Qualitative Feedback:** "The live transcription accuracy on technical engineering terms was significantly higher than cloud tools."

## 4. Open Pilot Questions & Resolutions
- **Question:** How do users manage local Whisper model updates?
  - *Resolution:* Demonstrated the in-app Model Download manager and explained manual offline deployment methods for air-gapped devices.

## 5. Next Steps to Pilot Conclusion
- [ ] Complete Jamf Pro silent deployment test on Thursday morning.
- [ ] Conduct final pilot debrief review on [YYYY-MM-DD].
- [ ] Deliver executable Order Form to client procurement lead.

Filled Real-World Example

Here is a realistic example demonstrating how this template captures actual deliberations:

---
title: "Pilot Evaluation: Quantum Therapeutics"
date: 2026-10-05
sales_engineer: "[[Devin Zhao]]"
account_executive: "[[Rachel Adams]]"
client_name: "Quantum Therapeutics"
tags: [poc, pilot, enterprise]
---

# Pilot Evaluation: Quantum Therapeutics

## 1. Pilot Status
Devin Zhao completed Day 10 of the pilot with Quantum Therapeutics. All 4 technical criteria passed verification. Client champion confirmed procurement authorization has been submitted to the CFO.

When to Use and When to Avoid

When to Use

• Use when launching time-boxed proof-of-concept (POC) evaluations with enterprise prospects.
• Use to establish binding written technical criteria before provisioning software access.
• Use to track weekly user cohort testing health and unblock technical evaluation questions.

When to Avoid

• Avoid using for unstructured informal product trials.
• Avoid using for standard self-serve consumer credit card purchases.

On-Device AI Integration

Summarize your recording in Oyma with this local instruction:

Analyze the pilot evaluation review meeting recording:
1. Extract agreed technical criteria and confirm the verification pass/fail status of each.
2. Outline cohort testing engagement metrics and user feedback quotes.
3. List remaining technical blockers and contract execution dates into clean Markdown.

Your data remains securely on your Mac with zero cloud exposure.