Ollama Local LLM Meeting Summaries

Introduction: Sovereign Intelligence on Modern Hardware

The rapid democratization of open-weights large language models (LLMs) has fundamentally transformed the economics and security of automated text summarization. Historically, extracting action items, key decisions, and executive briefings from conversational transcripts required sending raw text over the public internet to third-party commercial cloud APIs. For proprietary engineering organizations, legal practices, and healthcare researchers, this paradigm created insurmountable confidentiality and intellectual property risks.

With the release of advanced open-weights architectures—such as Meta’s Llama 3.1, Alibaba’s Qwen 2.5, and Mistral AI’s Nemo—paired with Apple Silicon’s unified memory architecture, running state-of-the-art inference on local workstations is no longer an experimental hobby. A contemporary MacBook Pro powered by an M3 or M4 processor possesses the unified memory bandwidth to execute 8-billion parameter models at inference speeds exceeding 40 to 60 tokens per second, completely offline.

Ollama has emerged as the premier CLI and background service for managing, quantizing, and serving open-weights models on macOS. In this architectural guide, we explore how to configure Ollama on Apple Silicon to process lengthy meeting transcripts, extract structured action items with strict JSON schema compliance, manage large context windows, and eliminate cloud API billing entirely.

---

1. Apple Silicon Hardware Mechanics: Why Unified Memory Wins

To understand why Apple Silicon dominates local LLM execution, we must examine the physical memory architecture of modern computing platforms.

Traditional x86 Workstation:
[CPU] <--- PCIe Bus (16-32 GB/s) ---> [Discrete GPU VRAM (8-24 GB)]
* High latency moving context between system RAM and GPU memory.

Apple Silicon Architecture:
[CPU Cores + Metal GPU + Neural Engine]
                     |
         [Unified Memory Controller]
                     |
       [Unified Memory Pool (18-128 GB)]
             (Up to 400-800 GB/s)
* Zero-copy memory access: CPU, GPU, and ANE read identical memory addresses directly.

In traditional PC architectures, large language models must fit into discrete GPU VRAM (typically 8 GB to 24 GB on consumer graphics cards). If a model or its context window exceeds VRAM capacity, execution degrades catastrophically as weights are swapped over the PCIe bus.

In Apple Silicon, the CPU, Metal GPU, and Apple Neural Engine (ANE) share a single pool of ultra-wide unified memory. A 36 GB or 64 GB MacBook Pro provides unified memory bandwidth ranging from 150 GB/s on base chips to over 800 GB/s on M3/M4 Max configurations. Because the GPU accesses weights directly in system RAM without PCIe bus bottlenecks, running an 8-billion parameter model quantized to 4-bit (requiring approximately 5.2 GB of RAM) leaves dozens of gigabytes available for multi-turn 128k context buffers.

---

2. Setting Up Ollama and Model Selection for Meeting Synthesis

Installing Ollama on macOS requires downloading the universal binary or executing the homebrew package manager:

# Install Ollama via Homebrew
brew install ollama

# Start the Ollama background daemon
brew services start ollama

2.1 Model Evaluation for Meeting Summarization

Not all open-weights models perform equally when confronted with messy, unstructured spoken transcripts. Audio transcripts feature colloquial speech, false starts, speaker overlap, and informal conversational drift. The model must possess:

1. Long Context Window: A 60-minute meeting transcript typically contains between 7,000 and 12,000 words (approximately 10,000 to 16,000 tokens). Models with 8k context limits will truncate the conversation.
2. Instruction Following Fidelity: The model must rigorously adhere to structured output schemas without injecting preamble fluff (“Certainly! Here is your summary:”).
3. Low Hallucination Rate: The model must never invent deadlines, names, or architectural commitments that were not verbalized.

Evaluation Matrix: Local LLMs on Apple Silicon

Model Name	Parameter Size	Quantization	Context Window	Inference Speed (M3 Pro)	Accuracy on Conversational Audio
Llama 3.1 8B Instruct	8.0 Billion	Q4_K_M (4.9 GB)	128,000 tokens	48.2 tokens/sec	Exceptional
Qwen 2.5 7B Instruct	7.6 Billion	Q4_K_M (4.7 GB)	128,000 tokens	52.1 tokens/sec	Exceptional
Mistral Nemo 12B	12.2 Billion	Q4_K_M (7.5 GB)	128,000 tokens	32.4 tokens/sec	High
Gemma 2 9B Instruct	9.2 Billion	Q4_K_M (5.8 GB)	8,192 tokens	41.5 tokens/sec	Moderate (Context Limited)
Llama 3.1 70B Instruct	70.6 Billion	Q4_K_M (42.0 GB)	128,000 tokens	6.8 tokens/sec (M3 Max)	State-of-the-Art

For most Mac workstations equipped with 18 GB to 36 GB of unified memory, Llama 3.1 8B Instruct and Qwen 2.5 7B Instruct represent the optimal sweet spot between lightning-fast execution and flawless comprehension.

# Pull the recommended model
ollama pull llama3.1:8b-instruct-q4_K_M

---

3. Engineering the Prompt: Constrained JSON Output Extraction

To ensure that the generated summary seamlessly integrates into your Markdown vault and automation scripts, you should force Ollama to emit structured JSON using strict schema constraints.

3.1 The Custom Modelfile

Create a custom Modelfile named MeetingSummarizer.modelfile:

FROM llama3.1:8b-instruct-q4_K_M

# Set inference temperature low to minimize hallucination
PARAMETER temperature 0.15
PARAMETER top_p 0.9
PARAMETER num_ctx 32768

# Define system persona and strict behavioral boundaries
SYSTEM '''
You are an expert executive secretary and software architecture analyst.
Your sole function is to analyze raw meeting transcripts and extract structured, verifiable information.
CRITICAL RULES:
1. Extract ONLY facts, decisions, and action items explicitly stated in the transcript.
2. NEVER extrapolate, infer, or hallucinate deliverables not discussed.
3. Assign each action item to a named individual if spoken; otherwise label as 'Unassigned'.
4. Format all output as a valid JSON object matching the requested schema without any markdown commentary outside the JSON block.
'''

Build the custom model into Ollama:

ollama create meeting-analyst -f MeetingSummarizer.modelfile

3.2 The Python Extraction Pipeline

Below is a complete, production-ready Python script utilizing the official ollama-python library to ingest a raw meeting transcript and produce structured Markdown with YAML frontmatter:

import json
import sys
from pathlib import Path
import ollama

def extract_meeting_intelligence(transcript_text: str) -> dict:
    prompt = f'''
Analyze the following meeting transcript. Return a JSON object with this exact structure:
{{
  "title": "Concise descriptive title of the meeting (max 8 words)",
  "executive_summary": "3-4 sentence comprehensive overview of the discussion",
  "key_decisions": ["Decision 1", "Decision 2"],
  "action_items": [
    {{
      "owner": "Name of person responsible",
      "task": "Specific description of deliverable",
      "deadline": "Deadline if specified, or 'None'"
    }}
  ],
  "technical_topics": ["Topic 1", "Topic 2"]
}}

TRANSCRIPT:
"""
{transcript_text}
"""
'''
    
    response = ollama.chat(
        model='meeting-analyst',
        messages=[{'role': 'user', 'content': prompt}],
        format='json',
        options={'temperature': 0.15}
    )
    
    return json.loads(response['message']['content'])

def format_markdown_note(data: dict) -> str:
    md = "---\n"
    md += f"title: \"{data['title']}\"\n"
    md += f"type: meeting-note\n"
    md += f"tags: [meeting, architecture, local-ai]\n"
    md += "---\n\n"
    
    md += f"# {data['title']}\n\n"
    md += "## Executive Summary\n"
    md += f"{data['executive_summary']}\n\n"
    
    md += "## Key Decisions\n"
    for decision in data.get('key_decisions', []):
        md += f"- [x] **Decision:** {decision}\n"
    md += "\n"
    
    md += "## Action Items\n"
    for item in data.get('action_items', []):
        md += f"- [ ] **{item['owner']}**: {item['task']} *(Due: {item['deadline']})*\n"
    md += "\n"
    
    return md

---

4. Benchmark: On-Device Processing vs Cloud API Latency and Cost

To evaluate the operational efficiency of local inference, we processed a 45-minute technical architecture meeting (9,200 words / 12,800 tokens) using local Ollama (Llama 3.1 8B Q4_K_M on an Apple M3 Pro) versus commercial cloud APIs (OpenAI GPT-4o and Anthropic Claude 3.5 Sonnet).

Latency and Financial Cost Comparison

Execution Vector	Input Tokens	Processing Latency	Per-Meeting Cost	Monthly Cost (40 Mtgs)	Network Transmission
Local Ollama (M3 Pro)	12,800	18.4 seconds	$0.00	$0.00	0.00 KB (Offline)
OpenAI GPT-4o	12,800	11.2 seconds	$0.068	$2.72 + Base Fees	68 KB JSON Payload
Claude 3.5 Sonnet	12,800	14.6 seconds	$0.082	$3.28 + Base Fees	71 KB JSON Payload
Commercial Cloud SaaS	N/A	Variable (Batch)	$30.00/user/mo	$30.00/user/mo	75 MB Raw Audio

While commercial cloud APIs boast modest marginal per-call token fees, the true cost lies in vendor lock-in, recurring team seat subscriptions, and the non-negotiable security compromise of transmitting confidential intellectual property over public networks. For an in-depth financial breakdown of local versus cloud transcription overhead, consult our analysis on the cost of AI transcription in Obsidian.

---

5. Security & Privacy Advantages: Air-Gapped Compliance

In heavily regulated sectors—including intellectual property litigation, defense contracting, biotech patent development, and M&A advisory—uploading conversational transcripts to multi-tenant cloud inference clusters violates enterprise compliance mandates.

The Subpoena Risk of Cloud Notetakers

When proprietary deliberation transcripts are stored on cloud vendor infrastructure, they become discoverable targets for civil litigation subpoenas, administrative inquiries, and national security directives without requiring notice to your legal counsel. Conversely, when inference executes locally via Ollama:

• No Remote Socket Connections: Inference executes within memory managed by the Darwin kernel. Network sockets remain inactive.
• Immediate Data Destruction: Audio buffers and context matrices evaporate from unified memory the moment the process terminates.
• Client Privilege Preservation: Maintaining conversational transcripts exclusively within encrypted APFS local storage upholds attorney-client privilege and work-product doctrine protections.

For a comprehensive network traffic audit proving that zero packets are emitted during on-device capture and processing, read our 60-minute meeting packet capture empirical research.

---

6. The Complete Local Stack: Oyma + Ollama

To achieve a seamless, automated end-to-end meeting workflow without lifting a finger:

[System Audio + Mic]
         |
    (ScreenCaptureKit)
         |
[Oyma On-Device Engine] ---> Local Whisper Transcription (.md)
                                         |
                               [Ollama Local Daemon]
                                         |
                              [Structured Markdown Note]
                                         |
                                  [Obsidian Vault]

1. Capture: Oyma records crisp system audio and microphone streams directly via macOS ScreenCaptureKit without injecting intrusive third-party meeting bots.
2. Transcribe: Audio is converted to text on your local Apple Silicon Neural Engine in seconds.
3. Summarize: The resulting transcript is digested locally by Ollama to produce bulleted decisions and assigned tasks.
4. Archive: Notes land directly inside your local Markdown directory or Obsidian vault, ready for instant search and long-term reference.

Explore our curated markdown note templates and learn how to construct a resilient local AI meeting assistant on Mac.

---

7. Context Optimization: Prompt Compression & Rolling Windows

When processing marathon meetings spanning 90 to 120 minutes, raw conversational transcripts can exceed 25,000 tokens. Even with modern 128k context windows, feeding massive unstructured text into local LLMs increases time-to-first-token latency.

Rolling Window Hierarchical Summarization:
[Raw 90-Min Transcript] ──► Chunk 1 (30m) ──► Mini-Summary A ──┐
                         ──► Chunk 2 (30m) ──► Mini-Summary B ──┼──► Master Executive Summary
                         ──► Chunk 3 (30m) ──► Mini-Summary C ──┘

7.1 Hierarchical Chunk Synthesis

1. Segment by VAD Pauses: Divide the transcript into 30-minute logical chapters using natural conversational pauses detected during transcription.
2. Extract Micro-Decisions: Run an ultra-fast quantized model (such as Qwen 2.5 7B Q4_K_M) across each chunk to extract interim commitments.
3. Consolidate Final Briefing: Pass the concatenated micro-summaries into a final synthesis pass, producing a clean, non-redundant executive brief in seconds.