ScreenCaptureKit Audio Capture Guide

Introduction: The Death of Virtual Audio Kernel Extensions

For more than two decades, capturing system audio on macOS—recording the audio emitted by web browsers, video conferencing clients, or media players—was one of the most frustrating engineering challenges in Apple software development. Because macOS historically lacked an official user-space API for loopback audio recording, developers were forced to rely on cumbersome workarounds: virtual CoreAudio HAL plugins, third-party kernel extensions (such as Soundflower), or virtual audio aggregate drivers (such as BlackHole).

These legacy approaches suffered from catastrophic operational flaws:

1. Security & Kernel Panic Risks: Third-party kernel drivers frequently crashed across OS point releases and required users to degrade macOS system security in Recovery Mode.
2. Aggressive Permission Barriers: Modern macOS security policies make installing third-party audio drivers an onerous ordeal for everyday users.
3. Fragile Audio Routing: Virtual loopback devices hijack the user’s default audio output device. If a driver glitched during a video conference, the user’s headphones went completely silent.

With the release of ScreenCaptureKit in macOS Monterey (12.3) and its comprehensive audio expansion in macOS Ventura (13.0) and Sonoma (14.0), Apple fundamentally resolved this problem. ScreenCaptureKit provides a high-throughput, low-latency, user-space framework capable of capturing display video and pristine system audio directly from the Darwin window management subsystem.

In this technical guide, we present a complete architectural walkthrough of building a native, botless system audio capture engine on macOS using ScreenCaptureKit and Swift.

---

1. System Architecture: How ScreenCaptureKit Captures Audio

To understand ScreenCaptureKit, we must look at how the macOS WindowServer and CoreAudio collaborate under the hood.

[Running Applications: Zoom, Chrome, Slack]
                    |
           [CoreAudio HAL Output]
                    |
          [macOS WindowServer] <--- ScreenCaptureKit Tap
                    |
        [SCStreamOutput Delegate]
                    | (Delivers CMSampleBuffer)
        +-----------+-----------+
        |                       |
  [Audio Track]           [Video Track (Discarded)]
        |
  [Extract Linear PCM AudioBufferList]
        |
  [16kHz Mono Resampling & Transcription Engine]

When an application plays audio, CoreAudio renders the buffers and presents them to the output hardware. ScreenCaptureKit attaches an authenticated capture tap at the compositor level. Instead of rerouting the user’s physical headphone output, ScreenCaptureKit duplicates the audio stream in memory, packaging the PCM samples into standard CoreMedia CMSampleBuffer structures and delivering them to your application delegate.

Crucially, the user’s physical audio playback is completely unaffected. Volume controls, AirPods spatial audio, and headphone routing remain 100% normal.

---

2. Entitlements and macOS TCC Privacy Permissions

Because ScreenCaptureKit has the capability to capture screen pixels alongside audio, macOS classifies it under the Screen and System Audio Recording privacy umbrella managed by Transparency, Consent, and Control (TCC).

2.1 Checking and Requesting Permissions

Before attempting to initialize an SCStream, your application must verify authorization:

import Foundation
import ScreenCaptureKit

final class CapturePermissionManager {
    static func verifyScreenCaptureAccess() -> Bool {
        // CGPreflightScreenCaptureAccess checks existing authorization
        if CGPreflightScreenCaptureAccess() {
            return true
        }
        
        // Triggers the macOS system prompt directing user to System Settings
        return CGRequestScreenCaptureAccess()
    }
}

If authorization has not been granted, macOS will present an alert directing the user to System Settings > Privacy & Security > Screen & System Audio Recording. Once enabled, your application can capture system audio streams across all running applications.

---

3. Implementation: Building the Audio Capture Stream

Building an audio-only capture pipeline using ScreenCaptureKit involves four key components:

1. SCShareableContent: Discovers available displays, windows, and running applications.
2. SCContentFilter: Defines what content to include or exclude from the capture session.
3. SCStreamConfiguration: Specifies stream parameters (disabling video frames to save CPU, configuring audio sample rates).
4. SCStreamOutput: The delegate protocol that receives incoming CMSampleBuffer frames.

3.1 The Swift Capture Engine

Below is a production-grade, complete Swift implementation for capturing system audio:

import Foundation
import ScreenCaptureKit
import AVFoundation

final class SystemAudioRecorder: NSObject, SCStreamOutput, SCStreamDelegate {
    private var stream: SCStream?
    private let audioProcessingQueue = DispatchQueue(
        label: "com.engine.audio.screencapturekit",
        qos: .userInitiated
    )
    
    // Callback delivering processed PCM float buffers
    var onAudioBufferReceived: (([Float]) -> Void)?

    func startAudioCapture() async throws {
        // Step 1: Query shareable content across macOS
        let shareableContent = try await SCShareableContent.excludingDesktopWindows(
            false,
            onScreenWindowsOnly: true
        )
        
        guard let primaryDisplay = shareableContent.displays.first else {
            throw NSError(domain: "CaptureEngine", code: 1, userInfo: [NSLocalizedDescriptionKey: "No display found"])
        }
        
        // Step 2: Configure filter to capture all system audio
        // We capture the primary display while excluding our own app to avoid feedback loops
        let currentApp = NSRunningApplication.current
        let excludedApps = shareableContent.applications.filter { $0.bundleIdentifier == currentApp.bundleIdentifier }
        
        let filter = SCContentFilter(
            display: primaryDisplay,
            excludingApplications: excludedApps,
            exceptingWindows: []
        )
        
        // Step 3: Configure stream parameters for pure audio extraction
        let config = SCStreamConfiguration()
        config.capturesAudio = true
        config.sampleRate = 48000
        config.channelCount = 2
        
        // CRITICAL PERFORMANCE OPTIMIZATION:
        // Set video dimensions to minimal 2x2 pixels to eliminate GPU composition overhead
        config.width = 2
        config.height = 2
        config.minimumFrameInterval = CMTime(value: 1, timescale: 1) // 1 frame per second
        
        // Step 4: Initialize and start the stream
        let newStream = SCStream(filter: filter, configuration: config, delegate: self)
        try newStream.addStreamOutput(self, type: .audio, sampleHandlerQueue: audioProcessingQueue)
        
        try await newStream.startCapture()
        self.stream = newStream
    }

    func stopAudioCapture() async throws {
        guard let stream = self.stream else { return }
        try await stream.stopCapture()
        self.stream = nil
    }

    // MARK: - SCStreamOutput Delegate

    func stream(_ stream: SCStream, didOutputSampleBuffer sampleBuffer: CMSampleBuffer, of type: SCStreamOutputType) {
        guard type == .audio else { return }
        guard sampleBuffer.isValid else { return }
        
        // Extract raw Linear PCM audio buffers from the CMSampleBuffer
        guard let formatDescription = sampleBuffer.formatDescription,
              let audioBufferList = sampleBuffer.audioBufferList else {
            return
        }
        
        let pcmData = extractFloatChannelData(from: audioBufferList)
        onAudioBufferReceived?(pcmData)
    }

    private func extractFloatChannelData(from bufferList: UnsafePointer<AudioBufferList>) -> [Float] {
        let buffers = UnsafeMutableAudioBufferListPointer(UnsafeMutablePointer(mutating: bufferList))
        guard let firstBuffer = buffers.first, let rawData = firstBuffer.mData else {
            return []
        }
        
        let floatCount = Int(firstBuffer.mDataByteSize) / MemoryLayout<Float>.size
        let floatPointer = rawData.bindMemory(to: Float.self, capacity: floatCount)
        
        return Array(UnsafeBufferPointer(start: floatPointer, count: floatCount))
    }
}

---

4. Performance Optimization: Eliminating Video Composition Overhead

Because ScreenCaptureKit was originally designed as a unified screen-and-audio framework, initializing a stream without careful configuration can inadvertently trigger heavy display capture. If you do not explicitly restrict the video dimensions, macOS will capture 4K or 5K display buffers at 60 frames per second, consuming massive GPU memory and battery power.

To make ScreenCaptureKit a true zero-overhead audio tap, apply these three configuration rules:

1. Set Minimal Dimensions: Set config.width = 2 and config.height = 2. This reduces the video render surface to a 4-pixel stub.
2. Throttle Frame Rate: Set config.minimumFrameInterval = CMTime(value: 1, timescale: 1), instructing the compositor to render at most one video frame per second.
3. Ignore Video Callbacks: In your SCStreamOutput delegate, immediately return early if type == .screen.

With these optimizations in place, ScreenCaptureKit captures pristine 48kHz audio while consuming less than 0.8% CPU on an Apple Silicon processor.

---

5. Handling Microphone Multiplexing

Recording incoming system audio captures what the other participants on the call are saying. However, to produce a complete meeting transcript, your application must simultaneously record your own voice via the physical microphone.

[System Audio (ScreenCaptureKit)] ---> [Circular Buffer A]
                                                |
                                        [Sample Time Alignment]
                                                |
[Microphone Audio (AVAudioEngine)] ----> [Circular Buffer B]
                                                |
                                    [Interleaved / Mixed PCM]
                                                |
                                  [Whisper Speech-to-Text]

The Recommended Architecture

• Input Stream A (Far-End Audio): Captured via ScreenCaptureKit at 48kHz stereo.
• Input Stream B (Near-End Audio): Captured via AVAudioEngine.inputNode at the microphone’s native hardware sample rate.
• Timestamp Synchronization: Both streams provide host hardware timestamps (mach_absolute_time() or CMSampleBufferGetPresentationTimeStamp). Use a dual-channel ring buffer to align audio frames before passing them into the resampler.

By keeping the microphone and system audio streams discrete until transcription, you also preserve the ability to identify whether a spoken phrase originated locally or remotely.

---

6. How Oyma Implements Botless Recording

In Oyma, ScreenCaptureKit is the foundational technology that makes botless meeting notes possible:

1. No Bot Injections: Unlike legacy cloud notetakers that force an unwanted AI attendee into your Zoom, Google Meet, or Teams calls, Oyma captures incoming audio directly from the macOS audio pipeline.
2. Zero Audio Glitches: Because ScreenCaptureKit operates without virtual audio drivers or kernel extensions, your Bluetooth headphones, volume sliders, and microphone settings remain completely unaffected.
3. Instant Local Transcription: Audio buffers are piped directly into our optimized on-device Whisper engine, producing structured Markdown notes in your vault seconds after the call concludes.

For comprehensive platform-specific guides on botless recording, explore our tutorials for recording Zoom calls without a bot and recording Google Meet calls without a bot.

---

7. Dynamic Audio Resampling with AVAudioConverter

When capturing system audio alongside physical hardware microphones, incoming audio streams often exhibit differing sample rates (e.g., ScreenCaptureKit emitting 48,000 Hz while an external USB microphone operates at 44,100 Hz). Whisper models require uniform 16,000 Hz single-channel PCM.

import AVFoundation

final class AudioResampler {
    private let converter: AVAudioConverter
    private let inputFormat: AVAudioFormat
    private let outputFormat: AVAudioFormat

    init?(sourceSampleRate: Double, channels: AVAudioChannelCount) {
        guard let inFmt = AVAudioFormat(
            commonFormat: .pcmFormatFloat32,
            sampleRate: sourceSampleRate,
            channels: channels,
            interleaved: false
        ),
        let outFmt = AVAudioFormat(
            commonFormat: .pcmFormatFloat32,
            sampleRate: 16000.0,
            channels: 1,
            interleaved: false
        ) else { return nil }

        self.inputFormat = inFmt
        self.outputFormat = outFmt
        guard let conv = AVAudioConverter(from: inFmt, to: outFmt) else { return nil }
        self.converter = conv
    }

    func convertBuffer(_ buffer: AVAudioPCMBuffer) -> AVAudioPCMBuffer? {
        let ratio = 16000.0 / buffer.format.sampleRate
        let outputFrameCapacity = AVAudioFrameCount(Double(buffer.frameLength) * ratio)
        guard let outBuffer = AVAudioPCMBuffer(pcmFormat: outputFormat, frameCapacity: outputFrameCapacity) else {
            return nil
        }

        var error: NSError?
        self.converter.convert(to: outBuffer, error: &error) { _, outStatus in
            outStatus.pointee = .haveData
            return buffer
        }
        return outBuffer
    }
}

This hardware-accelerated conversion pipeline runs with sub-millisecond latency, delivering normalized 16kHz audio tensors directly into the Whisper inference queue.