Transcribe audio

Use transcribe when you have audio data and want normalized text back.

Examples/Sources/TranscribeAudio/main.swift

import Foundation
import SwiftAISDK

@main
struct TranscribeAudioExample {
    static func main() async throws {
        let provider = try AIProviders.openAI()
        let model = try provider.transcriptionModel("gpt-4o-transcribe")
        let audio = try Data(contentsOf: URL(fileURLWithPath: "meeting.wav"))

        let result = try await model.transcribe(
            audio: audio,
            fileName: "meeting.wav",
            mimeType: "audio/wav",
            language: "en",
            prompt: "The speakers discuss Swift package design."
        )

        print(result.text)
    }
}

The lower-level request initializer is still available when you want to build the request separately.

let request = AudioTranscriptionRequest(
    audio: audio,
    fileName: "clip.m4a",
    mimeType: "audio/mp4"
)

let result = try await AI.transcribe(model: model, request: request)

Streaming Transcription

StreamingTranscriptionModel is the duplex WebSocket counterpart to batch transcription. Feed microphone or other incremental PCM chunks through an AIStreamingAudioInput pipe and consume partial/final transcript events while audio is still being produced:

let cartesia = try AIProviders.cartesia()
let model = try cartesia.streamingTranscription("ink-2")
let audioPipe = AIStreamingAudioInput.makeStream()

let result = try await model.stream(StreamingTranscriptionRequest(
    audio: audioPipe.input,
    inputAudioFormat: AIStreamingAudioFormat(
        mediaType: "audio/pcm",
        sampleRate: 16_000
    )
))

Task {
    for await chunk in microphonePCMChunks {
        _ = audioPipe.writer.send(chunk)
    }
    audioPipe.writer.finish()
}

for try await part in result.stream {
    switch part {
    case let .transcriptPartial(_, text, _, _, _, _):
        print("partial:", text)
    case let .transcriptFinal(_, text, _, _, _, _):
        print("final:", text)
    case let .finish(summary):
        print("complete:", summary.text)
    default:
        break
    }
}

Cartesia Ink 2 obtains a short-lived STT access token, supports server turn detection or explicit finalization, maps partial/final events, and redacts the access token from request metadata. Gateway uses the shared transcription WebSocket envelope, splits audio frames at 64 KiB, and exposes experimentalTranscription.getToken(...) for minting short-lived route-bound client tokens on a trusted server. The underlying AIDuplexWebSocketTransport is injectable for deterministic tests or custom networking; stopping consumption, result.cancel(), or an AIAbortSignal closes the socket and the audio producer.

Streaming transcription is intentionally narrower than a full bidirectional speech session. For audio/text responses, function-call events, and session configuration, use Realtime sessions. xAI is the first full Realtime V4 provider; ElevenLabs realtime STT and Google/OpenAI streaming translation remain deferred.

Options

• fileName and mimeType describe the submitted audio.
• language hints the expected language when the provider supports it.
• prompt gives the model context for names, vocabulary, or formatting.
• providerOptions carries provider-specific transcription settings.
• OpenAI chunkingStrategy accepts "auto" or a server_vad object with an optional threshold and nonnegative prefix/silence padding. The gpt-4o-transcribe-diarize model defaults to automatic chunking and diarized_json output.
• inputAudioFormat declares the raw encoding and sample rate for streaming models.
• extraBody, headers, abortSignal, retryPolicy, and telemetry follow the shared facade behavior.

Results

TranscriptionResult includes text plus optional timing and language metadata:

• text
• segments
• language
• durationInSeconds
• warnings
• providerMetadata
• requestMetadata
• responseMetadata

For diarized OpenAI output, normalized text remains in the shared result while speaker-aware segments are retained under providerMetadata["openai"]["segments"] with text, start/end seconds, and speaker identity.