Local Models

Local models are usually exposed through an OpenAI-compatible endpoint. Ollama works with SwiftyAI through the same OpenAICompatibleProvider used for hosted compatible APIs.

Local path	Use when
`OpenAICompatibleProvider`	You want explicit construction and easy dependency injection
`AI.configure { ai.ollama() }`	Your app wants `"ollama/model"` strings
Private gateway	Your team exposes local or internal models behind an OpenAI-compatible API
Streaming middleware	The local model is non-streaming but the UI wants incremental output

Direct Ollama Provider

Start Ollama locally, pull a model, then point SwiftyAI at the local API:

let local = OpenAICompatibleProvider(
    baseURL: "http://localhost:11434/v1",
    apiKey: "ollama",
    model: "llama3.2"
)
 
let response = try await generateText(
    model: local,
    prompt: "Summarize this README in two sentences."
)

The API key value is not used by Ollama, but the OpenAI-compatible adapter still expects a string.

Model String Setup

AI.configure { ai in
    ai.ollama(baseURL: "http://localhost:11434/v1")
}
 
let response = try await generateText(
    model: "ollama/llama3.2",
    prompt: "Draft a short local-only privacy note."
)

Configure this during app startup before user-driven requests.

Streaming Locally

for try await chunk in streamText(
    model: local,
    prompt: "Write a short note while streaming."
) {
    print(chunk.text, terminator: "")
}

Local models can be slower and may emit different metadata from hosted APIs. Treat usage and finishReason as optional.

Private Gateways

Any local or private service with an OpenAI-compatible chat endpoint can use the same provider.

let gateway = OpenAICompatibleProvider(
    baseURL: "http://localhost:8080/v1",
    apiKey: gatewayToken,
    model: "team/default"
)

If the gateway supports tools or media endpoints, the matching SwiftyAI APIs can use it.