What's New in Foundation Models

Model Lineup

The biggest architectural change is the new LanguageModel abstraction. A LanguageModelSession can now be backed by Apple's on-device model, the new Private Cloud Compute model, Core AI models, MLX models, or a Swift package from a provider.

import FoundationModels
import MLXFoundationModels

// On-device Apple Foundation Model
let model = SystemLanguageModel()

// Other OS 27 options can use the same session shape:
// let model = PrivateCloudComputeLanguageModel()
// let model = try await CoreAILanguageModel(resourcesAt: modelURL)
// let model = MLXLanguageModel(modelID: "mlx-community/my-model")

let session = LanguageModelSession(model: model)
let response = try await session.respond(to: "Summarize this note.")
print(response.content)

The practical effect is lower lock-in inside your app architecture. You can keep prompts, tools, transcripts, and guided generation close to the feature, then choose a model based on privacy, latency, capability, and cost.

Image Prompts

The rebuilt on-device model gains vision capabilities. Prompt builders can include image attachments from UIKit/AppKit images, Core Graphics, Core Image, Core Video pixel buffers, or file URLs. Apple says arbitrary sizes and aspect ratios are accepted, but larger images use more tokens and add latency.

let session = LanguageModelSession(model: SystemLanguageModel())

let response = try await session.respond {
    "What should I notice in this image?"
    Attachment(image)
}

Use this for user-facing reasoning over screenshots, receipts, reference photos, and visual context. Keep strict extraction tasks on specialized APIs when you need repeatable fields rather than prose.

Private Cloud Compute

PrivateCloudComputeLanguageModel exposes Apple's server-side model through the same session API. The WWDC session describes a larger model with a 32,000-token context window and reasoning levels. It also notes that PCC brings Foundation Models to watchOS 27.

let model = PrivateCloudComputeLanguageModel()
let session = LanguageModelSession(model: model)

let response = try await session.respond(
    to: "Compare these project ideas and pick the best next step.",
    contextOptions: ContextOptions(reasoningLevel: .deep)
)

Apple's stated launch policy gives no cloud API cost to qualifying developers with fewer than 2 million first-time App Store downloads, with higher user limits for iCloud+ subscribers. Treat those limits as service policy, not an app contract.

Provider Packages

The model abstraction is also for non-Apple models. Apple says Anthropic and Google are publishing Swift packages for Claude and Gemini, and provider packages can conform to the same LanguageModel protocol used by Apple models.

import FoundationModels
import ProviderFoundationModels

let token = try await authManager.fetchAccessToken()
let model = ProviderLanguageModel(accessToken: token)
let session = LanguageModelSession(model: model)

let response = try await session.respond(to: "Draft three support replies.")

Usage reporting is part of the updated surface too. Sessions and responses can expose token usage, including cached input tokens and reasoning output tokens, which matters once a third-party provider bills per token.

Built-in System Tools

OS 27 adds system-provided tools that a session can call. BarcodeReaderTool and OCRTool are backed by Vision, and Apple also describes a Spotlight-powered search tool for fully local retrieval-augmented generation over indexed content.

let session = LanguageModelSession(
    model: SystemLanguageModel(),
    tools: [
        OCRTool(),
        BarcodeReaderTool()
    ]
)

let answer = try await session.respond {
    "Read the important fields from this label."
    Attachment(labelImage)
}

Dynamic Profiles

Dynamic Profiles are the new context primitive for agentic app flows. A profile resolves to one active branch at a time, but the session can keep a continuous transcript while the active branch changes its instructions, tools, model, and configuration.

struct CraftProfile: LanguageModelSession.DynamicProfile {
    let state: CraftState

    var body: some DynamicProfile {
        switch state.mode {
        case .analysis:
            Profile {
                Instructions { "Analyze the user's craft image." }
                RecordImageAnalysisTool()
                SwitchModeTool(state: state)
            }
        case .brainstorm:
            Profile {
                Instructions { "Brainstorm project ideas from the analysis." }
                RecordBrainstormTool()
            }
            .model(PrivateCloudComputeLanguageModel())
            .reasoningLevel(.deep)
        }
    }
}

Reach for this when modes are real product states: analyze, search, plan, verify, summarize. If a feature only needs one prompt and one answer, a regular session is still clearer.

Evaluations, fm, and Python

Apple is also filling in the workflow around app code. The new Evaluations framework measures model-driven feature quality statistically, not as a single deterministic unit test. On Mac, the fm CLI can run the on-device model or PCC from Terminal, including image prompts and structured output. The Python SDK exposes the on-device model to notebooks and evaluation pipelines.

let response = try await session.respond(
    to: "Recommend a craft that does not require scissors.",
    contextOptions: ContextOptions(reasoningLevel: .light)
)

print(response.usage.input.totalTokenCount)
print(response.usage.output.reasoningTokenCount)

Apple says fm ships pre-installed on macOS 27. The Python SDK is installed into a Python environment and, per Apple's session, needs Python 3.10 or later, Xcode, and an Apple Silicon Mac.

Open Source Utilities

Apple says the core Foundation Models framework is going open source, alongside a Foundation Models framework utilities package that can evolve between OS releases. The utilities package starts with profile modifiers for transcript management, a skill API for procedural knowledge loading, and a language model that can talk to servers using the Chat Completions standard.

That package is the right place to watch for experimental building blocks. Keep app-critical behavior on stable SDK APIs, and treat utilities as a faster-moving layer you can update through Swift Package Manager.

Adoption Checklist