What does "on-device AI" mean, and why does it matter?

Two different things happen when you tap "generate"

Most AI apps work the same way: you type something, the app sends it over the internet to a company's servers, a large model running on that company's hardware generates a response, and the result is sent back to your phone. That's how ChatGPT, most chatbots, and most AI writing tools work — the intelligence lives in the cloud, and your device is mostly a window onto it.

On-device AI works differently. Apple's iPhones — the recent ones with enough processing power — can run a language model directly on the phone's own chip. Your prompt never has to leave the device, because the thing generating the response is sitting right there in your pocket. Apple calls this Apple Intelligence, built on their on-device Foundation Models framework.

Why this matters more for some apps than others

For a lot of AI use cases, it genuinely doesn't matter much whether the processing happens on your device or in the cloud. It matters a great deal more for apps that handle information about a child, because:

• Nothing to intercept. Data that never leaves the device can't be caught in transit, logged by a server, or included in a data breach at a company you've never heard of.
• Nothing to retain by accident. Cloud services vary wildly in how long they keep what you send them, even when their policy says they shouldn't. On-device processing sidesteps the question entirely.
• It works offline. A phone that can generate a story locally doesn't need a connection to do it — useful at bedtime, on a plane, or anywhere signal is patchy.

The honest limits

On-device AI isn't free of trade-offs. It needs a fairly recent, fairly capable phone — older devices simply can't run the models involved. And on-device models are generally smaller and somewhat less capable than the largest cloud models, which is part of why some apps (including ours) fall back to a cloud model on devices that can't run the on-device version, rather than just not working at all.

The honest way to think about it: on-device AI is a meaningfully stronger privacy default where it's available, and a well-designed app should be upfront about when it's using it and when it isn't — not just claim "on-device" as a marketing line while quietly sending everything to a server anyway.

How OnceUponaBookshelf uses it

OnceUponaBookshelf tries to write every story on-device first, using Apple's on-device Foundation Models running locally on your iPhone — in that case, your child's name, age, interests and story idea never leave the device. On a phone that can't run the on-device model, the app asks your permission and falls back to Claude Haiku, a cloud model from Anthropic, for that one story; nothing is kept once it's delivered back to your device either way. See exactly how in how it works.

Related reading

• Is it safe to let AI write your child's bedtime story?
• Do AI story apps keep your child's data private?
• What to look for in an AI bedtime story app