A compliance officer at a European bank recently asked how to handle an internal chatbot that employees were using to summarise customer call transcripts. The chatbot ran on a cloud API. No one had checked whether those transcripts contained personally identifiable information. They did. The bank had no data processing agreement in place with the API provider, and the model provider’s terms of service reserved the right to use API inputs for service improvement. That is the exact moment when the local-versus-cloud question stops being theoretical. This piece sets the order in which a business should decide, shows how to put real numbers on the cost, and adds what one Mac can actually serve before the cloud is needed.
The bank’s problem was not that the cloud was inherently insecure. The problem was that no one had asked the threshold question: does this workload belong in the cloud at all?
When does privacy rule out the cloud?
The first filter in the decision is not cost or latency. It is data sensitivity. The question is binary: can this data legally or contractually leave your infrastructure?
Where the answer is no, the question is settled. No amount of cloud cost efficiency justifies crossing that line.
The regulatory environment has made this filter more restrictive. The European Data Protection Board has said that whether a model trained on personal data counts as anonymous must be assessed case by case, and that it only does if extracting that personal data from it is very unlikely. If your cloud provider fine-tunes on user data without explicit consent mechanisms, the model weights themselves may carry compliance exposure. Local inference sidesteps this entirely because the data never leaves your own hardware.
With an API, every prompt is a request to a server you do not control, logged under a policy you did not write. With a local model, the prompt and the answer both live and die on your own machine: the client transcript, the half-finished draft, the file full of things you would never paste into a web form. That is not a privacy promise. It is where the computation happens, and it is the difference between a claim you assert and one you can prove.
When privacy is the constraint, the cost of self-hosting is justified by the binary nature of the requirement, not by the unit economics.
How does the cost math shift at scale?
For workloads that pass the privacy filter, the next question is volume. Cloud APIs are cheap at low volume and expensive at high volume. Local hardware is the opposite.
Cheap at low volume is also where the free tiers sit, and their price is paid in something other than money. The hidden costs of free cloud AI are the terms, the data and the lock-in you accept without an invoice.
Cloud inference sends a network request to a provider-managed GPU cluster, waits for a response, and pays per token. No hardware cost on your side, but every call crosses the network and every call is metered, so the more useful the tool becomes, the more it costs to keep using. Rate limits cut the same way: an API decides how many requests you get and how fast, and hitting the ceiling on a busy afternoon stops your tool until the vendor lets you back in. Local hardware carries an upfront capital cost, but the per-call cost after that is approximately electricity, and the only limit is the machine.
Most arguments about which is cheaper never get to a number. The method below puts your own numbers on both sides.
How do you add up your AI subscription costs?
Open your statements and list every AI subscription you pay for. Not the ones you remember, the ones you pay for. They are not the same list. Look for:
- The chat assistant.
- The coding assistant or agent.
- The image or design generator.
- The transcription or meeting-notes service.
- Any per-seat plan billed across a team.
- The API spend that shows up as usage, not a flat fee.
- The one someone added last quarter that nobody owns.
For each line, write the monthly cost, then multiply by 12 and add them into a single yearly number. Subscriptions hide inside “it is only a few dollars a month.” The annual total is where the real size shows up. If a plan is per-seat, multiply by seats too, because that is what you actually send, and per-seat pricing has a compounding problem of its own. Call that yearly total your rented cost.
How much does running AI locally really cost?
The local side has two parts. The first is the machine, a one-time cost that for many people is partly or fully sunk already. The Ollama floor is about 16GB of RAM recommended and 8GB as a minimum for small models. If the laptop you are reading this on meets that, your hardware cost for everyday local work may be zero. If you need more capability, the cost is whatever a suitable machine costs you, once. To compare fairly against a yearly subscription total, spread that price across the years you expect to keep the machine.
The second is electricity. Find your machine’s draw under load in watts (laptops are modest; a desktop with a discrete GPU is more), estimate the hours per day you actually run inference, and multiply by your electricity rate per kilowatt-hour from your utility bill. Watts times hours gives you energy; energy times your rate gives you cost. Do that per day and multiply out to a year. Add the per-year slice of the machine to the per-year electricity, and that is your local cost. The gap between the two annual figures is the whole argument, made of your numbers instead of someone’s marketing.
What is the cost of AI subscriptions vs local for a 10-person team?
This is the method worked through on published prices, so a team can see the shape before it gathers its own numbers. The seat prices are ranges from the vendors’ published prices on 17 October 2026: cloud coding assistants cost about $19 to $30 or more per developer a month on business plans, and a business transcription plan about $19.99 per user a month, billed annually. The machine is a desktop Mac with 64GB of memory and 1TB of storage, at Apple’s published list price of $3,199 on the same date.
Ten coding assistant seats cost about $190 to $300 or more a month (19 to 30, x 10). Add ten transcription seats at $199.90 (19.99 x 10) and the team pays about $389.90 to $499.90 or more a month, or $4,678.80 to $5,998.80 or more a year. The Mac’s price equals about 6.4 to 8.2 months of those combined fees (3,199 / 499.90 and 3,199 / 389.90), or about 10.7 to 16.8 months of the coding seats alone (3,199 / 300 and 3,199 / 190). That is arithmetic on list prices. It leaves out electricity, setup time and the person who keeps the machine running, and it assumes the machine does the work of every seat, which we have not shown. Our test machine is not the Mac in that sum, and we have not measured that one.
When will only a frontier model do?
The third filter is model capability. Local open-weight models have improved rapidly, but they still trail frontier proprietary models on complex reasoning, long-context tasks, and tool use.
Frontier-model needs such as complex reasoning, novel tasks, and anything user-facing where quality matters most should go to cloud APIs. Predictable high-volume tasks like classification and structured extraction can move to self-hosted open-weight models. The two are not the same decision.
Time matters too. A local model shipped today may trail the frontier by the time the hardware is paid off. Cloud APIs update continuously. For use cases where model quality directly affects product experience, that lag matters.
Cloud also wins on cost in a few cases. If running the local equivalent means a much more expensive machine you do not already own, the subscription may be the cheaper path until that changes. If you need a heavy capability rarely, paying per use can beat owning the hardware. And a managed service includes someone else keeping it running and updated, which has a value when your own time is the scarce thing.
Clustering moves the local ceiling a little. EXO Labs has shown exo running a 671B model across eight Mac minis at a reported 5.37 tokens a second, enough for a batch job and slow for a live chat, on hardware that stays in the room.
What hybrid architecture do most teams settle on?
A deployment does not have to be purely local or purely cloud. In a tiered design, a local or on-premise model handles routine requests while a cloud frontier model catches what the smaller model cannot, and privacy-bound data stays local whichever tier would be cheaper. Getting the routing right is an engineering decision, not an intuition.
What we run, and why
Our own split follows that design. We run gemma4:31b for writing and glm-4.7-flash for coding, both on a single Apple M4 Pro with 48 GB of unified memory, served through Ollama. We started on a single generalist model and moved to these two specialists as the local options got better: a generalist is convenient but average at everything, and swapping one file for two raised the floor on both jobs. The writing model drafts, edits and answers questions against our own notes all day. The coding model runs the build loop. We arrived at that default by doing the cost arithmetic above and finding that everyday work did not justify the recurring bill once it was annualised.
We keep one cloud subscription for the dense reasoning tasks that come up a few times a month rather than a few times an hour, where a top cloud model is ahead of what we run locally.
A local model also does not need the internet. The Wi-Fi can drop, the vendor can have an outage, and the tool keeps answering, because it is not reaching out to anything. That is part of why we own the stack rather than rent it: a local model cannot be rate-limited, deprecated, or taken offline by anyone but us. The upkeep is ours in return: the updates, the disk space, the occasional afternoon spent fixing something nobody else will fix. In practice it is an occasional model update and keeping enough disk free, not a second job.
Local vs cloud AI for a small team: how much can one Mac serve?
Cost comparisons assume the local machine can carry the load. For a small team that is the first thing to check.
How fast is one local model for one caller?
We measured single-caller speed on our test machine, an M4 Pro with 48 GB of unified memory, through LM Studio. A dense model is the slow case. Gemma 4 31B wrote 11.7 tokens a second from its GGUF file and 12.6 from its MLX build, as recorded in our Gemma 4 31B MLX vs GGUF test. A reply capped at 300 tokens took 19 to 28 seconds in our prompt processing benchmark. The longer wait is reading: a prompt of about 19,870 tokens took 286 seconds before the first word on the GGUF and 261 on MLX.
A mixture-of-experts model changes that. Qwen3.6 35B-A3B wrote 49.4 to 50.7 tokens a second on the llama.cpp engine and 59.1 to 61.5 on MLX, and it read a 20,033-token prompt in about 30 seconds on MLX against about 75 on llama.cpp, per our MLX vs llama.cpp measurement. Speed is not the whole answer. On a puzzle task where each model had to commit to one move, Qwen3.6 answered in 4 of 12 runs and Gemma in 11 of 12, in Qwen3.6 35B-A3B vs Gemma 4 31B on a Mac.
What happens with two callers at once?
We measured two requests at once through LM Studio in our parallel requests test. With the dense Gemma 4 31B on the llama.cpp engine, the machine’s combined output stayed at 11.4 tokens a second, so a second caller halved everyone’s speed to about 5.7 each. On the MLX engine the combined figure rose to 23.0, or 11.5 for each request. With the mixture-of-experts Qwen3.6 35B-A3B on MLX, two callers each got about 64.7 tokens a second, a combined 129.4.
That last figure can be put in volume terms. At 129.4 tokens a second, one hour of continuous generation comes to about 466,000 tokens (129.4 times 3,600 seconds). It is a ceiling for two callers on one model, not a measured working day, and real use has gaps between requests.
Two limits matter for planning. We measured two callers, not ten, so we cannot say where the queue starts to hurt, and we give no headcount for one Mac. Four and eight callers on our M4 Pro, and a logged working day of a small team’s requests with queue time recorded, would settle it, and both are on our list. And a second model generating at the same time is a different case: in our two-model test the faster model lost 65 percent of its speed. The tiered design above, routine work local and overflow to the cloud, rests on those two-caller numbers and nothing larger.
What should you consider before buying hardware?
The case for local AI is strong at scale, but the operational reality is more expensive than hardware amortization alone suggests.
Idle time is the cost that is easy to miss. A GPU cluster sitting idle at night still consumes power and depreciates. Cloud scales to zero.
Team size matters as well. Established companies with predictable workloads and existing infrastructure teams have more reasons to consider hybrid earlier because the operational cost is amortized and the engineering function already exists.
The decision framework reduces to three questions. Is the data sensitive? If yes, default to local. Is latency critical? If yes, default to local. Is the workload predictable and high-volume? If yes, local is almost certainly cheaper. If the answer to all three is no, cloud is the right call. In practice, most enterprise AI workloads answer yes to at least one of these questions. The teams that build durable systems are the ones that ask them before they buy hardware, and then run the cost comparison on their own numbers. A practical first step: pick one task you currently send to an API, run it once on a local model, and see whether the endpoint was earning its keep.
