The phone on our desk buzzed with a Telegram notification at 3 AM. It was a voice note from Stride, our local AI assistant, notifying us that the overnight data scrape had finished and the summaries were ready for review. We did not have to wake up a cloud server or check a subscription dashboard. The entire process happened on a studio work/test stack.
Why build a private AI assistant on a Mac?
Most AI interactions happen behind a corporate firewall. When we send a prompt to a cloud provider, we surrender the data to a third party. We also accept the risk of “model drift,” where a provider updates a model and changes how it follows instructions. A local assistant removes these variables. We own the weights and the system prompt. This setup provides a level of predictability that cloud APIs cannot match. We also eliminate the latency of round-trip network calls. The response time is limited only by the speed of the silicon on our desk.
What hardware is the baseline for a local agent?
A local assistant requires a machine that can hold a large model in its memory without swapping to the disk. We use an Apple M4 Pro with 48 GB of unified memory. This unified memory architecture is the key. Unlike a PC where the CPU and GPU have separate pools of RAM, the M4 Pro allows the GPU to access the entire 48 GB.
The 48 GB limit is a hard ceiling. In our testing, we found that we can run one large model hot at a time. For example, the Gemma-4-31B model loaded at a 64K context window takes up roughly 19 GB of memory. While this leaves room for the OS and other apps, trying to load a second large model causes the system to slow down. We frame this as a studio work/test stack because it is optimized for a specific workflow: one powerful brain that handles all requests.
How does LM Studio power the local server?
The brain of our assistant is a model served by LM Studio. We chose LM Studio because it provides an OpenAI-compatible endpoint at http://localhost:1234/v1. This allows any tool that speaks the OpenAI protocol to connect to a local model. No one needs to change the underlying code.
We load the gemma-4-31b-qat model for the main assistant. This model is a balance of reasoning power and memory efficiency. By using a quantized version, we keep the memory footprint manageable while retaining most of the intelligence of the full-weight model. LM Studio handles the GPU offloading automatically. We can see the memory usage in real time. This ensures that the 48 GB limit is respected. This setup allows us to maintain a 64K context window. This window is necessary for the assistant to remember long conversations or read large documents. You can find more about this on the LM Studio homepage.
How does Hermes Agent turn a model into an assistant?
A model by itself is just a text completion engine. To make it an assistant, we use the Nous Hermes Agent runtime. This is the layer that gives the model “hands.” Hermes Agent allows the model to use tools and access the filesystem.
Our assistant, Stride, uses a set of tools to manage our workflow. It can draft articles on stridenote.net via a custom Python helper script. It can search our local files using ripgrep. It can even schedule tasks using cron. The runtime manages the “loop” of the agent. The model decides to use a tool, the runtime executes the tool, and the result is fed back to the model.
We made Stride always-on by using a launchd gateway service. This ensures that the agent starts automatically when the Mac boots. We also integrated it with Telegram. We send a voice note to a bot, and the gateway routes that audio to a speech-to-text engine. That text goes to the Hermes Agent, and the final response returns to us as a voice note. This turns the M4 Pro into a physical presence in the studio. You can explore the full capabilities of the runtime in the Hermes Agent documentation.
What are the performance limits of a 48 GB studio stack?
The primary constraint is the memory-to-model ratio. On a 48 GB machine, we cannot run a 70B model with a large context window. We have to choose between model size and context length. We chose the 31B model because it fits comfortably while allowing for a 64K context.
Token speed is another factor. On the M4 Pro, we see respectable tokens per second. It is not as fast as a H100 cluster, but for an always-on assistant, this is acceptable. We do not need instant responses for background tasks. We care more about the quality of the reasoning and the privacy of the data.
We also found that the 48 GB limit affects our coding agents. We use OpenCode for heavy development work. Because OpenCode has a large system prompt, we use a smaller 12B model for it. This allows both the main assistant and the coding agent to exist on the disk. We only run one “hot” in the GPU at a time. This is a trade-off we accept to keep the studio stack lean.
Our setup is a practical example of how to run a local LLM on a Mac. By combining a unified memory machine with an OpenAI-compatible server and a tool-using runtime, we build a system that is private and autonomous. It is a different approach than the one described in our how we built our local AI stack. We focus on the agentic loop rather than just the chat interface. This is similar to how we evaluate other tools in our choosing between Ollama, LM Studio, and MLX, but with a focus on the runtime.
The hardware specifications for the M4 Pro are detailed on Apple’s official page. The unified memory is the single most important spec for anyone looking to build a similar stack.
The shift toward local agency is a move toward digital sovereignty. We no longer have to hope that a cloud provider keeps our data safe or maintains a specific model version. We can build a tool that knows our files and our preferences. The M4 Pro and Hermes Agent show that the barrier to entry for a private assistant is now just a matter of configuration.