The first thing Hermes Agent shows on its Local Models page is a single blue button that does three jobs at once. “Set up for me” installs a llama.cpp engine, downloads a 21.9 GB model and makes that model the default for every new chat. We opened the page on our test machine, a 48 GB M4 Pro, and took the slower route through Configure instead, so we could see each step on its own. This guide covers the setup of Hermes Agent local models on a Mac: what each button does, where the files land and how to keep LM Studio in charge if you already run it.
Everything below comes from Hermes desktop v0.21.1 on 11 September 2026, with timings from the file system rather than from the app.
How to install the Hermes Agent local models runtime on a Mac
Open Settings, then Providers, then Local Models. On a machine with no local engine the page opens on a one-click card for Qwen3.6 35B-A3B, with two buttons under it. Configure opens the full page and changes nothing. That is the safe starting point.
The full page has three sections. Local runtime sits at the top with an Install runtime button. This machine reports “48.0 GB GPU memory”, “48.0 GB RAM” and “Unified memory”. Models lists the catalog Hermes ships.
The Download buttons in the catalog stay grey until the engine is installed. Clicking one first does nothing, and nothing appears in Hermes’s logs either. Install the runtime before anything else.
Install runtime fetched llama.cpp build b10679 for macOS on Apple silicon and verified it in under ten seconds. The panel then read “llama.cpp runtime installed. Build b10679, metal backend” with a Ready badge naming the metal backend, and a Turn on button. Installing and turning on are separate steps: at this point nothing in ~/.hermes/config.yaml had changed.
Hermes’s page describes the download as a few hundred MB, and its Local Models documentation uses the same wording. On our machine the build took 26 MB, plus 11 MB for the archive Hermes keeps, so 37 MB in total. The binary lives at ~/.hermes/runtimes/llamacpp/b10679/metal/llama-b10679/llama-server.
The catalog on this 48 GB Mac listed four models:
| Model | Download | Label on our machine |
|---|---|---|
| Qwen3.8 27B | 16.2 GB | Fits your GPU, full 256K context, sees images |
| Qwen3.6 35B-A3B | 21.9 GB | Recommended, fits your GPU, full 256K context, sees images |
| Qwen3.8 Flash Next | 104.5 GB | Too big for this machine |
| DeepSeek V4 Flash | 154.6 GB | Too big for this machine |
The Recommended badge is a speed prediction rather than a quality ranking, which we tested in Qwen3.6 35B-A3B vs Gemma 4 31B on a Mac.
Where Hermes Agent downloads local models, and why LM Studio cannot see them
Models go straight into ~/.hermes/models. A download starts as a .part file that shows its full size immediately, so Finder is no use for progress; du shows the real figure. The vision projector that lets a model read images arrives afterwards in ~/.hermes/models/assets.
Qwen3.8 27B came from the unsloth/Qwen3.8-27B-GGUF repository on Hugging Face. The weights file was 16,464,440,224 bytes and took about four and a half minutes, roughly 61 MB a second, followed by a 931 MB projector. Qwen3.6 35B-A3B came from the unsloth MTP build of that model: 22.66 GB of weights plus a 0.90 GB projector in about six minutes. The two models together took 41 GB of disk.
The page runs one download at a time. While Qwen3.8 was downloading, the Download button for Qwen3.6 stayed grey, and it only became clickable once the first file was complete. When a download finishes, its row swaps the Download button for a Use button and a delete icon.
Hermes’s own logs record the runtime download and nothing about the models. There is no entry when a model download starts, finishes or fails. If a download stalls, the file size in ~/.hermes/models is the only evidence.
LM Studio does not see any of this. It scans ~/.lmstudio/models, and after both downloads its model list still showed only the Gemma 4 31B file we already had. The two apps keep separate folders, so a model downloaded in one is a second copy of anything held in the other.
Sharing works in the other direction. The Add model file button at the bottom of the page registers a GGUF that is already on the Mac. Hermes’s code makes a hard link to it where it can, so the model appears in Hermes with no extra disk used, and it falls back to a symbolic link or a copy only when a hard link fails. We added LM Studio’s Gemma 4 31B this way and the file kept a single copy on disk.
Set up for me or Configure: which Hermes button changes your default model
Two actions change what your chats run on, and neither says so loudly.
Set up for me, on the first card, installs the engine, downloads the recommended model and makes it the default for new chats, all in one click. The card states the last part in small grey text under the model name.
Use, on a downloaded model, does the same for that model. Hermes’s documentation puts it plainly: after you click Use, “New chats now run on the local model.” Behind the button, Hermes sets model.provider to llamacpp and points model.default at the file.
Download on its own changes nothing, and neither does Configure. If you want to try a model without moving your assistant off whatever it runs today, download it and test it outside the chat first.
Once a model is in use, Hermes runs a single llama-server process on 127.0.0.1 port 18434 and routes requests to it. The server can hold several models, loads them on demand and unloads one after 15 minutes idle. For each catalog model Hermes writes its own launch settings: an 8-bit KV cache, flash attention and, for both Qwen models, multi-token prediction. That last setting is the subject of our look at how MTP draft heads are packaged.
How to use Hermes Agent with LM Studio instead of its own engine
If LM Studio already serves your models, Hermes does not need its own engine at all. The documentation calls the managed runtime “a default, not a requirement”. Our assistant runs this way, as set out in our always-on assistant setup with Hermes Agent and LM Studio.
The model block in ~/.hermes/config.yaml for that setup reads:
model:
base_url: http://127.0.0.1:1234/v1/
default: google/gemma-4-31b-qat
provider: lmstudio
local_runtime:
enabled: falseWith local_runtime switched off, Hermes starts no llama-server of its own, and LM Studio stays the only process holding the model. Hermes reads this file when its gateway starts, so restart the gateway after editing it.
Running both on one 48 GB Mac is where it goes wrong. LM Studio loads Gemma 4 31B at the context length you give it, and a 64K window costs about 4.4 GB on top of the weights. Hermes’s engine loads its own copy with its own settings, so two copies of a 31B model compete for the same memory. On this machine we keep one engine per model, and that engine is LM Studio.
The Local Models page is useful for trying the catalog models Hermes has tuned settings for, since it sizes their context to fit. For a model you already run elsewhere, the three lines of config above give you the same assistant without a second engine.