GuidesClaude Code

Run Claude Code on a free model on your Mac

By Stephen Thorn · 3 minute read ·

Claude Code is the harness: it reads your files, runs commands and asks before it changes anything. The model behind it can be swapped. On 20 Oct 2026 I ran it on Qwen 3.8 27B, a free model, on my Mac Studio. I gave it 5 sample contracts and asked which renew in the next 60 days. It read every file, got every date right and flagged the one deadline that had already passed, in under 3 minutes. No Claude login was involved, and the files only went to the model on my Mac.

Anthropic doesn't support Claude Code on non-Claude models, so this runs on Ollama's and PrismML's setup, not Anthropic's. Use it for private files and simple jobs, and keep big builds on Claude.

What you need

  • A Mac with Apple silicon and enough memory. With a 64K context, Qwen 3.8 27B used about 21 GB on mine and Bonsai 2 used about 12 GB.

  • Claude Code.

  • Ollama (ollama.com) for Qwen, or PrismML's llama.cpp for Bonsai.

Option 1: Qwen 3.8 27B with Ollama

  1. Install Ollama from ollama.com.

  2. Give it a 64K context or more. Ollama's own advice: "Update the context length in Ollama's settings to at least 64000 tokens." If you run Ollama from the terminal instead of the app, start it with OLLAMA_CONTEXT_LENGTH=65536 ollama serve.

  3. Download the model (18 GB; mine took 4 minutes): 

    ollama pull qwen3.8:27b

  4. Open a terminal in the folder you want to work in and start Claude Code on the model. Give Claude Code the same context number, or it warns that the model "isn't recognized" and assumes 200K: 

    CLAUDE_CODE_MAX_CONTEXT_TOKENS=65536 ollama launch claude --model qwen3.8:27b

  5. Optional, for privacy: add CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 in front as well. It turns off Claude Code's auto-updates, telemetry and error reporting.

The header then shows qwen3.8:27b as the model. Ask it something about the files in that folder.

Option 2: Bonsai 2 27B (smaller, more setup)

Bonsai 2 is PrismML's compressed build of Qwen 3.8 27B, under the Apache 2.0 license. The PQ2_0 file is 7.2 GB, and PrismML says it scores 98.2% of the full model across 14 benchmarks. Ollama and LM Studio can't load it, so it runs on PrismML's version of llama.cpp.

  1. Build PrismML's llama.cpp (on a Mac, Metal is the default):

    git clone https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp
cmake -B build && cmake --build build -j

  2. Download the model:

    hf download prism-ml/Ternary-Bonsai-2-27B-gguf Ternary-Bonsai-2-27B-PQ2_0.gguf --local-dir .

  3. Start the server:

    ./build/bin/llama-server -m Ternary-Bonsai-2-27B-PQ2_0.gguf --alias bonsai-2-27b -ngl 99 -fa on -c 65536 --jinja --port 8090

  4. Add a small bridge. Claude Code puts system messages in the middle of a conversation, and the model's chat template only accepts one at the start. Pointed straight at llama-server, every request failed with "System message must be at the beginning." Ask Claude Code (on Claude) to build the bridge:

    Write a small Python proxy on port 8091 that accepts Anthropic Messages API requests (POST /v1/messages, streaming and not). Move any system message after the first into the user turn it follows, wrapped in tags. Convert the request to OpenAI chat completions, send it to llama-server at http://127.0.0.1:8090/v1/chat/completions, and convert the answer back to Anthropic format, tool calls included.

  5. Start Claude Code against the bridge, with every model slot on Bonsai:

    export ANTHROPIC_BASE_URL=http://127.0.0.1:8091 ANTHROPIC_AUTH_TOKEN=local
export ANTHROPIC_MODEL=bonsai-2-27b ANTHROPIC_DEFAULT_OPUS_MODEL=bonsai-2-27b
export ANTHROPIC_DEFAULT_SONNET_MODEL=bonsai-2-27b ANTHROPIC_DEFAULT_HAIKU_MODEL=bonsai-2-27b
export CLAUDE_CODE_SUBAGENT_MODEL=bonsai-2-27b CLAUDE_CODE_MAX_CONTEXT_TOKENS=65536 API_TIMEOUT_MS=1800000
claude --model bonsai-2-27b

Through the bridge, Bonsai got the same right answer on the contracts test in 3 min 35 s. PrismML lists a known issue: "Tool calls sometimes come out malformed or loop". Mine ran clean, but watch for it.

Worth knowing

  • Claude Code's built-in web search runs on Anthropic's servers, so on the local model it came back "unavailable". To search from a local model, add an MCP server that has a search tool.

  • The first reply is slow. It took about a minute while my Mac read Claude Code's instructions (9,892 tokens). Later replies reuse that work: the second came back in under 5 seconds.

The mistake to avoid

Leaving the context small. Claude Code's instructions alone were 9,892 tokens in my test, before any of your files. Set Ollama's context to 64K or more first, and give Claude Code the same number with CLAUDE_CODE_MAX_CONTEXT_TOKENS.