Workflows/Build a local coding assistant with Bonsai 2 · 27B

Coding / WORKFLOW

Build a local coding assistant with Bonsai 2 · 27B

Stand up a private, on-device coding assistant: pick a Bonsai 2 · 27B ternary GGUF package, serve it with the PrismML llama.cpp fork, connect the local OpenAI-compatible endpoint to Cline in VS Code, and complete one small, reviewable code change. Bonsai 2 · 27B is a 27.36B-parameter hybrid-attention model with a 262K-token context, so a single workstation can hold a full assistant-sized model; the packed PQ2_0 and PTQ1_0 types only run correctly in the PrismML fork, not stock llama.cpp or Ollama.

EstimatedSource checked 9/29/2026

LOCALRENTED GPU

Freshness 82 out of 100. Release age: 53 days old, Sep 17, 2026. Measures age, not quality.

Popularity 85 out of 100. 1,688,388 downloads in 30 days; 2,960 lifetime likes. Checked Nov 8, 2026. Measures adoption, not quality.

What do these scores mean?

Two ways to compare models. Neither score measures answer quality, benchmark performance, or whether a model fits your machine.

Freshness · release age

A new release starts at 100. The score halves every 180 days: 100 × 2^(−age in days / 180). When the release date is unavailable, a repository date is a fallback—not proof of when the model was released. Missing dates stay unscored.

Check the date source

Popularity · adoption

Downloads over the past 30 days carry 80% of the score; lifetime likes carry 20%. Both use a logarithmic scale, capped at 10 million downloads and 100,000 likes. If likes are unavailable, downloads carry all the weight. Missing download statistics stay unscored.

round(80 × min(1, log10(1 + downloads) / 7) + 20 × min(1, log10(1 + likes) / 5))

Check the statistics source

Scores as of Nov 9, 2026. Statistics checked Nov 8, 2026. Statistics are marked for refresh after 72 hours.Full scoring method

Before you begin

Hardware

Machine	PTQ1_0	PQ2_0	CRACK PTQ1_0	CRACK PQ2_0
Apple Silicon 8 GB unified memory	Possible but tight at ~5.95 GB; start 4K context, expect memory pressure.	Not recommended; PQ2_0 targets 16 GB unified memory; use PTQ1_0.	Same tight 5.95 GB fit as PTQ1_0; community weights, quality unverified.	Not recommended; same PQ2_0 16 GB unified memory target; use PTQ1_0.
Apple Silicon 16 GB unified memory	Comfortable target; Metal PTQ1_0 works in the PrismML fork.	Metal PQ2_0 supported; 16 GB unified memory is the target.	Same Metal PTQ1_0 route; community weights, quality unverified.	Same Metal PQ2_0 route; community weights, quality unverified.
NVIDIA GPU 8 GB VRAM	Fits at ~5.95 GB; lower context or offload layers if VRAM is short.	Tight at ~7.21 GB; 10–12 GB VRAM recommended, lower context.	Same CUDA PTQ1_0 fit; name CRACK file, not official pack.	Tight at ~7.21 GB; use 10–12 GB VRAM or lower context.
NVIDIA GPU 10–12 GB VRAM	Comfortable; CUDA PTQ1_0 supported by the PrismML fork.	Recommended VRAM range for CUDA PQ2_0.	Same CUDA PTQ1_0 route; community weights, quality unverified.	Same CUDA PQ2_0 route; name CRACK file, not official pack.
AMD GPU 8 GB VRAM	ROCm/HIP needs supported Radeon and toolchain; validate fit, as CUDA-sourced.	ROCm/HIP source-supported; validate on Radeon card and driver.	Same ROCm/HIP PTQ1_0 support; validate Radeon card and driver.	Same ROCm/HIP PQ2_0 support; validate Radeon card and driver.
AMD GPU 12 GB VRAM	ROCm/HIP supported source; validate on target card and driver.	ROCm/HIP source-supported; validate build on target Radeon GPU.	Same ROCm/HIP PTQ1_0 support; validate on target Radeon.	Same ROCm/HIP PQ2_0 support; validate on target Radeon.
Intel Arc GPU 12 GB VRAM	Vulkan PTQ1_0 exists but lacks coopmat2; verify driver and output.	No native Vulkan PQ2_0 kernel; use PTQ1_0 or CPU.	Same Vulkan PTQ1_0 caveats; verify output or fall back to CPU.	No native Vulkan PQ2_0 kernel; use PTQ1_0 or CPU.
CPU 16 GB system RAM	CPU-only supported but slower; 16 GB RAM minimum, 32 GB better.	CPU-only supported but slower; 16 GB RAM minimum, 32 GB better.	Same CPU-only PTQ1_0 route; 16 GB RAM minimum, 32 GB better.	Same CPU-only PQ2_0 route; 16 GB RAM minimum, 32 GB better.
CPU 32 GB system RAM	CPU-only supported but slower; 32 GB gives more headroom.	CPU-only supported but slower; 32 GB gives more headroom.	Same CPU-only PTQ1_0 route; 32 GB gives more headroom.	Same CPU-only PQ2_0 route; 32 GB gives more headroom.

Publisher model context limit is 262,144 tokens; starting contexts are practical runtime settings, not the model maximum.

Context window

Machine	PTQ1_0	PQ2_0	CRACK PTQ1_0	CRACK PQ2_0
Apple Silicon 8 GB unified memory	4096 tokens; tight, watch memory and raise only if headroom allows.	Not recommended; PQ2_0 targets 16 GB unified memory; use PTQ1_0.	4096 tokens; tight, watch memory; community weights quality unverified.	Not recommended; same PQ2_0 16 GB target; use PTQ1_0.
Apple Silicon 16 GB unified memory	4096 tokens; comfortable target, raise if memory allows.	8192 tokens; PQ2_0 recommended start, lower if memory pressure appears.	4096 tokens; same Metal PTQ1_0 route, raise if memory allows.	8192 tokens; same Metal PQ2_0 route, lower if memory pressure appears.
NVIDIA GPU 8 GB VRAM	4096 tokens; lower or offload layers if VRAM is short.	8192 tokens; PQ2_0 is tight at 8 GB, lower if VRAM is short.	4096 tokens; same CUDA PTQ1_0 fit, lower if VRAM is short.	8192 tokens; same CUDA PQ2_0 fit, lower if VRAM is short.
NVIDIA GPU 10–12 GB VRAM	4096 tokens; raise while watching VRAM headroom.	8192 tokens; recommended start, raise while watching VRAM headroom.	4096 tokens; same CUDA PTQ1_0 route, raise if memory allows.	8192 tokens; same CUDA PQ2_0 route, raise if memory allows.
AMD GPU 8 GB VRAM	4096 tokens; validate Radeon card and driver before raising.	8192 tokens; validate Radeon card and driver before raising.	4096 tokens; same ROCm/HIP route, validate Radeon card and driver.	8192 tokens; same ROCm/HIP route, validate Radeon card and driver.
AMD GPU 12 GB VRAM	4096 tokens; validate on target Radeon; raise if memory allows.	8192 tokens; validate on target Radeon; raise if memory allows.	4096 tokens; same ROCm/HIP route; validate on target Radeon.	8192 tokens; same ROCm/HIP route; validate on target Radeon.
Intel Arc GPU 12 GB VRAM	4096 tokens; Vulkan PTQ1_0 caveats apply; verify driver and output.	No native Vulkan PQ2_0 kernel; use PTQ1_0 or CPU.	4096 tokens; same Vulkan PTQ1_0 caveats; verify output or use CPU.	No native Vulkan PQ2_0 kernel; use PTQ1_0 or CPU.
CPU 16 GB system RAM	4096 tokens; CPU-only supported but slower; 32 GB gives more headroom.	8192 tokens; CPU-only supported but slower; 32 GB gives more headroom.	4096 tokens; same CPU-only route; 32 GB gives more headroom.	8192 tokens; same CPU-only route; 32 GB gives more headroom.
CPU 32 GB system RAM	4096 tokens; CPU-only supported but slower; raise if memory allows.	8192 tokens; CPU-only supported but slower; raise if memory allows.	4096 tokens; same CPU-only route; raise if memory allows.	8192 tokens; same CPU-only route; raise if memory allows.

Publisher model context limit is 262,144 tokens; starting contexts are practical runtime settings, not the model maximum.

LOCAL CODING / MAC · WINDOWS · LINUXSOURCE-LINKED SETUP

Build a local coding assistant with Bonsai 2 · 27B

Choose a Bonsai 2 · 27B package that fits your machine, start it with the PrismML llama.cpp fork, point Cline in VS Code at that local server, then ask for one concrete file change and review the diff before applying it. Client and server context settings must agree.

Choose a package Run its commands Check the result

BEFORE YOU BEGIN

Choose your package

Select the weights you want. The instructions, downloads and settings change with your choice.

Save your machine in My Hardware to get an automatic starting choice. You can always choose any package yourself.

CURRENT SETUP

PTQ1_0 · official

Capacity estimate from the 5.95 GB package size plus publisher guidance, not a measured test: a 6 GB GPU needs CPU/RAM offload, an 8 GB GPU is comfortable only at short context, and an 8 GB Apple Silicon Mac is tight. Start around 4K context and raise it while watching memory.

Freshness 82 out of 100. Release age: 53 days old, Sep 17, 2026. Measures age, not quality.

Popularity 85 out of 100. 1,688,388 downloads in 30 days; 2,960 lifetime likes. Checked Nov 8, 2026. Measures adoption, not quality.

What do these scores mean?

Two ways to compare models. Neither score measures answer quality, benchmark performance, or whether a model fits your machine.

Freshness · release age

Check the date source

Popularity · adoption

round(80 × min(1, log10(1 + downloads) / 7) + 20 × min(1, log10(1 + likes) / 5))

Check the statistics source

Scores as of Nov 9, 2026. Statistics checked Nov 8, 2026. Statistics are marked for refresh after 72 hours.Full scoring method

View weight source

THE WORKFLOW

Set up PTQ1_0 on your machine

Choose your computer first, then its GPU path. The instructions below update to match your selection.

CHOOSE YOUR SETUPMac

Use the PrismML llama.cpp fork with its Metal backend; the fork implements the Metal Hadamard/sign-flip and signed-fusion paths this packing needs. 8 GB unified memory is tight, 16 GB is the comfortable target, and the optional mmproj pack is loaded only for image input.

Install the PrismML Bonsai runtime (Metal fork)

Open Terminal, clone the publisher demo into your home folder and enter it. Bonsai 2 PTQ1_0 needs the PrismML llama.cpp fork because the fork implements the Metal Hadamard/sign-flip and signed-fusion paths this packing requires; stock llama.cpp or an Ollama tag cannot run it correctly. Run the setup script with the default weight downloads skipped, since this guide fetches the exact PTQ1_0 file separately and you do not want the launcher silently substituting the PQ2_0 pack. The script installs the compatible fork binary and leaves this Terminal free for the next commands.

PrismML Bonsai demo PrismML llama.cpp fork Backend and format support

cd ~
git clone https://github.com/PrismML-Eng/Bonsai-demo.git
cd Bonsai-demo
BONSAI_SKIP_GGUF=1 BONSAI_SKIP_MLX=1 BONSAI_OPENWEBUI=0 BONSAI_CODE_INTERPRETER=0 ./setup.sh

Download the exact PTQ1_0 weights

Still in the Bonsai-demo folder, create a dedicated model folder and download Ternary-Bonsai-2-27B-PTQ1_0.gguf (about 5.95 GB) from the publisher repository. The file size is not total runtime memory: the KV cache grows with the context you set. The optional mmproj pack is only for image input and is not needed for this coding task.

Put these files in ~/Bonsai-demo/models/yourunai/official-ptq1/1 required file

models/yourunai/official-ptq1/

Ternary-Bonsai-2-27B-PTQ1_0.ggufThe exact official PTQ1_0 language-model weight file used by the local server.Get file

Bonsai 2 GGUF repository

mkdir -p models/yourunai/official-ptq1
curl -fL --progress-bar "https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf/resolve/b072e1d3b35a0a630cece372c2127528e0994386/Ternary-Bonsai-2-27B-PTQ1_0.gguf" -o "models/yourunai/official-ptq1/Ternary-Bonsai-2-27B-PTQ1_0.gguf"
ls -lh models/yourunai/official-ptq1

Install VS Code and Cline

Install Visual Studio Code, then install the official Cline extension from its Marketplace page. Open Cline from the VS Code sidebar. Keep automatic approval disabled for this first task and review each requested file or command action.

Download VS Code Install Cline

Start the local Metal server with the pinned PTQ1_0 file

In the same Terminal, start the server with BONSAI_GGUF pointing at the exact PTQ1_0 file so the launcher cannot pick another installed package. BONSAI_CTX=4096 is the practical starting context for Apple Silicon PTQ1_0; the publisher model maximum is 262144 tokens, but the server only loads the value you pass here, and on an 8 GB Mac 4096 is already tight. The server keeps running and listens on http://127.0.0.1:8080. Leave this Terminal open and open a second Terminal for the client setup and test request. If macOS swaps or the server reports memory pressure, stop it with Ctrl+C and restart with BONSAI_CTX=2048.

PrismML environment variables

cd ~/Bonsai-demo
BONSAI_CTX=4096 BONSAI_GGUF="models/yourunai/official-ptq1/Ternary-Bonsai-2-27B-PTQ1_0.gguf" ./scripts/start_llama_server.sh

Open the test project in VS Code

Create a new empty folder named YouRunAI-task-check in your Documents folder. In VS Code choose File → Open Folder and select that exact folder. Confirm the Explorer shows YouRunAI-task-check before opening Cline. Changing a terminal directory alone does not change Cline’s project.

Read the served model ID and point Cline at it

In the second Terminal, ask the running server which model ID it serves; this is the OpenAI-compatible endpoint id, which is not necessarily the GGUF filename, so never guess it. Then open the Cline settings gear in VS Code and choose API Provider: OpenAI Compatible. Set Base URL to http://127.0.0.1:8080/v1, enter a non-secret local placeholder for the API key, and enter the exact id printed by the endpoint as the Model ID. Set Cline's context-window value to 4096 so it matches the server's active BONSAI_CTX; the 262144-token model maximum is not usable unless the server actually loads it. Leave command and file approval controls enabled.

Cline OpenAI-compatible configuration Cline local models overview

curl -fsS http://127.0.0.1:8080/v1/models

Make and check one small code change

After connecting Cline to the selected local model and matching the server context, switch Cline to Act mode and ask: “In this open project create counter.html, a self-contained page with a count starting at 0 and one button that adds 1. Ask before edits or commands.” Review the proposed file before approving it. Open counter.html in your browser and click twice; the count should be 2. Inspect the saved file in VS Code; newly created untracked files do not appear in a plain git diff.

Check the result and shut the server down cleanly

After you approve the proposed counter.html and the page counts to 2 on two clicks, review the saved file in the VS Code Explorer and Cline's activity log to confirm the request went to the local server rather than a remote provider. When you are finished, stop the server with Ctrl+C in the first Terminal; nothing else needs cleanup. If responses later stall or macOS reports memory pressure, start a new Cline task and relaunch the server with BONSAI_CTX=2048 before retrying rather than raising the context on a memory-constrained Mac.

SUCCESS CHECK · PTQ1_0

A reviewed change that runs

Cline shows the local Bonsai 2 · 27B server as the active provider, proposes one small, concrete file change, and asks for approval before applying it or running a command. After you read the diff, the result runs in a disposable project. Keep the client's configured context window at or below the server's active context, and keep command approvals on.

The PrismML llama.cpp fork is required for Bonsai 2 · 27B PQ2_0 and PTQ1_0; stock llama.cpp and Ollama are not substitutes, and a successful load in a stock build is not proof of compatibility. Cline's OpenAI-compatible local endpoint is source-backed but is not a hardware test. Start in a disposable repository, keep command approvals on, and treat every capacity figure here as an estimate derived from package size and publisher guidance.

Back to setup steps

When it doesn’t go to plan

The server rejects the file with an invalid ggml type, or loads it and produces unusable output.

Make sure the server binary comes from the PrismML llama.cpp fork or the Bonsai demo setup, not from stock llama.cpp or an Ollama tag. Stock builds either fail on the PQ2_0/PTQ1_0 custom types outright, or load a recognisable format while skipping the required Hadamard and sign-flip transforms and returning gibberish.

The machine slows down, swaps heavily, or the server exits with an out-of-memory error.

Close other GPU-heavy applications, lower the server context (for example to 4096), and reduce how many layers are offloaded so part of the model runs on CPU and system RAM. The GGUF file size is not total runtime memory — KV cache, longer context and the optional mmproj vision pack all add to it. On a 6 GB GPU expect partial offload.

Cline cannot reach the model, or reports a context or model error.

Confirm the local server is still running, then make Cline's OpenAI-compatible settings (base URL, model ID, context window) match the server that is actually running — including the exact model identifier the server exposes. Cline's configured context must not exceed the server's active context, so lower the client setting if the server was started small.

Windows starts a different model, or picks the wrong GPU backend.

Launch with a command that names the exact package file instead of letting a launcher scan a shared folder, so a second installed GGUF cannot be selected by accident. Then check the vendor: nvidia-smi for NVIDIA, HIP/ROCm tooling for AMD, vulkaninfo for Intel Arc. Re-run the publisher setup if the expected CUDA, HIP or Vulkan server binary is missing, and confirm the selected binary before launching.

An Intel Arc machine cannot load or misgenerates with PQ2_0.

Switch to PTQ1_0 for Intel Arc; the PrismML fork has no native Vulkan kernels for PQ2_0. If PTQ1_0 still misbehaves, check your Intel driver version — on the affected proprietary driver range the FWHT path is disabled and falls back to a much slower dense matrix multiply, which can look like a failure. The CPU route is the fallback when output stays wrong.

REFERENCE LIBRARY

Sources and files

Original instructions, model files, and compatibility notes behind this setup.

9 SOURCES

01PrismML Bonsai demo and runtimegithub.com 02PrismML llama.cpp forkgithub.com 03PrismML backend and format supportgithub.com 04Official Bonsai 2 · 27B GGUF model card (PQ2_0 and PTQ1_0)huggingface.co 05CRACK PQ2_0 community packagehuggingface.co 06CRACK PTQ1_0 community packagehuggingface.co 07Cline OpenAI-compatible provider configurationdocs.cline.bot 08Cline local models overviewdocs.cline.bot 09llama.cpp HTTP server referenceraw.githubusercontent.com

The model behind this workflow

Bonsai 2 · 27B

Keep exploring

Text

SETUP GUIDE

Qwen3.5 · 9B private chat setup

Source-backed package and platform-route plan for private local chat with Qwen3.5 · 9B. Preserves the official Ollama 9B package, a smaller 4B family fallback, and a clearly labeled community uncensored GGUF option.

Freshness 37 out of 100. Release age: 255 days old, Feb 27, 2026. Measures age, not quality.

Build a private coding assistant with Qwen3.8 27B

Run Qwen3.8 27B with Ollama, connect it to Cline in VS Code, and make one reviewable code change in a tiny local project.

Freshness 69 out of 100. Repository age: 96 days old, Aug 5, 2026. Measures age, not quality.

Arena Open Source Web coding: rank 12 of 62. Ranked model: Qwen3.8-27B. Original model ranking, not a benchmark of this package. Export dated Nov 6, 2026. Source checked Nov 8, 2026. Human preference ranking, not hardware suitability.+3

SOFTWAREOllama + Visual Studio Code + Cline + Python

Create images locally with FLUX.2 Klein

Generate images from text prompts in ComfyUI with the fast distilled or more adjustable base 4B model. Save your results and reuse the visual workflow.

Freshness 32 out of 100. Release age: 298 days old, Jan 15, 2026. Measures age, not quality.

SOFTWAREComfyUI

LOCALRENTED GPU

Verified source