Bonsai 2 · 27B
A compressed 27B-class reasoning model with publisher-provided GGUF packs and a dedicated llama.cpp fork for CUDA, Metal, and CPU.
What do you want to do?
Choose a task. Get the software, files, settings, and steps to run it.
Set up private chat with Bonsai 2 · 27B
Run Bonsai 2 · 27B locally for private chat using PrismML's llama.cpp fork. Choose an official ternary package or a community CRACK uncensored build, then follow the route for your operating system and accelerator backend.
Build a local coding assistant with Bonsai 2 · 27B
Stand up a private, on-device coding assistant: pick a Bonsai 2 · 27B ternary GGUF package, serve it with the PrismML llama.cpp fork, connect the local OpenAI-compatible endpoint to Cline in VS Code, and complete one small, reviewable code change. Bonsai 2 · 27B is a 27.36B-parameter hybrid-attention model with a 262K-token context, so a single workstation can hold a full assistant-sized model; the packed PQ2_0 and PTQ1_0 types only run correctly in the PrismML fork, not stock llama.cpp or Ollama.
Build a local tool-using agent with Bonsai 2 · 27B
Source-backed metadata for a local agent setup using Bonsai 2 · 27B with PrismML’s llama.cpp fork and Hermes Agent. Choose the smallest official PTQ1_0 pack, a community abliterated PTQ1_0 alternative, or the official PQ2_0 pack used in PrismML’s agent demo.
Run a private reasoning assistant with Bonsai 2 · 27B
Select a source-backed Bonsai 2 · 27B package, start the PrismML llama.cpp fork, then verify a multi-step reasoning answer locally.
Open a guide for its software, hardware requirements, and setup steps.
At a glance
- Parameters
- 27.36B
- Architecture
- Ternary hybrid attention
- Context length
- 262,144
- License
- Apache 2.0
- Disk space
- 7.21 GB
- Software
- PrismML llama.cpp fork
Will it run on your machine?
Save your machine to see a personalized rating and its reasoning.
Add my hardwareWhich Bonsai is this?
Bonsai 2 is the newer Qwen3.8-based release. The original Bonsai 27B is Qwen3.6-based and has separate ternary and smaller 1-bit packages. A different fit badge can also reflect different evidence: Bonsai 2 has a publisher RTX 5090 measurement; the original packages have a memory-based estimate for that card.
Best for
- Private reasoning and coding on hardware that cannot hold conventional 27B FP16 weights.
Tradeoffs
- Requires Prism ML’s modified llama.cpp runtime; stock builds do not support these packed weights.
- The optional vision tower increases memory use for image input.
Ways to run it
PrismML llama.cpp fork · Windows, macOS, Linux
Use the publisher fork or its pinned binary. Stock llama.cpp cannot run the ternary packs correctly.
hf download prism-ml/Ternary-Bonsai-2-27B-gguf Ternary-Bonsai-2-27B-PQ2_0.gguf --local-dir .Explore its uses
Sources and files
Primary sources and files behind this model page.
Keep exploring
DeepSeek V4.1 Flash
An open-weight model for reasoning, coding, agents, and image understanding. DeepSeek V4.1 Flash combines a 552B backbone with sparse activation and up to one million tokens of context. Choose the original checkpoint or a community uncensored variant below.
Qwen3.8
Explore 4 versions of Qwen3.8. Compare model sizes, packages, hardware requirements, and available setups.
Qwen3 8B
Qwen3-8B is a causal language model from the Qwen series, developed by the Qwen team. It is an 8.2B parameter model (6.95B non-embedding) with 36 layers and grouped-query attention (32 query heads, 8 key/value heads). The model supports text generation and is designed for both thinking and non-thinking modes within a single model. It has a native context length of 32,768 tokens, extendable to 131,072 tokens with YaRN. The model card highlights advancements in reasoning, instruction-following, agent capabilities, and multilingual support (100+ languages). It is licensed under Apache-2.0 and built on Qwen/Qwen3-8B-Base.