Huihui Qwen3.8 27B abliterated GGUF
Huihui Qwen3.8 27B abliterated GGUF is a proof-of-concept, uncensored GGUF conversion of Qwen/Qwen3.8-27B created by abliteration, distributed by huihui-ai under apache-2.0. The repository offers several GGUF series derived from different base GGUF sources, including bf16, UD, UD-DW, GSQ-RCO, and Ternary variants. Abliteration is applied to selected layer ranges, with differences between updates: the latest update ablates layers 17 to 52 (0-based indexing), while update 4 and later update 5/6 series reportedly ablate only layers 22 to 52. The base model card lists pipeline_tag image-text-to-text and notes that MTP and visual components were not modified. The ternary hybrid-attention files require the PrismML-Eng/llama.cpp fork and will not run with stock llama.cpp. Some weights below Q8_0 were converted to Q8_0 (filenames K_L), Q8_0 weights were changed to BF16 (Q8_0_L), and the resulting file sizes may differ from the original GGUF. The model card labels this as a test/validation and a crude proof-of-concept implementation.
At a glance
- Architecture
- unknown
- License
- apache-2.0
- Disk space
- 15.4 GB
- Software
- PrismML llama.cpp fork
Will it run on your machine?
Save your machine to see a personalized rating and its reasoning.
Add my hardwareBest for
- Local GGUF inference via llama.cpp-compatible tooling
- Loading with Transformers where supported by the model format
- Image-text-to-text prompting
- Experimentation with uncensored/abliterated model behavior
- Evaluating different GGUF quantization series (bf16, UD, UD-DW, GSQ-RCO, Ternary)
Tradeoffs
- Uncensored/abliterated model behavior may be unsuitable for some applications
- Ternary hybrid-attention files require the PrismML-Eng/llama.cpp fork, not stock llama.cpp
- Abliteration is partial and differs by update, affecting only selected layer ranges
- This is described by the source as a crude proof-of-concept and test/validation
- Converted GGUF sizes may differ from the original, and nonstandard quantization naming/sizing may be confusing
Ways to run it
PrismML llama.cpp fork · Windows, macOS, Linux
Repository-specific command in publisher documentation; confirm dependencies and hardware in the source.
llama-cli -m huihui-ai/Huihui-Qwen3.8-27B-abliterated-GGUF/Huihui-Qwen3.8-27B-abliterated-Q4_K.gguf -c 262144Explore its uses
Keep exploring
Bonsai 2 · 27B
A compressed 27B-class reasoning model with publisher-provided GGUF packs and a dedicated llama.cpp fork for CUDA, Metal, and CPU.
DeepSeek V4.1 Flash
A multimodal reasoning model with a compressed key-value cache, published as open weights.
Qwen3 8B
Qwen3-8B is a causal language model from the Qwen series, developed by the Qwen team. It is an 8.2B parameter model (6.95B non-embedding) with 36 layers and grouped-query attention (32 query heads, 8 key/value heads). The model supports text generation and is designed for both thinking and non-thinking modes within a single model. It has a native context length of 32,768 tokens, extendable to 131,072 tokens with YaRN. The model card highlights advancements in reasoning, instruction-following, agent capabilities, and multilingual support (100+ languages). It is licensed under Apache-2.0 and built on Qwen/Qwen3-8B-Base.