---
license: apache-2.0
base_model: Qwen/Qwen2.5-7B
tags:
  - quantization
  - ternary
  - balanced-ternary
  - tritllm
library_name: transformers
---

# Qwen2.5-7B-trit-uniform-d4

Balanced ternary quantization of [`Qwen/Qwen2.5-7B`](https://huggingface.co/Qwen/Qwen2.5-7B) at depth **d=4** (81 levels per weight, **6.64 bits per weight**).

Produced with the codec from **"Balanced Ternary Post-Training Quantization for Large Language Models"** (Stentzel, 2026). See [Entrit/tritllm-codec](https://huggingface.co/Entrit/tritllm-codec) for the codec source.

## Quick load

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Entrit/Qwen2.5-7B-trit-uniform-d4")
tokenizer = AutoTokenizer.from_pretrained("Entrit/Qwen2.5-7B-trit-uniform-d4")
```

The weights are dequantized to FP16 for stock-`transformers` compatibility. The on-disk size is therefore the same as the FP16 source. The 6.64-bpw figure refers to the *information content* of the quantized matrices and is what matters for inference on hardware that consumes the packed trit format directly (see [Entrit/tritllm-kernel](https://huggingface.co/Entrit/tritllm-kernel)).

## Quantization details

| Field | Value |
|---|---|
| Source model | [`Qwen/Qwen2.5-7B`](https://huggingface.co/Qwen/Qwen2.5-7B) |
| Depth | d=4 (81 levels) |
| Bits per weight | 6.64 |
| Group size | 16 |
| Scale codebook | 27-entry log-spaced (scale_depth=3) |
| Method | Uniform PTQ |
| Quantized layers | all 2D linear matrices |
| Kept FP16 | `lm_head`, token embeddings, all `*_norm` layers |
| Codec | tritllm v2 |

## Citation

```
@article{stentzel2026ternaryptq,
  title  = {Balanced Ternary Post-Training Quantization for Large Language Models},
  author = {Stentzel, Eric},
  year   = 2026,
  note   = {Entrit Systems}
}
```

## Reproducibility

```bash
git clone https://huggingface.co/Entrit/tritllm-codec
cd tritllm-codec
python quantize_model_v2.py --model Qwen/Qwen2.5-7B --configs uniform-d4 --out ./out
```