Prism ML Website  |  White Paper  |  Demo & Examples  |  Discord

Ternary-Bonsai-8B-mlx-2bit

Ternary (1.58-bit) language model for Apple Silicon

7.1x smaller than FP16 | 5.2x faster on M4 Pro | 27 tok/s on iPhone | runs on Mac, iPhone, iPad

Highlights

• 2.15 GiB (2.30 GB) packed 2-bit size (down from 16.38 GB FP16) — runs comfortably on any Mac or iPhone
• Ternary weights {-1, 0, +1} across embeddings, attention projections, MLP projections, and LM head
• 75.5 avg benchmark score across 6 categories — competitive with full-precision 8B models at 1/9th the size
• 5-point improvement over our earlier 1-bit Bonsai 8B (70.5) at only ~0.6 GB additional footprint
• MLX-native format with group size 128 and FP16 scaling

Resources

• White Paper
• Demo repo — examples for serving, benchmarking, and integrating Bonsai
• Discord — community support and updates
• Kernels: MLX (Apple Silicon) · mlx-swift (iOS/macOS) — 2-bit format is supported out of the box

Model Overview

Item	Specification
Base model	Qwen3-8B
Parameters	8.19B (~6.95B non-embedding)
Architecture	GQA (32 query / 8 KV heads), SwiGLU MLP, RoPE, RMSNorm
Layers	36 Transformer decoder blocks
Context length	65,536 tokens
Vocab size	151,936
Weight format	Ternary g128: {-1, 0, +1} with FP16 group-wise scaling
Packed 2-bit size	2.15 GiB (2.30 GB)
Ternary coverage	Embeddings, attention projections, MLP projections, LM head
License	Apache 2.0

Quantization Format: Ternary g128

Each weight takes a value from {-1, 0, +1}, with one shared FP16 scale per group of 128 weights:

w_i = scale_g * t_i,    t_i in {-1, 0, +1}

The information-theoretic cost is log2(3) ≈ 1.585 bits per weight, plus FP16 group scales (16 bits per 128 weights), for a theoretical minimum of ~1.71 bits/weight. This release uses the MLX 2-bit format, which stores each ternary value in 2 bits plus group scales, for an effective ~2.125 bits/weight.

The addition of a zero value compared to binary (1-bit) provides more expressive weight representations, allowing better preservation of model quality under extreme compression.

Memory

Format	Size	Reduction	Ratio
FP16	16.38 GB	--	1.0x
MLX 2-bit g128	2.15 GiB (2.30 GB)	86.0%	7.1x

Quickstart

MLX (Python)

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("prism-ml/Ternary-Bonsai-8B-mlx-2bit")

response = generate(
    model,
    tokenizer,
    prompt="Explain quantum computing in simple terms.",
    max_tokens=256,
)
print(response)

MLX Swift (iOS / macOS)

Ternary Bonsai 8B runs natively on iPhone and iPad via MLX Swift at 27 tok/s on iPhone 17 Pro Max. The 2-bit format is supported out of the box.

Throughput (MLX / Apple Silicon)

Platform	Backend	PP512 (tok/s)	TG128 (tok/s)	FP16 TG (tok/s)	Speedup
M4 Pro 48 GB	MLX (Python)	460	83	16	5.2x

iPhone 17 Pro Max (MLX Swift)

Platform	Backend	PP512 (tok/s)	TG128 (tok/s)	4-bit TG (tok/s)	Speedup
iPhone 17 Pro Max	MLX Swift	363	27	14	1.9x

Benchmarks

Evaluated with EvalScope v1.4.2 + vLLM 0.15.1 on NVIDIA H100 under identical infrastructure, generation parameters, and scoring. All models are in the 6B-9B parameter range.

Model	Size	Avg	MMLU-R	MuSR	GSM8K	HE+	IFEval	BFCL
Qwen 3 8B	16.38 GB	79.3	83	55	93	82.3	81.5	81
Ternary Bonsai 8B	1.75 GB	75.5	72.6	56.2	91	77.4	81.8	73.9
1-bit Bonsai 8B (prior)	1.15 GB	70.5	65.7	50	88	73.8	79.8	65.7
RNJ 8B	16.63 GB	73.1	75.5	50.4	93.7	84.2	73.8	61.1
Ministral3 8B	16.04 GB	71.0	68.9	53.8	87.9	72.6	67.4	75.4
Olmo 3 7B	14.60 GB	70.9	72	56.1	92.5	79.3	87.1	38.4

Ternary Bonsai 8B ranks 2nd among all compared models despite being 1/9th the size.

Intelligence Density

density = -ln(1 - score/100) / size_GB

Model	Size	Intelligence Density (1/GB)
Ternary Bonsai 8B	1.75 GB	0.803
1-bit Bonsai 8B (prior)	1.15 GB	1.062
Qwen 3 8B	16.38 GB	0.096
RNJ 8B	16.62 GB	0.079

Limitations

• Only MLX 2-bit format is available at initial release; more formats for other backends coming soon
• Mobile power measurement is estimated rather than hardware-metered
• The full-precision frontier continues to advance; the ternary methodology is architecture-agnostic

Citation

@techreport{ternarybonsai,
    title   = {Ternary Bonsai: 1.58-bit Language Models at 8B, 4B, and 1.7B Scale},
    author  = {Prism ML},
    year    = {2026},
    month   = {April},
    url     = {https://prismml.com}
}

Contact

For questions, feedback, or collaboration inquiries: contact@prismml.com