File size: 1,681 Bytes
1307249 0551c09 1307249 0551c09 1307249 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 | ---
license: apache-2.0
pipeline_tag: text-generation
library_name: mlx
tags:
- mlx
base_model: nex-agi/Nex-N2-Pro
---
[Nex-N2-Pro](https://huggingface.co/nex-agi/Nex-N2-Pro) optimized for MLX.
This is one of the best coding models that runs on a Mac Studio!
- A mixed-precision quant that balances speed, memory, and accuracy.
- 4-bit baseline with important layers at higher precision.
- Supports image input and requires a vision-capable MLX server.
# Usage
```sh
# Start server at http://localhost:8080/v1/chat/completions
uvx --from mlx-vlm mlx_vlm.server \
--host 127.0.0.1 \
--port 8080 \
--model spicyneuron/Nex-N2-Pro-MLX-5.3bit-vision
```
# Benchmarks
Tested on a Mac Studio M3 Ultra.
metric | this model
--- | ---
bpw | 5.349
base memory | 246.796
peak memory (1024/512) | 267.043
prompt tok/s (1024) | 475.490 ± 0.195
gen tok/s (512) | 30.802 ± 0.154
kl mean\* | 0.012 ± 0.001
kl p95\* | 0.029 ± 0.001
perplexity | 3.677 ± 0.023
ifbench_strict | 0.470 ± 0.050
ifbench_loose | 0.520 ± 0.050
arc_challenge | 0.696 ± 0.021
hellaswag | 0.922 ± 0.012
\*KL was measured against the largest quant I could run (~495GB), so real value is higher.
# Methodology
Quantized with a [mlx-vlm fork](https://github.com/spicyneuron/mlx-vlm/tree/override).
MLX quantization options differ than llama.cpp, but the principles are the same:
- Sensitive layers like MoE routing, attention, and output embeddings get higher precision
- More tolerant layers like MoE experts get lower precision
Related tooling:
- [Benchmark VLMs with `mlx_lm`](https://github.com/ml-explore/mlx-lm/pull/1033)
- [`mlx_lm.kld` command](https://github.com/ml-explore/mlx-lm/pull/1146)
|