Create README.md

README.md

@@ -0,0 +1,61 @@
+---
+license: apache-2.0
+pipeline_tag: text-generation
+library_name: mlx
+tags:
+- mlx
+base_model: nex-agi/Nex-N2-Pro
+---
+
+> [!NOTE]
+> Uploading... 🐌
+
+[Nex-N2-Pro](https://huggingface.co/nex-agi/Nex-N2-Pro) optimized for MLX.
+
+- A mixed-precision quant that balances speed, memory, and accuracy.
+- 4-bit baseline with important layers at higher precision.
+- Supports image input and requires a vision-capable MLX server.
+
+# Usage
+
+```sh
+# Start server at http://localhost:8080/v1/chat/completions
+uvx --from mlx-vlm mlx_vlm.server \
+  --host 127.0.0.1 \
+  --port 8080 \
+  --model spicyneuron/Nex-N2-Pro-MLX-5.3bit-vision
+```
+
+# Benchmarks
+
+Tested on a Mac Studio M3 Ultra.
+
+metric | this model
+--- | ---
+bpw | 5.349
+base memory | 246.796
+peak memory (1024/512) | 267.043
+prompt tok/s (1024) | 475.490 ± 0.195
+gen tok/s (512) | 30.802 ± 0.154
+kl mean\* | 0.012 ± 0.001
+kl p95\* | 0.029 ± 0.001
+perplexity | 3.677 ± 0.023
+ifbench_strict | 0.470 ± 0.050
+ifbench_loose | 0.520 ± 0.050
+arc_challenge | 0.696 ± 0.021
+hellaswag | 0.922 ± 0.012
+
+\*KL was measured against a stronger local reference using a cached baseline.
+
+# Methodology
+
+Quantized with a [mlx-vlm fork](https://github.com/spicyneuron/mlx-vlm/tree/override).
+MLX quantization options differ than llama.cpp, but the principles are the same:
+
+- Sensitive layers like MoE routing, attention, and output embeddings get higher precision
+- More tolerant layers like MoE experts get lower precision
+
+Related tooling:
+
+- [Benchmark VLMs with `mlx_lm`](https://github.com/ml-explore/mlx-lm/pull/1033)
+- [`mlx_lm.kld` command](https://github.com/ml-explore/mlx-lm/pull/1146)