MOSS-Transcribe-preview-2B-MLX-4bit

4-bit MLX quantization of OpenMOSS-Team/MOSS-Transcribe-preview-2B -- an English speech-to-text model (Qwen3-Omni-MoE audio encoder + gated-MLP adapter + Qwen3-1.7B decoder) -- running natively on Apple Silicon.

At a glance

Source	OpenMOSS-Team/MOSS-Transcribe-preview-2B @ c98175cb20e48bd9be4e95f6c85f2af18899f780 (bf16, 4.84 GB)
Architecture	Qwen3-Omni-MoE audio encoder (32 layers) + gated-MLP adapter + Qwen3-1.7B decoder (28 layers)
Format	MLX, single safetensors tree
Quantization	4-bit affine, group-size 64 (Qwen3 decoder only, 197 tensors; audio encoder + adapter preserved bf16)
Disk size	2.2 GB

Audio path

The MOSS audio path remains partly unquantized in BF16: model.audio_model.* (525 tensors) and model.audio_adapter.* (3 tensors) remain BF16, while only model.language_model.* is quantized.

Audio inference smoke

PASS — this variant passed the audio-inference smoke gate on 2026-08-02.

Gate design: the public smoke gate is the same three fixed clean-English clips from the recorded preview evidence — e1.wav, e2.wav, and e3.wav. non-English audio is out of scope for this preview card copy and does not participate in the release verdict.

clip	CER vs MLX-BF16	gating	wall_s
e1.wav	0.0	yes	0.109
e2.wav	0.0	yes	0.113
e3.wav	0.0	yes	0.126

• Decoder-only smoke (one-token forward, finite logits): pass
• Total wall clock: 72.682 s

Family

All MLX variants of this model ship together:

• majentik/MOSS-Transcribe-preview-2B-MLX-8bit
• majentik/MOSS-Transcribe-preview-2B-MLX-6bit
• majentik/MOSS-Transcribe-preview-2B-MLX-4bit (this repo)

Usage

Runs on the MOSS-Transcribe MLX harness in majek/pipelines/moss_transcribe_mlx (Qwen3-Omni audio encoder vendored from mlx-vlm, decoder via stock mlx_lm.models.qwen3):

python -m pipelines.moss_transcribe_mlx.generate \
    --path majentik/MOSS-Transcribe-preview-2B-MLX-4bit \
    --wav your_clip.wav

Stock mlx-lm/mlx-vlm alone cannot load this composite audio architecture. This is an English ASR model (Open ASR Leaderboard); non-English audio is out of domain.

Reproduce

python -m pipelines.mlx_direct_quantize --model moss-transcribe-preview-2b \
    --base-dir <upstream snapshot> --out-dir <out> \
    --bits 4 --mode affine --group-size 64

License

Apache-2.0, inherited from the upstream model.