MOSS-Image Text Encoder โ€” Heretic โ€” DHQ GGUF

GGUF builds of pottokao/MOSS-Image-Text-Encoder-Heretic (decensored Qwen3.5-9B) for MOSS-Image.

One file does two jobs: it rewrites a short idea into a full MOSS-Image caption, then encodes that caption for the DiT. Weights are loaded once. A Q4-class file fits an 8 GB card.

The quantization is DHQ (DiT-aware quantization): every tensor's format is chosen from measured damage on two rulers at once โ€” the error after MOSS-Image's own text stage (what the DiT sees) and the KL of the rewrite โ€” instead of llama.cpp's fixed per-type rules. At the same file size, every DHQ build beats the standard llama.cpp quant on both rulers.

How to use

Use the Dual-Head Dragon ComfyUI node: https://github.com/pottokao-dotcom/ComfyUI-DualHead-Dragon (MOSS-Image support: products/moss.py).

  1. Put a .gguf from this repo (and mmproj-moss-te-f16.gguf if you want image input) in ComfyUI/models/text_encoders/.
  2. Dual-Head Dragon Loader (GGUF) โ†’ gguf = this file, type = moss_image (auto also detects it).
  3. Connect its CLIP output to a normal CLIP Text Encode, or to Dual-Head Rewrite first to expand a short idea (the rewrite runs with thinking off, as a prompt enhancer does).
  4. DiT / VAE as usual: moss_image_sft_* (or a Turbo / DMD build) and flux2-vae_small from Comfy-Org/MOSS-Image.

The loader reads the last layer after the final RMSNorm with MOSS-Image's own chat template (open think block), exactly what the official pipeline encodes.

Loaders other than the Dual-Head Dragon node: as of this release, ComfyUI-GGUF does not map Qwen3.5 GGUF tensor names, so it cannot load these files yet. The K builds contain K-quants only (no IQ tensors, checked file by file), so they are ready for loaders that do. The IQ builds are for llama.cpp-based engines only (ComfyUI-GGUF dequantizes IQ types on the CPU).

Files

Error = relative L2 error after MOSS-Image's text stage, on 36 held-out prompts, run through the real llama.cpp engine (lower is better). "vs heretic" isolates quantization; "vs official" is what the DiT trained on (it includes the heretic shift itself, 13.5% for the unquantized model). Rewrite KL = llama-perplexity KL vs the BF16 model on 80 of its own rewrites.

Dual-head (rewrite + encode), K-quants only

file size error vs heretic error vs official rewrite KL same-size llama.cpp quant
moss-te-heretic-dhq-K-6.6G.gguf 6.64 GB 4.9% 14.5% 0.0053 Q5_K_M: 6.7% / 0.0135
moss-te-heretic-dhq-K-5.8G.gguf 5.78 GB 7.3% 15.5% 0.0115 Q4_K_M: 9.1% / 0.0229
moss-te-heretic-dhq-K-4.7G.gguf 4.74 GB 14.3% 20.2% 0.0472 Q3_K_M: 16.0% / 0.0741
moss-te-heretic-dhq-K-4.2G.gguf 4.17 GB 21.3% 25.7% 0.113 โ€”
moss-te-heretic-dhq-K-3.9G.gguf 3.87 GB 25.5% 29.1% 0.170 โ€”

Dual-head, IQ (llama.cpp engines only)

file size error vs heretic error vs official rewrite KL same-size llama.cpp quant
moss-te-heretic-dhq-IQ-5.3G.gguf 5.33 GB 8.7% 16.4% 0.0171 IQ4_XS: 9.3% / 0.0191
moss-te-heretic-dhq-IQ-4.5G-M.gguf 4.53 GB 15.4% 20.9% 0.0542 IQ3_M: 15.9% / 0.0601
moss-te-heretic-dhq-IQ-4.5G-S.gguf 4.48 GB 15.8% 21.1% 0.0578 IQ3_S: 16.3% / 0.0616
moss-te-heretic-dhq-IQ-4.2G.gguf 4.16 GB 19.2% 23.9% 0.090 โ€”

On this model the IQ builds gain little: the K 4.7 GB file beats IQ 4.5 GB on both rulers. Part of the reason is known: the simulated IQ errors used for allocation do not always match what llama.cpp writes (e.g. forcing layer 0 linear_attn.out_proj from IQ3_S to Q8_0 in a stock IQ3_M file changes nothing, though the simulation flagged it), so the IQ allocation is less well informed than the K one. The files are still measured on real GGUFs, as for all rows.

Encoder only (no lm_head, no MTP head โ€” cannot rewrite)

file size error vs heretic error vs official
moss-te-only-dhq-K-3.2G.gguf 3.21 GB 21.3% 25.6%
moss-te-only-dhq-K-2.9G.gguf 2.92 GB 26.1% 29.8%
moss-te-only-dhq-IQ-2.9G.gguf 2.92 GB 24.2% โ€”

Removing lm_head and the MTP head changes nothing for encoding (identical error to the full BF16 model).

Vision tower

mmproj-moss-te-f16.gguf (0.92 GB, F16) โ€” the Qwen3.5 vision tower, identical to the official one (Heretic did not touch it). Needed only for image input (looking at a reference image while rewriting, community edit LoRAs).

What the DiT sees

Same prompt and seed, MOSS-Image SFT Turbo int8 (8 steps, cfg 1), every encoder run through the Dual-Head Dragon node. The text-stage errors above barely show; the heretic encoder renders like the official one, and the signage is spelled right down to 3.9 GB.

cat sign portrait swimsuit

Low-bit files (4.2 GB and below) keep images intact, but their rewrite KL is high: expect a weaker prompt enhancer.

How it was made

  • Base: pottokao/MOSS-Image-Text-Encoder-Heretic โ†’ BF16 GGUF (llama.cpp, MTP head kept), imatrix from calibration_datav3 + 78 MOSS-style captions (PPL 7.29).
  • Sensitivity: per-tensor, per-format Fisher probes on both rulers (24 prompts ร— 4 probes; 9 formats from Q2_K to Q8_0, including IQ3_XXS / IQ3_S / IQ4_XS). The DiT ruler is MOSS-Image's text_embedder (proj + 2 blocks), the same weights in SFT and Turbo.
  • Allocation: start at the floor, buy upgrades by damage removed per byte until the budget (the same-size llama.cpp file) is spent. Rewrite and encoder are weighted equally in the dual-head builds; encoder only for the encoder-only builds.
  • The single most sensitive tensor is layer 0 linear_attn.out_proj. llama.cpp's rules leave it at Q4; moving only that tensor to Q8_0 (8 MB more) lowers the stock Q4_K_M error from 9.1% to 8.0% (measured). DHQ does this and re-allocates the other 247 tensors as well, which is where most of its gain comes from. (The linearized probes overstate this tensor's share โ€” they put it at 65% of the Q4 damage โ€” so treat per-tensor shares as a ranking, not as exact sizes.)
  • recipes/ holds every file's per-tensor type map and the imatrix, so each file can be rebuilt bit for bit with llama-quantize --imatrix recipes/imatrix.gguf --tensor-type ... <bf16.gguf> <out.gguf> <base type>. SHA256SUMS lists the file hashes.

Use responsibly

Adults only. Do not depict minors or age-ambiguous people, non-consensual acts, or real people without consent.

License

Apache-2.0, same as Qwen3.5-9B.

Downloads last month
16,648
GGUF
Model size
9B params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

We're not able to determine the quantization variants.

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF

Finetuned
Qwen/Qwen3.5-9B
Quantized
(4)
this model