Instructions to use pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF:F16 # Run inference directly in the terminal: llama cli -hf pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF:F16 # Run inference directly in the terminal: llama cli -hf pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF:F16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF:F16 # Run inference directly in the terminal: ./build/bin/llama cli -hf pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF:F16
Use Docker
docker model run hf.co/pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF:F16
- Llama new
- LM Studio
- Jan
- Ollama
How to use pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF with Ollama:
ollama run hf.co/pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF:F16
- Unsloth Desktop
- Pi
How to use pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF:F16
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF:F16" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF with Docker Model Runner:
docker model run hf.co/pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF:F16
- Lemonade
How to use pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF:F16
Run and chat with the model
lemonade run user.MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF-F16
List all available models
lemonade list
- Hermes Agent
How to use pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF:F16
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF:F16
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF:F16
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF:F16" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
MOSS-Image Text Encoder โ Heretic โ DHQ GGUF
GGUF builds of pottokao/MOSS-Image-Text-Encoder-Heretic (decensored Qwen3.5-9B) for MOSS-Image.
One file does two jobs: it rewrites a short idea into a full MOSS-Image caption, then encodes that caption for the DiT. Weights are loaded once. A Q4-class file fits an 8 GB card.
The quantization is DHQ (DiT-aware quantization): every tensor's format is chosen from measured damage on two rulers at once โ the error after MOSS-Image's own text stage (what the DiT sees) and the KL of the rewrite โ instead of llama.cpp's fixed per-type rules. At the same file size, every DHQ build beats the standard llama.cpp quant on both rulers.
How to use
Use the Dual-Head Dragon ComfyUI node: https://github.com/pottokao-dotcom/ComfyUI-DualHead-Dragon
(MOSS-Image support: products/moss.py).
- Put a
.gguffrom this repo (andmmproj-moss-te-f16.ggufif you want image input) inComfyUI/models/text_encoders/. - Dual-Head Dragon Loader (GGUF) โ
gguf= this file,type=moss_image(autoalso detects it). - Connect its CLIP output to a normal CLIP Text Encode, or to Dual-Head Rewrite first to expand a short idea (the rewrite runs with thinking off, as a prompt enhancer does).
- DiT / VAE as usual:
moss_image_sft_*(or a Turbo / DMD build) andflux2-vae_smallfrom Comfy-Org/MOSS-Image.
The loader reads the last layer after the final RMSNorm with MOSS-Image's own chat template (open think block), exactly what the official pipeline encodes.
Loaders other than the Dual-Head Dragon node: as of this release, ComfyUI-GGUF does not map Qwen3.5 GGUF tensor names, so it cannot load these files yet. The K builds contain K-quants only (no IQ tensors, checked file by file), so they are ready for loaders that do. The IQ builds are for llama.cpp-based engines only (ComfyUI-GGUF dequantizes IQ types on the CPU).
Files
Error = relative L2 error after MOSS-Image's text stage, on 36 held-out prompts, run through the real llama.cpp engine (lower is better). "vs heretic" isolates quantization; "vs official" is what the DiT trained on (it includes the heretic shift itself, 13.5% for the unquantized model). Rewrite KL = llama-perplexity KL vs the BF16 model on 80 of its own rewrites.
Dual-head (rewrite + encode), K-quants only
| file | size | error vs heretic | error vs official | rewrite KL | same-size llama.cpp quant |
|---|---|---|---|---|---|
moss-te-heretic-dhq-K-6.6G.gguf |
6.64 GB | 4.9% | 14.5% | 0.0053 | Q5_K_M: 6.7% / 0.0135 |
moss-te-heretic-dhq-K-5.8G.gguf |
5.78 GB | 7.3% | 15.5% | 0.0115 | Q4_K_M: 9.1% / 0.0229 |
moss-te-heretic-dhq-K-4.7G.gguf |
4.74 GB | 14.3% | 20.2% | 0.0472 | Q3_K_M: 16.0% / 0.0741 |
moss-te-heretic-dhq-K-4.2G.gguf |
4.17 GB | 21.3% | 25.7% | 0.113 | โ |
moss-te-heretic-dhq-K-3.9G.gguf |
3.87 GB | 25.5% | 29.1% | 0.170 | โ |
Dual-head, IQ (llama.cpp engines only)
| file | size | error vs heretic | error vs official | rewrite KL | same-size llama.cpp quant |
|---|---|---|---|---|---|
moss-te-heretic-dhq-IQ-5.3G.gguf |
5.33 GB | 8.7% | 16.4% | 0.0171 | IQ4_XS: 9.3% / 0.0191 |
moss-te-heretic-dhq-IQ-4.5G-M.gguf |
4.53 GB | 15.4% | 20.9% | 0.0542 | IQ3_M: 15.9% / 0.0601 |
moss-te-heretic-dhq-IQ-4.5G-S.gguf |
4.48 GB | 15.8% | 21.1% | 0.0578 | IQ3_S: 16.3% / 0.0616 |
moss-te-heretic-dhq-IQ-4.2G.gguf |
4.16 GB | 19.2% | 23.9% | 0.090 | โ |
On this model the IQ builds gain little: the K 4.7 GB file beats IQ 4.5 GB on both rulers. Part of the reason is
known: the simulated IQ errors used for allocation do not always match what llama.cpp writes (e.g. forcing layer 0
linear_attn.out_proj from IQ3_S to Q8_0 in a stock IQ3_M file changes nothing, though the simulation flagged it), so
the IQ allocation is less well informed than the K one. The files are still measured on real GGUFs, as for all rows.
Encoder only (no lm_head, no MTP head โ cannot rewrite)
| file | size | error vs heretic | error vs official |
|---|---|---|---|
moss-te-only-dhq-K-3.2G.gguf |
3.21 GB | 21.3% | 25.6% |
moss-te-only-dhq-K-2.9G.gguf |
2.92 GB | 26.1% | 29.8% |
moss-te-only-dhq-IQ-2.9G.gguf |
2.92 GB | 24.2% | โ |
Removing lm_head and the MTP head changes nothing for encoding (identical error to the full BF16 model).
Vision tower
mmproj-moss-te-f16.gguf (0.92 GB, F16) โ the Qwen3.5 vision tower, identical to the official one (Heretic did not
touch it). Needed only for image input (looking at a reference image while rewriting, community edit LoRAs).
What the DiT sees
Same prompt and seed, MOSS-Image SFT Turbo int8 (8 steps, cfg 1), every encoder run through the Dual-Head Dragon node. The text-stage errors above barely show; the heretic encoder renders like the official one, and the signage is spelled right down to 3.9 GB.
Low-bit files (4.2 GB and below) keep images intact, but their rewrite KL is high: expect a weaker prompt enhancer.
How it was made
- Base:
pottokao/MOSS-Image-Text-Encoder-Hereticโ BF16 GGUF (llama.cpp, MTP head kept), imatrix from calibration_datav3 + 78 MOSS-style captions (PPL 7.29). - Sensitivity: per-tensor, per-format Fisher probes on both rulers (24 prompts ร 4 probes; 9 formats from Q2_K to
Q8_0, including IQ3_XXS / IQ3_S / IQ4_XS). The DiT ruler is MOSS-Image's
text_embedder(proj + 2 blocks), the same weights in SFT and Turbo. - Allocation: start at the floor, buy upgrades by damage removed per byte until the budget (the same-size llama.cpp file) is spent. Rewrite and encoder are weighted equally in the dual-head builds; encoder only for the encoder-only builds.
- The single most sensitive tensor is layer 0
linear_attn.out_proj. llama.cpp's rules leave it at Q4; moving only that tensor to Q8_0 (8 MB more) lowers the stock Q4_K_M error from 9.1% to 8.0% (measured). DHQ does this and re-allocates the other 247 tensors as well, which is where most of its gain comes from. (The linearized probes overstate this tensor's share โ they put it at 65% of the Q4 damage โ so treat per-tensor shares as a ranking, not as exact sizes.) recipes/holds every file's per-tensor type map and the imatrix, so each file can be rebuilt bit for bit withllama-quantize --imatrix recipes/imatrix.gguf --tensor-type ... <bf16.gguf> <out.gguf> <base type>.SHA256SUMSlists the file hashes.
Use responsibly
Adults only. Do not depict minors or age-ambiguous people, non-consensual acts, or real people without consent.
License
Apache-2.0, same as Qwen3.5-9B.
- Downloads last month
- 16,648
We're not able to determine the quantization variants.
Model tree for pottokao/MOSS-Image-Text-Encoder-Heretic-DHQ-GGUF
Base model
Qwen/Qwen3.5-9B-Base


