Akahsizrr/fuse-1-Lite-GGUF

Fuse3 llama.cpp Integration

Full llama.cpp support for Fuse3 (fuse-1 Lite) — LFM2 host + Qwen3.6 coding experts.

Files

File	Purpose
fuse3.cpp	C++ graph builder (place in src/models/)
fuse3.h	C++ header (place in src/models/)
fuse3_converter.py	Python GGUF converter (place in conversion/)
export_gguf.py	Standalone GGUF exporter (no llama.cpp fork needed to create GGUF)
INTEGRATION.md	Step-by-step patches for llama.cpp source files

Quick Start

1. Create the GGUF file (no fork needed)

pip install gguf safetensors torch
python export_gguf.py --model-dir /path/to/fuse-1-Lite --output fuse-1-Lite-f16.gguf --outtype f16

2. Build llama.cpp with Fuse3 support

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp

# Apply patches (see INTEGRATION.md)
# Copy fuse3.cpp and fuse3.h to src/models/
# Copy fuse3_converter.py to conversion/
# Modify src/llama-arch.h, src/llama-arch.cpp, src/llama-model.cpp, src/CMakeLists.txt

# Build
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release

3. Run inference

./build/bin/llama-cli \
  -m fuse-1-Lite-f16.gguf \
  -p "Write a Python function to check if a number is prime." \
  -n 512 --temp 0.1

Architecture

Fuse3 extends llama.cpp's native LFM2 implementation:

• Host layers: Reuses LFM2's attention (Lfm2AttentionDecoderLayer) and short-conv (Lfm2ShortConvDecoderLayer) graph builders
• Dense FFN: Standard LFM2 SwiGLU FFN (always dense, not MoE)
• Expert augmentation: After each augmented layer's FFN, a custom MoE block runs:
  1. Router: sqrtsoftplus(x @ router_weight) → top-k selection → normalize
  2. Experts: Per-expert SwiGLU FFN (gate, up, down projections)
  3. Scale: softplus(expert_scale) clamped to 0.1
  4. Add: host_output + scale * expert_output

GGUF Tensor Names

Standard LFM2 tensors (host)

• token_embd.weight, token_embd_norm.weight, output.weight
• blk.{i}.attn_norm.weight, blk.{i}.attn_q/k/v.weight, blk.{i}.attn_output.weight
• blk.{i}.attn_q_norm.weight, blk.{i}.attn_k_norm.weight
• blk.{i}.ffn_norm.weight, blk.{i}.ffn_gate/up/down.weight
• blk.{i}.shortconv_conv/inproj/outproj.weight

Fuse3-specific tensors

• blk.{i}.fuse3_router.weight — {n_embd, n_exp} router gate
• blk.{i}.fuse3_expert_scale.weight — {1} learned scale parameter
• blk.{i}.fuse3_experts.gate.weight — {n_embd, n_ff_exp, n_exp} stacked expert gates
• blk.{i}.fuse3_experts.up.weight — {n_embd, n_ff_exp, n_exp} stacked expert up projections
• blk.{i}.fuse3_experts.down.weight — {n_ff_exp, n_embd, n_exp} stacked expert down projections

Custom GGUF metadata keys

• fuse3.augmented_layers — array of layer indices with experts
• fuse3.expert_counts — per-layer expert count array
• fuse3.swiglu_limit — float, expert activation clamp (default 10.0)
• fuse3.expert_scale_init — float, initial scale (not used at inference)

Quantization

The GGUF file can be quantized using llama.cpp's standard quantization tools:

# Q4_K_M (recommended)
./build/bin/llama-quantize fuse-1-Lite-f16.gguf fuse-1-Lite-q4_k_m.gguf q4_k_m

# Q8_0
./build/bin/llama-quantize fuse-1-Lite-f16.gguf fuse-1-Lite-q8_0.gguf q8_0

Note: Only the host LFM2 weights are quantized. The expert weights are kept in f16 for quality (they're small — 512 intermediate × ~30 experts × 30 layers ≈ 200MB).