Instructions to use Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF",
	filename="Qwen3.6-35B-A3B-Abliterated-Heretic-BF16/Qwen3.6-35B-A3B-Abliterated-Heretic-BF16.gguf-00001-of-00002.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF:Q4_K_M

Use Docker

docker model run hf.co/Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF:Q4_K_M

Ollama
How to use Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF with Ollama:
```
ollama run hf.co/Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF:Q4_K_M
```

Unsloth Studio new

How to use Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF to start chatting

Pi new

How to use Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF with Docker Model Runner:
```
docker model run hf.co/Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF:Q4_K_M
```

Lemonade

How to use Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF-Q4_K_M

List all available models

lemonade list

Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF

This is a GGUF release of an abliterated version of Qwen's Qwen3.6-35B-A3B.

By applying Heretic on the Qwen 3.6 sparse-MoE text stack, the base refusal behavior was removed at the weight level. The result keeps Qwen3.6-35B-A3B's multimodal architecture and general capability profile, while no longer defaulting to the original refusal pattern.

Quick Benchmarks

Check	Original Qwen3.6-35B-A3B	Abliterated Heretic
Official 25-prompt refusal check	22/25 refusals	1/25 refusals
Archived Heretic KL divergence	-	0.010655362159013748

Methodology & Model Notes

Qwen3.6-35B-A3B is a 35.95B sparse MoE vision-language model with roughly 3B active parameters per token, 40 text layers, 256 routed experts, and 8 active experts per token.

This release was produced with a Heretic MPOA/SOMA-style sibling-transfer run, finalized with a split-MoE input-side intervention on the accepted candidate.

The accepted candidate scored Refusals: 1/25 on the official 25-prompt marker suite used for the MiniMax M2.7 abliterated run.

The resulting abliterated checkpoint was exported to BF16 and then converted to GGUF for llama.cpp-compatible deployment.

Files

Qwen3.6-35B-A3B-Abliterated-Heretic-BF16/: BF16 GGUF source
Qwen3.6-35B-A3B-Abliterated-Heretic-Q8_0/: highest-fidelity quant
Qwen3.6-35B-A3B-Abliterated-Heretic-Q6_K/: near-lossless practical quant
Qwen3.6-35B-A3B-Abliterated-Heretic-Q4_K_M/: smaller general-use quant
mmproj-Qwen3.6-35B-A3B-Abliterated-Heretic.gguf: matching multimodal projector file for llama.cpp vision use

Running

llama-server \
  -m <quant-file.gguf> \
  --mmproj <mmproj-file.gguf> \
  -ngl 999 -c 32768 --jinja -fa

Model Architecture

Spec	Value
Total Parameters	35.95B (sparse MoE)
Active Parameters	~3B per token
Experts	256 routed, 8 per token
Layers	40
Hidden Size	2048
Family	`qwen3_5_moe`
Modality	Vision-language
Base Model	Qwen/Qwen3.6-35B-A3B

Disclaimer

This model has had refusal behavior removed at the weight level. It will answer prompts that the base model would normally refuse. You are responsible for how you use it.

Credits

Base model: Qwen/Qwen3.6-35B-A3B
Refusal removal pipeline: Heretic
GGUF runtime and quantization: llama.cpp

License

This release inherits the base Qwen3.6-35B-A3B license.

Apache-2.0.

Downloads last month: 11,287

GGUF

Model size

35B params

Architecture

qwen35moe

Hardware compatibility

4-bit

6-bit

8-bit

16-bit

Model tree for Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-GGUF

Base model

Qwen/Qwen3.6-35B-A3B

Quantized

(405)

this model

Quantizations

1 model