Instructions to use Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF:UD-IQ1_M
# Run inference directly in the terminal:
llama cli -hf Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF:UD-IQ1_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF:UD-IQ1_M
# Run inference directly in the terminal:
llama cli -hf Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF:UD-IQ1_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF:UD-IQ1_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF:UD-IQ1_M

Use Docker

docker model run hf.co/Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF:UD-IQ1_M

Llama new
LM Studio
Jan

vLLM

How to use Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF:UD-IQ1_M

Ollama
How to use Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF with Ollama:
```
ollama run hf.co/Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF:UD-IQ1_M
```
Unsloth Desktop

How to use Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF:UD-IQ1_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF:UD-IQ1_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF with Docker Model Runner:
```
docker model run hf.co/Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF:UD-IQ1_M
```

Lemonade

How to use Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF:UD-IQ1_M

Run and chat with the model

lemonade run user.MiniMax-M3.1-Flash-StrixHalo-GGUF-UD-IQ1_M

List all available models

lemonade list

Hermes Agent

How to use Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF:UD-IQ1_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF:UD-IQ1_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF:UD-IQ1_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF:UD-IQ1_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

MiniMax-M3.1-Flash for Strix Halo (128 GB)

Made for one AMD Strix Halo with 128 GB of unified memory

Each build is about 97 GB (90 GiB). That leaves room for a 64K context (about 8 GB of KV cache) inside a ~100 GiB GPU memory budget.

The size Hugging Face shows on the right side of this page adds up both builds. You download only one.

Vote on what we quantize next: harshavardhan3002.github.io/ROCmFPX.

Status: community test release. It is checked on the CPU and on an NVIDIA GB10 with CUDA (perplexity and GSM8K below), but it has not been run on a Strix Halo yet. Please report speed, memory and quality in the Community tab.

What this is

MiniMax-M3.1-Flash cut down to fit one AMD Ryzen AI Max+ 395 (Strix Halo) box. Unsloth's smallest dynamic quants are still 129 GB (UD-IQ1_M) and 135 GB (UD-IQ2_XXS), too big for 128 GB of shared memory. These builds take Unsloth's files and remove the least-used routed experts in every MoE layer. Nothing is re-quantized: every kept tensor is Unsloth's, byte for byte.

Files

Folder	Experts kept per layer	Based on	Size	Perplexity (lower is better)
`UD-IQ1_M-E93/` (recommended)	93 of 128	Unsloth UD-IQ1_M	96.6 GB	61.96
`UD-IQ2_XXS-E89/`	89 of 128	Unsloth UD-IQ2_XXS	96.8 GB	62.11
Unsloth UD-IQ1_M, not pruned (reference)	128	-	129.4 GB	52.02

Each build is 3 files. Download one folder and point the runtime at the first file.

Perplexity: 8 chunks of 2048 tokens of mixed English Wikipedia text and source code, CPU, same text for all rows. The two builds are about equal; keeping more experts at the lower bit width wins slightly, as with our DeepSeek build. The loss against the unpruned file is clear: this is the price of fitting into 128 GB.

Test results

Build	GSM8K, first 200 test questions	Answers cut off	Mean answer length	Prompt speed (pp512, GB10)
`UD-IQ1_M-E93`	191 / 200 (95.5%)	0	285 tokens	218 t/s
`UD-IQ2_XXS-E89`	191 / 200 (95.5%)	2	357 tokens	315 t/s

GSM8K method: llama-server with the embedded chat template, greedy decoding, up to 6144 output tokens, 4 requests in parallel. Each question ends with "Solve the problem step by step. End your reply with a last line of the form 'Final answer: '." and that number is scored. We have no unpruned score with the same setup to compare against. GSM8K is easy for a model of this size, so it shows that the pruned builds still reason correctly, not that they match the original on harder tasks.
Where it ran: an NVIDIA DGX Spark (GB10, CUDA), because that is where we have the compute. These are not Strix Halo numbers. Generation speed (tg128) was about 12.6 t/s, but other CPU jobs were running on the machine at the same time, so treat it as a lower bound.
Backend differences: on the same text, CUDA and CPU agree on the top token only 82 to 83% of the time (mean KL divergence 0.20 to 0.26). The router of the pruned model often has near-ties between experts, so tiny rounding differences between backends pick different experts. Both backends give sensible answers, but do not expect identical output on Vulkan, CUDA and CPU.

What is inside

All weights: from unsloth/MiniMax-M3.1-Flash-GGUF, copied unchanged. Only routed experts (and their router rows) are removed.
Which experts are kept: ranked per layer by the REAP score (router weight times expert output size, averaged over the tokens that reach the expert), measured over 98,304 tokens of mixed text and code. There is no full-precision copy on our side, so the scores were measured on Unsloth's UD-IQ2_XXS file. The 93-expert build keeps 88.1% of the routing weight, the 89-expert build 86.0%. The kept lists are in keep/.
Text only. The vision projector and the speculative draft model of the original are not included and were not tested.

How to run

git clone https://github.com/HarshaVardhan3002/ROCmFPX
cd ROCmFPX
cmake -B build -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j

hf download Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF --include "UD-IQ1_M-E93/*" --local-dir .

./build/bin/llama-server \
  -m UD-IQ1_M-E93/MiniMax-M3.1-Flash-StrixHalo-UD-IQ1_M-E93-00001-of-00003.gguf \
  -ngl 99 -c 65536 --jinja

Notes:

The GPU must be able to use about 100 GiB. On Windows set the largest graphics memory in the BIOS or AMD Software that still leaves the system enough RAM; on Linux raise the GTT limit. Close other GPU programs.
It is a thinking model: give it enough output tokens.
For perplexity on raw text, add --override-kv tokenizer.ggml.add_bos_token=bool:true,tokenizer.ggml.bos_token_id=int:200034 (one flag, comma separated; the chat template already adds the BOS token).
Quick speed test: ./build/bin/llama-bench -m UD-IQ1_M-E93/MiniMax-M3.1-Flash-StrixHalo-UD-IQ1_M-E93-00001-of-00003.gguf -ngl 99 -p 512 -n 128
Expect low speed at first: the 1 to 2 bit expert kernels on Vulkan are not tuned yet. They are the next target in the ROCmFPX fork.

Credits and license

Original model: MiniMaxAI/MiniMax-M3.1-Flash, MiniMax Community License (copy in LICENSE). Non-commercial use is free. Commercial use has conditions (a "Built with MiniMax M3.1-Flash" notice, a notice or a written authorization from MiniMax depending on revenue) and the license lists prohibited uses. Read LICENSE before you use these files.
Quantization: Unsloth (UD dynamic quants). All credit for the quantized weights goes to them.
Expert ranking: REAP criterion (router-weighted expert activation pruning).

This repository only selects and repackages those weights.

Downloads last month: 1,371

GGUF

Model size

313B params

Architecture

minimax-m3

Hardware compatibility

1-bit

2-bit

Model tree for Harsha001/MiniMax-M3.1-Flash-StrixHalo-GGUF

Base model

MiniMaxAI/MiniMax-M3.1-Flash

Quantized

(39)

this model