Instructions to use IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF:F16 # Run inference directly in the terminal: llama cli -hf IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF:F16 # Run inference directly in the terminal: llama cli -hf IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF:F16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF:F16 # Run inference directly in the terminal: ./build/bin/llama cli -hf IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF:F16
Use Docker
docker model run hf.co/IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF:F16
- Llama new
- LM Studio
- Jan
- vLLM
How to use IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF:F16
- Ollama
How to use IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF with Ollama:
ollama run hf.co/IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF:F16
- Unsloth Desktop
- Pi
How to use IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF:F16
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF:F16" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF with Docker Model Runner:
docker model run hf.co/IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF:F16
- Lemonade
How to use IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF:F16
Run and chat with the model
lemonade run user.Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF-F16
List all available models
lemonade list
- Hermes Agent
How to use IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF:F16
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF:F16
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF:F16
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF:F16" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Groot-19B-A3B-261023 APEX-I MiniPlus V2.1
Run it in V-Agent
V-Agent is a free, open-source desktop agent for Windows, made by the author of these quants. It runs this model locally, with tools and clear permissions.
- Native engines: llama.cpp (CUDA, Vulkan or CPU), Strata and Strata Rust, installed from their official releases and matched to your hardware.
- Built for these quants: APEX files get their chat template from the official repo, and each quant shows whether it fits your GPU.
- Real tools, safely: files, web search, a browser and MCP servers, with each chat in its own isolated folder and approvals for every action.
- Private: no account, no telemetry; everything runs on your PC.
- Download V-Agent and install it (Windows 10/11, x64).
- Open Models, search
Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF, and pick the file marked Fits on GPU. - V-Agent downloads it with SHA-256 verification and loads it with this model's chat template.
APEX Quantization Series by IsValorum
Independently computed using dedicated cloud compute clusters, funded 100% out-of-pocket.
If you consider these releases to be of high quality and they help you in your day-to-day workflow, consider supporting on Ko-fi. Every contribution directly sustains future cloud compute hours and helps support my university studies.
Family cross-links
| Tier | Footprint | Base | Abliterated |
|---|---|---|---|
| MiniPlus V2.1 | 8.42 GB (7.84 GiB) | Groot-19B-A3B-261023 | Not published |
Only the MiniPlus V2.1 tier is published for this model. NanoPlus and MicroPlus are not part of this release.
Architecture and precision
- Base model:
GrootTeam/Groot-19B-A3B-261023(Apache-2.0) - Architecture: MoE with 40 language layers plus one MTP block; 30 linear-attention/SSM layers and 10 full-attention layers; 128 routed experts plus 1 shared expert, 8 routed experts active per token; hidden size 2048; expert intermediate size 512; 262,144-token context.
- iMatrix: public matrix from bartowski,
GrootTeam_Groot-19B-A3B-261023-imatrix.gguf; 573 chunks of 512 tokens fromcalibration-v6; 510 entries. It covers 400 of the 413 quantized tensors. The 13 uncovered tensors areoutput.weight,token_embd.weightand the MTP block, which use K-quants or Q4_0 and do not need an importance matrix. - Recurrent shielding:
ssm_alpha,ssm_a,ssm_dt.bias,ssm_conv1dand all RMSNorms remainF32. - Vision: two companion projectors are available:
mmproj-GrootTeam_Groot-19B-A3B-261023-f16.gguf(original F16) andmmproj-GrootTeam_Groot-19B-A3B-261023-q8_0.gguf(Q8_0, 610.7 MB). Vision quality was not re-evaluated after quantization. - MTP: block 40 is included in the main GGUF. Use
--spec-type draft-mtpwith llama.cpp b12017 or newer. - Tensor count: 753 tensors, matching the 753-rule MiniPlus V2.1 MTP recipe.
File and metrics
| Item | Value |
|---|---|
| File | Groot-19B-A3B-261023.APEX-I-MiniPlus-V2.1.gguf |
| Size | 8,418,216,832 bytes (8.42 GB / 7.84 GiB) |
| Bits per weight (file) | 3.5472 |
| Bits per weight (payload) | 3.5425 |
| Tensor count | 753 |
| Type histogram | 340 F32, 150 Q3_K, 123 Q5_K, 60 IQ3_XXS, 35 Q4_K, 30 Q8_0, 12 Q6_K, 3 Q4_0 |
| Quantized tensors | 413; the other 340 are norms, router logits, SSM scalars and gates held in F32 |
| Tensor map vs GGUF header | 753 of 753 names, 0 type mismatches, 0 parameter mismatches |
| Header and alignment | 10,990,976 bytes, 0.13 percent of the file |
Audited tensor map sample
| Tensor | Precision |
|---|---|
token_embd.weight |
Q4_K |
output.weight |
Q6_K |
output_norm.weight |
F32 |
blk.*.attn_qkv.weight (30 SSM layers) |
Q3_K |
blk.*.ssm_out.weight (30 SSM layers) |
Q3_K |
blk.*.ssm_beta.weight (30 SSM layers) |
Q3_K |
blk.*.attn_q.weight, attn_k.weight, attn_v.weight (10 full-attention layers plus the MTP block) |
Q4_K |
blk.*.attn_output.weight (11 layers) |
Q6_K |
blk.*.attn_gate.weight (30 SSM layers) |
Q8_0 |
blk.*.ffn_up_exps.weight, ffn_gate_exps.weight, ffn_down_exps.weight |
Q3_K in layers 0-9 and 30-39; IQ3_XXS in layers 10-29; Q4_0 in the MTP block 40 |
blk.*.ffn_up_shexp.weight, ffn_gate_shexp.weight, ffn_down_shexp.weight |
Q5_K |
blk.*.ffn_gate_inp.weight |
F32 |
blk.*.ffn_gate_inp_shexp.weight |
F32 |
blk.*.ssm_alpha.weight, blk.*.ssm_a, blk.*.ssm_dt.bias |
F32 |
blk.*.ssm_conv1d.weight |
F32 |
blk.*.attn_norm.weight, post_attention_norm.weight, ssm_norm.weight, attn_q_norm.weight, attn_k_norm.weight |
F32 |
blk.40.nextn.eh_proj.weight |
Q4_K |
blk.40.nextn.enorm.weight, hnorm.weight, shared_head_norm.weight |
F32 |
Full tensor map
Complete list: 753 tensors
| Tensor | Precision |
|---|---|
output.weight |
Q6_K |
output_norm.weight |
F32 |
token_embd.weight |
Q4_K |
blk.0.attn_gate.weight |
Q8_0 |
blk.0.attn_norm.weight |
F32 |
blk.0.attn_qkv.weight |
Q3_K |
blk.0.ffn_down_exps.weight |
Q3_K |
blk.0.ffn_down_shexp.weight |
Q5_K |
blk.0.ffn_gate_exps.weight |
Q3_K |
blk.0.ffn_gate_inp.weight |
F32 |
blk.0.ffn_gate_inp_shexp.weight |
F32 |
blk.0.ffn_gate_shexp.weight |
Q5_K |
blk.0.ffn_up_exps.weight |
Q3_K |
blk.0.ffn_up_shexp.weight |
Q5_K |
blk.0.post_attention_norm.weight |
F32 |
blk.0.ssm_a |
F32 |
blk.0.ssm_alpha.weight |
F32 |
blk.0.ssm_beta.weight |
Q3_K |
blk.0.ssm_conv1d.weight |
F32 |
blk.0.ssm_dt.bias |
F32 |
blk.0.ssm_norm.weight |
F32 |
blk.0.ssm_out.weight |
Q3_K |
blk.1.attn_gate.weight |
Q8_0 |
blk.1.attn_norm.weight |
F32 |
blk.1.attn_qkv.weight |
Q3_K |
blk.1.ffn_down_exps.weight |
Q3_K |
blk.1.ffn_down_shexp.weight |
Q5_K |
blk.1.ffn_gate_exps.weight |
Q3_K |
blk.1.ffn_gate_inp.weight |
F32 |
blk.1.ffn_gate_inp_shexp.weight |
F32 |
blk.1.ffn_gate_shexp.weight |
Q5_K |
blk.1.ffn_up_exps.weight |
Q3_K |
blk.1.ffn_up_shexp.weight |
Q5_K |
blk.1.post_attention_norm.weight |
F32 |
blk.1.ssm_a |
F32 |
blk.1.ssm_alpha.weight |
F32 |
blk.1.ssm_beta.weight |
Q3_K |
blk.1.ssm_conv1d.weight |
F32 |
blk.1.ssm_dt.bias |
F32 |
blk.1.ssm_norm.weight |
F32 |
blk.1.ssm_out.weight |
Q3_K |
blk.2.attn_gate.weight |
Q8_0 |
blk.2.attn_norm.weight |
F32 |
blk.2.attn_qkv.weight |
Q3_K |
blk.2.ffn_down_exps.weight |
Q3_K |
blk.2.ffn_down_shexp.weight |
Q5_K |
blk.2.ffn_gate_exps.weight |
Q3_K |
blk.2.ffn_gate_inp.weight |
F32 |
blk.2.ffn_gate_inp_shexp.weight |
F32 |
blk.2.ffn_gate_shexp.weight |
Q5_K |
blk.2.ffn_up_exps.weight |
Q3_K |
blk.2.ffn_up_shexp.weight |
Q5_K |
blk.2.post_attention_norm.weight |
F32 |
blk.2.ssm_a |
F32 |
blk.2.ssm_alpha.weight |
F32 |
blk.2.ssm_beta.weight |
Q3_K |
blk.2.ssm_conv1d.weight |
F32 |
blk.2.ssm_dt.bias |
F32 |
blk.2.ssm_norm.weight |
F32 |
blk.2.ssm_out.weight |
Q3_K |
blk.3.attn_k.weight |
Q4_K |
blk.3.attn_k_norm.weight |
F32 |
blk.3.attn_norm.weight |
F32 |
blk.3.attn_output.weight |
Q6_K |
blk.3.attn_q.weight |
Q4_K |
blk.3.attn_q_norm.weight |
F32 |
blk.3.attn_v.weight |
Q4_K |
blk.3.ffn_down_exps.weight |
Q3_K |
blk.3.ffn_down_shexp.weight |
Q5_K |
blk.3.ffn_gate_exps.weight |
Q3_K |
blk.3.ffn_gate_inp.weight |
F32 |
blk.3.ffn_gate_inp_shexp.weight |
F32 |
blk.3.ffn_gate_shexp.weight |
Q5_K |
blk.3.ffn_up_exps.weight |
Q3_K |
blk.3.ffn_up_shexp.weight |
Q5_K |
blk.3.post_attention_norm.weight |
F32 |
blk.4.attn_gate.weight |
Q8_0 |
blk.4.attn_norm.weight |
F32 |
blk.4.attn_qkv.weight |
Q3_K |
blk.4.ffn_down_exps.weight |
Q3_K |
blk.4.ffn_down_shexp.weight |
Q5_K |
blk.4.ffn_gate_exps.weight |
Q3_K |
blk.4.ffn_gate_inp.weight |
F32 |
blk.4.ffn_gate_inp_shexp.weight |
F32 |
blk.4.ffn_gate_shexp.weight |
Q5_K |
blk.4.ffn_up_exps.weight |
Q3_K |
blk.4.ffn_up_shexp.weight |
Q5_K |
blk.4.post_attention_norm.weight |
F32 |
blk.4.ssm_a |
F32 |
blk.4.ssm_alpha.weight |
F32 |
blk.4.ssm_beta.weight |
Q3_K |
blk.4.ssm_conv1d.weight |
F32 |
blk.4.ssm_dt.bias |
F32 |
blk.4.ssm_norm.weight |
F32 |
blk.4.ssm_out.weight |
Q3_K |
blk.5.attn_gate.weight |
Q8_0 |
blk.5.attn_norm.weight |
F32 |
blk.5.attn_qkv.weight |
Q3_K |
blk.5.ffn_down_exps.weight |
Q3_K |
blk.5.ffn_down_shexp.weight |
Q5_K |
blk.5.ffn_gate_exps.weight |
Q3_K |
blk.5.ffn_gate_inp.weight |
F32 |
blk.5.ffn_gate_inp_shexp.weight |
F32 |
blk.5.ffn_gate_shexp.weight |
Q5_K |
blk.5.ffn_up_exps.weight |
Q3_K |
blk.5.ffn_up_shexp.weight |
Q5_K |
blk.5.post_attention_norm.weight |
F32 |
blk.5.ssm_a |
F32 |
blk.5.ssm_alpha.weight |
F32 |
blk.5.ssm_beta.weight |
Q3_K |
blk.5.ssm_conv1d.weight |
F32 |
blk.5.ssm_dt.bias |
F32 |
blk.5.ssm_norm.weight |
F32 |
blk.5.ssm_out.weight |
Q3_K |
blk.6.attn_gate.weight |
Q8_0 |
blk.6.attn_norm.weight |
F32 |
blk.6.attn_qkv.weight |
Q3_K |
blk.6.ffn_down_exps.weight |
Q3_K |
blk.6.ffn_down_shexp.weight |
Q5_K |
blk.6.ffn_gate_exps.weight |
Q3_K |
blk.6.ffn_gate_inp.weight |
F32 |
blk.6.ffn_gate_inp_shexp.weight |
F32 |
blk.6.ffn_gate_shexp.weight |
Q5_K |
blk.6.ffn_up_exps.weight |
Q3_K |
blk.6.ffn_up_shexp.weight |
Q5_K |
blk.6.post_attention_norm.weight |
F32 |
blk.6.ssm_a |
F32 |
blk.6.ssm_alpha.weight |
F32 |
blk.6.ssm_beta.weight |
Q3_K |
blk.6.ssm_conv1d.weight |
F32 |
blk.6.ssm_dt.bias |
F32 |
blk.6.ssm_norm.weight |
F32 |
blk.6.ssm_out.weight |
Q3_K |
blk.7.attn_k.weight |
Q4_K |
blk.7.attn_k_norm.weight |
F32 |
blk.7.attn_norm.weight |
F32 |
blk.7.attn_output.weight |
Q6_K |
blk.7.attn_q.weight |
Q4_K |
blk.7.attn_q_norm.weight |
F32 |
blk.7.attn_v.weight |
Q4_K |
blk.7.ffn_down_exps.weight |
Q3_K |
blk.7.ffn_down_shexp.weight |
Q5_K |
blk.7.ffn_gate_exps.weight |
Q3_K |
blk.7.ffn_gate_inp.weight |
F32 |
blk.7.ffn_gate_inp_shexp.weight |
F32 |
blk.7.ffn_gate_shexp.weight |
Q5_K |
blk.7.ffn_up_exps.weight |
Q3_K |
blk.7.ffn_up_shexp.weight |
Q5_K |
blk.7.post_attention_norm.weight |
F32 |
blk.8.attn_gate.weight |
Q8_0 |
blk.8.attn_norm.weight |
F32 |
blk.8.attn_qkv.weight |
Q3_K |
blk.8.ffn_down_exps.weight |
Q3_K |
blk.8.ffn_down_shexp.weight |
Q5_K |
blk.8.ffn_gate_exps.weight |
Q3_K |
blk.8.ffn_gate_inp.weight |
F32 |
blk.8.ffn_gate_inp_shexp.weight |
F32 |
blk.8.ffn_gate_shexp.weight |
Q5_K |
blk.8.ffn_up_exps.weight |
Q3_K |
blk.8.ffn_up_shexp.weight |
Q5_K |
blk.8.post_attention_norm.weight |
F32 |
blk.8.ssm_a |
F32 |
blk.8.ssm_alpha.weight |
F32 |
blk.8.ssm_beta.weight |
Q3_K |
blk.8.ssm_conv1d.weight |
F32 |
blk.8.ssm_dt.bias |
F32 |
blk.8.ssm_norm.weight |
F32 |
blk.8.ssm_out.weight |
Q3_K |
blk.9.attn_gate.weight |
Q8_0 |
blk.9.attn_norm.weight |
F32 |
blk.9.attn_qkv.weight |
Q3_K |
blk.9.ffn_down_exps.weight |
Q3_K |
blk.9.ffn_down_shexp.weight |
Q5_K |
blk.9.ffn_gate_exps.weight |
Q3_K |
blk.9.ffn_gate_inp.weight |
F32 |
blk.9.ffn_gate_inp_shexp.weight |
F32 |
blk.9.ffn_gate_shexp.weight |
Q5_K |
blk.9.ffn_up_exps.weight |
Q3_K |
blk.9.ffn_up_shexp.weight |
Q5_K |
blk.9.post_attention_norm.weight |
F32 |
blk.9.ssm_a |
F32 |
blk.9.ssm_alpha.weight |
F32 |
blk.9.ssm_beta.weight |
Q3_K |
blk.9.ssm_conv1d.weight |
F32 |
blk.9.ssm_dt.bias |
F32 |
blk.9.ssm_norm.weight |
F32 |
blk.9.ssm_out.weight |
Q3_K |
blk.10.attn_gate.weight |
Q8_0 |
blk.10.attn_norm.weight |
F32 |
blk.10.attn_qkv.weight |
Q3_K |
blk.10.ffn_down_exps.weight |
IQ3_XXS |
blk.10.ffn_down_shexp.weight |
Q5_K |
blk.10.ffn_gate_exps.weight |
IQ3_XXS |
blk.10.ffn_gate_inp.weight |
F32 |
blk.10.ffn_gate_inp_shexp.weight |
F32 |
blk.10.ffn_gate_shexp.weight |
Q5_K |
blk.10.ffn_up_exps.weight |
IQ3_XXS |
blk.10.ffn_up_shexp.weight |
Q5_K |
blk.10.post_attention_norm.weight |
F32 |
blk.10.ssm_a |
F32 |
blk.10.ssm_alpha.weight |
F32 |
blk.10.ssm_beta.weight |
Q3_K |
blk.10.ssm_conv1d.weight |
F32 |
blk.10.ssm_dt.bias |
F32 |
blk.10.ssm_norm.weight |
F32 |
blk.10.ssm_out.weight |
Q3_K |
blk.11.attn_k.weight |
Q4_K |
blk.11.attn_k_norm.weight |
F32 |
blk.11.attn_norm.weight |
F32 |
blk.11.attn_output.weight |
Q6_K |
blk.11.attn_q.weight |
Q4_K |
blk.11.attn_q_norm.weight |
F32 |
blk.11.attn_v.weight |
Q4_K |
blk.11.ffn_down_exps.weight |
IQ3_XXS |
blk.11.ffn_down_shexp.weight |
Q5_K |
blk.11.ffn_gate_exps.weight |
IQ3_XXS |
blk.11.ffn_gate_inp.weight |
F32 |
blk.11.ffn_gate_inp_shexp.weight |
F32 |
blk.11.ffn_gate_shexp.weight |
Q5_K |
blk.11.ffn_up_exps.weight |
IQ3_XXS |
blk.11.ffn_up_shexp.weight |
Q5_K |
blk.11.post_attention_norm.weight |
F32 |
blk.12.attn_gate.weight |
Q8_0 |
blk.12.attn_norm.weight |
F32 |
blk.12.attn_qkv.weight |
Q3_K |
blk.12.ffn_down_exps.weight |
IQ3_XXS |
blk.12.ffn_down_shexp.weight |
Q5_K |
blk.12.ffn_gate_exps.weight |
IQ3_XXS |
blk.12.ffn_gate_inp.weight |
F32 |
blk.12.ffn_gate_inp_shexp.weight |
F32 |
blk.12.ffn_gate_shexp.weight |
Q5_K |
blk.12.ffn_up_exps.weight |
IQ3_XXS |
blk.12.ffn_up_shexp.weight |
Q5_K |
blk.12.post_attention_norm.weight |
F32 |
blk.12.ssm_a |
F32 |
blk.12.ssm_alpha.weight |
F32 |
blk.12.ssm_beta.weight |
Q3_K |
blk.12.ssm_conv1d.weight |
F32 |
blk.12.ssm_dt.bias |
F32 |
blk.12.ssm_norm.weight |
F32 |
blk.12.ssm_out.weight |
Q3_K |
blk.13.attn_gate.weight |
Q8_0 |
blk.13.attn_norm.weight |
F32 |
blk.13.attn_qkv.weight |
Q3_K |
blk.13.ffn_down_exps.weight |
IQ3_XXS |
blk.13.ffn_down_shexp.weight |
Q5_K |
blk.13.ffn_gate_exps.weight |
IQ3_XXS |
blk.13.ffn_gate_inp.weight |
F32 |
blk.13.ffn_gate_inp_shexp.weight |
F32 |
blk.13.ffn_gate_shexp.weight |
Q5_K |
blk.13.ffn_up_exps.weight |
IQ3_XXS |
blk.13.ffn_up_shexp.weight |
Q5_K |
blk.13.post_attention_norm.weight |
F32 |
blk.13.ssm_a |
F32 |
blk.13.ssm_alpha.weight |
F32 |
blk.13.ssm_beta.weight |
Q3_K |
blk.13.ssm_conv1d.weight |
F32 |
blk.13.ssm_dt.bias |
F32 |
blk.13.ssm_norm.weight |
F32 |
blk.13.ssm_out.weight |
Q3_K |
blk.14.attn_gate.weight |
Q8_0 |
blk.14.attn_norm.weight |
F32 |
blk.14.attn_qkv.weight |
Q3_K |
blk.14.ffn_down_exps.weight |
IQ3_XXS |
blk.14.ffn_down_shexp.weight |
Q5_K |
blk.14.ffn_gate_exps.weight |
IQ3_XXS |
blk.14.ffn_gate_inp.weight |
F32 |
blk.14.ffn_gate_inp_shexp.weight |
F32 |
blk.14.ffn_gate_shexp.weight |
Q5_K |
blk.14.ffn_up_exps.weight |
IQ3_XXS |
blk.14.ffn_up_shexp.weight |
Q5_K |
blk.14.post_attention_norm.weight |
F32 |
blk.14.ssm_a |
F32 |
blk.14.ssm_alpha.weight |
F32 |
blk.14.ssm_beta.weight |
Q3_K |
blk.14.ssm_conv1d.weight |
F32 |
blk.14.ssm_dt.bias |
F32 |
blk.14.ssm_norm.weight |
F32 |
blk.14.ssm_out.weight |
Q3_K |
blk.15.attn_k.weight |
Q4_K |
blk.15.attn_k_norm.weight |
F32 |
blk.15.attn_norm.weight |
F32 |
blk.15.attn_output.weight |
Q6_K |
blk.15.attn_q.weight |
Q4_K |
blk.15.attn_q_norm.weight |
F32 |
blk.15.attn_v.weight |
Q4_K |
blk.15.ffn_down_exps.weight |
IQ3_XXS |
blk.15.ffn_down_shexp.weight |
Q5_K |
blk.15.ffn_gate_exps.weight |
IQ3_XXS |
blk.15.ffn_gate_inp.weight |
F32 |
blk.15.ffn_gate_inp_shexp.weight |
F32 |
blk.15.ffn_gate_shexp.weight |
Q5_K |
blk.15.ffn_up_exps.weight |
IQ3_XXS |
blk.15.ffn_up_shexp.weight |
Q5_K |
blk.15.post_attention_norm.weight |
F32 |
blk.16.attn_gate.weight |
Q8_0 |
blk.16.attn_norm.weight |
F32 |
blk.16.attn_qkv.weight |
Q3_K |
blk.16.ffn_down_exps.weight |
IQ3_XXS |
blk.16.ffn_down_shexp.weight |
Q5_K |
blk.16.ffn_gate_exps.weight |
IQ3_XXS |
blk.16.ffn_gate_inp.weight |
F32 |
blk.16.ffn_gate_inp_shexp.weight |
F32 |
blk.16.ffn_gate_shexp.weight |
Q5_K |
blk.16.ffn_up_exps.weight |
IQ3_XXS |
blk.16.ffn_up_shexp.weight |
Q5_K |
blk.16.post_attention_norm.weight |
F32 |
blk.16.ssm_a |
F32 |
blk.16.ssm_alpha.weight |
F32 |
blk.16.ssm_beta.weight |
Q3_K |
blk.16.ssm_conv1d.weight |
F32 |
blk.16.ssm_dt.bias |
F32 |
blk.16.ssm_norm.weight |
F32 |
blk.16.ssm_out.weight |
Q3_K |
blk.17.attn_gate.weight |
Q8_0 |
blk.17.attn_norm.weight |
F32 |
blk.17.attn_qkv.weight |
Q3_K |
blk.17.ffn_down_exps.weight |
IQ3_XXS |
blk.17.ffn_down_shexp.weight |
Q5_K |
blk.17.ffn_gate_exps.weight |
IQ3_XXS |
blk.17.ffn_gate_inp.weight |
F32 |
blk.17.ffn_gate_inp_shexp.weight |
F32 |
blk.17.ffn_gate_shexp.weight |
Q5_K |
blk.17.ffn_up_exps.weight |
IQ3_XXS |
blk.17.ffn_up_shexp.weight |
Q5_K |
blk.17.post_attention_norm.weight |
F32 |
blk.17.ssm_a |
F32 |
blk.17.ssm_alpha.weight |
F32 |
blk.17.ssm_beta.weight |
Q3_K |
blk.17.ssm_conv1d.weight |
F32 |
blk.17.ssm_dt.bias |
F32 |
blk.17.ssm_norm.weight |
F32 |
blk.17.ssm_out.weight |
Q3_K |
blk.18.attn_gate.weight |
Q8_0 |
blk.18.attn_norm.weight |
F32 |
blk.18.attn_qkv.weight |
Q3_K |
blk.18.ffn_down_exps.weight |
IQ3_XXS |
blk.18.ffn_down_shexp.weight |
Q5_K |
blk.18.ffn_gate_exps.weight |
IQ3_XXS |
blk.18.ffn_gate_inp.weight |
F32 |
blk.18.ffn_gate_inp_shexp.weight |
F32 |
blk.18.ffn_gate_shexp.weight |
Q5_K |
blk.18.ffn_up_exps.weight |
IQ3_XXS |
blk.18.ffn_up_shexp.weight |
Q5_K |
blk.18.post_attention_norm.weight |
F32 |
blk.18.ssm_a |
F32 |
blk.18.ssm_alpha.weight |
F32 |
blk.18.ssm_beta.weight |
Q3_K |
blk.18.ssm_conv1d.weight |
F32 |
blk.18.ssm_dt.bias |
F32 |
blk.18.ssm_norm.weight |
F32 |
blk.18.ssm_out.weight |
Q3_K |
blk.19.attn_k.weight |
Q4_K |
blk.19.attn_k_norm.weight |
F32 |
blk.19.attn_norm.weight |
F32 |
blk.19.attn_output.weight |
Q6_K |
blk.19.attn_q.weight |
Q4_K |
blk.19.attn_q_norm.weight |
F32 |
blk.19.attn_v.weight |
Q4_K |
blk.19.ffn_down_exps.weight |
IQ3_XXS |
blk.19.ffn_down_shexp.weight |
Q5_K |
blk.19.ffn_gate_exps.weight |
IQ3_XXS |
blk.19.ffn_gate_inp.weight |
F32 |
blk.19.ffn_gate_inp_shexp.weight |
F32 |
blk.19.ffn_gate_shexp.weight |
Q5_K |
blk.19.ffn_up_exps.weight |
IQ3_XXS |
blk.19.ffn_up_shexp.weight |
Q5_K |
blk.19.post_attention_norm.weight |
F32 |
blk.20.attn_gate.weight |
Q8_0 |
blk.20.attn_norm.weight |
F32 |
blk.20.attn_qkv.weight |
Q3_K |
blk.20.ffn_down_exps.weight |
IQ3_XXS |
blk.20.ffn_down_shexp.weight |
Q5_K |
blk.20.ffn_gate_exps.weight |
IQ3_XXS |
blk.20.ffn_gate_inp.weight |
F32 |
blk.20.ffn_gate_inp_shexp.weight |
F32 |
blk.20.ffn_gate_shexp.weight |
Q5_K |
blk.20.ffn_up_exps.weight |
IQ3_XXS |
blk.20.ffn_up_shexp.weight |
Q5_K |
blk.20.post_attention_norm.weight |
F32 |
blk.20.ssm_a |
F32 |
blk.20.ssm_alpha.weight |
F32 |
blk.20.ssm_beta.weight |
Q3_K |
blk.20.ssm_conv1d.weight |
F32 |
blk.20.ssm_dt.bias |
F32 |
blk.20.ssm_norm.weight |
F32 |
blk.20.ssm_out.weight |
Q3_K |
blk.21.attn_gate.weight |
Q8_0 |
blk.21.attn_norm.weight |
F32 |
blk.21.attn_qkv.weight |
Q3_K |
blk.21.ffn_down_exps.weight |
IQ3_XXS |
blk.21.ffn_down_shexp.weight |
Q5_K |
blk.21.ffn_gate_exps.weight |
IQ3_XXS |
blk.21.ffn_gate_inp.weight |
F32 |
blk.21.ffn_gate_inp_shexp.weight |
F32 |
blk.21.ffn_gate_shexp.weight |
Q5_K |
blk.21.ffn_up_exps.weight |
IQ3_XXS |
blk.21.ffn_up_shexp.weight |
Q5_K |
blk.21.post_attention_norm.weight |
F32 |
blk.21.ssm_a |
F32 |
blk.21.ssm_alpha.weight |
F32 |
blk.21.ssm_beta.weight |
Q3_K |
blk.21.ssm_conv1d.weight |
F32 |
blk.21.ssm_dt.bias |
F32 |
blk.21.ssm_norm.weight |
F32 |
blk.21.ssm_out.weight |
Q3_K |
blk.22.attn_gate.weight |
Q8_0 |
blk.22.attn_norm.weight |
F32 |
blk.22.attn_qkv.weight |
Q3_K |
blk.22.ffn_down_exps.weight |
IQ3_XXS |
blk.22.ffn_down_shexp.weight |
Q5_K |
blk.22.ffn_gate_exps.weight |
IQ3_XXS |
blk.22.ffn_gate_inp.weight |
F32 |
blk.22.ffn_gate_inp_shexp.weight |
F32 |
blk.22.ffn_gate_shexp.weight |
Q5_K |
blk.22.ffn_up_exps.weight |
IQ3_XXS |
blk.22.ffn_up_shexp.weight |
Q5_K |
blk.22.post_attention_norm.weight |
F32 |
blk.22.ssm_a |
F32 |
blk.22.ssm_alpha.weight |
F32 |
blk.22.ssm_beta.weight |
Q3_K |
blk.22.ssm_conv1d.weight |
F32 |
blk.22.ssm_dt.bias |
F32 |
blk.22.ssm_norm.weight |
F32 |
blk.22.ssm_out.weight |
Q3_K |
blk.23.attn_k.weight |
Q4_K |
blk.23.attn_k_norm.weight |
F32 |
blk.23.attn_norm.weight |
F32 |
blk.23.attn_output.weight |
Q6_K |
blk.23.attn_q.weight |
Q4_K |
blk.23.attn_q_norm.weight |
F32 |
blk.23.attn_v.weight |
Q4_K |
blk.23.ffn_down_exps.weight |
IQ3_XXS |
blk.23.ffn_down_shexp.weight |
Q5_K |
blk.23.ffn_gate_exps.weight |
IQ3_XXS |
blk.23.ffn_gate_inp.weight |
F32 |
blk.23.ffn_gate_inp_shexp.weight |
F32 |
blk.23.ffn_gate_shexp.weight |
Q5_K |
blk.23.ffn_up_exps.weight |
IQ3_XXS |
blk.23.ffn_up_shexp.weight |
Q5_K |
blk.23.post_attention_norm.weight |
F32 |
blk.24.attn_gate.weight |
Q8_0 |
blk.24.attn_norm.weight |
F32 |
blk.24.attn_qkv.weight |
Q3_K |
blk.24.ffn_down_exps.weight |
IQ3_XXS |
blk.24.ffn_down_shexp.weight |
Q5_K |
blk.24.ffn_gate_exps.weight |
IQ3_XXS |
blk.24.ffn_gate_inp.weight |
F32 |
blk.24.ffn_gate_inp_shexp.weight |
F32 |
blk.24.ffn_gate_shexp.weight |
Q5_K |
blk.24.ffn_up_exps.weight |
IQ3_XXS |
blk.24.ffn_up_shexp.weight |
Q5_K |
blk.24.post_attention_norm.weight |
F32 |
blk.24.ssm_a |
F32 |
blk.24.ssm_alpha.weight |
F32 |
blk.24.ssm_beta.weight |
Q3_K |
blk.24.ssm_conv1d.weight |
F32 |
blk.24.ssm_dt.bias |
F32 |
blk.24.ssm_norm.weight |
F32 |
blk.24.ssm_out.weight |
Q3_K |
blk.25.attn_gate.weight |
Q8_0 |
blk.25.attn_norm.weight |
F32 |
blk.25.attn_qkv.weight |
Q3_K |
blk.25.ffn_down_exps.weight |
IQ3_XXS |
blk.25.ffn_down_shexp.weight |
Q5_K |
blk.25.ffn_gate_exps.weight |
IQ3_XXS |
blk.25.ffn_gate_inp.weight |
F32 |
blk.25.ffn_gate_inp_shexp.weight |
F32 |
blk.25.ffn_gate_shexp.weight |
Q5_K |
blk.25.ffn_up_exps.weight |
IQ3_XXS |
blk.25.ffn_up_shexp.weight |
Q5_K |
blk.25.post_attention_norm.weight |
F32 |
blk.25.ssm_a |
F32 |
blk.25.ssm_alpha.weight |
F32 |
blk.25.ssm_beta.weight |
Q3_K |
blk.25.ssm_conv1d.weight |
F32 |
blk.25.ssm_dt.bias |
F32 |
blk.25.ssm_norm.weight |
F32 |
blk.25.ssm_out.weight |
Q3_K |
blk.26.attn_gate.weight |
Q8_0 |
blk.26.attn_norm.weight |
F32 |
blk.26.attn_qkv.weight |
Q3_K |
blk.26.ffn_down_exps.weight |
IQ3_XXS |
blk.26.ffn_down_shexp.weight |
Q5_K |
blk.26.ffn_gate_exps.weight |
IQ3_XXS |
blk.26.ffn_gate_inp.weight |
F32 |
blk.26.ffn_gate_inp_shexp.weight |
F32 |
blk.26.ffn_gate_shexp.weight |
Q5_K |
blk.26.ffn_up_exps.weight |
IQ3_XXS |
blk.26.ffn_up_shexp.weight |
Q5_K |
blk.26.post_attention_norm.weight |
F32 |
blk.26.ssm_a |
F32 |
blk.26.ssm_alpha.weight |
F32 |
blk.26.ssm_beta.weight |
Q3_K |
blk.26.ssm_conv1d.weight |
F32 |
blk.26.ssm_dt.bias |
F32 |
blk.26.ssm_norm.weight |
F32 |
blk.26.ssm_out.weight |
Q3_K |
blk.27.attn_k.weight |
Q4_K |
blk.27.attn_k_norm.weight |
F32 |
blk.27.attn_norm.weight |
F32 |
blk.27.attn_output.weight |
Q6_K |
blk.27.attn_q.weight |
Q4_K |
blk.27.attn_q_norm.weight |
F32 |
blk.27.attn_v.weight |
Q4_K |
blk.27.ffn_down_exps.weight |
IQ3_XXS |
blk.27.ffn_down_shexp.weight |
Q5_K |
blk.27.ffn_gate_exps.weight |
IQ3_XXS |
blk.27.ffn_gate_inp.weight |
F32 |
blk.27.ffn_gate_inp_shexp.weight |
F32 |
blk.27.ffn_gate_shexp.weight |
Q5_K |
blk.27.ffn_up_exps.weight |
IQ3_XXS |
blk.27.ffn_up_shexp.weight |
Q5_K |
blk.27.post_attention_norm.weight |
F32 |
blk.28.attn_gate.weight |
Q8_0 |
blk.28.attn_norm.weight |
F32 |
blk.28.attn_qkv.weight |
Q3_K |
blk.28.ffn_down_exps.weight |
IQ3_XXS |
blk.28.ffn_down_shexp.weight |
Q5_K |
blk.28.ffn_gate_exps.weight |
IQ3_XXS |
blk.28.ffn_gate_inp.weight |
F32 |
blk.28.ffn_gate_inp_shexp.weight |
F32 |
blk.28.ffn_gate_shexp.weight |
Q5_K |
blk.28.ffn_up_exps.weight |
IQ3_XXS |
blk.28.ffn_up_shexp.weight |
Q5_K |
blk.28.post_attention_norm.weight |
F32 |
blk.28.ssm_a |
F32 |
blk.28.ssm_alpha.weight |
F32 |
blk.28.ssm_beta.weight |
Q3_K |
blk.28.ssm_conv1d.weight |
F32 |
blk.28.ssm_dt.bias |
F32 |
blk.28.ssm_norm.weight |
F32 |
blk.28.ssm_out.weight |
Q3_K |
blk.29.attn_gate.weight |
Q8_0 |
blk.29.attn_norm.weight |
F32 |
blk.29.attn_qkv.weight |
Q3_K |
blk.29.ffn_down_exps.weight |
IQ3_XXS |
blk.29.ffn_down_shexp.weight |
Q5_K |
blk.29.ffn_gate_exps.weight |
IQ3_XXS |
blk.29.ffn_gate_inp.weight |
F32 |
blk.29.ffn_gate_inp_shexp.weight |
F32 |
blk.29.ffn_gate_shexp.weight |
Q5_K |
blk.29.ffn_up_exps.weight |
IQ3_XXS |
blk.29.ffn_up_shexp.weight |
Q5_K |
blk.29.post_attention_norm.weight |
F32 |
blk.29.ssm_a |
F32 |
blk.29.ssm_alpha.weight |
F32 |
blk.29.ssm_beta.weight |
Q3_K |
blk.29.ssm_conv1d.weight |
F32 |
blk.29.ssm_dt.bias |
F32 |
blk.29.ssm_norm.weight |
F32 |
blk.29.ssm_out.weight |
Q3_K |
blk.30.attn_gate.weight |
Q8_0 |
blk.30.attn_norm.weight |
F32 |
blk.30.attn_qkv.weight |
Q3_K |
blk.30.ffn_down_exps.weight |
Q3_K |
blk.30.ffn_down_shexp.weight |
Q5_K |
blk.30.ffn_gate_exps.weight |
Q3_K |
blk.30.ffn_gate_inp.weight |
F32 |
blk.30.ffn_gate_inp_shexp.weight |
F32 |
blk.30.ffn_gate_shexp.weight |
Q5_K |
blk.30.ffn_up_exps.weight |
Q3_K |
blk.30.ffn_up_shexp.weight |
Q5_K |
blk.30.post_attention_norm.weight |
F32 |
blk.30.ssm_a |
F32 |
blk.30.ssm_alpha.weight |
F32 |
blk.30.ssm_beta.weight |
Q3_K |
blk.30.ssm_conv1d.weight |
F32 |
blk.30.ssm_dt.bias |
F32 |
blk.30.ssm_norm.weight |
F32 |
blk.30.ssm_out.weight |
Q3_K |
blk.31.attn_k.weight |
Q4_K |
blk.31.attn_k_norm.weight |
F32 |
blk.31.attn_norm.weight |
F32 |
blk.31.attn_output.weight |
Q6_K |
blk.31.attn_q.weight |
Q4_K |
blk.31.attn_q_norm.weight |
F32 |
blk.31.attn_v.weight |
Q4_K |
blk.31.ffn_down_exps.weight |
Q3_K |
blk.31.ffn_down_shexp.weight |
Q5_K |
blk.31.ffn_gate_exps.weight |
Q3_K |
blk.31.ffn_gate_inp.weight |
F32 |
blk.31.ffn_gate_inp_shexp.weight |
F32 |
blk.31.ffn_gate_shexp.weight |
Q5_K |
blk.31.ffn_up_exps.weight |
Q3_K |
blk.31.ffn_up_shexp.weight |
Q5_K |
blk.31.post_attention_norm.weight |
F32 |
blk.32.attn_gate.weight |
Q8_0 |
blk.32.attn_norm.weight |
F32 |
blk.32.attn_qkv.weight |
Q3_K |
blk.32.ffn_down_exps.weight |
Q3_K |
blk.32.ffn_down_shexp.weight |
Q5_K |
blk.32.ffn_gate_exps.weight |
Q3_K |
blk.32.ffn_gate_inp.weight |
F32 |
blk.32.ffn_gate_inp_shexp.weight |
F32 |
blk.32.ffn_gate_shexp.weight |
Q5_K |
blk.32.ffn_up_exps.weight |
Q3_K |
blk.32.ffn_up_shexp.weight |
Q5_K |
blk.32.post_attention_norm.weight |
F32 |
blk.32.ssm_a |
F32 |
blk.32.ssm_alpha.weight |
F32 |
blk.32.ssm_beta.weight |
Q3_K |
blk.32.ssm_conv1d.weight |
F32 |
blk.32.ssm_dt.bias |
F32 |
blk.32.ssm_norm.weight |
F32 |
blk.32.ssm_out.weight |
Q3_K |
blk.33.attn_gate.weight |
Q8_0 |
blk.33.attn_norm.weight |
F32 |
blk.33.attn_qkv.weight |
Q3_K |
blk.33.ffn_down_exps.weight |
Q3_K |
blk.33.ffn_down_shexp.weight |
Q5_K |
blk.33.ffn_gate_exps.weight |
Q3_K |
blk.33.ffn_gate_inp.weight |
F32 |
blk.33.ffn_gate_inp_shexp.weight |
F32 |
blk.33.ffn_gate_shexp.weight |
Q5_K |
blk.33.ffn_up_exps.weight |
Q3_K |
blk.33.ffn_up_shexp.weight |
Q5_K |
blk.33.post_attention_norm.weight |
F32 |
blk.33.ssm_a |
F32 |
blk.33.ssm_alpha.weight |
F32 |
blk.33.ssm_beta.weight |
Q3_K |
blk.33.ssm_conv1d.weight |
F32 |
blk.33.ssm_dt.bias |
F32 |
blk.33.ssm_norm.weight |
F32 |
blk.33.ssm_out.weight |
Q3_K |
blk.34.attn_gate.weight |
Q8_0 |
blk.34.attn_norm.weight |
F32 |
blk.34.attn_qkv.weight |
Q3_K |
blk.34.ffn_down_exps.weight |
Q3_K |
blk.34.ffn_down_shexp.weight |
Q5_K |
blk.34.ffn_gate_exps.weight |
Q3_K |
blk.34.ffn_gate_inp.weight |
F32 |
blk.34.ffn_gate_inp_shexp.weight |
F32 |
blk.34.ffn_gate_shexp.weight |
Q5_K |
blk.34.ffn_up_exps.weight |
Q3_K |
blk.34.ffn_up_shexp.weight |
Q5_K |
blk.34.post_attention_norm.weight |
F32 |
blk.34.ssm_a |
F32 |
blk.34.ssm_alpha.weight |
F32 |
blk.34.ssm_beta.weight |
Q3_K |
blk.34.ssm_conv1d.weight |
F32 |
blk.34.ssm_dt.bias |
F32 |
blk.34.ssm_norm.weight |
F32 |
blk.34.ssm_out.weight |
Q3_K |
blk.35.attn_k.weight |
Q4_K |
blk.35.attn_k_norm.weight |
F32 |
blk.35.attn_norm.weight |
F32 |
blk.35.attn_output.weight |
Q6_K |
blk.35.attn_q.weight |
Q4_K |
blk.35.attn_q_norm.weight |
F32 |
blk.35.attn_v.weight |
Q4_K |
blk.35.ffn_down_exps.weight |
Q3_K |
blk.35.ffn_down_shexp.weight |
Q5_K |
blk.35.ffn_gate_exps.weight |
Q3_K |
blk.35.ffn_gate_inp.weight |
F32 |
blk.35.ffn_gate_inp_shexp.weight |
F32 |
blk.35.ffn_gate_shexp.weight |
Q5_K |
blk.35.ffn_up_exps.weight |
Q3_K |
blk.35.ffn_up_shexp.weight |
Q5_K |
blk.35.post_attention_norm.weight |
F32 |
blk.36.attn_gate.weight |
Q8_0 |
blk.36.attn_norm.weight |
F32 |
blk.36.attn_qkv.weight |
Q3_K |
blk.36.ffn_down_exps.weight |
Q3_K |
blk.36.ffn_down_shexp.weight |
Q5_K |
blk.36.ffn_gate_exps.weight |
Q3_K |
blk.36.ffn_gate_inp.weight |
F32 |
blk.36.ffn_gate_inp_shexp.weight |
F32 |
blk.36.ffn_gate_shexp.weight |
Q5_K |
blk.36.ffn_up_exps.weight |
Q3_K |
blk.36.ffn_up_shexp.weight |
Q5_K |
blk.36.post_attention_norm.weight |
F32 |
blk.36.ssm_a |
F32 |
blk.36.ssm_alpha.weight |
F32 |
blk.36.ssm_beta.weight |
Q3_K |
blk.36.ssm_conv1d.weight |
F32 |
blk.36.ssm_dt.bias |
F32 |
blk.36.ssm_norm.weight |
F32 |
blk.36.ssm_out.weight |
Q3_K |
blk.37.attn_gate.weight |
Q8_0 |
blk.37.attn_norm.weight |
F32 |
blk.37.attn_qkv.weight |
Q3_K |
blk.37.ffn_down_exps.weight |
Q3_K |
blk.37.ffn_down_shexp.weight |
Q5_K |
blk.37.ffn_gate_exps.weight |
Q3_K |
blk.37.ffn_gate_inp.weight |
F32 |
blk.37.ffn_gate_inp_shexp.weight |
F32 |
blk.37.ffn_gate_shexp.weight |
Q5_K |
blk.37.ffn_up_exps.weight |
Q3_K |
blk.37.ffn_up_shexp.weight |
Q5_K |
blk.37.post_attention_norm.weight |
F32 |
blk.37.ssm_a |
F32 |
blk.37.ssm_alpha.weight |
F32 |
blk.37.ssm_beta.weight |
Q3_K |
blk.37.ssm_conv1d.weight |
F32 |
blk.37.ssm_dt.bias |
F32 |
blk.37.ssm_norm.weight |
F32 |
blk.37.ssm_out.weight |
Q3_K |
blk.38.attn_gate.weight |
Q8_0 |
blk.38.attn_norm.weight |
F32 |
blk.38.attn_qkv.weight |
Q3_K |
blk.38.ffn_down_exps.weight |
Q3_K |
blk.38.ffn_down_shexp.weight |
Q5_K |
blk.38.ffn_gate_exps.weight |
Q3_K |
blk.38.ffn_gate_inp.weight |
F32 |
blk.38.ffn_gate_inp_shexp.weight |
F32 |
blk.38.ffn_gate_shexp.weight |
Q5_K |
blk.38.ffn_up_exps.weight |
Q3_K |
blk.38.ffn_up_shexp.weight |
Q5_K |
blk.38.post_attention_norm.weight |
F32 |
blk.38.ssm_a |
F32 |
blk.38.ssm_alpha.weight |
F32 |
blk.38.ssm_beta.weight |
Q3_K |
blk.38.ssm_conv1d.weight |
F32 |
blk.38.ssm_dt.bias |
F32 |
blk.38.ssm_norm.weight |
F32 |
blk.38.ssm_out.weight |
Q3_K |
blk.39.attn_k.weight |
Q4_K |
blk.39.attn_k_norm.weight |
F32 |
blk.39.attn_norm.weight |
F32 |
blk.39.attn_output.weight |
Q6_K |
blk.39.attn_q.weight |
Q4_K |
blk.39.attn_q_norm.weight |
F32 |
blk.39.attn_v.weight |
Q4_K |
blk.39.ffn_down_exps.weight |
Q3_K |
blk.39.ffn_down_shexp.weight |
Q5_K |
blk.39.ffn_gate_exps.weight |
Q3_K |
blk.39.ffn_gate_inp.weight |
F32 |
blk.39.ffn_gate_inp_shexp.weight |
F32 |
blk.39.ffn_gate_shexp.weight |
Q5_K |
blk.39.ffn_up_exps.weight |
Q3_K |
blk.39.ffn_up_shexp.weight |
Q5_K |
blk.39.post_attention_norm.weight |
F32 |
blk.40.attn_k.weight |
Q4_K |
blk.40.attn_k_norm.weight |
F32 |
blk.40.attn_norm.weight |
F32 |
blk.40.attn_output.weight |
Q6_K |
blk.40.attn_q.weight |
Q4_K |
blk.40.attn_q_norm.weight |
F32 |
blk.40.attn_v.weight |
Q4_K |
blk.40.ffn_down_exps.weight |
Q4_0 |
blk.40.ffn_down_shexp.weight |
Q5_K |
blk.40.ffn_gate_exps.weight |
Q4_0 |
blk.40.ffn_gate_inp.weight |
F32 |
blk.40.ffn_gate_inp_shexp.weight |
F32 |
blk.40.ffn_gate_shexp.weight |
Q5_K |
blk.40.ffn_up_exps.weight |
Q4_0 |
blk.40.ffn_up_shexp.weight |
Q5_K |
blk.40.nextn.eh_proj.weight |
Q4_K |
blk.40.nextn.enorm.weight |
F32 |
blk.40.nextn.hnorm.weight |
F32 |
blk.40.nextn.shared_head_norm.weight |
F32 |
blk.40.post_attention_norm.weight |
F32 |
The machine-readable version is tensor-map.tsv.
Files
| File | Size | Notes |
|---|---|---|
Groot-19B-A3B-261023.APEX-I-MiniPlus-V2.1.gguf |
8,418,216,832 bytes | Main model, 753 tensors, MTP included |
mmproj-GrootTeam_Groot-19B-A3B-261023-f16.gguf |
899,283,264 bytes | Vision projector, original F16 |
mmproj-GrootTeam_Groot-19B-A3B-261023-q8_0.gguf |
610,655,424 bytes | Vision projector, Q8_0 |
| tensor-map.tsv | 34,081 bytes | Audited per-tensor types |
| chat_template.jinja | 7,764 bytes | Extracted from the final GGUF |
Usage
Tested with llama.cpp b12017. Groot needs a recent build; older builds such as b11512 may not load this model or its MTP block.
llama-cli
llama-cli \
-m Groot-19B-A3B-261023.APEX-I-MiniPlus-V2.1.gguf \
-p "Explain why the sky is blue in one paragraph." \
-ngl 99 -c 32768 -t 8
llama-server
llama-server \
-m Groot-19B-A3B-261023.APEX-I-MiniPlus-V2.1.gguf \
--mmproj mmproj-GrootTeam_Groot-19B-A3B-261023-q8_0.gguf \
--jinja -ngl 99 -c 32768
With MTP:
llama-server \
-m Groot-19B-A3B-261023.APEX-I-MiniPlus-V2.1.gguf \
--mmproj mmproj-GrootTeam_Groot-19B-A3B-261023-q8_0.gguf \
--jinja -ngl 99 -c 32768 --spec-type draft-mtp
The Q8_0 projector is the smaller default; use the F16 file if you prefer the original projector precision.
The model starts in thinking mode. Disable it per request with the chat template kwargs, for example enable_thinking: false, if your client exposes them.
Recommended sampling
| Mode | Temperature | top_p | top_k | presence_penalty |
|---|---|---|---|---|
| General, thinking | 1.0 | 0.95 | 20 | 1.5 |
| Precise coding | 0.6 | 0.95 | 20 | 0.0 |
| Instruct, non-thinking | 0.7 | 0.80 | 20 | 1.5 |
Use min_p = 0.0 and repetition_penalty = 1.0.
Limitations
- The base model benchmarks are self-reported by GrootTeam. They were not independently reproduced for this quantized release.
- Vision was not re-evaluated after quantization. The Q8_0 projector was generated from the original F16 projector, but vision quality was not tested.
- MTP speculative decoding was not benchmarked.
- GrootTeam does not disclose the exact method used to scale the 35B base down to 19B. Evaluate the model on your workload.
- Groot requires llama.cpp b12017 or newer.
- This release is not affiliated with GrootTeam.
License
Apache-2.0, inherited from the base model. See the base model card for the original license and terms.
- Downloads last month
- 13,016
We're not able to determine the quantization variants.
Model tree for IsValorum/Groot-19B-A3B-261023-APEX-I-MiniPlus-V2.1-GGUF
Base model
GrootTeam/Groot-19B-A3B-261023