Instructions to use Arki05/North-Mini-Code-1.0-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Arki05/North-Mini-Code-1.0-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M

Use Docker

docker model run hf.co/Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use Arki05/North-Mini-Code-1.0-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Arki05/North-Mini-Code-1.0-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Arki05/North-Mini-Code-1.0-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M

Ollama
How to use Arki05/North-Mini-Code-1.0-GGUF with Ollama:
```
ollama run hf.co/Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M
```
Unsloth Desktop

How to use Arki05/North-Mini-Code-1.0-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Arki05/North-Mini-Code-1.0-GGUF with Docker Model Runner:
```
docker model run hf.co/Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M
```

Lemonade

How to use Arki05/North-Mini-Code-1.0-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.North-Mini-Code-1.0-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use Arki05/North-Mini-Code-1.0-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Arki05/North-Mini-Code-1.0-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Arki05/North-Mini-Code-1.0-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Arki05 commited on Jun 10

Commit

d556012

verified ·

1 Parent(s): 80349a8

Upload folder using huggingface_hub

Browse files

Files changed (15) hide show

.gitattributes +12 -0
BLS-Mini-Code-1.0-BF16-00001-of-00002.gguf → North-Mini-Code-1.0-BF16-00001-of-00002.gguf +1 -1
BLS-Mini-Code-1.0-BF16-00002-of-00002.gguf → North-Mini-Code-1.0-BF16-00002-of-00002.gguf +0 -0
BLS-Mini-Code-1.0-IQ2_M.gguf → North-Mini-Code-1.0-IQ2_M.gguf +1 -1
BLS-Mini-Code-1.0-IQ2_XS.gguf → North-Mini-Code-1.0-IQ2_XS.gguf +1 -1
BLS-Mini-Code-1.0-IQ2_XXS.gguf → North-Mini-Code-1.0-IQ2_XXS.gguf +1 -1
BLS-Mini-Code-1.0-IQ3_M.gguf → North-Mini-Code-1.0-IQ3_M.gguf +1 -1
BLS-Mini-Code-1.0-IQ4_XS.gguf → North-Mini-Code-1.0-IQ4_XS.gguf +1 -1
BLS-Mini-Code-1.0-Q4_K_M.gguf → North-Mini-Code-1.0-Q4_K_M.gguf +1 -1
BLS-Mini-Code-1.0-Q5_K_M.gguf → North-Mini-Code-1.0-Q5_K_M.gguf +1 -1
BLS-Mini-Code-1.0-Q6_K.gguf → North-Mini-Code-1.0-Q6_K.gguf +1 -1
BLS-Mini-Code-1.0-Q8_0.gguf → North-Mini-Code-1.0-Q8_0.gguf +1 -1
BLS-Mini-Code-1.0.imatrix → North-Mini-Code-1.0.imatrix +0 -0
README.md +23 -11
eval-corpora.tar.zst +2 -2

.gitattributes CHANGED Viewed

@@ -45,3 +45,15 @@ BLS-Mini-Code-1.0-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
 BLS-Mini-Code-1.0-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
 BLS-Mini-Code-1.0-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
 BLS-Mini-Code-1.0.imatrix filter=lfs diff=lfs merge=lfs -text

 BLS-Mini-Code-1.0-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
 BLS-Mini-Code-1.0-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
 BLS-Mini-Code-1.0.imatrix filter=lfs diff=lfs merge=lfs -text
+North-Mini-Code-1.0-BF16-00001-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
+North-Mini-Code-1.0-BF16-00002-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
+North-Mini-Code-1.0-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
+North-Mini-Code-1.0-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
+North-Mini-Code-1.0-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+North-Mini-Code-1.0-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
+North-Mini-Code-1.0-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+North-Mini-Code-1.0-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+North-Mini-Code-1.0-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+North-Mini-Code-1.0-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+North-Mini-Code-1.0-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+North-Mini-Code-1.0.imatrix filter=lfs diff=lfs merge=lfs -text

BLS-Mini-Code-1.0-BF16-00001-of-00002.gguf → North-Mini-Code-1.0-BF16-00001-of-00002.gguf RENAMED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4070c9914714241ca3e1facc069c8a337afd0406761ed97a6f41098457467e8f
 size 44796475616

 version https://git-lfs.github.com/spec/v1
+oid sha256:e037609a4170dddde94277bf3497d8e1450068aae4bbdbcdcb023137cb397b2e
 size 44796475616

BLS-Mini-Code-1.0-BF16-00002-of-00002.gguf → North-Mini-Code-1.0-BF16-00002-of-00002.gguf RENAMED Viewed

File without changes

BLS-Mini-Code-1.0-IQ2_M.gguf → North-Mini-Code-1.0-IQ2_M.gguf RENAMED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4389183c1268066a65e673351995b18736aacd4e6005667ec8715ce38f167e39
 size 10258257728

 version https://git-lfs.github.com/spec/v1
+oid sha256:76bf6be2729ee57dc8e2889861e0eca93a82a8165e25cb1d433d9212f02f8ee7
 size 10258257728

BLS-Mini-Code-1.0-IQ2_XS.gguf → North-Mini-Code-1.0-IQ2_XS.gguf RENAMED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6f02052c3140c3dad78e05f7d436ea698b1269a53fe5e736fcd0e83ab6229b8f
 size 9208223552

 version https://git-lfs.github.com/spec/v1
+oid sha256:a0c1f6cc368f4712c721a4ebc4c04ca5d26939d6d6f6fed82857a9eb62a128af
 size 9208223552

BLS-Mini-Code-1.0-IQ2_XXS.gguf → North-Mini-Code-1.0-IQ2_XXS.gguf RENAMED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4d461356a56907cb57f10b2ea8862396d3b92e017b7a76982e4ed9fe3bc1dfca
 size 8306022208

 version https://git-lfs.github.com/spec/v1
+oid sha256:57a5f7b8f571388e23bddb3f481ed4cdd905e428a4b3aaf4a0d1843a34b2cb7f
 size 8306022208

BLS-Mini-Code-1.0-IQ3_M.gguf → North-Mini-Code-1.0-IQ3_M.gguf RENAMED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1d4c034a4179540646537386906d63afe692b1d4994ff4238daee42ca2486bd2
 size 13560125248

 version https://git-lfs.github.com/spec/v1
+oid sha256:c9983a8c53abef1ea7f63525702748d4cb6443b2f809e115aee91d01925db4eb
 size 13560125248

BLS-Mini-Code-1.0-IQ4_XS.gguf → North-Mini-Code-1.0-IQ4_XS.gguf RENAMED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6ab79c61ff36ad7280e588887df870f460fae4821cd3e7a403bde442f7cac2a1
 size 16412456768

 version https://git-lfs.github.com/spec/v1
+oid sha256:ed667972eb5553f4286cd509f3b5f282f4a031d705680c43bbcfef65435ff7c0
 size 16412456768

BLS-Mini-Code-1.0-Q4_K_M.gguf → North-Mini-Code-1.0-Q4_K_M.gguf RENAMED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b55a5fefbfb3e26bdd135badc63deeb99c1b732ea67da8895e8ab319a6f6ed60
 size 18593978176

 version https://git-lfs.github.com/spec/v1
+oid sha256:1dae9d226358088d3518b4fe7badd7c081cd1f64d11b5541187c788532c018e6
 size 18593978176

BLS-Mini-Code-1.0-Q5_K_M.gguf → North-Mini-Code-1.0-Q5_K_M.gguf RENAMED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:672791ad83e7117dc9960294d957d0dfc4a8142c9dd0a5ec75e2b79f489975ee
 size 21727778624

 version https://git-lfs.github.com/spec/v1
+oid sha256:24c2a4360e5af87f859b65e9517cfe691929b98daa40cd21ab26d15e6efc2c91
 size 21727778624

BLS-Mini-Code-1.0-Q6_K.gguf → North-Mini-Code-1.0-Q6_K.gguf RENAMED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:81986d9a4fc5131fde0f4ed30fc5be6bca0d5a9564015269fd36cce5b17c26f5
 size 25057441600

 version https://git-lfs.github.com/spec/v1
+oid sha256:e1aa31df862f134858e59cfbd88bf73abf379978235a505fbc1da12d5c8f013b
 size 25057441600

BLS-Mini-Code-1.0-Q8_0.gguf → North-Mini-Code-1.0-Q8_0.gguf RENAMED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6ba4a1c8bafca1f70321efddb55517fdd59a6803b914be505808d8ac4cf695bc
 size 32437286720

 version https://git-lfs.github.com/spec/v1
+oid sha256:f3fec34f80131923eeb5ce40e53fc1bcc1eb0830362a2afe94556fe719360e87
 size 32437286720

BLS-Mini-Code-1.0.imatrix → North-Mini-Code-1.0.imatrix RENAMED Viewed

File without changes

README.md CHANGED Viewed

@@ -1,5 +1,6 @@
 ---
-base_model: CohereLabs/BLS-Mini-Code-1.0
 pipeline_tag: text-generation
 library_name: gguf
 tags:
@@ -7,30 +8,40 @@ tags:
   - moe
   - code
   - reasoning
   - gguf
 quantized_by: Arki05
 ---
-# BLS-Mini-Code-1.0 — GGUF
-GGUF quantizations of [CohereLabs/BLS-Mini-Code-1.0](https://huggingface.co/CohereLabs/BLS-Mini-Code-1.0),
 a 30.5B-total / ~2.9B-active sparse MoE code model by Cohere (`cohere2moe`
 architecture: Command-R7B-style hybrid SWA/full attention with NoPE on global
 layers, parallel residual blocks, 128 fine-grained experts with sigmoid top-8
-routing, reasoning-by-default chat format).
 > **Status / requirements:** needs llama.cpp with `cohere2moe` support —
 > [PR #24260](https://github.com/ggml-org/llama.cpp/pull/24260) (not yet merged).
-> Build that branch until it lands. The upstream model repo currently ships
-> **no license**; these files inherit whatever terms Cohere attaches to the
-> original weights.
 ## Quants
 All quality numbers are measured against the **bf16 model as ground truth**.
 The headline table uses **wikitext-2 (test)** — the only evaluation set that is
 fully held out from the imatrix calibration data — plus HumanEval/HumanEval+
-(pass@1, greedy, thinking on, 6k token budget; remaining quants in progress).
 | file | size | PPL | mean KLD | top-1 % | HumanEval | HumanEval+ |
 |---|---|---|---|---|---|---|
@@ -143,7 +154,7 @@ support automatically (`thinking = 1`), separates `reasoning_content` from
 working; rendering is byte-identical for native invocations.
 ```bash
-llama-server -m BLS-Mini-Code-1.0-Q5_K_M.gguf --jinja
 ```
 - thinking on (default): response arrives as `reasoning_content` + `content`
@@ -153,7 +164,7 @@ llama-server -m BLS-Mini-Code-1.0-Q5_K_M.gguf --jinja
 ## imatrix
-`BLS-Mini-Code-1.0.imatrix` (included) was computed on the **bf16** model over
 the v3 + code + chat mix described above (326x512-token chunks), reaching full
 coverage of all 128 experts in every layer.
@@ -164,5 +175,6 @@ coverage of all 128 experts in every layer.
   26/27, mean |dlogprob| 0.012 - the only disagreement a 0.013 near-tie.
 - Tool calling, parallel calls, multi-turn with reasoning passback, and a live
   agentic tool-execution loop verified end to end via `llama-server`.
-- 500k context advertised by the model; KV cache at long context stays small
   thanks to iSWA (only 13 of 49 layers are global; ~13.6 GB KV at 500k).

 ---
+base_model: CohereLabs/North-Mini-Code-1.0
+license: apache-2.0
 pipeline_tag: text-generation
 library_name: gguf
 tags:
   - moe
   - code
   - reasoning
+  - agent
   - gguf
 quantized_by: Arki05
 ---
+# North-Mini-Code-1.0 — GGUF
+GGUF quantizations of [CohereLabs/North-Mini-Code-1.0](https://huggingface.co/CohereLabs/North-Mini-Code-1.0),
 a 30.5B-total / ~2.9B-active sparse MoE code model by Cohere (`cohere2moe`
 architecture: Command-R7B-style hybrid SWA/full attention with NoPE on global
 layers, parallel residual blocks, 128 fine-grained experts with sigmoid top-8
+routing, reasoning-by-default chat format). Trained with SFT followed by RL
+with verifiable rewards, aimed at agentic coding and terminal/tool-use work —
+see the [release blog post](https://huggingface.co/blog/CohereLabs/introducing-north-mini-code).
+> **Provenance:** these GGUFs were converted from the weights Cohere first
+> published as `CohereLabs/BLS-Mini-Code-1.0` (the pre-release name; the repo
+> was renamed in place for the public launch). All 49 safetensors shards and
+> the tokenizer of the final `North-Mini-Code-1.0` release are SHA256-identical
+> to that pre-release — same weights, new name. Only file names and the GGUF
+> `general.name`/`general.basename` metadata were updated here; tensor data is
+> untouched, so all measurements below remain valid.
 > **Status / requirements:** needs llama.cpp with `cohere2moe` support —
 > [PR #24260](https://github.com/ggml-org/llama.cpp/pull/24260) (not yet merged).
+> Build that branch until it lands. Weights are released under **Apache 2.0**,
+> and these files inherit that license.
 ## Quants
 All quality numbers are measured against the **bf16 model as ground truth**.
 The headline table uses **wikitext-2 (test)** — the only evaluation set that is
 fully held out from the imatrix calibration data — plus HumanEval/HumanEval+
+(pass@1, greedy, thinking on, 6k token budget).
 | file | size | PPL | mean KLD | top-1 % | HumanEval | HumanEval+ |
 |---|---|---|---|---|---|---|
 working; rendering is byte-identical for native invocations.
 ```bash
+llama-server -m North-Mini-Code-1.0-Q5_K_M.gguf --jinja
 ```
 - thinking on (default): response arrives as `reasoning_content` + `content`
 ## imatrix
+`North-Mini-Code-1.0.imatrix` (included) was computed on the **bf16** model over
 the v3 + code + chat mix described above (326x512-token chunks), reaching full
 coverage of all 128 experts in every layer.
   26/27, mean |dlogprob| 0.012 - the only disagreement a 0.013 near-tie.
 - Tool calling, parallel calls, multi-turn with reasoning passback, and a live
   agentic tool-execution loop verified end to end via `llama-server`.
+- The official model card states 256K input / 64K output context; the config's
+  `max_position_embeddings` is 500k. KV cache at long context stays small
   thanks to iSWA (only 13 of 49 layers are global; ~13.6 GB KV at 500k).

eval-corpora.tar.zst CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:52c1d605d8429819d8e4f56a5fabfccee8b799e29e0e1e61c88a13f30a49551e
-size 640688

 version https://git-lfs.github.com/spec/v1
+oid sha256:03eb4c521fe39eddea66bc581123f9a239b98b877586a65d050e787e8068d1b3
+size 638292