Hugging Face's logo

IFM
/
K2-Horizon-32B-FP8

Text Generation
Transformers
Safetensors
English
k2_horizon
k2-horizon
32b
dense
open-weights
ifm
conversational
custom_code
compressed-tensors
Model card Files
xet
 Community
1
LiqunMa commited on
Commit
f51a202
·
verified ·
1 Parent(s): 5f2f399

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +69 -457
README.md CHANGED
@@ -16,7 +16,7 @@ tags:
16
  - ifm
17
  ---
18
 
19
- # K2-Horizon-32B-FP8
20
 
21
  [Training Code](https://github.com/LLM360/xllm) - [Evaluation Code](https://github.com/LLM360/Eval360-V2) - [Pretraining Data](https://huggingface.co/datasets/IFM/K2-Horizon-Pretrain-Data) - [Midtraining Data](https://huggingface.co/datasets/IFM/K2-Horizon-Midtrain-Data)
22
 
@@ -29,504 +29,116 @@ tags:
29
  >
30
  > The FP8 model performs closely in line with the original BF16 model on our evaluations, while reducing memory footprint and enabling faster inference on FP8-capable hardware.
31
 
32
- K2-Horizon-32B is a large dense base model in the IFM K2-Horizon family.
33
 
34
- K2-Horizon is an open-weight IFM model family built for transparent foundation-model research, staged checkpoint analysis, and practical deployment. The series spans compact dense models for local experimentation, dense mid-size and large models for high-quality research workloads, and sparse expert models for higher-capacity serving.
 
 
35
 
36
- All model cards in this family intentionally share the same structure and family-level reference material. Each repository focuses on one model size, but the card also describes the full release series so users can compare architectures, context length, serving requirements, and benchmark behavior before selecting a model.
37
 
38
- The repositories provide base/pretraining and staged midtraining revisions, making it possible to study capability changes across training phases instead of evaluating only one checkpoint. The `main` branch contains the default checkpoint, while stable stage tags such as `base_final`, `mid_1_final`, and `mid_4_final` identify specific model states for reproducible use.
 
 
 
39
 
 
40
 
41
- ## Model Series Overview
 
 
42
 
43
- K2-Horizon is released as a family of checkpoints that share the same post-training recipe, release process, and user-facing deployment guidance while covering dense, MoE, and MoVA configurations. The table below summarizes the public family view; individual repositories contain the files and branches for one model size.
44
 
45
- | Model | Architecture | Stored parameters | Active parameters/token | Context length | Vocab size | Intended use |
46
- | --- | --- | --- | --- | --- | --- | --- |
47
- | K2-Horizon-0.9B | K2HorizonForCausalLM; dense decoder-only | 0.9B | 0.9B | 8,192 | 64,256 | Lightweight experimentation, local development, evaluation dry runs |
48
- | K2-Horizon-3.7B | XllmForCausalLM; dense decoder-only | 5.06B | 5.06B | 524,288 | 250,624 | Efficient research, evaluation, and single-node serving |
49
- | K2-Horizon-7B | XllmForCausalLM; dense decoder-only | 9.00B | 9.00B | 524,288 | 250,624 | General research, fine-tuning, and cost-conscious deployment |
50
- | K2-Horizon-32B | K2HorizonForCausalLM; dense decoder-only | 34.78B | 34.78B | 524,288 | 250,624 | Stronger long-context and reasoning experiments |
51
- | K2-Horizon-36B | K2HorizonForCausalLM; MoVA + MoE | 37.44B | 5.95B | 524,288 | 250,624 | Multistage research and production-style serving experiments |
52
- | K2-Horizon-375B | XllmForCausalLM; sparse MoE | 379.17B | 26.67B | 524,288 | 250,624 | Frontier-scale open research, long-context evaluation, and high-capacity serving |
53
 
54
- Tokenizer files are distributed with each model repository. The compact K2-Horizon-0.9B model comes from the IFM Mobile line and uses a 64,256-token vocabulary. The 3.7B, 7B, 32B, 36B, and 375B models use a 250,624-token vocabulary.
55
 
56
- The native 0.9B `main`, 32B `mid_4`, and 36B `main`/`mid_4` revisions publish `K2HorizonForCausalLM`, `model_type: k2_horizon`, and matching `configuration_k2_horizon.py` and `modeling_k2_horizon.py` modules. The deployment preflights below validate that public contract before loading weights.
57
 
58
- ## This Repository
59
-
60
- | Field | Value |
61
- | --- | --- |
62
- | Model name | K2-Horizon-32B |
63
- | Repository | IFM/K2-Horizon-32B |
64
- | Model family | K2-Horizon |
65
- | Architecture class | K2HorizonForCausalLM |
66
- | Classification | Dense |
67
- | Stored parameters | 32.21B core; 34.78B including embeddings |
68
- | Active parameters per token | 32.21B core; 34.78B including embeddings |
69
- | Context length | 512K tokens |
70
- | Transformer layers | 64 |
71
- | Hidden size | 5,120 |
72
- | Dense / expert intermediate size | 26,624 / 0 |
73
- | Attention heads / KV heads | 64 / 8 |
74
- | Dense attention / sparse attention blocks | 64 / 0 |
75
- | Dense FFN / sparse FFN blocks | 64 / 0 |
76
- | MoE experts / active experts | 0 / 0 |
77
- | MoVA count / active values | 0 / 0 |
78
- | Vocab size | 250,624 |
79
- | Tokenizer metadata | Hugging Face tokenizer files; vocab size 250,624 |
80
- | Checkpoint storage dtype | float32 (FP32) |
81
- | Checkpoint config dtype metadata | float32 |
82
- | Evaluation and sample dtype | bfloat16 (BF16) |
83
- | Shard/index summary | 64 shards |
84
-
85
- ## Checkpoint Revisions
86
-
87
- Each repository uses branches for model stages and stable tags for specific checkpoints. Pin a revision tag when reproducibility matters; use `main` for the default model checkpoint.
88
-
89
- | Stage | Branch | Revision tag | Max context | Description |
90
- | --- | --- | --- | --- | --- |
91
- | Base / pretraining | `pretrain` | `base_final` | 8K tokens | Base model checkpoint |
92
- | Midtraining 1 | `mid_1` | `mid_1_final` | 32K tokens | Midtraining 1 checkpoint |
93
- | Midtraining 2 | `mid_2` | `mid_2_final` | 131,072 tokens | Midtraining 2 checkpoint |
94
- | Midtraining 3 | `mid_3` | `mid_3_final` | 512K tokens | Midtraining 3 checkpoint |
95
- | Midtraining 4 | `mid_4` | `mid_4_final` | 512K tokens | Midtraining 4 checkpoint |
96
- | Default | `main` | `main` | 512K tokens | Default release view of `mid_4_final` |
97
-
98
- Example revision pinning:
99
-
100
- <details>
101
- <summary>Show revision pinning example</summary>
102
-
103
- ```python
104
- # Validated with Python 3.14,
105
- # Transformers 5.15.0, and
106
- # Safetensors 0.8.0.
107
- from transformers import AutoModelForCausalLM, AutoTokenizer
108
-
109
- model_id = "IFM/K2-Horizon-32B"
110
- revision = "mid_4_final"
111
-
112
- tokenizer = AutoTokenizer.from_pretrained(model_id, revision=revision, trust_remote_code=True)
113
- model = AutoModelForCausalLM.from_pretrained(
114
- model_id,
115
- revision=revision,
116
- device_map="auto",
117
- dtype="bfloat16",
118
- low_cpu_mem_usage=True,
119
- trust_remote_code=True,
120
- )
121
- ```
122
-
123
- </details>
124
-
125
-
126
-
127
- ## Training Provenance
128
-
129
- Each row identifies the last W&B process record associated with the selected checkpoint, together with its Slurm job and exact xLLM source revision. A training stage can span multiple W&B process records when checkpointed training is resumed.
130
-
131
- | Stage | Revision | Checkpoint context | xLLM source | W&B last run | Slurm job | Training nodes |
132
- | --- | --- | --- | --- | --- | --- | --- |
133
- | Base / pretraining | `base_final` | 8K tokens | [`ad6dac16`](https://github.com/LLM360/xllm/commit/ad6dac16031ae74d3e02c10bf3f37384815503a2) | [`k2v3-32b/5ptu101d`](https://wandb.ai/mbzuai-llm/k2v3-32b/runs/5ptu101d) | `443488` | 152 |
134
- | Midtraining 1 | `mid_1_final` | 32K tokens | [`b69a6b2e`](https://github.com/LLM360/xllm/commit/b69a6b2ed17091dfee8af68a570f78816f765bad) | [`k2v3_midtrain/6zcj31kj`](https://wandb.ai/mbzuai-llm/k2v3_midtrain/runs/6zcj31kj) | `498822` | 152 |
135
- | Midtraining 2 | `mid_2_final` | 131,072 tokens | [`9ab31836`](https://github.com/LLM360/xllm/commit/9ab3183694cbf36cf77a5c60761576435f550433) | [`k2v3_midtrain/4lo14vpx`](https://wandb.ai/mbzuai-llm/k2v3_midtrain/runs/4lo14vpx) | `968789` | 152 |
136
- | Midtraining 3 | `mid_3_final` | 512K tokens | [`9ab31836`](https://github.com/LLM360/xllm/commit/9ab3183694cbf36cf77a5c60761576435f550433) | [`k2v3_midtrain/4753a9v1`](https://wandb.ai/mbzuai-llm/k2v3_midtrain/runs/4753a9v1) | `974704` | 152 |
137
- | Midtraining 4 | `mid_4_final` | 512K tokens | [`5494c84c`](https://github.com/LLM360/xllm/commit/5494c84cc7a2a66e5d6439bf9aa07454097ce9be) | [`k2v3_midtrain/lagg1h7w`](https://wandb.ai/mbzuai-llm/k2v3_midtrain/runs/lagg1h7w) | `1189061` | 152 |
138
-
139
-
140
-
141
- ## Training Loss
142
-
143
- The chart reconstructs the release checkpoint lineage for Pretraining, Midtraining 1, Midtraining 2, Midtraining 3, Midtraining 4. Training loss is the next-token prediction objective; lower values indicate that the model assigns greater probability to the observed next token. Training loss is not a validation score and should not be compared across different tokenizers or data mixtures without additional controls.
144
-
145
- ![K2-Horizon-32B training loss versus cumulative training tokens](./assets/k2-horizon-32b-training-loss-vs-tokens.png)
146
-
147
- Each panel combines the W&B process records that contribute to the release checkpoint. Faint lines show sampled raw training loss; bold lines show an unpadded centered moving average with a raised-cosine taper over the final half-window. The final sample retains 20% of its base weight, limiting right-edge bias without hiding the raw tail. Checkpoint restart overlap is replaced by the later process record.
148
-
149
-
150
- ## Deployment Guide
151
-
152
- These are base checkpoints. They can be served for research, evaluation, retrieval-augmented generation experiments, tool-use research, and downstream adaptation. They are not instruction-aligned assistants by default, so production deployments should add application-level safety filters, system prompting, and task-specific validation.
153
-
154
- All reported K2-Horizon evaluation used BF16. Some released checkpoint shards and configuration metadata remain FP32; the examples below explicitly load or serve them as BF16 to match the evaluation runtime and reduce accelerator memory usage.
155
-
156
- ### Validated Independent Dependency Versions
157
-
158
- Use a separate Python environment for every workflow. Direct Transformers loading, xLLM conversion, vLLM, and the pinned SGLang production revision have independent dependency contracts. Do not install one row into another row's environment. Versions intentionally differ when a framework or remote-model contract requires them.
159
-
160
- | Workflow | Python | PyTorch | CUDA | Transformers | Safetensors | Attention kernel | Framework revision |
161
- | --- | --- | --- | --- | --- | --- | --- | --- |
162
- | Direct Transformers loading | 3.14 | 2.13.0 | 12.6 | 5.15.0 | 0.8.0 | PyTorch SDPA 2.13.0 | Transformers 5.15.0 remote code |
163
- | xLLM checkpoint conversion | 3.12 | 2.8.0 | 12.8 | 4.57.3 | 0.8.0 | FlashAttention 2.8.3.post1 | xLLM `7cfa3b2c` |
164
- | vLLM serving | 3.12 | 2.13.0 | 12.9 | 5.16.1 | 0.8.0 | FlashInfer 0.6.17 | vLLM 0.26.1rc1.dev1212 source `d9fd5f11` (PR #53806) |
165
- | SGLang serving | 3.12 | 2.9.1 | 12.8 (CUDA Python 12.9) | 5.3.0 | 0.8.0 | FA3 (sglang-kernel 0.4.1); FlashInfer 0.6.7.post2 available | SGLang `ffb41d77` (stable-only metadata) |
166
-
167
- ### Inference Parameter Starting Points
168
-
169
- K2-Horizon does not use one family-wide sampling preset. The values below are operational baselines: temperature 0 is the validated deterministic serving comparison, while sampled settings are conservative starting points that should be tuned on application prompts. Benchmark reproduction must use the decoding configuration attached to that benchmark.
170
-
171
- | Use case | Temperature | Top-p | Max new tokens | Notes |
172
- | --- | --- | --- | --- | --- |
173
- | Checkpoint smoke test and deterministic evaluation | 0.0 | 1.0 | 512-4,096 | Greedy decoding is the reproducible baseline used by serving comparisons. |
174
- | Base-model continuation | 0.6 | 0.95 | 1,024-4,096 | A conservative sampling baseline; tune on a held-out prompt set. |
175
- | Long-context continuation or summarization | 0.2 | 0.9 | 2,048-8,192 | Start with one request and increase context only after KV-cache capacity checks. |
176
- | Reasoning and tool-call validation | 0.0 | 1.0 | 4,096 | Use a compatible chat template and the K2Horizon parser stack below. |
177
-
178
- ### Transformers
179
-
180
- The selected default checkpoint is loaded with Transformers 5.15.0 and PyTorch 2.13.0 on CUDA 12.6. Select the corresponding PyTorch build for your accelerator when that CUDA runtime is unavailable. The selected default checkpoint uses the self-contained native K2 remote implementation. Its saved Transformers 5.13.0 contract is source-compatible with the newer stable 5.15.0 release, and it does not import xLLM native extensions.
181
-
182
- <details>
183
- <summary>Show Transformers installation commands</summary>
184
-
185
- ```bash
186
- python -m pip install "torch==2.13.0" \
187
- --index-url "https://download.pytorch.org/whl/cu126"
188
- python -m pip install \
189
- "transformers==5.15.0" \
190
- "safetensors==0.8.0" \
191
- "accelerate==1.14.0"
192
- ```
193
-
194
- </details>
195
-
196
- <details>
197
- <summary>Show Transformers inference example</summary>
198
-
199
- ```python
200
- from transformers import AutoModelForCausalLM, AutoTokenizer
201
-
202
- MODEL_ID = "IFM/K2-Horizon-32B"
203
- REVISION = "mid_4"
204
-
205
- tokenizer = AutoTokenizer.from_pretrained(
206
- MODEL_ID,
207
- revision=REVISION,
208
- trust_remote_code=True,
209
- )
210
- model = AutoModelForCausalLM.from_pretrained(
211
- MODEL_ID,
212
- revision=REVISION,
213
- device_map="auto",
214
- dtype="bfloat16",
215
- low_cpu_mem_usage=True,
216
- trust_remote_code=True,
217
- )
218
-
219
- prompt = "Explain why long-context evaluation is difficult."
220
- inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
221
- inputs.pop("token_type_ids", None)
222
- outputs = model.generate(
223
- **inputs,
224
- max_new_tokens=512,
225
- temperature=0.7,
226
- top_p=0.9,
227
- do_sample=True,
228
- )
229
- print(tokenizer.decode(outputs[0], skip_special_tokens=True))
230
- ```
231
-
232
- </details>
233
-
234
- ### xLLM
235
-
236
- xLLM provides the training framework, checkpoint converter, serving bridges, and runtime kernels used by the XLLM-based K2-Horizon repositories. For checkpoint conversion, parity work, or loading a legacy 36B `XllmForCausalLM` stage that imports xLLM MoVA kernels, use the isolated legacy contract and reviewed bridge revision [`7cfa3b2c`](https://github.com/LLM360/xllm/commit/7cfa3b2c61fad3b2ef48f7f7b497ef360087c72f):
237
-
238
- <details>
239
- <summary>Show xLLM setup and conversion commands</summary>
240
-
241
- ```bash
242
- git clone https://github.com/LLM360/xllm.git
243
- cd xllm
244
- git checkout 7cfa3b2c61fad3b2ef48f7f7b497ef360087c72f
245
-
246
- python -m pip install "torch==2.8.0" \
247
- --index-url "https://download.pytorch.org/whl/cu128"
248
- export XLLM_BUILD_ROOT="${XLLM_BUILD_ROOT:-$PWD/.xllm-build}"
249
- export TMPDIR="$XLLM_BUILD_ROOT/tmp"
250
- export PIP_CACHE_DIR="$XLLM_BUILD_ROOT/pip-cache"
251
- mkdir -p "$TMPDIR" "$PIP_CACHE_DIR"
252
- python -m pip install packaging ninja
253
- python -m pip install "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.3.post1/flash_attn-2.8.3.post1+cu12torch2.8cxx11abiTRUE-cp312-cp312-linux_x86_64.whl"
254
- python -m pip install -r requirements.txt
255
- python -m pip install -e . --no-build-isolation \
256
- --config-settings editable_mode=compat
257
- python -m pip install \
258
- "transformers==4.57.3" \
259
- "safetensors==0.8.0" \
260
- accelerate fire
261
-
262
- python xllm_bridges/huggingface/xllm_to_hf_main.py --help
263
- ```
264
-
265
- </details>
266
-
267
- The converter requires an original xLLM distributed checkpoint and tokenizer. Released Hugging Face revisions are already converted. On K2-Horizon-32B, `mid_4` uses the native Transformers 5 contract above; `pretrain` through `mid_3` use the pure Transformers 4.57.6/SDPA direct contract, without xLLM. The xLLM row remains the converter contract. Pinning a revision does not make runtime contracts interchangeable.
268
-
269
- ### vLLM
270
-
271
- The example pins [vLLM PR #53806](https://github.com/vllm-project/vllm/pull/53806) at [`d9fd5f11`](https://github.com/vllm-project/vllm/commit/d9fd5f11423a1a5628fe29e7296ceb9de91aac3c). That source registers `K2HorizonForCausalLM` plus the built-in `k2_horizon` reasoning and tool parsers. The command uses vLLM's `vllm` model implementation for this checkpoint contract.
272
-
273
- <details>
274
- <summary>Show vLLM setup and serving commands</summary>
275
-
276
- ```bash
277
- git clone --filter=blob:none --no-checkout \
278
- https://github.com/vllm-project/vllm.git
279
- cd vllm
280
- git fetch origin \
281
- pull/53806/head:refs/remotes/origin/pr-53806
282
- git checkout --detach d9fd5f11423a1a5628fe29e7296ceb9de91aac3c
283
- test "$(git rev-parse HEAD)" = "d9fd5f11423a1a5628fe29e7296ceb9de91aac3c"
284
-
285
- python3.12 -m venv .venv
286
- source .venv/bin/activate
287
- python -m pip install --upgrade pip uv
288
- export UV_LINK_MODE=copy
289
- VLLM_USE_PRECOMPILED=1 uv pip install --upgrade --editable . \
290
- --torch-backend=auto
291
- uv pip install \
292
- "transformers==5.16.1" \
293
- "safetensors==0.8.0"
294
- python -m pip check
295
-
296
- python - <<'PY'
297
- from vllm import ModelRegistry
298
- from vllm.reasoning import ReasoningParserManager
299
- from vllm.tool_parsers import ToolParserManager
300
-
301
- assert "K2HorizonForCausalLM" in ModelRegistry.get_supported_archs()
302
- assert ReasoningParserManager.get_reasoning_parser("k2_horizon") is not None
303
- assert ToolParserManager.get_tool_parser("k2_horizon") is not None
304
- PY
305
-
306
- export MODEL_ID="IFM/K2-Horizon-32B"
307
- export MODEL_REVISION="mid_4"
308
- export TENSOR_PARALLEL_SIZE=2
309
- export MAX_MODEL_LEN=131072
310
-
311
- vllm serve "$MODEL_ID" \
312
- --revision "$MODEL_REVISION" \
313
  --model-impl vllm \
314
- --tensor-parallel-size "$TENSOR_PARALLEL_SIZE" \
315
  --trust-remote-code \
316
  --dtype bfloat16 \
317
- --served-model-name "$MODEL_ID" \
318
- --max-model-len "$MAX_MODEL_LEN" \
319
  --reasoning-parser k2_horizon \
320
  --enable-auto-tool-choice \
321
  --tool-call-parser k2_horizon
322
  ```
323
 
324
- </details>
325
-
326
- The source checkout is immutable at `d9fd5f11423a1a5628fe29e7296ceb9de91aac3c`. The command serves `mid_4`; the repository's longest checkpoint configuration reports 524,288 tokens, while this example starts at 131,072 tokens. Increase context and concurrency only after measuring KV-cache capacity. The preflight fails immediately if the native architecture or either Horizon parser registration is absent.
327
-
328
- ### SGLang
329
 
330
- Native K2 Horizon support is provided by
331
- [sgl-project/sglang#37654](https://github.com/sgl-project/sglang/pull/37654).
332
- Use a `lmsysorg/sglang:dev` image built after that PR is merged. Once support is
333
- included in a tagged SGLang release, use the corresponding versioned image.
334
-
335
- <details>
336
- <summary>Show the SGLang serving command</summary>
337
-
338
- ```bash
339
- docker run --gpus all \
340
- --shm-size 32g \
341
- -p 30000:30000 \
342
- -v ~/.cache/huggingface:/root/.cache/huggingface \
343
- --ipc=host \
344
- lmsysorg/sglang:dev \
345
- python3 -m sglang.launch_server \
346
- --model-path "IFM/K2-Horizon-32B" \
347
- --revision main \
348
- --tp 2 \
349
- --dtype bfloat16 \
350
- --context-length 131072 \
351
- --attention-backend fa3 \
352
- --reasoning-parser k2_horizon \
353
- --tool-call-parser k2_horizon \
354
- --mem-fraction-static 0.85 \
355
- --host 0.0.0.0 \
356
- --port 30000
357
  ```
358
 
359
- </details>
360
 
361
- This uses SGLang's native `K2HorizonForCausalLM` implementation; no
362
- `--trust-remote-code`, source patch, or external parser plugin is required. The
363
- 131,072-token limit is a conservative starting point; increase it only after
364
- measuring KV-cache capacity.
365
-
366
- ### Reasoning and Tool Parsers
367
-
368
- Parser configuration is part of the deployment contract. The canonical K2-Horizon format uses `<ifm|...>` reasoning and tool tokens; test streaming and non-streaming responses before exposing an OpenAI-compatible endpoint.
369
-
370
- | Goal | vLLM guidance | SGLang guidance | When to use |
371
- | --- | --- | --- | --- |
372
- | Plain text generation | Do not enable a reasoning or tool parser. | Do not enable a reasoning or tool parser. | Base-model evaluation, completion-style generation, and checkpoint smoke tests. |
373
- | Canonical K2-Horizon reasoning | Use the built-in `--reasoning-parser k2_horizon`. | Use `--reasoning-parser k2_horizon`. | Checkpoints and chat templates using `<ifm|think>`, `<ifm|think_fast>`, or `<ifm|think_faster>`. |
374
- | Canonical K2-Horizon tool calls | Use `--enable-auto-tool-choice --tool-call-parser k2_horizon`. | Use `--tool-call-parser k2_horizon`. | XML, typed-XML, or JSON calls wrapped in `<ifm|tool_call>`. |
375
- | Legacy K2 reasoning traces | Use the legacy parser only in a compatibility image that registers it. | Use `--reasoning-parser k2_horizon_legacy`. | Older traces using `<think>`, `<think_fast>`, or `<think_faster>` instead of canonical IFM tokens. |
376
- | Mixed tool-call corpora | Use `multi_format` only in a build that includes and tests that parser. | Use `--tool-call-parser multi_format`. | Evaluation corpora that intentionally mix K2, GLM, or JSON tool-call dialects. |
377
-
378
- PR #53806 registers both Horizon parsers in-tree; no external parser plugin is required.
379
-
380
- For OpenAI-compatible requests, select reasoning effort through chat-template kwargs:
381
-
382
- <details>
383
- <summary>Show OpenAI-compatible request example</summary>
384
 
385
  ```python
386
  from openai import OpenAI
387
 
388
- client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
389
  response = client.chat.completions.create(
390
  model="IFM/K2-Horizon-32B",
391
  messages=[{"role": "user", "content": "Explain the result step by step."}],
392
- temperature=0,
393
- max_tokens=4096,
 
394
  extra_body={"chat_template_kwargs": {"reasoning_effort": "high"}},
395
  )
396
- print(response.choices[0].message)
 
 
397
  ```
398
 
399
- </details>
400
-
401
- ### Other Serving Frameworks
402
-
403
- Text Generation Inference, TensorRT-LLM, llama.cpp, Ollama, and LM Studio may require model-specific conversion, quantization, or custom architecture support. Validate tokenizer behavior, remote-code requirements, attention implementation, and long-context behavior before publishing service results.
404
-
405
-
406
- ## Hardware Planning
407
-
408
- The table below gives capacity-planning starting points for self-hosted inference. Weight memory uses the parameter count including input and output embeddings. On-disk FP32 shards and dtype conversion can require substantially more host RAM during loading than the final BF16 GPU footprint. KV-cache estimates use BF16 keys and values for one request at the smaller of the model maximum and 128K tokens; batching, allocator overhead, activations, CUDA graphs, and framework workspaces require additional memory.
409
-
410
- | Model | Stored params | Active params/token | BF16 weight memory | FP32 weight memory | BF16 KV cache / request | Starting TP | Recommended starting point |
411
- | --- | --- | --- | --- | --- | --- | --- | --- |
412
- | K2-Horizon-0.9B | 0.9B | 0.9B | 2.0 GiB | 4.0 GiB | 0.4 GiB @ 8K tokens | 1 | 1 GPU with 16 GiB is a practical starting point for the model's 8K context; allow additional memory for batching. |
413
- | K2-Horizon-3.7B | 5.06B | 5.06B | 9.4 GiB | 18.8 GiB | 18.0 GiB @ 128K tokens | 1 | 1 GPU with 24 GiB can serve short contexts in BF16; use 48-80 GiB for larger KV caches and 128K-class requests. |
414
- | K2-Horizon-7B | 9.00B | 9.00B | 16.8 GiB | 33.5 GiB | 18.0 GiB @ 128K tokens | 1 | 1 GPU with 40-48 GiB is suitable for short-context BF16 serving; 80 GiB is the practical starting point for 128K-class requests. |
415
- | K2-Horizon-32B | 34.78B | 34.78B | 64.8 GiB | 129.6 GiB | 32.0 GiB @ 128K tokens | 2-4 | 1x 80 GiB fits BF16 weights only at short context; start with 2x 80 GiB for 128K requests and 4x 80 GiB for larger context or throughput. |
416
- | K2-Horizon-36B | 37.44B | 5.95B | 69.7 GiB | 139.5 GiB | 24.0 GiB @ 128K tokens | 2-4 | Use at least 256 GiB of host RAM to load the current 139.5 GiB FP32 shard set. Start with 2x 80 GiB GPUs for BF16 weight sharding, or 1x H200 141 GiB for short single-request validation; use 4x 80 GiB for 128K-class requests. |
417
- | K2-Horizon-375B | 379.17B | 26.67B | 706.3 GiB | 1.38 TiB | 30.5 GiB @ 128K tokens | 8 H200 or 16x 80 GiB | The full 512K profile is validated on 8x H200 141 GiB GPUs at TP=8 and 0.92 memory utilization; use at least 16x 80 GiB GPUs on lower-memory hardware. |
418
-
419
- ## Evaluation
420
-
421
- Benchmark values are reported as percentages unless otherwise noted. A dash means the metric was not reported for that model or revision in the release evaluation records. The family tables below use each model's latest listed midtraining release checkpoint.
422
-
423
- ### Family Evaluation: Knowledge and Instruction Following
424
-
425
- | Model | Revision | IFEval | MMLU-Pro | MMLU | BBH |
426
- | --- | --- | --- | --- | --- | --- |
427
- | K2-Horizon-0.9B | `mid_2_final` | 43.1 | 40.3 | 44.4 | 43.8 |
428
- | K2-Horizon-3.7B | `mid_4_final` | 85.7 | 73.0 | 80.7 | 84.4 |
429
- | K2-Horizon-7B | `mid_4_final` | 89.9 | 75.7 | 83.3 | 87.8 |
430
- | K2-Horizon-32B | `mid_4_final` | 88.0 | 79.5 | 87.5 | 89.1 |
431
- | K2-Horizon-36B | `mid_4_final` | 89.6 | 79.2 | 87.3 | 88.6 |
432
- | K2-Horizon-375B | `mid_4_final` | 92.1 | 82.6 | 89.5 | 87.5 |
433
-
434
- ### Family Evaluation: STEM, Reasoning, and Coding
435
-
436
- | Model | Revision | AIME26 | GPQA-Diamond | GSM8K | MBPP |
437
- | --- | --- | --- | --- | --- | --- |
438
- | K2-Horizon-0.9B | `mid_2_final` | 12.3 | 27.8 | 53.1 | 53.2 |
439
- | K2-Horizon-3.7B | `mid_4_final` | 88.2 | 73.3 | 92.3 | 86.4 |
440
- | K2-Horizon-7B | `mid_4_final` | 87.6 | 73.9 | 95.1 | 91.0 |
441
- | K2-Horizon-32B | `mid_4_final` | 91.7 | 75.4 | 95.5 | 87.8 |
442
- | K2-Horizon-36B | `mid_4_final` | 92.7 | 75.4 | 95.2 | 87.2 |
443
- | K2-Horizon-375B | `mid_4_final` | 92.5 | 84.9 | 95.1 | 93.8 |
444
-
445
- ### Family Evaluation: Agentic and Long-Context Tasks
446
-
447
- | Model | Revision | SWE-Bench Verified (avg@3) | Terminal-Bench 2.1 (avg@3) | GDPval (acc@75%) | Tau3 Banking | BrowseComp+ (acc@1) | Apex Agents (pass@1) | WildClawBench | LCR |
448
- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
449
- | K2-Horizon-7B | `mid_4_final` | 57.5 | 28.1 | 34.5 | 10.4 | 52.2 | 6.0 | 23.9 | - |
450
- | K2-Horizon-375B | `mid_4_final` | 67.8 | 38.2 | 39.6 | 11.4 | 58.1 | 13.8 | 34.8 | 56.0 |
451
-
452
- Agentic results are joined only to exact selected release checkpoints. SWE-Bench Verified and Terminal-Bench 2.1 report three-run averages where available; GDPval reports accuracy at 75 percent; BrowseComp+ reports accuracy at one; Apex Agents reports pass at one. Fractional workbook values are converted to percentages. When a source cell contains multiple reported aggregates, the first aggregate is shown unless the cell contains only three slash-separated trial values, in which case their mean is reported. A dash means the exact release checkpoint has no reported result for that benchmark.
453
-
454
- ### Family Evaluation: Advanced Research Benchmarks
455
-
456
- | Model | Revision | MCP Atlas | HLE | SciCode | OmniScience Accuracy | OmniScience Non-Hallucination | CritPT |
457
- | --- | --- | --- | --- | --- | --- | --- | --- |
458
- | K2-Horizon-7B | `mid_4_final` | 53.6 | 13.8 | - | - | 43.7 | - |
459
- | K2-Horizon-375B | `mid_4_final` | 59.0 | 21.2 | 40.3 | 20.6 | 49.2 | 5.1 |
460
-
461
- These results use exact final-stage checkpoint records. Benchmark availability differs by model, so unreported cells remain `-`.
462
-
463
- ### This Repository: Base / Pretraining Snapshot
464
-
465
- | Revision | ArabicMMLU | ARC-C | BBH | GPQA-Diamond-CoT | GSM8K | MMLU | MMLU-Pro | HumanEval | MBPP | IFEval |
466
- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
467
- | `base_final` | 73.5 | 69.8 | 84.7 | 18.2 | 86.0 | 78.9 | 57.9 | 64.0 | 68.2 | 27.5 |
468
-
469
- ### This Repository: Midtraining Snapshot
470
-
471
- | Revision | IFEval avg | MMLU-Pro | GSM8K | MMLU | BBH | MBPP | AIME26 | GPQA-Diamond | LongBench-v2 | LCR | SWE-Bench Verified | Terminal Bench 2 |
472
- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
473
- | `mid_1_final` | 89.6 | 79.7 | 95.8 | 87.9 | 89.2 | 88.2 | 79.8 | 69.9 | - | - | - | - |
474
- | `mid_2_final` | 88.5 | 79.7 | 95.6 | 87.8 | 88.9 | 89.6 | 92.3 | 77.3 | - | - | - | - |
475
- | `mid_3_final` | 88.0 | 79.5 | 95.5 | 87.5 | 89.1 | 87.8 | 91.7 | 75.4 | - | - | - | - |
476
-
477
- ### This Repository: Agentic and Long-Context Results by Stage
478
-
479
- | Revision | SWE-Bench Verified | Terminal-Bench | GDPval (acc@75%) | Tau3 Banking | BrowseComp+ | Toolathlon | Apex Agents | LongBench-v2 | LCR |
480
- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
481
- | `mid_3_final` | - | - | 26.4 | - | 56.0 | - | 5.7 | - | - |
482
-
483
-
484
-
485
-
486
- ## Datasets and Mixtures
487
-
488
- K2-Horizon training is organized around transparent data releases:
489
-
490
- - Pretraining data: [IFM/K2-Horizon-Pretrain-Data](https://huggingface.co/datasets/IFM/K2-Horizon-Pretrain-Data), derived from the TxT360 lineage.
491
- - Midtraining data: [IFM/K2-Horizon-Midtrain-Data](https://huggingface.co/datasets/IFM/K2-Horizon-Midtrain-Data), following the TxT360-Midas reasoning and long-context release pattern.
492
- - The public data-preparation lineage is maintained in [LLM360/TxT360](https://github.com/LLM360/TxT360).
493
 
494
- ## Source Code
495
 
496
- - [LLM360/xllm](https://github.com/LLM360/xllm) - Training framework and Hugging Face/vLLM bridges. Per-stage training commits are linked in the provenance table.
497
- - [LLM360/Eval360-V2 at `f5081bfb`](https://github.com/LLM360/Eval360-V2/commit/f5081bfba9ef8e7b380f5bd85ed1cf61a91f90b0) - Evaluation harness revision reviewed against the release benchmark workbooks.
498
- - [LLM360/agent-evals at `f3788399`](https://github.com/LLM360/agent-evals/commit/f3788399a974e49121984f11556138f23f907cee) - Agentic benchmark harness, Slurm launchers, and serving profiles used to reconcile the agentic release results.
499
- - [vLLM PR #53806 at `d9fd5f11`](https://github.com/vllm-project/vllm/commit/d9fd5f11423a1a5628fe29e7296ceb9de91aac3c) - Native K2-Horizon architecture plus built-in reasoning and tool parsers used by the serving example.
500
- - [LLM360/Inference360 at `a7ee94f9`](https://github.com/LLM360/Inference360/commit/a7ee94f9ad9a561e65a67ede22d4e7a2c8428b57) - Serving-image and deployment-validation references.
501
- - [LLM360/sglang at `ffb41d77`](https://github.com/LLM360/sglang/commit/ffb41d77d8b96b8a953351cb4d66d6bf4ed061a7) - SGLang serving base used by the hash-guarded K2-Horizon namespace migration.
502
- - [LLM360/TxT360](https://github.com/LLM360/TxT360) - Public data-preparation and source-data lineage reference.
503
 
504
- ## Intended Use
 
 
 
 
505
 
506
- K2-Horizon-32B is intended for open research on foundation models, transparent evaluation, staged checkpoint analysis, downstream fine-tuning, and reproducibility studies. It is suitable for researchers and developers who want to inspect training-stage behavior, serve a model privately, compare model sizes, or build downstream adaptations.
 
 
 
 
507
 
508
- ## Limitations and Safety
509
 
510
- - This is a base checkpoint family and may produce inaccurate, biased, unsafe, or hallucinated content.
511
- - The model is not optimized for moderation, refusal behavior, or production assistant safety.
512
- - Long-context behavior depends on serving stack, prompt construction, retrieval quality, and KV-cache configuration.
513
- - Tool-use and agentic results depend heavily on parser format, prompt template, tool schema, and execution harness.
514
- - The model family is documented primarily with English-language evaluations; additional multilingual evaluation should be performed before broad multilingual claims.
515
- - Users are responsible for complying with the model license and for validating deployments in their own risk domain.
516
 
517
  ## Citation
518
 
519
- Please cite this model as:
520
-
521
  ```bibtex
522
- @misc{ifmk2horizon32b2026,
523
- title = {K2-Horizon Model Family: K2-Horizon-32B},
524
- author = {IFM Team},
525
- year = {2026},
526
- url = {https://huggingface.co/IFM/K2-Horizon-32B},
527
  }
528
  ```
529
-
530
- ## Contact
531
-
532
- For questions about this release, contact the IFM / K2-Horizon team through the model repository or the IFM organization.
 
16
  - ifm
17
  ---
18
 
19
+ # K2-Horizon-32B-Stage1-FP8
20
 
21
  [Training Code](https://github.com/LLM360/xllm) - [Evaluation Code](https://github.com/LLM360/Eval360-V2) - [Pretraining Data](https://huggingface.co/datasets/IFM/K2-Horizon-Pretrain-Data) - [Midtraining Data](https://huggingface.co/datasets/IFM/K2-Horizon-Midtrain-Data)
22
 
 
29
  >
30
  > The FP8 model performs closely in line with the original BF16 model on our evaluations, while reducing memory footprint and enabling faster inference on FP8-capable hardware.
31
 
32
+ K2-Horizon-32B-Stage1 is the large dense member of the K2-Horizon family: a 32B decoder-only model with a 512K context window. <span style="color:#d92d20;font-weight:600">Note: final checkpoint to be released.</span>
33
 
34
+ <p align="center">
35
+ <img src="assets/k2-horizon-32b-benchmarks.png" alt="K2-Horizon-32B-Stage1 benchmark results against open MoE, dense, and closed models" width="100%">
36
+ </p>
37
 
38
+ ## K2-Horizon-32B-Stage1 Highlights
39
 
40
+ - **Strong dense baseline.** A 32B dense model evaluated on the same agentic, coding, and reasoning benchmarks as the rest of the family (see [Benchmark Results](#benchmark-results)). Results are for stage 1 of the final model training; results for stage 2 will be out soon.
41
+ - **512K context.** Native 524,288-token context from the midtraining stages onward.
42
+ - **Intermediate checkpoints.** Intermediate checkpoints will be released so capability changes can be studied across training rather than at a single checkpoint.
43
+ - **Fully open.** Training data/recipe and the training code will be made public.
44
 
45
+ ## Benchmark Results
46
 
47
+ <!-- TABLE:START -->
48
+ <div style="font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif;margin:0 auto;padding:8px 0 16px;overflow-x:auto"><table style="display:table;width:100%;table-layout:fixed;border-collapse:collapse;font-size:12px;margin:0"><thead><tr><th style="width:30%;border-bottom:none"></th><th colspan="1" style="padding:6px 4px 2px;text-align:center;font-size:11px;font-weight:600;letter-spacing:0.04em;text-transform:uppercase;opacity:0.65;border-bottom:1px solid rgba(128,128,128,0.25)"></th><th colspan="3" style="padding:6px 4px 2px;text-align:center;font-size:11px;font-weight:600;letter-spacing:0.04em;text-transform:uppercase;opacity:0.65;border-bottom:1px solid rgba(128,128,128,0.25)">Open-weight dense models</th></tr><tr><th style="padding:10px 6px;text-align:left;border-bottom:2px solid #2450D6"></th><th style="padding:10px 3px;text-align:center;font-weight:600;border-bottom:2px solid #2450D6;color:#2450D6;font-size:12.5px;line-height:1.2;width:17.50%;overflow-wrap:anywhere;background:rgba(36,80,214,0.08);">K2-Horizon-32B-Stage1</th><th style="padding:10px 3px;text-align:center;font-weight:600;border-bottom:2px solid #2450D6;color:#2450D6;font-size:12.5px;line-height:1.2;width:17.50%;overflow-wrap:anywhere;">Qwen3.8-27B</th><th style="padding:10px 3px;text-align:center;font-weight:600;border-bottom:2px solid #2450D6;color:#2450D6;font-size:12.5px;line-height:1.2;width:17.50%;overflow-wrap:anywhere;">Muse Glimmer-30B</th><th style="padding:10px 3px;text-align:center;font-weight:600;border-bottom:2px solid #2450D6;color:#2450D6;font-size:12.5px;line-height:1.2;width:17.50%;overflow-wrap:anywhere;">IBM Granite 4.2 30B</th></tr></thead><tbody><tr><td style="padding:7px 4px 7px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;font-weight:600"># Params</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">32B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">27B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">30B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">30B</td></tr><tr><td style="padding:7px 4px 7px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;font-weight:600"># Activated params</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">32B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">27B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">30B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">30B</td></tr><tr><td style="padding:7px 4px 7px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;font-weight:600">Architecture</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">Dense</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">Dense</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">Dense</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">Dense</td></tr><tr><td colspan="5" style="padding:6px 10px;font-weight:600;font-size:12.5px;color:#2450D6;border-bottom:1px solid rgba(36,80,214,0.25);background:rgba(36,80,214,0.12)">Agents</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">tau3-Banking</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Agentic tool use</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">22.5</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>48.0</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">23.5</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">14.4</td></tr><tr><td colspan="5" style="padding:6px 10px;font-weight:600;font-size:12.5px;color:#2450D6;border-bottom:1px solid rgba(36,80,214,0.25);background:rgba(36,80,214,0.12)">Coding</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">Terminal-Bench 2.1</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Agentic terminal use</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">36.6</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>79.8</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">51.7</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">26.6</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">SciCode</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Scientific coding</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">30.2</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>44.7</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">43.6</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">36.6</td></tr><tr><td colspan="5" style="padding:6px 10px;font-weight:600;font-size:12.5px;color:#2450D6;border-bottom:1px solid rgba(36,80,214,0.25);background:rgba(36,80,214,0.12)">Scientific Reasoning</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">Humanity's Last Exam (without tools)</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Expert-level reasoning</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">22.8</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>33.9</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">22.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">11.2</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">GPQA Diamond</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Graduate-level science QA</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">82.3</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>90.5</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">83.5</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">64.4</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">CritPt</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Frontier physics reasoning</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">1.4</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>5.4</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">2.6</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">0.3</td></tr><tr><td colspan="5" style="padding:6px 10px;font-weight:600;font-size:12.5px;color:#2450D6;border-bottom:1px solid rgba(36,80,214,0.25);background:rgba(36,80,214,0.12)">General</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">AA-LCR</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Long-context reasoning</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">65.3</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">77.3</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>80.0</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">46.7</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">AA-Omniscience Accuracy</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Factual accuracy</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">16.8</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">15.6</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>27.0</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">10.1</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2;color:inherit">AA-Omniscience Non-Hallucination</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Non-hallucination rate</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">58.3</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">69.7</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">18.1</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;"><strong>74.4</strong></td></tr></tbody></table></div>
49
+ <!-- TABLE:END -->
50
 
51
+ <p style="font-size:1.25em;line-height:1.6">Scores in %. Bold marks the best score in each row. Sections follow the <a href="https://artificialanalysis.ai/methodology/intelligence-benchmarking">Artificial Analysis Intelligence Index</a> categories. Baseline scores are from Artificial Analysis; Muse Glimmer-30B at high reasoning effort, other open models in their reasoning mode.</p>
52
 
53
+ ## Quickstart
 
 
 
 
 
 
 
54
 
55
+ ### Serving
56
 
57
+ vLLM, recipe at [recipes.vllm.ai/IFM](https://recipes.vllm.ai/IFM):
58
 
59
+ ```shell
60
+ vllm serve IFM/K2-Horizon-32B \
61
+ --revision main \
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
62
  --model-impl vllm \
63
+ --tensor-parallel-size 2 \
64
  --trust-remote-code \
65
  --dtype bfloat16 \
66
+ --max-model-len 131072 \
 
67
  --reasoning-parser k2_horizon \
68
  --enable-auto-tool-choice \
69
  --tool-call-parser k2_horizon
70
  ```
71
 
72
+ SGLang recipe validated on 2× H200 in the [SGLang K2 Horizon cookbook](https://docs.sglang.io/cookbook/autoregressive/IFM/K2-Horizon):
 
 
 
 
73
 
74
+ ```shell
75
+ python3 -m sglang.launch_server \
76
+ --model-path IFM/K2-Horizon-32B \
77
+ --revision main \
78
+ --tp 2 \
79
+ --dtype bfloat16 \
80
+ --attention-backend fa3 \
81
+ --reasoning-parser k2_horizon \
82
+ --tool-call-parser k2_horizon \
83
+ --host 0.0.0.0 --port 30000
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
84
  ```
85
 
86
+ ### API Usage
87
 
88
+ > [!Tip]
89
+ > Recommended settings: `reasoning_effort="high"`, `temperature=1.0`, `top_p=0.95`.
90
+ > Reasoning depth is selected per request through `chat_template_kwargs`. Thinking is returned in `reasoning_content` and the answer in `content`.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
91
 
92
  ```python
93
  from openai import OpenAI
94
 
95
+ client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
96
  response = client.chat.completions.create(
97
  model="IFM/K2-Horizon-32B",
98
  messages=[{"role": "user", "content": "Explain the result step by step."}],
99
+ temperature=1.0,
100
+ top_p=0.95,
101
+ max_tokens=32768,
102
  extra_body={"chat_template_kwargs": {"reasoning_effort": "high"}},
103
  )
104
+ message = response.choices[0].message
105
+ print("Reasoning:", getattr(message, "reasoning_content", None))
106
+ print("Answer:", message.content)
107
  ```
108
 
109
+ ### Transformers
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
110
 
111
+ Validated with Transformers 5.15.0, PyTorch 2.13.0, Safetensors 0.8.0.
112
 
113
+ ```python
114
+ from transformers import AutoModelForCausalLM, AutoTokenizer
 
 
 
 
 
115
 
116
+ model_id = "IFM/K2-Horizon-32B"
117
+ tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
118
+ model = AutoModelForCausalLM.from_pretrained(
119
+ model_id, device_map="auto", dtype="bfloat16", low_cpu_mem_usage=True, trust_remote_code=True
120
+ )
121
 
122
+ inputs = tokenizer("Explain why long-context evaluation is difficult.", return_tensors="pt").to(model.device)
123
+ inputs.pop("token_type_ids", None)
124
+ outputs = model.generate(**inputs, max_new_tokens=32768, temperature=1.0, top_p=0.95, do_sample=True)
125
+ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
126
+ ```
127
 
128
+ ## Best Practices
129
 
130
+ 1. **Reasoning effort: always `high`.** All reported results use high reasoning effort. Pass `{"chat_template_kwargs": {"reasoning_effort": "high"}}` on every request.
131
+ 2. **Sampling parameters.** `temperature=1.0`, `top_p=0.95`.
132
+ 3. **Serving.** Use the validated SGLang recipe above: BF16, TP=2, FlashAttention-3. Full recipes for every K2-Horizon size, with measured H200 latency and throughput, are in the [SGLang cookbook](https://docs.sglang.io/cookbook/autoregressive/IFM/K2-Horizon) and the [vLLM recipe](https://recipes.vllm.ai/IFM).
133
+ 4. **Parsers.** Enable the `k2_horizon` reasoning parser for chat, and add the `k2_horizon` tool-call parser for agent use. Leave both off for plain completion-style generation.
 
 
134
 
135
  ## Citation
136
 
 
 
137
  ```bibtex
138
+ @misc{k2horizon2026,
139
+ title = {Introducing K2 Horizon: Frontier Performance, Radically Open},
140
+ author = {{IFM Team}},
141
+ year = {2026},
142
+ url = {https://ifm.ai/blog/k2/},
143
  }
144
  ```