AweAI-Team/CalibForge
Updated • 20.3k • 4
Following ricdomolm's feedback and published exclusion list, this model was retrained with the 405 flagged tasks excluded from its training data. The updated CalibForge dataset contains 5,026 tasks.
📄 Paper · 💻 Repository · 🤗 Dataset
CalibForge-35B-A3B-v2 is a terminal-agent model fine-tuned from Qwen3.5-35B-A3B using multi-turn SFT on CalibForge solver trajectories.
| Model | Terminal-Bench 2.1 Acc. (%, mean ± SEM) ↑ |
|---|---|
| CalibForge-35B-A3B-v2 | 46.44 ± 1.63 |
Results are reported as mean ± SEM over three runs. Evaluation uses CalibForge-Eval, with a one-hour per-task timeout and 16 CPUs / 48 GiB RAM per sandbox.
Use a recent SGLang release with Qwen3.5 MoE support:
python -m sglang.launch_server \
--model-path AweAI-Team/CalibForge-35B-A3B-v2 \
--tp 8 \
--dp 1 \
--trust-remote-code \
--reasoning-parser qwen3 \
--tool-call-parser qwen3_coder \
--mem-fraction-static 0.85 \
--port 40003
The model is released under the Apache License 2.0.
@misc{meng2026calibforge,
title = {CalibForge: Adversarial Solver Calibration for Scaling Learnable Terminal Tasks},
author = {Fanzhe Meng and Guoxin Chen and Jiale Zhao and Shuang Sun and Yuzhi Lin and Wayne Xin Zhao and Ruihua Song and Ji-Rong Wen and Kai Jia},
year = {2026},
eprint = {2608.06352},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2608.06352}
}