Document v0.0.1 LoRA training recipe and results

Records completed seq2048 LoRA SFT data recipe, training parameters, held-in eval results, and release caveats. Excludes the in-progress QLoRA run.

README.md

@@ -5,100 +5,124 @@ datasets:
 - benchflow/general-agent-qwen35-9b-azure-gpt54mini-sft
 tags:
 - lora
-- qlora
+- peft
 - sft
 - general-agent
-- env-0
 - qwen
 ---
 
-# Qwen3.5-9B General-Agent SFT Adapter
+# Qwen3.5-9B General-Agent SFT LoRA Adapter
 
-This repository publishes the current PEFT adapter for the Prime `general-agent` SFT reproduction. It does not include the frozen base weights. Load the adapter on top of the full, non-prequantized `Qwen/Qwen3.5-9B` checkpoint.
+`v0.0.1` is the completed LoRA SFT release for the Prime `general-agent` reproduction using the full, non-prequantized `Qwen/Qwen3.5-9B` base checkpoint. It does **not** include the base weights; load this adapter on top of `Qwen/Qwen3.5-9B`.
 
-## Current Published Adapter
+This release intentionally excludes the next QLoRA run that is currently in progress. That run will be documented and tagged separately after its training and eval finish.
+
+## Release Summary
 
 | Field | Value |
 | --- | --- |
+| Release tag | `v0.0.1` |
+| Adapter repo | `benchflow/benchflow-qwen35-9b` |
 | Base checkpoint | `Qwen/Qwen3.5-9B` |
-| Base checkpoint form | Full, non-quantized source checkpoint; frozen during adapter training |
-| Adapter type | LoRA |
-| Adapter rank / alpha / dropout | `r=16`, `alpha=32`, `dropout=0.0` |
-| Target modules | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` |
-| Trainable params | about `29.1M` |
-| Source run id | `general-agent-qwen35-9b-sft-seq2048-fresh-20260624T131847Z` |
+| Base checkpoint form | Full, non-quantized source checkpoint; frozen during LoRA SFT |
+| Adapter type | LoRA / PEFT |
+| Source completed run | `general-agent-qwen35-9b-sft-seq2048-fresh-20260624T131847Z` |
 | W&B project | `general-agent-qwen35-9b-sft-seq2048-fresh-20260624T131847Z` |
-| Raw artifacts | `benchflow/env0-experiment-trajectories/experiments/general-agent/general-agent-qwen35-9b-sft-seq2048-fresh-20260624T131847Z` |
+| HF training artifacts | `benchflow/env0-experiment-trajectories/experiments/general-agent/general-agent-qwen35-9b-sft-seq2048-fresh-20260624T131847Z` |
+| Published at | `2026-06-24 22:27:07 UTC` |
+
+## Research Reproduction Scope
+
+The goal of this adapter is to reproduce the SFT-stage lift from Prime Intellect's `general-agent` work as closely as possible while using a smaller student model that can train on one H100. The stack keeps the Prime-style task and verifier path:
 
-## Completed SFT Training Recipe
+- Source tasks: open-source `PrimeIntellect-ai/research-environments/environments/general_agent` task corpus.
+- Teacher trace generation: `general-agent-solver-rlm` + Azure GPT-5.4-mini through native Verifiers / `vf-eval --save-results` artifacts.
+- SFT trainer: Prime-RL SFT.
+- Student: full, non-quantized `Qwen/Qwen3.5-9B` loaded in BF16 with LoRA adapters.
+- Eval: `general-agent-solver-local` through native `vf-eval --save-results` on the same held-in task sets before and after SFT.
+
+## Data Recipe
 
 | Field | Value |
 | --- | --- |
-| Trainer | Prime-RL SFT |
 | Dataset | `benchflow/general-agent-qwen35-9b-azure-gpt54mini-sft` |
-| Dataset rows | `4414` valid Azure GPT-5.4-mini teacher traces |
-| Source tasks | Prime `general_agent` open-source task corpus |
+| Dataset rows | `4414` |
+| Original source task count | `4417` |
+| Teacher model | Azure GPT-5.4-mini |
+| Teacher harness | Prime/Verifiers `general-agent-solver-rlm` |
+| Artifact format | Native `vf-eval --save-results` trajectories converted to Prime-RL `messages` + `tool_defs` SFT rows |
+| Excluded source tasks | `dog_breeding_t1`, `skydiving_center_t1`, `skydiving_center_t2` |
+| Exclusion reason | Stable Azure content-filter blocks during teacher trace generation |
+| Full teacher sweep artifact | `benchflow/env0-experiment-trajectories/experiments/general-agent/general-agent-daytona-teacher-full4417-tunnel8-20260624T015706Z` |
+| Data validation | Prime SFT JSONL validator rejected non-leading system messages and leakage fields before training |
+
+## Training Parameters
+
+| Field | Value |
+| --- | --- |
+| Trainer | Prime-RL SFT |
+| Model loaded for SFT | `Qwen/Qwen3.5-9B` full BF16 base weights |
+| Quantization | None for the completed `v0.0.1` LoRA run |
+| Adapter | LoRA |
+| LoRA rank | `16` |
+| LoRA alpha | `32` |
+| LoRA dropout | `0.0` |
+| Target modules | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` |
+| Trainable params | about `29.1M` |
+| Adapted base params | about `5.30B` |
+| Total base params loaded | about `9.44B` |
 | Sequence length | `2048` |
 | Global batch size | `8` |
 | Micro batch size | `1` |
 | Pack function | `cat` |
-| Shuffle / seed | `true` / `0` |
-| Max steps | `200` |
+| Shuffle | `true` |
+| Seed | `0` |
 | Optimizer | `AdamW` |
 | Learning rate | `5e-5` |
 | Weight decay | `0.01` |
-| Grad norm clip | `1.0` |
 | Betas | `0.9`, `0.999` |
+| Grad norm clip | `1.0` |
 | Scheduler | Linear |
 | Warmup steps | `20` |
-| Loss mask | Assistant messages only; system/user/tool context masked |
+| Decay steps | `180` |
+| Minimum LR | `0.0` |
+| Max steps | `200` |
+| Checkpoint interval | `20` |
+| Keep last | `3` |
+| Keep interval | `100` |
+| Save format | `safetensors` |
+| Loss mask | Assistant messages only; system, user, and tool messages are context-only |
+
+## Training Result
+
+| Metric | Value |
+| --- | ---: |
+| Completed step | `200` |
 | Final loss | `0.11897` |
-| NaN count | `0` |
+| `loss/nan_count` | `0` |
 | Peak GPU memory | about `40.8 GiB` |
+| Final adapter | `adapter_model.safetensors` in this repo |
+
+The initial `data.seq_len=8192` Prime-RL BF16 LoRA attempt OOMed on one H100. The completed `v0.0.1` run used `data.seq_len=2048`, system CUDA 12.8 `nvcc`/`ptxas`, and `g++-12` for the required FLA/TileLang kernels.
 
 ## Evaluation Results
 
-All evaluations below use `general-agent-solver-local`, native `vf-eval --save-results`, `max_model_len=4096`, `--enable-auto-tool-choice`, and `--tool-call-parser qwen3_xml`.
+All evaluations below use native Verifiers `vf-eval --save-results`, `general-agent-solver-local`, serving context length `4096`, `--enable-auto-tool-choice`, and `--tool-call-parser qwen3_xml`. Dynamic vLLM LoRA loading was not reliable for this stack, so eval served a merged local checkpoint built from this adapter plus `Qwen/Qwen3.5-9B`.
 
-| Task set | Base pass rate | SFT pass rate | Delta | Notes |
+| Task set | Base pass rate | LoRA SFT pass rate | Delta | Notes |
 | --- | ---: | ---: | ---: | --- |
-| Held-in 36 tier-0/1 tasks | `20/36 = 55.56%` | `23/36 = 63.89%` | `+8.33%` | No regressions; recovered `3d_print_shop_t1`, `accounting_firm_t1`, `allergy_clinic_t0` |
-| Held-in 50 tier-0/1 tasks | `27/50 = 54.00%` | `30/50 = 60.00%` | `+6.00%` | Latest assembled result; the final 14-task slice had no net delta |
+| Held-in 5 smoke | `1/5 = 20.00%` | `2/5 = 40.00%` | `+20.00 pp` | First serving/eval smoke |
+| Held-in 20 | `11/20 = 55.00%` | `13/20 = 65.00%` | `+10.00 pp` | Recovered `3d_print_shop_t1`, `accounting_firm_t1` |
+| Held-in 36 | `20/36 = 55.56%` | `23/36 = 63.89%` | `+8.33%` | No regressions; recovered `3d_print_shop_t1`, `accounting_firm_t1`, `allergy_clinic_t0` |
+| Held-in 50 assembled | `27/50 = 54.00%` | `30/50 = 60.00%` | `+6.00%` | Latest wider held-in result; final 14-task slice had no net delta |
 | Held-in 50 final 14-task slice | `7/14 = 50.00%` | `7/14 = 50.00%` | `+0.00%` | Recovered `animation_studio_t0`; regressed `antiquarian_bookshop_t0` |
 
-Held-in 50 provenance:
+Evaluation artifact prefixes:
 
-- Held-in36 comparison: `benchflow/env0-experiment-trajectories/experiments/general-agent/general-agent-qwen35-eval-heldin36-compare-20260624`
-- Held-in50 final 14-task run: `benchflow/env0-experiment-trajectories/experiments/general-agent/general-agent-qwen35-eval-heldin50-gap-20260624T190517Z`
-
-## Current QLoRA Seq8192 Training Run
-
-A longer-context QLoRA ablation is currently running and is not the adapter published on `main` yet. Its final metrics should replace this section only after `train_summary.json`, merged serving, and held-in eval finish.
-
-| Field | Current value |
-| --- | --- |
-| Run id | `general-agent-qwen35-qlora-seq8192-20260624T215352Z` |
-| Prime pod | `e0d97ec9c2db4b3f93529f1bbf61da60` |
-| GPU | `1x H100 80GB`, `$2.35/hr` |
-| Trainer | Custom Transformers + PEFT + bitsandbytes QLoRA |
-| Base checkpoint | `Qwen/Qwen3.5-9B` full, non-prequantized source checkpoint |
-| Quantization | `4bit NF4`, double quantization, BF16 compute |
-| Dataset | `benchflow/general-agent-qwen35-9b-azure-gpt54mini-sft/train.jsonl` |
-| Dataset rows | `4414` |
-| Sequence length | `8192` |
-| Max steps | `200` |
-| Micro batch size | `1` |
-| Gradient accumulation | `8` |
-| Effective batch size | `8` |
-| Learning rate | `5e-5` |
-| Warmup steps | `20` |
-| Weight decay | `0.01` |
-| Seed | `0` |
-| LoRA | `r=16`, `alpha=32`, `dropout=0.0` |
-| Target modules | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` |
-| Loss mask | Assistant turns only, built by chat-template prefix diff |
-| W&B project | `general-agent-qwen35-qlora-seq8192-20260624T215352Z` |
-| Current status at model-card update | In tokenization/training startup; no final loss or eval metric committed yet |
+- Held-in 5 smoke: `benchflow/env0-experiment-trajectories/experiments/general-agent/general-agent-qwen35-eval-smoke4096-20260624T152150Z`
+- Held-in 20 comparison: `benchflow/env0-experiment-trajectories/experiments/general-agent/general-agent-qwen35-eval-heldin20-compare-20260624`
+- Held-in 36 comparison: `benchflow/env0-experiment-trajectories/experiments/general-agent/general-agent-qwen35-eval-heldin36-compare-20260624`
+- Held-in 50 final 14-task run: `benchflow/env0-experiment-trajectories/experiments/general-agent/general-agent-qwen35-eval-heldin50-gap-20260624T190517Z`
 
 ## Loading
 
@@ -106,14 +130,18 @@ A longer-context QLoRA ablation is currently running and is not the adapter publ
 from peft import PeftModel
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
-base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-9B", torch_dtype="auto", trust_remote_code=True)
+base = AutoModelForCausalLM.from_pretrained(
+    "Qwen/Qwen3.5-9B",
+    torch_dtype="auto",
+    trust_remote_code=True,
+)
 model = PeftModel.from_pretrained(base, "benchflow/benchflow-qwen35-9b")
 tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-9B", trust_remote_code=True)
 ```
 
 ## Caveats
 
-- This is an SFT-stage reproduction artifact, not the full Prime paper recipe with the original teacher/model stack.
-- The training data was regenerated from open-source tasks with Azure GPT-5.4-mini teacher traces; three Azure-filtered source prompts were excluded from the trainable dataset.
-- Held-in eval lift is positive but modest: the latest held-in50 assembled result is `+6.00 pp`.
-- The active seq8192 QLoRA run is still in progress as of `2026-06-24 22:26:04 UTC` and should not be treated as a released checkpoint.
+- This is an SFT-stage reproduction artifact, not the full Prime paper recipe with the original teacher and student model stack.
+- The trainable dataset has `4414` rows rather than `4417` because three Azure teacher prompts were blocked by content filtering.
+- The latest held-in50 assembled lift is positive but modest at `+6.00 pp`; gains are concentrated in a small number of tasks rather than broad across-the-board recovery.
+- The next QLoRA seq8192 experiment is excluded from `v0.0.1` and should receive its own update/tag only after it completes.