AUBIN-Learn: instant self-learning memory, fast skills, self-calibration + reports

README.md

@@ -18,3 +18,54 @@ Part of the **AUBIN by Norovox** family: [AUBIN-12B](https://huggingface.co/emre
 | MindAct Flan-T5-XL | 52.0 | 38.9 | 39.6 |
 
 Same protocol as AUBIN-12B-Web: MindAct multiple-choice, 200 steps per split. The model was trained from scratch on the full Mind2Web training split for 5,000 steps.
+
+<!-- AUBIN-LEARN:BEGIN -->
+## AUBIN-Learn — instant self-learning (experimental, Norovox core)
+
+AUBIN can learn from feedback without retraining. Verified cases go into an external **decision memory** (a write takes well under
+a millisecond with the hashing embedder), and a **self-calibrator** (Hedge / multiplicative weights) shifts trust per source between
+the model, the memory and their fusion — so where memory is not useful yet, AUBIN keeps trusting itself. Measured on Kev's public
+suites; full tables, protocol and code: [`reports/AUBIN_LEARN.md`](reports/AUBIN_LEARN.md), [`code/aubin/learn.py`](code/aubin/learn.py).
+
+* **Feedback stream, kev_test:** all 6 AUBIN variants improve, **+1.0 to +1.3 points** (best 84.3 → 85.6). Separate protocol — the
+  label is revealed after each answer — so it is not comparable to static scores (Kev-9B 87.4 static).
+* **Never-seen sources (transfer, memory starts empty):** −0.4 to +0.1 points — it does not hurt; a few hundred feedbacks per source
+  are not enough to help yet.
+* **Fast skills, static locked test:** per-source classifiers learned from memory in seconds (switched on only where dev proves
+  them) lift kev_test for all 4 measured AUBIN variants, **+0.3 to +0.6 points** (ensemble 84.3 → 84.8; banking77 59.5 → 65.5).
+* **Raw memory (kNN) on the static test:** no reliable gain (−1.0 to +0.7) — AUBIN already learned these sources.
+
+## Results, 3 October 2026 (full report: [`reports/AUBIN_RESULTS_2026-10-03.md`](reports/AUBIN_RESULTS_2026-10-03.md))
+
+| benchmark | AUBIN | reference |
+|---|---|---|
+| Typed decisions (2,000 decisions) | **77.55** (AUBIN-Learn, weights fixed before test) | Laya 76.65 · meraGPT 76.8 · Jev 72.7 |
+| Kev suites, Kev's training sources (kev_test) | **85.7** (AUBIN ensemble, selected on cal split) | Kev-0.8B 83.8 · Kev-4B 86.5 · Kev-9B 87.4 |
+| Kev suites, transfer test | **86.5** ensemble · **89.0** AUBIN-31B | – |
+| Mind2Web cross-domain step SR (200 steps) | **48.5** AUBIN-31B, no web training | MindAct-XL 39.6 · GPT-4 26.4 |
+| Grid control, 100 unseen episodes | **92% success, 0 lava deaths** (12B-Control + shield) | greedy rule + same shield 89% |
+
+On Kev's own training sources AUBIN is still 1.7 points behind Kev-9B; this is stated, not hidden.
+
+**Evening update (3 Oct, all measured, details in the results report):**
+
+| benchmark | AUBIN | reference |
+|---|---|---|
+| ScreenSpot click accuracy (visual computer use) | **69.3** AUBIN-E4B-Screen (zero-shot 48.5; 67.7 after round 1) | SeeClick 53.4 · CogAgent 47.4 · UGround-7B 73.3 · UI-TARS-7B 89.5 |
+| Mind2Web step success, cross-task / website / domain | 47.0 / 37.0 / 43.5 (12B) · 47.0 / 34.5 / 42.0 (E4B, ≈2× faster) | MindAct-XL 52.0 / 38.9 / 39.6 |
+| ViZDoom FPS, kills per episode (30 episodes) | **17.3** with in-game self-learning (AUBIN-Learn) · 15.6 without (AUBIN-12B, 0.6 s/move) | random 1.3 · scripted rule 18.8 |
+| Kev training sources + learned skills (selected on kev_dev + cal only) | **87.08**, NLL 0.43 (ensemble 85.69, NLL 0.79) | Kev-9B 87.4 (not yet beaten) · Kev-27B 87.0 · Kev-4B 86.5 |
+
+New in `code/`: `AubinLearning.acquire_skill` (learns a skill, self-tests on held-out data, enables only on proven gain), `learn_skill2/3.py`, `fuse_multi.py`, `screenspot_train/eval.py`, `fps_vizdoom.py`.
+
+### The AUBIN family: every ability, one interface (`AubinEngine` routes between them)
+
+| ability | model | measured |
+|---|---|---|
+| typed decisions (choice / score / yes-no), calibrated | [AUBIN-31B](https://huggingface.co/emrevrg/AUBIN-31B) · [AUBIN-12B](https://huggingface.co/emrevrg/AUBIN-12B) · [12B-v3b](https://huggingface.co/emrevrg/AUBIN-12B-v3b) · [12B-v3d](https://huggingface.co/emrevrg/AUBIN-12B-v3d) · [E4B-v3](https://huggingface.co/emrevrg/AUBIN-E4B-v3) | typed-decisions 77.55 (#1) · Jev's set 8/8 · Kev unseen sources 89.8 · Kev training sources 87.08 |
+| web agent | [AUBIN-12B-Web](https://huggingface.co/emrevrg/AUBIN-12B-Web) · [AUBIN-E4B-Web](https://huggingface.co/emrevrg/AUBIN-E4B-Web) (fast) · 31B web training running | Mind2Web cross-domain 43.5 (MindAct-XL 39.6) |
+| visual computer use (click on screenshots) | [AUBIN-E4B-Screen](https://huggingface.co/emrevrg/AUBIN-E4B-Screen) · 12B screen training running | ScreenSpot 69.3 |
+| real-time control | [AUBIN-12B-Control](https://huggingface.co/emrevrg/AUBIN-12B-Control) · [AUBIN-E4B-Control](https://huggingface.co/emrevrg/AUBIN-E4B-Control) | 92% success, 0 lava deaths |
+| FPS play, self-learning in game | AUBIN-12B + AUBIN-Learn | ViZDoom 17.3 kills/episode vs 15.6 without learning (30 episodes) |
+| self-learning, self-acquired skills | `AubinLearning` (`learn`, `acquire_skill`) in every repo | skills switch on only after a held-out self-test proves a gain |
+<!-- AUBIN-LEARN:END -->

SUPPORT.md

@@ -7,7 +7,7 @@ When I started this project I had no team and no budget: just one computer, free
 Even so, AUBIN has measured results today:
 - **#1** on the typed-decisions benchmark, ahead of Laya, meraGPT and Jev.
 - Ahead of Jev on **all 8 metrics** of Jev's own published split.
-- **67.7%** on ScreenSpot for visual computer use, and an FPS agent that learns while it plays.
+- **69.3%** on ScreenSpot for visual computer use, and an FPS agent that learns while it plays.
 
 Every number comes with code and result files, so anyone can reproduce it.
 
@@ -24,7 +24,7 @@ Bu projeye başladığımda elimde ne bir ekip ne bir bütçe vardı; sadece bir
 Buna rağmen AUBIN bugün ölçülmüş sonuçlara sahip:
 - Tipli karar kıyaslamasında **birinci**; Laya, meraGPT ve Jev'in önünde.
 - Jev'in kendi yayımladığı kümede **sekiz ölçütün sekizinde** önde.
-- Görsel bilgisayar kullanımında (ScreenSpot) **%67,7**; oynarken öğrenen bir FPS ajanı.
+- Görsel bilgisayar kullanımında (ScreenSpot) **%69,3**; oynarken öğrenen bir FPS ajanı.
 
 Her sayının kodu ve sonuç dosyası açık, herkes kendisi tekrar üretebilir.
 
@@ -36,4 +36,4 @@ Destek olmak, yatırım yapmak ya da sadece tanışmak isterseniz **zgremre@gmai
 
 ---
 *Built with Claude Opus 5.5 and GPT-5.6 Sol; because of OpenAI usage limits, the final stretch was completed with Claude Opus 5.5.*
-*Geliştirme Claude Opus 5.5 ve GPT-5.6 Sol ile başladı; OpenAI'ın kullanım limitleri nedeniyle son noktayı Claude Opus 5.5 ile koyduk.*
+*Geliştirme Claude Opus 5.5 ve GPT-5.6 Sol ile başladı; OpenAI'ın kullanım limitleri nedeniyle son noktayı Claude Opus 5.5 ile koyduk.*

reports/AUBIN_RESULTS_2026-10-03.md

@@ -136,7 +136,8 @@ Code is in `code/`. Every number above can be reproduced from the scripts and ru
 
 | model | accuracy |
 |---|---|
-| **AUBIN-E4B-Screen** (8k leak-free wave-ui examples) | **67.7** |
+| **AUBIN-E4B-Screen**, round 2 (16k leak-free wave-ui examples) | **69.3** |
+| AUBIN-E4B-Screen, round 1 (8k examples) | 67.7 |
 | AUBIN-E4B zero-shot | 48.5 |
 | SeeClick | 53.4 |
 | CogAgent | 47.4 |