AriannaV6

AriannaV6 is an all-in-one, agent-native large language model built on Google's gemma-4-12B (the gemma4_unified architecture). It is one release in the Arianna line — a from-scratch, self-improving fine-tune project run by a solo builder on a home GPU fleet (project name "Aria" internally; "Arianna" is the public/shipped name). This is the sixth public checkpoint and, as of this card, the strongest one released: it beats its immediate predecessor (V5) on 3 of 5 held-out benchmark suites while holding steady on the rest, and it exists specifically to correct a knowledge/commonsense regression discovered in earlier versions (details below — we're not hiding it).

If you're new to this model family, read the "Which version should I use" and "Known limitations" sections before anything else. This is a hobbyist/independent research project, not a lab release — it is honestly documented, including its failures, because that's the only way version numbers this high mean anything.

---

TL;DR

• Base: google/gemma-4-12B (~12B parameters, encoder-free natively multimodal architecture — the same weights process text, images, and audio; there is no separate frozen vision/audio tower).
• What changed: a QLoRA adapter trained on top of the base, then merged into full weights. The adapter teaches tool-calling, an unprompted persistent identity ("Arianna" / creator provenance), long-horizon memory behavior, and general capability uplift (code, reasoning, physics, grammar, storytelling, safety refusal) — while this specific version (V6) was trained explicitly to also hold onto the base model's raw knowledge and commonsense reasoning, which earlier versions partially lost.
• Formats: full-precision merged safetensors (5 shards, ~22.3 GiB, load directly with transformers) and a GGUF ladder for llama.cpp / Ollama / LM Studio-style local inference.
• License: inherited from Gemma — cc-by-nc-sa-4.0. Non-commercial, share-alike, attribution required. See Gemma's usage policy — this fine-tune does not grant any rights the base license doesn't.
• Context length: up to 262,144 tokens (inherited from base architecture; practical usable context depends on your inference stack and hardware).
• Sidecars, not baked-in: real-time voice (STT/TTS), image generation, and video generation are deliberately not trained into these weights. They run as separate services that call this model as the reasoning core. Baking them into the weights was tried early in this project and caused a severe capability collapse (see History). If you're looking for a model that speaks out loud or draws pictures, this is the brain, not the mouth or the hands — pair it with a TTS/diffusion sidecar.

---

Model family history — read this before choosing a version

This project ships fast and iterates in public, including public failures. Version numbers do not all represent monotonic improvement — some were course-corrections, and the numbering has gaps because not every internal experiment (v18–v28 in the internal adapter registry) was promoted to a public release.

Public release	What it is	Status
AriannaV1	First all-in-one merge: the adapter (internally "v17") merged directly into gemma-4-12b-it. Established the "core weights own everything except sidecars" architecture.	Shipped, private, superseded
AriannaV2	AriannaV1 + full GGUF quant ladder + end-to-end voice/audio pipeline verification. First release with a production-readiness gate (every capability had to pass before upload).	Shipped, private, superseded
AriannaV3	Diagnostic/experimental, not a clean uplift. External evaluation (see below) found V1/V2 scored 8.5 percentage points below the base model on MMLU and HellaSwag under identical conditions — a real regression, not measurement noise. Root cause: the adapter had directly fine-tuned lm_head and embed_tokens (the output/input embedding layers — 1B+ parameters, ~89% of that adapter's file size), which is the layer most responsible for raw factual recall. A cheap fix (stripping just those two tensors back to the frozen base) was tried and did not fully recover the loss — proving the rest of the adapter's weights had co-adapted to the damaged embeddings. This ruled out the easy fix and justified a full retrain.	Internal/experimental only
AriannaV4	A fresh adapter trained without touching lm_head/embed_tokens at all (the lesson from V3). First fully public release. ~5,400+ downloads.	Shipped, public, superseded
AriannaV5	Iteration on V4's recipe with training-data reweighting aimed specifically at closing the remaining knowledge/commonsense gap (oversampling MMLU/HellaSwag/Winogrande/ARC/GSM8K-style examples in the replay mix).	Shipped, public, superseded
AriannaV6 (this model)	Current best. Gate-evaluated against the V5 champion on five held-out benchmark suites and promoted on merit (see Evaluation below).	Current / this card

If you just want the best available model today, use V6. If you're specifically studying the project's history or comparing what fine-tuning embeddings does to a model, V3 through V5 are useful reference points and their write-ups are preserved in the project's internal records.

---

Architecture

AriannaV6 is built on google/gemma-4-12B's gemma4_unified architecture (via the community unsloth/gemma-4-12b-it loading path used during training). Key specs, read directly from the shipped config:

Property	Value
Total parameters	~11.96B
Hidden size	3,840
Layers	48
Attention heads	16 query heads / 8 KV heads (grouped-query attention), with 8 of the 48 layers additionally using a wide per-layer head config (512 head-dim, 1 KV head)
Attention pattern	Mostly sliding-window (1,024 tokens) with a full-attention layer every 6th layer
Vocabulary	262,144 tokens
Max position embeddings	262,144
Multimodal	Native, encoder-free. Vision (16px patches, 280 soft tokens) and audio (640-dim embedding) are projected directly into the same embedding space the text tokens live in — there is no separate CLIP-style vision tower or Whisper-style audio encoder with its own weights. This means every gradient step during fine-tuning touches the same parameters used for image/audio understanding, which is why multimodal capability has to be actively protected during training (see Training below) rather than assumed safe by default.
Tied embeddings	Yes (tie_word_embeddings: true)

What this model does NOT include as trained weights: speech synthesis (TTS), speech recognition beyond native audio-token understanding, image generation, or video generation. These are architecturally absent from the base model and were never added. Any product built on Arianna that offers voice or image generation is calling this model for reasoning/tool-orchestration and a separate specialized model for the generative modality.

---

What was trained (the adapter, before merging)

Every Arianna release starts from the same base and trains a LoRA (QLoRA, 4-bit base, bf16 compute) adapter on top, then merges it into full weights for distribution. The adapter for V6 (internal id adapter-v29) targets the attention and MLP projection matrices only:

q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Deliberately excluded: lm_head and embed_tokens. This is the single most important lesson baked into V6 and every release since V4 — see the AriannaV3 note above. Fine-tuning the embedding/output layers directly is the fastest way to move benchmark numbers in training, and the fastest way to quietly destroy the base model's world knowledge. It also produces a misleadingly large adapter file (an adapter that touches embed_tokens is roughly 9x larger than one that doesn't, because that single tensor is over a billion parameters).

Training mix (general shape, consistent across the V4→V6 line): a task-specific corpus (tool-calling formats, persistent-identity examples with and without a system prompt, multi-turn memory-conditioning examples) blended with a replay pool of general text, code, and reasoning data, plus a held-out slice of multimodal examples (~25–30% floor) to protect vision/audio understanding from drifting during text-only-heavy steps. V6 specifically increased the sampling weight on general-knowledge and commonsense-style examples relative to V5, in direct response to the gap that V3's diagnosis identified.

Capabilities the adapter is trained for, each with its own held-out evaluation probe: math (GSM8K-style), tool selection and calling, unprompted identity/creator-provenance, multi-turn memory conditioning, code, general reasoning, physics/science, grammar, storytelling, safety refusal, and vision/audio understanding (multimodal pass/fail).

---

Evaluation

Two separate evaluation lineages exist for this project and it's important not to conflate them:

1. The gated promotion suite (used to decide whether a candidate adapter replaces the production incumbent). Held-out, seeded, training-disjoint probes per capability, each with its own noise-tolerance band, pooled case-weighted. An adapter is only promoted if it passes or beats the incumbent on this suite. The V6 lineage held roughly: math ≈0.92, tools 1.0, identity 0.90, memory 0.90, code/reasoning/grammar/storytelling/safety in the 0.80–1.0 range (0–1 scale, task-specific rubrics).

2. The knowledge/commonsense diagnostic (the one that produced the V3→V6 story). Standard academic benchmarks — MMLU and HellaSwag — run generatively through the model's own chat template (not the multiple-choice-logprob method most published leaderboards use; see the methodology note below), n≈400 per condition, fixed seed, compared directly against stock gemma-4-12b run through the identical harness.

AriannaV6's gate result (internal id adapter-v29) vs. the V5 champion, five held-out suites:

Suite	V6	Δ vs. V5
MMLU	71.6	+1.8
ARC-Challenge	91.8	+1.0
HellaSwag	87.6	+4.0
GSM8K	93.2	−0.4
Winogrande	70.2	−1.0

V6 improved on 3 of 5 suites (including the two — MMLU and HellaSwag — that were the specific target of the V3 diagnosis) with only minor regressions on the other two, both within noise tolerance. It was promoted over V5 on this basis.

Methodology note (please read before comparing to other leaderboards): these numbers are produced by a generative, chat-template-based harness running the actual model as end users will run it — not the standard academic multiple-choice log-likelihood scoring most public MMLU/HellaSwag leaderboard numbers use. The two methodologies are not directly comparable, and generative scoring is typically a few points lower for a given model, all else equal, because it depends on the model producing the correctly-formatted answer rather than just ranking the correct option highest internally. When this project's own internal diagnostics compare Arianna to stock Gemma, both sides are always run through the identical harness so the comparison is apples-to-apples even if the absolute numbers don't match what you'd see on a public leaderboard for "gemma-4-12b."

---

Files in this repository

• Merged full-precision model (repository root): model-0000X-of-00005.safetensors (5 shards, ~22.3 GiB total) + config.json, generation_config.json, tokenizer.json, tokenizer_config.json, chat_template.jinja, processor_config.json. Load directly with transformers — no adapter application needed, this is the fully fused model.
• gguf/ — quantized GGUF builds for llama.cpp-based local inference (Ollama, LM Studio, koboldcpp, text-generation-webui, etc.). Check this folder's current contents on the Hub for the exact quant levels available at any given time — the ladder is built and uploaded incrementally (F16 first, then progressively smaller K-quants) and may still be filling in shortly after a release goes live. Smaller quants trade some quality for dramatically lower VRAM/RAM requirements and faster inference; if you have the hardware, prefer a higher-precision quant (Q8_0 or F16) for anything requiring careful reasoning, and drop to Q4_K_M/Q5_K_M for lower-resource or latency-sensitive use.

How to use it

With transformers (full precision, needs a real GPU with ~24GB+ VRAM for bf16, or 4-bit quantization for less):

from transformers import AutoModelForImageTextToText, AutoTokenizer
import torch

model_id = "SurgeFF/AriannaV6"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto"
)

messages = [{"role": "user", "content": "Introduce yourself."}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=256)
print(tok.decode(out[0], skip_special_tokens=True))

With llama.cpp / Ollama / LM Studio (GGUF, runs on far less hardware): download the GGUF file matching your available RAM/VRAM from the gguf/ folder, point your inference tool at it. The chat template is embedded in the merged model's chat_template.jinja and mirrored into the GGUF metadata — most modern local-inference front ends will apply it automatically.

For image or audio input, this model accepts them natively (encoder-free multimodal) — pass them through your inference stack's standard multimodal message format; no separate vision/audio adapter is required.

---

Known limitations

• Not a match for the base model on every knowledge axis, yet. V6 closed most, but not all, of the gap identified in V3 (GSM8K and Winogrande saw small regressions relative to V5 in this release's own gate). If your use case is knowledge-intensive and you don't need Arianna's tool-calling/identity/memory behavior, stock google/gemma-4-12B may still outperform on pure trivia/commonsense recall.
• Non-commercial license. Inherited cc-by-nc-sa-4.0 from Gemma. Do not use in a commercial product without separately satisfying Gemma's terms.
• No baked-in voice or image generation. See Architecture. If a downstream product built on this model appears to speak or generate images, that's a sidecar system, not this checkpoint.
• Solo-maintained, fast-moving project. This is not a corporate-backed release with an SLA. Version history has real course-corrections in it (see V3). Check the project's Hugging Face activity for what has shipped after this card was written before assuming V6 is still the latest.
• Generative-harness benchmark numbers are not directly comparable to standard leaderboards. See the Methodology note above.

Intended use

General-purpose assistant use, tool-calling / agentic workflows, and as the reasoning core for larger multimodal systems that add voice or image generation as separate components. Research and personal/non-commercial use per the license. Not evaluated or warrantied for high-stakes domains (medical, legal, financial decision-making) — treat outputs like any other LLM's: verify before acting on anything consequential.

Acknowledgements

Built on google/gemma-4-12B. Training tooling: Unsloth (QLoRA acceleration), PEFT, llama.cpp (GGUF conversion/quantization). This is an independent project, not affiliated with or endorsed by Google.