AmelieSchreiber/toricgt-checkpoints

ToricGT Checkpoints

This repository stores compact ToricGT checkpoint artifacts from the OpenAI Parameter Golf BPB experiments. The published artifact is an OpenAI baseline adaptation trained for low bits-per-byte (BPB) while carrying ToricGT's graph, tropical, toric, topological, memory, and category-theoretic training signals as training-time structure.

The core contest-facing model is intentionally small and self-contained. It uses a ConvexTok-2048 deterministic tokenizer path, first-class TokenGT-style FineWeb graphification, graph-output flattening for BPB scoring, tropical/toric tokenization-DAG features, embedding-space GFlowNet and Forest-of-Thought training heads, multi-token prediction, and score-first evaluation controls. Training-only probes include GraphCG, analogical memory retrieval, toric geometry, toric vector-bundle 1D-cone/sheaf metrics, Toric BGG category O, Koszul persistence, combinatorial toric commutative algebra, and derived signature losses/metrics. Those probes are used to shape representations and audit behavior; the minimal competition artifact keeps only what is needed for the BPB model.

Best Published Artifact

• uploaded UTC: 2026-06-21T17:46:33.018781+00:00
• source campaign: tg-bpb-bestof10-convextok2048-det-20260621T040717Z
• source phase: best-of-10-short-runs
• selected run: tg-bpb-bestof10-convextok2048-det-20260621T040717Z-r008-gate2500_experimental_family_selective-20260621T154110Z
• profile: gate2500_experimental_family_selective
• checkpoint step: 1000
• train BPB: 0.5036
• validation BPB: 0.4924
• int8+zlib round-trip BPB: 0.49533264
• compressed artifact bytes: 12,202,796
• source checkpoint path: checkpoints/tg-bpb-bestof10-convextok2048-det-20260621T040717Z-r008-gate2500_experimental_family_selective-20260621T154110Z_step_001000.pt

Lower BPB is better. The values above are the local ToricGT campaign export and round-trip measurements for this selected artifact, not an official OpenAI leaderboard verification.

Files

• final_model.int8.ptz - minimal int8+zlib Parameter-Golf-style model payload.
• final_model.pt - full state dict copy for local debugging and inspection.
• checkpoints/..._step_001000.pt - selected PyTorch checkpoint payload.
• train.log and logs/*.txt - run logs for the selected checkpoint.

Minimal Artifact Smoke Run

The snippet below loads the minimal final_model.int8.ptz artifact, reconstructs the OpenAI-compatible ToricGT baseline from the local ToricGT checkout, and scores a small nontrivial token-id sequence. This is a smoke inference/scoring example for the exported artifact. Full BPB evaluation should use the competition FineWeb validation stream and the matching tokenizer/shards.

git clone https://github.com/amelie-iska/ToricGT.git
cd ToricGT
python -m pip install torch huggingface_hub

PYTHONPATH=src:amelie-iska/parameter-golf python - <<'PY'
import io
import os
import zlib

import torch
from huggingface_hub import hf_hub_download

# Shape and graphification settings for the published int8 artifact.
os.environ.update({
    "VOCAB_SIZE": "2048",
    "MODEL_DIM": "512",
    "NUM_LAYERS": "9",
    "NUM_HEADS": "8",
    "NUM_KV_HEADS": "4",
    "MLP_MULT": "2",
    "TIE_EMBEDDINGS": "1",
    "FINEWEB_GRAPHIFY": "1",
    "TOKENGT_FIRST_CLASS": "1",
    "TOKENGT_GRAPH_RADIUS": "4",
    "TOKENGT_TOKEN_CLASS_BUCKETS": "64",
    "TOKENGT_POSITION_BUCKETS": "256",
    "CONVEXTOK_DAG_FEATURES": "1",
    "OAI_FINEWEB_OUTPUT_FLATTENING": "1",
    "GRAPH_OUTPUT_FLATTENING": "1",
    "GRAPH_OUTPUT_VIRTUAL_EDGE_TOKENS": "1",
    "GRAPH_OUTPUT_SCORE_CORRECTION": "1",
})

from train_gpt import GPT, Hyperparameters, dequantize_state_dict_int8

args = Hyperparameters()
model = GPT(
    args.vocab_size,
    args.num_layers,
    args.model_dim,
    args.num_heads,
    args.num_kv_heads,
    args.mlp_mult,
    args.tie_embeddings,
    args.tied_embed_init_std,
    args.logit_softcap,
    args.rope_base,
    args.qk_gain_init,
    args.fineweb_graphify,
    args.tokengt_first_class,
    args.tokengt_graph_radius,
    args.tokengt_distance_features,
    args.tokengt_token_class_buckets,
    args.tokengt_position_buckets,
    args.tokengt_structural_weight,
    args.tokengt_edge_weight,
    args.tokengt_torus_weight,
    args.tokengt_identifier_dim,
    args.tokengt_identifier_weight,
    args.tokengt_endpoint_weight,
    args.tokengt_edge_token_weight,
    args.convextok_dag_features,
    args.convextok_dag_feature_weight,
    args.graph_output_flattening,
    args.graph_output_edge_radius,
    args.graph_output_distance_features,
    args.graph_output_node_weight,
    args.graph_output_edge_weight,
    args.graph_output_virtual_edge_tokens,
    args.graph_output_edge_token_weight,
    args.graph_output_score_correction,
    args.graph_output_score_correction_weight,
)

artifact = hf_hub_download(
    repo_id="AmelieSchreiber/toricgt-checkpoints",
    filename="final_model.int8.ptz",
)
with open(artifact, "rb") as f:
    packed = f.read()
state = torch.load(io.BytesIO(zlib.decompress(packed)), map_location="cpu")
model.load_state_dict(dequantize_state_dict_int8(state), strict=True)
model.eval()

# Nontrivial token-id exemplar.  For real BPB, replace this with ConvexTok/FineWeb
# tokenized validation chunks and divide total NLL by log(2) times decoded bytes.
tokens = torch.tensor(
    [[4, 17, 128, 512, 1024, 33, 71, 777, 12, 44, 91, 203, 8, 19, 21, 5]],
    dtype=torch.long,
)
with torch.no_grad():
    loss, hidden, per_token_nll = model.forward_aux(
        tokens[:, :-1],
        tokens[:, 1:],
        flatten_graph_output=True,
    )

print(f"mean token NLL: {loss.item():.4f}")
print("hidden shape:", tuple(hidden.shape))
print("per-token NLL shape:", tuple(per_token_nll.shape))
PY

Why This Repo Matters

ToricGT is a research program for making graph-token reasoning auditable while still optimizing ordinary compression metrics. The Parameter Golf adaptation is not the full graph research model; it is the compact BPB-facing path that keeps the highest-value structure:

• ConvexTok tokenization is treated as a byte-boundary DAG. Candidate token edges, LP relaxation scores, selected rounded paths, and byte-fallback edges become TokenGT graph features.
• Tropical attention and min-plus dynamic-programming views organize path, routing, and active-face behavior. Finite tropical certificates are embedded into toric charts for exact Sage/Macaulay2 audits.
• FineWeb is graphified as causal token/edge structure, then flattened only for OAI BPB scoring. General graph data remains graph-in/graph-out.
• GraphCG, memory retrieval, GFlowNet/FoT search, persistent homology, Toric BGG category O, Koszul and module-category checks, vector-bundle/sheaf metrics, divisors, one-dimensional cones, and combinatorial commutative algebra are treated as training/audit signals, not as decorative dashboard metrics.

For visual reports, methodology notes, and the current project status, see the project page and GitHub repository linked above.