canho/lateon-ar-ours6k-ckpt30

lightonai/LateOn fine-tuned with PyLate losses.Contrastive, checkpoint at step 30 of 126 (epoch 0.71).

  • Training data: HF canho/ours-6k train split (5,469 pairs; query + 1 positive + 9 hard negatives, title. abstract).
  • Recipe: LightOn's LateOn fine-tuning script (scripts/finetune/english_late_interaction.py), same as the step-1 wave-1 run (canho/lateon-ar-step1-wave1-ckpt*): 8 GPUs, global batch 128 (split_batches, 16 per GPU, in-batch negatives per GPU), 7 of the 9 negatives sampled per step, temperature 0.02, lr 3e-6 (linear, no warmup), 3 epochs, bf16, checkpoint every 10 steps.
  • Query [Q] + query, 64 tokens; document [D] + text, 300 tokens. No instruction.
from pylate import models
m = models.ColBERT("canho/lateon-ar-ours6k-ckpt30")
q = m.encode(["..."], is_query=True); d = m.encode(["..."], is_query=False)
Downloads last month
9
Safetensors
Model size
0.1B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for canho/lateon-ar-ours6k-ckpt30

Base model

lightonai/LateOn
Finetuned
(42)
this model