laya-multilingual-lt-med

Lithuanian medical decision model. It is a fine-tune of convaiinnovations/laya-multilingual on AIML-VU/MEDLINE-S1-LT.

Give it a Lithuanian title and abstract (state) and the typed questions from that dataset. In one forward pass it returns a probability for every option. It does not generate text.

Load it with the laya package, not with AutoModelForSequenceClassification. The weights are a merged Laya checkpoint: model.safetensors, encoder/, tokenizer/, and rl_agent_config.json.

This is a research model. It is not a medical device and it is not for clinical decisions.

Result on the held-out test split

The comparison is the official MEDLINE-S1-LT test split: 4,857 abstracts, 29,654 decisions. The same questions and the same gold labels were used before and after fine-tuning.

Before is the released convaiinnovations/laya-multilingual checkpoint, scored as a decision model. It is not a randomly initialized head. After is this checkpoint, with temperatures fitted on the calibration split only. Temperature changes calibration and loss. It does not change the argmax, so the accuracy gain is from the fine-tune.

	Before	After	Change
Decision accuracy	0.448304	0.923720	+0.475416
Correct decisions	13,294 / 29,654	27,392 / 29,654	+14,098
Record accuracy	0.031089	0.627548	+0.596459
Abstracts with every question correct	151 / 4,857	3,048 / 4,857	+2,897
Expected calibration error	0.343504	0.003805	−0.339699
Loss	1.331877	0.195695	−1.136182

Record accuracy is the stricter number: an abstract counts only when every question on it is right. That is the "improved records" result. It moves from 151 abstracts (3.11%) to 3,048 abstracts (62.75%).

The training log stored rates rounded to six decimals, not the integer counts. Each count above is the only integer whose ratio rounds back to the logged rate.

No per-question accuracy was stored, so this card does not break the gain down by design, disease_area, or the yes/no questions. eval_results.json is the same report in JSON, without any local path.

After fine-tuning only

Development and calibration were not scored before training. These rows are not a before/after comparison.

Split	Role	Decisions	Decision accuracy	Fully correct abstracts	ECE	Loss
Development	model selection	10,969 / 11,930	0.919447	1,213 / 1,964	0.016392	0.202076
Calibration	temperature fit	2,169 / 2,352	0.922194	241 / 390	0.009611	0.198909
Test	held out, reported above	27,392 / 29,654	0.923720	3,048 / 4,857	0.003805	0.195695

The saved weights are the best development-loss state kept during training (development loss 0.202145). Temperatures were fitted after that, on calibration data that was not used to choose the weights.

What the test split contains

These are question counts in the dataset, not model scores.

Question	Type	Test decisions
design	choice, 6 designs	1,944
disease_area	choice, MeSH C-tree area	1,014
humans	yes/no	4,857
animals	yes/no	4,857
adverse	yes/no	4,857
drug_therapy	yes/no	4,857
group_female	yes/no	1,052
group_male	yes/no	1,032
group_older_adults	yes/no	1,012
group_infants	yes/no	1,024
group_adolescents	yes/no	1,088
group_children	yes/no	1,061
group_adults	yes/no	999
Total		29,654

Option keys stay in English (systematic_review, Neoplasms, true, false). Instructions and option descriptions are Lithuanian. Gold labels come from NLM MeSH and publication types, not from a language model. The dataset card describes the rules.

Use

pip install laya

import laya

agent = laya.load("AIML-VU/laya-multilingual-lt-med")

state = (
    "Atsitiktinių imčių kontroliuojamajame tyrime 240 suaugusių pacientų "
    "gavo tiriamąjį vaistą arba placebą. Vertintas nepageidaujamas poveikis."
)
questions = {
    "design": {
        "type": "choice",
        "instructions": "Koks tai tyrimo ar straipsnio tipas?",
        "criteria": {
            "systematic_review": "ankstesnių tyrimų sisteminė apžvalga arba metaanalizė",
            "randomized_trial": "atsitiktinių imčių kontroliuojamasis tyrimas",
            "other_trial": "klinikinis tyrimas be atsitiktinių imčių",
            "case_report": "vieno ar kelių pacientų atvejo aprašymas",
            "observational": "stebimasis tyrimas (kohortinis, atvejo ir kontrolės arba skerspjūvio)",
            "narrative_review": "naratyvinės apžvalgos straipsnis",
        },
    },
    "humans": {
        "type": "noul",
        "instructions": "Ar šiame tyrime buvo tiriami žmonės (pacientai ar dalyviai)?",
        "criteria": {
            "true": "Teiginys teisingas arba atsakymas yra taip.",
            "false": "Teiginys klaidingas arba atsakymas yra ne.",
        },
    },
}

result = agent.predict(state, questions)
print(result["answers"]["design"]["choice"])
print(result["answers"]["design"]["probabilities"])
print(result["answers"]["humans"]["noul"])  # P(true); at least 0.5 means yes

A local copy of this folder loads the same way:

agent = laya.load("laya-multilingual-lt-med")

state may be a string or a JSON object. questions maps an id to one of choice, score, or noul, which is the Laya API. This fine-tune saw only choice and noul. A yes/no answer is answers[name]["noul"], the probability of true, not a boolean. A choice answer is answers[name]["choice"] plus probabilities.

The checkpoint stores max_len 2048 and head_max_len 256. Training and the numbers above used that limit. The mmBERT encoder can hold 8,192 positions, but this fine-tune was not evaluated at a longer limit. Fifteen of 29,654 test decisions and 114 of 236,720 training decisions were truncated at 2,048 tokens.

If laya.load hangs while TensorFlow is installed, the original Laya card's workaround is to run with USE_TF=0.

Training

Base	convaiinnovations/laya-multilingual
Encoder	jhu-clsp/mmBERT-base (22 layers, hidden 768, vocab 256,000)
Data	AIML-VU/MEDLINE-S1-LT
Train	38,950 abstracts, 236,720 decisions
Method	Unsloth FastDecisionModel and DecisionTrainer, LoRA, then merged
LoRA	r = 16, alpha = 16, dropout 0, all linear modules
Learning rate	2e-4 body, 1e-4 decision head, cosine, 10 warmup steps
Epochs	2
Batch	16, gradient accumulation 2, seed 3407, bf16, full precision (not 4-bit)
Trainable parameters	18,348,547 of 325,288,195 (5.64%)
Steps	14,796 in 16,835 s (1.138 s/step)
Train loss	0.216299
Peak memory	5.77 GiB
Finished	2026-10-17

Fitted temperatures, applied at prediction time:

Question type	Temperature	Fitted
choice	1.217570	yes
score	1.0	no; this dataset has no score questions
noul	1.184432	yes

The released Laya checkpoint ships temperatures [1.0, 1.0, 1.0]. The original model card says that checkpoint is systematically over-confident and that a temperature refit is required before the probabilities are trusted. The test ECE here, 0.343504 before and 0.003805 after, is consistent with that warning and with the refit on MEDLINE-S1-LT calibration data.

Original model

This repository does not replace convaiinnovations/laya-multilingual. Use that checkpoint for the general multilingual decision model. Use this one for the Lithuanian MEDLINE questions above. Fine-tuning was measured only on MEDLINE-S1-LT. It was not re-measured on the original Laya suites, so this card does not claim that those results still hold.

Laya is a non-autoregressive System 1 decision model from Convai Innovations. A state and typed questions go in; option probabilities come out of one forward pass. There is no generated string to parse. The multilingual checkpoint is the member of the family intended for languages other than English.

Checkpoint	Encoder	Parameters	Released context	Intended use
convaiinnovations/laya	ModernBERT-large	421M	512	English
convaiinnovations/laya-multilingual	mmBERT-base	322M	1,024, up to 8,192	100+ languages
convaiinnovations/laya-typed-decisions	ModernBERT-large	421M	1,024	typed-decision workflows

The original multilingual card reports its own MASSIVE, XNLI, speed, and long-document measurements, and these limits of the released checkpoint:

• it ships uncalibrated, and confidence stays high when accuracy does not
• it is weaker on English than convaiinnovations/laya
• zero-shot typed decisions are near chance until the model is fine-tuned
• choice questions should stay under about 20 options, because the options share a 256-token head budget
• score questions are the weakest type and have a measured position bias (laya issue 131)
• noul can under-report true on the released checkpoint (laya issue 156)

Those statements describe the original checkpoint. They are not measurements of this fine-tune, apart from the calibration result in the table above.

• Docs: https://nandhakishorm.github.io/laya/
• Family: https://huggingface.co/convaiinnovations/laya
• Code: https://github.com/NandhaKishorM/laya
• Package: https://pypi.org/project/laya/
• Demo: https://huggingface.co/spaces/convaiinnovations/laya-demo
• Encoder: https://huggingface.co/jhu-clsp/mmBERT-base
• Encoder paper: https://arxiv.org/abs/2509.06888

The original model is Apache 2.0, Convai Innovations. LICENSE in this folder is that license. NOTICE names the upstream works.

Data

Train, development, calibration, and test text are the Lithuanian MEDLINE-S1 translation:

https://huggingface.co/datasets/AIML-VU/MEDLINE-S1-LT

The parent English dataset is pritamdeka/MEDLINE-S1, introduced in Pritam Deka, From Retrieval to Typed Decisions: Calibrated System One Models from Biomedical Sentence Encoders, arXiv:2610.02486.

Titles and abstracts were translated with facebook/nllb-200-distilled-1.3B (eng_Latn to lit_Latn, revision 7be3e24664b38ce1cac29b8aeed6911aa0cf0576). Question ids, option keys, and gold labels are unchanged. 1,239 abstracts were dropped because the translation collapsed into repetition.

Licence

The weights are Apache 2.0, because they derive from convaiinnovations/laya-multilingual. Keep LICENSE and NOTICE with any redistribution. The encoder ancestry is jhu-clsp/mmBERT-base, whose card declares MIT.

The training text is not Apache. AIML-VU/MEDLINE-S1-LT offers its package under CC BY 4.0 and asks for attribution to Pritam Deka, arXiv:2610.02486, and to the NLLB translation model. Two limits from that card still apply:

• NLM does not own the abstracts. Copyright in the English text remains with the publishers or authors. NLM's terms apply to that text and to the translations, and NLM does not endorse this model.
• facebook/nllb-200-distilled-1.3B is CC BY-NC 4.0. Its card says the translation model is for research, not for production, medical text, or document translation.

Limits

• Measured on Lithuanian MEDLINE-S1-LT only. Not a general Lithuanian medical chatbot, and not a substitute for the original multilingual checkpoint.
• Not a medical device. Do not use the probabilities as a diagnosis, a screening result, or a treatment recommendation.
• score questions were not in the training data. Their temperature is still 1.0.
• The act/escalate head is part of the Laya output and was not the reported metric.
• Fifteen test decisions were truncated at 2,048 tokens.

Files

File	Role
model.safetensors	merged weights, 644 MB
rl_agent_config.json	Laya runtime config, including temperatures
encoder/config.json	mmBERT encoder config
tokenizer/tokenizer.json	tokenizer, 34 MB
tokenizer/tokenizer_config.json	tokenizer config
eval_results.json	before/after report
LICENSE	Apache 2.0
NOTICE	attribution
README.md	this model card

laya.load needs the weights, rl_agent_config.json, encoder/, and tokenizer/. There is no trainer/ directory and none should be added.

Citation

@misc{petkevicius2026laya-lt-med,
  title        = {laya-multilingual-lt-med: a Lithuanian MEDLINE decision model},
  author       = {Petkevi{\v{c}}ius, Linas},
  year         = {2026},
  howpublished = {Artificial Intelligence Methods Lab, Vilnius University},
  url          = {https://huggingface.co/AIML-VU/laya-multilingual-lt-med}
}

@misc{deka2026retrievaltypeddecisionscalibrated,
  title         = {From Retrieval to Typed Decisions: Calibrated System One Models from Biomedical Sentence Encoders},
  author        = {Pritam Deka},
  year          = {2026},
  eprint        = {2610.02486},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2610.02486}
}

@article{nllbteam2022language,
  title   = {No Language Left Behind: Scaling Human-Centered Machine Translation},
  author  = {{NLLB Team}},
  journal = {arXiv preprint arXiv:2207.04672},
  year    = {2022},
  url     = {https://arxiv.org/abs/2207.04672}
}

Also credit Convai Innovations, convaiinnovations/laya-multilingual, and the mmBERT encoder, https://arxiv.org/abs/2509.06888.

This project was funded by the European Union (project No S-ITP-26-11) under the agreement with the Research Council of Lithuania (LMTLT).