Colab RAM crash error - Fine-tuning RoBERTa in Colab

rgwatwormhill
Tue, 15 Dec 2020 16:45:37 +0000

Hi, you could try reducing max-length.

For a bert-base model, I found that I needed to keep maxlen x batchsize below about 8192. I think that limit would be even lower for a bert-large model.

Do you need roberta-large, or would roberta-base be sufficient?
(Or even distilroberta-base)