Key Error 'loss' while fine tuning GPT-2 with the Trainer utility

sebansunny
Thu, 17 Dec 2020 15:10:28 +0000

Input data is a text file with entries differentiated by a newline in the following sequence:
<|startoftext|> sentence1… <|endoftext|>
<|startoftext|> sentence2… <|endoftext|>
<|startoftext|> sentence3… <|endoftext|>

This is the input to the Trainer:

DatasetDict({
train: Dataset({
features: [‘attention_mask’, ‘input_ids’],
num_rows: 25
})
test: Dataset({
features: [‘attention_mask’, ‘input_ids’],
num_rows: 10
})
})