class SQUAD(Dataset): def __init__(self): # Load our training dataset and tokenizer self.dataset = load_dataset("squad", split="train") self.encoded_context = self.dataset.map( convert_to_features_context, batched=True ) #self.dataset = self.da…

Fetching data takes too too much time

mariosasko June 13, 2022, 5:53pm 2

Answered here.

Topic		Replies	Views
Dataloader time problem on custom dataset based on huggingface Beginners	2	1094	June 14, 2022
Hugging face datasets -- reading image shape takes very long time Beginners	1	303	April 4, 2023
Why is it so slow to access data through iteration with hugginface dataset? Intermediate	2	2925	July 21, 2022
Slow Iteration speed (with and without keep_in_memory=True) 🤗Datasets	3	1512	March 14, 2023
Improve performance IterableDataset (with tokenization) 🤗Datasets	2	836	November 2, 2023