IndexError: Invalid key: 16 is out of bounds for size 0

envizzion
Thu, 29 Jun 2023 04:04:28 +0000

i’m getting the same issue . with custom dataset .can anyone explain the what this function does here exactly _check_valid_index_key.

i’m running the example from one of the huggingface blogpost

Colab notebook to fine-tune Falcon-7B on Guanaco dataset using 4bit and PEFT

the dataset is custom and looks like the following when loaded via Dataset class

print(dataset[0])
{'text': "### Human: What is Sarina Landcare Catchment Management Association's ABN? ### Assistant: 75953668479."}

following is the error trace .

│ in <cell line: 3>:3                                                                              │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer.py:1645 in train                    │
│                                                                                                  │
│   1642 │   │   inner_training_loop = find_executable_batch_size(                                 │
│   1643 │   │   │   self._inner_training_loop, self._train_batch_size, args.auto_find_batch_size  │
│   1644 │   │   )                                                                                 │
│ ❱ 1645 │   │   return inner_training_loop(                                                       │
│   1646 │   │   │   args=args,                                                                    │
│   1647 │   │   │   resume_from_checkpoint=resume_from_checkpoint,                                │
│   1648 │   │   │   trial=trial,                                                                  │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer.py:1659 in _inner_training_loop     │
│                                                                                                  │
│   1656 │   │   self._train_batch_size = batch_size                                               │
│   1657 │   │   logger.debug(f"Currently training with a batch size of: {self._train_batch_size}  │
│   1658 │   │   # Data loader and number of training steps                                        │
│ ❱ 1659 │   │   train_dataloader = self.get_train_dataloader()                                    │
│   1660 │   │                                                                                     │
│   1661 │   │   # Setting up training control variables:                                          │
│   1662 │   │   # number of training epochs: num_train_epochs                                     │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer.py:919 in get_train_dataloader      │
│                                                                                                  │
│    916 │   │   │   │   pin_memory=self.args.dataloader_pin_memory,                               │
│    917 │   │   │   )                                                                             │
│    918 │   │                                                                                     │
│ ❱  919 │   │   train_sampler = self._get_train_sampler()                                         │
│    920 │   │                                                                                     │
│    921 │   │   return DataLoader(                                                                │
│    922 │   │   │   train_dataset,                                                                │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer.py:841 in _get_train_sampler        │
│                                                                                                  │
│    838 │   │   │   │   lengths = None                                                            │
│    839 │   │   │   model_input_name = self.tokenizer.model_input_names[0] if self.tokenizer is   │
│    840 │   │   │   if self.args.world_size <= 1:                                                 │
│ ❱  841 │   │   │   │   return LengthGroupedSampler(                                              │
│    842 │   │   │   │   │   self.args.train_batch_size * self.args.gradient_accumulation_steps,   │
│    843 │   │   │   │   │   dataset=self.train_dataset,                                           │
│    844 │   │   │   │   │   lengths=lengths,                                                      │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer_pt_utils.py:571 in __init__         │
│                                                                                                  │
│    568 │   │   if lengths is None:                                                               │
│    569 │   │   │   model_input_name = model_input_name if model_input_name is not None else "in  │
│    570 │   │   │   if (                                                                          │
│ ❱  571 │   │   │   │   not (isinstance(dataset[0], dict) or isinstance(dataset[0], BatchEncodin  │
│    572 │   │   │   │   or model_input_name not in dataset[0]                                     │
│    573 │   │   │   ):                                                                            │
│    574 │   │   │   │   raise ValueError(                                                         │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/datasets/arrow_dataset.py:2792 in __getitem__            │
│                                                                                                  │
│   2789 │                                                                                         │
│   2790 │   def __getitem__(self, key):  # noqa: F811                                             │
│   2791 │   │   """Can be used to index columns (by string names) or rows (by integer index or i  │
│ ❱ 2792 │   │   return self._getitem(key)                                                         │
│   2793 │                                                                                         │
│   2794 │   def __getitems__(self, keys: List) -> List:                                           │
│   2795 │   │   """Can be used to get a batch using a list of integers indices."""                │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/datasets/arrow_dataset.py:2776 in _getitem               │
│                                                                                                  │
│   2773 │   │   format_kwargs = kwargs["format_kwargs"] if "format_kwargs" in kwargs else self._  │
│   2774 │   │   format_kwargs = format_kwargs if format_kwargs is not None else {}                │
│   2775 │   │   formatter = get_formatter(format_type, features=self._info.features, **format_kw  │
│ ❱ 2776 │   │   pa_subtable = query_table(self._data, key, indices=self._indices if self._indice  │
│   2777 │   │   formatted_output = format_table(                                                  │
│   2778 │   │   │   pa_subtable, key, formatter=formatter, format_columns=format_columns, output  │
│   2779 │   │   )                                                                                 │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/datasets/formatting/formatting.py:583 in query_table     │
│                                                                                                  │
│   580 │   │   _check_valid_column_key(key, table.column_names)                                   │
│   581 │   else:                                                                                  │
│   582 │   │   size = indices.num_rows if indices is not None else table.num_rows                 │
│ ❱ 583 │   │   _check_valid_index_key(key, size)                                                  │
│   584 │   # Query the main table                                                                 │
│   585 │   if indices is None:                                                                    │
│   586 │   │   pa_subtable = _query_table(table, key)                                             │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/datasets/formatting/formatting.py:526 in                 │
│ _check_valid_index_key                                                                           │
│                                                                                                  │
│   523 def _check_valid_index_key(key: Union[int, slice, range, Iterable], size: int) -> None:    │
│   524 │   if isinstance(key, int):                                                               │
│   525 │   │   if (key < 0 and key + size < 0) or (key >= size):                                  │
│ ❱ 526 │   │   │   raise IndexError(f"Invalid key: {key} is out of bounds for size {size}")       │
│   527 │   │   return                                                                             │
│   528 │   elif isinstance(key, slice):                                                           │
│   529 │   │   pass                                                                               │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
IndexError: Invalid key: 0 is out of bounds for size 0.