Hi, I have created a custom dataset as csv file on a google sheet and the format is correct but when I uploaded it to hugging face the features answers says they are strings even though I copied exactly as per the original Squad dataset. In the original Squad dataset for the feature answers it says…

Perhaps this? [image] How to create dataset from CSV to training Question answering？ Beginners You should be able to parse the second pattern with: pd.read_csv(..., converters={"answers": lambda x: json.loads(x)})

Thank you, I’ll have a look. I just uploaded the file to github where I saw the “” aka string elements, so right now I am removing by hand the “” and will download the file again from github as CSV and see if it works , if not I will try the other post .

As a last resort, you can specify the data format directly, but it would be better if it could be recognized automatically. [image] Dataset features We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Thank you , very good I guess I will use that as I was unable to make the new CSV file working. It’s a bit odd that it’s being recognised as strings and not sequence but maybe that could be google sheets too and wasn’t sure how to make them a sequence in google sheet. Also I used a for loop to get t…

Problem with Hugging face customised SQuad dataset

Beginners

John6666 January 20, 2025, 4:15pm 2

Perhaps this?

Topic		Replies	Views
How to create dataset from CSV to training Question answering？ Beginners	6	2552	January 20, 2025
Question answering bot: fine-tuning with custom dataset Beginners	6	6189	June 23, 2022
CSV File Being Misinterpreted as Text in Hugging Face Dataset 🤗Datasets	1	75	December 27, 2024
Correct way to create a Dataset from a csv file Beginners	13	14631	March 25, 2022
Custom SQuAD2.0 dataset gives an error when using run_qa.py script 🤗Transformers	3	3512	July 30, 2021

Problem with Hugging face customised SQuad dataset

Related topics