[Solved] Image dataset seems slow for larger image size

lhoestq
Thu, 16 Dec 2021 15:59:38 +0000

Hi ! We recently did some optimizations in datasets regarding the way we handle arrays.

Note that for vision datasets like cifar10, we’re adding support for efficiently loading the images directly as PIL images rather than keeping raw lists of integers. This will be available in the next version of datasets. The documentation is also on its way