[Solved] Image dataset seems slow for larger image size

ydshieh
Thu, 16 Dec 2021 17:52:55 +0000

@lhoestq

I installed by pip install --upgrade .[dev], and fixed a minor issue of types:
it works now!!

You guys are amazing!! Thank you so much.

By the way, would you mind to tell me which PR did this optimization. I want to get some ideas about it! Thanks!

Average times_1 in data_loader: 0.12009784579277039

batch: 0 | time: 0.11190962791442871
batch: 1 | time: 0.14724326133728027
batch: 2 | time: 0.12398481369018555
batch: 3 | time: 0.12642168998718262
batch: 4 | time: 0.08752894401550293
batch: 5 | time: 0.09127330780029297
batch: 6 | time: 0.09010076522827148
batch: 7 | time: 0.09955096244812012
batch: 8 | time: 0.11309647560119629
batch: 9 | time: 0.09490156173706055
batch: 10 | time: 0.0921792984008789
batch: 11 | time: 0.10338354110717773
batch: 12 | time: 0.10463619232177734
batch: 13 | time: 0.11239099502563477
batch: 14 | time: 0.11689925193786621
batch: 15 | time: 0.10281705856323242
batch: 16 | time: 0.10169291496276855
batch: 17 | time: 0.15300273895263672
batch: 18 | time: 0.10869407653808594
batch: 19 | time: 0.08719658851623535
batch: 20 | time: 0.1176912784576416
batch: 21 | time: 0.13638067245483398
batch: 22 | time: 0.12096691131591797
batch: 23 | time: 0.08860659599304199
batch: 24 | time: 0.10601067543029785
batch: 25 | time: 0.11217999458312988
batch: 26 | time: 0.1530930995941162
batch: 27 | time: 0.14607572555541992
batch: 28 | time: 0.17520356178283691
batch: 29 | time: 0.2369546890258789
batch: 30 | time: 0.13903379440307617
batch: 31 | time: 0.14586281776428223