AMD GPU for local inference

ChiefScientist
Sat, 30 Aug 2025 16:47:39 +0000

I’m considering buying a graphics card and besides using it for games, I want to use it for some local AI experiments, specifically using the transformers and diffusers libraries with HuggingFace models. I currently have an AMD RX 5600XT which does not seem to be supported by rocm. I was considering mainly AMD cards because of their compatibility with games and Linux.

If I were considering another AMD card like a 7900XT 24GB or a 9070 16GB, what would that make available to me in terms of AI experiments? Which models would I be able to use, even if it’s a bit slow, and which models would still be off limits?

thanks for any advice!