Problem: In GPT4All, under Settings > Application Settings > Device, I've selected my AMD graphics card, but I'm seeing no improvement over CPU performance. In both cases (AMD graphics card or CPU), it crawls along at about 4-5 tokens per second. The interaction in the screenshot below took 174 seconds to generate the response.

Question: Do I have to use a specific model to benefit from this advancement? Do I need to install a different AMD driver? What steps can I take to troubleshoot this?

Sorry if this is an obvious question. Sometimes I feel like the answer is right in front of me, but I'm unsure of which key words from the documentation should jump out at me.

My system info:

GPU: Radeon RX 6750 XT
CPU: Ryzen 7 5800X3D processor
RAM: 32 GB @ 3200 MHz
OS: Linux Bazzite
I've installed GPT4All as a flatpak

you are viewing a single comment's thread
view the rest of the comments

[–] yo_scottie_oh@lemmy.ml 2 points 1 month ago (1 children)

I wonder why ROCm 6.4 doesn’t support you, but ROCm 6.3 does.

Wait, where are you reading that 6.3 supports the 6950 XT? I dug up the System Requirements (Linux) page for 6.3 and it lists the same cards as the 6.4 page. Is there another document out there that covers this topic?

[–] pebbles@sh.itjust.works 4 points 1 month ago* (last edited 1 month ago) (1 children)

Okay I rechecked and it looks like 6.4 and 6.3 have similar compatibility/incompatibility with certain cards.

Here are the gfx versions of different amd cards:

https://rocm.docs.amd.com/en/develop/reference/gpu-arch-specs.html

Here are the supported versions of 6.4

https://rocm.docs.amd.com/en/docs-6.4.0/compatibility/compatibility-matrix.html

So given this extra bit of research it looks like you may be able to run ROCm on a 6950XT but I'm not sure about a 6750XT.

From my experience ROCm supports more than they say they do. They say they support the cards they've tested, but other's still may work. I was running ROCm on my 7900 XTX before they officially supported it.

[–] yo_scottie_oh@lemmy.ml 2 points 1 month ago

I probably would not have noticed that. I'll have to look into this some more. Thanks for all your help.