11
Is there a good ai thats run fast on a amd rx 7600? (feddit.rocks)
submitted 5 months ago* (last edited 5 months ago) by mariah@feddit.rocks to c/fosai@lemmy.world
8 comments fedilink hide all child comments

Ive been playing koboldai horde but the queue annoys me. I want a nsfw ai for playing on tavernai chat

you are viewing a single comment's thread
view the rest of the comments
[-] projectmoon@lemm.ee 1 points 4 months ago

How much speed are you actually getting on Mixtral (I assume that's the 8x7b). I have 64 GB of RAM and an AMD RX 6800 XT with 16 GB of VRAM. I get like 4 tokens per second with Q5_K_M quant.

this post was submitted on 04 Jun 2024
11 points (100.0% liked)

Free Open-Source Artificial Intelligence

2889 readers
1 users here now

Welcome to Free Open-Source Artificial Intelligence!

We are a community dedicated to forwarding the availability and access to:

Free Open Source Artificial Intelligence (F.O.S.A.I.)

More AI Communities

LLM Leaderboards

Developer Resources

GitHub Projects

FOSAI Time Capsule

founded 1 year ago
MODERATORS
Blaed@lemmy.world
fosai@lemmy.world