Thanks, I added the checkout link to the OP
I got some weird specialised hardware over USB working via WinBoat. Might be an option for some.
Unfortunately, the AI community prefers rushed buggy development over proper, tested releases, so the quants and maybe the PR weren’t fully working.
As of 3 hours ago, unsloth was still updating their quants and guide. I don’t have time to test now but I wouldn’t judge the base model performance in the first few days when the bugs are still being worked out.
They also recommend some unconventional parameters in the Unsloth guide.
It could also be that the model is truly shit of course.
Edit I just took a look at the llama.cpp repo and there are still issues with the implementation as well.
cross-posted from: https://piefed.zip/c/fosai/p/958141/30b-a3b-glm-4-7-flash-released
Small/fast model with MIT license for local use.
Benchmarks look good for the size. But IMO these smaller models aren’t consistent enough to live up to their promises.
Small/fast model with MIT license for local use.
Benchmarks look good for the size. But IMO these smaller models aren’t consistent enough to live up to their promises.
What features are still missing after this gets merged? Right now, I still use a glitchy adapter that randomly drops out every few minutes and sometimes crashes my whole Hyprland WM due to an unsteady state when no displays are recognised/connected.
I would love to use my GPU and display with the features I paid for.
Can confirm that I can’t open piefed.zip from Voyager now. Lemmy.zip works fine.