"Your honor, my archive of Linux ISOs were acquired under the pretense that they were 'publicly available' and the copyright holders didn't 'opt-out' using the 'up-for-grabs.txt' standard I invented."

Still much better, especially with respecting opt-outs, than most other LLMs

Legally, it seems it does, at least in the US and EU. I assume China too.

Whether or not it should is a different argument, but copyright is a legal framework, not an ethical one.

Yes. Although they don’t host the dataset binaries.

In case you're not aware, there are a decent number of open weight (and some open source) large language models.

The Ollama project makes it very approachable to download and use these models.

Links in the article. Hugging Face and Swiss Telecoms host

8b parameter models are relatively fast on 3rd gen RTX hardware with at least 8gigs of vram, CPU inferencing is slower and requires boatloads of ram but is doable on older hardware. These really aren't designed to run on consumer hardware, but the 8b model should do fine on relatively powerful consumer hardware.

If you have something that would've been a high end gaming rig 4 years ago, you're good.

If you wanna be more specific, check huggingface, they have charts. If you're using linux with nvidia hardware you'll be better off doing CPU inferencing.

Edit: Omg y'all I didn't think I needed to include my sources but this is quite literally a huge issue on nvidia. Nvidia works fine on linux but you're limited to whatever VRAM is on your video card, no RAM sharing. Y'all can disagree all you want but those are the facts. Thays why AMD and CPU inferencing are more reliable, and allow for higher context limits. They are not faster though.

Sources for nvidia stuff https://github.com/NVIDIA/open-gpu-kernel-modules/discussions/618

https://forums.developer.nvidia.com/t/shared-vram-on-linux-super-huge-problem/336867/

https://github.com/NVIDIA/open-gpu-kernel-modules/issues/758

https://forums.opensuse.org/t/is-anyone-getting-vram-backed-by-system-memory-with-nvidia-drivers/185902

For fastest inference, you want to fit the entire model in VRAM. Plus, you need a few GB extra for context.

Context means the text (+images, etc) it works on. That's the chat log, in the case of a chatbot, plus any texts you might want summarized/translated/ask questions about.

Models can be quantized, which is a kind of lossy compression. They get smaller but also dumber. As with JPGs, the quality loss is insignificant at first and absolutely worth it.

Inference can be split between GPU and CPU, substituting VRAM with normal RAM. Makes it slower, but you'll probably will still feel that it's smooth.

Basically, it's all trade-offs between quality, context size, and speed.

Please read the article before commenting.

"The model is named Apertus – Latin for “open” – highlighting its distinctive feature: the entire development process, including its architecture, model weights, and training data and recipes, is openly accessible and fully documented."

This https://github.com/swiss-ai/pretrain-data ?