我一直在考虑购买一台个人电脑，或者自己组装一台，用于在本地运行您自己的LLMs而无需订阅和与外部供应商共享数据。 我在想，对于小型家庭实验室服务器/个人电脑来说，何为有负担且性能足够的价格呢？ 我尝试了量化模型，它们表现相当不错，但通常在CPU上速度很慢，然而GPU价格昂贵且内存较少…也许将模型的一部分卸载至GPU可能是一种选择，但这样做就需要调整许多其他因素以适应最终的个人电脑建造。谢谢！

一个适用于运行您本地AI的价格实惠的硬件

I been considering to acquire a PC, or build one for running locally your own LLMs without needing to subscribe and share your data with external vendors. 
I was wondering what would be the affordable for small home lab server/pc to make it "good enough", whatever that could be. I tried quantised models and they work quite well, but usually quite slow on CPU, however GPUs have crazy prices and little RAM… Perhaps off loading part of models to GPU might be an option, but this so as many other factors need to be adjusted to the final PC build. Thx!

Tu znajdują się posty, które nie były przypisane do żadnej grupy 

这里是未分配到任何组的帖子。

ccFOUND 档案

ccFOUND Archiwum 

I been considering to acquire a PC, or build one for running locally your own LLMs without needing to subscribe and share your data with external vendors.  
I was wondering what would be the affordable for small home lab server/pc to make it "good enough", whatever that could be.

I tried quantised models and they work quite well, but usually quite slow on CPU, however GPUs have crazy prices and little RAM… Perhaps off loading part of models to GPU might be an option, but this so as many other factors need to be adjusted to the final PC build. Thx!