Saya telah mempertimbangkan untuk mendapatkan PC, atau membangun satu untuk menjalankan LLMs Anda sendiri tanpa perlu berlangganan dan berbagi data dengan vendor eksternal. Saya bertanya-tanya berapa biayanya untuk server/PC lab rumahan kecil agar menjadi "cukup baik", apa pun itu mungkin. Saya telah mencoba model kuantisasi dan mereka cukup baik, tetapi biasanya cukup lambat pada CPU, namun GPU memiliki harga yang sangat tinggi dan sedikit RAM... Mungkin memindahkan sebagian model ke GPU bisa menjadi pilihan, tetapi demikian banyak faktor lainnya perlu diatur untuk membangun PC akhir. Terima kasih!

Perangkat keras terjangkau untuk menjalankan AI lokal Anda.

I been considering to acquire a PC, or build one for running locally your own LLMs without needing to subscribe and share your data with external vendors. 
I was wondering what would be the affordable for small home lab server/pc to make it "good enough", whatever that could be. I tried quantised models and they work quite well, but usually quite slow on CPU, however GPUs have crazy prices and little RAM… Perhaps off loading part of models to GPU might be an option, but this so as many other factors need to be adjusted to the final PC build. Thx!

Tu znajdują się posty, które nie były przypisane do żadnej grupy 

"Di sini terdapat postingan-postingan yang belum ditetapkan ke dalam grup manapun"

"ccFOUND Arsip"

ccFOUND Archiwum 

I been considering to acquire a PC, or build one for running locally your own LLMs without needing to subscribe and share your data with external vendors.  
I was wondering what would be the affordable for small home lab server/pc to make it "good enough", whatever that could be.

I tried quantised models and they work quite well, but usually quite slow on CPU, however GPUs have crazy prices and little RAM… Perhaps off loading part of models to GPU might be an option, but this so as many other factors need to be adjusted to the final PC build. Thx!