I would like to buy a home server in this Black Friday. My primary needs is to host NextCloud, Immich.

Additionally, would be cool if I can run some 13B - 30B LLMs, train a model on some light-weight documents (ebooks, pdf files, etc), run pfsense (maybe I can buy a separated cheap one for this).

This server will runs 24/7, so I would also like to take into account power consumption, noise and space.

I have a few options below:

  • Lenovo IdeaCentre 3 i5-12/8GB/256GB stationær computer
  • LENOVO IDEACENTRE GAMING 5 17IAB7 (GTX 1650 SUPER) STATIONÆR PC, Intel® Core™ i5-12400F, NVIDIA GeForce GTX 1650 SUPER 4 GB (this is a compact-sized tower)
  • MEDION AKOYA S23005 I5 STATIONÆR PC, Intel® Core™ i5-1135G7, 8 GB RAM, 512 GB M.2 SSD
  • Intel NUC, NUC 11 Mini PC, Windows 10 Pro Desktop Mini Computer, Intel® Core i5-1135G7, 28W Intel Iris™ Xe Graphics, WiFi 6 | Bluetooth 5.2 | Thunderbolt 3 | 16GB RAM, 512GB PCIe SSD
  • Intel NUC, NUC11PAHi7 Desktop Mini PC, Intel® Core i7-1165G7, 32GB RAM, 1TB PCIe SSD, Support for 8K | WiFi 6 | 4K Quad Display | Bluetooth 5.2 | Thunderbolt 3 | Win 10 Pro

Would you give me some suggestions?

  • PermanentLiminality@alien.topB
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    I think the LLM need about half the parameters in vram so a 30b model needs a video card with 16gb of vram. The 13b models need 8gb of vram.

    Those aren’t cheap.