HomeLab

228 readers

1 users here now

A homelab is a server or multiple server setup that resides in your home and where you host sevelra applications and virtualized systems for testing and developing

Its a sandbox environment where you can experience and break and fix things in with no repercussions while its down

This is a community where you can share, discuss, or post news relating to homelabs

founded 2 years ago

MODERATORS

anzo@programming.dev

Open weight models reviewed (wagtail.org)

submitted 3 days ago by anzo@programming.dev to c/homelab@programming.dev

2 comments fedilink hide all child comments

Apparently Gemma 4 (by deepmind) are very energy-efficient for their capabilities.

The post itself seems to recommend a company (neuralwatt.com) but it's unrelated to project wagtail (an open source CMS made with django) so I consider it legit, they're just vouching for this new approach that shows users their ecological footprint basically.

(Of course, running these at a homelab is possible. hence the relevance of this info for the community)

top 2 comments

sorted by: hot top controversial new old

[–] e0qdk@reddthat.com 4 points 3 days ago

If you're running on AMD hardware, drop the power cap to save energy. (e.g. by poking values into /sys/class/drm/card1/device/hwmon/.../power1_cap or similar; check the min and max files adjacent to it for the ranges.) I can lower the power usage by 100W on a discrete AMD GPU and only lose about ~3% tok/s generation (inference is limited more by memory bandwidth than by compute typically on modern hardware). Test different settings on your actual hardware to find the sweet spot balancing performance vs energy usage!

[–] troed@fedia.io 1 points 3 days ago

Yeah I run the Gemma 4 12B QAT on my homeserver. It has a 12GB GPU (Nvidia A2000) and that model gives the most bang for the buck being both fast and multimodal.