Whilst a 100W delta seems unlikelly, a 50W delta seems realistic as the kind of stuff you have in a NAS will use maybe 5W (about the same as a Raspberry PI, possibly less) whilst the typical desktop PC uses significantly more even outside graphics mode (part of the reason to use Linux in text mode only is exactly to try and save power there). It mainly depends on what the desktop was used for before: a "gaming PC" with a dedicated graphics card from an old enough generation (i.e. with HW from back before the manufactures of GPUs started competing on power usage) will use signiificantly more power than integrated graphics even in idle mode.

That said, making it a "home server" as you suggest makes a lot of sense - if that thing is an "All In One" server (media server, NAS, print server, torrent download server and so on) loaded with software of your choice (and hence stuff that respects your privacy and doesn't shove Ads in your face) it's probably a superior solution to getting those things as separate standalone devices, especially in the current era of enshittification.