132

This morning I woke up to my phone using mobile data and my home assistant automations not working. Initially I thought it the power was out, but I could turn on the lights just fine. I checked my UniFi app and saw that the server was not connected to the network at all. This meant that the cable got unplugged, the switch isn't working, or the server isn't working. It said the switch was connected and another device was connected to the switch so that narrows it down to just 2 cases. So I opened my server closet in the basement and immediately noticed something was wrong. I couldn't figure out what was wrong but I just felt like something was wrong. Everything was plugged in, the network switch lights were blinking like normal, my raspberry pi was running just fine, even the server indicator lights were on. My main server is an old gaming PC so it has a glass side panel so I looked inside and I could see the fan spinning, but I could not hear it. Usually I have it set to full speed and I can hear full speed very well. I tried rebooting the server with the power button and the fans didn't go to full speed. As a last resort, I brought down a keyboard and monitor. As soon as I plugged in the monitor, I saw that there was a prompt to set the time on the BIOS! Picture of the prompt In my opinion, this was the stupidest reason for an outage.

Further investigations

I dug a little deeper and discovered that the BIOS had been reset during a power outage right before all of this happened. So far I have consulted the motherboard manual and found absolutely nothing about this. After a bit of research, I think it could have been that the CMOS battery has died. This is a really simple fix but I don't have the replacement battery right now. This means that I will have the same exact issue after the next power outage unless I replace the battery.

Preventing this in the future

From what I can see, I just need to replace the CMOS battery. But this computer has been running for over 4 years, so what is stopping this from happening again around 2028? The most effective solution is going to be preventing power outages in the first place. This can be done using a battery backup or a standby generator. Standby generators will last longer during a power outage but are typically more expensive and harder to setup than a simple battery backup.

you are viewing a single comment's thread
view the rest of the comments
[-] dgriffith@aussie.zone 157 points 1 month ago* (last edited 1 month ago)
  1. Replace CMOS battery.
  2. Get small UPS.
  3. Discover that small UPS's fail regularly, usually with cooked batteries.
  4. Add maintenance routine for UPS battery.
  5. Begin to wonder if this is really worth it when the rest of the house has no power during an outage.
  6. Get small generator.
  7. Discover that small generators also need maintenance and exercise.
  8. Decide to get a whole house battery backup a-la Tesla Powerwall topped off by solar and a dedicated generator.
  9. Spend 15 years paying this off while wondering if the payback was really worth it, because you can count on one hand the number of extended power outages in that time.
  10. In the end times a roving band of thugs comes around and kills you and strips your house of valuable technology, leaving your homelab setup behind and - sadly - without power. Your dream of unlimited availability has all been for nought.

Conclusion: just replace the CMOS battery on a yearly basis during planned system downtime.

[-] mattgolsen@lemmy.ml 8 points 1 month ago

I'm somewhere between step 7 and 8.

[-] poVoq@slrpnk.net 3 points 1 month ago

Currently at step 9. Waiting for the roving bands of thugs to arrive 😅

[-] jjlinux@lemmy.ml 4 points 1 month ago

I scheduled them before you, so I'll let you know once they're done here (maybe).

load more comments (13 replies)
this post was submitted on 04 Aug 2024
132 points (94.6% liked)

Selfhosted

39143 readers
264 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS