dan

joined 3 years ago
[–] dan@upvote.au 1 points 1 day ago* (last edited 1 day ago) (1 children)

All the data gathered by Cambridge Analytica was gathered through the public API though, after users had consented to share it (by logging into a quiz app that requested the permissions). That's why the API is very locked down now, and the approval process to get any sort of data access is very strict.

The main issue was that they gathered data from people whose profiles were set to be visible only to friends. If someone logged into the quiz and granted permissions, their friends' data was also accessible via the API.

[–] dan@upvote.au 2 points 1 day ago* (last edited 1 day ago)

With your idea, you either have to list a local IP in your public DNS record, or highjack your local DNS to point to the local IP. Both feel inelegant

The DNS records for your internal servers don't have to be public - they can be only on an internal DNS server if you want to do that. Only the _acme-challenge subdomain has to be public. Let's Encrypt does follow CNAMEs.

And you have to give your NAS write access to your API key of your DNS registrar

You can use a separate DNS server just for Let's Encrypt, as it follows CNAMEs. I use acme-dns for this. Let's Encrypt supports IPv6-only DNS servers so I have my acme-dns instance listening on an IPv6 address in the /64 range on one of my VPSes.

[–] dan@upvote.au 7 points 1 day ago (1 children)

Makes sense - thanks.

[–] dan@upvote.au 13 points 1 day ago* (last edited 1 day ago) (4 children)

Debian is ready - as of Debian Trixie (released in August 2025), all software in the official repo is being compiled with 64-bit time. https://wiki.debian.org/ReleaseGoals/64bit-time

For your home NAS, I'd recommend using Let's Encrypt with Certbot. You can use it for internal systems, as long as you have a real domain name. Use DNS verification instead of HTTP. Renewal isn't an issue if it's entirely automated.

[–] dan@upvote.au 4 points 1 day ago (13 children)

Doesn't the water evaporate and become part of the water cycle? Water can't just disappear? Maybe I'm missing something.

It would be good to cut down water usage... Not just for data centers but also for things like lawns and golf courses.

[–] dan@upvote.au 8 points 1 day ago* (last edited 1 day ago) (3 children)

aggressively guard

tbh it's a hard balance for any social media company.

Guard content too little and you end up with Cambridge Analytica, which was literally because the public APIs allowed too much access (third-party apps could see any data through the API that you could see through your Facebook account, including friends profiles). You also end up with headlines talking about big data leaks which really just end up being compilations of public data (which has happened to both Facebook and LinkedIn).

Guard content too much and you restrict users' freedom too much.

[–] dan@upvote.au 7 points 1 day ago

It's not too bad if you use an outbound SMTP relay for sending. SMTP2Go is pretty good, and they have a free plan with 1000 emails per month. I use Mailcow and you can configure relays in their web UI, but it works just as well with the sender_dependent_relayhost_maps setting in Postfix.

Sure, it's not fully self-hosted, but the interesting part to self-host is the storage of your emails, not the sending (which will just relay through other SMTP servers along the way anyways).

[–] dan@upvote.au 1 points 2 days ago* (last edited 2 days ago)

That's probably my fault for not wording it well. I edited my comment to say "they sell plenty of them" instead.

[–] dan@upvote.au 1 points 2 days ago

In my experience, no large business would decide to only accept encrypted inbound SMTP

For submission (connections coming from users that have an account on the server) or for relay/target (connections coming from other email clients)? All email clients support encryption so I think requiring encryption for submission is reasonable. Server-to-server (port 25) can't have it enforced though, like you said.

SMTP is one of the worst protocols I have ever seen so widely used

It's from a era where everyone trusted everyone else. All connections were unencrypted, spam protection and rate limiting weren't needed, and security really wasn't on people's minds. Modern security and spam protection is hacky because it's built on top of protocols that weren't designed for it.

The other major issue with old protocols is that they're stateful. Modern protocols are mostly stateless since it's generally easier to deal with. They've also had more and more features hacked into them over time, so the specs are enormous.

There's been one major attempt at modernizing it: JMAP. It's stateless, uses JSON, and intends to replace both IMAP and SMTP. FastMail started the project. https://jmap.io/why-jmap/

However, they've only looked at the "easier" part to replace: Communication between a user and their email server. They're not looking to replace server-to-server communication at all.

[–] dan@upvote.au 3 points 2 days ago* (last edited 2 days ago) (1 children)

I've used that on automated systems. No need to worry about email quota and everything incoming is single-use input for other systems so there's no need to store messages on the mail server

You can do this with IMAP as well, you just need to delete and expunge the emails. Any good email client or library will have an option for this.

POP is literally just downloading all the emails, then deleting them. That's it.

You can't have a script that only downloads emails that match a search (for example emails from a particular sender), since POP doesn't support search. A basic use case is to check for bills/invoices from certain companies and import them into an accounting system, while leaving other emails untouched.

You can't receive emails in real-time and have to instead poll, since POP doesn't support real time notifications. IMAP supports IMAP IDLE.

If you have rules that filter emails into folders, you can't download them via POP, as it doesn't support folders.

For automated systems, if you don't want to store the emails, you can configure the email server to pipe the emails directly to a script. That way they're not stored at all, and your script gets them immediately rather than having to poll.

[–] dan@upvote.au 3 points 2 days ago

I used to use POP for some situations, but the protocol is extremely limited. I don't think there's anything POP can do that IMAP can't.

[–] dan@upvote.au 10 points 2 days ago (1 children)

Yeah, there's no risk of the mortgage falling through, and not as much dealing with banks. I don't really know the specifics but it was something I had to be aware of when buying my house. Luckily I was buying while it was a buyers market a few years ago, so prices were lower, fewer people were looking, and there weren't any competing all-cash offers.

84
submitted 1 week ago* (last edited 1 week ago) by dan@upvote.au to c/linux@lemmy.ml
 

I run dnstools.ws which lets you perform DNS lookups, pings, traceroutes, etc. from 25 locations around the world. Each location is powered by a VPS running Debian, running a C# service that's compiled to native code ahead-of-time using Native AOT. It uses ~60MB RAM.

Six of the the locations are powered by tiny "NAT VPSes" (native IPv6 with shared NAT IPv4) that only cost a few dollars a year, sponsored by various server providers. These usually have 256 MB RAM and 4-5 GB disk space.

This is great with OpenVZ and LXC. Since they're containers that share the kernel with the host, kernel memory doesn't count towards the container's memory limit. I'm using ~75 MB RAM on those systems: ~60MB for the DNSTools worker and ~15MB for everything else (sshd, systemd, cron, rsyslogd, and unattended-upgrades). Plenty of room left.

I also have a few KVM systems with 256 MB RAM. These are what I'm struggling with.

Debian 13 (Trixie) increased the minimum hardware requirements from 256 MB to 512 MB RAM. It seems like this is a hard requirement - When running on a system using 256 MB RAM, the installer complains about having too little RAM, and OOMs during the installation. Even with a successful installation (e.g. upgrading from bookworm to trixie), it kernel panics on boot: "System is deadlocked on memory".

I could try debootstrap to bootstrap a basic system, or Clonezilla to clone a working disk image over the network, but I think I'd hit the memory deadlock too.

Does Debian have smaller kernel images for VM environments, that use less RAM? Or should I just give up on Debian for this use case?

Does anyone have a recommendation for another distro I should use? I've been considering trying Alpine. C# does support compiling to use musl instead of glibc, so that's not an issue. I'm also not tightly-coupled to systemd and can get rid of it.

I can mount a custom ISO on the systems, so booting from an ISO isn't an issue.

Thanks!

Edit: Alpine looks very promising - no issue installing it and running my app on a 256MB VM. This is probably what I'll end up using.

 

I noticed that Spectacle has an option to upload to Imgur and Nextcloud. Is there a way to allow it to upload to an SFTP server?

Ideally I'd like for it to upload the file via SFTP then put the URL on my clipboard, which is what I do with ShareX on Windows.

 

I love Sentry, but it's very heavy. It runs close to 50 Docker containers, some of which use more than 1GB RAM each. I'm running it on a VPS with 10GB RAM and it barely fits on there. They used to say 8GB RAM is required but bumped it to 16GB RAM after I started using it.

It's built for large-scale deployments and has a nice scalable enterprise-ready design using things like Apache Kafka, but I just don't need that since all I'm using it for is tracking bugs in some relatively small C# and JavaScript projects, which may amount to a few hundred events per week if that. I don't use any of the fancier features in Sentry, like the live session recording / replay or the performance analytics.

I could move it to one of my 16GB or 24GB RAM systems, but instead I'm looking to evaluate some lighter-weight systems to replace it. What I need is:

  • Support for C# and JavaScript, including mapping stack traces to original source code using debug symbols for C# and source maps for JavaScript.
    • Ideally supports React component stack traces in JS.
  • Automatically group the same bugs together, if multiple people hit the same issue
    • See how many users are affected by a bug
  • Ignore particular errors
  • Mark a bug as "fixed in next release" and reopen it if it's logged again in a new release
  • Associate bugs with GitHub issues
  • Ideally supports login via OpenID Connect

Any suggestions?

Thanks!

 

On a small form factor PC with an i5-9500, Debian 12, 6.2.16 kernel, running Proxmox, powertop shows the following idle stats:

PowerTOP 2.14     Overview   Idle stats   Frequency stats   Device stats   Tunables   WakeUp


           Pkg(HW)  |            Core(HW) |            CPU(OS) 0
                    |                     | C0 active   2.8%
                    |                     | POLL        0.0%    0.0 ms
                    |                     | C1          1.1%    0.4 ms
C2 (pc2)    7.2%    |                     |
C3 (pc3)    5.5%    | C3 (cc3)    0.0%    | C3          0.1%    0.1 ms
C6 (pc6)    1.5%    | C6 (cc6)    1.9%    | C6          2.2%    0.6 ms
C7 (pc7)   75.2%    | C7 (cc7)   92.8%    | C7s         0.0%    0.0 ms
C8 (pc8)    0.0%    |                     | C8         21.5%    2.5 ms
C9 (pc9)    0.0%    |                     | C9          0.0%    0.0 ms
C10 (pc10)  0.0%    |                     |
                    |                     | C10        72.8%   12.5 ms
                    |                     | C1E         0.4%    0.2 ms

                    |            Core(HW) |            CPU(OS) 1
                    |                     | C0 active   1.4%
                    |                     | POLL        0.0%    0.0 ms
                    |                     | C1          0.7%    0.9 ms
                    |                     |
                    | C3 (cc3)    0.1%    | C3          0.1%    0.2 ms
                    | C6 (cc6)    1.0%    | C6          1.1%    0.8 ms
                    | C7 (cc7)   96.3%    | C7s         0.0%    0.0 ms
                    |                     | C8         18.9%    2.9 ms
                    |                     | C9          0.0%    0.0 ms
                    |                     |
                    |                     | C10        78.3%   24.8 ms
                    |                     | C1E         0.0%    0.0 ms
...

On a custom-built server with an i5-13500, Asus Pro WS W680M-ACE SE motherboard, Unraid (which uses Slackware), 6.1.38 kernel, it shows the following output:

PowerTOP 2.15     Overview   Idle stats   Frequency stats   Device stats   Tunables   WakeUp


           Pkg(HW)  |            Core(HW) |            CPU(OS) 0   CPU(OS) 1
                    |                     | C0 active   5.9%        0.9%
                    |                     | POLL        0.1%    0.0 ms  0.0%    0.0 ms
                    |                     | C1_ACPI    14.2%    0.2 ms  1.0%    0.1 ms
C2 (pc2)    0.0%    |                     | C2_ACPI    39.2%    0.8 ms 27.0%    0.9 ms
C3 (pc3)    0.0%    | C3 (cc3)    0.0%    | C3_ACPI    33.6%    1.2 ms 69.7%    3.0 ms
C6 (pc6)    0.0%    | C6 (cc6)    1.1%    |
C7 (pc7)    0.0%    | C7 (cc7)    0.0%    |
C8 (pc8)    0.0%    |                     |
C9 (pc9)    0.0%    |                     |
C10 (pc10)  0.0%    |                     |

                    |            Core(HW) |            CPU(OS) 2   CPU(OS) 3
                    |                     | C0 active  10.4%        0.5%
                    |                     | POLL        0.0%    0.0 ms  0.0%    0.0 ms
                    |                     | C1_ACPI    17.4%    0.2 ms  0.4%    0.2 ms
                    |                     | C2_ACPI    14.3%    0.8 ms  4.9%    0.6 ms
                    | C3 (cc3)    0.0%    | C3_ACPI    41.8%    5.4 ms 93.5%    5.5 ms
                    | C6 (cc6)    5.9%    |
                    | C7 (cc7)   26.7%    |
                    |                     |
                    |                     |
                    |                     |

                    |            Core(HW) |            CPU(OS) 4   CPU(OS) 5
                    |                     | C0 active  11.7%        0.2%
                    |                     | POLL        0.0%    0.1 ms  0.0%    0.0 ms
                    |                     | C1_ACPI    19.0%    0.1 ms  0.0%    0.0 ms
                    |                     | C2_ACPI    11.3%    0.7 ms  0.0%    0.0 ms
                    | C3 (cc3)    0.0%    | C3_ACPI    39.6%    7.7 ms 99.6%    7.0 ms
                    | C6 (cc6)    1.3%    |
                    | C7 (cc7)   25.4%    |
...

Both systems have C-states enabled in the BIOS.

I have a few questions I'm hoping someone can help with:

  • Why does the older system show more C-states in the right-most "CPU(OS)" column?
  • What does it mean when they're suffixed with "_ACPI" like in the output from the new system?
  • How do I debug the new system not hitting any CPU package C-states?

I can't find any documentation about this, neither on the man page nor on Intel's site (the official powertop URL https://01.org/powertop doesn't go anywhere useful any more).

Thanks!

 

Google Analytics is broken on a bunch of my sites thanks to the GA4 migration. Since I have to update everything anyways, I'm looking at the possibility of replacing Google Analytics with something I self-host that's more privacy-focused.

I've tried Plausible, Umami and Swetrix (the latter of which I like the most). They're all very lightweight and most are pretty efficient due to their use of a column-oriented database (Clickhouse) for storing the analytics data - makes way more sense than a row-oriented database like MySQL for this use case.

However, these systems are all cookie-less. This is usually fine, however one of my sites is commonly used in schools on their computers. Cookieless analytics works by tracking sessions based on IP address and user-agent, so in places like schools with one external IP and the same browser on every computer, it just looks like one user in the analytics. I'd like to know the actual number of users.

I'm looking for a similarly lightweight analytics system that does use cookies (first-party cookies only) to handle this particular use case. Does anyone know of one?

Thanks!

Edit: it doesn't have to actually be a cookie - just being able to explicitly specify a session ID instead of inferring one based on IP and user-agent would suffice.

 

I'm replacing an SFF PC (HP ProDesk 600 G5 SFF) I'm using as a server with a larger one that'll function as a server and a NAS, and all I want is a case that would have been commonplace 10-15 years ago:

  • Fits an ATX motherboard.
  • Fits at least 4-5 hard drives.
  • Is okay sitting on its side instead of upright (or even better, is built to be horizontal) since it'll be sitting on a wire shelving unit (replacing the SFF PC here: https://upvote.au/post/11946)
  • No glass side panel, since it'll be sitting horizontally.
  • Ideally space for a fan on the left panel

It seems like cases like this are hard to find these days. The two I see recommended are the Fractal Design Define R5 and the Cooler Master N400, both of which are quite old. The Streacom F12C was really nice but it's long gone now, having been discontinued many years ago.

Unfortunately I don't have enough depth for a full-depth rackmount server; I've got a very shallow rack just for networking equipment.

Does anyone have recommendations for any cases that fit these requirements?

My desktop PC has a Fractal Design Define R4 that I bought close to 10 years ago... I'm tempted to just buy a new case for it and repurpose the Define R4 for the server.

 

Sorry for the long post. tl;dr: I've already got a small home server and need more storage. Do I replace an existing server with one that has more hard drive bays, or do I get a separate NAS device?


I've got some storage VPSes "in the cloud":

  • 10TB disk / 2GB RAM with HostHatch in LA
  • 100GB NVMe / 16GB RAM with HostHatch in LA
  • 3.5TB disk / 2GB RAM with Servarica in Canada

The 10TB VPS has various files on it - offsite storage of alert clips from my cameras, photos, music (which I use with Plex on the NVMe VPS via NFS), other miscellaneous files (using Seafile), backups from all my other VPSes, etc. The 3.5TB one is for a backup of the most important files from that.

The issue I have with the VPSes is that since they're shared servers, there's limits in terms of how much CPU I can use. For example, I want to run PhotoStructure for all my photos, but it needs to analyze all the files initially. I limit Plex to maximum 50% of one CPU, but limiting things like PhotoStructure would make them way slower.

I've had these for a few years. I got them when I had an apartment with no space for a NAS, expensive power, and unreliable Comcast internet. Times change... Now I've got a house with space for home servers, solar panels so running a server is "free", and 10Gbps symmetric internet thanks to a local ISP, Sonic.

Currently, at home I've got one server: A HP ProDesk SFF PC with a Core i5-9500, 32GB RAM, 1TB NVMe, and a single 14TB WD Purple Pro drive. It records my security cameras (using Blue Iris) and runs home automation stuff (Home Assistant, etc). It pulls around 41 watts with its regular load: 3 VMs, ~12% CPU usage, constant ~34Mbps traffic from the security cameras, all being written to disk.

So, I want to move a lot of these files from the 10TB VPS into my house. 10TB is a good amount of space for me, maybe in RAID5 or whatever is recommended instead these days. I'd keep the 10TB VPS for offsite backups and camera alerts, and cancel the other two.

Trying to work out the best approach:

  1. Buy a NAS. Something like a QNAP TS-464 or Synology DS923+. Ideally 10GbE since my network and internet connection are both 10Gbps.
  2. Replace my current server with a bigger one. I'm happy with my current one; all I really need is something with more hard drive bays. The SFF PC only has a single drive bay, its motherboard only has a single 6Gbps SATA port, and the only PCIe slots are taken by a 10Gbps network adapter and a Google Coral TPU.
  3. Build a NAS PC and use it alongside my current server. TrueNAS seems interesting now that they have a Linux version (TrueNAS Scale). Unraid looks nice too.

Any thoughts? I'm leaning towards option 2 since it'll use less space and power compared to having two separate systems, but maybe I should keep security camera stuff separate? Not sure.

 

I have a 10Gbps internet connection. On a system with a 10Gbps Ethernet card, I can get ~8Gbps down and ~6Gbps up:

I'd expect this to easily max out a 2.5Gbps network connection. However, while the upload is maxed (or close to it), I can only ever get ~1.0 to 1.5Gbps down:

Both tests were performed on the same system. The only difference is that the first one uses a TRENDnet 10Gbps PCIe network card (which uses an Aquantia AQC107 chipset) whereas the second one uses the onboard NIC on my motherboard (Intel I225-V chipset).

This is consistent across two devices that have 10Gbps ports and two devices that have 2.5Gbps ports.

I'm using an AdTran 622v ONT provided by my internet provider, a TP-Link ER8411 router, and a MikroTik CRS312-4C+8XG-RM switch. I'm using CAT6 cabling, except for the connection between the router and the switch which uses an SFP+ DAC cable.

I haven't been able to figure it out. The 'slower' speeds are still great, I just don't understand why it can't achieve more than 1.5Gbps down over a 2.5Gbps network connection.

Any ideas?

67
submitted 2 years ago* (last edited 2 years ago) by dan@upvote.au to c/selfhosted@lemmy.world
 

I couldn't find a "Home Networking" community, so this seemed like the best place to post :)

My house has this small closet in the hallway and thought it'd make a perfect place to put networking equipment. I got an electrician to install power outlets in it, ran some CAT6 myself (through the wall, down into the crawlspace, to several rooms), and now I finally have a proper networking setup that isn't just cables running across the floor.

The rack is a basic StarTech two-post rack (https://www.amazon.com/gp/product/B001U14MO8/) and the shelving unit is an AmazonBasics one that ended up perfectly fitting the space (https://www.amazon.com/gp/product/B09W2X5Y8F/).

In the rack, from top to bottom (prices in US dollars):

  • TP-Link ER8411 10Gbps router. My main complaint about it is that the eight 'RJ45' ports are all Gigabit, and there's only two 10Gbps ports (one SFP+ for WAN, and one SFP+ for LAN). It can definitely reach 10Gbps NAT throughput though. $350
  • Wiitek SFP+ to RJ45 module for connecting Sonic's ONT (which only has an RJ45 port), and 10Gtek SFP+ DAC cable to connect router to switch.
  • MikroTik CRS312-4C+8XG-RM managed switch (runs RouterOS). 12 x 10Gbps ports. I bought it online from Europe, so it ended up being ~$520 all-in, including shipping.
  • Cable Matters 24-port keystone patch panel.
  • TP-Link TL-SG1218MPE 16-port Gigabit PoE switch. 250 W PoE power budget. Used for security cameras - three cameras installed so far.
  • Tripp Lite 14 outlet PDU.

Other stuff:

  • AdTran 622v ONT provided by my internet provider (Sonic), mounted to the wall.
  • HP ProDesk 600 G5 SFF PC with Core i5-9500. Using it for a home server running Home Assistant, Blue Iris, Node-RED, Zigbee2MQTT, and a few other things. Bought it off eBay for $200.
    • Sonoff Zigbee dongle plugged in to the front USB port
  • (next to the PC) Raspberry Pi 4B with SATA SSD plugged in to it. Not doing anything at the moment, as I migrated everything to the PC.
  • (not pictured) Wireless access point is just a basic Netgear one I bought from Costco a few years ago. It's sitting on the top shelf. I'm going to replace it with a TP-Link Omada ceiling-mounted one once their wifi 7 access points have been released.

Speed test: https://www.speedtest.net/my-result/d/3740ce8b-bba5-486f-9aad-beb187bd1cdc

Edit: Sorry, I don't know why the image is rotated :/ The file looks fine on my computer.

 

Hi!

I just created a Lemmy server at https://upvote.au/ for my personal use. I created a test community with a test post, but searching for it in Mastodon doesn't work. I tried searching for both @dan@upvote.au and @[!dan@upvote.au](/c/dan@upvote.au). I see the requests in the Nginx log:

172.19.0.5 - - [13/Jun/2023:22:57:06 -0700] "GET /.well-known/webfinger?resource=acct:test@upvote.au HTTP/1.1" 200 312 "-" "http.rb/5.1.1 (Mastodon/4.1.2; +https://toot.d.sb/)"
172.19.0.5 - - [13/Jun/2023:22:57:06 -0700] "GET /c/test HTTP/1.1" 200 10033 "-" "http.rb/5.1.1 (Mastodon/4.1.2; +https://toot.d.sb/)"

However, no results appear in Mastodon.

Any ideas?

view more: next ›