191
submitted 1 year ago by favrion@lemmy.ml to c/fediverse@lemmy.ml

Ugh.

you are viewing a single comment's thread
view the rest of the comments
[-] r00ty@kbin.life 1 points 1 year ago* (last edited 1 year ago)

I don't know that it's a DB design flaw if we're talking about federation messages to other instances inboxes (which created rows of that magnitude for updates does sound like federation messages outbound to me). Those need to be added somewhere. On kbin, if installed using the instructions as-is, we're using rabbitmq (but there is an option to write to db). But failures do end up hitting sql still and rabbit is still storing this on the drive. So unless you have a dedicated separate rabbitmq server it makes little difference in terms of hits to storage.

It's hard to avoid storing them somewhere, you need to be able to know when they've been sent or if there are temporary errors store them until they can be sent. There needs to be a way to recover from a crash/reboot/restart of services and handle other instances being offline for a short time.

EDIT: Just read the issue (it's linked a few comments down) it actually looks like a weird pgsql reaction to a trigger. Not based on the number of connected instances like I thought.

[-] RoundSparrow@lemmy.ml 1 points 1 year ago

(which created rows of that magnitude for updates does sound like federation messages outbound to me)

rows=1675 from lemmy.ca here: https://github.com/LemmyNet/lemmy/issues/3165#issuecomment-1646673946

It was not about outbound federation messages. It was about counting the number of comments and posts for the sidebar on the right of lemmy-ui to show statistics about the content. site_aggregates is about counting.

[-] r00ty@kbin.life 1 points 1 year ago

Yep I read through it in the end. Looks like they were applying changes to all rows in a table instead of just one on a trigger. The first part of my comment was based on reading comments here. I'd not seen the link to the issue at that stage. Hence the edit I made.

this post was submitted on 31 Jul 2023
191 points (89.0% liked)

Fediverse

17508 readers
103 users here now

A community dedicated to fediverse news and discussion.

Fediverse is a portmanteau of "federation" and "universe".

Getting started on Fediverse;

founded 4 years ago
MODERATORS