581
you are viewing a single comment's thread
view the rest of the comments
[-] deur@feddit.nl 36 points 4 months ago

The answer to your question is no, federation is not an appropriate model for internet scale search.

[-] Sigh_Bafanada@lemmy.world 5 points 4 months ago

Yeah I think you need a centralized system with decentralized ownership, so that no single party can fuck it up by themselves

[-] ben_dover@lemmy.world 2 points 4 months ago
[-] Sigh_Bafanada@lemmy.world 1 points 4 months ago

I mean yeah exactly

[-] avidamoeba@lemmy.ca 1 points 4 months ago

Yeah, decentralized ownership or democratic ownership would be another way to achieve this. A federated system even if possible would almost certainly be less efficient resource-wise.

[-] avidamoeba@lemmy.ca 1 points 4 months ago* (last edited 4 months ago)

Just to be clear, what I'm referring to here is that a search would occur on a single instance. E.g. searches on lemmy.world occur on the lemmy.world instance, and load lemmy.world's servers. The federated part is in the building the database on lemmy.world. E.g. a crawler or a user on lemmy.ca adds a new web site and that record is federated to lemmy.world to add to its database. Another user on feddit.de upvotes a search result and that upvote is federated to lemmy.world so that the search result shows higher for users searching on lemmy.world. In this kind of model individual search instances could in fact be very large based on their usage. If there's no limit to what's federated, that would put a lower bound on the size of instances. If there's a limit (something dumb like federate only search records for *.fr domains) then that would allow for smaller instances that don't have the compute and storage for the complete index.

this post was submitted on 21 Feb 2024
581 points (97.9% liked)

Technology

55919 readers
2650 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS