12
submitted 1 year ago* (last edited 1 year ago) by doostee@programming.dev to c/programming@programming.dev

From a technical and legal standpoint, ignoring ethics and dignity, is there anything preventing us from scripting a scraper that recreates reddit posts in a lemmy instance? Like maybe top 50 posts of the top 20 subreddits, without comments. I think it would help convince people to join, since the major argument for sticking with reddit is that it has more content. Thoughts?

you are viewing a single comment's thread
view the rest of the comments
[-] auv_guy@programming.dev 13 points 1 year ago

Why should someone switch when the content is the same but you cannot reach the OP? In fact you would need to go to reddit to reach the OP! I think this would drive people back to reddit.

[-] feylec@mander.xyz 4 points 1 year ago

One of Reddit's main pressure points for forcing reopening was that it was "unfair for the users" to keep data hidden and inaccessible. Mirroring all that data takes away some of the leverage. So I can understand the value... we can move and we don't have to worry about Reddit taking its ball and going home or claiming it deserves to be hostile because it is the steward of so much information.

[-] pixelpop3@programming.dev 2 points 1 year ago

Leverage for what purpose? To fix reddit? Let reddit die or not die.

Reddit has always come after mirrors and they will easily get courts to take down the instances. Don't forget that prior to the API change they came after pushshift.

Additionally, anyone mirroring reddit on the moral basis that the content is owned by the creators and reddit is an exploitative rentseeker, has an obligation to not become a rentseeker themselves. This means things like ensuring that content that users voluntarily delete is also deleted in the mirrors. Reddit in fact had a large battle with pushshift about this years ago such that pushshift supposedly now only keeps history of moderator and admin edits. I agree with that ethically.

And in many cases you may be legally required to do this. To be clear Reddit made pushshift change to respecting user delete requests because of legal exposure and compliance risks.

Not to mention that you don't really know that anyone intends their content to be mirrored on sites they do not use. Particularly now that Reddit seems to be forcing private subreddits to be open. There's no moral high ground for doing this.

this post was submitted on 28 Jun 2023
12 points (64.3% liked)

Programming

16968 readers
311 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev



founded 1 year ago
MODERATORS