[-] RoundSparrow@lemmy.ml 18 points 1 year ago

Why is there a lack of gifs/videos on Lemmy?

Lemmy's internal data performance is so horribly slow and crash-causing that I think the last thing they want is even more popular data.

Video is simply the most superior type of media there is, and I think that not having easy access to it on Lemmy is hurting it.

Video is more data, popularity is more data. For whatever reason, at every turn, I've seen developers turn away from scaling options like Memcache, Redis, or just abandoning ORM data management and rewriting the data interfaces by hand....

since the sites on which the videos are hosted can track you.

That's already true for images that are hot linked routinely, so I don't think video really changes it.

I've been baffled since June why data and fixing lemmy's data coding hasn't been front and center. It's pretty wild to witness so many come to Lemmy and then turn away... Elon Musk has been flocking people, Reddit, etc. It's as if the project wants to make code that won't work on any data. It's baffeling.

[-] RoundSparrow@lemmy.ml 18 points 1 year ago

That feature you linked to is to flair users.... there is a different issue to flair posts: https://github.com/LemmyNet/lemmy/issues/317

3
submitted 1 year ago* (last edited 1 year ago) by RoundSparrow@lemmy.ml to c/lemmydev@lemm.ee

I'm starting lemmy-ui with:

LEMMY_UI_LEMMY_INTERNAL_HOST=lemmy-alpha:8541 node dist/js/server.js

Running against the drone-lemmy instances created by lemmy_server's api-tests scripts. I'm running latest main checkout on both projects.

My browser gives me:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://localhost:8536/api/v3/community/list?type_=Local&sort=TopMonth&limit=50&page=1. (Reason: CORS request did not succeed). Status code: (null).

api-tests scripts build dev mode, I thought CORS was turned off? I've played around with adding LEMMY_CORS_ORIGIN=* - but it doesn't help.

EDIT: reading this post closer, I see 8536 vs. 8541. If I manually refresh my browser against lemmy-ui on port 1234, it works... but I wonder where 8536 is getting picked up?

Thank you and I hope you are having a great weekend.

2

Right now, every single comment and post vote is federated and is a single row in PostgreSQL tables.

Each person and community has a home instance. They only accurate counts of total post, comments, and votes is on that home instance. And even then it is theoretically possible that to save disk space and/or improve performance, the home of a community could purge older data (or have loss of data).

For lemmy to lemmy, I think instead of federating the votes independent of the posts and comments, there could be sharing of aggregate data directly.

The model for this is how profiles of a person are federated. When a person revises their profile on their home instance, every other instance has to get the updated change. Such as a new image or revised bio. Same with the profile of a community is revised, such as changing image or sidebar of a community.

The code redesign could start out by making person and community aggregate count sharing part of those revisions. Those numbers are not that time-sensitive, the statistics of the number of users, posts, comments in a community could be behind by many hours without any real impact on the end-user experience of reading posts and comments on Lemmy.

With votes, posts it is more tricky. But some experiments could be done such as only sending post aggregates when a comment on that post is created, deleted, or edited.... and a more back-fill-oriented bulk operation take care of correcting and discovering out of sync information.

11
submitted 1 year ago by RoundSparrow@lemmy.ml to c/rust@lemmy.ml

In lemmy_server, at this code point: https://github.com/LemmyNet/lemmy/blob/91c024fd987b61ac8892b9e7d1896ee4574751da/crates/db_schema/src/impls/comment.rs#L62

How would I determine if it was an SQL UPDATE or INSERT and skip the remaining blocks of code that do work that does not need to be repeated when a comment edit (UPDATE) is being performed.

Thank you.

4

Secrets that we shared, mountains that we moved

114
submitted 1 year ago by RoundSparrow@lemmy.ml to c/memes@lemmy.ml
2
submitted 1 year ago* (last edited 1 year ago) by RoundSparrow@lemmy.ml to c/lemmyfederation@lemmy.ml

or federation.lemmy as root... allowing:
federation.kbin
federation.lemmy
etc.

And it emphasizes owner/operator/home of community, instance_name (subdomain/domain name).

5
10
submitted 1 year ago* (last edited 1 year ago) by RoundSparrow@lemmy.ml to c/rust@lemmy.ml

I haven't had this much trouble since Ada coding in 1986 when I was age 16.

Can someone please help me get the procedural syntax right?

https://github.com/LemmyNet/lemmy/pull/3805

3
4
submitted 1 year ago* (last edited 1 year ago) by RoundSparrow@lemmy.ml to c/lemmyperformance@lemmy.ml

This is the first post or comment in Lemmy history to say log_min_duration_statement ... ;)

It is possible to instruct PostgreSQL to log any query that takes over a certain amount of time, 2.5 seconds what I think would be a useful starting point. Minimizing the amount of logging activity to only those causing the most serious issues.

"Possibly the most generally useful log setting for troubleshooting performance, especially on a production server. Records only long-running queries for analysis; since these are often your "problem" queries, these are the most useful ones to know about. Used for pg_fouine." - https://postgresqlco.nf/doc/en/param/log_min_duration_statement/

I think it would really help if we could get lemmy.world or some other big site to turn on this logging and share it so we can try to better reproduce the performance overloads on development/testing systems. Thank you.

[-] RoundSparrow@lemmy.ml 15 points 1 year ago

I miss the days when the internet was populated largely by nerds aiming to make a better world

The BBS and early Internet days were dominated by people who read non-fiction books. RTFM was a common saying in those days.

does anyone else feel enslaved?

“Everything in our background has prepared us to know and resist a prison when the gates begin to close around us . . . But what if there are no cries of anguish to be heard? Who is prepared to take arms against a sea of amusements? To whom do we complain, and when, and in what tone of voice, when serious discourse dissolves into giggles? What is the antidote to a culture’s being drained by laughter?”
Neil Postman
Amusing Ourselves to Death

5
submitted 1 year ago* (last edited 1 year ago) by RoundSparrow@lemmy.ml to c/lemmycode@lemmy.ml

lemmy.world
lemmy.ml

See the difference? Can someone submit/locate an Issue and/or Pull Request? Any others?

2
submitted 1 year ago by RoundSparrow@lemmy.ml to c/vanlife@lemmy.ml
[-] RoundSparrow@lemmy.ml 17 points 1 year ago

On the technical topic of renaming a domain of a Lemmy server... I think it is worth experimenting with the code. At minimum, I think it should be an option to try and keep the same login/passwords for users from the old install of Lemmy. But even that could prove tricky if a particular domain changed underllying ownership more than once - and user@domain became rewritten by an entirely different person. I guess in the real-world people do often get mail for previous residence of a house.

My biggest concern is legality because Lemmy claims to support privacy. I honestly think it's a bad idea to claim privacy because you run into so many problems. If the user never knows that their lemmy instance changed names and can't find it again, etc. Especially on technical topics, 15+ years of having Reddit keep messages from deleted user accounts offered a lot of great search engine hits. With Lemmy, a person moving to a different instance and deleting their account, so much content is going to get black-hole in favor of 50 instances having copies of a meme post or trivial website link - and solid original content (often in comment discussions) gets removed.

[-] RoundSparrow@lemmy.ml 15 points 1 year ago

Up until early July, Lemmy was damned if you do, damned if you don't. Federation had massive performance overhead due to some bugs and each additional instance that went online and subscribed to the big 4 popular servers was causing an even worse load problem than if say 30 users had joined directly. Especially instances that wanted a fully populated All listing, that meant every single thing was being sent to the server even if nobody was really reading that stuff.

And things like searching for topic content are going to be pretty limited given these newer servers don't have much history.

The aftermath of this attempt to scale is that there is also likely a lot of duplicate data, conversations that are mostly repetitive and posts to the same topics. Let alone the bugs Lemmy has federating deletes and moderation removal that doesn't impact direct users on the main servers as much.

[-] RoundSparrow@lemmy.ml 17 points 1 year ago* (last edited 1 year ago)

Welcome to the Lemmy network.

There is a pretty steep learning curve to this, and there have been a lot of little tweaks to the install that it's hard to know for certain what is causing your problems.

Lemmy does not bring in messages automatically. You have to subscribe to each community from one user on your server. I see some of your listed communities have zero subscribers: https://globe.pub/communities?listingType=All&page=1

You also have many that do list 1 subscriber, did it ever go past "pending" for that one user to "joined" for the community?

Are you running on Ubuntu 22.04 server? How did you do the install?

[-] RoundSparrow@lemmy.ml 17 points 1 year ago

Another instance was hacked too: https://lemmy.blahaj.zone/

[-] RoundSparrow@lemmy.ml 16 points 1 year ago

The "Hot" sort topic:

[-] RoundSparrow@lemmy.ml 18 points 1 year ago

The JWT are likely a hot issue, already some Issues on GitHub about them not being revoked properly.

[-] RoundSparrow@lemmy.ml 16 points 1 year ago

There is no built-in “real-time” methods for admins via the UI to identify suspicious activity from their users, I am only able to fetch this data directly from the database. I don’t think it is even exposed through the rest api.

The people doing the development seem to have zero concern that their all the major servers are crashing with nginx 500 errors on their front page under routine moderate loads, nothing close to a major website. There is no concern to alert operators of internal federation failures, etc.

I am only able to fetch this data directly from the database.

I too had to resort to this, and published an open source tool - primitive and non-elegant, to try and get something out there for server operators: !lemmy_helper@lemmy.ml

[-] RoundSparrow@lemmy.ml 17 points 1 year ago

And I would like to see a federation-wide policy that all bots must be clearly identified as bots (an attribute on their account). And features in the site code to block all bots as a user preference.

view more: ‹ prev next ›

RoundSparrow

joined 1 year ago
MODERATOR OF