RoundSparrow

joined 2 years ago
MODERATOR OF
[–] RoundSparrow@lemmy.ml 2 points 2 years ago

I haven't looked around at alternatives.

Lemmy has a lot of front-end app development going on and I think that's one of the big strengths. The API can be bloated with a lot of duplicate data in JSON responses but it is usable.

[–] RoundSparrow@lemmy.ml 2 points 2 years ago* (last edited 2 years ago) (6 children)

Lemmy is pretty immature as code to actually run in production. It may be well over 4 years old, but the whole thing seems to have very little in the way of information that a server operator can look at to check the health and problems under the covers. It also doesn't deal with unrecognized data very well and hides a lot of errors in a log where the messages are often not very much of a hint what is going on.

Lemmy surely is unique, as I almost never see people using it actually criticize the code for quality assurance and testing. More often than not, I see people cheering and defending it. I've had to look through this experience and code as it is more run like an art project or a music band than any serious focus on data integrity or performance concern.

[–] RoundSparrow@lemmy.ml 2 points 2 years ago (4 children)

if it were me right now with Lemmy 0.18.4, I'd take the server offline, do a PostgreSQL dump file - keep a copy, then hand-edit the sequence numbers in the dump file - and do a restore.

you probably only had a few users, so I would set user to 100, person id can be higher because of federation - but jump ahead to 10000 maybe. Post and comment set ahead to 10000 ... and community set ahead to 10000 because that gets federated

the PostgreSQL sequence numbers should only get used on newly created objects here-forward.

[–] RoundSparrow@lemmy.ml 1 points 2 years ago* (last edited 2 years ago)

This feature is being added to another federated platform, Pixelfed, featured called 'groups' that kind of allows attributes. https://lemmy.ml/post/3702181

The community structure in Lemmy is the only federated entity I can think of that allows multiple editors, moderators can all edit the community or act to lock posts / feature posts / ban members - write activities - and the language settings could be a model for setting individual attributes on the community and mod abilities themselves.

[–] RoundSparrow@lemmy.ml 1 points 2 years ago (3 children)

Right now lemmy backend only supports a single 'subscribed' list, that would be like a music player only having a single playlist. I know people are using multiple logins just to have more flexibility in managing their lists.

[–] RoundSparrow@lemmy.ml 1 points 2 years ago (6 children)

GitHub issue about this topic: https://github.com/LemmyNet/lemmy/issues/3782

Some of it might be avoided by tweaking the PostgreSQL database with higher key values after the new install, but the whole situation isn't really planned for or recognized in the code

[–] RoundSparrow@lemmy.ml 1 points 2 years ago (8 children)

You really have blind faith that federation even works, when I've been validating data and highlighting that delivery is not reliable when it has so much overhead it crashes servers.

[–] RoundSparrow@lemmy.ml 14 points 2 years ago* (last edited 2 years ago) (3 children)

This basically shuts my idea down

it's not very difficult to modify the code for something like this.... and closing off registration wont' let anyone else login and create new content form your istance.

Personally the load on the major servers by having one more instance that subscribes to everything is why I think people should back off from creating more than the 1500 instances Lemmy network already has. Delivery of every single vote, comment, post 24 hours a day just so one person can read content for an hour or two a day.

That makes sense for email systems where all that content doesn't have to be sent, but for Lemmy it's a huge amount of overhead.

[–] RoundSparrow@lemmy.ml 2 points 2 years ago

ok, I'm going to delete this post. People actually aren't discussing privacy and are just debating if they think Lemmy needs Multi-Reddit. And I just want to get the code finished. I am probably moving ahead on the code with ZERO sharing of any existing data.

[–] RoundSparrow@lemmy.ml 0 points 2 years ago

dismissing client-side techniques as nonsense without seeming to understand why they were being discussed in the first place.

I'm the one who started a post about a server-side solution that entirely is based on Reddit's code for a server-side solution. YOU are the one coming along with this wild idea that a server change isn't needed at all. yet, you have not demonstrated this wild claim you made!

I’m not interested in any multireddit feature that reduces sub privacy. I’d consider it a net loss for lemmy.

It does NOT require it. I will repeat it: IT IS NOT REQUIRED! It is a sub-feature that facilities better openness that I am suggesting be added as part of the core feature I'm developing.

On Reddit, multi-reddits personal in nature.

10 years ago Reddit announced it as entirely not being personal! That sharing them was the whole point. I again question if you even understand what multi-reddit is!

[–] RoundSparrow@lemmy.ml 1 points 2 years ago

shouldn’t require relaxing privacy constraints in any case.

It isn't at all essential to the feature.

I have already coded it so that it does NOT require sharing of anyone's data, at all. No way shape or form. I'm proposing it as a discussion topic because it's easy to implement and goes along with the whole spirit of a public forum where people share their public stuff. That people might actually want an easy way to help others out...

But, it's easier for me just to avoid any privacy topic entirely and not allow sharing of anything. Just build the whole design with opt-in only empty list.

[–] RoundSparrow@lemmy.ml 2 points 2 years ago (2 children)

I’m suggesting that multireddits are a “local” function. Theu are so local that they’re possible without server-side support at all,

Again, how? If I want a blend of 50 different communities, how can Reddit or Lemmy do that without 50 API calls if you do not add server-side MultiReddit code?

50 API calls is the overhead and nonsense that is being avoided here....

 

Someone was asking about it in support, so I was curious to try it. The Rust code has HideCommunity, I even tried to bypass the JavaScript client and do a direct API call.

Does anyone see the function call to send a HideCommunity object?

Where it might be disabled in the Rust code? I know it goes all the way into the database and is on SELECT statements.

Thank you

 

"502 bad gateway"

 

Since I found out about my autism in 2009, at age 40, I struggled to find older people who I could relate to that had mental disorders.

I also feel like late-diagnosed autism is not well understood by the medical community. I keep witnessing more and more adults who are diagnosed even at age 60 (who work in science news reporting, of all fields!).

With autism, age is so focused on children in the medical research funding. But schizophrenia learning is often focused on adults: "symptoms generally start in the mid- to late 20s, though it can start later, up to the mid-30s. Schizophrenia is considered early onset when it starts before the age of 18."

It's an interesting time period in autism right now. Since I became active on the topic in 2009, I keep hearing story after story on social media about people not getting diagnosed with autism until age 40, 50, etc. Again: diagnosed with autism even at age 60

Terry A Davis and I have a lot in common: https://en.wikipedia.org/wiki/Terry_A._Davis

  1. I was born in 1969 too. Just a couple months older.

  2. I learned programming on a Commodore 64 and published social media software.

  3. He worked at "Ticketmaster as a programmer for VAX machines" - I had a VAX/VMS system at my high school and got a job with a pentagon contractor based on them also having a VAX system at age 16.

  4. I became an expert in operating systems and wrote books about them in the early 1990's.

  5. I have traveled and lived as a digital nomad since 1999, I've lived all over the USA states he was spending time in.

After 2017, he struggled with periods of homelessness and incarceration. In 2018, he was struck by a train and died at the age of 48.

I have never had hallucinations of any kind. And his interest in religion strikes me as someone who would have really benefited from a 1970 lecture by Joseph Campbell: Joseph Campbell - Inward Journey: Schizophrenia and Mythology

I've lived in the Middle East for years, lived in several Islamic nations, experienced different views of religion. Dr. John Weir Perry is a reference. As Joseph Campbell is pointing out in that lecture, religion itself is a form of Schizophrenia. I think Terry A Davis is exactly the kind of person who would have benefited from having that knowledge, and even translating it to Arizona (where he lived) areas of mythology - like Navajo teaching. I do not subscribe to any one religion as "correct" any more than films, songs, fiction. I think James Joyce combined with Marshall McLuhan and Neil Postman are an excellent set of teachers about how to understand mythology (religion) stories and not get caught up in the specifics of Middle East interpretations. The Navajo Pollen Path I consider better than what I've seen in most Bible/Torah/Quran teaching.

Terry A Davis's visions of inner voices also fits a lot with the fiction and meta writings of Phillip K. Dick. PKD was very good with metaphors and I think well understood what Joseph Campbell was saying about the psychology of religion. It would not surprise me at all if PKD knew of the 1970 lecture by Campbell in his lifetime.

 

For lemmy server testing and performance baseline measurement, I think it would be cool to have every API call exercised.

Anyone willing to create and share some JavaScript client code? Normally these are run with Jest via NodeJS - but we can make it an extra step to integrate into Jest. I'm just thinking someone doing front-end app work can do a well organized hit on every API surface.

You can skip the creation of a user if you want, that code is already in the testing

Probably ideal to organize moderator vs non-moderator.

Something like: edit profile with every option one at a time, create a community, edit it, create posts, edit, delete, undelete, reply, etc. Imagine you were doing interactive tests of a major upgrade and wanted to hit every feature and button.

Right now most of that testing is done in independent scripts, such as a user test: https://github.com/LemmyNet/lemmy/blob/main/api_tests/src/user.spec.ts

And you can see it only tests editing a profile is working, not that actual features change their behavior. Although I've started to add that for back-end behaviors like showing read/unread posts on the list. Front-end devs are the ones who know what end-users do and the fringe cases to look out for. Thank you.

 

I thought some people were out there in June creating stress-testing scripts, but I haven't seen anything materializing/showing results in recent weeks?

I think it would be useful to have an API client that establishes some baseline performance number that can be run before a new release of Lemmy and at least ensure there is no performance regression?

The biggest problem I have had since day 1 is not being able to reproduce the data that lemmy.ml has inside. There is a lot of older content stored that does not get replicated, etc.

The site_aggregates UPDATE statement lacking a WHERE clause and hitting 1500 rows (number of known Lemmy instances) of data instead of 1 row is exactly the kind of data-centered problem that has slipped through the cracks. That was generating a ton of extra PostgreSQL I/O for every new comment and post from a local user.

The difficult things to take on:

  1. Simulating 200 instances instead of just 5 that the current API testing code does. First, just to have 200 rows in many of the instance-specific tables so that local = false API calls are better exercised. And probably about 25 of those instances have a large number of remote subscribers to communities.

  2. async federation testing. The API testing in lemmy right now does immediate delivery with the API call so you don't get to find out the tricky cases of servers being unreachable.

  3. Bulk loading of data. On one hand it is good to exercise the API by inserting posts and comments one at a time, but maybe loading data directly into the PostgreSQL backend would speed up development and testing?

  4. The impact of scheduled jobs such as updates to certain aggregate data and post ranking for sorting. We may want to add special API feature for testing code to trigger these on-demand to stress test that concurrency with PostgreSQL isn't running into overloads.

  5. Historically, there have been changes to the PostgreSQL table layout and indexes (schema) with new versions of Lemmy, which can take significant time to execute on a production server with existing data. Some kind of expectation for server operators to know how long an upgrade can take to modify data.

  6. Searching on communities, posts, comments with significant amounts of data in PostgreSQL. Scanning content of large numbers of posts and comments can be done by users at any time.

  7. non-Lemmy federated content in database. Possible performance and code behavior that arises from Mastodon and other non-Lemmy interactions.

I don't think it would be a big deal if the test takes 30 minutes or even longer to run.

And I'll go out and say it: Is a large Lemmy server willing to offer a copy of their database for performance troubleshooting and testing? Lemmy.ca cloned their database last Sunday which lead to the discovery of site_aggregates UPDATE without WHERE problem. Maybe we can create a procedure of how to remove private messages and get a dump once a month from a big server to analyze possible causes of PostgreSQL overloads? This may be a faster path than building up from-scratch with new testing logic.

 

Autism brain can get latched on technical issues and have no awareness of just how much people with different attitudes about art and entertainment, like Rock Stars making music, do things different.

I've had to think of analogies of the situation. Like criticizing the maintenance cost or oil leak of an exotic sports car. And not realizing it's all about the 'hand crafted' nature and not about practicality.

On the Reddit adult autism community people have shared experiences about drinking. For some autistic people it helps, others say it causes them problems.

I haven't drank heavy since New Years, but I had to drink to reset my brain over my social interpretation mistake.

I just got too worked up over the style and fashion of a art-centered creative project. And I really regret how I didn't realize the social situation. I really regret that I worried about a problem taking so long to be fixed when I haven't found a single other person who cares about it. Stupid brain. Social things like this can be so confusing some times, I didn't even realize it.

Street vendor food. wildly different social conversions and standards people have I don't have troubles with those, but "Rock Star" culture in business/workplace and music/entertainment/art cultures can confuse me. I get bewildered why people would let Harvey Weinstein get away with so much for so many years. That kind of "Rock Star" culture and how people have different standards for what they consider normal.... bewildering. Rock Star politicians, I've never seen the appeal of wearing clothing with a politician on it. I have such a hard time seeing these situations, my mind doesn't go there. But I can be that way about actual rock stars, art, so the problem I am having I am just shifting my mind to interpret the whole situation that way.

But I feel so dumb and stupid for not recognizing it earlier.

 

I'm supposed to be able to live in the current era without spectrum of years

Many nations: Rent/Housing perspective/10 years ago?

 

Reference, June 4, 2023: https://github.com/LemmyNet/lemmy/issues/2910

Questions:

  1. Lemmy.ml was crashing every 10 minutes of every single day since May 25. Do you dispute this claim I make? Server log Evidence?

  2. June 4 issue 2910 was on Github and the same developers who run lemmy.ml put a tag on the issue June 4.

  3. Did it say server-crashing topic?

  4. Were you aware of the end-of-June Reddit API change?

  5. When was the issue fixed, date and who?

  6. Did you ask PostgreSQL communities on Lemmy for help? Which posts?

  7. Is lemmy.ml a developer run instance? Is this your idea of "supported" example of Lemmy's rust lemmy_server code?

Please explain. Like PostgreSQL EXPLAIN?

Was this what was given as an Issue 2910 to you on June 4, 2023?

 

IIRC, it was lemmy.ca full copy of live data that was used (copy made on development system, not live server - if I'm following). Shared Saturday July 22 on GitHub was this procedure:

...

Notable is the AUTO_EXPLAIN SQL activation statements:

LOAD 'auto_explain';
SET auto_explain.log_min_duration = 0;
SET auto_explain.log_analyze = true;
SET auto_explain.log_nested_statements = true;

This technique would be of great use for developers doing changes and study of PostgreSQL activity. Thank you!

view more: ‹ prev next ›