782
submitted 1 year ago* (last edited 1 year ago) by rcmaehl@lemmy.world to c/fediverse@lemmy.world

Hi all,

If you're just now signing in for the first time in 12+ hours, you may just now be finding out that Lemmy World and other instances where hijacked. The hijackers had the full abilities of hijacked user, mod, and admin accounts. At this time, I am only aware of instance defacing and URL redirections to have been done by the hijackers.

If you were not forced to sign back in this morning, contact your instance admin to verify mitigations were completed on your instance.

How?

This occurred due to an XSS attack in the recently added custom emojis. Instance admins should follow the issue tracker on the LemmyNet GitHub, as well as the Matrix Chat. Post-Incident Activity is still on-going.

Currently, it is likely that just your session cookie was stolen, with instance admins being targeted specifically by checking for navAdmin, an HTML element only instance admins had. I do not believe this to affect users across instances, but I have yet to confirm this.

What happens next?

As I am not the developers or affected instance admins, I cannot make any guarantees. However, here is what you'll likely see:

  1. Post Incident investigation continues. This will include inspecting code, posts, websites, and more used by the hijackers. An official incident writeup may occur. You should expect the following from that report:
  • Exactly what happened, when.
  • The incident response that occurred from instance admins
  • Information that might have helped resolve the issue sooner
  • Any issues that prevented successful resolution
  • What should have been done differently by admins
  • What should be improved by developers
  • What can be used to identify the next attack
  • What tools are needed to identify that information
  1. A CVE is created. This is an official alert of the issue, and notifies security experts (and enthusiasts), even those not using lemmy, about the issue.

  2. A code security audit is done. This will likely just be casual reviews by technical lemmy users. However, I will be reaching out to the Mozilla Foundation and Cure53 as they recently did an audit of Mastodon. If there is interest in an external audit of lemmy and the costs are affordable, I'll look into crowdfunding this cost.

top 50 comments
sorted by: hot top controversial new old
[-] redcalcium@c.calciumlabs.com 187 points 1 year ago* (last edited 1 year ago)

This incident made me realize not to use an admin account for my primary lemmy account in my personal instance. I setup another account for instance admin purpose (with 2FA enabled) and keep it logged out, then remove my primary account from the instance admin list.

[-] Renacles@lemmy.world 73 points 1 year ago

This is a good mindset in general, when working in AWS you are not supposed to use your root account unless it's absolutely necessary even if you are the only user. Hosting a Lemmy instance should be no different.

[-] kakes@sh.itjust.works 11 points 1 year ago

Yeah, even in Windows or what-have-you, you should always keep your admin account separate from your daily driver account for exactly this reason.

load more comments (2 replies)
[-] Crackhappy@lemmy.world 17 points 1 year ago

I work with 2 factor, Oath, SAML, etc. all the time for work, and for the life of me I can't get it working properly with Lemmy.

[-] Notorious@lemm.ee 21 points 1 year ago

Lemmy decided to go with SHA256 for TOTP seed. This is a very odd move since many 2FA apps don’t support SHA256. I actually had to write a quick python script to spit out my 2FA code since Bitwarden doesn’t support it. Hopefully either Lemmy will change to SHA-1 or Bitwarden will start to support SHA256 seeds.

[-] redcalcium@c.calciumlabs.com 17 points 1 year ago* (last edited 1 year ago)

Wait, I use bitwarden and it works just fine. And yes, it's using SHA256.

Screenshot:

load more comments (8 replies)
[-] Renacles@lemmy.world 12 points 1 year ago

SHA-1 has been broken already.

load more comments (1 replies)
[-] ZeroNationality@lemmy.one 10 points 1 year ago

Preferably the latter, nobody should really be using SHA1 for anything security focused and new

load more comments (3 replies)
load more comments (1 replies)
[-] Uriel238@lemmy.fmhy.ml 99 points 1 year ago

Having recently migrated from Reddit (and kept up with commercial social media hacks) I'm used to Nothing To See Here! We totally didn't store your personal information in plaintext for hackers to snatch. Oh and maybe please change your passwords. All Part Of The Show!

So, by comparison, the response here is downright heartwarming.

load more comments (1 replies)
[-] 0x4E4F@sh.itjust.works 59 points 1 year ago

If there is interest in an external audit of lemmy and the costs are affordable, I’ll look into crowdfunding this cost.

It could get VERY, VERY expensive... depends on code complexity.

[-] static@kbin.social 59 points 1 year ago* (last edited 1 year ago)

Not at this stage.
Lemmy grew too fast, got many more eyes.
Step 1 is getting a security focus group selected from the people who contribute code to lemmy.

Just like the admins and coders volonteer their time, security specialists will too, money might be needed, but that is not in the the first steps.

[-] 0x4E4F@sh.itjust.works 15 points 1 year ago

Agreed, this is wise.

[-] rcmaehl@lemmy.world 22 points 1 year ago

Yeah, specifically why I mentioned "affordable".

load more comments (3 replies)
[-] barsoap@lemm.ee 11 points 1 year ago
  1. Set up a fake Nazi lemmy
  2. Have it online during the next Chaos Communication Congress
  3. ???
  4. 50 CVEs.
[-] rcmaehl@lemmy.world 17 points 1 year ago
  1. Be Lemmy Devs
  2. Loudly Claim Lemmy is Bulletproof and unhackable
  3. ???
  4. 50 CVEs
load more comments (1 replies)
load more comments (2 replies)
[-] p03locke@lemmy.dbzer0.com 28 points 1 year ago

A code security audit is done. This will likely just be casual reviews by technical lemmy users. However, I will be reaching out to the Mozilla Foundation and Cure53 as they recently did an audit of Mastodon. If there is interest in an external audit of lemmy and the costs are affordable, I’ll look into crowdfunding this cost.

You don't need to pay money. You just need to listen to the recommendations already made by free tools.

Here, fix this shit first and then worry about a professional audit later.

[-] where_am_i@sh.itjust.works 26 points 1 year ago

also, I cannot properly login into my lemmy.world account anymore. username/password work, but when I try to upvote it tells me i gotta be logged in. Tried apps and web.

[-] Yoz@lemmy.world 14 points 1 year ago
load more comments (3 replies)
[-] roy_mustang76@lemmy.world 11 points 1 year ago

Try clearing cookies (or app cache, on app), that will most likely resolve the issue.

load more comments (4 replies)
[-] Dreamer_joy@lemmy.ml 25 points 1 year ago
[-] Pekka@feddit.nl 40 points 1 year ago

Only Lemmy instances with custom emoticons were affected based on the Recap of the Lemmy XSS incident. So if Lemmy.ml doesn't have these it should not have been affected.

[-] dan1101@lemmy.world 11 points 1 year ago

I think generally the less flashy features (pun intended) the better. Text and links (and well sanitized) is all we need.

load more comments (2 replies)
load more comments (2 replies)
[-] PagingDoctorLove@lemmy.world 24 points 1 year ago

How do I contact my instance admin?

I went ahead and logged out and back in on my own, but I was not prompted to do so by Lemmy and I don't know enough about this stuff to say whether or not doing it on my own would've helped. I'm still pretty overwhelmed trying to figure this site out, so I apologize if this is a dumb question, just want to make sure I'm doing what I can to protect my own privacy.

I don't want to fall for the millennial version of a Nigerian prince scam, lol.

load more comments (5 replies)
[-] samae@lemmy.menf.in 22 points 1 year ago

Good reminder that: small instances = small impact. Both true for the positive and the negative.

[-] chunkystyles@sopuli.xyz 17 points 1 year ago

Or if you're on a small instance, you're a lot less likely to be targeted in the first place.

[-] overzeetop@lemmy.world 16 points 1 year ago

Security by obscurity is the best policy I always say 😉

load more comments (1 replies)
[-] Notorious@lemm.ee 10 points 1 year ago

Not sure that’s entirely true. Thankfully this attack vector required custom emojis, so it was limited to those specific Lemmy instances. Other attack vectors we may not be so lucky and it could spread through federation.

[-] phileashog@sh.itjust.works 17 points 1 year ago

XSS, classic

Thanks for your hard work, fuck the trolls that always have to poke holes in shit

[-] hackitfast@lemmy.world 39 points 1 year ago

Honestly, I see it as a win.

The people that did this didn't really act out in a coordinated attack. They were just kind of playing around, redirecting to lemonparty, changing page elements.

It could have been a lot worse. The site could have been redirecting to malicious websites, downloading trojans, doing a lot of bad things. Instead, we got direct attention to the security vulnerabilities in question, and they're being worked on and patched out relatively quickly. Helps that a lot of those on these communities are focused in programming and cybersecurity.

[-] drdaeman@lemmy.zhukov.al 14 points 1 year ago

The fundamental issue is not that emoji XSS (that's just a vector), but how JWTs are implemented and [not] secured. I've read that it was reported at least this January (https://akkoma.nrd.li/notice/AXXhAVF7N5ZH1V972W).

So, developers were already aware, yet - as I'm checking 0.18.1 - they have not fixed the unsafe-inline and unsafe-eval CSP, haven't made jwt cookie HttpOnly, and haven't done anything about exp and jti in the JWTs. I hope the recent events will make them do to so, and not just patch this particular XSS.

load more comments (1 replies)
[-] phileashog@sh.itjust.works 10 points 1 year ago

That's fair, I was a little harsh. Not all hackers are looking to cause damage!

load more comments (2 replies)
load more comments (1 replies)
[-] choroalp@programming.dev 16 points 1 year ago
load more comments (2 replies)
[-] monerobull@monero.town 14 points 1 year ago

Good thing we don't have custom emojis on monero.town and the admin account isn't used for things outside of the local community :D

load more comments (1 replies)
[-] kep@lemmy.world 12 points 1 year ago

This post is weird. You're typing like you're in charge of things, but you're apparently not.

It's one thing to show some initiative, but you're literally demanding a full report like the Lemmy devs work for you. You sound like someone who does this kind of thing for a living and felt the need to flex. Because otherwise, what the hell are you even doing?

Setting neurotically-specific demands for the developers makes sense if you represent a big instance or something, but you're literally just a dude. You could have framed this entire post in a different way and gotten away with it. Right now, it's creepy to anybody who actually reads the entire thing.

load more comments (1 replies)
[-] nobloat@lemmy.ml 9 points 1 year ago

Is the sudden unannounced disappearance of Vlemmy related to this ? I had my main account there and it just disappeared

load more comments (1 replies)
load more comments
view more: next ›
this post was submitted on 10 Jul 2023
782 points (98.4% liked)

Fediverse

28736 readers
128 users here now

A community to talk about the Fediverse and all it's related services using ActivityPub (Mastodon, Lemmy, KBin, etc).

If you wanted to get help with moderating your own community then head over to !moderators@lemmy.world!

Rules

Learn more at these websites: Join The Fediverse Wiki, Fediverse.info, Wikipedia Page, The Federation Info (Stats), FediDB (Stats), Sub Rehab (Reddit Migration), Search Lemmy

founded 2 years ago
MODERATORS