Hi all,

If you’re just now signing in for the first time in 12+ hours, you may just now be finding out that Lemmy World and other instances where hijacked. The hijackers had the full abilities of hijacked user, mod, and admin accounts. At this time, I am only aware of instance defacing and URL redirections to have been done by the hijackers.

If you were not forced to sign back in this morning, contact your instance admin to verify mitigations were completed on your instance.

How?

This occurred due to an XSS attack in the recently added custom emojis. Instance admins should follow the issue tracker on the LemmyNet GitHub, as well as the Matrix Chat. Post-Incident Activity is still on-going.

Currently, it is likely that just your session cookie was stolen, with instance admins being targeted specifically by checking for navAdmin, an HTML element only instance admins had. I do not believe this to affect users across instances, but I have yet to confirm this.

What happens next?

As I am not the developers or affected instance admins, I cannot make any guarantees. However, here is what you’ll likely see:

  1. Post Incident investigation continues. This will include inspecting code, posts, websites, and more used by the hijackers. An official incident writeup may occur. You should expect the following from that report:
  • Exactly what happened, when.
  • The incident response that occurred from instance admins
  • Information that might have helped resolve the issue sooner
  • Any issues that prevented successful resolution
  • What should have been done differently by admins
  • What should be improved by developers
  • What can be used to identify the next attack
  • What tools are needed to identify that information
  1. A CVE is created. This is an official alert of the issue, and notifies security experts (and enthusiasts), even those not using lemmy, about the issue.

  2. A code security audit is done. This will likely just be casual reviews by technical lemmy users. However, I will be reaching out to the Mozilla Foundation and Cure53 as they recently did an audit of Mastodon. If there is interest in an external audit of lemmy and the costs are affordable, I’ll look into crowdfunding this cost.

  • kep@lemmy.world
    link
    fedilink
    English
    arrow-up
    14
    arrow-down
    2
    ·
    1 year ago

    This post is weird. You’re typing like you’re in charge of things, but you’re apparently not.

    It’s one thing to show some initiative, but you’re literally demanding a full report like the Lemmy devs work for you. You sound like someone who does this kind of thing for a living and felt the need to flex. Because otherwise, what the hell are you even doing?

    Setting neurotically-specific demands for the developers makes sense if you represent a big instance or something, but you’re literally just a dude. You could have framed this entire post in a different way and gotten away with it. Right now, it’s creepy to anybody who actually reads the entire thing.

    • RCMaehl [Any]@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      3
      arrow-down
      1
      ·
      edit-2
      1 year ago

      These aren’t demands, but I can definitely see how they can come off that way. These are industry standard post cybersecurity incident review questions by defined by NIST (NIST SP 800-61 Rev 2 Section 3.4.1) slightly rephrased.

  • redcalcium@c.calciumlabs.com
    link
    fedilink
    English
    arrow-up
    9
    ·
    edit-2
    1 year ago

    This incident made me realize not to use an admin account for my primary lemmy account in my personal instance. I setup another account for instance admin purpose (with 2FA enabled) and keep it logged out, then remove my primary account from the instance admin list.

    • Renacles@lemmy.world
      link
      fedilink
      English
      arrow-up
      5
      ·
      1 year ago

      This is a good mindset in general, when working in AWS you are not supposed to use your root account unless it’s absolutely necessary even if you are the only user. Hosting a Lemmy instance should be no different.

    • BarterClub@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 year ago

      Yup. Basics of running a server for anything. Never use your admin account and make a default backup with 2 factor.

    • Crackhappy@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 year ago

      I work with 2 factor, Oath, SAML, etc. all the time for work, and for the life of me I can’t get it working properly with Lemmy.

      • Notorious@lemm.ee
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        Lemmy decided to go with SHA256 for TOTP seed. This is a very odd move since many 2FA apps don’t support SHA256. I actually had to write a quick python script to spit out my 2FA code since Bitwarden doesn’t support it. Hopefully either Lemmy will change to SHA-1 or Bitwarden will start to support SHA256 seeds.

          • Trapping5341@lemmy.world
            link
            fedilink
            English
            arrow-up
            0
            ·
            1 year ago

            wait wait wait. how did you get lemmy 2FA into bitwarden. I can only get it to set up in authy or google auth

            • redcalcium@c.calciumlabs.com
              link
              fedilink
              English
              arrow-up
              0
              ·
              1 year ago

              You’ll need to either subscribe to bitwarden premium ($10 / year), or deploy vaultwarden in your own server and have your bitwarden extension/app use that vaultwarden server instead of the official server.

              • Trapping5341@lemmy.world
                link
                fedilink
                English
                arrow-up
                0
                ·
                1 year ago

                Yeah I have premium and every account I have that offers 2fa is set up but Lemmy will only let me set it up in authy or Google auth. It won’t give bitwarden as an option or give me the code to manually add it to bitwarden.

                • redcalcium@c.calciumlabs.com
                  link
                  fedilink
                  English
                  arrow-up
                  1
                  ·
                  1 year ago

                  Just right click at the 2fa button and select “copy link”, then paste the link into bitwarden TOTP field. The link you copied should already be in the format otpauth://totp/xxxx which will be recognized by bitwarden.

  • P03 Locke@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    7
    ·
    1 year ago

    A code security audit is done. This will likely just be casual reviews by technical lemmy users. However, I will be reaching out to the Mozilla Foundation and Cure53 as they recently did an audit of Mastodon. If there is interest in an external audit of lemmy and the costs are affordable, I’ll look into crowdfunding this cost.

    You don’t need to pay money. You just need to listen to the recommendations already made by free tools.

    Here, fix this shit first and then worry about a professional audit later.

  • Uriel-238@lemmy.fmhy.ml
    link
    fedilink
    English
    arrow-up
    7
    ·
    1 year ago

    Having recently migrated from Reddit (and kept up with commercial social media hacks) I’m used to Nothing To See Here! We totally didn’t store your personal information in plaintext for hackers to snatch. Oh and maybe please change your passwords. All Part Of The Show!

    So, by comparison, the response here is downright heartwarming.

    • Alkider@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 year ago

      Whoops! Reddit spazzed out and couldn’t send your post because it hurts spez’s feelings!

  • CifrareVerba@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    edit-2
    1 year ago

    The way the hack was utilized is honest very creative and interesting; either way, if all big communities could crowdsource money for security audits, I believe that it could help prevent something like this in the future.

    Edit for clarification: I am NOT talking about how the exploit itself was created or achieved, I am talking about the act of having the power they did and just doing shock crap.

  • 0x4E4F@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    If there is interest in an external audit of lemmy and the costs are affordable, I’ll look into crowdfunding this cost.

    It could get VERY, VERY expensive… depends on code complexity.

    • static@kbin.social
      link
      fedilink
      arrow-up
      1
      ·
      1 year ago

      Not at this stage.
      Lemmy grew too fast, got many more eyes.
      Step 1 is getting a security focus group selected from the people who contribute code to lemmy.

      Just like the admins and coders volonteer their time, security specialists will too, money might be needed, but that is not in the the first steps.

      • dan1101@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        I think generally the less flashy features (pun intended) the better. Text and links (and well sanitized) is all we need.

        • Wintermute@lemmy.villa-straylight.social
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          I agree. Sadly it seems the rest of the world does not. Hopefully as Lemmy matures we can get to a point where features are not pushed put half-baked because there aren’t enough people willing and able to give thorough code reviews.

    • RCMaehl [Any]@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      1
      ·
      edit-2
      1 year ago

      To add context to this. What I’ve been told is that a community running on a lemmy fork with 5 digit users had used this code for a while and backported(?) the code upstream when they federated back. I guessing there was an assumption of safety as they had been using the custom emojis code for quite a while without it being exploited.

    • FightMilk@lemmy.world
      link
      fedilink
      English
      arrow-up
      1
      arrow-down
      2
      ·
      1 year ago

      Yeah it’ll be hard to regain my trust after this one. I mean I’ll still use Lemmy but for now I’ll assume mine or any other account could be hacked at any time and act accordingly. This is a really amateur mistake even by FOSS standards.

  • ZuckerbergIsCancer@lemmy.ml
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    Friendly remind that Lemmy.world is considering federating with Meta while Lemmy.ml will not.

    I would strongly suggesting ditching Lemmy.world for Lemmy.ml before they can even get started with that stupidity.

  • theangryseal@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 year ago

    I haven’t been able to change my password on Lemmy.world. When I click save, nothing happens and the password doesn’t update.

    That’s probably something someone wants to look into.

  • where_am_i@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 year ago

    also, I cannot properly login into my lemmy.world account anymore. username/password work, but when I try to upvote it tells me i gotta be logged in. Tried apps and web.

    • hackitfast@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 year ago

      Honestly, I see it as a win.

      The people that did this didn’t really act out in a coordinated attack. They were just kind of playing around, redirecting to lemonparty, changing page elements.

      It could have been a lot worse. The site could have been redirecting to malicious websites, downloading trojans, doing a lot of bad things. Instead, we got direct attention to the security vulnerabilities in question, and they’re being worked on and patched out relatively quickly. Helps that a lot of those on these communities are focused in programming and cybersecurity.

  • monerobull@monero.town
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 year ago

    Good thing we don’t have custom emojis on monero.town and the admin account isn’t used for things outside of the local community :D