In the past two weeks I set up a new VPS, and I run a small experiment. I share the results for those who are curious.

Consider that this is a backup server only, meaning that there is no outgoing traffic unless a backup is actually to be recovered, or as we will see, because of sshd.

I initially left the standard “port 22 open to the world” for 4-5 days, I then moved sshd to a different port (still open to the whole world), and finally I closed everything and turned on tailscale. You find a visualization of the resulting egress traffic in the image. Different colors are different areas of the world. Ignore the orange spikes which were my own ssh connections to set up stuff.

Main points:

  • there were about 10 Mb of egress per day due just to sshd answering to scanners. Not to mention the cluttering of access logs.

  • moving to a non standard port is reasonably sufficient to avoid traffic and log cluttering even without IP restrictions

  • Tailscale causes a bit of traffic, negligible of course, but continuous.

  • Decronym@lemmy.decronym.xyzB
    link
    fedilink
    English
    arrow-up
    157
    arrow-down
    6
    ·
    edit-2
    5 months ago

    Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:

    Fewer Letters More Letters
    DNS Domain Name Service/System
    HTTP Hypertext Transfer Protocol, the Web
    IP Internet Protocol
    SSH Secure Shell for remote terminal access
    UDP User Datagram Protocol, for real-time communications
    VPN Virtual Private Network
    VPS Virtual Private Server (opposed to shared hosting)
    nginx Popular HTTP server

    7 acronyms in this thread; the most compressed thread commented on today has 8 acronyms.

    [Thread #42 for this sub, first seen 14th Aug 2023, 15:55] [FAQ] [Full list] [Contact] [Source code]

  • James@lemmy.ca
    link
    fedilink
    English
    arrow-up
    75
    ·
    1 year ago

    Public key auth, and fail2ban on an extremely strict mode with scaling bantime works well enough for me to leave 22 open.

    Fail2ban will ban people for even checking if the port is open.

    • devtimi@lemmy.world
      link
      fedilink
      English
      arrow-up
      21
      ·
      1 year ago

      Honest question, is there a good default config available somewhere or is what apt install fail2ban does good to go? All the tutorials I’ve found have left it to the reader to configure their own rules.

      • Sleepkever@lemm.ee
        link
        fedilink
        English
        arrow-up
        6
        ·
        1 year ago

        Honestly the default config is good enough to prevent brute force attacks on ssh. Just installing it and forgetting about it is a definite option.

        I think the default block time is 10 minutes after 5 failed login attempts in 10 minutes. Not enough to ever be in your way but enough to fustrate any automated attacks. And it’s got default config for a ton of services by default. Check your /etc/fail2ban/jail.conf for an overview.

        I see that a recidive filter that bans repeat offenders for a week after 10 fail2ban bans in one day is also default now. So I’d say that the results are perfect unless you have some exotic or own service you need fail2ban for.

    • JubilantJaguar@lemmy.world
      link
      fedilink
      English
      arrow-up
      1
      arrow-down
      7
      ·
      1 year ago

      If Fail2Ban is so important, why the h*** does it not come installed and enabled as standard?!

      Security is the number-1 priority for any OS, and yet stock SSHD apparently does not have Fail2Ban-level security built in. My conclusion is that Fail2Ban cannot therefore be that vital.

  • notabot@lemm.ee
    link
    fedilink
    English
    arrow-up
    74
    arrow-down
    12
    ·
    1 year ago

    You really shouldn’t have something kike SSHD open to the world, that’s just an unnecessary atrack surface. Instead, run a VPN on the server (or even one for a network if you have several servers on one subnet), connect to that then ssh to your server. The advantage is that a well setup VPN simply won’t respond to an invalid connection, and to an attacker, looks just like the firewall dropping the packet. Wireguard is good for this, and easy to configure. OpenVPN is pretty solid too.

    • douglasg14b@lemmy.world
      link
      fedilink
      English
      arrow-up
      23
      arrow-down
      2
      ·
      1 year ago

      You say this and are downvoted.

      While we are coming off the tail of Def Con where there where a plethora or small talks and live examples of taking advantage and abusing just this.

      • teawrecks@sopuli.xyz
        link
        fedilink
        English
        arrow-up
        22
        arrow-down
        1
        ·
        1 year ago

        Just trying to parse your comment, I assume your first “this” and second “this” are referring to different things, right?

      • TheLemming@feddit.de
        link
        fedilink
        English
        arrow-up
        3
        ·
        1 year ago

        I don’t understand your comment, what you are saying. Could you elaborate a bit, please? I’m interested why it’s a bad idea what previous comment suggested.

        Of course I can dig into DefCon videos and probably would do if needed, but perhaps you know what exactly the issue is

        • MaggiWuerze@feddit.de
          link
          fedilink
          English
          arrow-up
          3
          ·
          1 year ago

          The first this means the comment he answered to and the second one means ssh being used as an attack surface, being described in defCon talks

    • JDubbleu@programming.dev
      link
      fedilink
      English
      arrow-up
      7
      ·
      1 year ago

      I usually just run a ZeroTier client on my Pi connected to a private P2P network to solve this issue, and then have ProtonVPN over Wireguard for all internet traffic in and out of the Pi.

  • Clou42@feddit.de
    link
    fedilink
    English
    arrow-up
    59
    arrow-down
    5
    ·
    1 year ago

    I’ll take that tiny amount of traffic telling scanners there’s no password auth over having to remember port settings for ssh, scp and rsync any day.

  • u_tamtam@programming.dev
    link
    fedilink
    English
    arrow-up
    53
    arrow-down
    2
    ·
    1 year ago

    Or, you know, just use key auth only and fail2ban. Putting sshd behind another port only buys you a little time.

    • nomadjoanne@lemmy.world
      link
      fedilink
      English
      arrow-up
      22
      ·
      1 year ago

      Yeah but the majority of bots out there are going after easy prey. Honestly, if you use public key authentication with ssh you should be fine, even if it is on port 22. But it does of course clog up access logs.

      • marcos@lemmy.world
        link
        fedilink
        English
        arrow-up
        15
        ·
        1 year ago

        The majority of bots out there are stopped by just using a hard to guess password. It’s not them that you should be worried about.

        • aard@kyu.de
          link
          fedilink
          English
          arrow-up
          10
          ·
          1 year ago

          The majority of bots doesn’t even show up in the logs if you disable password auth in the server config, as you typically should.

    • MangoPenguin@lemmy.blahaj.zone
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 year ago

      I’ve noticed that a lot of the scans these days almost always switch IPs after 2-3 attempts, making IP blocking a lot more difficult.

      • u_tamtam@programming.dev
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        Let’s say that you could ban for an indefinitely large amount of time after a single failure to authenticate, that’d make them run out of IPs much quicker than you’d run out of CPU/BW, so I don’t really see the issue

          • u_tamtam@programming.dev
            link
            fedilink
            English
            arrow-up
            2
            ·
            1 year ago

            True, but very unlikely (once your ssh client is configured once and for all), and in that event you can always switch connection (use a data network, proxy, vpn, hop from another server you have ssh access, etc)

  • Feathercrown@lemmy.world
    link
    fedilink
    English
    arrow-up
    36
    ·
    1 year ago

    I opened a raw text channel on the Telnet port for a personal game engine project and someone tried to enable commands and do some shady stuff. Unfortunately for them, that’s not a valid chess move.

  • spagnod@lemmy.world
    link
    fedilink
    English
    arrow-up
    30
    ·
    1 year ago

    Just do it properly and configure sshd securely. When you have a machine exposed to the internet, you should expect it to be attacked. If you really want to give the finger to bots, run endlessh on port 22 and keep sshd on a non-standard port. Stay safe.

  • z3bra@lemmy.sdf.org
    link
    fedilink
    English
    arrow-up
    25
    arrow-down
    1
    ·
    1 year ago

    I get what you say, and you’re definitely not wrong to do it. But as I see it, you only saved ~80Kib of ingress and a few lines of logs in the end. From my monitoring I get ~5000 failed auth per day, which account for less than 1Mbps average bandwidth for the day.

    It’s not like it’s consuming my 1Gbps bandwidth or threatening me as I enforce ssh key login. I like to keep things simple, and ssh on port 22 over internet makes it easy to access my boxes from anywhere.

    • aesir@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      8
      ·
      edit-2
      1 year ago

      ssh -p 12345 would leave your boxes accessible from anywhere too. Other blocks of IPs receive 10 times or more requests, as scanners can focus on blocks of ips from major providers.

      • z3bra@lemmy.sdf.org
        link
        fedilink
        English
        arrow-up
        3
        arrow-down
        2
        ·
        1 year ago

        Yeah I know, I just don’t really care about that traffic to bother changing it :) Also, I’m talking about a server hosted on Hetzner, so I feel like it’s scanned a lot.

    • oatscoop@midwest.social
      link
      fedilink
      English
      arrow-up
      1
      arrow-down
      1
      ·
      edit-2
      1 year ago

      I don’t get why people leave interfaces the public doesn’t need access to open to the public – especially SSH.

      Use a VPN if you need access to those interfaces from the “outside”. They’re stupidly easy to set up these days, particularly with Wireguard.

      • z3bra@lemmy.sdf.org
        link
        fedilink
        English
        arrow-up
        8
        arrow-down
        1
        ·
        1 year ago

        A VPN is easy to setup (and I have it setup by the way), but no VPN is even easier. SSH by itself is sufficiently secure if you keep it up to date with a sane configuration. Bots poking at my ssh port is not something that bother me at all, and not part of any attack vector I want to be secure against.

        Out of all the services I expose to the clear web, SSH is probably the one I trust the most.

        • GreenBottles@lemmy.world
          link
          fedilink
          English
          arrow-up
          3
          ·
          edit-2
          1 year ago

          I would generally agree with this a strong password and SSH without keys has never gone sideways for me and over 15 years of having public Linux servers. but I also make sure to install all security updates on a regular basis on any server no matter what SSH configuration is.

        • oatscoop@midwest.social
          link
          fedilink
          English
          arrow-up
          1
          ·
          edit-2
          1 year ago

          Defense in depth – maybe I’m paranoid, but just because something is unlikely doesn’t mean an extra layer of security isn’t advantageous. Particularly when I already have a VPN, so there’s little reason not to use it.

          Plus, my logs are easily checked as a side effect.

          • z3bra@lemmy.sdf.org
            link
            fedilink
            English
            arrow-up
            1
            ·
            1 year ago

            To each their own ! Security is a complex topic which usually resolves to adjusting the “security/annoyance” cursor to the best position.

            In my case the constraints of using a VPN simply outweighs the security benefits.

    • entropicdrift@lemmy.sdf.org
      link
      fedilink
      English
      arrow-up
      6
      ·
      1 year ago

      Right? PiVPN is easy AF and uses WireGuard. No reason not to set up something yourself if you’re already selfhosting.

      • glasgitarrewelt@feddit.de
        link
        fedilink
        English
        arrow-up
        10
        ·
        1 year ago

        Please ELI5: How does this solution work? You tunnel yourself in your home network with a VPN on your PI? How is that safer, isn’t there still a port open?

        • entropicdrift@lemmy.sdf.org
          link
          fedilink
          English
          arrow-up
          7
          ·
          1 year ago

          I have a port open, yes, but it’s not port 22. The problem with Tailscale is you’re trusting a third party. With my setup, it’s just me connecting directly to my Pi. Thanks to port forwarding the only open port pointing to my Pi is the one that I use for WireGuard.

          I already have a DDNS and domain name pointing to my house, so there’s effectively no added risk compared to my existing setup with a couple webapps being reverse proxied behind Caddy on a different device.

            • hatedbad@lemmy.sdf.org
              link
              fedilink
              English
              arrow-up
              1
              ·
              1 year ago

              no, and that’s be a pretty bad idea, you’re opening up all your internal hosts to the public internet.

              a VPN is specifically designed to keep all your internal hosts off the public internet. When you authenticate with the VPN server the remote device you are using effectively “joins” the internal network, using the VPN to act like a tunnel between you and your network.

              it has the benefits of better security as well as the fact that once you set it up, you can access any services you host, not just HTTP ones.

  • elscallr@lemmy.world
    link
    fedilink
    English
    arrow-up
    21
    arrow-down
    2
    ·
    edit-2
    1 year ago

    If you do want to open 22, and there are plenty of good reasons to want to, just implement something called port knocking and you can do it safely.

    Note with this you still need good authentication. That means no passwords, key based auth only.

    • FarraigePlaisteach@kbin.social
      link
      fedilink
      arrow-up
      7
      ·
      1 year ago

      I have read elsewhere that port knocking is just security through obscurity and isn’t worth considering. I found it when searching for ways to set it up and that put me off.

      • elscallr@lemmy.world
        link
        fedilink
        English
        arrow-up
        21
        ·
        edit-2
        1 year ago

        It is and it isn’t. It prevents random scans from opening 22 and attempting to authenticate, that’s basically the entire purpose. You still need good authentication after because you’re right, it’s not a security measure, it’s just a way to keep your logs useful and to keep botnets from beating the hell out of 22.

        By “good authentication” I mean a key pair based authentication. That is impossible to brute force. If you use a password on 22 you shouldn’t open it at all and you should rethink allowing any remote access.

        Put another way: You’re the doorman at a speak easy. You can answer the little window with “what’s the password?” to every jack ass that approaches, and you’d be asking all the time. But if they don’t know they have to knock “shave and a haircut” first, your job gets a lot easier and you’re dealing with a lot fewer nuisance password promptings.

        You can also use it to blacklist. If someone tries to hit 22 without knocking you can blacklist that IP entirely because you know it’s nuisance.

        • FarraigePlaisteach@kbin.social
          link
          fedilink
          arrow-up
          5
          ·
          1 year ago

          When you put it that way, it sounds beneficial and like something I’d like to use. Thank you!

          The article may have been dissing it as a sole strategy.

          • elscallr@lemmy.world
            link
            fedilink
            English
            arrow-up
            4
            ·
            1 year ago

            It’s not uncommon for people to try using it as their sole authentication so that wouldn’t be a surprise. But for it’s purpose, it’s perfect.

        • Hector_McG@programming.dev
          link
          fedilink
          English
          arrow-up
          5
          arrow-down
          1
          ·
          edit-2
          1 year ago

          But if they don’t know they have to knock “shave and a haircut” first, your job gets a lot easier and you’re dealing with a lot fewer nuisance password promptings.

          Very good explanation. And the benefits are even greater, because there is absolutely no response until the entire secret knock is correctly used, the random guy trying to get in doesn’t even know if there’s anyone at that address. (In fact, set up correctly, they won’t even know if there’s really a door there or not)

      • Ondergetekende@feddit.nl
        link
        fedilink
        English
        arrow-up
        15
        arrow-down
        2
        ·
        1 year ago

        Security through obscurity is a very valid way to secure something. It shouldn’t be the only way of securing something, but it can be a valid additional layer.

        The knock sequence is a secret, just like a password. It may not be a particularly strong secret, but is is strong enough to keep out casual attackers. You’ll still need additional security, but sshd is well equipped to provide that.

      • Hector_McG@programming.dev
        link
        fedilink
        English
        arrow-up
        5
        arrow-down
        4
        ·
        edit-2
        1 year ago

        If you want to go down that path, a password is only security by obscurity.

        Port knocking is an extra layer of security, and one that can stop attackers from ever knowing your private server even exists. A random scanner won’t even see any open ports.

        Always bear in mind that any random guy advising people not to use port knocking may be doing it with malicious intent. I’m sure there’s someone out there advising that random passwords are a waste of time, and everyone should just use monkey123.

    • elscallr@lemmy.world
      link
      fedilink
      English
      arrow-up
      32
      ·
      1 year ago

      Computers communicate across networks using ports. Port 22 is a commonly used remote administration port called ssh. Bots go around probing computers with an open port 22 hoping to find badly secured or outside misconfigured ssh servers to turn them into bots and crypto miners, etc.

      • YⓄ乙 @aussie.zone
        link
        fedilink
        English
        arrow-up
        10
        ·
        edit-2
        1 year ago

        Its crazy people can do all these stuff. I can’t even edit my word document. Being said that I want to learn IT but looks hell stressful for $100-$200k job. I was in content & marketing making $130k and we used to discuss about shades of color to use in a font for 3-4 weeks. Its crazy how you guys have to fix issues within minutes on those tickets. The more i learn about IT, I feel like I should stay away from it.

        • elscallr@lemmy.world
          link
          fedilink
          English
          arrow-up
          3
          ·
          1 year ago

          I can’t edit Word documents for shit lol. I edit everything using Markdown (the same formatting used here) because I don’t have to think about it.

          That said every job can get stressful now and again, and this line of work is no different, but most days it’s just work. Make this change, make this thing do something else, kill this thing that’s costing money and everyone stopped using last year without telling anyone, etc. Typical things.

        • xeddyx@lemmy.nz
          link
          fedilink
          English
          arrow-up
          2
          ·
          edit-2
          1 year ago

          Not everyone in IT needs to fix tickets or work in a high-stress environment. In one of my previous roles, I was a projects engineer, and I was basically given a bunch of projects to work on (like there was a small python-based project - they needed to automate something; then there was one to get them into a hybrid cloud setup; another project to upgrade something and so on). I didn’t really have any break-fix tickets to work on, although I was occasionally asked if I could help, when there was some spare time or if it was something high-level the ops guys couldn’t fix. Basically a total chill job, I was free to allocate time on my projects as I saw fit, no hard deadlines, no SLAs to meet, and the best part - no users to deal with.

          Of course, it wasn’t always like this. To get here, I had to do those grunt roles first, those stressful jobs with tickets that needed to be fixed in minutes, dealing with angry users and stuff. But thankfully my career has progressed past that stage now.

        • Saigonauticon@voltage.vn
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          One of the sad aspects of my job (in IT) is building tools to eliminate less stressful jobs, especially ones that pay well (usually management or accounting, in my case). Design has definitely been a specific target in recent years though – off the top of my head I could at least imagine two approaches to writing a tool that automates color and font selection with results comparable to human expertise.

          This is one reason it’s a good idea to regularly study new things (IT or otherwise). I have to retool every few years as whatever I know becomes obsolete – this used to mainly be a frustration in IT, but is rapidly becoming a necessary process in other fields. It won’t be necessary to become an IT expert, but I would keep up-to-date on how to use the new tools technology provides… especially if I wanted to keep a job in say, graphics design or copywriting!

          (Incidentally, my first job in this country was in marketing! It was high-stress and I did not earn 130k. I recall font and color choice processes vividly :D)

          • YⓄ乙 @aussie.zone
            link
            fedilink
            English
            arrow-up
            1
            ·
            1 year ago

            Yea dont create anything bro. 100k-200k is nothing in 2023. Like you can’t even afford a decent house for a family. I do my 9-5 with the least work possible and go home. Tried starting a business but failed miserably so back to 9-5.

            • Saigonauticon@voltage.vn
              link
              fedilink
              English
              arrow-up
              1
              ·
              1 year ago

              Yeah… I couldn’t cope with that unfortunately (I’m a bit jealous, it sounds nice). I need to work long hours and make things, it’s a compulsion. “Taking it easy” can stress me out to the point where I end up in a hospital.

              So I sold all my worldly possessions and immigrated to the developing world on an investment visa (where things are made). My timing was a few years early, but I had no path to a decent life left except having my own company in a growth economy – my entire industry vanished twice overnight in my home country due to changes in legislation.

              Nowadays, looking at the local economy, there is no path to home ownership except for people who own companies, and maybe senior executives or senior software engineers. An average university-educated couple would have to save 100% of their income for their entire adult life to afford a nice home – if they don’t have kids. I think this kind of cruel equation is slowly coming to the West too – although you guys have more land so I guess it takes longer.

              • YⓄ乙 @aussie.zone
                link
                fedilink
                English
                arrow-up
                1
                arrow-down
                1
                ·
                1 year ago

                Feel for you brother. Dont get dominated by stupid 9-5 managers, directors, CEO instead try dominating them. They cant do all thr work by themselves , even if they fire you, its a 9-5 job. You have to take what’s yours legally or illegally. Look at the history from colonization to new era Capitalism. More power to you bro!

                • Saigonauticon@voltage.vn
                  link
                  fedilink
                  English
                  arrow-up
                  1
                  ·
                  1 year ago

                  Perhaps ironically, I live in a nominally Communist country that went through decolonization quite a number of times. It doesn’t change much in my daily life (I’m not really political), although I arguably own some tiny slice of the means of production these days. So maybe in retirement I’ll provide public access to those for working class people. That would be really fun, I think. Who knows what we might create together? Certainly if the machines are sitting unused in my retirement, they are creating nothing, and I would feel sad for the machines.

                  I don’t do the whole 9-5 thing. That would stress me out! I work as long as I feel like, any day of the week I feel like. Generally, this is really nice for both managing stress (there’s always tomorrow!) and steamrolling over any competition.

                  I’m just a mercenary (and a bureaucrat) though. You pay my fee in filthy lucre, and the job gets done – legally, and reliably. If someone annoys me with politics at a client, I just try and replace them with a computer program. The result is that several of my best coworkers are machines these days. I foresee that trend increasing with time.

  • FrederikNJS@lemm.ee
    link
    fedilink
    English
    arrow-up
    13
    arrow-down
    2
    ·
    1 year ago

    As others have already said, set up a VPN like wireguard, connect to the VPN and then SSH to the server. No need to open ports for SSH.

    I do have port 22 open on my network, but it’s forwarded to an SSH tarpit: https://github.com/skeeto/endlessh

    • Don Corleone@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 year ago

      I have wireguard for other purposes but I also have ssh open on a different port. I don’t much understand the argument of exchanging ssh for wireguard. In the end, we’re just trading an attack vector for another.

      My ssh only allows connections from my user. If I’m using password auth, I also request a 2FA.

      Tail scale is also a good idea but I don’t like having my control plane under someone else’s control.

      • 486@kbin.social
        link
        fedilink
        arrow-up
        5
        ·
        1 year ago

        There is quite a significant difference. An ssh server - even when running on a non-default port - is easily detectable by scanning for it. With a properly configured Wireguard setup this is not the case. As someone scanning from the outside, it is impossible to tell if there is Wireguard listening or not, since it simply won’t send any reply to you if you don’t have the correct key. Since it uses UDP it isn’t even possible to tell if there is any service running on a given UDP port.

      • FrederikNJS@lemm.ee
        link
        fedilink
        English
        arrow-up
        4
        ·
        1 year ago

        The reason a VPN is better to expose than SSH, is the feedback.

        If someone tries connecting to your SSH with the wrong key or password, they get a nice and clear permission denied. They now know that you have SSH, and which version. Which might allow them to find a vulnerability.

        If someone connects to your wireguard with the wrong key, they get zero response. Exactly as if the port had not been open in the first place. They have no additional information, and they don’t even know that the port was even open.

        Try running your public IP through shodan.io, and see what ports and services are discovered.

      • barsoap@lemm.ee
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        If someone finds a 0day in your SSH server and goes on drive-by attacking the whole internet you’re toast.

        Already moving off port 22 reduces much of the risk, essentially reducing the attack surface for drive-by attacks to zero while still being susceptible to targeted attacks – that is, still susceptible to attackers bothering to scan the whole range. Anything that makes you unscannable (VPN, portknockd, doesn’t matter) mitigates that. Even state-level actors would have to be quite determined to get through that one.

        Yes it’s security through obscurity. Yes it’s a good idea: There’s a difference between hiding your unlocked front door and hiding your military-grade front door lock, one of them is silly the other isn’t.

  • axum@kbin.social
    link
    fedilink
    arrow-up
    19
    arrow-down
    8
    ·
    1 year ago

    ITT: People who don’t understand Tailscale or are allergic to it for ‘reasons’