I have an ISR4331 that I’m using to replace an average branch office environment. You need to watch out for a clock issue, but not all of them are affected by it and a reputable seller is a must. Most also include the licenses you need so no extra cost there unless you need more services. I don’t run a FW yet because my lab doesn’t have Internet access, but I’d go for something from Sophos.
Air gapped, no Internet access. I don’t use Internet services for any of my stuff though so I can get away without direct Internet access
I just happen to have a spreadsheet that covers compute density for all models of dell, HPE, and supermicro…
Your cost/density sweet spot is going to be the 2u/4 node platform from Dell or supermicro that use the xeon e5-2600v4 or scalable v2. There’s a wide selection on eBay for this stuff, so it’s definitely available. At 16 cores/CPU, 128 cores/2u unit, you’ll need 6 units.
For the dell fx2s with 4 x fx630 nodes, 256gb memory/node, 32 cores/node, I spec’d one out for $2400 before storage and GPU
Why not VMs? Dell poweredge fx2s with 4 x fc630 or any other multi-node server will not only give you density but also the ability to scale out with the option of just turning off one or more of the nodes. There’s also the Hyve Zeus 1u servers that run pretty quiet, and can also scale out depending on how many you have turned on. They’re absolutely no-frills, only has room for two 2.5" drives, but it’s supermicro-based so there’s plenty of documentation.
My optiplex 9010 sff is what I use for experimenting with services and as a staging area for moving VMs to my main lab because it’s air gapped. At max load it runs at 140w but it has a GTX 1650 that I use for gaming as well.
Otherwise the rest of my lab is only turned on when I’m using it or forget to turn it off when I leave the house. When I get a laptop again I’ll leave it on more. None of it is more than $150 to replace though. It’s a Hyve Zeus, Cisco isr 4331, and a catalyst 3750x so nothing heavy, just a little loud.
First up - don’t make your own rack patch cables. They sell sets of tiny ones and you fingers and sanity will thank you. They usually get the terminations right the first time too, while you’re sure to do a few wrong and have to redo them.
Don’t get any switch that does less than 1gbe. My Dell power connect 2824 handles L2 switching without issues, it’s quiet, and it can be rack or desk mounted.
Rather than a NUC I always recommend an old business desktop like an optiplex or an elite desk. They can take abuse, the motherboards are cheap enough to replace when you mess up your first BIOS update, the non-micro ones have decent room for storage as well as a pcie slot, and you won’t feel bad about modding it to suit your needs… Like drilling a bunch of holes in the side of the case and slapping on a fan so a pcie slot is being cooled better.
While fiber is cheaper than copper, it uses more power and getting the right transceiver can be hard at first.
If your goal is to learn and your employer uses Cisco, go with Cisco gear. If they use Windows, stick with evaluations. If they use VMware, lol don’t use it bc licensing is painful at best.