I have the possibility of getting my hands on a practically unlimited supply of S905X2 Android boxes, they have 2GB of RAM and 8GB eMMC, only 100Mbit LAN but USB-3 as well.
I was joking around with colleagues, what about making an ARM cluster out of them? Use the 100Mbps port as a management port and add a USB3 Gigabit or 2.5G adapter for general traffic. Power the cluster by PoE using PoE to 5v splitters from a smart, management switch with PoE (10W each box?).
Lets say I got 40 boxes with 4 cores, that’s 160 cores and 80GB of RAM. It could run Armbian with Kubernetes or something lighter. I personally have no use for it, but it could be a fun proof of concept?
Tell me why I shouldn’t do this! Tell me why it won’t work and why I’d be wasting my time.
That sounds like a 12-15$ TV box.
Their downside is IO on network/storage, adding the cost of that and you start passing thin clients/minis in actual cost.
But with the arm option you are close to same consumption for cores that are 10% of the performance that cheap x86 would be.
but it could be a fun proof of concept?
Having gigantic clusters of small ARM computers, has been around almost as long as ARM is on consumer hardware. Definitly since the very first Pi 1. Loads of people have constructed many different configurations with all kind of different hardware. Loads of people here even run K8s and other stuff on their ARM cluster.
Tell me why I shouldn’t do this!
If you have no use for it, why bother? Only if you want to learn from it, it can be useful. But still, you really have to think about what you want to do with it.
Oh, and power reasons. If you’re in the USA, and your power is cheap, then why not. If you’re not so lucky with the powerbill, then just get one big system (x64 or ARM) and just run stuff on there.
BOINC.
Not x86. I think this answer all questions.
There are plenty of examples of Arm machines running at scale now, so that’s a pretty narrow perspective.
Not your case. And depends on your use case.
Sounds like you would be building the poor man’s version of an AmpereOne CPU.
A modern multicore x64 with half that RAM would run any workload that cluster could do… faster (carved into a handful of containers or VMs if you needed isolation).
It’s a cool project, for fun! It would be a horrible way to do any useful work though. Just the thought of wiring it all up makes me shudder…
Not only a ton of work but likely also much more expensive than you might think even with free boxes. A Gbit USB adapter costs lie $10, you might also need a cable, a switch, maybe (some) PSUs, a power strip, maybe some box to put everything into… And now you suddenly are spending $400-800 on this project just to get a rather lame cluster. Those cores are rater slow, 2GB of RAM is not that much, and 8GB of storage means you also need some kind of NAS.
If you like the project go for it but do not expect this to be a wise way to spend your time and money. 3-4 NUCs are likely less expensive and much more fun.
Plus the entire thing would likely be pulling way more power too, and take up 10-20x the space.
I switched a 6 year server over to a 3 node pi k8s and it runs with half the power. The one annoying thing is building images for Arm arch as a lot of common helm charts aren’t built for multi arch builds yet.
£5 for a USB NIC - £100 £3 for a PoE to 5V DC splitter - £60 £2 in short cables - £40 Shelf? £20-50
PoE Switch I already have spare Leaf switch I already have spare Network storage I have in abundance.
Cost wise it’s not prohibitive, about £250
I think the biggest worry is, as you say, it might not be the most stable or impressive compute.
£5 x 40 = 200. £3 x 40 = 120. Seems like you calculated with 20 and not 40. This helps to bring the cost down but even for 250 you can at least a single decent NUC.
It’s a good and fun idea for learning but the fact that it’s ARM may be a bit wonky and get in the way.
Projects are fun. Do it if you don’t mind failing. Just keep failing until you get it right.
Would it be fun? Probably. Would you learn stuff? Almost certainly yes. Useful as a full time production environment? Nah.
There are worse things to spend your money on. If it all goes tits up, usb network adapters are always useful, especially with the amount of laptops that don’t come with network ports these days.
I think this mainly depends on your workloads. As others have stated, the CPU isn’t all that fast RAM is even tighter.
Remember, it doesn’t matter how much ram you have in total, if your individual nodes don’t have enough resources for your most demanding workload, it can’t run.
It could be interesting at least to see if these ARM devices of yours could be used as the control plane for K8S.
Again, RAM would be tight, but they could handle a small cluster I think. The only question would be whether the emmc is fast enough for etcd, but you can test that with fio.
I would play around with it. The worst thing that can happen is you gain experience with k8s and learn that those devices aren’t really suitable.
If you’re looking to try learning about cluster management without running up a huge AWS bill, sure. But you’re not going to be doing anything with it you couldn’t do with some random PC with way more cores and horsepower, plus way more RAM (the vast majority of that 2GB in each box is going to be going to running the OS and the cluster client).
If you could get a bunch of PS3s or ASICs free, then you could actually do work that could net you some money: mine crypto (if it’s even worth doing at any given time based on electricity cost).