I’m impressed that Lemmy only needs 150 MB RAM and uses virtually no CPU
This seems like an ideal thing to install on an embedded system such as a Raspberry Pi.
Problem: Where are the builds?
In the installation page, you have three options: Docker, Ansible, From Scratch
I’m currently doing the from scratch option since my OS distributor is not a fan of docker or ansible (and neither am I), but the build is taking ~1hr.
Are there no nightly builds with .deb packages?
Providing binary packages is kind of a futile effort since you’d need one for each distro/version and architecture. I don’t like/use Docker either but it is at least a distro-agnostic format. If you want distro packages, ask your distro maintainers to package it.
Are you compiling it directly on the Pi? If so, try cross-compiling, it’s going to be a lot faster that way. I just cross-compiled the server in about 5 minutes on my machine as a test.
Other packages do this though, and builds are easy to automate. Is it because it’s not stable software, and so hasn’t passed the usual unstable, testing, stabile repo vetting process? If so, I’m surprised, it really looks mature.
Ah cross-compiling is something I hadnt considered… do you have a preferred toolchain for this?
My preferred toolchain for most things these days, including this, is the Nix package manager. It makes cross-compiling most programs (or other similar things like compiling fully statically linked binaries, or a combination of the two) really easy, in this case with a single command, without having to deal with setting up any of the cross-compiler toolchain itself or anything else like that. It also has a huge package collection including the Lemmy packages. For example, this is the command I used for testing:
nix build nixpkgs#pkgsCross.armv7l-hf-multiplatform.lemmy-serverFor building and copying to a remote host (both the build and target host should have Nix installed), something like this should work instead:
nix copy --to ssh://raspberry-pi nixpkgs#pkgsCross.armv7l-hf-multiplatform.lemmy-serverThere is NixOS, the distro built on top of Nix, but Nix itself works on any distro and doesn’t come into conflict with your distro’s package manager, so there’s no issue with having it installed in parallel.
I actually make a lot of use of the cross-compiling functionality since my router is also an armv7l box which isn’t very powerful, so I cross-compile the whole NixOS system (including some custom packages) on my PC and then copy it over. It’s very nice to work with.
Here’s the website: https://nixos.org/
Wow, I tried NixOS at one point but I couldn’t keep it as a daily driver because it didn’t play nicely with my conda-based dev environment.
I knew nix like guix (another failed experiment I wanted to make my daily driver) could be run as a package manager but I had no idea it could cross-compile and copy in one command!
These are static builds right?
Wow, I tried NixOS at one point but I couldn’t keep it as a daily driver because it didn’t play nicely with my conda-based dev environment.
I’m not familiar with it but it looks like conda heavily relies on prebuilt non-static binaries as well… which do cause issues the further the target system is from the system that built it (and NixOS is very far away from most Linux systems). Another reason why I don’t like providing those anywhere unless they’re statically linked, I guess :P
However it looks like conda in nixpkgs is packaged with an additional conda-shell script which drops you into a shell with the expected library paths so stuff should work, if you want to try it again. There’s a small section about it in the NixOS Wiki. (The same workaround also exists with a lot of other prebuilt binaries that expect FHS paths, steam-run being a convenient wrapper for that which makes most stuff just work. It uses the Steam runtime, that’s why it’s called that, but it works for anything, not just games.)
I knew nix like guix (another failed experiment I wanted to make my daily driver) could be run as a package manager but I had no idea it could cross-compile and copy in one command!
Yup, that’s pretty much the result of a lot of work in nixpkgs done to make cross-compilation for various compilers work well without having to explicitly support it in each package, and especially without having to package cross-compiled stuff separately. So if you write a remotely standard Nix package for some software and it builds normally, chances are you’ll be able to cross-compile it without any further work on your part. Nixpkgs also automatically exposes these cross-compiled packages for you under the pkgsCross attribute set, which contains the exact same packages as top-level nixpkgs but with a different build host target, so for packages inside nixpkgs you don’t even have to instantiate it with custom options to set up cross-compilation.
And nix copy will then simply build the package you specify if it’s not already in the local store or the build cache before sending it to the target host. You can also do the reverse, specify one or more build hosts (that are more powerful, or have a different architecture that a package needs) so that nix will do distributed builds on those other hosts and then automatically copy the result back.
I did try Guix as well because Scheme seems very cool and it hasn’t accumulated as much tech debt as NixOS and a couple better designs from what I’ve seen (for example grafts), but failed at the lack of proprietary software (:
These are static builds right?
Unfortunately not, I tried statically building it but it seems at the very least PostgreSQL which is a dependency fails to build that way (there’s an open issue here). I guess a lot of software simply isn’t tested for static building because it’s usually a giant pain to do, since you need each transitive dependency built as a static library to end up with a static binary in the end.
But Nix binaries are linked in a way that they will load exactly the shared libraries they were built against by their store path (in most cases), so effectively it should behave the same and never load incompatible libraries. It just means you can’t copy it around and need to leave it in the store.
A bit OT, but what are the storage requirements for running your own instance? I am assuming things can quickly get out of hand depending on what your server is federating with.
Short answer, not that I have seen. You could always build/package the .deb’s yourself and provide them to the community, otherwise it’s time the devs probably don’t have for a niche use case.
You can always run another machine or traditional VM for docker (or kubernetes). Otherwise, building from scratch seems your best option.
Also, docker (or in my case kubernetes) runs just fine on a Raspberry PI (using a Pi 4 4GB). In my experience most of the overhead comes in the form of storage, since different images require difference base images which start to add up. Otherwise I haven’t noticed any noticeable overhead (except with my Ceph Rook setup).
True, I’ve never had my own build server before… maybe this is something I can start looking at.
