I’ve been a Software Engineering Student for 2 years now. I understand networks and whatnot at a theoretical level to some degree.
I’ve developed applications and hosted them through docker on Google Cloud for school projects.
I’ve tinkered with my router, port forwarded video game servers and hosted Discord bots for a few years (familiar with Websockets and IP/NAT/WAN and whatnot)
Yet I’ve been trying to improve my setup now that my old laptop has become my homelab and everything I try to do is so daunting.
Reverse proxy, VPN, Cloudfare bullshit, and so many more things get thrown around so much in this sub and other resources, yet I can barely find info on HOW to set up this things. Most blogs and articles I find are about what they are which I already know. And the few that actually explain how to set it up are just throwing so many more concepts at me that I can’t keep up.
Why is self-hosting so daunting? I feel like even though I understand how many of these things work I can’t get anything actually running!
Theory + experience => skill
I say this because I am not sure how much of what you find “rough” comes from you not being familiar with it and how much comes from real hurdle (not that getting familiar with a subject is not a hurdle in itself).
Those are nice to get an idea but come on, read the doc. Do spend that time. Unless the service was extremely well designed and small in scope, getting a serviceable mental map of how it works can take days (or weeks if you really have no clue). There is no real shortcut to this. If the doc is not good enough, either walk away or engage with the dev/community to get things right.
It might be the first time in your life where you are confronted to having to get things perfectly right for a service to work at all (I say this neutrally, without meaning to offend) but this is how it is: this is what “work” means.
When I say at a theoretical level I mean I’m familiar with it from University lectures and reading about what it is, but it is true I’ve never actually tweaked my networking in a practical sense enough to be familiar with it, which is exactly why I want to get into self-hosting.
As for the docs, I read them, I truly do. But docs are not where you find how to do something, is where you find how to implement it. By this I mean, if I wanna setup an authoritative DNS server, I need to find how I set one up. Once I know what software I need to use, I can read the docs to figure out how to wield said software. Just stuck on the step before being able to dive into the docs (or stuck on having too many docs to read, no middle ground)
No, they are. All the other resources are shortcuts to get a proof-of-concept out there more quickly (or to get one common case implemented quickly). Actually the best-written software bring you to PoC stage from the get-go. E.g. for OpenSSH, just install the package and start the server. Now it works and you can peruse the rest of the doc to do what you want.
Yes, so you read the doc. That’s a pretty unfortunate choice for an example because bind has excellent doc, walking you through an introduction to DNS and Bind itself, then what machine you need to run it and then how to get your PoC. The rest of the doc is all about how to wield it correctly, as you said.
I am aware that some software does not document nearly all of that but the vast majority brings you to PoC state without trouble before you can tune things, perusing the rest of the doc.
Well, you’ve proven my point. In order to know how to setup an authoritative DNS server I need to read the docs for bind. But in order to know bind is the answer to my problem I need to read articles and blogs. There is no way to go from Authoritative DNS server to bind without reading some more on the internet in blogs and whatnot.
Once I know about bind, I can read it’s docs to set it up or to figure out if it’s the right thing for me, but I need to know about it first.
I only ever use something other than the docs when I’m either looking for something more specific than the docs, the docs suck or I can’t find it in the docs. Really not against reading through them at all.
But with a lot of programs that’s also an issue cause a lot of docs just expect you to be familiar with that area of knowledge (at least with some libraries I work with such as Spring in Java, which assumes constantly you know about HTTP and APIs when explaining how to set an HTTP API with Spring. Not saying it’s bad, you probably need that background knowledge anyway, and the doc writers cannot be bothered to bake it into the docs, but it gives people who are completely clueless like me more and more homework in a snowball that becomes quickly unmanageable)
Yes and no. Either you have the experience/knowledge to know what you need (e.g. “I need an authoritative DNS server to solve problem X”), in which case it all comes down to “what are my options?”, which any search engine will gladly answer (and the doc will detail without ambiguity whether or not it’s a good fit). Or you don’t, in which case you either need to build that knowledge or you walk away.
Blogs and articles get stale very very quickly and very often, they are not written by competent people. In the grand days of “host your own mail server”, this very thing has led to so many open relays that ISPs and server providers started to block SMTP by default, with convoluted steps to unlock it.
Yes. It’s just like a mechanics does not like to read literature explaining for a 1000th time how an engine works or a surgeon who loathes having to go through an anatomy lesson every time they try to read content. If you don’t know what you are doing, learn about it first. Often, a wikipedia article and a couple more random reads go a long way towards understanding these kinds of things.
Also with time, your area of soft expertise expends. I have never had to configure bind as an authoritative DNS, yet I know what DNS is, roughly how it works and how to navigate to the right places to get the specific info I need quickly. That’s what experience brings.
Why do you think people have do different trades and learn about these kind of things at Uni level? That’s the point of any degree or education system actually: you build an understanding, lesson after lesson, year after year, just to be able to understand/manage what you are going to be taught next.
If you/I need to perform heart surgery tomorrow, your/my patient will undoubtedly die. Or phrased differently, it gives people who are completely clueless at heart surgery, like me, more and more homework in a snowball that becomes quickly unmanageable. That’s how knowledge works. And as I’ve been repeating (again, without offense), you either learn about it, step by step, or you do something else.
These things are not easy and getting competent/skilled at them requires work.
I might be very wrong about what I am going to say, and again, this is not a slight, I don’t mean to offend anyone, but it looks like, so far, you were able to find articles and blog posts about most of the stuff you wanted to achieve. So you’ve been mostly following tutorials. It’s rewarding: you get things working. But the problem with said resources (besides going stale quickly or not being competently done) is that they are made, originally (before it’s copied 100s of times by copycats), by people who took the time understanding how things work and do the hard work for you. Now that you want more bespoke or niche things, you seem baffled that no one had written a convenient tutorial for you, even to get yourself started. But the “tutorial world” is an illusion. The best way of doing self-hosting with minimal amount of frustration is by getting up to speed with all that background knowledge first or as you go, this is how you go beyond “tutorials”. I am aware this is not how you phrased it but tutorials are the only way “clueless” people get something working. Most things out there are not written for clueless: they assume a minimum amount of knowledge.
Finally I want to reiterate that I am not judging you. I stand by what I said: it is hard and not everyone want to spend time getting experts at those things. It’s fine. Conversely, if you really want to and stick with it, I am certain you can achieve that level of expertise and, in a couple years, maybe, look at that post again and think “I really got upset over little there but it’s true it looked insurmountable back then, good that I stuck with it”. :)
I wanna preface this making something clear, I have probably never managed to follow a tutorial. I usually have an idea and try to run it. What I’m getting stuck at is precisely at the “what are my options”
I did say that I understand why docs expect you to be familiar with knowledge, I’m even complaining that I don’t need to read what something is a million times and sometimes I just need to get a solution to my problem.
My biggest issue here is lack of experience, but after two years in an IT education, I know some basics. I am familiar with countless topics and have a general idea of how things work in theory, but getting these ideas into a setup is what’s hard. Uni doesn’t help at all with this.
I think my problem is quite honestly the opposite of what you present, I need to know deeper knowledge than what’s enough and have trouble actually conceptualizing things that are presented as magic. For example, Docker presents itself as magic in most of the docs. Volumes, layers, and so much more are explained with how to make one and what to use them for rather than what they are. It might just be the way I memorize things is weird, or that I’m stuck with too little knowledge and way too deep.
I do mostly agree with your comment though, I just feel like I’m shit at explaining myself cause I’m clearly not getting my point across