Part of working with tech is knowing what to search for und using the right keywords. If you could give an example what guide / information you are unable to find, someone could give you an example how to search for it. I personally know a few junior devs and junior devops which use llama2 / chatgpt since they dont know how to search for it or read the docs.
this post was submitted on 26 Oct 2023
13 points (93.3% liked)
Self-Hosted Main
502 readers
1 users here now
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
For Example
- Service: Dropbox - Alternative: Nextcloud
- Service: Google Reader - Alternative: Tiny Tiny RSS
- Service: Blogger - Alternative: WordPress
We welcome posts that include suggestions for good self-hosted alternatives to popular online services, how they are better, or how they give back control of your data. Also include hints and tips for less technical readers.
Useful Lists
- Awesome-Selfhosted List of Software
- Awesome-Sysadmin List of Software
founded 1 year ago
MODERATORS
I actually hate GPT, dislike it’s answers and find myself knowing better than it most times.
I’ve been trying to setup a DNS server to create my own domains internally within my VPN but I keep finding info on how DNS servers work, and how to make a records on registrars, but nothing on what I actually need to install and run to have my own DNS for example. Same thing goes for many other services, but that’s the one bugging me for the longest time because it should be so simple.
I’ve found plenty of tutorials on how to make a cache DNS, just not an authoritative name server btw, and I’ve searched for both DNS and name server to no avail. If it was Linux I’d write some custom rules in my hostfiles and be done with it, but it’s so much harder to do on Windows and that’s my daily use OS for now…
"Selfhost authorative dns" returns this for example https://wiki.selfhosted.show/DNS/
And we have two guides for the most common dns implementantations. Depending on the vpn software you can push the dns ip as part of the configuration or by using DHCP.
I also found the pihole docs for unbound rather helpful https://docs.pi-hole.net/guides/dns/unbound/.
load more comments
(8 replies)
I use Pihole for this in my home network. It has the capability to configure local domains for your internal network resolution as well as create whitelist/blacklist and use as a DHCP server alternative to your home router -- you just need to configure your router to point to the Pihole instance for DNS on your LAN network configuration to get started with the DNS piece
load more comments
(3 replies)
Each one of these things is a separate topic/project. There is of course, overlap, but concentrate on just one at a time. If you try to do everything, you'll go mad. Application Administrator, Network Engineer, Sysadmin, Network Security, are often separate full time jobs. Just concentrate on your own needs, one at a time, then focus on how simple you can pare down that one thing you need/want to self host. What is the minimum set of capabilities you must have for this one thing? Then, expect to have as much research for this one thing as taking a 1 credit college course, at least. Maybe more, maybe less, depending what it is. That's a more realistic set of expectations.
I've done a lot of web design and web hosting, and messing with Linux server stacks for around 30 years, now. You absorb a LOT over time, but it is all always changing. When I set up my first server, Cloudflare didn't exist yet. VPNs weren't a thing yet. The only people I knew with a network in their home were Computer Science professors. Wifi was not a thing yet. It isn't you being inadequate, it's you trying to do multiple jobs that each require constant professional development.
Check YouTube. I haven’t found a lot of written out guides but someone has made a video on it, I bet. Christian Lempa, Techno Tim, Db Tech, NetworkChuck come to mind for self hosting tutorials on containers and new applications. I’m sure there’s a lot more as well.
Thanks for the suggestions! The algorithms keep feeding me people who just explain what stuff is and it drives me nuts. You wouldn’t believe the amount of videos, articles and blogs I’ve seen on setting up a DNS server just for it to be about either a cache or an explanation of how it works. I’ll look into these later!
Jeff Geelong is another good resource on youtube, especially if you ever want to get into infrastructure as a career.
Github is also an amazing resource and always RTFM.
Edit: spelling
Maybe you should read the documentation for the programs you try to install
I can barely find info on HOW to set up this things. Most blogs and articles I find are about what they are which I already know. And the few that actually explain how to set it up are just throwing so many more concepts at me that I can’t keep up.
This is big part of why I, even as an IT professional, don't do much homelabbing or selfhosting. So often I'll be scrolling through these subs and see something someone has done and I'll decide I want to do that too, but so often I end up abandoning whatever it is because doing x requires googling for y and z because without y and z, x doesn't work. And after a day of dealing with issues and tinkering on work stuff, do I want to spend more time troubleshooting? Often not.
Because most people who develop these things are, frankly, terrible at good documentation, or understanding the end-user perspective.
There's also a downward spiral effect when you start getting into these things, because lots of them require dependencies, or ask you to do things but don't explain why, and you're just left wondering why you added that line to a config file somewhere, but if you don't put it there, nothing works.
A vertical slice of the amount of knowledge you need passes through so many different disciplines, operating systems, GUIs, and programming languages that it would look like a Milhojas cake.
I've been a technical writer in the software industry for 17 years. The number one challenge in my work is extracting all of the information I need to write good documentation from the experts elsewhere in my company.
You don't have to do it, 99.9999%^([citation needed]) of the population don't. If it doesn't interest or excite you then it's not worth it.
You're kidding?
I'm an avid home-labber trying to get into software engineering and I'm finding the software dev work to be super daunting lol
I can setup a reverse proxy in my sleep - but write a JS function without double checking google? forget it!
It comes with experience I guess, I’ve got a bad habit of researching to the core and many times have a hard time grasping things like containers without understanding how it’s setup technically. Sometimes I find a decent explanation, but specially for libraries that do “magic” I gotta go diving into the source to understand what’s going on, else I have trouble understanding what I am doing and what I should be doing.
Which makes it so hard because networking is very low level and I’m very unfamiliar with this environment
load more comments
(1 replies)
That's not bad
at a theoretical level
Theory + experience => skill
I say this because I am not sure how much of what you find "rough" comes from you not being familiar with it and how much comes from real hurdle (not that getting familiar with a subject is not a hurdle in itself).
Most blogs and articles I find...
Those are nice to get an idea but come on, read the doc. Do spend that time. Unless the service was extremely well designed and small in scope, getting a serviceable mental map of how it works can take days (or weeks if you really have no clue). There is no real shortcut to this. If the doc is not good enough, either walk away or engage with the dev/community to get things right.
It might be the first time in your life where you are confronted to having to get things perfectly right for a service to work at all (I say this neutrally, without meaning to offend) but this is how it is: this is what "work" means.
When I say at a theoretical level I mean I’m familiar with it from University lectures and reading about what it is, but it is true I’ve never actually tweaked my networking in a practical sense enough to be familiar with it, which is exactly why I want to get into self-hosting.
As for the docs, I read them, I truly do. But docs are not where you find how to do something, is where you find how to implement it. By this I mean, if I wanna setup an authoritative DNS server, I need to find how I set one up. Once I know what software I need to use, I can read the docs to figure out how to wield said software. Just stuck on the step before being able to dive into the docs (or stuck on having too many docs to read, no middle ground)
As for the docs...
No, they are. All the other resources are shortcuts to get a proof-of-concept out there more quickly (or to get one common case implemented quickly). Actually the best-written software bring you to PoC stage from the get-go. E.g. for OpenSSH, just install the package and start the server. Now it works and you can peruse the rest of the doc to do what you want.
if I wanna setup an authoritative DNS server, I need to find how I set one up
Yes, so you read the doc. That's a pretty unfortunate choice for an example because bind has excellent doc, walking you through an introduction to DNS and Bind itself, then what machine you need to run it and then how to get your PoC. The rest of the doc is all about how to wield it correctly, as you said.
I am aware that some software does not document nearly all of that but the vast majority brings you to PoC state without trouble before you can tune things, perusing the rest of the doc.
Well, you’ve proven my point. In order to know how to setup an authoritative DNS server I need to read the docs for bind. But in order to know bind is the answer to my problem I need to read articles and blogs. There is no way to go from Authoritative DNS server to bind without reading some more on the internet in blogs and whatnot.
Once I know about bind, I can read it’s docs to set it up or to figure out if it’s the right thing for me, but I need to know about it first.
I only ever use something other than the docs when I’m either looking for something more specific than the docs, the docs suck or I can’t find it in the docs. Really not against reading through them at all.
But with a lot of programs that’s also an issue cause a lot of docs just expect you to be familiar with that area of knowledge (at least with some libraries I work with such as Spring in Java, which assumes constantly you know about HTTP and APIs when explaining how to set an HTTP API with Spring. Not saying it’s bad, you probably need that background knowledge anyway, and the doc writers cannot be bothered to bake it into the docs, but it gives people who are completely clueless like me more and more homework in a snowball that becomes quickly unmanageable)
But in order to know bind is the answer to my problem I need to read articles and blogs.
Yes and no. Either you have the experience/knowledge to know what you need (e.g. "I need an authoritative DNS server to solve problem X"), in which case it all comes down to "what are my options?", which any search engine will gladly answer (and the doc will detail without ambiguity whether or not it's a good fit). Or you don't, in which case you either need to build that knowledge or you walk away.
Blogs and articles get stale very very quickly and very often, they are not written by competent people. In the grand days of "host your own mail server", this very thing has led to so many open relays that ISPs and server providers started to block SMTP by default, with convoluted steps to unlock it.
a lot of docs just expect you to be familiar with that area of knowledge
Yes. It's just like a mechanics does not like to read literature explaining for a 1000th time how an engine works or a surgeon who loathes having to go through an anatomy lesson every time they try to read content. If you don't know what you are doing, learn about it first. Often, a wikipedia article and a couple more random reads go a long way towards understanding these kinds of things.
Also with time, your area of soft expertise expends. I have never had to configure bind as an authoritative DNS, yet I know what DNS is, roughly how it works and how to navigate to the right places to get the specific info I need quickly. That's what experience brings.
but it gives people who are completely clueless like me more and more homework in a snowball that becomes quickly unmanageable
Why do you think people have do different trades and learn about these kind of things at Uni level? That's the point of any degree or education system actually: you build an understanding, lesson after lesson, year after year, just to be able to understand/manage what you are going to be taught next.
If you/I need to perform heart surgery tomorrow, your/my patient will undoubtedly die. Or phrased differently, it gives people who are completely clueless at heart surgery, like me, more and more homework in a snowball that becomes quickly unmanageable. That's how knowledge works. And as I've been repeating (again, without offense), you either learn about it, step by step, or you do something else.
These things are not easy and getting competent/skilled at them requires work.
I might be very wrong about what I am going to say, and again, this is not a slight, I don't mean to offend anyone, but it looks like, so far, you were able to find articles and blog posts about most of the stuff you wanted to achieve. So you've been mostly following tutorials. It's rewarding: you get things working. But the problem with said resources (besides going stale quickly or not being competently done) is that they are made, originally (before it's copied 100s of times by copycats), by people who took the time understanding how things work and do the hard work for you. Now that you want more bespoke or niche things, you seem baffled that no one had written a convenient tutorial for you, even to get yourself started. But the "tutorial world" is an illusion. The best way of doing self-hosting with minimal amount of frustration is by getting up to speed with all that background knowledge first or as you go, this is how you go beyond "tutorials". I am aware this is not how you phrased it but tutorials are the only way "clueless" people get something working. Most things out there are not written for clueless: they assume a minimum amount of knowledge.
Finally I want to reiterate that I am not judging you. I stand by what I said: it is hard and not everyone want to spend time getting experts at those things. It's fine. Conversely, if you really want to and stick with it, I am certain you can achieve that level of expertise and, in a couple years, maybe, look at that post again and think "I really got upset over little there but it's true it looked insurmountable back then, good that I stuck with it". :)
I wanna preface this making something clear, I have probably never managed to follow a tutorial. I usually have an idea and try to run it. What I’m getting stuck at is precisely at the “what are my options”
I did say that I understand why docs expect you to be familiar with knowledge, I’m even complaining that I don’t need to read what something is a million times and sometimes I just need to get a solution to my problem.
My biggest issue here is lack of experience, but after two years in an IT education, I know some basics. I am familiar with countless topics and have a general idea of how things work in theory, but getting these ideas into a setup is what’s hard. Uni doesn’t help at all with this.
I think my problem is quite honestly the opposite of what you present, I need to know deeper knowledge than what’s enough and have trouble actually conceptualizing things that are presented as magic. For example, Docker presents itself as magic in most of the docs. Volumes, layers, and so much more are explained with how to make one and what to use them for rather than what they are. It might just be the way I memorize things is weird, or that I’m stuck with too little knowledge and way too deep.
I do mostly agree with your comment though, I just feel like I’m shit at explaining myself cause I’m clearly not getting my point across
Senior DevOps Engineer here with 15+ years of experience in system engineering now, even with my skillset and experience I am improving my setup piece by piece.
When I look at my old docker-compose versions and general setup, I sometimes laugh how I could've been so dumb.
You don't have to and won't learn and set everything up at once, ever.
I've studied automotive and never really graduate, but I love reading manuals since I was a kid and always believe that even I don't know what am I doing as long I can read It's manual, guide, documentation and read support I could get a grasp of it even if it takes more time.
A part that made it easier for me to understand self-hosting is the raspberry pi. It let me thinker self-hosting without worrying about power bill, how can I easily reinstall its OS even if I make bunch of mistake, make different copies of my setup on different sd card without worries of hdd or ssd.
Then from that just upgraded stuff from what I've learned.
I wish I could afford a Pi. Would be so cool. Unfortunately I’m stuck with my gaming PC from 6 years ago that I recently updated from. Much more powerful hardware but I can’t just swap out the drive or not worry about power usage sadly haha.
Still, I’ve had to reinstall the OS about 8 times last year alone haha, but we’re still learning (most because I forgot the password tbh haha
Most blogs and articles I find are about what they are which I already know.
Exactly.. I neither want "what z is" nor all the history behind z, the Z inventors life, his grand parents lives etc. I just want a solution to the problem im having..
Honestly, I get they’re trying to be educational for beginners way more clueless than me. But after two years of an IT degree I know some stuff, and the sheer amount of internet text I’ve read just to find absolutely nothing new and no solution even though the title is exactly my problem is unreal
load more comments
(1 replies)
Self-hosting is like any other topic in computing. You have to read, read and read. If you want to self-host a reverse proxy, go to the official documentation of the reverse proxy. What you are doing is searching for magic recipes in blogs and YouTube, that is why you feel it so "daunting".
Quite the contrary, I’m stuck at finding a reverse proxy in the first place. If I didn’t know nginx had a reverse proxy, which is the only one I know about, where would I even start finding the docs? I can’t repeat this enough, but I rarely ever do tutorials, I find them basic and lackluster
I don’t think I need specifically a reverse proxy rn so I don’t really have a clue about that kind of service specifically, but even finding WHAT to use to do an authoritative DNS was a challenge in its own right that I only solved somewhere else in this thread.
I'd say, start with a need and not with a tool. Tools are much easier to understand once you know the problem you have to solve.
I implement everything with the help of chatgpt without problems. Of course, as the service on github has a lot of issues I skip it.
The responses you get here are from people that have already perfected their setup so take them with a grain of salt.
I mean I have my setup running with Traefik, Crowdsec, Authentik and that's before the request even reaches the application.
2 months ago that was only Traefik.
A year before that I was using Nginx Proxy Manager instead of Traefik because it was easier to manage and understand.
Half the fun is evolving your homelab. Trying to start out with the full stack of things someone suggests is daunting and nearly impossible.
Take things one step at a time. And honestly if you don't understand what the documentation is talking about, YouTube videos are great. I've had to use it lots to understand how Authentik works but now I understand the docs
I would say the opposite: self hosting nowadays is very easy. Is the multitude of options and configuration possibilities that is daunting. We want too much and this increases complexity.
load more comments
(1 replies)
Because that means nothing?
"Why are computers hard"
"Why is car hard"
Maybe figure out what you're trying to do? "I can't figure out VPNs" is a pretty weird starting point considering you can just google VPN and click literally anything
While this post is a bit of a generic rant, I do know what I’m trying to do, and the issue is that as soon as you go into anything slightly more complex than setting up a VPN, you’ll be bombarded with a thousand words that barely mean anything, everyone and their mother has a different opinion on what’s optimal, minimal and desired and to top all of that, most resources out there focus on making you understand what things are rather than how to set it up.
My issue and what I was ranting is why is most shit on the internet so unhelpful, hoping to find someone who’s had a similar struggle and learn how to get better. And I’ve succeeded, many people have given me useful advice.
I never said anything remotely as vague as “computers are hard” I think my post clearly states my issue is with resources being unhelpful for complete beginners
I am right there with you. I am a Mechanical Engineer and if we tried putting out a product with the limited documentation that these come out with, we would be taken off the project. ME projects have documentation meant to guide a 10 year old through the process and details. The documentation would fill a 3" 3ring binder.
Yet these network products are released with 1 page of upper level description of the product. Yeah I already know what the product already does, that's why I got it, how about the actual documentation?
This is a swiss army knife of hobbies. I would say that yes it can be hard to get into if you try to do everything at once. I would pick one to three goals you want to accomplish and do those items. Once you have those documented and implemented, then pick up another task.
The amount of times I've completely rebuilt my home setups is... many. But to get where I am now, I would never have been able to plan it properly without the learning experience of "failing" so many times. And there will always be a better way to do it as things need upgrading so in a way it is perpetual. Start with the projects you want first. Get them working and make sure you can export the data properly before you begin to rely on them. Then once you have small bits done, add extras, like cloudflare, vpn, portainer, proxmox. There will be a lot of formatting and reinstalling and going back to the drawing board, just accept it as a learning experience but take each piece once by one. If there is a specifc
yet I can barely find info on HOW to set up this things.
There are plenty of guides out there that makes assumptions that you should already know things, which can be frustrating, I found this especially true with docker related documentation. If you find yourself on a github page, check the Releases tab. If it's still confusing there, check for a docker folder in the github and search "docker ". Start with popular more documented projects that will usually include beginner instructions that less popular software will omit. Even then if you keep looking you can usually fine one good doc that clears things up.
Also, in this realm, chat GTP excels at obscure configurations so it's worth giving it a shot when stuck.
If you want security and ease of use for remote access, just use Tailscale. Twingate if you share access with others.
That doesn’t even scratch the surface with the issues I face…
Its only daunting if you want to do everything from scratch. Depending on your needs you could easily build off an existing platform for home labs and servers. Its more of a tinker / experiment / use existing, then reverse engineer the parts you need to.
A good article on Home Server OS is here:
https://www.smarthomebeginner.com/best-home-server-os-2023/
You can Find some good Self-Hosting projects / apps here:
https://selfhosted.libhunt.com/
And finally, here are two examples of decent, simple to install all-in-one solutions, Yunohost is my go to for friends who want to start self-hosting as it sets up everything including SSO, DNS (via Adguard Home) and Certs via Lets Encrypt all in a nice web gui and admin panel.
Yunohost:
https://yunohost.org/#/
HomelabOS:
https://homelabos.com/
As with everything in life you need to crawl before you run. And as someone mentioned before skills issue. You can assume most people who self host are Linux engineers / Devops engineers. So they do these kind of setups daily. This is why these channels exist. So we can assist or guide. So you can learn as well.
I feel like Self-hosting is like playing the game of life on the hardest mode possible.
The easiest is always to pay for services or let others have the responsibility. That's the Easy Mode.
When Self-hosting, mind blowing fucked up things will happen, your head will sometimes explode because you will doubt everything you learned, things that always worked will stop working for no apparent logical reason.
Maybe you'll understand that some data was corrupted, maybe you backed your servers up, maybe you didn't, maybe you can recover data, maybe you can't.
Sometimes you will feel amazing for your achievements, sometimes things just stop working even for things you won't control. ( Energy and Internet Suppliers ).
Most people even with high curriculum and studies opt for the easy mode. And that's okay.
Others enjoy the struggle and the amount of knowledge that comes with that.
Choose your path, and stick with it.
Pros of Self-hosting: self enlightenment, joy of achieving great things, freedom, saving money, power.
Cons of Self-hosting: Time consuming, stress, dangerous if not done with responsibility, requires patience and perseverance, may be actually more expensive on your personal and professional time, if your time can be indeed valuable. You may be able to actually sell more, or be more profitable if you don't lose time thinking about the hosting itself.
I’d just install Docker and add a CloudFlare Tunnel to securely host your stuff. No need to open any ports, your WAN IP remains hidden and you have the benefits of DDoS protection. It’s free as well; all you need is a domain name for which you can configure the nameservers.
load more comments
(1 replies)
Invest some time in understanding Docker containers and then install something like Portainer (for example on top of Proxmox + VM, or ProxMOX + LXC, or some Linux or bare metal) and then installing new apps will become very trivial once you understand how to "create" the containers and how to map the volumes and ports.
view more: next ›