Nowadays I build things so that they move and I have moved things about a bit so I know they work.
Quite. I'm old enough to remember machine uptime being a badge of honour.
However, being older and not really wiser, I look for service uptime these days. Yes we did have similar back in the day, that's why MX and the like DNS records exist.
Old school clusters were pretty esoteric but the lessons were learned (split brain n that) and that's why we still argue the toss with kiddies about why a Proxmox cluster with two nodes is fucked and why we recommend an additional "witness".
I don't care that VMware glossed over the whole two node HA cluster thing years ago with a massive bodge. They were wrong then and they are probably still wrong because that nonsense is probably still baked in.
Sorry, slight digression.
High uptime implies no patching. We all love patching.
The more you know!
>a Proxmox cluster with two nodes is fucked and why we recommend an additional "witness".
Reminds me of the three Magi from Evangelion: https://magi.kinta.ma/
For a physical machine which you can't easily copy, it's a different story.
Mostly ~/apps/appname, where each appname has a docker compose file, and the data directories mounted under appname... I can compose down and (s)ftp the data out for hard archives or to move a site/service. I had been running a few VMs under a dedicated server, but switched to separate VPSes on OVH. Only gotcha with OVH is if you want to run mail, you want to avoid the local zone VMs that don't allow mail hosting.
YMMV
I think FreeBSD has some interesting technical merits, but like it or not Docker is the default for a lot of open source software, and I have neither the time nor the inclination to translate everything to freebsd jails.
Note, I did try Artix, but when it broke last week after a restart (in which evidently something had gone wrong with an earlier kernel update), and I had to pull out a rescue ISO, I decided I didn't want to mess with that. I switched that machine to Devuan, but the jury is still out for me. I don't have any major complaints, but I'm still in the burn-in phase. :) I'm running Arch on a laptop, but they have been a bit hostile in the community with censorship, so I'm just waiting for a free weekend to blast it and put something else on. I don't want political drama in my software.
This all comes at an interesting time, though. This is the first time that I purchased a new laptop and didn't even let it boot into Windows, but instantly installed Linux. And everything "just worked". And now that I'm excited to try Linux, so many of the big players are embracing the steps to erode privacy (AI everywhere... age attestation/verification... telemetry on by default...). It's sad, and I'm just going to "nope" out of any interactions with them.
I use Debian now much more. With all the supply chain attacks, Debian Stable feels like an absolute jewel, even if there are always a few packages I need to handle separately because I want or need a more updated version. But I love the old school no-nonsense engineering ethos.
You've been misled.
I've also got a couple of test systems running Chimera - going to wait until it hits stable before relying on it too much though. Experimenting a bit with AerynOS too.
- PM2 was buggy on FreeBSD, which I used to manage my processes
- An alternative, using `rc.d` to run daemons was just so hard to get logs working.
- The firewall required too much self configuration to get it right with all the best security practices (ie. What does one do with ICMP.) I was missing something like a template with the defaults that come with UFW, for instance.
FreeBSD does include this! It's implemented using IPFW instead of PF. Check out `firewall_type` key in `rc.conf`: https://cgit.freebsd.org/src/tree/libexec/rc/rc.conf?id=8e08...
For a very easy single-machine firewall, one could set `firewall_type=client` or `firewall_type=workstation` if you want to host anything. For the latter, `firewall_myservices` and `firewall_allowservices` control what ports are enabled and who (other networks/IPs) have access to them.
For a very simple NAT gateway, one could set `firewall_type=simple` and then `firewall_simple_(iif|inet|oif|onet)(_ipv6)?` to configure the ISP-side and internal-side interface names and IPv4 and IPv6 network ranges for each.
For more details and to see exactly what each option actually does, check out `/etc/rc.firewall` where this is all implemented: https://cgit.freebsd.org/src/tree/libexec/rc/rc.firewall?id=...
For supervision?
> - An alternative, using `rc.d` to run daemons was just so hard to get logs working.
The unix way is to use logger(1) If you only want some simple message, or redirect to files using newsyslog(8) for managing the sizes of the files.
> The firewall required too much self configuration to get it right with all the best security practices (ie. What does one do with ICMP.) I was missing something like a template with the defaults that come with UFW, for instance.
I would recommend The Book of PF[0]. While FreeBSD has syntax difference with OpenBSD's pf, this should give you enough insight on how a firewall operates to get a sense of what rules to write.
For a while I used CentOS 7 on all of those small VMs, because it got security updates for a really long time. With minimal risk of breaking things on updates.
PS: after a bit of research Alma/Rocky Linux are probably the best choices for now. 10 years of support. But are they maintained well?
Alma has a few affordances as it's no longer RHEL source compatible, which means it could ship priviledge escalation fixes with new kernel updates faster.
Rocky responded with an extra, optional to enable, security repo to provide mitigations to the exploits while waiting for RHEL to downstream.
Look pretty well maintained to me. If only judging by recent events.
The manuals, indeed are good, though for more esoteric issues I land too often on a gated answer page.
I don't care much about being fully RHEL compatible, or no ABI changes at all. I just want a system that gets security fixes quickly with as little chances of breaking things as possible.
Then I figured that version upgrades were miserable, not just because I had painted myself in a weird corner with ungodly packages mix-ups, but because the upgrade path was always best-effort. I think I gave up during the 6 to 7 transition, as I realised that all I needed was fedora: with yearly or half-yearly updates I have no need to fight the distro's packages: stuff stays current and in working order, major distro upgrades go smoothly, downtime is minimal. I'm not considering going back to any "server distribution" ever.
I have been running NixOS on several servers for more than a decade. No reinstalling, upgrading, or any breaks whatsoever.
It has been the most headache-free Linux I've used, simply because I'm less scared to play with and fix stuff. The fact that rollbacks are trivial and snapshots are automatic, and since everything is declarative in a text file anyway, I am way braver. If I do something like screw up the video driver, or the wifi driver or make it so the system doesn't boot anymore, all I need to do is reboot and choose a previous generation.
But I have had, uh, non-trivial breakages happen also when I upgrade the system itself to the next yearly release. Non-bootable kernel kind of breakages.
But I will give you that I can just boot from the generation before the upgrade, and it works again. So there's that :)
RHEL is definitely the most stable major distribution. Alma and Rocky are essentially downstream clones of RHEL.
ELTS is 10 years and paid. It's great that it exists, but not relevant for my toy projects.
I haven't used a lot of different distros, but for me, Debian has been a good balance of those factors. You may need to do more upgrades per decade, but the ones that you do are more liable to go smoothly.
Just my 2¢ on the topic (:
Either the 1-2 hours is a drop in the bucket compared to what you spend on it anyway (like a blog you still regularly update), or you don't actively update the project but still care enough about it to spend half an evening every few years, or you should just admit you don't care about it enough anymore to do even that. In the last case just delete the project.
I've upgraded Debian stable (both pure and with some cherry-picked backports) and Ubuntu (non-LTS and LTS) systems in place and rarely broken anything, for years and years. When stuff has broken it's been a quick google and then slapping myself for not having read the upgrade guide.
I do generally wait about 2-3 weeks before upgrading, giving time for them to catch stuff that was missed until the great masses were set loose on it.
Not the OP, but I support Ubuntu as desktop and server OS for an engineering collage and have for 10ish years. Some LTS upgrades don't require many changes (mostly minor package name changes) and some take months of work to get rolled out (mostly for workstations, the server upgrades are usually quick.). Not everything gets upgraded every new OS release. If we had to upgrade everything every 6-12 months it would eat up a significant amount of time for our small team.
If you use Copr (Nvidia Drivers, Non-Free Stuff) you need to ensure all your Copr packages work fine in the next version of Fedora. A ton of packages haven't been updated for Fedora 44 and this will cause issues.
The same applies if you use Terra
I've had issues with Ubuntu/Debian upgrades more than once. Some third party binaries breaking with the update. Or some specific config tweaks that break, because the structure of /etc changed too much.
For some small VM with a specific purpose I prefer a distribution that changes as little as possible for as long as possible. Less work, more uptime.
The naming conventions drive me crazy as well. When you deal with 2 things that have dumbshit naming conventions, like ubuntu and ROS, its really obnoxious to pretend to case enough to keep track of.
For updates, Debian and Ubuntu are great. For upgrades… not so much for me.
It used to be hard to find dedicated servers or VPSs with any of the BSDs, I think I settled on Panix.com or something?
Before that I remember some company called 15MinuteServers (NAC?) out of NJ I think that offered them. Just kind of rambling down memory lane at this point though.
Be careful if you have anything that autostarts that reaches out
kqueue was a huge win too.
A huge thank you to the FreeBSD developers. I ran my first company for 15 years on FreeBSD with incredible uptime and resilience.
I just setup Semaphore the other night which adds a web UI to manage Ansible playbooks, it works like this:
1. I host my own Forgejo git repos
2. Semaphore is granted access to the Ansible repo
3. FreshRSS notifies me when a service I am running has new release
4. Check the release note, then run Semaphore to run the ansible-playbook
I could fully automate it all but I have the need to read release notes.
As for the OS, they are Debian 13 Netinst and fully local only, I could run them until the services can no longer run, which the ansible-playbook can spin up another LXC container running Debian 14 or whatever.
The goal is to automate everything as much as possible.
The applications run just fine, but I don't even know where to start. Apparently I coded them directly into the server, no dev machine!
[0]: https://cdn.idiallo.com/images/assets/daily/98/old_servers.j...
My guess would be that fastfetch probably reports actual memory usage while btop probably reports the total usage of all processes. The former is probably higher because of things like filesystem caching
If someone is willing to use something like Hugo instead of garbage sites like Medium why not use a VPS? For many people working in tech $10/month and free are the same thing.
And yes you can have preferences to keep things simple while others can make something unnecessarily complex. For personal projects this is fine and part of learning. If you had said “I much prefer… because…” it would have been fine but you said “objectively inferior in every meaningful way” which ignores people’s subjective preference for over engineering hobby stuff for learning.
On the other hand, dedicated servers are more fun, even if the cpus I get for $35/month are ancient. Is it $30/month fun? Probably not, but it's near zero given my situation.
At least I'm sensible and don't have an actual colo space to visit.
In the unlikely event I go viral, I'm pretty sure my server can manage serving https at 1gbps, and that's plenty. Maybe TLS is too much though, the cpus are too old for AESNI.
On GitHub?!
There's additional concern with tying your work to something like github it makes it more of a pain to pull it off and put it somewhere else.
I'm not really sure what you mean by objectively inferior. It's trade offs like everything in this field.
As far as harder, I don't really think the lift for a personal VPS is that high. Again it's a fun hobby project for most. It's fun to run your own stack.
If you want to opt into the github cloudflare goodness that's fine they're good services but I wouldn't say it's better or degnegrate others for not doing that.
You will never win this crusade, because there are too many people here who know from experience a VPS is neither expensive, nor under-performing up to millions of users a day, nor hard.
You can put it behind cloudflare for free.
Many of these small VPSs can be had for less than a couple bucks a month. Tons of popular influencers run their own machines for their blog.
insinuating that it's unsafe to run your own machine is insanity. I don't understand this mindset of being scared to run your own stuff. Especially if you're doing doing it at such a large scale there's nothing wrong with doing it with nginx and a linux box on a vps. You'll learn a hell of a lot more and be fine. At the end of the day it's a computer. We've been hosting websites since the 70's. With the advant of cloud compute is easier than every to run your own.
(edited to be less mean)
I guess if you want to call being informed about the online threat landscape "scared", that's your perogative. For me, it's common sense to avoid completely unnecessary threat vectors to my digital infrastructure, but power to you if you like dealing with extra maintenance overhead and constantly wondering whether you're providing free cryptomining to some random international criminal.
I'm also not sure that I really need a CDN for a simple blog . I'm not going to benefit from the caching as it's not video or images.
The flexibility and learning is more important for me. For example I want to aggregate HN comments and lobste.rs comments and inject that into the HTML before serving. (on the server side so no CORS or other additions)
I was considering adding additional metrics to see who is hitting the server and how at the reverse proxy level.
This is all stuff I can't really do on a github pages blog.
I see what you're saying if you want set and forget that's fine, but like I said above it's a tradeoff.
The one server I have just has 80 and 443 open with nginx. I expect it to run indefinitely with little maintenance.
I've owned and operated enough stacks e2e both personally and professionally to have gotten over the novelty. The less shit that can go wrong, the better. I sleep better at night not wondering whether any of the constant stream of IPs in my fail2ban log is wielding a yet-to-be-CVE'd zero-day, or finding out that my site has been down for 6 weeks because of some fucking stupid bug in the latest kernel patch or whatever.
It's an incredible journey to take--whether you stick with it or not. Migrating to FreeBSD gives you new eyes into what Linux was, is, and the awesomeness of FreeBSD that is so hard to articulate; like describing the color blue. It must be taken as a whole to appreciate it; and I'm not just saying the OS, it's commands, kernel features, but, the end-to-end compute experience, over time.
If I could draw an equivelent, it would be like when Djistrka savagely destroyed the GOTO statement with a single, short, paper. It took a brilliant mind to articulate that and there has yet to be such a mind to describe the beauty of FreeBSD. So, the best I can do, is just to challenge you to try it.
