How I Accidentally Fixed My Server by Doing Everything Wrong

Well, here’s a tiny update from the last few very full days.

The trouble started when I decided to upgrade my server from Debian 12 to 13. And wow. Things broke. Things got weird. Things that had no business changing suddenly developed opinions.

The first three times I tried, I had to crawl back to my backups and snapshots because everything went sideways, no matter how many “best practices” I followed like a responsible adult.

Then I saw some guy on omg.lol saying he had upgraded his server in the most chaotic, “whatever happens, happens” way possible — and it worked just fine.

So naturally I thought: great, maybe I’m just an idiot. Let’s try again.

And somehow, by doing it the wrong way, it worked. Not only worked — it worked better than expected, which is frankly insulting.

I also learned a few painful lessons along the way. One of them being that I’m apparently not as good with Unix-like systems as I thought. While I was fighting the upgrade, someone was trying to brute-force my system and DDoS it at the same time. Beautiful. Very relaxing. Exactly the kind of spa weekend every sysadmin dreams of.

Turns out they were trying to break into the company’s WordPress blogs. Incredible. Truly, humanity continues to innovate in the dumbest possible directions.

So I threw Fail2ban at the problem, hardened Nginx’s response to the attacks, and after a few hours they stopped. They haven’t come back since, which I’m choosing to interpret as victory and not just them taking a coffee break.

Closed-source software continues to be a problem, as usual. Though, to be fair, I used to defend it. That’s a story for another day, preferably one with drinks.

After the Debian 12 to 13 upgrade, I somehow ended up with no network, which was… bold. Very avant-garde. So I had to wrestle with VNC just to get back into the VPS and figure out what was still alive.

In the end, I also migrated from OpenClaw to Hermes, because apparently one disaster wasn’t enough and I wanted the full tasting menu.

Anyway, that’s the short version of a “quick little server upgrade” that politely stole several days of my life.

Webmentions

No webmentions yet. If something links here and sends a mention, it will show up after the next site build.