I’ve been putting it off for over a year: our home server is running an old Ubuntu Linux installation, and needs to be upgraded. Research and teaching and life intervened, and this upgrade fell farther and farther down the list. In the meantime, Ubuntu produced two stable releases… making the chore of upgrading twice as hard. Let’s take a look at some lessons I learned by not upgrading, and my state of mind as I proceeded through the first phase of the upgrade today.
Lessons from NOT upgrading
One of the key lessons from holding back on upgrading was the following: when they stop updating kernels for your old release, you’re playing a dangerous game of kernel roulette. What bugs might be left in that last kernel they pushed to the old release? Well, I learned recently: the “Out-of-Memory Killer” kernel bug. A well-documented bug was left in the last range of kernel versions released to Ubuntu 14.04, wherein a Linux Kernel aggressively kills large-memory programs because it perceives a lack of system memory, even when plenty is available.
It took me weeks to track this down. It began with having to write a script to recover strange failures of our Gluster disk, the one shared across computers in our house. From log files, I could see that processes like Spamassassin were occasionally being killed by the Kernel. All very strange. I expanded the size of my “swap” file (a disk-based memory overflow area). Nothing helped. Finally, I went in search of kernel bugs. By this point, I could not keep our Gluster disk up for more than a few minutes and any significant file input/output operation led to its murder by the kernel.
So the lesson here is: grab upstream kernels, even if you’re not going to upgrade your OS. But, seriously…. upgrade your OS.
Other problems that cropped up in the last 6 months:
- The PHP version on my server was so old I couldn’t even do basic things like upgrade the Dokuwiki software I run there.
- OwnCloud server versions were pretty unhappy with running on something as old as Ubuntu 14.04, and we rely on OwnCloud for distributed and cloud-based file sharing and hosting.
The upgrade
You can follow my activities today (still not complete!) from my tweets:
What now?
Where am I now? Email is all working fine again. Most basic web services, like blogs and Dokuwiki, seem to be behaving themselves. My social media instance of GNU Social seems to be OK, and tbe bridging to the Pump.io instance is also fine. I am presently backing up my old OwnCloud data folder before deploying a new OwnCloud server, and trying to get Diaspora* working again.
And I still have to make the leap from 16.04 to 18.04…!
