One Power Outage, One Bricked Machine: The Complete Rescue Log of an Interrupted Ubuntu 26.04 Upgrade

Introduction

The National Day holiday is finally here! Staying home tinkering with my HomeLab, K8s, and NAS server clusters feels great! I really got my hands dirty with a lot of things this holiday — readers who are interested should look forward to the upcoming flood of posts. If you can’t wait, you can also go straight to the PRs and commits in my repo east4ming/homelab2.

During the National Day holiday, while everyone else was packed shoulder to shoulder at tourist spots, I was at home upgrading my HomeLab cluster — taking several nodes from Ubuntu 24.04 to 26.04.1.

To be honest, the first few upgrades went unusually smoothly: do-release-upgrade, hit Enter all the way through, reboot, SSH connects instantly, clean and neat. After several successes in a row, I got cocky. 😆

The price of getting cocky: while upgrading n100-cheshi-0, I hit confirm and then went out. When I came back, the upgrade had gone wrong and was stuck. 🧠 In a moment of heat plus panic, I cut the power directly — instant GG 😭, and it stayed dead for hours.

The picture that followed is very familiar: repeated reboots, stuck at GRUB, a black screen, or dropped into the (initramfs) prompt, and SSH was out of the question. A machine that was only supposed to “get an upgrade” had been turned into a brick. 😑

Fortunately, I fixed it in the end.

Why “an interrupted upgrade” is trickier than “a normal crash”

A major Ubuntu version upgrade is essentially a multi-stage transaction, roughly: download packages → unpack → configure → rebuild initramfs → update the GRUB bootloader. Where the power cut happens completely changes the way it breaks:

Where the power cut happened	Typical symptoms
Packages unpacked but not configured	dpkg database stuck in half-configured state; apt throws all kinds of errors
initramfs half-generated	Kernel can’t find the root filesystem and drops into (initramfs)
GRUB write interrupted	Boot record corrupted; boot goes straight to BIOS or a black screen
ext4 journal not flushed	Filesystem corrupted; read-only mount or even unmountable

The troublesome part is that these layers are very likely broken at the same time. You fix the filesystem first, then it gets stuck because the packages aren’t configured; once the packages are configured, GRUB is broken again. One link after another — it’s enough to drive you crazy.

📝Notes:
The good news: the vast majority of this is repairable, unlike physical disk failure, which is an immediate death sentence. There’s only one core idea: boot a temporary “rescue system” from an Ubuntu Live USB, then chroot into the original system to fix it.

The rescue log: the six-step Live USB + chroot method

Step 0: Prepare a Live USB

On another computer, download the Ubuntu 26.04 desktop ISO, write it to a USB drive of ≥4GB with Rufus or balenaEtcher, then set the USB drive as the first boot entry in BIOS/UEFI.

📝Notes:
Use the desktop edition rather than the Server edition, because we need the temporary graphical environment from Try Ubuntu, which makes things much easier.

Step 1: Boot from the USB drive and choose “Try Ubuntu”

Press F12/F11/Del at boot to enter the boot menu and select the USB drive. Once the installer appears, do not click Install; choose Try Ubuntu to enter the temporary desktop. All subsequent operations happen in this environment.

Step 2: Identify the disks and repair the filesystem

First confirm the partition layout:

lsblk -f

Identify which is the root partition (usually ext4, e.g. /dev/sda2) and which is the EFI partition (usually vfat, e.g. /dev/sda1).

An unexpected power loss very easily corrupts the filesystem, and that’s the most common reason a machine won’t boot. Fix it first:

sudo fsck -y /dev/sda2

📝Notes:
If it says “device is mounted”, it has already been auto-mounted; run sudo umount /dev/sda2 first and then retry. The EFI partition generally doesn’t need fsck.

Step 3: Set up the chroot environment

sudo mount /dev/sda2 /mnt
sudo mount /dev/sda1 /mnt/boot/efi

for i in /dev /dev/pts /proc /sys /run; do
  sudo mount --bind $i /mnt$i
done

sudo chroot /mnt

The point of this step: let the original system “borrow” the running kernel of the current Live environment. Virtual filesystems such as /dev, /proc, and /sys must be bind-mounted in, otherwise apt/dpkg will fail with missing device nodes.

Step 4: Fix the packages

After entering the chroot, finish off the package transaction interrupted by the power loss:

apt --fix-broken install
dpkg --configure -a

These two commands solve 80% of “half-configured” problems.

Step 5: Reinstall GRUB and rebuild initramfs

For UEFI systems:

grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=ubuntu
update-grub
update-initramfs -u -k all

If grub-install fails because of corrupted packages, force a reinstall:

apt install --reinstall grub-efi-amd64 grub-efi-amd64-signed shim-signed

Step 6: Exit, unmount, reboot

exit
sudo umount -R /mnt
sudo reboot

Pull out the USB drive and witness the miracle. 🎉

Layered rescue options: how to choose among five approaches

For the same machine there is actually more than one way to rescue it. Ordered by “invasiveness from low to high”:

Approach	When it applies	Pros	Cons
init=/bin/bash	GRUB works, the system just won’t start	Fastest; root shell in 10 seconds	Minimal environment only, no networking or services
Recovery Mode	The GRUB menu still shows	Graphical menu, one-click fsck/dpkg	Can’t fix GRUB itself
Live USB + chroot	Boot, packages, and filesystem all broken	Most capable; can fix almost anything	Many steps, needs a USB drive
boot-repair	Complex GRUB issues, don’t want to fiddle manually	Highly automated	Sometimes “too clever for its own good”
Backup + reinstall	Everything else failed	Ends the matter once and for all	All config lost; the last resort

How to use init=/bin/bash: in the GRUB menu press e to edit the boot entry, append init=/bin/bash to the end of the linux line, press F10 to boot, and you drop straight into a root shell. It suits the case where “the filesystem is fine, the services just won’t start”.

📝Notes:
boot-repair can be installed in the Live environment: sudo add-apt-repository ppa:yannubuntu/boot-repair && sudo apt install boot-repair. My personal preference: if I can fix it manually with chroot, I fix it manually and see every step clearly — that’s more reassuring than a one-click tool.

Post-mortem discipline: a few iron rules so upgrades don’t derail

Technically there was nothing new about this failure; the real problem was in the process:

1. Someone must watch the machine during an upgrade. This isn’t a “just run a command” operation; it’s a transaction that can fail.
2. Confirm the power supply is stable before upgrading. A UPS isn’t a luxury; it’s a necessity for a HomeLab.
3. Back up before upgrading. At least etckeeper or a tar archive of key configs.
4. Don’t let your guard down because the first few went smoothly. Honestly, the inexplicable overconfidence that builds up after several consecutive successes is the real culprit behind this incident.

📝Notes:
Rule 4 is the core one. A thousand-mile dike can collapse from an ant hole. The upgrade itself isn’t hard; what’s hard is staying as careful as the first time after consecutive successes.

Conclusion

The full rescue chain was: Live USB boot → lsblk -f to identify disks → fsck to repair the filesystem → mount + bind virtual filesystems → chroot → apt/dpkg to fix packages → reinstall GRUB + update-initramfs → umount -R → reboot.

Two key takeaways:

• A “brick” caused by an interrupted upgrade is almost always repairable; don’t rush to wipe the disk and reinstall;
• Use rescue methods in layers: if you can get in via GRUB, don’t reach for a Live USB; if you can fix it manually with chroot, don’t rush to boot-repair; reinstalling the system is always the last resort.

As for that n100-cheshi-0, it came back to life at full health in the end. The cost was one less day of holiday fun, traded for a blog post — not a bad deal. 😆

If you are as careful at the end as at the beginning, nothing will fail.

That’s all.

📚️ References

• Ubuntu official documentation - system installation and booting
• Ubuntu Community Help Wiki - Grub2
• boot-repair tool
• fsck manual
• chroot manual