Btrfs Seed Devices for A/B System Updates

A/B system updates, as described here, provide a way for an operating system (OS) to seamlessly update from an old version to a new version, while ensuring that any failure in the upgrade process will allow for fallback to the known-working old version of the OS.

Typically A/B updates are implemented using separate old and new filesystem images, atop separate, equally sized disk partitions. However, modern copy-on-write filesystems offer some more performant and space efficient possibilities, as described below.

A/B Updates Using Btrfs Subvolume Snapshots

Linux's Btrfs filesystem provides support for snapshots at a subvolume level, which can be used for A/B system updates. A typical procedure would be:

• The current OS version is running atop an old read-only subvolume
• When an update is available, the old subvolume is cloned as a writeable snapshot under a newly created path within the filesystem
  
• The upgrade is written to the new snapshot subvolume path (e.g. via btrfs receive)
• The new snapshot is configured as the default subvolume, causing it to be mounted on next boot
• If any issues are encountered during or post update, any default subvolume change is reverted, the old OS version is booted and the new subvolume is subsequently discarded
  

This procedure works well; it's space efficient, allows for as many old versions to be retained as desired and also doesn't require any specific block device partitioning scheme. Given these benefits, it's unsurprising that SUSE uses a similar approach to provide Transactional Update functionality. However, there are still some minor caveats:

• Currently Btrfs only provides atomic snapshots for single subvolumes, meaning that the above procedure shouldn't be used if an OS update modifies multiple subvolumes
• The update procedure must be aware of the new subvolume path to target for I/O
• An alternative may be to create a read-only snapshot before upgrading in-place, similar to snapper based rollback
  

A/B Updates Using Btrfs Seed Devices

Btrfs seed devices offer copy-on-write support at a block device level, which also can be used to provide A/B system updates, with fallback between new and old block devices instead of subvolumes.

The following seed device example requires two or more separate block devices (or partitions), with one acting as a read-only seed device and one a read-write "sprout" device.

• The currently running OS version is backed by an old block device, flagged as a read-only seed via

  btrfstune -S 1 /dev/old_block_dev

• When an update is available, the new writeable "sprout" device is added to the Btrfs filesystem via

  btrfs device add /dev/new_block_device /

• The filesystem is remounted read-write
  
• The update is written in-place, with Btrfs ensuring that all update I/O is written to the newly added block device
• The new block device is flagged for the bootloader as the default boot device
• If any issues are encountered during or post update, any default boot device change is reverted and the new block device can be discarded
• The previous OS version remains untouched on the old device for fallback
  
• Once the new OS version is deemed stable, the old seed device should be removed from the filesystem, which will cause dependent data from the old device to be merged into the new

This seed device approach removes some of the constraints of the Btrfs subvolume approach, namely:

• The update procedure can atomically apply changes across multiple subvolumes, with seed-device rollback safely reverting all subvolume changes made
• After read-write remount, the update process can perform I/O to the running system in-place, without any specific knowledge of the seed device usage or underlying filesystem

This functionality may be attractive for Linux distributions, particularly if adding A/B update support to an existing update process with little filesystem integration. However, there remain a number of trade-offs to consider:

• Seed devices are significantly less space efficient compared to snapshot based A/B updates
• Each block device must have sufficient capacity to store the OS
• I/O performed when the old seed device is removed from the updated filesystem is a significant overhead and is avoided with snapshot based A/B updates
• Btrfs at least provides some compensation for this by verifying data checksums
• Btrfs seed device support appears somewhat niche compared to regular subvolume snapshots, so it likely receives less filesystem test focus

A/B Updates Using Copy On Write Virtual Block devices

2024-10-10 update: Interestingly, since I wrote this article a couple of years ago, Android has moved from a fully provisioned, partition based A/B approach to now using device-mapper, which provides layered, block level copy-on-write A/B updates.

Conclusions

Btrfs subvolume snapshots and seed devices can both be used to provide seamless and reliable A/B system updates. Snapshot based updates offer more efficient storage and CPU resource utilization, so should likely be considered the optimal choice for implementers.

Seed device based updates are a viable alternative, particularly for multi-subvolume updates, but implementers should carefully consider the described trade-offs.

Thanks

• Linux Btrfs developers
• My employer, SUSE
• Wikipedia users anon and Naturenow, for publishing the image and animation under open licenses
  

 Changelog

• 2024-10-10: add "A/B Updates Using Copy On Write Virtual Block devices" note
   

Samba and Snapper: Previous Versions with Windows Explorer

Snapper is a neat application for managing snapshots atop a Btrfs filesystem.

The upcoming release of Samba 4.2 will offer integration with Snapper, providing the ability to expose snapshots to remote Windows clients using the previous versions feature built into Windows Explorer, as demonstrated in the following video:

The feature can be enabled on a per-share basis in smb.conf, e.g.:

...
[share]
        vfs objects = snapper
        path = /mnt/btrfs_fs

The share path must correspond to a Btrfs subvolume, and have an associated Snapper configuration. Additionally, Snapper must be configured to grant users snapshot access - see the vfs_snapper man page for more details.

Many thanks to:

• Arvin Schnell and other Snapper developers.
• My colleagues at SUSE Linux, and fellow Samba Team members, for supporting my implementation efforts.
• Kdenlive developers, for writing a great video editing suite.

Samba Transparent File Compression on Btrfs

I recently implemented support for the SMB ioctls required to allow clients to remotely manipulate file and directory compression flags on Btrfs.

Windows Explorer provides the ability to flag files and folders for transparent compression via the File->Properties->Advanced dialog box:

Windows Explorer Advanced Attributes Dialog

 
Files flagged for compression are transparently compressed and uncompressed by the underlying filesystem when accessed or modified, which normally means that storage capacity is saved at the expense of extra CPU overhead during file I/O.

New files and folders inherit the compression flag from the parent folder, unless created with the FILE_NO_COMPRESSION option.

Once flagged, Windows explorer presents compressed files and folders differently to those that aren't compressed:

Windows Explorer Directory Listing

This feature will be available with the upcoming release of Samba 4.2, and is enabled with the Btrfs Samba VFS module. E.g.:

...
[share]
        vfs objects = btrfs
        path = /mnt/btrfs_fs

Samba Server-Side Copy Offload

I recently implemented server-side copy offload support for Samba 4.1, along with Btrfs filesystem specific enhancements. This video compares server-side copy performance with traditional copy methods.

A few notes on the demonstration:

• The Windows Server 2012 client and Samba server are connected via an old 100 Mb/s switch, which obviously acts as a network throughput bottleneck in the traditional copy demonstration.
• The Samba server resembles the 4.1.0 code-base, but includes an extra patch to disable server-side copy requests on the regular share.

Many thanks to:

• My colleagues at SUSE Linux, for supporting my implementation efforts.
• The Samba Team, particularly Metze and Jeremy, for reviewing the code.
• Kdenlive developers, for writing a great video editing suite.

Update (July, 2014): Usage is now fully documented on the Samba Wiki.