Trimming the FAT: Flash Raspberry Pi OS images faster

Did you know that when you download and flash one of our Raspberry Pi OS Lite images, you’re largely flashing… nothing? Looking at our latest 64-bit lite image from 13 May, it’s supposedly 2,632 MiB decompressed, yet for some reason it occupies only 1,643 MiB of disk space, just 62.4% of that amount. Why is our disk image one-third empty? And, more importantly, can we use this knowledge to reduce image flash times and SD/eMMC wear?

Raspberry Pi OS Lite image sizes

First of all, what do we mean by ‘empty’? When we decompress our image file, the xz software tool will, by default, detect long sequences of binary zeros and skip over them, creating ‘holes’ in the output file. Many file systems support this sparse feature because, by using it, we both reduce the time spent writing zeroes to disk and economise on disk space. We can use tools like filefrag and binvis.io to discover where these holes are in our decompressed disk image and visualise them:

We have a culprit! The largest hole in our disk image coincides with our FAT32 512 MiB bootfs partition, which is used in early boot to load firmware or the Linux kernel. It’s hardly surprising that this is the case: this 512 MiB partition contains only 65 MiB of files, leaving plenty of extra space for future software updates.

Although we’ve identified the main source of our holes, we still have a problem to solve if we wish to reduce flash and verification times: how do we differentiate between holes that are deliberately zero, and holes that are uninitialised or unused clusters in our bootfs partition?

A solution with bmap

Thankfully, there’s a simple solution. We recently augmented the Raspberry Pi OS image builder, pi-gen, to add bmap support; some readers may have already noticed the .bmap files appearing in the downloads section. The bmap files are a record of which blocks of the image file contain important information that must be flashed and verified, and which blocks can be skipped. Using the information in the .bmap file allows us to keep a record of blocks that are unimportant and can be left uninitialised (white), and blocks that contain useful data (black/blue):

In the case of our 2025-05-13 Lite image, 23.8% (627 MiB) of the blocks can be left uninitialised. Ignoring these results in a significant speed-up when flashing and verifying the image. We can, for instance, use bmaptool to copy our latest 64-bit Lite image to a 32GB Raspberry Pi SD Card:

$ bmaptool copy https://downloads.raspberrypi.org/raspios_lite_arm64/images/raspios_lite_arm64-2025-05-13/2025-05-13-raspios-bookworm-arm64-lite.img.xz /dev/mmcblk0
bmaptool: info: discovered bmap file 'https://downloads.raspberrypi.org/raspios_lite_arm64/images/raspios_lite_arm64-2025-05-13/2025-05-13-raspios-bookworm-arm64-lite.bmap'
bmaptool: info: block map format version 2.0
bmaptool: info: 673792 blocks of size 4096 (2.6 GiB), mapped 513250 blocks (2.0 GiB or 76.2%)
bmaptool: info: copying image '2025-05-13-raspios-bookworm-arm64-lite.img.xz' to block device '/dev/mmcblk0' using bmap file '2025-05-13-raspios-bookworm-arm64-lite.bmap'
# ...
bmaptool: info: copying time: 1m 17.1s, copying speed 26.0 MiB/sec

Whilst bmap is a useful format to preserve hole information and thus enable space-saving sparseness in standard Raspberry Pi OS images, there are differing requirements when mass-provisioning devices with rpi-sb-provisioner.

rpi-sb-provisioner uses Fastboot to flash individual partitions, so we took the opportunity to have a closer look at Android’s sparse image format to see whether greater efficiency could be achieved. Whilst the option to create Android-sparse format ext4 images has been available in verions of mke2fs for many years (via -E android_sparse), no such tool existed for FAT — until now, that is! Introducing…

fat2simg: a new tool to create Android-sparse format FAT32 disk images

Instead of detecting blocks to be discarded by looking at holes in the underlying filesystem (typically with a 4096-byte granularity), fat2simg works by having a deeper understanding of the data it’s reading — in our case, FAT partitions. fat2simg reads the File Allocation Table in the FAT Region to determine which clusters are in use and which may be discarded. Because clusters can be as small as 512 bytes, this approach can offer a greater granularity than relying on filesystem holes. It can also discard deleted files.

By way of a synthetic example, let’s take a look at a 40MiB FAT32 partition image with 512-byte clusters. We create a couple of directories in our FAT image: one containing a 512-byte file of all zeros, and the other containing a 512-byte file of random data.

The first 512 KiB of the image is visualised below for both bmap and Android-sparse formats:

The Android-sparse format not only works on the more granular 512-byte block size, but also tracks which blocks are zero-initialised (shown in blue). This can lead to further performance improvements, as it’s possible to skip zero-initialised blocks on storage that is blank or has already been erased. In this case, there’s 336 KiB of data to write with bmap, but only 4.5 KiB to write with Android-sparse. Additionally, the Android-sparse format both compresses smaller (1.1K vs 7.1K with xz -6) and does so without the additional 2.3K file that bmap needs to track the holes.

Faster secure boot with rpi-sb-provisioner

To achieve secure boot on Raspberry Pi 4- and 5-generation devices, rpi-sb-provisioner also makes use of FAT ramdisks (referred to as boot.img) . We’ve recently updated our tool rpi-make-boot-image to ensure that the smallest possible ramdisk is created in order to speed up boot times. If FAT32 is used, the created images are now cluster-exact. This functionality mirrors that of the mke2fs -d option for ext4, which creates exact-sized disk images for a given set of files.

$ mkdir data_for_disk_image
$ truncate -s 40M ./data_for_disk_image/EMPTY
$ sudo rpi-make-boot-image -d ./data_for_disk_image -o boot.img
# ...
$ fsck.fat boot.img
fsck.fat 4.2 (2021-01-31)
boot.img: 2 files, 81921/81921 clusters

The various Raspberry Pi tools that I discuss in this post are packaged and available in Raspberry Pi OS repositories. We encourage you to take a look at fat2simg, rpi-make-boot-image and rpi-sb-provisioner on GitHub and explore what might be useful for your applications. We hope these tools will benefit enthusiast and industrial customers alike, and they help achieve feature parity for FAT filesystem use.

The post Trimming the FAT: Flash Raspberry Pi OS images faster appeared first on Raspberry Pi.

from News - Raspberry Pi https://ift.tt/8GFjz5q