Squashfs Binary Format

SquashFS is a compressed, read-only filesystem for Linux that can also be used as a flexible, general purpose, compressed archive format, optimized for fast random access with support for Unix permissions, sparse files and extended attributes.

SquashFS supports data and metadata compression through zlib, lz4, lzo, lzma, xz or zstd.

For fast random access, compressed files are split up in fixed size blocks that are compressed separately. The block size can be set between 4k and 1M (default for squashfs-tools and squashfs-tools-ng is 128K).

This document attempts to specify the on-disk format in detail. It is based on a previous on-line version that was originally written by Zachary Dremann and subsequently expanded by David Oberhollenzer during reverse engineering attempts and available here: https://dr-emann.github.io/squashfs/.

SquashFS always stores integers in little endian format. The data blocks that make up the SquashFS archive are byte aligned, i.e. they typically do not care for alignment. The implementation in the Linux kernel requires the archive itself to be a multiple of either 1k or 4k in size (called the device block size) and user space tools typically use 4k to be compatible with both.

A SquashFS archive consists of a maximum of nine parts:

Although the super block details the exact positions of each section, most implementations, including the one in the Linux kernel, insist on this exact order.

The superblock is the first section of a SquashFS archive. It is always 96 bytes in size and contains important information about the archive, including the locations of other sections.

The Xattr table, fragment table and export table are optional. If they are omitted from the archive, the respective fields indicating their position must be set to 0xFFFFFFFFFFFFFFFF (i.e. all bits set).

Most of the flags only serve an informational purpose and are only useful when editing the archive to convey the original packer settings.

The only flag that actually carries information is the "Compressor options are present" flag. In fact, this is the only flag that the Linux kernel implementation actually tests for.

The compressor options, however, are also only there for informal purpose, as most compression libraries understand their own stream format irregardless of the options used to compress and in fact don’t provide any options for the decompressor. In the Linux kernel, the XZ decompressor is currently the only one that processes those options to pre-allocate the LZMA dictionary if a non-default size was used.

As outlined in 2.1, file data is packed by dividing the input files into fixed size chunks (the block size from the super block) that are stored in sequence.

The picture below tries to illustrate this concept:

In the above diagram, file A consists of 3 blocks and a single tail end, file B has 2 blocks and one tail end while file C is smaller than block size.

For each file, the blocks are individually compressed and stored on disk in order.

The tail ends of A and B, together with the entire contents of C are packed together into a fragment block F, that is compressed and stored on disk once it is full.

This tail-end-packing is completely optional. The tail ends (or in case of C the entire file) can also be treated as truncated blocks that expand to less than block size when uncompressed.

There are no headers in front of data or fragment blocks and there MUST NOT be any gaps between data blocks from a single file, but a SquashFS packer is free to leave gaps between two different files or fragment blocks. The packer is also free to decide how to arrange fragments within a fragment block and what fragments to pack together.

To locate file data, the file inodes store the following information:

Since a fragment block will likely be referred to by multiple files, inodes don’t store its on-disk location and size directly, but instead use a 32 bit index into a fragment block lookup table (see the Fragment Table).

If a data block other than the last one unpacks to less than block size, the rest of the buffer is filled with 0 bytes. This way, sparse files are implemented. Specifically if a block has an on-disk size of 0 this translates to an entire block filled with 0 bytes without having to retrieve any data from disk.

The on-disk locations of file blocks MAY overlap and different file inodes are free to refer to the same fragment. Typical SquashFS packers would explicitly use this to for files that are duplicates of others. Doing so is NOT counted as a hard link.

If an inode references on-disk locations outside the data area, the result is undefined.

Inodes are packed into metadata blocks and are not aligned, i.e. they can span the boundary between metadata blocks. To save space, there are different inodes for each type (regular file, directory, device, etc.) of varying contents and size.

To further save more space, inodes come in two flavors: simple inode types optimized for a simple, standard use case, and extended inode types where extra information has to be stored.

SquashFS more or less supports 32 bit UIDs and GIDs. As an optimization, those IDs are stored in a lookup table (see ID Table) and the inodes themselves hold a 16 bit index into this table. This allows to 32 bit UIDs/GIDs, but only among 2¹⁶ unique values.

The location of the first metadata block is indicated by the inode table start in the superblock. The inode table ends at the start of the directory table.

For each directory inode, the directory table stores a linear list of all entries, with references back to the inodes that describe those entries.

The entry list itself is sorted ASCIIbetically by entry name and split into multiple runs, each preceded by a short header.

The directory inodes store the total, uncompressed size of the entire listing, including headers. Using this size, a SquashFS reader can determine if another header with further entries should be following once it reaches the end of a run.

To save space, the header indicates a metadata block and a reference inode number. The entries that follow simply store a difference to that inode number and an offset into the specified metadata block.

Every time, the inode block changes or the difference of the inode number to the reference in the header cannot be encoded in 16 bits anymore, a new header is emitted.

A header must be followed by AT MOST 256 entries. If there are more entries, a new header MUST be emitted.

Typically, inode allocation strategies would sort the children of a directory and then allocate inode numbers incrementally, to optimize directory entry listings.

Since hard links might be further further away than ±32k of the reference number, they might require a new header to be emitted. Inode number allocation and picking of the reference could of course be optimized to prevent this.

The directory header has the following structure:

The counter is stored off-by-one, i.e. a value of 0 indicates 1 entry follows. This also makes it impossible to encode a size of 0, which wouldn’t make any sense. Empty directories simply have their size set to 0 in the inode instead, so no extra dummy header has to be stored or looked up.

The header is followed by multiple entries that each have this structure:

In the entry structure itself, the file names are stored without trailing null bytes. Since a zero length name makes no sense, the name length is stored off-by-one, i.e. the value 0 cannot be encoded.

The inode type is stored in the entry, but always as the corresponding basic type.

While the field is technically 16 bits, the kernel implementation currently imposes an arbitrary limit of 255 on the name size field. Since the field is off-by-one, this means that a file name in SquashFS can be at most 256 characters long.

Tail-ends and smaller than block size files can be combined into fragment blocks that are at most 'block size' bytes long.

The fragment table describes the location and size of the fragment blocks (not the tail-ends within them).

This is a lookup table which stores entries of the following shape:

The table is stored on-disk as described in Storing Lookup Tables.

The fragment table location in the superblock points to an array of 64 bit integers that store the on-disk locations of the metadata blocks containing the lookup table.

Each metadata block can store up to 512 entries (8192 / 16).

The "unused" field is there for alignment and SHOULD be set to 0, however the Linux kernel currently ignores this field completely, making it impossible for Linux to ever re-purpose this field.

To support NFS exports, SquashFS needs a fast way to resolve an inode number to an inode structure.

For this purpose, a SquashFS archive can optionally contain an export table, which is basically a flat array of 64 bit inode references, with the inode number being used as an index into the array.

Because the inode number 0 is not used (reserved as a sentinel value), the array actually starts at inode number 1 and the index is thus inode_number - 1.

The array itself is stored in a series of metadata blocks, as outlined in Storing Lookup Tables.

Since each block can store 1024 references (8192 / 8), there will be ceil(inode_count / 1024) metadata blocks for the entire array.

As outlined in Common Inode Header, SquashFS supports 32 bit user and group IDs. To compact the inode table, the unique UID/GID values are collected in a lookup table and a 16 bit table index is stored in the inode instead.

This lookup table is stored as outlined in Storing Lookup Tables.

Each metadata block can store up to 2048 IDs (8192 / 4).

Extended attributes are arbitrary key value pairs attached to inodes. The key names use dots as separators to create a hierarchy of name spaces.

The key value pairs of all inodes are stored consecutively in a series of metadata blocks.

The values can either be stored inline, i.e. a key entry is directly followed by a value, or out-of-line to deduplicate identical values and use a reference instead. Typically, the first occurrence of a value is stored in line and every consecutive use of the same value uses an out-of-line reference back to the first one.

The keys are stored using the following data structure:

Following the key, this structure is used to store the value:

The metadata block location given by an out-of-line reference is relative to the location of the first block.

To actually address a block of key value pairs associated with an inode, a lookup table is used that specifies the start and size of a sequence of key value pairs.

All an inode needs to store is a 32 bit index into this table. If two inodes have an identical attribute sets, the key/value sequence is only written once, there is only one lookup table entry and both inodes have the same index.

Each lookup table entry has the following structure:

This lookup table is stored as outlined in Storing Lookup Tables

Each metadata block can hold 512 (8192 / 16) entries.

However, in contrast to Storing Lookup Tables, additional data is given before the list of metadata block locations, to locate the key-value pairs, as well as the actual number of lookup table entries that are not specified in the super block.

The 'Xattr table' entry in the superblock gives the absolute location of the following data structure which is stored on-disk as is, uncompressed:

If an inode has a a valid xattr index (i.e. not 0xFFFFFFFF), the metadata block index is computed as

which is then used to retrieve the metadata block index from the locations array.

Once the block has been read from disk and uncompressed, the byte offset into the metadata block can be computed as

From this position, the structure can be read that holds a reference to the metadata block that contains the key/value pairs (and byte offset into the uncompressed block where the pairs start), as well as the number of key/value pairs and their total, uncompressed size.