The superblock

The superblock is the first section of a squashfs archive, and contains important information about the archive, including the locations of other sections of the archive.

Superblock Flags

Metadata Blocks

Metadata blocks are compressed in 8KiB blocks. A metadata block is prefixed by a u16 header. The highest bit of the header is set if the block is stored uncompressed (this will happen if the block grew when compressed, or e.g. the UNCOMPRESSED_INODES superblock flag is set). The lower 15 bits specifies the size of the metadata block (not including the header) on disk.

To read a metadata block, read a u16. If the highest bit is set (size & 0x8000 == 0x8000) the following data is uncompressed. Mask out the highest bit to get the size of the block data on disk (this should always be <= 8KiB). Read that many bytes. If the data is compressed, uncompress the data. In pseudocode:

header = read_u16(offset=offset)
data_size = header & 0x7FFF
compressed = !(header & 0x8000)
data = read(offset=offset+2, len=data_size)
if(compressed) {
    data = uncompress(data)
}
return data

Neither the size on disk, nor the compressed size should exceed 8KiB. The uncompressed size should always be equal to 8KiB, with the exception of the last metadata block of a section, which may have an uncompressed size less than 8KiB.

Compression Options

If the COMPRESSOR_OPTIONS flag is set, this section will be present immediately after the superblock, otherwise this section will not be present. If this section is present, it consists of a single metadata block, which is always uncompressed. The data is interpreted differently based on the compressor (compression_id).

For LZ4, the compressor options always have to be present.

Datablocks and Fragments

Datablocks and fragments contain the data which is contained by the files in this archive. A single file's data is stored in a number of data blocks, which are stored sequentially in this section. Because blocks are stored sequentially, the inode for a file only needs to store the position of the first block, and the compressed sizes of each block. All data blocks must be of size block_size.

If the size of a file is not equally divisible by block_size, the final chunk can either be stored in a short block that does not uncompress to full size, or it can be stored in a fragment, if fragments are enabled (NO_FRAGMENTS is not set).

Fragments of multiple files are combined into data blocks of at most size block_size, and compressed as a single block (unless compression fails to shrink the fragment block).

Datablocks do not have headers. Information about the size and position of datablocks is stored in the inode of the file to which the datablocks belong. Information about the size and position of fragment blocks are stored in the Fragment Table, and the size and offset of fragments within the blocks are stored in the inode of the file to which the fragment belongs.

In both the fragment table, and file inodes, the size of a data block is represented by a u32. If the 1 << 24 bit is set, the data block is stored uncompressed. The size of the block on disk is described by this u32 when this bit is masked out, though the value should always be less than the max block size (1MiB).

Sparse files are handled at block_size granularity. If an entire block is found to be full of zero bytes, the block isn't written to disk. Instead a size of zero is stored in the inode.

Inode Table

The inode table starts at inode_table_start and ends at directory_table_start. In this range are stored enough metadata blocks to contain all inodes. All metablocks in the table (except for the last block) should have an uncompressed size of 8KiB.

Inodes are packed into metadata blocks. Inodes are not aligned to block boundaries, and can therefore span the boundary between metadata blocks. To maximise compression there are different inodes for each item type (regular file, directory, device, etc.), the inode contents and length varying with the type.

To further maximise compression, inodes come in two flavors: simple inode types optimised for frequently occurring items, and extended inode types where extra information has to be stored.

If the UNCOMPRESSED_INODES flag is set, all metadata blocks should be stored uncompressed. If the flag is not set, metadata blocks will be stored compressed if compression decreases the size of the block as described in the metadata block section.

Inode References

Entries in the Inode table are referenced (for example, in directory entries) with u64 values. The upper 16 bits are unused. The next 32 bits are the position of the first byte of the metadata block, relative to the start of the inode table. The lower 16 bits describe the (uncompressed) offset within the metadata block where the inode begins (remember that an inode may straddle the border between two metadata blocks). For example, an inode reference with the value 0x0000_000001FF_01A0 will be located in the metadata block that starts at byte inode_table_start+0x000001FF. After decompressing the block, the inode itself then starts at the byte at index 0x01A0 (the 417th byte) of the uncompressed data.

Inode Header

All Inodes share a common header, which contains some common information, as well as describing the type of Inode which follows. This header has the following structure:

Inode Types

There are seven types of inodes, and each type comes in two variants, a basic variant which is smaller and contains only the most-used properties, and an extended variant which has more properties, and will be used when those less used properties are required (e.g. xattrs). Note that some inodes types are variable sized (symlinks targets, sizes of file data blocks, etc).

The value of inode_type (in the inode header) can have any of the following values:

Directory Table

For each directory inode, the directory table stores a list of all entries stored inside, with references back to the inodes that describe those entries.

The directory inodes store the total, uncompressed size of the entire listing, including headers. Using this size, a SquashFS reader can determine if another header with further entries should be following once it reaches the end of a run.

The entry list is self is sorted ASCIIbetically by entry name. To save space, a delta encoding is used to store the inode number, i.e. the list is preceeded by a header with a reference inode number and all entries store the difference to that. Furthermore, the header also includes the location of a metadata block that the inodes of all of the following entries are in. The entries just store an offset into the uncompressed metadata block.

Directory Header

Every time, the inode block changes or the difference of the inode number cannot be encoded in 16 bits anymore, a new header is emitted.

A header must have at least one entry. A header must not be followed by more than 256 entries. If there are more entries, a new header is emitted.

The file names are stored without trailing null bytes. Since a zero length name makes no sense, the name length is stored off-by-one, i.e. the value 0 cannot be encoded.

Directory Entry

The basic and extended inode types both have a size field that stores the uncompressed size of all the directory entries (including all headers) belonging to the inode. This field is used to deduce if more data is following while iterating over directory entries, even without knowing how many headers and partial lists there will be.

Directory Index

To speed up lookups on directories with lots of entries, the extended directory inode can store an index table, holding the locations of all directory headers and the name of the first entry after the header.

To allow for fast lookups, a new directory header should be emitted every time the entry list crosses a metadata block boundary.

Fragment Table

Fragments are combined into fragment blocks of at most block_size bytes long. This table describes the location and size of these fragment blocks, not the fragments within them.

This table is stored in two levels: The fragment block entries are stored in metadata blocks, and the file offsets to these metadata blocks are stored at the offset specified by the fragment_table_start field of the superblock.

Each metadata block can store 512 fragment block entries (16 bytes per fragment block entry), so there will be ceil(fragment_entry_count / 512.0) metadata blocks (and the same number of u64 offsets stored at fragment_table_start)

To read the list of fragment block entries, read ceil(fragment_entry_count / 512.0) u64 offsets starting at fragment_table_start, then read the metadata blocks at the offsets read, interpreting the data of the metadata blocks as a packed array of fragment block entries.

Fragment Block Entry

Export Table

To support NFS exports, squashfs needs a fast way to resolve an inode number to an inode structure.

For this purpose, a squashfs archive can optionally contain an export table, which is basically a flat array of 64 bit inode references with the inode number being used as an index into the array.

Because the inode number 0 is not used (as it is reserved as a sentinel value in Linux and other UNIX-like OS kernels), the array actually starts at inode number 1 and the index is thus inode_number - 1.

The array itself is stored in a series of metadata blocks. Since each block can store 1024 references (8 byte per reference), there will be ceil(inode_count / 1024.0) metadata blocks for the entire array.

To locate the metadata blocks, a secondary list is used, containing the absolute, on-disk locations of the blocks. This list is stored uncompressed and starts at export_table_start.

UID/GID Table

UID/GIDs are both stored as u32s. Both UIDs and GIDs are treated as IDs, if a file is owned by user 1000, and group 1000, the ID 1000 will be stored only once. UID/GIDs in inodes are stored as u16 indexes into this table (and thus, a maximum of 65535 unique IDs may be stored).

This table is stored in two levels: The IDs are stored in metadata blocks, and the file offsets to these metadata blocks are stored at the offset specified by the id_table_start field of the superblock.

Each metadata block can store 2048 UID/GIDs (4 bytes per ID), so there will be ceil(id_count / 2048.0) metadata blocks used for IDs (and the same number of u64 offsets stored at id_table_start).

To read the list of IDs, read ceil(id_count / 2048.0) u64 offsets starting at id_table_start, then read the metadata blocks at the offsets read.

Xattr Table

Extended attributes are arbitrary key value pairs attached to inodes. The key names use dots as separators to create a hierarchy of namespaces.

Squashfs uses multiple levels of indirection to store Xattr key value pairs associated with inodes. To saves space, the topmost namespace prefix is removed and encoded as an integer ID instead. This approach limits squashfs xattr support to the following, commonly used namespaces:

This means that on the one hand squashfs can store SELinux labels or capabilities since those are stored in the security.* namespaces, but cannot store ACLs which are stored in system.posix_acl_access because it has no way to encode the system. prefix yet.

The key value pairs of all inodes are stored consecutively in metadata blocks. The values can be either be stored inline, i.e. an Xattr Key Entry is directly followed by an Xattr Value Entry, or out of line to deduplicate identical values.

If a value is stored out of line, the value entry structure holds a 64 bit reference instead of a string that specifies the location of the value string, similar to an inode reference, but relative to the the first metadata block containing the key value pairs.

Typically, the first occurrence of a value is stored in line and every consecutive use of the same value uses an out of line value to refer back to the first one.

Xattr Key Entry

Xattr Value Entry

To actually address a block of key value pairs associated with an inode, a lookup table is used that specifies the start and size of a block of key value pairs.

All an inode needs to store is a 32 bit index into this table. If two inodes have the identical xattrs (e.g. they have the same SELinux labels and no other attributes), the key/value block is only written once, there is only one lookup table entry and both inodes have the same index.

Xattr Lookup Table

In order to locate both tables on disk, an approach similar to ID and fragment tables is used. The following data structure is stored directly on in the archive (i.e. uncompressed and without additional headers).

The xattr_id_table_start in the superblock stores the absolute position of this table.

Xattr ID Table