What is a sparse file and why do we need it?

Question

What is a sparse file and why do we need it? The only thing that I am able to get is that it is a very large file and it is efficient(in gigabytes). How is it efficient ?

Answer 1

A sparse file is a file that is mostly empty, i.e. it contains large blocks of bytes whose value is 0 (zero).

On the disk, the content of a file is stored in blocks of fixed size (usually 4 KiB or more). When all the bytes contained in such a block are 0, a file system that implements sparse files does not store the block on disk, instead it keeps the information somewhere in the file meta-data.

Advantages of using sparse files:

• empty blocks of data do not occupy disk space; they are not stored as the regular blocks of data, their identifiers (that use only several bytes) are stored instead in the file meta-data; this way 4 KiB of disk space (or more) are saved for each empty block;
• reading an empty block of data from a sparse file does not take time; this happens because no data is read from disk; since the file system knows all the bytes in the block are 0, it just sets to 0 all the bytes in the input buffer and the data is ready; there is no need to access the slow storage device;
• writing an empty block of data into a sparse file does not take time; on writing, the file system detects that the block is empty (all its bytes are 0) and puts the block ID into the list of empty blocks (in the file meta-data); no data is written to the disk.

More information about sparse files can be found on the Wikipedia page.