Sparse file creation, not zero fill

If all we wanted to do was create a file of size N, then we can simply write at byte N in the file. This can be a significant savings in IO and is called sparse file creation.

Here is a simple Python script which does just that:

#
# Does not account for size < 1
#
def punch(file, size) :
    try :
        f = open(file, "wb")
        f.seek(size-1)
        f.write(b'\x00')

    finally:
        f.close()

punch("p1024.out", 1024)
punch("p1023.out", 1023)
punch("p64.out", 64)
punch("p1025.out", 1025)
punch("p10250.out", 10250)

And it yields:

[thomas@snakey src]$ ./punch.py 
[thomas@snakey src]$ ls -la p*out
-rw-r--r-- 1 thomas wheel  1023 Feb 13 16:05 p1023.out
-rw-r--r-- 1 thomas wheel  1024 Feb 13 16:05 p1024.out
-rw-r--r-- 1 thomas wheel 10250 Feb 13 16:05 p10250.out
-rw-r--r-- 1 thomas wheel  1025 Feb 13 16:05 p1025.out
-rw-r--r-- 1 thomas wheel    64 Feb 13 16:05 p64.out

Can we detect a difference between the two outputs?

[thomas@snakey src]$ uname -a
Linux snakey 2.6.35.11-83.fc14.x86_64 #1 SMP Mon Feb 7 07:06:44 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
[thomas@snakey src]$ cmp p1023.out h1023.out 
[thomas@snakey src]$ cmp p10250.out h10250.out 

That asks the question over whether the OS is dumping data down there for me or not. Hmm, let's try this:

[thomas@snakey src]$ du -sh p10250.out h10250.out 
4.0K	p10250.out
12K	h10250.out

So the sparse one is 1/3rd the size. If we do an additional file 10x the size of this one, we see:

[thomas@snakey src]$ cmp p102500.out h102500.out 
[thomas@snakey src]$ du -sh p102500.out h102500.out 
4.0K	p102500.out
104K	h102500.out

So the sparse file creation is working in the sense that we are seeing a block being written. (Which must perforce be 4k.)

So why then does cmp not squawk? Well, when this OS reads the missing pages, it reports them as 0s to the caller. We can also see that this OS is zero filling the page for us.