Large-file Sequential Write Comparison Using FFSB

Test bed:

Kernel: 2.6.27-rc1 and  ext4-patch-queue-b73ebf59e77eeb6482788604078a9c62dec31185 (update of August 12, 2008)
Partition size: 5 TB (IBM DS4100 Storage System with 400GB SATA 7200rpm disks)
IO scheduler: CFQ

Tests are done on a bi-Xeon machine with 2G of RAM and with hyper-threading enabled (-> 4 CPUs).

processor      : 4
vendor_id      : GenuineIntel
cpu family     : 15
model          : 4
model name     : Intel(R) Xeon(TM) CPU 2.80GHz
cpu MHz        : 2793.078
cache size     : 1024 KB
bogomips       : 5586.59

#hdparm -t /dev/md0

/dev/md0:
 Timing buffered disk reads:  332 MB in  3.00 seconds = 110.51 MB/sec

#cat /proc/mdstat
Personalities : [linear]
md0 : active linear sdg[2] sdf[1] sde[0]
      5462560320 blocks 64k rounding

Tools version used:
e2fsprogs v1.41.0-53-g63b5e35 from the "master" branch of e2fsprogs git tree (update of August 10, 2008)
xfsprogs-2.9.8
btrfs-v0.16

Synopsis:

FFSB filesystem benchmarking software 5.2.1 available at
http://sourceforge.net/projects/ffsb/

ffsb profile used:
num_filesystems=1
num_threadgroups=1
directio=0
time=600

[filesystem0]
        location=/mnt/test/
        num_files=0
        num_dirs=0
        max_filesize=1073741824
        min_filesize=1073741824
[end0]

[threadgroup0]
        num_threads=128
        write_size=65536
        write_blocksize=65536
        create_weight=1
[end0]

This profile runs 128 threads which create 128 1-GB files. We collect the throughput scores and the CPU utilization values given by ffsb output.

The partition used is formated with mkfs and mounted with appropriate options before each run of ffsb. A FS check is done after each run.
The number of extents per file is given by filefrag command.

The script used is available here.

Results

.

mkfs options
mount options   Throughput
 MB/s
  CPU usage
       %   
average number of
extents per file
 e2fsck
elapsed time
in seconds
ext3 -I 256
data=writeback 75.9
56.8
264 2854.12
xfs "defaults" 41.5
129.5
32.83
btrfs "defaults" 36.6
36.5
segfault
ext4
-I 256 data=writeback 77.1
32
131 2613
-I 256 data=writeback,
nodelalloc
77.2 57.6 133 2939.07
-I 256
-O uninit_groups
data=writeback 78.7
32.2
128 253.04
-I 256
-O uninit_groups
data=writeback,
nomballoc
74.8
32.8
1078 253.31
-I 256
-O uninit_groups
data=writeback,
nomballoc,nodelalloc
78.4
64.5
249 252.94
-I 256
-Ouninit_groups
-G 64
data=writeback,
journal_async_commit
78
32.1
125  252.49
-I 256
-Ouninit_groups,flex_bg
-G 64
data=writeback 76.8
31.7
125  243.28
-I 256
-Ouninit_groups,flex_bg
-G 64
data=writeback,
nomballoc
78.6 32.6 1090 243.65
-I 256
-Ouninit_groups,flex_bg
-G 64
data=writeback,
nomballoc,nodelalloc
78.6
64.3
249 242.77
-I 256
-Ouninit_groups,flex_bg
-G 64
data=writeback,
journal_checksum
76.3
31
125 243.60
-I 256
-Ouninit_groups,flex_bg
-G 64
data=writeback,
journal_async_commit
75.8
31.1
125 243.37