Large-file Sequential Write Comparison Using FFSB

Test bed:

Kernel: 2.6.27-rc1 and  ext4-patch-queue-b73ebf59e77eeb6482788604078a9c62dec31185 (update of August 12, 2008)
Partition size: 5 TB (IBM DS4100 Storage System with 400GB SATA 7200rpm disks)
IO scheduler: CFQ

Tests are done on a bi-Xeon machine with 2G of RAM and with hyper-threading enabled (-> 4 CPUs).

processor      : 4
vendor_id      : GenuineIntel
cpu family     : 15
model          : 4
model name     : Intel(R) Xeon(TM) CPU 2.80GHz
cpu MHz        : 2793.078
cache size     : 1024 KB
bogomips       : 5586.59

#hdparm -t /dev/md0

/dev/md0:
 Timing buffered disk reads:  332 MB in  3.00 seconds = 110.51 MB/sec

#cat /proc/mdstat
Personalities : [linear]
md0 : active linear sdg[2] sdf[1] sde[0]
      5462560320 blocks 64k rounding

Tools version used:
e2fsprogs v1.41.0-53-g63b5e35 from the "master" branch of e2fsprogs git tree (update of August 10, 2008)
xfsprogs-2.9.8
btrfs-v0.16

Synopsis:

FFSB filesystem benchmarking software 5.2.1 available at
http://sourceforge.net/projects/ffsb/

ffsb profile used:
num_filesystems=1
num_threadgroups=1
directio=0
time=600

[filesystem0]
        location=/mnt/test/
        num_files=0
        num_dirs=0
        max_filesize=1073741824
        min_filesize=1073741824
[end0]

[threadgroup0]
        num_threads=128
        write_size=65536
        write_blocksize=65536
        create_weight=1
[end0]

This profile runs 128 threads which create 128 1-GB files. We collect the throughput scores and the CPU utilization values given by ffsb output.

The partition used is formated with mkfs and mounted with appropriate options before each run of ffsb. A FS check is done after each run.
The number of extents per file is given by filefrag command.

The script used is available here.

Results

.

 mkfs options
 mount options   Throughput
 MB/s
  CPU usage
       %   
average number of
extents per file		  e2fsck
elapsed time
in seconds
ext3 -I 256
 data=writeback 75.9
 56.8
 264 2854.12
xfs "defaults" 41.5
 129.5
 32.83
btrfs "defaults" 36.6
 36.5
 segfault
ext4
 -I 256 data=writeback 77.1
 32
 131 2613
-I 256 data=writeback,
nodelalloc		 77.2 57.6 133 2939.07
-I 256
-O uninit_groups
 data=writeback 78.7
 32.2
 128 253.04
-I 256
-O uninit_groups
 data=writeback,
nomballoc
 74.8
 32.8
 1078 253.31
-I 256
-O uninit_groups		 data=writeback,
nomballoc,nodelalloc		 78.4
 64.5
 249 252.94
-I 256
-Ouninit_groups
-G 64		 data=writeback,
journal_async_commit		 78
 32.1
 125  252.49
-I 256
-Ouninit_groups,flex_bg
-G 64		 data=writeback 76.8
 31.7
 125  243.28
-I 256
-Ouninit_groups,flex_bg
-G 64		 data=writeback,
nomballoc		 78.6 32.6 1090 243.65
-I 256
-Ouninit_groups,flex_bg
-G 64		 data=writeback,
nomballoc,nodelalloc
 78.6
 64.3
 249 242.77
-I 256
-Ouninit_groups,flex_bg
-G 64		 data=writeback,
journal_checksum
 76.3
 31
 125 243.60
-I 256
-Ouninit_groups,flex_bg
-G 64		 data=writeback,
journal_async_commit		 75.8
 31.1
 125 243.37