Large-file Sequential Write Comparison Using FFSB

Test bed:

Kernel: 2.6.27-rc1 and  ext4-patch-queue-b73ebf59e77eeb6482788604078a9c62dec31185 (update of August 12, 2008)
Partition size: 5 TB (IBM DS4100 Storage System with 400GB SATA 7200rpm disks)
IO scheduler: CFQ

Tests are done on a bi-Xeon machine with 2G of RAM and with hyper-threading enabled (-> 4 CPUs).

processor      : 4
vendor_id      : GenuineIntel
cpu family     : 15
model          : 4
model name     : Intel(R) Xeon(TM) CPU 2.80GHz
cpu MHz        : 2793.078
cache size     : 1024 KB
bogomips       : 5586.59

#hdparm -t /dev/md0

/dev/md0:
 Timing buffered disk reads:  332 MB in  3.00 seconds = 110.51 MB/sec

#cat /proc/mdstat
Personalities : [linear]
md0 : active linear sdg[2] sdf[1] sde[0]
      5462560320 blocks 64k rounding

Tools version used:
e2fsprogs v1.41.0-53-g63b5e35 from the "master" branch of e2fsprogs git tree (update of August 10, 2008)
xfsprogs-2.9.8
btrfs-v0.16

Synopsis:

FFSB filesystem benchmarking software 5.2.1 available at
http://sourceforge.net/projects/ffsb/

ffsb profile used:
num_filesystems=1
num_threadgroups=1
directio=0
time=1800

[filesystem0]
        location=/mnt/test/
        num_files=5000
        num_dirs=1000
        max_filesize=524288
        min_filesize=4096
[end0]

[threadgroup0]
        num_threads=128
        write_size=4096
        write_blocksize=4096
        read_blocksize=4096
        create_weight=1
        write_weight=1
        readall_weight=2
[end0]

This profile runs 128 threads which create 128 1-GB files. We collect the throughput scores and the CPU utilization values given by ffsb output.

The partition used is formated with mkfs and mounted with appropriate options before each run of ffsb. A FS check is done after each run.
The number of extents per file is given by filefrag command.

The script used is available here.

Results

.

 mkfs options
 mount options   Transactions
 per second
   CPU usage
       %   
 e2fsck
elapsed time
in seconds
ext3 -I 256
 data=writeback 6341
 8.1
 2881.74
xfs "defaults" 6852
 8.3
 xfs_repair 12.65
btrfs "defaults" 632.10
 38.3
 segfault
ext4
 -I 256 data=writeback 6305
 6.4
 2630
-I 256 data=writeback,
nodelalloc		 9881   14  2888 
-I 256
-O uninit_groups
 data=writeback 7020
 7.6
 255.42
-I 256
-O uninit_groups
 data=writeback,
nomballoc
  6149
 8.5
 353.07
-I 256
-O uninit_groups		 data=writeback,
nomballoc,nodelalloc		 9353
12
 255.22
-I 256
-O uninit_groups		 data=writeback,
journal_async_commit		  7872 8.9 255.99
-I 256
-Ouninit_groups,flex_bg
-G 64		 data=writeback 9469
9.3
 255.63
-I 256
-Ouninit_groups,flex_bg
-G 64		 data=writeback,
nomballoc		 9908  10.1  249.61 
-I 256
-Ouninit_groups,flex_bg
-G 64		 data=writeback,
nomballoc,nodelalloc
 10763
 15.1
316.70
-I 256
-Ouninit_groups,flex_bg
-G 64		 data=writeback,
journal_checksum		 9375
9.2
 245.64
-I 256
-Ouninit_groups,flex_bg
-G 64		 data=writeback,
journal_async_commit		 9913
10.2
245.62