Benchmarking of ext3 extents on 64 bits

2005/04/11
Pierre Peiffer (Pierre.Peiffer@bull.net)

Introduction

This document briefly describes the performance tests done during the development of 64 bits extents.

The development is done in three phases :

Phase 1: convert on-disk structures to architecture independent data using little endian values (not specific to 64 bits in fact).
Phase 2: use 64 bits values for addressing physical blocks.
Phase 3: use 64 bits values for addressing logical blocks.

At each phases, the test ran with and without the new code to see the performance impact of the new development.

Here after, the tests done at each phase and the results are described.
The tests are basic high level tests using cp and rm. The system time consumed by both commands is measured with time.
All development and tests are done on kernel version 2.6.11.7 with extents version 0.4.

Patches:

The patch for the 1st phase is ext3-extents-little_endian.patch.
The patch for the 2nd phase is ext3-extents-0.4-le-physical-block64-2.6.11.7.patch.
The last patch for the 3rd phase is ext3-extents-0.4-logical-block64-2.6.11.7.patch.

These patches must be applied in that order on top of Alex Tomas patch ext3-extents-2.6.11-0.4.patch.

Phase 1: architecture independence of on-disk structure

As the format used for on-disk data is little endian, the tests have been done on power machine, which natively works with big endian values.

The test is done on a PL250 (dual Power 5 with 4 GB of RAM).
It uses a loopback file, which is filled and emptied with /usr (about 2GB of data) as follow:

dd if=/dev/zero of=~/loop.img bs=1k count=2000000 2>&1 1>/dev/null       
losetup /dev/loop0 ~/loop.img
mkfs.ext3 /dev/loop0 2>&1 1>/dev/null
mount -o extents /dev/loop0 /mnt

TIME1=`/usr/bin/time -f "%U %S %e" cp -r /usr /mnt 2>&1`

umount /dev/loop0
mount -o extents /dev/loop0 /mnt

TIME2=`/usr/bin/time -f "%U %S %e" rm -rf /mnt/usr 2>&1`

umount /dev/loop0
losetup -d /dev/loop0

echo "cp: $TIME1 rm: $TIME2"



The test ran 100 times with and without code modification. The results before the development are here and the one after are here. At each line, the values indicate the user-, the system- and the real time consumed for the command.

On average, the results are : 

cp
rm
 Before
 After 
Difference
 Before
 After 
Difference
User time
0.46 0.45
0.05 0.05
System time
17.98 18.10 +0.67%
3.47 3.49 +0.58%
Elapse time
206.60 207.30
3.54 3.55

Phase 2: use 64 bits values for addressing physical blocks

The test was run on a dual Xeon 2.8GHz with 2GB of RAM.
The system was installed on the first disk. The test consists of repeating several copy/remove on the tested filesystem installed on a dedicated disk/partition.

The script used is quite the same as before:

mkfs.ext3 /dev/sdc1 2>&1 1>/dev/null
mount -o extents /dev/sdc1 /mnt

TIME1=`/usr/bin/time -f "%U %S %e" cp -r /usr /home /boot /lib /bin /etc /sbin /var /mnt`

umount /dev/sdc1
mount -o extents /dev/sdc1 /mnt

TIME2=`/usr/bin/time -f "%U %S %e" rm -rf /mnt/usr /mnt/home /mnt/boot /mnt/lib /mnt/bin /mnt/etc /mnt/sbin /mnt/var`

umount /dev/sdc1
echo "cp: $TIME1 rm: $TIME2"

The previous script has been run 400 times on the machine without other special activities. The amount of data copied each time is about 16 GB. The results before are here and the one after are here (the format of the results is as in previous section).

On average, the results are:


cp
rm
 Before
 After 
Difference
 Before
 After 
Difference
User time
0.74 0.75

0.10
0.09

System time
14.32
14.30
- 0.14%
2.55
2.55
+0.00%
Elapse time
126.70
126.65

13.86
13.88


Phase 3: use 64 bits values for addressing logical blocks.

The test was run on a dual Xeon 3.0GHz with 4GB of RAM.
As before, the system was installed on the first disk and the tests were ran on the third one.
The test done is exactly the same. The previous script has been repeated 330 times in both cases (with and without development) and the amount of copied data each time is about 4 GB.
The results are here (before) and here (after).

On average, it gives the following result:


cp
rm
 Before
 After 
Difference
 Before
 After 
Difference
User time
1.19
1.21

0.17
0.17

System time
24.54
24.64
+0.40%
5.04
5.03
-0.20%
Elapse time
298.74
298.39

40.58
40.61