
value we could expect to be the hardware limit, 
about 102 MiB/s. Below this block size, we observe 
a progressive drop of performance, which can be 
explained as the duration of the hardware I/O 
operation which takes a fixed amount of time 
independently of the amount of data being 
transferred. 
At the same time, we notice that the buffering 
operation, with bigger data blocks, incurs a penalty 
of about 3 MiB/s, which corresponds to about 4% of 
the total bandwidth. 
3.3  Experiment 2: Storage/var Data 
Size 
The second measurement performed on the host 
machine “storage” was performed by holding the 
block size to a very large value, 1 MiB and varying 
the total data size. The purpose of this measurement 
was to examine the point at which the result 
stabilizes and the effect of CPU caches and 
scheduler non-determinism is no longer dominant. 
Results are reported in Table 2 and analyzed in 
Figure 4. 
Table 2: Results for read/write measurement on native 
host “storage”, with and without the O_DIRECT flag, with 
fixed block size of 1MiB and variable block size. 
Size
(B)
Write1MiBblock
Write1MiBblock
withO_DIRECT
Read1MiBblock
withO_DIRECT
Time
(s)
Speed
(MiB/s)
Time
(s)
Speed
(MiB/s)
Time
(s)
Speed
(MiB/s)
1Mi 0,03 37,3 0,03 37,8 0,04 26,8
2Mi 0,03 67,7 0,03 67,8 0,02 83,5
4Mi 0,06 62,0 0,05 83,9 0,04 109,6
8Mi 0,11 73,3 0,10 79,6 0,08 106,0
16Mi 0,21 76,9 0,17 92,0 0,16 102,6
32Mi 0,42 76,4 0,34 93,5 0,30 106,4
64Mi 0,81 78,9 0,67 96,1 0,61 105,5
128Mi 1,57 81,6 1,29 98,9 1,26 101,4
256Mi 2,84 90,0 2,57 99,6 2,51 101,9
512Mi 5,44 94,1 5,08 100,8 5,01 102,2
1Gi 10,53 97,3 10,07 101,7 10,01 102,3
2Gi 20,57 99,6 20,10 101,9 20,03 102,2
4Gi 40,59 100,9 40,30 101,6 40,04 102,3
8Gi 80,92 101,2 80,44 101,8 80,13 102,2
16Gi 162,84 100,6 162,38 100,9 161,44 101,5
It is easy to notice that the noise is big with 
smaller amounts of data: this is obviously the effect 
of scheduling, context switches, and other 
background activities occurring at the operating 
system level. The obtained values tend to converge 
at bigger amounts of data and the value selected for 
the subsequent experiments was 1 GiB, which is a 
good compromise between data size and stability of 
the measurement. 
 
Figure 4: Chart for read/write measurement on native host 
with fixed block size relatively to Table 2. 
3.4  Experiment 3: Centos1 
After building solid frame of reference for the 
expected results, it is time to proceed with 
measuring the I/O performance of the 
paravirtualized installations. The “centos1” VM was 
freshly restarted, which is the one with the file 
image storage. The file is saved on the file system of 
the host OS, which is running an ext4 partition. The 
measurement was executed with read/write 
operations on the empty partition /dev/xvda2 
using the same dd command as before, with variable 
block size and fixed data size of 1 GiB. The results 
are reported in Figure 5. 
 
Figure 5: Chart for results of experiment 3, machine 
“centos1”, read/write on file image based storage in 
comparison with the host OS results from experiment 1. 
We note that the buffered write operations reach 
the maximum average speed even for small block 
sizes (1 KiB), while the read/write operations with 
O_DIRECT flag reach fast speed with very large 
block sizes (about 1 MiB), but pay a very high 
overhead price for each I/O operation. 
3.5  Experiment 4: Centos2 
After experiment 3, the VM “centos1” was shut 
down and the VM “centos2” was started up, and the 
same measurement was repeated. 
0,0
10,0
20,0
30,0
40,0
50,0
60,0
70,0
80,0
90,0
100,0
110,0
120,0
1Mi 2Mi 4Mi 8Mi 16Mi 32Mi 64Mi 128Mi256Mi512Mi 1Gi 2Gi 4Gi 8Gi 16Gi
Speed(MiB/s)
Filesize (B)
Read1MiBblockwith O_DIRECT
Write1MiBblockwith O_DIRECT
Write1MiBblock
0,0
10,0
20,0
30,0
40,0
50,0
60,0
70,0
80,0
90,0
100,0
110,0
120,0
512 1ki 2ki 4ki 8ki 16ki 32ki 64ki 128ki 256ki 512ki 1Mi
Speed(MiB/s)
Block size (B)
Write1GBwith O_DIRECT
Nativ e write with O_DIRECT
Read1GBwith O_DIRECT
Nativ e write
Write1GB
CLOSER 2011 - International Conference on Cloud Computing and Services Science
660