ArmA 2 I/O analysis results

[qwertz]

THIS THREAD IS CONTINUING THE "RAMDISK" THREAD - PLEASE CONTINUE THE RAMDISK DISCUSSION HERE!

Original thread: http://forums.bistudio.com/showthread.php?t=88388

__________________________________________________ _______________
Update 19 Oct 2009

I have pimped my analysis tool a little so that everyone can download and use it to easily analyze his/her own ArmA 2 disk performance.

I am not a programmer so this is one large Excel file. It includes all instructions, and the process is rather simple (read the instructions carefully though).

If you use it, It would be great if you guys would share the results here in the forum so we can all learn more.

Here's the download link to the ArmA_2_IO_Analysis_v0.22.zip file (now version 0.22, should work with everyone's data now):

ArmA_2_IO_Analysis_v0.22.zip

Please let me know what you think, i.e. if there are any problems with the tool.

__________________________________________________ _________________

OK guys,

here's the results of a few hours of analysis (I will just post them and comment later/tomorrow in detail) - very interesting findings:



Arma does not use file caching - period. This means all the tools that are advertising to optimize the windows cache are useless for our purpose. On the other hand, it means that Arma is really 100% streaming the content. Because as a 32 bit app it can only address 2GB max (and I have yet to see it use more than 1GB), with >9GB of game assets, it means that there must be a LOT of disk activity.

Lets have a closer look:

This is 3 x the same data, sorted (1) by PBO name (2) by MB/s per PBO and (3) by total number of read I/Os per PBO. You can use the two tables in the middle and on the right to see which PBO's you should prioritize when you have limited RAM available for a RAMdisk - just look at the "FS (MB) cum.(ulated)" column. Also, note that my 6 minutes of gameplay left some PBOs completely untouched - this alone saves 1.7GB, but might be caused by the rather short game play time for the dataset.



This dataset is based upon 6 minutes of gameplay, with ProcMon capturing all file activity in the ArmA 2 folder (for this exercise, the ArmA 2 folder was on my 2 x Velociraptor RAID0 disk array, not on the RAMdrive). As you can see, ProcMon recorded over 10,000 read events, totalling ~1GB of data reads in 360 seconds . This is mostly "random read" activity - wait a moment - hard disks are not very good at random read - this might get interesting.

Here is another look at the data, now on the 360 seconds timeline - and this is the key to understanding why we have stutter with HDDs but not with RAMdisks:





On average, ArmA 2 creates a load of 30 IOPS and reads 2.9 MB/s. This might sound harmless, but as the graph shows, the distribution of I/Os is not at all even - it shows heavy spiking !

As a reference, this is what a fast HDD (Velociraptor) is capable of:



As you can see, in random read the fastes consumer HDD is capable of 50-141 IOPS, and 0.07-50.00 MB/s (mind you, your average 7200 rpm HDD is way slower). What is more important is that as you can see above, the average I/O size across all reads recoreded is 97 KB. The mighty Velociraptor is somewhere between 50 and 127 IOPS for this I/O size. So, the spikes on the IOPS graph are definitely saturating the HDD, causing stutter.

But it gets worse. The 97k average I/O size is not evently spread out on the timeline, either. As you can see on the graph above, it oscillates between a few KB to up to 1000 KB, and many times it dips into the 200k and bigger numbers.

At these I/O chunk sizes, even a single Velociraptor can not deliver the 50+ Iops and 10+ MB/s that ArmA 2 is churning through !!

This is what causes the stutter - ArmA 2 simply oversaturates the average hard disk subsystem. And this is why a RAMdisk helps so much in increasing the smoothness of the game!

This also means that this data is not reflective of what the real I/O need for ArmA 2 is - without the HDDs bottlenecking, we would see higher numbers here. I will do the same test running off the RAMdrive when I have time to verify/falsify this thesis.

Also have in mind that I am still working with averages here and one-second "buckets" - there might be more extreme oscillation going on that is just not visible at this resolution.

First conclusion: put ArmA 2 on the fastest disk subsystem you have. This test was run on a Veloci RAID0, and I had a lot of stutter (well, coming from my RAMdisk, that is). Even on my Vertex SSD RAID0, there is still stutter. Only the RAMdisk removed the bottleneck completely - go figure.

I'm tired now, will share some more observations with you tomorrow.

Best,

qwertz

[Tankbuster]

Resize those images, fella. You'll break the internet posting more than 100k pics!

[Peonza]

Cool find. Now it explains why some people playing with 8800 cards owns ones with 4870's in arma mark

[Greg]

Very interesting findings Qwertz. This kind of profiling would be useful to BIS if they chose to pay attention to it.

[4 IN 1]

I personally dont think there is much for BI can do on this matter

[.kju [PvPscene]]

Sincere thanks for the in depth analysis Qwertz!

One aspect nice to verify is that if lower graphic settings
mean less data to stream / read from the HDD.

PS: BI can make special contracts with SSD or RAM drive companies. :P

[Inkompetent]

Very nice analysis indeed!

It was already known that RAM/SSD drives vastly improve ArmA2 performance, but it's lovely to get see some actual numbers on it

[Alex72]

Thanks a lo Qwertz!

RAMDISK it is! Since ARMA2 is the only thing i play really.

[Bono_LV]

Thank you for taking your time to do this analysis. Really informative.

On other side, I must agree with 4in1. I don't think there is anything that BIS can improve with that.

Also... It would be nice if someone of community could find explanation why buildings in Arma 2 are such a fps killer.

[PogMoThoin]

This explains why the Sahrani caa1 mod runs much smoother than Chernarus, no constant loading of the plants2_tree.pbo or structures.pbo as the map is less dense in plants and buildings.

Now I've only 4GB ram (wow, I just said it, 4GB is loads), there must be a way we can add the most used .pbo's to a 1GB ramdisk or force the game to load them to ram. Hope BI are watching, this is an idea for the next patch

Nice work qwertz