THIS THREAD IS CONTINUING THE "RAMDISK" THREAD - PLEASE CONTINUE THE RAMDISK DISCUSSION HERE!
Original thread: http://forums.bistudio.com/showthread.php?t=88388
Update 19 Oct 2009
I have pimped my analysis tool a little so that everyone can download and use it to easily analyze his/her own ArmA 2 disk performance.
I am not a programmer so this is one large Excel file. It includes all instructions, and the process is rather simple (read the instructions carefully though).
If you use it, It would be great if you guys would share the results here in the forum so we can all learn more.
Here's the download link to the ArmA_2_IO_Analysis_v0.22.zip file (now version 0.22, should work with everyone's data now):
Please let me know what you think, i.e. if there are any problems with the tool.
here's the results of a few hours of analysis (I will just post them and comment later/tomorrow in detail) - very interesting findings:
Arma does not use file caching - period. This means all the tools that are advertising to optimize the windows cache are useless for our purpose. On the other hand, it means that Arma is really 100% streaming the content. Because as a 32 bit app it can only address 2GB max (and I have yet to see it use more than 1GB), with >9GB of game assets, it means that there must be a LOT of disk activity.
Lets have a closer look:
This is 3 x the same data, sorted (1) by PBO name (2) by MB/s per PBO and (3) by total number of read I/Os per PBO. You can use the two tables in the middle and on the right to see which PBO's you should prioritize when you have limited RAM available for a RAMdisk - just look at the "FS (MB) cum.(ulated)" column. Also, note that my 6 minutes of gameplay left some PBOs completely untouched - this alone saves 1.7GB, but might be caused by the rather short game play time for the dataset.
This dataset is based upon 6 minutes of gameplay, with ProcMon capturing all file activity in the ArmA 2 folder (for this exercise, the ArmA 2 folder was on my 2 x Velociraptor RAID0 disk array, not on the RAMdrive). As you can see, ProcMon recorded over 10,000 read events, totalling ~1GB of data reads in 360 seconds . This is mostly "random read" activity - wait a moment - hard disks are not very good at random read - this might get interesting.
Here is another look at the data, now on the 360 seconds timeline - and this is the key to understanding why we have stutter with HDDs but not with RAMdisks:
On average, ArmA 2 creates a load of 30 IOPS and reads 2.9 MB/s. This might sound harmless, but as the graph shows, the distribution of I/Os is not at all even - it shows heavy spiking !
As a reference, this is what a fast HDD (Velociraptor) is capable of:
As you can see, in random read the fastes consumer HDD is capable of 50-141 IOPS, and 0.07-50.00 MB/s (mind you, your average 7200 rpm HDD is way slower). What is more important is that as you can see above, the average I/O size across all reads recoreded is 97 KB. The mighty Velociraptor is somewhere between 50 and 127 IOPS for this I/O size. So, the spikes on the IOPS graph are definitely saturating the HDD, causing stutter.
But it gets worse. The 97k average I/O size is not evently spread out on the timeline, either. As you can see on the graph above, it oscillates between a few KB to up to 1000 KB, and many times it dips into the 200k and bigger numbers.
At these I/O chunk sizes, even a single Velociraptor can not deliver the 50+ Iops and 10+ MB/s that ArmA 2 is churning through !!
This is what causes the stutter - ArmA 2 simply oversaturates the average hard disk subsystem. And this is why a RAMdisk helps so much in increasing the smoothness of the game!
This also means that this data is not reflective of what the real I/O need for ArmA 2 is - without the HDDs bottlenecking, we would see higher numbers here. I will do the same test running off the RAMdrive when I have time to verify/falsify this thesis.
Also have in mind that I am still working with averages here and one-second "buckets" - there might be more extreme oscillation going on that is just not visible at this resolution.
First conclusion: put ArmA 2 on the fastest disk subsystem you have. This test was run on a Veloci RAID0, and I had a lot of stutter (well, coming from my RAMdisk, that is). Even on my Vertex SSD RAID0, there is still stutter. Only the RAMdisk removed the bottleneck completely - go figure.
I'm tired now, will share some more observations with you tomorrow.