PvPscene Benchmark Suite

[.kju [PvPscene]]

= PvPscene Benchmark Suite

The goal is to create a better system to benchmark the ArmA engine to
understand the effects of different game video settings, different hardware
components and system configuration.

The approach is to improve the suite step by step by adding
new benchmark scenarios, improving the benchmark system
and automating the process more and more.

The system is largely based on the work and insights from Mac
and his ArmA2 FPS analyzer - benchmarking tool.
Please show him your gratitude and motivate him to update the tool
to OA and extend the functionality.


The suite can be run fully automated and logs the result into the rpt file.

Sample output:

"_Preloading - PvPscene_BM_City (Chernarus)"
"avg fps	avg min	highest	lowest"
"20	14	26	5"

The suite contains 12 missions so far and takes 25 minutes to process.

You can also run a subset with configuration or copy the benchmark missions
to arma2\missions and run them one by one as singleplayer mission.


== Current scenarios

The current scenarios focus on the benchmarking the rendering only.
In other words no AI and no interaction - a simple camera scene moving along
some waypoints twice:

1) First a preloading run with faster speed.
2) Followed by the same run in normal speed.

All given scenarios enforce very high terrain detail (most dense grass/clutter
with highest radius) and 3000 viewdistance (10k for LongVD) to put enough
pressure on GPU and CPU.

The types so far are:

• City
• Coastline
• OpenFields
• Woods
• LongViewDistance

on Chernarus, Takistan, Zargabad and ProvingGrounds.


== Possible new scenarios

• Camera to zoom in and out to simulate RMB zoom.
• Camera that simulates the optic view.
• Non combat AI scenarios with different numbers of AI (10, 100, 250, 500, 1000).

== Install guide

1) Download the suite.
(file section at: http://dev-heaven.net/projects/clanbase/files)

2) Extract to the arma2 folder, like

c:\arma2\autotest\PvPscene_BM

3) Configure your arma2 path in _configureSettings.bat, like
(right click the _configuresettings file and choose edit)

set PATHTOARMA=c:\arma2

You can also select the set of missions to be run in there.
As a start it is best to run the complete suite.

You can also set custom startup parameters, if you want to:

set CUSTOMPARAMS="-profiles=c:\arma2" -window "-name=test"

4) Start the suite with the run.bat or runBeta.bat in .\arma2\autotest\PvPscene_BM.

5) Wait for the benchmarks to finish and check the rpt for the result.

6) You can run the suite again with different settings and compare the results yourself
or submit your results to the online database (no registration required).


== Notes

If you run arma in window mode, you have a look at the GPU use
while the benchmarks are run with a tool like MSI Afterburner.

You can follow the CPU use, if you have the task manager running with the
performance tab open.


=== Online Database

Please provide your values the following graphic settings along with your test results
to make the comparison between different systems and graphic settings useful.

For some graphic settings, you need to look in the arma2.cfg.
To find out your hardware, you can use the tool PC-Wizard for example.


To make the copy-paste action less effort, remove the duplicate lines in the rpt first.
You can do this with EditPadPro (trial/lite) for example nicely.
Just open the rpt / drag it into the text editor window, select in the menu 'extra',
and in the middle 'remove duplicate lines'.


The abbreviations for the graphic settings values are:

• D - Disabled/Default
• VL - Very Low
• L - Low
• N - Normal
• H - High
• VH - Very High

[Sickboy]

Really cool mate! Congrats on release!

[[DirTyDeeDs]-Ziggy-]

ok I realize a lot of effort has gone into this, so I will try to participate.

[.kju [PvPscene]]

Lots of time? Download, unpack, configure - once. Can be done in 2 minutes.
The benchmarks are fully automated - you can go watch TV, eat or whatever meanwhile.

If you want a meaningful benchmark, you found one here.

[[DirTyDeeDs]-Ziggy-]

there are 420 fields to fill out? is that a random number or by design?

my benchmark results reflect gfx setting i use in normal gameplay.

Spoiler:

show spoiler

Version 1.57.76815
Item str_disp_server_control listed twice
Warning: looped for animation: ca\wheeled\data\anim\uaz_cargo01_v0.rtm differs (looped now 0)! MoveName: kia_uaz_cargo02
Warning: looped for animation: ca\wheeled\data\anim\uaz_cargo01_v0.rtm differs (looped now 1)! MoveName: uaz_cargo02
"Type avg fps avg min highest lowest"
"Preloading 33 25 45 13 PvPscene_BM_City (Chernarus)"
"Type avg fps avg min highest lowest"
"Benchmark 37 30 52 12 PvPscene_BM_City (Chernarus)"
"Type avg fps avg min highest lowest"
"Preloading 42 32 62 10 PvPscene_BM_Coastline (Chernarus)"
"Type avg fps avg min highest lowest"
"Benchmark 45 36 68 14 PvPscene_BM_Coastline (Chernarus)"
"Type avg fps avg min highest lowest"
"Preloading 44 31 59 6 PvPscene_BM_OpenFields (Chernarus)"
"Type avg fps avg min highest lowest"
"Benchmark 47 37 72 8 PvPscene_BM_OpenFields (Chernarus)"
"Type avg fps avg min highest lowest"
"Preloading 46 33 59 11 PvPscene_BM_Woods (Chernarus)"
"Type avg fps avg min highest lowest"
"Benchmark 46 34 59 13 PvPscene_BM_Woods (Chernarus)"
"Type avg fps avg min highest lowest"
"Preloading 47 34 65 9 PvPscene_BM_City (takistan)"
"Type avg fps avg min highest lowest"
"Benchmark 51 43 65 11 PvPscene_BM_City (takistan)"
"Type avg fps avg min highest lowest"
"Preloading 45 28 66 12 PvPscene_BM_OpenFields (takistan)"
"Type avg fps avg min highest lowest"
"Benchmark 46 30 58 17 PvPscene_BM_OpenFields (takistan)"
"Type avg fps avg min highest lowest"
"Preloading 50 33 65 9 PvPscene_BM_Woods (takistan)"
"Type avg fps avg min highest lowest"
"Benchmark 52 36 68 18 PvPscene_BM_Woods (takistan)"
"Type avg fps avg min highest lowest"
"Preloading 41 32 45 11 PvPscene_BM_LongViewDistance (takistan)"
"Type avg fps avg min highest lowest"
"Benchmark 41 34 45 26 PvPscene_BM_LongViewDistance (takistan)"
"Type avg fps avg min highest lowest"
"Preloading 39 28 48 12 PvPscene_BM_City (Zargabad)"
"Type avg fps avg min highest lowest"
"Benchmark 42 34 70 14 PvPscene_BM_City (Zargabad)"
"Type avg fps avg min highest lowest"
"Preloading 36 23 49 13 PvPscene_BM_OpenFields (Zargabad)"
"Type avg fps avg min highest lowest"
"Benchmark 38 27 47 15 PvPscene_BM_OpenFields (Zargabad)"
"Type avg fps avg min highest lowest"
"Preloading 25 22 27 13 PvPscene_BM_LongViewDistance (Zargabad)"
"Type avg fps avg min highest lowest"
"Benchmark 25 23 28 19 PvPscene_BM_LongViewDistance (Zargabad)"
"Type avg fps avg min highest lowest"
"Preloading 33 21 38 13 PvPscene_BM_OpenFields (ProvingGrounds_PMC lite)"
"Type avg fps avg min highest lowest"
"Benchmark 33 22 44 8 PvPscene_BM_OpenFields (ProvingGrounds_PMC lite)"

[AC337]

Hey guys might be a stupid question but i can't set my shortcut up, can anyone give me a step by step on how to?

[[DirTyDeeDs]-Ziggy-]

Start the suite with the run.bat or runBeta.bat.

from within the ArmA 2/autotest/PvPscene_BM folder



also
right click the _configuresettings file and choose edit
looks like this

[AC337]

from within the ArmA 2/autotest/PvPscene_BM folder



also
right click the _configuresettings file and choose edit
looks like this

Thanks alot man!

[Birdseye]

Nice work on this benchmark suite!
Since it forces high terrein details, and also VD, maybe make it enforce all
in game video options, and create .bats for a few different resolutions?


Spoiler:

show spoiler

================================================== ===================
== c:\arma2\arma2oa.exe
== "c:\arma2\arma2oa.exe" -nosplash -skipintro -world=empty -noFilePatching "-name=test" "-autotest=c:\arma2\autotest\PvPscene_BM\MissionSets \autotestAll.cfg"
================================================== ===================
Exe timestamp: 2011/01/23 14:36:06
Current time: 2011/02/18 09:56:37

Version 1.57.76815
Item str_disp_server_control listed twice
Warning: looped for animation: ca\wheeled\data\anim\uaz_cargo01_v0.rtm differs (looped now 0)! MoveName: kia_uaz_cargo02
Warning: looped for animation: ca\wheeled\data\anim\uaz_cargo01_v0.rtm differs (looped now 1)! MoveName: uaz_cargo02
"Type avg fps avg min highest lowest"
"Preloading 42 32 52 13 PvPscene_BM_City (Chernarus)"
"Type avg fps avg min highest lowest"
"Benchmark 50 41 72 15 PvPscene_BM_City (Chernarus)"
"Type avg fps avg min highest lowest"
"Preloading 70 60 95 18 PvPscene_BM_Coastline (Chernarus)"
"Type avg fps avg min highest lowest"
"Benchmark 74 66 99 26 PvPscene_BM_Coastline (Chernarus)"
"Type avg fps avg min highest lowest"
"Preloading 72 60 86 23 PvPscene_BM_OpenFields (Chernarus)"
"Type avg fps avg min highest lowest"
"Benchmark 77 72 105 40 PvPscene_BM_OpenFields (Chernarus)"
"Type avg fps avg min highest lowest"
"Preloading 84 73 102 37 PvPscene_BM_Woods (Chernarus)"
"Type avg fps avg min highest lowest"
"Benchmark 86 80 103 31 PvPscene_BM_Woods (Chernarus)"
"Type avg fps avg min highest lowest"
"Preloading 66 53 87 11 PvPscene_BM_City (takistan)"
"Type avg fps avg min highest lowest"
"Benchmark 72 61 89 19 PvPscene_BM_City (takistan)"
"Type avg fps avg min highest lowest"
"Preloading 98 83 139 23 PvPscene_BM_OpenFields (takistan)"
"Type avg fps avg min highest lowest"
"Benchmark 99 89 129 38 PvPscene_BM_OpenFields (takistan)"
"Type avg fps avg min highest lowest"
"Preloading 104 90 139 20 PvPscene_BM_Woods (takistan)"
"Type avg fps avg min highest lowest"
"Benchmark 110 101 145 56 PvPscene_BM_Woods (takistan)"
"Type avg fps avg min highest lowest"
"Preloading 59 50 67 22 PvPscene_BM_LongViewDistance (takistan)"
"Type avg fps avg min highest lowest"
"Benchmark 62 52 70 42 PvPscene_BM_LongViewDistance (takistan)"
"Type avg fps avg min highest lowest"
"Preloading 55 44 73 13 PvPscene_BM_City (Zargabad)"
"Type avg fps avg min highest lowest"
"Benchmark 58 47 73 14 PvPscene_BM_City (Zargabad)"
"Type avg fps avg min highest lowest"
"Preloading 70 59 83 26 PvPscene_BM_OpenFields (Zargabad)"
"Type avg fps avg min highest lowest"
"Benchmark 74 66 85 26 PvPscene_BM_OpenFields (Zargabad)"
"Type avg fps avg min highest lowest"
"Preloading 31 28 35 2 PvPscene_BM_LongViewDistance (Zargabad)"
"Type avg fps avg min highest lowest"
"Benchmark 32 29 35 24 PvPscene_BM_LongViewDistance (Zargabad)"
"Type avg fps avg min highest lowest"
"Preloading 72 58 84 24 PvPscene_BM_OpenFields (ProvingGrounds_PMC lite)"
"Type avg fps avg min highest lowest"
"Benchmark 75 66 85 38 PvPscene_BM_OpenFields (ProvingGrounds_PMC lite)"

[[DirTyDeeDs]-Ziggy-]

= PvPscene Benchmark Suite

The goal is to create a better system to benchmark the ArmA engine to
understand the effects of different game video settings, different hardware
components and system configuration.

in line with the goal of this 'addon', please submit your results to the online database so they can be compared with other results.
Most of the fields can be copy pasted.
Adding your information to the database will also express your appreciation to the author for the hard work that has gone into this effort.