Dedicated server performance

[Dwarden]

definitely as pointed out, MinErrorToSend default value was changed to 0.001 since 1.60 and new setting was introduced: MinErrorToSendNear

read updated
http://community.bistudio.com/wiki/basic.cfg
more details explained by SUMA: http://forums.bistudio.com/showpost....45&postcount=5

additionally take in mind new packet sizes were introduced (so make sure Your MaxSize* values aren't higher than 1300 but that's not case of this server in OP)

---------- Post added at 10:56 ---------- Previous post was at 10:50 ----------

I've read all your posts paying a lot of attention to all the matters.

One thing that really interested me, following your suggestion about the regularCheck command is this:

From the BIS WIKI:

If you do not include the regularCheck option or set regularCheck=""; it will be activated. This means the server checks files from time to time by hashing them and comparing the hash to the hash values of the clients. Since newer server versions this has lead to some lag spikes on certain systems, because the whole file is hashed in one burst. (The heavy I/O operation essentially blocking the whole server application for 1-5 secs, depending on the file size)
regularCheck is also known to cause sporadic (10 mins - 2 hrs) disconnects, terminating the client with "You were kicked off the game." on the client side and "Player Test disconnected." in the console log. To turn this function off, write regularCheck="{}";. But beware, this will also make the server more prone to cheating (even though most cheats are averted when connecting)

The description of the effects of the regularCheck on "some" systems is amazingly similar to what I've come up with in fact my own description of the problem in the first post was:
"we have higher FPS which actually stays in the 30-40 zone....but suddenly it goes down to around zero, than after a second it comes back up to high values. This gives smooth game for about 30 secs and than a big lag to everybody: unplayable.
Further investigating showed that maybe those lags can be due to sudden big I/O operations so I thought of a disk bottleneck and started looking at the possibility of using a ramdisk and tried the Fancycache program, but we are still having these problems."

How can I disable regularCheck?
Is it the regularCheck="{}"; that Das Attorney suggested?



Thank you.

obsolete info,
unless someone proves me it's still happening on actual latest betas....

also do not disable the regularcheck as you technically sack all the security with that decision

i see where you get this http://community.bistudio.com/wiki/server.cfg#Comments

the page was re-created in June 2011 that means the comment is even older (1.59 or 1.58 RC times)

thus the info is completely flawed for actual release 1.60 and 1.61 betas - i cleaned up the BIKI page and replaced it with bit more valuable info

[Sibolo]

Thank you Dwarden for clearing this out, although I was hoping it could be a solution.
I have disabled this checks and I must say that I haven't had those lag spikes anymore. On the most serious test we made the spikes were gone, but for some strange reason we had other problems like scripts not initialized, and about 20 people suddenly moved to open sea after a few minutes of play. So the test was aborted after about 10 minutes from start. When we moved to the other server, we managed to play for about 1 hour and then we had an enourmous freeze for about 3 minutes. When the server recovered, a lot of peolpe were kicked out by BE, and could not even see the server in the list to rejoin. The remaining people could instead continue playing.
Another strange thing that is happening (since 1.60) is at mission start, we have a syncronization process in which the clients wait for public variables to be initialized by the server. This used to go almost unseen previously, but now it always takes about 1 minute, (in fact the first time we believed the server had frozen or crashed), but then it recovers and seems to work fine.

All these problems are really making it hard to continue and keep up our ATC campaign, so I ask you all to continue suggesting solutions to these problems.

Thank you.

[MadDogX]

I'm a little confused on the math regarding MaxMsgSend. According to the Biki, the value determines the maximum number of messages that can be sent "per simulation cycle". By my understanding, a server can perform up to 50 simulation cycles per second, meaning any bandwidth calculations would have to be multiplied by 50.

With that in mind, setting the "MaxSizeXXX" values to 1300 (bytes per message) and MaxMsgSend to 1024, we would be sending 1300x1024=1331200 bytes per message. Per second would be multiplied x50, so 66560000 bytes, or ~63.48MB. Per second...

Isn't that a little too much for a 100MBit connection, which should have a maximum throughput of 12.5MB/sec? Looking at this post, it seems to me that either my understanding of the values is wrong, or VisceralSyn is missing something.

[VisceralSyn]

If the server 'Frame per second' is synonymous with "per simulation cycle". The only time the server will ever be at 50FPS is when its idle, or you are running a mission with 0 scripts in it. You'll see the server sends very little data when its not churning away on scripts, easy to test in a 100Mbps LAN.

That said, MaxMsgSend would be used for setting the maximum amount of messages, which should allow you to control bandwidth. The MaxSizeGuaranteed and MaxSizeNonguaranteed, when changed from default should net you some very large numbers. While its very hard to confirm, with google-searching, I vaguely came up with using smaller messages is supposed to make gaming more efficient, while large message sizes enhance file transferring. This comes into the territory of Gigabit Ethernets' jumbo packets.

OK: my math, MaxMsgSend, is what should the maximum amount of messages you want sent. Remembering that data is measured in Bytes, and data transfer measurements in bits.

MaxMsgSend=1024
MaxSizeGuaranteed=1300 ( 1300 is the max size in bytes, in bits thats 10,400bits )

1024*10,400bits = 10,649,600bits ( 10.6Mbps )
If you want to assume that the server is sending that up to 50 times a second, at the most. Then yes you should come up with an insane number like 532,480,000bits.

In 100Mbps LAN I use MaxMsgSend=8192; and MaxSizeGuaranteed=484; and MaxSizeNonguaranteed=242;

484byte = 3,872bits
3,872 bits * 8192 = 1,874,048bits

and If you multiply that times 50, the maxbandwidth used should be 93,702,400bits. Which, should come up to about 93.7Mbps. Or in case you want to use bytes/s= 11.71MB/s.

So yes, if you don't do your math right, you get real funny numbers.

Also I use this tool, for testing different network settings.

Course, I might also be missing something, too!!!

[MadDogX]

1024*10,400bits = 10,649,600bits ( 10.6Mbps )
If you want to assume that the server is sending that up to 50 times a second, at the most. Then yes you should come up with an insane number like 532,480,000bits.

I don't want to assume anything, I'm simply trying to figure out how to optimize the server bandwidth settings. And since the Biki describes MaxMsgSend and messages per cycle, not per second, I believe this to be correct. Of course the server never really maintains that framerate in large missions, so we could use something more realistic for our calculations. Let's say 30. Still doesn't help much.

So yes, if you don't do your math right, you get real funny numbers.

Which part of my math is wrong? Between this comment, the "insane number" comment, and your attitude as a whole, you're coming off a little condescending for my taste. Which is funny, because:

484byte = 3,872bits
3,872 bits * 8192 = 1,874,048bits

and If you multiply that times 50, the maxbandwidth used should be 93,702,400bits. Which, should come up to about 93.7Mbps. Or in case you want to use bytes/s= 11.71MB/s.

Emphasis mine. That highlighted part didn't even look plausible at first glance, so I did the math myself. 3,872 bits * 8192 is 31,719,424 bits. How the hell did you come up with a number over ten times lower?

Anyway, if you multiply that times 50, you get around 1.5Gbps. Even if you assume the server fps will never go over 30, it'll still need a gigabit connection to send that number of messages. Unless, of course, my math went insanely wrong somewhere.

Also I use this tool, for testing different network settings.

Thanks, I'll check that out.

[MadDogX]

Okay, I thought it would be best to PM Suma about this, and he has just confirmed that a "simulation cycle" as mentioned on the Biki is, in fact, a "frame" on the server.

Am I correct in interpreting "simulation cycle" as "frame"? Meaning that if MaxMsgSend=128 and the server FPS is at 20, the server will send out a maximum 20*128=2560 messages per second?

Exactly. The above is correct.

In other words, the server framerate must be taken into account when calculating maximum bandwidth:

"maximum bandwidth usage in bytes per second" = MaxMsgSend * Framerate * MsgSize

If we plug in 1024 for MaxMsgSend, 40 for the framerate (fairly conservative), and 1300 for both MsgSize values:

1024 * 40 * 1300 = 53248000 bytes = ~50.78 MB per second.

Too much for a 100MBit server, as 100Mbits = 12.5 MB/sec. (Though in practise it's more like 11 MB/sec.)

All things considered, it seems like the default MaxMsgSend of 128 isn't so bad after all.

[.kju [PvPscene]]

Well done MDX. Can you guys please merge the info into the BIKI page:
http://community.bistudio.com/wiki?title=basic.cfg

[friznit2]

Interesting read and will definitely have a go tweaking some of these settings.

VCB and 16AA just attempted a joint op on our server with 65 people. Needless to say it wasn't so much unplayable, as completely unplayed - we never got beyond the load screen. We were running several popular scripts (ACE, ACRE, UPSMON) and we had TS running on the same dedicated box.

We eventually gave up and restarted with just 25 of us. After a several shaky minutes at the start it calmed down and despite a low server framerate (4) it was very playable.

Given that PvP clans regularly play with those numbers and other coop groups have reportedly played with 70+, I'd be very interested to hear any other tips on how to make this work.

I don't suppose there's anyway to test multiple clients without imposing on 70 people to spend 2 hours watching a server pile in?

[MadDogX]

Well done MDX. Can you guys please merge the info into the BIKI page:
http://community.bistudio.com/wiki?title=basic.cfg

Okay, I've added some info on bandwidth optimization: here.

I've tried to keep it simple with some examples, but it can definitely use some polishing.

[.kju [PvPscene]]

Don't forget that AI means additional traffic AND CPU use - BOTH for the server AND the clients.