CSS - Community sign server

[.kju [PvPscene]]

Abstract

To avoid addon abuse in Multiplayer one needs to have the signature system active.

Currently every addon creator is meant to create his key and sign files to be
used by the players and server admins.

Instead an automated system to create the necessary files and provide them
to the players and admins could make everyone's life much easier.


Why?

• Addon makers do sign their addons at times.
• The process is complicated and annoying.
• One mass download and upload source for end users.
• No need to mess with the files. Just press dl all/update.
• Only need to add this one additional Modfolder with zillion of bisign files on the client.

Basic concept

• There is a server or set of server one can send any pbo to.
  
• The upload process would be best to open to anyone. If abuse is a problem,
  probably restricted ability to a given group of people might be the alternative.
  
• The server automatically looks into his database if the addon was already processed.
  
• If not the server generates a new key file based on the pbo name and signs the pbo with it.
  
• The pbo file is dropped. So this server does NOT act as a pbo download server.
  
• Instead the server ONLY provides the key and sign files.
  
• Now there is a client application to download all sign files in a very simple
  process. That is all for the client. He only needs to sync with the server
  before every game session (in which he wants to use addons).
  
• The server administrators have an access to the CSS to easily filter, select
  and download or sync the key files of their choice with their server.

Summary

• Client: Can easily download all sign files for any addon out there (and
  signed by the CSS). As sign files are very small, there is no problem to download thousands or more of em.
  
• Server: Admins have a central place to look for key files and easy access.
  
• Addon creator: There is no longer the need to worry about the sign process.
  Only needs to submit his work to the CSS for every release.

Challenges

• The CSS server. The server hardware to host the system.
• As sign and key files are very small, the download traffic should be minimal.
• The upload interface could be a bottleneck and abuse needs to be ruled out.
• The sign process is simple, however it needs CPU power. That said the
  number of PBOs is limited. So mainly spikes in CPU use, not overall huge load.
• Group of people to code the upload interface.
• Group of people to code the server admin interface to find, filter, select and
  download the key files of choice.
• For the client Yoma Addon Sync or Sickboy's git updater are already available.
• Group of people to code the backend processing
  
  • filter out already known PBOs via a hash check
  • dump the information about known pbos into a DB (hash, name, date, sign and key file)
  • interaction to upload interface and admin key file interface

Additional thoughts

• Both pbo has and file name could be used to find PBO files at community download sites.
  
• This system does NOT handle the problem of addons needed by a server or
  mission. This is a separate problem. Tools are in development to allow this.

FAQ

To be asked first.


Thoughts, feedback and participation in the CSS project is very welcome!

[whisper]

I'd add also on the server CSS the original addonmakers keys, for players who would not use this tool but have keys installed, so that servers are correctly updated

[.kju [PvPscene]]

So how should that work - automated upload of key by anyone or a internal team to do
that or just to accept or deny keys?

After all a key by cheater X can make all his modified addons and pbo based cheats available.

[whisper]

Mmhh, in this vein, what prevents a hacker to upload his hacked .pbo for having the key generated by the server itself?

[.kju [PvPscene]]

Nothing prevents him. The system works different.

It creates an unique key for each pbo.
So the clients can download all sign files (very small) and update easily. Nothing else to worry for em.
Server admins have a central server to pick keys from.
They still need to verify the pbo is not hostile themselves.
Either by looking at the content, by asking for other expert advice, to compare hash of
the pbo vs official release, by comment system etc.

[.kju [PvPscene]]

Possible implementation details

Design wise I think it is wise to start simple and still keep possible extensions in mind to a degree.

Pretext:
A very simple client side app could improve the system in the way to create a hash client side and
ask the server if the PBO was already signed.
If not, it could 7z the file before uploading to safe traffic / time. This could replace the web-front-end
altogether.


For the front-end indeed a simple web system should do the job:

1) Upload one pbo. Select the file located on your HDD.
2) Not sure if possible to do a client side hash generation. The hash in general is to compare if the file
was already uploaded. If not possible to do client side, the upload probably always has to happen.
Or do you have ideas - the name itself isn't sufficient.
3) If not done, generate a hash on the server for the pbo.
4) Compare the generated hash with all hashes, saved in a SQLiteDB / some DB of choice.
5) If already available, drop PBO and notify the uploader about the existing key.
6) The key is named just like the PBO along with a date tag. I think the day itself is enough like:
myPbo_2009_20_08
7) If has is not found, the system calls DSUtils\DSCreateKey.exe creates the key based on the said schema.
8) (Might be done later - just to get the idea) Move the created public key in a location accessible
to the common user. A location available via FTP might be the simplest solution.

This part can be extended though. Meaning either HTTP, or a web-front end to search and filter for
available keys.

For the location probably it might be a good to have .\bikey\datetag\myPbo_2009_20_08.bikey
Maybe the datetag folder level is unnecessary though.

9) We now use the created private key to sign the PBO:
Usage: dsSignFile private_key_filename file_to_sign_filename

This creates a bisign file in addition to PBO. Very small that is. Just a few bytes (306).

10) Check if the creation was successful.
Usage: dsCheckSignatures checked_dir keys_dir

So it is wise to have the PBO, its bisign file alone in a temporary folder during this process to avoid
complexity if another PBO is the reason for the app to fail.

If PBO, bisign and public key fit, the app will return nothing. On error it will report a message.

11) If successful (should be always the case unless PBO is corrupt?), drop the private key.
It is no longer needed and only used once to sign this one very PBO.

12) Create a hash for each PBO, sign and public key, put the hash, their filename and maybe more
information in the DB.

13) Depending on the server, drop the PBO or keep it available for download.
End user download probably would be way too huge traffic. It could serve as a master server for
other servers to mirror it though.

This part might be for a second/later phase to expand.
At least an API to get the hash of PBO for the respective key and/or sign file is possible.
This way an addon download server could query this one to get the key and sign file for the PBO.

14) Move the sign file to a folder with date tag, like .\bisign\2009-08-20\myPbo_2009_20_08.bisign.

15) At this point either have six installer or YAS create the file repository they need to deliver the files.
Both tools allow a very simple install and update system for the end user.

As stated the end user should always only download and update all. As the sign files are so small,
there is no problem to even download thousands of these.

Done.

The core element is to make the admins job easy to look through the available key files for him
to decide, find the one(s) he wants, and easily transfer them to his pc or maybe even server.
The key files are also very small. xxx bytes.

[[GCA] Salah ad Din]

Hi,

I have some questions:

Can your system somehow automatically update server key files for client side addons?

AFAIK, a client side addon does not need to be installed on the server (for example, simple interface changes like this).

Please correct me if I am wrong.

[galzohar]

The #1 issue with addons is that the server admin actually has to decide which addons are "ok" and which are "not ok", so you're going to need at least some kind of a comment system so people can comment on which addons are good and which are not, and even that probably won't cut it as you will still have most people wanting to use these "small addons that most people don't use and didn't have anyone comment on them" and not be able to because no admin should take the chance of approving unknown addons.

At the end, it seems like even if that system goes into a fully functional mode, it'll still be way too much work for admins to keep people happy.

[[GCA] Salah ad Din]

It's not really about keeping people happy, but about facilitating server management. I can see quite a lot of harmless, client side only addons that should be allowed. The problem here is that currently, for each of these addons a server admin must keep the keys up to date, even though he doesn't have to update any actual pbo files on the server.

For server side addons, this system is slightly overkill since you already HAVE to download the updated pbo files and then can just copy the key file.

[.kju [PvPscene]]

A server admin only needs key files, unless the addon introduce new classes or affects
AI behavior.

It is not the all-perfect-solution, yet its a doable one.
The design goal is to automate all aspects as much as possible.

We operate a similar system on a private basis. It works quite well.
There are only a small percentage of problematic addons.