Voting - Ticket System

[Suma]

To continue discussion where is it relevant:

Quote[/b] ]The votes for meta-bugs just indicate that the whole topic is important for the community and all related child should be looked into.

Actually most users vote on metabugs not because they think all related child should be looked into, but because some of them seem important to them (or sometimes even them mean some issues which they think are covered by this, but not specified in the BTS at all).

While not that obvius with anticheat metabug, which has only a few childern, thus is very obvious with AI metabug - http://bugs.armed-assault.net/view.php?id=2202. I think most people will agree some issues here are important, and other more a nit picking.

If the priority system should allow us to decide what to fix, it must allow us to identify a few top issues. If 30 issues are considered to be of same importance, it does not help us to make any decision at all.

I repeat: I ignore votes on metabugs, as they do not fit into my decision making process at all.

Quote[/b] ]What would be the point in meta-bugs otherwise?

I do not think they serve any real purpose, I think they support bad reporting habits. Bugs need to be reported as individual issues with concrete particular repro steps, not as "Everything is broken and needs to be reworked".

[sbsmac]

Now I've been reminded of this thread - and it does clearly state you would ignore meta-bug votes! .....

Quote[/b] ]I do not think they serve any real purpose, I think they support bad reporting habits. Bugs need to be reported as individual issues with concrete particular repro steps, not as "Everything is broken and needs to be reworked".

Can I respectfully suggest that this is not necessarily the best way to think of things ? (I don't at all wish to tell you how BIS should run it's develoment activities but I do have extensive experience in gathering customer requirements and bug reports so this is not just an academic post )


Classic bug-reports do come with clear repro steps - "do this and you will see that incorrect thing happen". In real-life any software without an explicit acceptance spec tends to suffer from less clear-cut complaints - "response is too slow" (what would be fast enough?), "GUI is confusing" (what would be clearer?), "can't transfer data whilst device is off" (doesn't user realise this is physically impossible?). The complaints are still useful feedback for developers in focussing on broad areas where effort might be spent. The danger of requiring very clear descriptions of features is that you run the risk of users trying to state the bug in the form of a required implementation. Eg, "server doesn't automatically disconnect users with 10 digit ids" when actually the problem statement is that the server does not detect fake users - 10 digit ids are just one possible symptom and obviously the implementor (BIS) is in a much better position to state the possible solution to the problem than the user is. The AI meta-bug is another good example. Many specific examples of potential AI improvements have been suggested but collectively they provide information to BIS that the AI behaviour is felt to be less than convincing. Some of the specific suggestions may be impossible to implement but you might be able to persuade users that AI had been improved by implementing a completely unrelated feature. (I'm reminded of the story of the Halo AI where MS found that user's perception of the AI was vastly improved when the aliens were made to verbally describe what they were doing, even though the AI routines themselves were untouched.)

Anyway, it is not my place to tell you how to prioritise your development but as someone who enjoys your products and would like to see them succeed I can only suggest that 'meta-bugs' are a useful source of market information even if they are difficult to use for specific actions.

[Suma]

OK. I guess I should make a little bit more cleared what I mean:

With votes on normal bugs, it is quite easy to have a formalised process how to decide which to fix, which might be something like:

* browse bugs with most votes
* evaluate the workload needed to fix them
* check their report quality (most of all reproducibility)
* re-asses their importance, and put them into internal queues for fixing

Such formalized process is very hard to be applied to metabugs, and as a results, they are skipped in this process.

Frankly, I think given the number of different feedback routes we have (including this forum, reviews, contacts with players) I do not think coarse feedback provided by metabugs provides any useful information not seen in the other channels.

Feel free to vote for any metabugs, it will do no harm, but in the end it is the same as voicing your opinion here in the forum. I would like to avoid those voting having impressing BTS voting is some magical wand which will immediately make sure what was voted for is fixed - in the end it ends in the hand of the very same people who are reading this forum.

On the other hand, if someone really wants to help us, he can do it by post a good bug report, or a good feature design - such ticket on BTS has much better chance of being accepted into our internal queues.

[sbsmac]

Quote[/b] ]I would like to avoid those voting having impressing BTS voting is some magical wand which will immediately make sure what was voted for is fixed

Absolutely agree. As I said elsewhere, I understand the difference between customer interest and developer priority ! BTS does at least give the reassurance that it is clear that BIS review it, even if requests are rejected. In that sense its value is actually as a feedback mechanism for users. (In my own business we found customers were much happier when support tickets were visibly tracked - even a rejected ticket caused less stress than an untracked ticket.)

I thought it was also quite interesting what you posted in the multiplayer forum...

Quote[/b] ]? Everyone knows cheating is an important problem. There is no BTS voting needed for this. Forum poll would be probably more appropriate for such subject.

One of the reasons the meta-bug and 'call for votes' happened was that I don't think it _was_ obvious what BIS's attitude to MP cheating was. Again, no disrespect meant - I'm sure that BIS believes this is an important issue - but in the absence of feedback it is easy for users to assume that an issue is being ignored or not noticed. Straying off-topic a bit here but maybe one way of giving feedback to the community would be an irregular 'BIS-blog' from various members of the development team ? Even an off-hand comment like "we've noticed the concern over on-line cheating on the forums and are considering ways to stop some of the more serious cheats" would go a long way to placating the users (and would probably take less time than posting to the forums as much as you have today )

[[DirTyDeeDs]-Ziggy-]

from what I understand, the BTS is not a tool for communicating to BIS about the game hacking. ok.

We must be content to voice our concerns about the hacking on the forum alone, and hope, without any official statements from BIS, that they are working on a solution.

heres to hoping

[.kju [PvPscene]]

Warning - very long post! Hopefully quality and useful though

It took me more than two hours to write that one.. and even
more time to think this through..
However sometimes it's hard to describe something with less
word, at least it's been very hard for me here.


At first, Suma sorry about the creating a meta for the public
cheating and promoting people to vote it!
I've been out of the ArmA community for 6 months just before this.
So i wasn't aware of it. Yet I agree mostly with you - more
about this later in this post.
Let's get to actual topic in here.



Preface

It is all about communication here.
You are right Suma that you have different channels for this.
Most are very time consuming and have a limited use.
PM and mail's are among the more useful ones I'd say.

However the CBTS has the potential to be a very efficient and
effective one!
The reasons are the technical framework can lead to high quality
and the technical gateway hurdle leads to more competent
people using this tool.

To get to this level, i'd like to propose a few changes.
There are three parties involved:
The community / reporters, the CBTS moderators and you (BI).

As preface - we can work on the first two,
yet we need your thoughts, comments and if you like the idea
and actual process, we need your commitment to this project.
From a realistic economic and updated over time perspective of
course - nothing is made forever!

Overall background:
Certain parts are moddable in ArmA, engine problems and
additions aren't.
However it even makes sense to get moddable parts into the
core as the core unmodified product is used by many people
(especially MP).

This could become a project to regain much lost credit and
sympathy from certain individuals.



Vision

It is all about time and money.
This means as long as the CBTS isn't useful to BI, it has no
reason to exist.
So the CBTS needs to become an efficient and effective tool for
bug and solution( ! ) reporting.
To some point it should be a tool to get improvements into ArmA(1).
Via various ways it should tell BI what issues need most and/or
the first attention.
It might serve as a platform to gather ideas for ArmA(2) as well.
Even more it could help improve ArmA(2) due to being based on
the same but improved engine.

In return to the hard work of the reporters and CBTS
moderators, BI is committing itself once the effort is appropriate
to integrate fixes and improvements into core product via game
patches.



Goal

~ Overall high quality in the reports. This includes:
Precise issue summary line.
Short and precise issue description (ie. max 10 short or 5 long
sentences).
Steps to reproduction required for bugs.
Big plus: actual solutions (ie for configs/missions or maybe
suggestions for script topics).
The additional information field can contain additional thoughts
and ideas. Everything which didn't fit in the description field.

~ Screenshots and small videos to visualize the problem / suggestion.

~ Demo missions once useful with short description and
steps to reproduce.

~ Active and strict filtering of issues before presented to you /
BI to lessen the effort on your side significantly. This is about a
recommendation system to tell you once issues are worth to
be checked from your side, as well as the general process
beforehand to get an issue to a quality level to take the
quality( ! ) barrier.

~ Active feedback about a ticket's status. No hundreds of dead
tickets. Either work in progress or resolved / closed. This
includes "Won't Fix / Never", "ArmA(2) / Future", "Not in this
patch, maybe in the following one(s)" (pushing tickets). People
prefer even a repulse / negative response than no response at all!

~ Community voted issues once the effort is appropriate might
get some priority.

~ However the actual criterion has to be quality. If we have a
very well presented problem / suggestion which just takes little
or appropriate effort, this one should have highest priority.
Especially if an appropriate and plug-in solution is presented
(best on the config/mission level).

~ _Goal_ No active tickets for the current work-in-progress
patch. Either the issue is work-in-progress (assigned), won't
be handled in ArmA(1) ("Won't fix or "future products") or is
assigned to a future patch version (somewhat the milestone
principle). That is the vision and the goal to be aimed for. The
practical part is only the process to get as close as possible to
the vision/goal!



Strategy


I) CBTS itself or report problems / suggestions

~ First and foremost is quality. This means people have to
provide quality and the CBTS moderators have to support and
even enforce this. Setting an issue to feedback means that the
ticket creator has to supply additional information and improve
the ticket quality. The requirements are outlined by the CBTS
staff. Of course other users can provide the requirements too.
A ticket in feedback status is no meant to get a review by BI.
The CBTS moderators have to change the status first after
giving the ticket a review for themselves.

~ Tickets with great report quality and especially with solutions
provided get a recommendation by attested "high quality" report.

~ Quality gateway: BI only looks at issues in the confirmed
status. None else!

The process works like this. The CBTS moderators review
issues and once the quality of a report seems sufficient, they
will set an issue to confirmed.
Now it's the the turn of you (BI). Once you (BI) need additional
information or cannot reproduce the problem, you will
document this with a few words and set the ticket back to
feedback. This triggers the process anew. As long as the
request isn't fulfilled the ticket won't get the confirmed status
by the CBTS moderators and therefore continue the process.
Otherwise you assign the ticket to yourself / BI members and
hopefully solve it as soon as possible.

Very important: We want you to be fair to yourself and the
reporter. If you think the issues is out of the scope of
ArmA(1) resources, please set it to "Won't fix" or "Future
products" - if you intend to review this for ArmA(2). Think
back to the opening, people prefer to know the truth and they
will accept it.

Another option in the process is to assign an issue to a patch
version after the one work-in-progress to highlight that this
one won't be resolved in the next patch. Via filtering you can
remove these tickets from your attention and also get them
back once the patch version is reached. This might be a bit of
an overkill though.

Yet the overall topic is here time spent. If you already take the
time to review a ticket and you decide not to work on it, you
have already spent quite some time for the review. The time
needed to use the technical options to close a ticket, change
the target version, request feedback just needs a few clicks
and a few words. Yet your part of documentation, your
thoughts about the issue are very crucial to the overall
process. Without it, no proper work flow is possible!

~ Priority could be used to have the CBTS moderators indicate a
possible priority from the community view (very soft and spare
use assumed though - NOT the same as number of votes. A bit
more unbiased hopefully).

~ Report quality "high", severity as well as number votes should
serve as the primary's indicator what should have priority of
attention on your side (BI).


II) SVN or provide solutions

Suma this is a topic which you have discussed a bit with IandC already.

The idea is to get solutions more into the focus of what we are
aiming for.

The simple part is just to report solutions in a CBTS ticket.

In addition to the CBTS we will provide an SVN server.
Everyone will have access to the SVN server with read access.
Everyone with a CBTS account will be able to commit changes
to the repository (write access).
Commit comments will be required and can be interlinked to
CBTS tickets (pre and post commit hook).
The repository will contain a basic ArmA installation - yet only
the standard ArmA configs in human readable format, scripts and
description.ext files.
NO textures, p3d and sound files! I hope that won't infringe your
copyright etc.

People will be able to checkout the repos and change the
standard ArmA files and commit them. By this you will be able to
get the diff for each commit, the diff itself. As long as commits
are thematic atomic (related to a specific issue solely), have
good commit comments and are linked to the correct ticket, this
option could serve as a good addition to the current CBTS
system.
Each commit will require a ticket to be interlinked with. So as
long as people won't abuse the system, this could generate
additional solutions on the code level.
This will just outline solutions, it will be still up to the quality of a
report and up to your decision whether to integrate a commit
into the core product or not. If not, a short comment in the
CBTS ticket from your side (after the ticket went through the
basic work flow described above) about the reason is to be made.
These commit and tickets can serve as concrete base for
discussion on the community side whether this should be
implemented or not.

Two practical examples:
~ AI infantry uses AT rounds on infantry - this is definitely not
in the interest of the community.
~ Standard AI values, ie skillXXX and precisionXXX, are not very
well set for the normal player. Different standard values are
in the interest of the normal player.

Severity tweak could be used to indicate that the issue "merely"
needs a config/mission/script tweak to be solved.

The concept of metabugs needs review. I think we should
disable votes in metabugs (close them) and metabugs should
only serve a stickies to pinpoint people to existing tickets of one
specific area.



Closing words

I hope you guys are still with me!

Hopefully Suma you like the overall idea of the suggestion or at
least parts of it.
After all it's down to your decision how to treat your product
and how much time and effort you still like to invest.
We can only try to lessen your effort and help you with making
this and maybe future products better.
Again i think this project could serve as strong positive basis to
heal wounds and to regain the lost respect amongst certain
individuals.

Your recent blog post is indicating that you plan to support
ArmA(1) in the future. So we have good faith.

Every feedback, thoughts and comments are very welcome -
especially how to improve the concept from the POV of you Suma (BI)!


best regards
Q


Credits and supporters

IandC
Boecko



PS: Now i have to hurry .. 3 hours late for work already ..
I'll try to polish to post tonight and fix spelling errors.