A more balanced DC Vault score

[carlfrs]

Hi,
I have followed the discussion around how score is calculated in the DC Vault. It has been mentioned the unfairness of how projects with few teams gives much better score than projects with many teams, and that creating hoax teams can improve the score a lot in projects with few teams.

I would suggest that a better position in projects with many teams should give a "better" score than in projects with few.
One way to do this, is to use the number of teams in the project with most active teams as shown below:

Project score = 10000 - ((Position - 1) * (10000 / Total number of teams in the project with most active teams))

This will give a better distribution and a more stable score.
Activity and improvement in all projects will then be similar, and it is much easier to "plan" competitions.

What do you mean about this?

[russkris]

If people keep taking about, how to cheat, then more people will read it and get the idea...

Whats the real problem??????????????????????????????????????

[carlfrs]

If people keep taking about, how to cheat, then more people will read it and get the idea...

Whats the real problem??????????????????????????????????????

The real problem is that a project where a lot of teams compete you can improve your position by e.g. 10 positions, but still get a low score of the effort while in projects with few teams the situation gives a much better score. This means that mainstream projects like many boinc projects (with more than 1000 teams) are not treated fair wrt to more exotic projects where few teams are producing (less than 100).
By removing this unfairness, position 10 will have the same score in all projects similarly to position 1 that gives 10000 points in all projects today.

[Nanobot]

Let me start off by stating that the scoring system will not be changing but if you want to create your own version of the vault with your own scoring system you are welcome to download our consolidated stats files which contain the positions of each team in each project http://www.team-ninja.com/vbulletin/showpost.php?p=324898&postcount=4

During the planning of the Vault we went through a multitude of scoring mechanisms and all can be manipulated in one way or another, including yours, and the one we chose was done so because it provided the best compromise which covers projects containing team numbers from 19 to 44,676. The scoring mechanism you have come up with is very close to one which was rejected as being too unstable. Currently Folding has the highest participation so the divisor would be 21,196 but if someone were to create 5 Vault teams in Seti then Seti would have the highest participation so the divisor would become 44,676 and every teams scores would change dramatically.

Also you say that your scoring method would remove the unfair advantage of participating in the less popular projects but it would actually make it more unfair as if you were in poition 19 in PSP (last place) that would give you the same amount of point as a team in position 19 in Seti (a far harder thing to achieve).

[carlfrs]

Also you say that your scoring method would remove the unfair advantage of participating in the less popular projects but it would actually make it more unfair as if you were in poition 19 in PSP (last place) that would give you the same amount of point as a team in position 19 in Seti (a far harder thing to achieve).
That is exactly my point! Teams that get a higher position will earn e.g. 500 points for every change in position when only 20 teams. Even though there are only 19 teams in a project, this project will do a dramatical influence on the total score related to a project with more teams. You value the top position the same in all projects, but other positions are much less valued. The first place in SETI is much harder to get than the first place in PSP. Maybe that could be changed such that the projects with few participants/teams should have a lower top score (not 10000).

[N.V.M.]

the bottom line is that there is no perfect scoring system. the current system was decided upon after much thought, weighing many factors including the ones you bring up.

if it can be done "better", i'd like to see it.

[russkris]

Look the thing is it would be totally unfair to every other team in the Vault to change a scoring system.

I can pick big holes in BOK's stats but I dont post about it because I know the massive amount of work that went into it....

[carlfrs]

Look the thing is it would be totally unfair to every other team in the Vault to change a scoring system.

I can pick big holes in BOK's stats but I dont post about it because I know the massive amount of work that went into it....
I respect all the work and attention that has been used in this work. My "problem" or "critique" is not against DC-Vault or any of the funders. I am a small contributor, but acknowledge that the system proposed by DC-Vault has some merit. Anyway, it is not easy to see that my team is using a lot of effort to get a new position in a team that results in an increase of 0.21 points in a project that has many teams and at the same time see that we could increase our DC-Vault score by only inreasing our efforts in projects where a few teams participate. Our RAH is valued 9785 points in place 85. Our position 13th in PSP-S is worth 6000. It does not make sense...

[Toutouf]

I respect all the work and attention that has been used in this work. My "problem" or "critique" is not against DC-Vault or any of the funders. I am a small contributor, but acknowledge that the system proposed by DC-Vault has some merit. Anyway, it is not easy to see that my team is using a lot of effort to get a new position in a team that results in an increase of 0.21 points in a project that has many teams and at the same time see that we could increase our DC-Vault score by only inreasing our efforts in projects where a few teams participate. Our RAH is valued 9785 points in place 85. Our position 13th in PSP-S is worth 6000. It does not make sense...

Yes, it does make sense, as one of the known effect, or better, goal of DC-Vault is to help "little" or "not so popular" projects to get new members.
The way points are calculated on DC-Vault should make more teams join little projects where they would never have been involved in without DCV.
If you decide to increase your effort in projects where a few teams participate, I think this it is a good thing. :)