GIMPS?

[Biggles]

It's one of the oldest distributed computing projects around.

As far as I can tell it meets all your inclusion criteria.

www.mersenne.org (http://www.mersenne.org)

[Ragnarog]

I've briefly checked the page but I dont seem to be able to find any team stats?

All I see is a top individual producers list.

Rag

[Biggles]

Actually, that is the team list!

I'm far from a GIMPS guru, but the way the stats work is that an Account ID represents a team. Within an account ID it is possible to track computer IDs, which represent users on the team. This means that even the user stats are trackable, although they don't appear on the GIMPS homepage. An example of that is shown here (http://www.teamprimerib.com/rr1/bin/member.php).

There are very few DC teams that are present in GIMPS, although all the top users are of course teams. curtisc for instance is the Central Missouri State University team, curtisc being a legacy username and account ID. It has 700 computers on campus and 100 or so off campus. So really, teams do exist, just few of the DC teams actual run this as teams. From that point of view, that gives an argument for inclusion since it would encourage more distributed computing teams to get involved.

[Ragnarog]

Uh but it also gives a slight advantage to those who figured this means of "teaming up" out a long time ago :)

Can "teams" be merged?

Maintaining this could be painful as the teams don't have their proper names in most cases...

Rag

[LinearB]

The way GIMP works is different to nearly all other DC projects in that really teams are just the same as individual's accounts. There is no difference between a Team and an individual user. Tracking an individual's stats within a team is a lot more complicated as this informatioin is lost over time and requires a local stats system to maintain user contributions (admittedly tracking individual's contributions is not required for DC-Vault).

There is also the question of whether factoring is considered as another stat/project to collect as per 17orbust.

Personally I don't feel GIMPS Team support meets the requirements for DC-Vault, Although I feel on every other measure it does. This is only my opinion though and it's not my decision ;)

For the record Team Ninja does have an account ;)

[garo]

Being a longtime GIMPS veteran I can answer some of the questions here.

0. GIMPS is one of the oldest projects and is pretty large. They also found the largest ever prime last month. So I think GIMPs should be added!!

1. Yes it is possible to merge teams or for users to leave a team and join a new team - but the stats are not carried over just like in dnet.

2. Teams may not have proper names but that is true of a lot of different projects. For example, Ars gives a different name to each of its teams. However, the Ars team on GIMPS is Team Prime Rib - the same as SOB - being as it was, the first team called Team Prime Rib.

3. Almost all the big accounts are teams. There is a team for FreeDC, Northwestern, Ars, CMSU, UChicago, TeamNinja etc. etc.

4. LinearB admits that tracking individual contributions is hard for GIMPS but is not required. So I think GIMPS does meet all the criteria. It is a math project, team creation is very easy, it is an old and established project and is pretty large - about 4 times as large as SOB. Stats are updated every hour.

5. When you register for GIMPS first, there is a checkbox which allows you to state whether this is a team or not. So individual accounts and teams are not the same. I am attaching a screenshot of the registration screen as proof. Granted the team formation is different from most other projects but teams do exist and it is possible for existing individual accounts to be merged into a team.

6. I think factoring should be a different project as it is with Rieselsieve and Prime Sierpinski and should be with SOB sieving.

Hope this helps you folks make up your mind.

[LinearB]

Great Information there garo. I agree with all of your points, nearly ;)

Point 5, my concern really was that you can't distinguish from a stats point of view whether a name corresponds to an individual or a team, as luck would have it I can't get to the GIMPS site at the moment so can't check this, but that was my understanding.

Personally I'd love to see GIMPS added as it was one of the first DC projects I got involved with. I'll post back once I get a little more info.

LinearB

[garo]

Hi LinearB,
Thanks for your response. Unfortunately the server seems to be having problems for the past couple of days. Hopefully you'll be able to see for yourself when it comes back up.
Regarding point 5, you are right in that just looking at the stats you cannot tell if it is an individual or a team account. The only place where you mark that is in the tickbox in the screengrab I posted above. So basically you have two types of accounts - team and non-team (or individual) which are treated differently by the server. There is a single stats table though so it is not possible to tell which account is a team and which is non-team from the stats.
One way to look at it is that all accounts in GIMPS are teams. Just that many teams consist of a single user. And the possibility remains for that user to merge with another team at any time. A second way to look at things is that most other projects lump users without a team into a "no team" account whereas GIMPS gives them all their own "team".
Going by Ragnarog's criteria listed in the FAQ thread nowhere does it say that individual accounts need to exist. All it says it that there should be parseable team stats :)
Of course it is your decision ultimately but I hope I have done a good enough job of convincing you guys :D

[Ragnarog]

Almost convinced here :)

One more stats related problem would be that these are not _really_ team stats but a listing which mixes teams AND individuals.

Why is that a problem? Because the DC Vault points are based solely on _team_ positions.... now is there any list available that lists team accounts only? (can't check now as the site is down...but maybe someone knows)

Rag

[Nanobot]

I have just tried to grab all the stats pages and I stopped it after half an hour as it was only half way through, not very good when you campare it to the fact that the whole of the Vault stats collection process takes less than ten minutes. This would impact the whole of the Vault stats collection and calculation so GIMPS will not be included yet. If in the future the stats pages can be consistantly grabbed far quicker than currently I can see no reason not to include GIMPS.

[Biggles]

*bangs GIMPS drum again*

The server problems you experienced the last time you had a look have long since been fixed. Updates are hourly, and as long as you grab stuff after say... ten minutes past the hour, very quick. It can be a tad sluggish immediately following the update, which is why I recommend to leave it for that ten minutes or so.

I definitely think you should reconsider this. I know the setup is a bit different from most projects, but it is the oldest DC project and I think if three _alpha_ projects can be included, then so should this.

Factoring team list (http://mersenne.org/primenet/ftops.txt)
Prime testing team list (http://mersenne.org/ips/topproducers.shtml)

Factoring and prime testing are effectively two seperate projects, in the way that RieselSieve and RieselSieve LLR are different projects. I got both of those pages loaded fully in less than a minute. Don't think that can be considered too long, considering it has all the accounts in it.

[NeoGen]

The factoring team list link doesn't seem to show anything... at least to me.

The prime testing list shows me 28510 items, are all those really teams or is it teams and individuals alltogether?

[Biggles]

Going by the time of your post, you tried the link during the hourly update. It worked fine for me when I clicked it immediately after reading your post.

That is all accounts. An account is either a team or an individual. A bunch of people form a team by using a common account ID. There are few big teams on GIMPS, athough I know both Team Ninja and Free-DC have teams on it. I haven't looked through the list for other teams.

EDIT - a quick check found 2CPU and what I presume is the Dutch Power Cows. There was a dpc and a DPCows.

[NeoGen]

Yea... now the factoring list link worked. Just my luck to strike right at the update time... lol

But back to topic, since this is all accounts (members+teams), and the Vault's formula for calculating each team's points at a project specifically needs the total number of teams at that project... how can we tell how many teams are there on that mix?

[Biggles]

To all intents and purposes they are teams. Just a very high proportion of one man teams. I don't think that can be considered grounds for disqualification considering it's the same way that EulerNet works, and it's in the Vault.

[NeoGen]

No... actually Eulernet does clearly split teams from individuals. Indeed there might be many one man teams there in the list, but it's not their fault if people want to show off as owning their own team. (I also did that a long time ago on Seti Classic)

The top producers page at Eulernet has two tables, one for teams with 248 registered teams, and another for users, with 692 users.
http://euler.free.fr/top.htm

[Biggles]

Actually, that's not users in the second table, it's nodes. Which is why I'm in it more than once, and the top guy has many many entries. It doesn't distinguish between teams and users, just teams and nodes.

[NeoGen]

Well put there. I never thought it that way, as the project creator clearly wrote on the tables "TEAM" and "USER", but it makes sense for the repeated names.

So it makes that a team is in fact a collection of nodes, and not really of users. Still it is a better solution than on the GIMPS case, because not everyone is a team. I'm pretty sure that on Euler there's plenty of people on the nodes table that is not represented on the teams table. If they're teamless they won't show up as teams.

[Ragnarog]

:nuts::nuts::nuts::nuts::nuts::nuts::nuts::nuts:

Rag

[Biggles]

The teamless show up as teams too, just like GIMPS.

If you take me for example, I have two nodes that have run EulerNet, so they show up in the users table. They also have brackets after my name with Ars Technica Team Olive Oil in them, my team.

But if you take a random person from the users table, with no team name after their username (and so not a member of a team), you'll also find them in the team table. An example of that is tsider. He/she does not have a team name after their username, but are counted as a team anyway. Node/user number 677, team number 240.

EulerNet's "team" listing is exactly like GIMPS, in that it shows individual users as teams unless they properly join a team. The only thing that is different is that it shows each node with the username as well - but that isn't even needed for the Vault. Really, the top true team is Rechenkraft, not Laurent Evrard.

[LinearB]

Okay, getting silly here but I feel the need to join in ;)

No it isn't the same.
:p

For starters we have 2 lists, one Teams one Users. GIMPS has one list.

Now you'll notice the Team List only has 248 teams, Whilst Users has 692.

You'll also notice, LinearA appears in the Users but not in the Teams, wheras Team Ninja appears in the Teams (surprise, surprise) and not in the Users.

Now I agree if you don't elect to add your team name to your id then you do appear as a team but that's fine, you are a one man band so to speak. But from a stats point of view there are 248 teams taking part not 692. Now GIMPs on the other hand does not allow me to register as LinearA and then join a team i.e. Team Ninja, which are distinct entities. All other Vault DC projects recognise and allow this distinction, either through team names embedded within their usernames or by distinct team entities which allow you to associate a username with. So on these grounds GIMPS doesn't technically meet the requirements of the DC vault. The only way GIMPS supports teams is by effectively all crunching for the same username!

Having said all the above I would still like to see if we could add GIMPS to the DC-Vault. The problem is going to be the parsing the 47000+ "Teams". I will report back when I have a better idea of what we can achieve from a stats gathering/parsing perspective. In the mean time please feel free to continue the discussion, but just for the record GIMPS IS DIFFERENT!!!!

[NeoGen]

I say we should nag the project administrator to come here and answer us clearly how the stats work. :P

[N.V.M.]

i can just give you Linear's home phone number if that helps. ...:D

[Biggles]

It's still looking the same to me...

Look at it this way. In EulerNet, if you do not join a team, you become a team. Rechenkraft have 59 users, or more precisely, nodes that have run for that team. Thus, there are 59 nodes in the users list that are not in the team list. Each and every node that has ever returned a result is in the users table. It is possible to publicly view the stats for each node, although there is no way of checking the overall scores for a user.

Sure, you can do it manually by adding things since you can see each node, but there is no built in way of seeing the totals on a per user basis, only on a team or node basis.

In GIMPS, if you do not join a team, you become a team. If you join a team, you can still track the progress on a per node basis, as in EulerNet. The only real difference is that it isn't publicly available as you need the team password.

The only real difference I can see is that EulerNet makes it easy to see totals for a node. But that concerns user based stats and not team stats, and it's the team stats that count.

GIMPS, both primality testing and factoring, are:

Active
Accept new members and teams immediately upon registration
Have parsable team stats
Have hourly stats updates
Provide a client program that runs on a local PC
Is not a keylogger
Has no maximum number of users or teams


That's all the stated criteria.

I know 47,000 teams seems a lot, and a great deal of them are one-man bands. But you don't disqualify EulerNet on the basis that it's team stats has a lot of one-man bands. It's just a much smaller project. SETI has almost 39,000 teams and it isn't barred for having too many. UD has almost 44,000 teams and it's not too big. In reality, many of those teams have only one member also.

OK, GIMPS is a bit different from just about every other project out there. But there is one very similar to it, that being EulerNet, so it has a precedent. 47,000 teams to parse seems a lot, but considering modern processors can perform many millions of operations per second, it's not exactly insurmountable. What it comes down to is whether or not you want the grandaddy of all DC projects in the Vault. I personally think it deserves to be there easily as much as SETI does. It does after all meet all your stated criteria for inclusion.

I'd say yes, but not sure he'd appreciate a call after 11 pm to talk about GIMPS stats :D

[Nanobot]

As

The main problem is the number of "teams" and the time taken to load the stats page which is why we have the "parsable team stats" requirement. Even after the improvement of the server speed it took the parsing process ovr one hour to get the team data into a format which was machine readable for ALL teams, which is what is reqired for the Vault. Other projects with large participation, e.g. Seti, UD, distributed.net, provide alternate methods to obtain a teams position within the project which is why they can be processed. The amount of system resources required to process GIMPS would be three times that of all the other projects put together and on a shared server that is not acceptable.

I am not against adding GIMPS to the Vault as it is one of the founding DC projects but unless the stats are presented in a more "friendly" method then it will not be added.

Rather than "bang the GIMPS drum" here perhaps you could put some pressure on the project administrators to provide better stats so that it could be added, or find someone to host a dedicated DC Vault Server ;)

[LinearB]

2nd what the boss says.

P.S I'm still up at 0:45 AM GMT trying to get to grips with the stats we're already doing :D

[Biggles]

What if someone else were to do the parsing for you?

My idea goes like this - you provide a single page with team names of participants in the Vault that run GIMPS. Once a day or so, I parse that page and use the data from it to parse the GIMPS stats, and then I generate a page elsewhere with only the rankings of teams that are participants within the Vault? That way you would only have to parse a list of at most 320 team names (and realistically, it'd likely be far, far less than that).

That way it wouldn't take you long to parse the stats at all.

What do you think?

[LinearB]

This was going to form part of one of the options I was going to propose. We are currently a little busy trying to resolve some other issues within the DC-Vault at the moment but I'll look into it as soon as I can.

[russkris]

Just letting you know all is not forgotten.
We are still currently a little busy trying to resolve some other issues within the DC-Vault at the moment..


Can't forget a founding DC project :thumbsup:

[DigitalConcepts]

Has the month of May helped get things under control?

I think LinearB and Biggles have come close to an agreement.

Just a list of Teams and this could move forward.

[KAMCOBILL]

I'm not sure, but isn't the Account ID The Team?

I think this is the way to work the team as in readme.txt:

The User Information menu choice lets you change your name and email
address. Your name will be used in credits and "top producers"
web pages. Your email address may be used to send you a
newsletter once every few months. Select the "Receive occasional
newsletters..." checkbox to enable these emails. The userid you
use will appear in the status pages on the PrimeNet server. You can
also set the ComputerID field if you have several computers and want
to keep track of which computers produced which results. You can create
a team by choosing the "Create a team..." checkbox. The user name you
entered becomes the team name and a message is sent to the server which
prevents the team name from changing. Thus, you can handout the userid
and password for others to join your team and these team members are not
able to accidentally or maliciously change the team name that appears
on the statistics page.

[ArtfulDodger]

I know this is an old thread, but I wondering if the idea of including GIMPS in DC Vault has since been revisited? There seems to be a page with the stats for all teams that loads relatively quickly now. (http://mersenne.org/ips/topproducers.shtml) (assuming that was truly the only obstacle to its inclusion in the past)

[russkris]

OK... Kool... I will look into it