Point Drop per Work Unit.

Message boards : Number crunching : Point Drop per Work Unit.
Message board moderation

To post messages, you must log in.

AuthorMessage
Conan

Send message
Joined: 3 Jan 21
Posts: 5
Credit: 187,663
RAC: 1,174
Message 1126 - Posted: 2 Sep 2021, 3:28:48 UTC

Recently I have been getting between 10 and 16 points per WU, with run times from 1000 to 1500 seconds.
Today I noticed that a number (at least 5) have been granted from 1 to 4 points for similar run times.

Any idea why? or are we using Credit New so it gives what it wants no matter what?

Thanks
Conan
ID: 1126 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Bryn Mawr

Send message
Joined: 16 Aug 21
Posts: 5
Credit: 1,158,210
RAC: 12,358
Message 1127 - Posted: 2 Sep 2021, 7:24:27 UTC - in response to Message 1126.  

Recently I have been getting between 10 and 16 points per WU, with run times from 1000 to 1500 seconds.
Today I noticed that a number (at least 5) have been granted from 1 to 4 points for similar run times.

Any idea why? or are we using Credit New so it gives what it wants no matter what?

Thanks
Conan


I’ve also seen a points drop but there’s also been a corresponding drop in time taken - I see nothing as low as your points, my lowest has been only just sub 10.
ID: 1127 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
hoarfrost
Project administrator
Project developer

Send message
Joined: 11 Oct 20
Posts: 151
Credit: 11,396,033
RAC: 21,493
Message 1133 - Posted: 4 Sep 2021, 8:58:42 UTC - in response to Message 1126.  

Any idea why? or are we using Credit New so it gives what it wants no matter what?

Hello Conan!

We reviewed a data related to two workunits of your computer 2983. For each task took near 1150 seconds, but for first task computer get 5.76 CS and for second 18.96.
In computations for this two workunits 3 computers were involved. And one of coefficient (named like 'correction factor') for "wingman computer" in workunit with low credit differ several times from another two computers. This coefficient computed by project server for each computer by set of tasks completed by it. And it used on computing of claimed and granted credit. This process is a part of CreditNew system (part Computing averages and further). This coefficients change during computer participation and claimed / granted credit of different computers is balanced over time.
ID: 1133 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Vato
Avatar

Send message
Joined: 23 Oct 20
Posts: 3
Credit: 410,442
RAC: 715
Message 1135 - Posted: 4 Sep 2021, 11:10:03 UTC - in response to Message 1133.  

that's a reasonable description of why CreditNew is widely disliked.
you get different credit based on what someone elses computer does!
ID: 1135 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
csbyseti

Send message
Joined: 18 Mar 21
Posts: 2
Credit: 1,138,535
RAC: 27
Message 1204 - Posted: 18 Sep 2021, 16:03:53 UTC

The whole Credit System of Sidock is bullshit
1. the difference of workunits in the same group does not reflect any calculation time.
2. the credits for the short ~30 min runtime WU can be higher than for the long 6 hour runtime WU.
3. if i calculate the Number of threads of my 5820k CPU running 11 Threads up to the 23 Threads of my 3900x = doubling the Rac of the 5820k
i'll see a leadership of the 5820k of about 15% but the 5820k has more WU's in pending. It's not the Pending !
But the calculation time of the 5820k is ~9hour for the long WU's instead of ~6 hours of the 3900X
So the Number of Results is much higher with the 3900X

The rac of the 3900x should be min 33% higher and not 15% lower.
It could not be that a much faster and much more energy efficient CPU get lower Credits for a better work.

looks like Intel has paid for this credit system.

https://www.sidock.si/sidock/hosts_user.php?userid=2582

greeting

csbyseti
ID: 1204 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
hoarfrost
Project administrator
Project developer

Send message
Joined: 11 Oct 20
Posts: 151
Credit: 11,396,033
RAC: 21,493
Message 1207 - Posted: 19 Sep 2021, 7:01:36 UTC

Yesterday we separate templates for different kinds of tasks and began step-by-step changes for bring the values into them to the realities of corresponding tasks.
ID: 1207 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
pschoefer
Avatar

Send message
Joined: 1 Jan 21
Posts: 9
Credit: 1,408,324
RAC: 2,174
Message 1209 - Posted: 19 Sep 2021, 8:22:54 UTC - in response to Message 1204.  

looks like Intel has paid for this credit system.

This is a nice conspiracy theory, but it really is not an Intel vs AMD (or Windows vs Linux) issue. It is just CreditNew turning things into a lottery, as it is prone to do.

Basically, for CreditNew to work smoothly, the following two assumptions have to be met:

  1. task run time scales reasonably well with estimated FLOP count (definitely not true here until now, as both the short 3CLpro and the long Eprot tasks had the same FLOP count estimate; I don't know yet how it will work out once hoarfrost's recent changes take effect)
  2. computation speed of each computer is constant (a very bold assumption, especially with modern CPUs and GPUs that can adjust their clock frequencies on the fly based on temperature or power draw)


As soon as reality deviates from these assumptions, CreditNew does all sorts of weird things that may or may not average out long term (and definitely not short term, e.g. over the duration of a typical competition like the one going on right now). I have two screenshots to illustrate the complete mess CreditNew created here:


  • A set of 3CLpro_v5 tasks. The longest task took ~50% longer than the shortest, but credit varies by a factor of 4.3 and there is no correlation between run time and credit. And this is just a small subset, I have also seen 3CLpro_v5 tasks with credit as low as ~20 and as high as ~210, i.e. 10 times the credit for roughly the same amount of work.
  • A set of Eprot tasks. In this case, the longest task took only ~10% longer than the shortest, but again, credit varies a lot more and there is no correlation with run time.


Three years ago, even David Anderson came to realise that CreditNew is not working all that well and now recommends it only in cases where no better option is available. Let's take a look at the other options:

Pre-assigned credit: I think this is the best option available. There could be a fixed amount of credit based on the target, e.g. 30 Cobblestones for 3CLpro tasks, 450 Cobblestones for Eprot tasks. Yes, as shown above, there is some variation in task run time even for the same target, but a ~50% difference in credit per second is much better than the ~1000% difference we see with CreditNew. It is also cheat-proof, device-neutral, and immediately rewards a CPU switching to turbo mode rather than punish it. The downside is of course that it takes a bit more work when preparing a new target, as the right amount of credit needs to be known in advance.

Post-assigned credit: AFAIK, the run time variations are caused by different run times for the docking simulations for each ligand, while the number of ligands processed in each task is constant. So this credit option would require implementing some sort of FLOP count. I don't think this is feasible.

Runtime-based credit: In theory, this sounds like a good choice for this project, as long as there is no GPU application. In reality, however, it is a complete nightmare that really should not be used by any project any more. It already was a nightmare back when this was the standard credit system, because some users used clients that reported inflated benchmark values; nowadays, the benchmark does not really mirror the true CPU performance even without those "optimisations", because it runs on only one CPU core and the CPU might therefore run at a lot higher frequency than during the actual computations.

Ironically, that leaves adaptive credit aka CreditNew as the recommended option for exactly those cases where it performs the worst. I would argue that even in those cases, some fixed amount of credit per task would be much less of a lottery than the mess created by CreditNew.


ID: 1209 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
walli

Send message
Joined: 2 May 21
Posts: 5
Credit: 4,085,318
RAC: 9,963
Message 1210 - Posted: 19 Sep 2021, 13:02:33 UTC - in response to Message 1209.  

I had some trouble viewing the images. For everyone who also runs into problems, here are the screenshots hosted on imgur:


ID: 1210 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
csbyseti

Send message
Joined: 18 Mar 21
Posts: 2
Credit: 1,138,535
RAC: 27
Message 1216 - Posted: 20 Sep 2021, 8:56:27 UTC - in response to Message 1210.  

First, sorry for my not nice word in the open line of my post but this expresses really good what im am thinking about the credit system at this moment of typing.

As psschoefer knows, i am a Boinc User for a really long time. So i got some experience in that area.
"looks like Intel has paid for this credit system." should be a joke because all slower CPU's got more Credit for at last the same WU.

I compared only my Boinc only systems which do nothing else.
Even my 3900X have nearly a constant Frequency if they run only one sort of computation.

I agree that a fixed credit system for the same Group of workunits would be the best solution for this Projekt.
Even if the runtime differs 100% in a group it will be ok.

Perhaps it would be a good idea to create a team badge for participating in this event.

Thanks to hoarfrost that he is working on the problems
ID: 1216 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Timber

Send message
Joined: 13 Nov 20
Posts: 6
Credit: 913,164
RAC: 2
Message 1359 - Posted: 15 Nov 2021, 7:02:21 UTC

Has this been fixed? or are we still on the credit screw?
ID: 1359 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
hoarfrost
Project administrator
Project developer

Send message
Joined: 11 Oct 20
Posts: 151
Credit: 11,396,033
RAC: 21,493
Message 1360 - Posted: 15 Nov 2021, 20:11:41 UTC

Hello Timber! Several weeks ago we change estimated number of FLOPS per workunit to values close to real amount of computations for workunits.
ID: 1360 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Point Drop per Work Unit.

©2021 SiDock@home Team