Will not queue more than about 4 work units per core despite set for 10 days of work

Questions and Answers : Windows : Will not queue more than about 4 work units per core despite set for 10 days of work
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile danwat1234
Avatar

Send message
Joined: 10 Feb 22
Posts: 19
Credit: 352,896,040
RAC: 1,109,842
Message 1699 - Posted: 14 Aug 2022, 3:02:01 UTC

i have noticed that for say an 8th gen Intel box, 6 core, it won't queue more than about 24 work units at a time despite having my queue settings maxxed out in BOINC at 10 days in both fields. Each work unit usually takes less than 2 hours some less than 1 hour. Is this a bug or perhaps my BOINC clients are confused by Rosetta occasionally sending a few work units out so it is holding off on queuing up SiDock?
ID: 1699 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jm7

Send message
Joined: 31 Jul 22
Posts: 16
Credit: 30,465
RAC: 0
Message 1700 - Posted: 15 Aug 2022, 19:54:10 UTC - in response to Message 1699.  

There are a couple of obvious possibilities.

1: The project does not have work to hand out right now.
2. If any task is in any danger of not being returned on time, BOINC will not fetch new tasks. This depends on settings and usage history.
ID: 1700 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile danwat1234
Avatar

Send message
Joined: 10 Feb 22
Posts: 19
Credit: 352,896,040
RAC: 1,109,842
Message 1701 - Posted: 15 Aug 2022, 21:46:19 UTC - in response to Message 1700.  

Thank you.
1. SiDock does consistently have work "Tasks ready to send 22058" right now on server status page.
2. Deadline for SiDock work unit is at least 3 days after receiving so that's not the issue since the work units don't take long to compute.
My settings BOINC settings are maxxed out and my machines do nothing but BOINC 24/7. Also the settings on my online SiDock profile is maxxed out
Store at least 10 days of work
Store up to an additional 10 days of work
Switch between tasks every 120 minutes
Request tasks to checkpoint at most every 60 second

Hmmm.
ID: 1701 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile danwat1234
Avatar

Send message
Joined: 10 Feb 22
Posts: 19
Credit: 352,896,040
RAC: 1,109,842
Message 1703 - Posted: 17 Aug 2022, 17:19:06 UTC

Seriously in all my 50+ machines, if i unplug a WIFI stick, it stops crunching within 3 hours despite 10 days 10 days set locally and also server side.
ID: 1703 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jim1348

Send message
Joined: 30 Oct 20
Posts: 57
Credit: 9,112,528
RAC: 0
Message 1704 - Posted: 17 Aug 2022, 18:42:23 UTC - in response to Message 1699.  

i have noticed that for say an 8th gen Intel box, 6 core, it won't queue more than about 24 work units at a time despite having my queue settings maxxed out in BOINC at 10 days in both fields.

There is a limit, based on the number of cores you have. I am not quite sure what it is, but I have never gotten more than a day's work either on any of my machines (currently a Ryzen 5950X).
ID: 1704 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
hoarfrost
Volunteer moderator
Project administrator
Project developer

Send message
Joined: 11 Oct 20
Posts: 323
Credit: 23,092,230
RAC: 11,084
Message 1705 - Posted: 17 Aug 2022, 22:13:42 UTC
Last modified: 17 Aug 2022, 22:16:30 UTC

Hi folks!

Yes, it's a server limit. Very useful for some purposes (within the existing infrastructure). But you can get around it by using ncpus tag in cc_config.xml during task request.

Thank you for participation!
ID: 1705 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile danwat1234
Avatar

Send message
Joined: 10 Feb 22
Posts: 19
Credit: 352,896,040
RAC: 1,109,842
Message 1706 - Posted: 18 Aug 2022, 11:04:25 UTC - in response to Message 1705.  

Thank you, I will try to find that file. Could you say why that limit is in place when the deadline for work units is multiple days out?
ID: 1706 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile danwat1234
Avatar

Send message
Joined: 10 Feb 22
Posts: 19
Credit: 352,896,040
RAC: 1,109,842
Message 1707 - Posted: 18 Aug 2022, 11:30:21 UTC - in response to Message 1705.  

Thanks, could you say why that severe limit is in place when the deadline shown in the client is multiple days? The file does not seem to exist anywhere so I gather I create the file with only that tag within the SiDock directory. Correct format? I need because i am waiting on some WIFI sticks and would like to lessen the frequency of walking to the internet-less machines with a WIFI stick.
cc_config.xml
<cc_config>
<options>
<ncpus>15</ncpus>
</options>
</cc_config>
ID: 1707 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
hoarfrost
Volunteer moderator
Project administrator
Project developer

Send message
Joined: 11 Oct 20
Posts: 323
Credit: 23,092,230
RAC: 11,084
Message 1708 - Posted: 18 Aug 2022, 21:03:53 UTC - in response to Message 1707.  

Hi! Уour cc_config.xml file is correct! "Short tasks queue" on participant computers reduce a time that need for completion of whole tasks set (and any subsets within it). Of course, first of all it is useful for large tasks (when for each tasks a hours of CPU time is needed).
But if tasks became short, drastic increase of number of workunits in "In Progress" state will "inflate" a project database and server work more "heavy".
ID: 1708 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile danwat1234
Avatar

Send message
Joined: 10 Feb 22
Posts: 19
Credit: 352,896,040
RAC: 1,109,842
Message 1710 - Posted: 20 Aug 2022, 11:40:41 UTC - in response to Message 1708.  

Thank you i see what you mean. Many of the work unit groups can be crunched by the community in a week or so I have noticed and you run a tight ship. One group gets to about 98% and then the next group begins to have results sent in and within a few days later the first group is completed no gaps in completion. Enabling larger queues would increase server load by computers not using as large % of the work units given to them 4 whatever reason and more needing to reissued.
Is it possible to get a list of future groups to be crunched on the site? Is there an 'end' to this stage of research where the community has completed all work 4 Covid?
ID: 1710 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile danwat1234
Avatar

Send message
Joined: 10 Feb 22
Posts: 19
Credit: 352,896,040
RAC: 1,109,842
Message 1711 - Posted: 20 Aug 2022, 11:48:39 UTC - in response to Message 1708.  

I should note that even though you have a server side limit of work units sent for a given estimate computational power of a computer, that does not mean you will have a same-day turnaround. If the user has multiple projects going they could have a significant queue to crunch through before resuming SiDock, depending on BOINC % resource allocation settings and luck. But the proof is in the pudding, err Project Status work unit group completions.
ID: 1711 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Natalia
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 9 Oct 20
Posts: 181
Credit: 2,689,264
RAC: 75
Message 1716 - Posted: 24 Aug 2022, 8:01:47 UTC - in response to Message 1710.  

At the beginning, we followed the list of targets (groups of work units) that was published in the presentation. Today, new targets are selected following the intermediate computational results and new scientific data. For instance, the target PLpro was investigated 4 times instead of 2 planned initially. This is why it is difficult to publish a planned list in advance.
ID: 1716 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jm7

Send message
Joined: 31 Jul 22
Posts: 16
Credit: 30,465
RAC: 0
Message 1726 - Posted: 5 Sep 2022, 17:33:02 UTC

If I recall correctly, BOINC assumes that when you ask for 10 days worth of tasks that the computer will be disconnected from the internet for 10 days, and that any task with a deadline within that time is in danger of not meeting its deadline.
ID: 1726 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jm7

Send message
Joined: 31 Jul 22
Posts: 16
Credit: 30,465
RAC: 0
Message 1727 - Posted: 5 Sep 2022, 17:35:17 UTC

Minor correction:

Store at least X days - means that the computer coule be disconnected for up to that long.
Store up to an additional Y days. Means that it is OK to store more on the computer, but not required.
ID: 1727 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Questions and Answers : Windows : Will not queue more than about 4 work units per core despite set for 10 days of work

©2024 SiDock@home Team