Mailing List ArchivesPublic Access |
![[Computer Systems Lab]](https://www-auth.cs.wisc.edu/pics/csl_logo.gif){kind=logo}
|
Mailing List ArchivesPublic Access |
|
Hi Richard,
I don't know much about DAG, but I've now read through the section in the manual. One thing I'm wondering is if the -maxjobs option will span multiple DAGs. For example, we can have several people submitting jobs - I suppose each submitter will end up using their own condor_submit_dag. In this case, I am still interested in limiting the total number of running jobs to a specific number, regardless of the number of submitters. Am I correct to say that the -maxjobs option applies only to one DAG?
Yes, you're right. So it doesn't do precisely what you want. I overlooked that, sorry. In principle you could work around that by having a custom made submit script which keeps track of how many licences are already in use and sets maxjobs accordingly. The licensed software must have a way of knowing how many licenses are in use, typically by means of a license server. You could query that with your script.
The other way DAGman could be useful to resolve your issue is:I assume that when your try to start a job when no license is available it fails with a specific error code (or can be made to do so by wrapping it in a suitable script). Well, then you make sure to keep resubmitting that job until it either runs or fails with some other error code. DAGman allows you to do that quite easily.
Regards, Patrick
Thanks, Richard. Patrick Huber wrote:Hi Richard,what about using DAGman and the -maxjobs option to condor_submit_dag. That does precisely what you want.PatrickHi Jason, This does work, and is sufficient to fix my problem for now.However, I see now that accounting does not do exactly what I want to accomplish. In my case, I have different types of jobs, with different numbers of available licenses. For example, Type A has 5 licenses and Type B has 10 licenses. What I really want to do is limit the number of jobs for each type, to the appropriate number of running jobs. But I see that the "sum of all quotas must be less than or equal to the number of machines in the pool." What will happen if this requirement is not met? In my case, I have 15 machines in the pool, so I may allocate quotas of 5 and 10, but if a machine drops out, what will happen? And if I want to add another group for license limit reason, I need to reduce the quota for the other groups (or add more machines). Is there any chance to implement a feature (or extension to the group accounting feature) that allow to specify a max number of jobs for a given group? In this way, I could have an arbitrary number of groups, with appropriately set limits, without the requirement that the sum of all quotas is > the # of machines.Thanks, Richard. Jason Stowe wrote:Richard, So accounting Groups are set to group.user@xxxxxxxxxxxxxxx I would set your +AccountingGroup="group_vsim@xxxxxxxxxxxx" or some such. You can also explicitly set autoregroup to false. Let me know how that works out, as there are more detailed steps to debug this if that doesn't work. Hope that helps, Jason_______________________________________________ Condor-users mailing list To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a subject: Unsubscribe You can also unsubscribe by visiting https://lists.cs.wisc.edu/mailman/listinfo/condor-usersThe archives can be found at: https://lists.cs.wisc.edu/archive/condor-users/------------------------------------------------------------------------ _______________________________________________ Condor-users mailing list To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a subject: Unsubscribe You can also unsubscribe by visiting https://lists.cs.wisc.edu/mailman/listinfo/condor-usersThe archives can be found at: https://lists.cs.wisc.edu/archive/condor-users/
--
Dr. Patrick Huber Physics Department
University of Wisconsin
Tel.:+1 608 262 2886 1150 University Avenue
http://pheno.physics.wisc.edu/~phuber Madison, WI 53706, USA