Re: [Condor-users] about multiple scheduler issues

["Carson Hung" <carson@xxxxxxxxxxxxxx>]

Thanks very much for your clear explanation.
I will setup a simple experiment in lab to test it.

One more additional question (really sorry for many questions).

In the negotiation cycle, the negotiator will match the resources to users
in what order. what I mean here is that, if I have 3 grid sites, each can
accommodate 3 jobs, and I have 3 users with infinity jobs queuing.

Suppose they are all with the same user priority and job priority. In this
case, so theoretically, each users should be able to submit 3 jobs to
these 3 grid sites. But the job of user 1 will all be matched to grid site
1 or distributed to 1 to each grid site.

Thanks,
Carson

>
>
> Carson Hung wrote:
>
>>I think what Erik's mean in replacing claiming of resources in condor-g
>> is by setting the CurMatches and WantAdRevaluate, right?
>>
>
> Yes.
>
>>However, I am not very understand why Dagman is necessary in this case?
>>
>
> DAGMan is not necessary in Condor-G matchmaking. DAGMan is used by USCMS
> to coordinate the different stages of a job (stage-in, run, stage-out,
> cleanup).
>
>>and in Dagman, all the jobs that is needed to be executed will be
>> pre-grouped into certain size and managed into a Dag file. In this
>> case, all the member of jobs in the same group can only be submited to
>> the same site, right?
>>
>
> Yes. Multiple DAGs are submitted. The matchmaking process directs them
> to target different sites, so each of the jobs within the DAGs are
> directed to the same site. This is just one way of doing things. It's
> not the only way.
>
>> and the decision will be done in the negotiator site, but not
>>the matchmaker site, right? is there any method that I can change the
>> scheduling based on the conditions of grid sites in matchmaker site?
>>
>
> The negotiator and the matchmaker are the same thing. The system
> described in "USCMS Matchmaking" is sensitive only to the number of jobs
> we have submitted to a given site and the maximum number of jobs we have
> configured for that site. To first order, this is a good way of doing
> matchmaking between grid sites. To second order, you may also want to
> monitor live queue depths at remote sites and feed this back into the
> site ClassAds, perhaps to lower the rank of busy sites and increase the
> rank of non-busy ones. However, I don't consider it necessary to worry
> about the second order optimizations until you have the first order ones
> working:)
>
> --Dan
>
>>>http://www.cs.wisc.edu/condor/USCMS_matchmaking.html
>>>
>>>Carson Hung wrote:
>>>
>>>
>>>
>>>>Hi Erik,
>>>>
>>>>One more question to ask, where can I find the USCMS writeup :)
>>>>
>>>>Thanks,
>>>>Carson
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>Hi Erik,
>>>>>
>>>>>Thanks very much for your clear explanation.
>>>>>
>>>>>You are right, I am actually asking if it is possible to have more
>>>>> than one negotiators with multiple schedulers. Since I think that
>>>>> case is possible to exist in condor-g environment.
>>>>>
>>>>>and I think you have answered my questions. Really thanks a lot.
>>>>>
>>>>>Thanks,
>>>>>Carson
>>>>>
>>>>>
>>>>>I have more understanding about the condor and condor-g
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>On Sat, Feb 19, 2005 at 04:40:24PM +0800, Carson Hung wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>Hi,
>>>>>>>
>>>>>>>Since it is possible to have more than one schedulers/submittors
>>>>>>> in the clusters.
>>>>>>>So I would like to know if it is possible that 2 or more
>>>>>>> schedulers match the same resource to the different jobs if they
>>>>>>> come from different schedulers/submittors.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>I think you really mean "if it is possible that 2 or more
>>>>>> negotiators match the same resources to different jobs if they come
>>>>>> from
>>>>>>different schedds" - you can't use the term 'scheduler' twice :)
>>>>>>
>>>>>>In Condor, it's not really possible for the negotiator to match a
>>>>>> resource twice, because:
>>>>>>
>>>>>>A. There's only one negotiator, so it will only match a resource
>>>>>> once per neogotiaition cycle.
>>>>>>
>>>>>>B. It is possible for a resource to be matched twice - if it gets
>>>>>> matched in one negotiation cycle, but doesn't update the collector
>>>>>> before the next  negotiation cycle starts. We prevent this from
>>>>>> happening in two ways:
>>>>>>  1. The negotiation CANNOT start until at least 20 seconds have
>>>>>>passed
>>>>>>since the last one. That usually is good enough.
>>>>>>  2. The startd can detect if it's been matched twice, and can
>>>>>>recover,
>>>>>>if the 20 seconds wasn't good enough.
>>>>>>
>>>>>>In Condor-G, because there is no claiming (ie the resource doesn't
>>>>>> automatically tell the collector that it is no longer able to be
>>>>>> matched), matching twice is more of a problem. We ususally build-in
>>>>>> some sort of  claiming when we've deployed matchmaking in the past
>>>>>> (see the USCMS writeup I pointed you at eariler).
>>>>>>
>>>>>>If you advertise your resource to more than one
>>>>>> collector/negotiator, then matching twice is a real problem. In
>>>>>> regular condor, it's not possible for a startd to report to two
>>>>>> different collectors, so it's not a problem.
>>>>>>
>>>>>>-Erik
>>>>>>_______________________________________________
>>>>>>Condor-users mailing list
>>>>>>Condor-users@xxxxxxxxxxx
>>>>>>https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>_______________________________________________
>>>>>Condor-users mailing list
>>>>>Condor-users@xxxxxxxxxxx
>>>>>https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>_______________________________________________
>>>>Condor-users mailing list
>>>>Condor-users@xxxxxxxxxxx
>>>>https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>>>>
>>>>
>>>>
>>>>
>>>_______________________________________________
>>>Condor-users mailing list
>>>Condor-users@xxxxxxxxxxx
>>>https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>>>
>>>
>>
>>
>>