Re: [classad-users] Dependencies and repetitions in DAGMan

[Marc Tardif <marc@xxxxxxxxxxxxx>]

* Alain Roy <roy@xxxxxxxxxxx> [2009-06-05 11:43 -0500]:
> On Jun 5, 2009, at 10:33 AM, Marc Tardif wrote:
>> I just read some PPT presentation which gave me the impression it  
>> might
>> be possible to use a PRE script in a nested DAG which would call
>> condor_submit_day in order to generate nodes dynamically. For example,
>> lets say I specified the following DAGs:
>>
>>    # Filename: A.dag
>>    JOB A A.condor
>>    JOB B B.dag
>>    PARENT A CHILD B
>>
>>    # Filename: B.dag
>>    Script PRE B loop-script
>>    JOB B B.condor
>>
>> Then, my understanding is that it might be possible to build loop- 
>> script in
>> such a way that it could discover the available hosts and  
>> automatically
>> submit B.condor for each host. Is my understanding correct?
>
> Sort of. You could do that. You can write loop-script so that it does  
> anything, but you probably don't want to have it submit jobs. Instead,  
> you can have it edit B.condor. That way DAGMan will monitor it.
>
> An interesting way to edit B is to turn it into a DAGMan job. Then you  
> get multiple jobs for that single node.

I'm not sure I completely understand but I suspect this might provide
exactly the kind of flexibility needed to address my remaining use
cases for using condor. To make sure I understand, lets take part of
the above example:

    # Filename: B.dag
    Script PRE B modify-script
    JOB B B.condor

So, when you say "edit B", does that mean that the modify-script would
modify B.condor on the fly? For example, it could perhaps create the
file to contain multiple jobs:

    # Generated filename: B.condor
    executable = foo
    ...
    queue

    executable = bar
    ...
    queue

If this behavior of a DAGMan job changing a submit job on the fly is
documented somewhere, please let me know.

Thanks,
Marc