Re: [HTCondor-users] A couple DAGman questions

[Michael V Pelletier <Michael.V.Pelletier@xxxxxxxxxxxx>]

From: Michael Fienen <mike@xxxxxxxxxxx>
Date: 07/26/2016 05:05 PM

> My issue is, the results of Job A will influence what queue I want
for JOB
> B. Your idea of using POST from A to write a file the PRE for B will
> generally work but can I then rewrite the sub file for B with the
PRE 
> SCRIPT such that I update the queue for B.
> 
> Specifically, in this case, Job A is splitting a large file into chunks
to
> be run in parallel by job B. Butâthe code in Job A breaks into a
difficult
> to predict number of such chunks.
> 
> Thanks, and I hope that makes sense.
> 
> Cheers,
> mike

Sounds like you should check out the utterly sweet
new "queue" options,
"queue from" "queue in" and "queue
matching" introduced in the 8.4
release.

In this case, you'd want the "queue matching"
statement:

===
matching - Each item in the <list of items with
file globbing> will be
matched against the names of files and directories
relative to the
current directory; the set of matching names is the
resulting list of
items.
===

So your Job B would just do a "queue matching"
against whatever Job A
produced, and would automatically queue the right
number of jobs based
entirely on the list of files Job A produced.

I love this new syntax, it makes so many things so
much simpler.

        -Michael Pelletier.
_