Re: [HTCondor-users] A couple DAGman questions

[Michael Fienen <mike@xxxxxxxxxxx>]

Great idea, Michael!

Thanks - that might work exactly how I need!

Cheers,

mike

On Jul 26, 2016, at 5:41 PM, Michael V Pelletier <Michael.V.Pelletier@xxxxxxxxxxxx> wrote:

From: Michael Fienen <mike@xxxxxxxxxxx>
Date: 07/26/2016 05:05 PM

> My issue is, the results of Job A will influence what queue I want for JOB
> B. Your idea of using POST from A to write a file the PRE for B will
> generally work but can I then rewrite the sub file for B with the PRE
> SCRIPT such that I update the queue for B.
>
> Specifically, in this case, Job A is splitting a large file into chunks to
> be run in parallel by job B. Butâthe code in Job A breaks into a difficult
> to predict number of such chunks.
>
> Thanks, and I hope that makes sense.
>
> Cheers,
> mike

Sounds like you should check out the utterly sweet new "queue" options,
"queue from" "queue in" and "queue matching" introduced in the 8.4
release.

In this case, you'd want the "queue matching" statement:

===
matching - Each item in the <list of items with file globbing> will be
matched against the names of files and directories relative to the
current directory; the set of matching names is the resulting list of
items.
===

So your Job B would just do a "queue matching" against whatever Job A
produced, and would automatically queue the right number of jobs based
entirely on the list of files Job A produced.

I love this new syntax, it makes so many things so much simpler.

        -Michael Pelletier.
_
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/