Re: [HTCondor-users] PyDagman - a convenience library for writing DAGs

[Keith Brown <keith6014@xxxxxxxxx>]

i suggest ALL jobs be DAGs. Even jobs that don't have relationships. There are some advantages to this. 1) Control number of idle/running processes without quotas/concurrency limits. 2) Built in PARENT CHILD relationship (obviously). Â3) Much easier monitoring when submitting with DEBUG 7. The entire process status is much easier to see with `tail -f`

I have built something like this. I used YAML as my configuration method. let me know if anyone is interested.Â

On Thu, Aug 28, 2014 at 4:54 AM, Brian Candler <b.candler@xxxxxxxxx> wrote:

ÂI'm writing to announce a Python package I've written to simplify the programmatic creation of DAG files in Python. The package is called PyDagman, and is available on github at https://github.com/brandentimm/pydagman

This is neat (and simple), thank you.

I also wrote a library with a similar aim:
https://github.com/candlerb/htcondor_dag.py

However it's a lot more complex, because what I wanted was for each DAG node to be a python function invocation. So it pickles the arguments to each deferred function call, stores them in an input file, and each node job calls the appropriate function with the appropriate args.

The code is not as clean as it could be - especially if you want to write out multiple DAGs.

Regards,

Brian.

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/