[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [HTCondor-users] purpose of pegasus
- Date: Sun, 27 Oct 2013 10:49:43 -0700
- From: Mats Rynge <rynge@xxxxxxx>
- Subject: Re: [HTCondor-users] purpose of pegasus
On 10/27/2013 08:48 AM, Keith Brown wrote:
> I am trying to understand what is the purpose of pegasus. It seems its a
> frontend for DAGMan, is that a correct observation?
Pegasus is using DAGMan and the rest of the HTCondor stack to execute
workflows. The value added is at a higher level. Some of the features
Pegasus provides on top of DAGMan:
Pegasus workflows are described in an abstract format, which means that
the description does not include data and software locations. These are
looked up at planning time, and this enables portability of the
workflow. You can move the same workflow around to resources such as
campus grids, supercomputers, or clouds.
Pegasus tracks all the data in the workflow, and will add data
management tasks, such as data staging and cleanup, to the executable
workflow (i.e. the DAG).
At planning time, Pegasus can apply graph transforms to the workflow,
such as cluster short tasks into a larger jobs.
Pegasus collects extra provenance data during the execution. This allows
the user to get reports at the end of the run (pegasus-statistics) and
debugging help if the workflow fails (pegasus-analyzer).
Some more information:
Please let me know if you would like to try out Pegasus. We can help
getting you requirements / current setup translated into a Pegasus workflow.
USC/ISI - Pegasus Team <http://pegasus.isi.edu>