[HTCondor-users] Non trivial way of using DAG

[Lorenzo Mobilia <l.mobilia@xxxxxxxxxxxxxxxx>]

Hi,Â

I am finding some difficulties in using DAG. Basically I need toÂ

1. Take a dataset D1

2. Split it in N subdataset

3. Perform some computation in these N subdataset

4. Merge these subdataset in another dataset D2

5. Restart the process (back to point 1) now using D2

And continuing until specific characteristics have been achieved by the final dataset. The problems are:

A. I don't know a priori how many times I need to split D1

B. I don't know a priori how many times I need to perform this cycle

The solution I came up with is to build a main which controls this flow, but afterÂsome cycles it crashes.Â

If anyone has some suggestions or is interested in this problem in order to have some other information,Âplease let me know!

Hi,Â

Lorenzo