[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[HTCondor-users] condor_submit_dag independent child processes
- Date: Wed, 20 Sep 2017 10:50:35 +0200
- From: Labounek René <xlabou01@xxxxxxxxxxxxxxxxxx>
- Subject: [HTCondor-users] condor_submit_dag independent child processes
Dear HT-condor Users,
I am trying to optimally paralelize our data processing pipeline via
condor and I have found one issue which I am not able to solve...
In my file.dag, I have 6 different processes and following PARRENT and
JOB mprage_bet /md1/NH-DHP/NH-DHP-002/condor_logs/NH-DHP-002-mprage_bet.condor
JOB mprage_seg /md1/NH-DHP/NH-DHP-002/condor_logs/NH-DHP-002-mprage_seg.condor
JOB mprage2mni /md1/NH-DHP/NH-DHP-002/condor_logs/NH-DHP-002-mprage2mni.condor
PARENT data_convert CHILD dmri_preproc mprage_bet
PARENT mprage_bet CHILD mprage_seg mprage2mni
PARENT dmri_preproc mprage_seg CHILD mprage2diff
After data_convert, both CHILD precosses (dmri_preproc and mprage_bet)
start. Process dmri_preproc takes much more time than other processes.
Process mprage_bet is finished much earlier. When mprage_bet is
finnished, the process mprage2mni is started first and mprage_seg is
started after mprage2nii is finished and not imediatelly when
mprage_bet is finished.
and i have enough free threads in my pool to be able to compute all
three processes at once.
Does somebody have an idea why it behaves in that way. mprage_seg and
mprage2mni should be independent on themselves.