Re: [Condor-users] Transfering directories structure using the Condor file transfer mechanisem

[Ajasja Ljubetič <ajasja.ljubetic@xxxxxxxxx>]

Thank you for the quick reply. 

But if I try to set transfer_input_files to the name of directory I get: 

ERROR: Can't open "C:\ctest\hello_world_transfer_files_dir\in"  with flags 00 (Permission denied)

(I'm have read/write permissions for the directory)

running on:

$CondorVersion: 7.5.5 Feb  3 2011 BuildID: 311217 $

$CondorPlatform: X86-WINDOWS_x86_winnt_5.1-tst $

Any suggestions?

Best regards,

Ajasja

On Fri, Mar 18, 2011 at 12:37, Mark Calleja <mc321@xxxxxxxxx> wrote:

Hi Ajasja,

The current stable version (7.4.x) only supports the transfer of files from a scratch directory, so sub-directories need to be zipped up in order to be returned. However, the automatic transfer of directories has been added to the development version of Condor as of version 7.5.4. From the release notes of 7.5.4:

"Transfer of directories is now supported by transfer_input_files and transfer_output_files for non-grid universes and Condor-C. The auto-selection of output files, however, remains the same: new directories in the job's output sandbox are not automatically selected as outputs to be transferred. "

Best regards,
Mark

On 18/03/2011 11:28, Ajasja Ljubetič wrote:

Dear Condor users,

When using the Condor file transfer mechanism, is there perhaps a way to transfer the directory structure (and the files as well, of course)?

Something akin perhaps to "--relative" switch of rsync?

I am planning to use Condor to run multiple NAMD MD simulations. There are a lot of input files for the simulations, which I have structured in many directories. (an example layout is attached to the end of the message).

Since Condor transfers all the files to the top execute directory I have to rename and edit a lot of scripts (and change all the analysis scripts as well..). 

Perhaps the easiest way is to just zip the whole directory structure, then specify the zip file as the input and write a wrapper that unzips the file with the directory structure intact, runs the simulation and at the end zips the right output directories.

But if I'm missing an (undocumented) switch somewhere, or if there is a better way to do this, then I'd really like to know.

Best regards,

Ajasja Ljubetič,

Young researcher,

Laboratory of biophysics,

Institute Jozef Stefan,

Ljubljana, Slovenia  

/My_sim (directory)
      submit.condor (the submit description file)
      namd (the executable)
      FF (directory)
          param (file)
          rtf (file) 

log (directory)

* log files go here

out (dir)

min (dir)

* output files of minimization go here 

(these are also input files for equilibration)

equ (dir)

* output files of equilibration go here

(these are also input files for the run)

run (dir)

* output files of minimization go here

namd.conf (file)

      (various other conf files)

_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/condor-users/

-- 
The Cavendish Laboratory, University of Cambridge,
J J Thomson Avenue, Cambridge, CB3 0HE, UK
Tel. (+44/0) 1223 746627
http://www.escience.cam.ac.uk/~mcal00