Re: [Condor-users] retrieving output subdirectories

[Javier Forment <jforment@xxxxxxxxxxxx>]

Excuse my ignorance, but... I can't see how to do that: if I run my
application with condor_submit, Condor run it, and, when finished, it
moves back all the output files in the plain execute directory (ignoring
subdirectories), and remove all the files in the execute directory
(including subdirectories). If I put the tarring of subdirectory output
files just after condor_submit in a shell script, it will not find them,
will it?

How must I tar all the output files (including the subdirectories) just
after process completion, but just before moving the output files and
removing the execute directory?

Thanks again,

Javier.



On Mon, 2008-02-04 at 15:46 +0000, Smith, Ian wrote:
> I've used something similar under Condor-G which doesn't( didn't ?)
> have
> 
> this functionality. I create a time stamp file before the start of
> 
> application then after running the app use
> 
>  
> 
> $ find -newer time_stamp_file_name 
> 
>  
> 
> to work out which files have been created/modified and transfer them
> back. If
> 
> you use DAGman then a POST script could automatically unzip/untar the
> output on the
> 
> submit host to make things easier.
> 
>  
> 
> -ian.
> 
>  
> 
>  
> 
>  
> 
> From:condor-users-bounces@xxxxxxxxxxx
> [mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of carl langlois
> Sent: 04 February 2008 14:57
> To: Condor-Users Mail List
> Subject: Re: [Condor-users] retrieving output subdirectories
> 
> 
>  
> 
> Hi Javier,
> 
>   I have the same problem as you, what i'm planning to do is wrap the
> application in a .bat file or .sh if on unix and after the jobs is
> complete juste zip everything and tell condor to retrived the zip
> file. 
> 
> 
> 
> On Feb 4, 2008 5:37 AM, Javier Forment <jforment@xxxxxxxxxxxx> wrote:
> 
> Hi, all... I'm trying to run an application in Condor which creates an
> output subdirectory and write output files on it. I know that Condor
> has
> the feature of retrieving (and sending back to submit node) only the
> output files created/modified in the plain execute directory at the
> corresponding execute node.
> 
> Is there any way to cope with this? I've browsed the archives at the
> mailing list, but I only find messages telling that it is a well-known
> issue, and the solution is to modify the application or writing a
> custom
> wrapper for condor to tar (and maybe gzip) the output
> subdirectory/ies.
> 
> Since I cannot modify the application, where can I find documentation
> to
> write such a custrom wrapper for condor?
> 
> Thanks a lot,
> 
> Javier.
> 
> 
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx
> with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/
> 
> 
>  
> 
> 
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 
> The archives can be found at: 
> https://lists.cs.wisc.edu/archive/condor-users/