[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Problems Submitting Multiple Jobs



John,
     Both machines are identical.  Even the directory structures and installed software are the same.  They are both windows XP pro with the same version of the gdal software.  

Awe

-----Original Message-----
From: "Kewley, J (John)" <j.kewley@xxxxxxxx>

Date: Sat, 16 Feb 2008 01:32:51 
To:"Condor-Users Mail List" <condor-users@xxxxxxxxxxx>
Subject: Re: [Condor-users] Problems Submitting Multiple Jobs


Is submit machine and execute machine the same architecture and OS?
If not, they won't match.

Can the execute machine see the c: directories? Are they cross-mounted or on
NFS?

JK


-----Original Message-----
From: condor-users-bounces@xxxxxxxxxxx on behalf of estrada.adam@xxxxxxxxx
Sent: Fri 15/02/2008 21:23
To: condor-users@xxxxxxxxxxx
Subject: Re: [Condor-users] Problems Submitting Multiple Jobs
 
A bit more information:

I realized that the executable I am running is linked to several otehr
dll's which means that I would have to move those files with the one I
am processing.  So...I found in the documentation that you can run a
script that first sets the environment variables and then runs the app
in question.  This script is still getting placed in to idel mode in
Condor.  My description file now looks like this.

#
# TEST CONDOR SCRIPT
#

Universe   = vanilla
Initialdir = C:\WorkSpace
Executable = C:\WorkSpace\batch.bat
getenv     = true
Log        = simple.log
Output     = simple.out
Error      = simple.error
Queue

And the script that I am planning to run resembles the following.

<!---Start Script-->

SET FWTOOLS_DIR=C:\Program Files\FWTools2.0.4

REM ===========================================================
REM This script is normally invoked from FWTools\setfw.bat

REM The FWTOOLS_OVERRIDE environment variable allows you to set a the
REM install directory in your environment, instead of having to edit
REM setfw.bat.  This is especially useful if you frequently upgrade to
new
REM versions, and don't want to have to edit the file every time.

IF exist "%FWTOOLS_OVERRIDE%\setfw.bat" SET FWTOOLS_DIR=
%FWTOOLS_OVERRIDE%

IF exist "%FWTOOLS_DIR%\setfw.bat" goto skip_err

echo FWTOOLS_DIR not set properly in setfw.bat, please fix and rerun.
goto ALL_DONE

:SKIP_ERR

PATH=%FWTOOLS_DIR%\bin;%FWTOOLS_DIR%\python;%PATH%
set PYTHONPATH=%FWTOOLS_DIR%\pymod
set PROJ_LIB=%FWTOOLS_DIR%\proj_lib
set GEOTIFF_CSV=%FWTOOLS_DIR%\data
set GDAL_DATA=%FWTOOLS_DIR%\data
set GDAL_DRIVER_PATH=%FWTOOLS_DIR%\gdal_plugins
REM set CPL_DEBUG=ON

:ALL_DONE

REM =========================================================
REM start.vbs
REM *** START PROCESSING THE MOSAIC SERIES ***

"C:\Program Files\FWTools2.0.4\bin\gdal_translate.exe" -of GTiff -co
"TFW=YES" -a_nodata 0 -a_srs "EPSG:102645" \\geo127\RAW\36119h7se.jpg \
\geo127\TIFF\36119h7se.tif
"C:\Program Files\FWTools2.0.4\bin\gdalwarp.exe" -multi -wm 500000 -
srcnodata 0 -dstnodata 0 -of GTiff -co "TFW=YES" -s_srs "EPSG:102645" -
t_srs "EPSG:26911" -wo "SAMPLE_GRID=YES" -wo "SKIP_NOSOURCE=YES" -wo
"INIT_DEST=NO_DATA" -wo "WRITE_FLUSH=YES" \\geo127\Geofields\TIFF
\36119h7se.tif \\geo127\UTM\36119h7se_utm.tif
del \\geo127\Geofields\TIFF\36119h7se.tif
"C:\Program Files\Leica Geosystems\Geospatial Imaging 9.1\bin
\ntx86\mrsidcompress.exe" -erdas -i \\geo127\Geofields\UTM
\36119h7se_utm.tif -o \\geo127\MrSID\36119h7se.sid -truecolor 1 2 3 -c
18 -g 2 -w 4 -n 3 -b 512 -orient -gen 2 -lossless
"C:\Program Files\FWTools2.0.4\bin\mrsidsdw.exe" \\geo127\MrSID
\36119h7se.sid
del \\geo127\UTM\36119h7se_utm.tif
dir /b /s \\geo127\MrSID\36119h7se.sid > \\geo127\ProcessedFiles.txt

<!--End Script-->

So you can see that just like the documentation states, I am calling a
script that is calling the other applications that are doing the
processing.  Am I going about this correctly?

AWE

On Feb 15, 3:47 pm, estrada.a...@xxxxxxxxx wrote:
> All,
>    I am able to submit this job to my queue with what I think is the
> correct way to submit the arguments for the application.  The
> executable is passed in to the queue but is immediately placed in idle
> mode.  Why is this not running correctly?  Does anyone have any
> feedback for me here?  I have thousands of processes just like this
> that take an hour or so at a time to process so I would think that
> distributing this over a cluster or processors would greatly help to
> speed up processing times.
>
> #
> # TEST CONDOR SCRIPT
> #
>
> Universe   = vanilla
> Executable = C:\Program Files\FWTools2.0.4\bin\gdal_translate.exe
> Arguments  = "-of GTiff -co 'TFW=YES'"\
>             "-a_nodata 0 -a_srs 'EPSG:102645'"\
>             "C:\WorkSpace\36119h7se.jpg C:\WorkSpace\36119h7se.tif"
> Log        = simple.log
> Output     = simple.out
> Error      = simple.error
> Queue
>
> Thanks,
> AWE
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-requ...@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visitinghttps://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
> The archives can be found at:https://lists.cs.wisc.edu/archive/condor-users/

_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users

The archives can be found at: 
https://lists.cs.wisc.edu/archive/condor-users/



_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users

The archives can be found at: 
https://lists.cs.wisc.edu/archive/condor-users/