Re: [Condor-users] HoldReason = "Streaming not supported"

[Sean Manning <seangwm@xxxxxxx>]

Jaime Frey wrote:

>On Jun 23, 2008, at 6:50 PM, Sean Manning wrote:
>
>>  I am working on a Web Services interface to submit jobs to our Globus
>> grid.  It uses the condor and birdbath Java packages.  We can
>> successfully submit the attached JDL on the command line of a condor
>> head node (the metascheduler of our grid)  and see it complete, but
>> when we submit it with the Java program from an external Condor client
>> machine the job stays Idle then Halts with an error.  Running the
>> condor daemons as root got rid of one error, but now we get another
>> one: HoldReason = "Streaming not supported".  I can't find any
>> information about this error in the usergroup archives.  Does anyone
>> here have an idea what could be causing this?
>
>For GT4 GRAM jobs, if StreamOut and StreamErr aren't explicitly set to  
>False in the job ad, then Condor assumes you want stdout and stderr to  
>be streamed, which isn't supported by Condor for GT4 GRAM jobs. This  
>appears to be a bug, as the default behavior for other job types is no  
>streaming.
>
>If you add the following two attributes to your job ads, it should  
>eliminate the problem:
>StreamOut = False
>StreamErr = False
>
>Thanks and regards,
>Jaime Frey
>UW-Madison Condor Team
>

Dear Jaime,

  Thanks for the reply.  

  I made that change, but jobs are still hanging with HoldReason = 
"Streaming not supported."  I can submit the new file with 
condor_submit from the grid metascheduler and see it appear on the head 
node of a worker cluster, when condor_config has SOAP enabled.  The 
output and error come back to the machine I submitted the job from just 
like they are supposed to.  But when I  submit the same JDL to the grid 
metascheduler using our Web Services code, the job always holds after a 
delay.

  Right now, the Condor daemons are running as root.  The web services 
code is running on my personal account (seangwm) on my workstation.  
The spool directory on the metascheduler 
($CONDOR_LOCATION/local.babargt4/spool) belongs to condor:root..  We 
have been changing the owner of the job folder on the spool 
($CONDOR_LOCATION/local.babargt4/spool/cluster5252.proc0.subproc0) by 
hand from root:root to my personal account and group, because jobs stay 
idle until I do so.  I think that this has to do with the fact that the 
proxy file must have very specific permissions so the grid will trust 
it.  If I change the owner of the spool folder to root:root I get a 
HoldReason = "Failed to get expiration time of proxy" instead.

  In principle, if we can submit a job to the grid using condor_submit, 
then the web services submission should work as well.  I would be very 
grateful if you have any further advice about what I am missing.  

  I have attached our main Java class for job submittion and the JDL 
which I have been trying with the Web Services code.  In the attached 
files, babargt4 is the grid metascheduler and ugdev07 is the head of 
one of the clusters of worker nodes.

Regards,

Sean Manning

Attachment: testjdl-gt4
Description: Binary data

/*
 * CondorJobSubmitter.java
 *
 * Created on December 1, 2007, 5:38 PM
 *
 * To change this template, choose Tools | Template Manager
 * and open the template in the editor.
 */

package condorwsgui;

import org.globus.tools.proxy.GridProxyInit;
import org.globus.gsi.GlobusCredential;
import org.globus.gsi.GlobusCredentialException;
import org.globus.tools.proxy.ProxyListener;
import condor.*;
//import condor.CondorCollectorLocator;	// SM Not used
//import org.apache.axis.constants.Style;	// SM Not used
import birdbath.*;

import java.io.*;
import java.net.URL;
//import java.util.*;	// SM Not used
import java.util.regex.*;
import javax.swing.JOptionPane;

//import org.glite.jdl.JobAd;
//import org.glite.jdl.*;		// SM Not used
//import condor.classad.*;	// SM Not used
//import java.util.Iterator;	// SM Not used
import java.util.Vector;
import condor.ClassAdStructAttr;
//import condor.ClassAdAttrType;	// SM Not used
import java.nio.charset.Charset;

/**
 *
 * To implement proxyListener, a public void proxyCreated(GlobusCredential proxy) is required.
 *
 * @author Core by David Gong; updated and expanded by Sean Manning
 */
public class CondorJobSubmitter implements ProxyListener {
	/*
	 * The following variables are used chiefly for the reader's convenience, 
	 * to explicitly record implicit information such as whether the proxy certificate 
	 * already exists:
	 * isFromFile, isExistingProxy, jdlFile, jobUniverse, myGlobusCredential
	 * 
	 * They may be removed without effect, but can help in debugging.
	 * 
	 */
	
	/*
	 * These variables all represent fasts about the environment and about Condor
	 */	
    private boolean isFromFile;			// SM True iff the job will be submitted from a file???
    private boolean isSubmissionNeeded; // true if myProxy == null or wantNewProxy ().  Also called in proxyCreated ()
    private boolean isExistingProxy;	// True iff the proxy certificate already exists
    private Schedd schedd;				// Represents the Condor schedd daemon
    private String condorHome;			// The location on this machine which will be treated as the root for placing other files
    private String jdlFile;				// SM Path to the JDL description of the job to be submitted.
    private GlobusCredential myProxy;	// A proxy certificate
    private String userProxySubject;	// The subject DN of the proxy certificate.  See the half of each line which is enclosed in quotes in $CONDOR_LOCATION/etc/user_map

    /*
     * These variables all represent lines in the JDL file
     */
    private String owner;	// The user id of the job owner eg. "j_smith"
    private String user;
    private int uid;		// User ID, for Unix/Linux systems only.
    private String command; // The actual command or shell script which you will execute
    private String arguments;
    private String requirements;
    private String jobUniverse;
    private Vector <File> stageInFiles; // SM A vector is an array which re-sizes itself as needed.  In Java 5+ it can be parameterized (SM did so).
    private Vector <ClassAdStructAttr> jobAdAttrs;	// SM Added parameterization
    
    private String proxyFileLocation;	// The path to the proxy certificate on the system
    private GlobusCredential myGlobusCredential;	// SM What is this for?
    private OS myOS;	// Represents the family of operating system running on the system
    
    private static enum OS {LINUX, WINDOWS, OTHERS};
    
    /** Creates a new instance of CondorJobSubmitter */
    public CondorJobSubmitter () {
        myProxy = null;
        user = System.getProperty("user.name");
        owner = user;
        command = null;
        arguments = null;
        stageInFiles = new Vector <File> ();
        jobAdAttrs = new Vector <ClassAdStructAttr> ();
        proxyFileLocation = null;
//        myGlobusCredential = null;
        condorHome = "/usr/local/condor-6.8.5/local.babargt4"; // Special sub-folder of $CONDOR_LOCATION
        determinteOS();	// Prints out the OS name, inter alia
//        System.out.println ("User: " + user);
        uid = getUID ();
    }
    
    /**
     * Takes a string which contains a job description and tries to use it 
     * to submit a file.
     * 
	 * Creates a simple GUI to receive the password and other information 
	 * about the certificate from the user
	 * 
	 * The functions used are proxtInitGUI () and wantNewProxy ()
	 * @param jdlFile The job description file as a string
     */
    public void submitFromFile (String jdlFile) throws Exception {
    	/*
    	 * Define more of the variables of this object
    	 * since the job to submit is now known
    	 */
        this.jdlFile = jdlFile;	// The string is a path to the file which we wish to execute.
        isFromFile = true;        
        isSubmissionNeeded = false;
        JobHelper myJobHelper = null; // Part of birdbath but rewritten by DG; converts a JDL file into an object
        System.out.println("+++Submit job from JDL file:" + jdlFile); // Line 2 of output

        // Convert the JDL file at the given path into an object
        try { myJobHelper = new JobHelper(jdlFile);}	// SM Uses getJobAttrFromJDL (String file)
        catch (Exception err) {          
            System.out.println("+++Error on parsing jdl file");
            throw new Exception("+++Error on parsing jdl file");
        }
        
        // The A? B: C; format avoids calling a function on a null object.  DG chose it.
        
        // owner defaults to user
        owner = (myJobHelper.getOwner() == null)? user:myJobHelper.getOwner().stringValue();
        // jobUniverse defaults to "GLOBUS".  Was "VANILLA" in some code which DG had commented out.
        jobUniverse = (myJobHelper.getJobUniverse() == null)? "GLOBUS":myJobHelper.getJobUniverse().stringValue();
        
        command = (myJobHelper.getCommand()==null)? null:myJobHelper.getCommand().stringValue();
        arguments = (myJobHelper.getArguments()==null)? null:myJobHelper.getArguments().stringValue();
        requirements = (myJobHelper.getRequirements()== null)? null: myJobHelper.getRequirements().stringValue();
        jobAdAttrs = myJobHelper.getJobAttrVector();
        
        // Organize a list of files to stage in
        String[] stageInFilesString = myJobHelper.getStageInFiles();
        try {
            for (int i=0; i<stageInFilesString.length; i++) {
                stageInFiles.add(new File(stageInFilesString[i]));
            }
        } 
        catch (Exception err) {
            System.out.println("+++Can not open file for stageIn");
            throw new Exception("+++Error on opening stageIn files");
        }
        
        // SM myProxy is a GlobusCredential from org.globus.gsi
        // SM Note that the second part of an AND is not executed 
        // if the first part is false
        // SM What purpose does the outer if/else serve?  
        // Both branches have essentially the same effect.
        if( (myProxy != null) && (!wantNewProxy())){
        	// Is there a certificate already, and does the user want to use it?
            if (getExistingProxy() && (!wantNewProxy())) {
                isExistingProxy = true;
                System.out.println ("+++Using existing proxy.1");
                submit ();
            }
            else {
                proxyInitGUI();
                System.out.println("+++Using newly created proxy.1");
                isExistingProxy = false;
                isSubmissionNeeded = true;
            }
        } 
        else {	// SM Execution flows here, even if a proxy already exists
        	// Is there a certificate already, and does the user want to use it?
        	if (getExistingProxy() && (!wantNewProxy())) {
                System.out.println("+++Using existing proxy.2");
                isExistingProxy = true;
                isSubmissionNeeded = true;
                submit ();
            }
            else { // SM Execution flows here, even if a proxy already exists
                proxyInitGUI();
                System.out.println("+++Using newly created proxy.2");
                isExistingProxy = false;
                isSubmissionNeeded = true;
            }
        }           
    }

    /**
     * Creates a GUI to ask the user whether he or she wishes to create a new proxy.
     * 
     * @return true if the user selects "Yes"; false if the user selects "No"
     */
    private boolean wantNewProxy() {
        String message = "The following proxy exists:\n" + myProxy.toString() + "\n\nDo you want to create a new proxy?\n";
        int response = JOptionPane.showConfirmDialog(null, message, "Proxy Creation Option", JOptionPane.OK_CANCEL_OPTION);
//        System.out.println("response is:" + response);
        return (response == 0);
    }
    
    /**
     * Asks the question "Does a valid proxy certificate for the current user exist?"
     * 
     * @return true iff a valid proxy certificate for the current user exists, false otherwise.  
	 * 
	 * Outputs some text to the console if true.
     */
	public boolean getExistingProxy() {
        String username = System.getProperty("user.name");
        /*
         * Get the path to such a hypothetical proxy certificate
         */
        // SM Why the final underscore for linux path?
        switch(myOS) {
            case WINDOWS:
                  proxyFileLocation = "C:\\Users\\" + username + "\\AppData\\Local\\Temp\\x509up_u_" + owner;
                  break;
            case LINUX:
                  proxyFileLocation = "/tmp/x509up_u" + uid/*"/tmp/x509up_u_" + owner*/;
                  break;
            default:
            	// Take linux as default
            	proxyFileLocation = "/tmp/x509up_u" + uid/*"/tmp/x509up_u_" + owner*/;
        }

        try {
            myProxy = new GlobusCredential (proxyFileLocation);
        } catch (GlobusCredentialException err) {
            return false;
        }
        if (this.myProxy != null){
            if(this.myProxy.getTimeLeft() > 0){
                System.out.println("+++*****Proxy Info*****");
                System.out.println("+++Subject\t:" + myProxy.getSubject());
                System.out.println("+++Issuer \t:" + myProxy.getIssuer());
                System.out.println("+++Strength\t:" + myProxy.getStrength() + "bits");
                System.out.println("+++Timeleft\t:" + myProxy.getTimeLeft() + " sec");
                System.out.println("+++Proxy type\t:" + myProxy.getProxyType());
                System.out.println("+++*****End of Proxy Info*****");
//				System.out.println(myProxy.toString());
//				this.myProxy = myProxy;	// SM No effect
                this.userProxySubject = myProxy.getSubject();
                return true;
            }
        }

        return false;
	}
    
	/**
	 * Creates a GUI to request the user to enter his password.
	 */
	public void proxyInitGUI() {
		GridProxyInit myProxyInit = new GridProxyInit();
        myProxyInit.setSize(500, 500);
        myProxyInit.setTitle("Creating proxy...");
        myProxyInit.setResizable(true);
        myProxyInit.setRunAsApplication(true);
        myProxyInit.setCloseOnSuccess(true);
        myProxyInit.addProxyListener(this);
        myProxyInit.setVisible(true);
	}
 
	/**
	 * This function is called when a new proxy certificate has been created.  
	 * It assigns that certificate to this object, updates userProxySubject, 
	 * and (if necessary) tries to submit the job to the grid.
	 * 
	 * Right now, it is only called by other classes (probably one of the ones 
	 * that handles the "Create New Proxy" window).
	 * @param proxy The new proxy certificate
	 */
	public void proxyCreated(GlobusCredential proxy)
    {
		System.out.println("+++ In proxyCreated ()");
		myProxy = proxy;
        userProxySubject = proxy.getSubject();
        
        if (isSubmissionNeeded) {
        	try {submit ();}
        	catch(Exception err){
        		System.out.println ("Error caught in proxyCreated: " + err); // SM Error caught here
        		err.printStackTrace();
        	}
        }     
    }
     
    
    /**
     * Detetermines which OS the system uses, assigning the correct value to myOS.
     */
    private void determinteOS(){
        String myOSName = System.getProperty("os.name");
        Pattern p1 = Pattern.compile("WIN"); // Compiles regular expression into Pattern.
        Pattern p2 = Pattern.compile("Lin");
        Matcher m1 = p1.matcher(myOSName);
        Matcher m2 = p2.matcher(myOSName);
        if (m1.find()){
            myOS = OS.WINDOWS;
            System.out.println("CondorJobSubmitter.determinteos (): Windows");
        }
        else if (m2.find()){
            myOS = OS.LINUX;
            System.out.println("CondorJobSubmitter.determinteos (): Linux");
        }
        else
            myOS = OS.OTHERS;
    }
    
    /**
     * Adds the proxy certificate to the jobs to be staged over from this machine 
     * to the Condor server.
     * @param cluster The cluster which the proxy will be associated with
     * @param job The job ID which the proxy will be associated with
     * @throws Exception
     */
    private void addProxy(int cluster, int job) throws Exception{
        try {
            stageInFiles.add(new File("/tmp/x509up_u" + uid)); // Was "/tmp/x509up_u_daobgong"
            ClassAdStructAttr subjectAttr = JobHelper.createStringAttr("X509UserProxySubject", userProxySubject);
            createProxyFileLocation(cluster, job);
            ClassAdStructAttr locationAttr = JobHelper.createStringAttr("X509UserProxy", proxyFileLocation);
            jobAdAttrs.add(subjectAttr);
            jobAdAttrs.add(locationAttr);
        }
        catch (Exception err) {
            throw new Exception("Error on creating proxy related attributes");
        }
    }
    
    /**
     * Sets proxyFileLocation to the job's folder in the spool folder.  
     * According to DG's workterm report p. 28, this is necessary because the SOAP 
     * interface can only access this folder and not /tmp where the proxy 
     * certificates are usually stored.
     * 
     * @param cluster The batch of jobs which this job is part of
     * @param job The number of the job within that cluster.  This starts at 0 and increments by one.
     */
    private void createProxyFileLocation(int cluster, int job){
    	// SM Was condorHome + "/spool/cluster" + cluster +".proc" + job + ".subproc0/x509up_u_" + owner.substring(0, owner.length()) 
    	proxyFileLocation = (condorHome + "/spool/cluster" + cluster +".proc" + job + ".subproc0/x509up_u" + uid);
        return;
    }

    /**
     * Submits a job to the grid once the schedd and this instance have been initialized.  
     * Specifically, it submits a job ClassAd to the application's condor_schedd.
	 * 
	 * See "Developer APIs to Condor + Condor Web Services" powerpoint slide 27, 28
     */
    public void submit () throws Exception {
        
    	/*
    	 * A transaction is a discrete unit of work.  It has a limited lifespan.
    	 */
        birdbath.Transaction xact = schedd.createTransaction();
        try {xact.begin(3000);} // Argument is max duration in seconds
        catch (java.rmi.RemoteException err)
        {
        	System.out.println("+++Failed to start transaction");
        	err.printStackTrace ();
        	throw (new Exception ("Failed to start transaction"));
        }
        
        // Define some final parameters of the job
        int cluster = xact.createCluster();
        int job = xact.createJob(cluster);       
        addProxy(cluster, job);
        ClassAdStructAttr[] jobExtraAttrArray = (ClassAdStructAttr[])jobAdAttrs.toArray(new ClassAdStructAttr[0]);
        File[] files =  (File[])stageInFiles.toArray(new File[0]);
        
        // Submits the job ClassAd (as a collection of parameters) to the Schedd
        /*Was UniverseType.VANILLA*/
        xact.submit(cluster, job, owner, UniverseType.GLOBUS, command, arguments, requirements, jobExtraAttrArray, files );
        xact.commit();	// Commits the transaction to make it take effect
        System.out.println("+++Job submitted successfully. ClusterID=" + cluster + "  jobId=" + job);
    }
    
    /**
     * Assigns a new Schedd to this object
     * @param mySchedd The Schedd to be assigned to this object
     */
    public void setSchedd(Schedd mySchedd){
        schedd = mySchedd;
    }

    /**
     * @return The Schedd associated with this object
     */
    public Schedd getSchedd(){
        return schedd;
    }
    
    /**
     * This requests a condor_reschedule from the condor_schedd daemon.  
     * 
     * condor_reschedule updates the information about a given machine's resources 
     * and jobs to the central manager. This can be used if one wants to see the 
     * current status of a machine. In order to do this, one would first run 
     * condor_reschedule, and then use the condor_status command to get specific 
     * information about that machine. condor_reschedule also starts a new 
     * negotiation cycle between resource owners and resource providers on the 
     * central managers, so that jobs can be matched with machines right away. 
     * This can be useful in situations where the time between negotiation cycles 
     * is somewhat long, and an administrator wants to see if a job they have 
     * in the queue will get matched without waiting for the next negotiation cycle.
     * 
     * (The above was borrowed from Sections 5 and 4 of the online manual 
     * version 6.0.3 at http://www.cs.wisc.edu/condor/manual/v6.0/ref.html
     */
    public void requestReschedule() throws Exception {
        schedd.requestReschedule();
    }
    
    /**
     * TODO Implement in a generic way without mangling the original file.
     *
     */
    public void parseJdlFile (String jdlFile) {
    	// Turn A = B into A = "B";
    	
    	// Rename variable Error err, and variable Output out
    	
    	//Turn queue A into queue = "A";
    	
    	// Add an InputSandbox
    }
    /**
     * Gets the Unix user ID of the current user.  This determines where a 
     * pre-existing proxy certificate is stored on a Unix system.  It will fail 
     * if myOS has not yet been set, or if the operating system is not Linux.
     * 
     * @return The user ID (on a Unix system) or -1 (on any other OS, 
     * or in case of an error on a Unix system)
     */
    private int getUID () {
    	if (myOS == OS.LINUX) {
    		/* 
    		 * We break the command up into several short strings to ensure 
    		 * that is is parsed correctly.  It is equivalent to typing 
    		 * "/bin/bash -c 'id'" on the command line and means 
    		 * "submit the string 'id' to be processed as a bash shell command".  
    		 * 
    		 * See http://java.sun.com/developer/JDCTechTips/2003/tt0304.html
    	     */
    		String[] cmd = { "/bin/bash", "-c", "id" };
    	    try {
    	    	Process p = Runtime.getRuntime().exec(cmd);
    	    	InputStream processIn = p.getInputStream ();
    	    	InputStreamReader reader = new InputStreamReader (processIn, "US-ASCII");
    	    	int uid = 0;
    	    	char c;
    	    	boolean equalsFound = false;	

    	    	// Parse the input looking for a uid.  Expected format is:
    	    	// uid=58858(seangwm) gid=34244(hep) groups=34244(hep)
    	    	do {
    	    		c = (char) reader.read ();
    	    		if (c == '=') {
    	    			equalsFound = true;
    	    		}
    	    		else if (equalsFound) {
    	    			// If the current character is a digit, multiply the old uid by 10 
    	    	    	// and add the current digit eg. 5 -> 50 + 8 = 58 -> 580 + 8 = 588
    	    			if (Character.isDigit(c)) {
    	    				uid *= 10;
    	    				uid += Character.getNumericValue (c); 
    	    			}
    	    		}
    	    	} while (c != '(');
    	    	
    	    	System.out.println ("uid = " + uid);
    	    	return uid;
    	    }
    	    catch (IOException err) {
    	    	System.out.println ("Failed to get uid");
    	    	return -1;
    	   	}
    	}
    	else {
    		return -1;
    	}
    }
    
    /**
     * 
     * Performs several simple tests of this class's functionality.
     * 
     * @param args A list of command-line arguments.  This method expects 
     * to take a single one, which defines which sample job to submit.  
     * 
     */
    /*
     * Output (6 lines):
     * CondorJobSubmitter.determinteos (): Linux
	 * +++Submit job from JDL file:/hepuser/seangwm/ashokProjects/CondorWebService/CondorWSProjectRon/src/supportfiles/test.jdl
	 * +++Using newly created proxy.2
	 * +++RequestReschedule Failed
	 * +++Failed to start transaction
	 * java.lang.Exception: Failed to start transation
     */
    public static void main(String[] args){
    	/* 
    	 * Set system properties (cp. lide 38 of "Developer APIs for Condor" PPT)
    	 */    	

    	//System.out.println ("In CondorJobSubmitter.Main");	// SM
    	// For Windows
//       java.lang.System.setProperty("javax.net.ssl.trustStore", "c:\\Program Files\\Java\\jre1.6.0_03\\bin\\truststore");
    	// For Unix
    	java.lang.System.setProperty("javax.net.ssl.trustStore", "/hepuser/seangwm/ashokProjects/CondorWebService/CondorWSProjectRon/src/supportfiles/truststore");
    	// Was 
    	
    	// For Windows
//       java.lang.System.setProperty("javax.net.ssl.keyStore", "c:\\Users\\Daobgong\\JavaProject\\CondorWSGUI\\keystore");
//       java.lang.System.setProperty("javax.net.ssl.keyStore", "c:\\Users\\Daobgong\\JavaProject\\CondorWSGUI\\DavidGridKeyStore");
    	// For Unix
    	java.lang.System.setProperty("javax.net.ssl.keyStore", "/hepuser/seangwm/ashokProjects/CondorWebService/CondorWSProjectRon/src/supportfiles/keystore");
    	// Was "/hepuser/seangwm/ashokProjects/CondorWebService/CondorWSProjectRon/src/supportFiles/DavidGridKeyStore"
    	
    	java.lang.System.setProperty("javax.net.ssl.keyStoreType", "PKCS12");
    	java.lang.System.setProperty("javax.net.ssl.keyStorePassword", "An5sh6An3-");

    	/*
    	 * Create a CondorJobSubmitter to submit the job
    	 */
    	//System.out.println ("Creating CondorJobSubmitter");
    	CondorJobSubmitter mySubmitter = new CondorJobSubmitter();	// SM Line 1 of output

    	/*
    	 * Create an instance of birdbath.Schedd located at a particular port, 
    	 * and make it the schedd of this CondorJobSubmitter.
    	 */
       Schedd mySchedd = null;
       //System.out.println ("Creating Schedd");
       try { // Listen on a particular port
 //           String tmpStr="https://ugdev01.phys.uvic.ca:1980";;
    	   	String tmpStr="https://babargt4.phys.uvic.ca:1980";;
            URL scheddLocation = new URL(tmpStr);
            mySchedd = new Schedd(scheddLocation);
       }
       catch (Exception err) {
            System.out.println("+++Failed to create scheduler, System is exiting.");
            System.exit(-1);
       }
       mySubmitter.setSchedd(mySchedd);
       
       /* "/bin/bash", "-c", "echo $PPID" means run the following string (-c) 
        * under bash shell (/bin/bash).  It 
        * http://java.sun.com/developer/JDCTechTips/2003/tt0304.html
        */
       
       /*
        * Submit a test job to the grid.  Creates a simple graphical user interface to do so.
        */
       /*  following line is for linux. For windows change it */
       // SM This creates lines 2-3 of output
       // SM Was "/hepuser/daobgong/ashokProject/CondorWebService/supportFiles/test.jdl"
       try {
    	   System.out.println();
    	   String fileToSubmit = null;
    	   
    	   if (args[0].contains ("4")) {		// testjdl-gt4
    		   fileToSubmit = "/hepuser/seangwm/ashokProjects/CondorWebService/CondorWSProjectRon/src/supportfiles/testjdl-gt4";
    	   }
    	   else if (args[0].contains ("2")) {	// testjdl-gt2
    		   fileToSubmit = "/hepuser/seangwm/ashokProjects/CondorWebService/CondorWSProjectRon/src/supportfiles/testjdl-gt2";
    	   }
    	   else if (args[0].contains ("DG")) {	// DG's test.jdl
    		   fileToSubmit = "/hepuser/seangwm/ashokProjects/CondorWebService/CondorWSProjectRon/src/supportfiles/test.jdl";
    	   }
    	   else {								// Default
    		   fileToSubmit = "/hepuser/seangwm/ashokProjects/CondorWebService/CondorWSProjectRon/src/supportfiles/test.jdl";
    	   }

    	   mySubmitter.submitFromFile(fileToSubmit);
       }
       catch (Exception err) {
            System.out.println("+++Error on submit");
            err.printStackTrace();	// SM At JobHelper.java line 166 <= CondorJobSubmitter.java line 143 <= CondorJobSubmitter.java line 455
            System.exit(0);
       }
        
       try {
    	   mySubmitter.requestReschedule();
    	   System.out.println("+++RequestReschedule succeeded");
       }	// SM just does Schedd schedd.requestReschedule ()
       catch (Exception err) {
    	   //System.out.println (" " + err);
//        	err.printStackTrace();
    	   System.out.println("+++RequestReschedule failed");
       }
       
//       System.out.println ("+++Exiting job submission program");
    }
}