Re: [Condor-users] Problems using Condor Web Services Client

["Natarajan, Senthil" <senthil@xxxxxxxx>]

Hi,
Are you using gridport Helper class, and condor 6.8.0. I am also having
the same problem, looks like something needs to be changed in the Helper
class to support condor 6.8.0. But it is working fine for condor 6.7.19
and Matt aware of this.

Regards,
Senthil

-----Original Message-----
From: condor-users-bounces@xxxxxxxxxxx
[mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of Nathan Harmston
Sent: Friday, August 25, 2006 11:08 AM
To: Condor-Users Mail List
Subject: Re: [Condor-users] Problems using Condor Web Services Client

Hi,

I looked through my code and found that there wasnt anything to actually
send files to the server (whoops!-weird that it wasnt in the IBM
download though). 

So I ve added the sendFileHelper{} to the WebServicesHelper so now I get
a different error:

query collector for the schedd address, please be patient ...
http://128.243.186.210:35501
Submitting job: ClusteId=31   JobId=0
/home/nharmston/test.sh
15
test.sh
Error Declaring File Invalid transaction
Exception in thread "main" SendFileException: Error Declaring File
        at WebServicesHelper.sendFileHelper(WebServicesHelper.java:279)
        at WebServicesHelper.submitJobHelper(WebServicesHelper.java:118)
        at WebServicesHelper.submitJobHelper(WebServicesHelper.java:204)
        at wsTest.main(wsTest.java:52)

It says its an invalid transaction but I dont see how this can be? Any
ideas? Code below!!!

Aswell is there a better way of getting the error from Condor via this
interface?

Many Thanks

Nathan

import condor.*;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;

class JobSubmissionException extends Exception
{
      public JobSubmissionException()
      {
            super();
      }
      public JobSubmissionException(String s)
      {
            super(s);
      }
}


class SendFileException extends Exception
{
      public SendFileException()
      {
            super();
      }
      public SendFileException(String s)
      {
            super(s);
      }
}


class GetFileException extends Exception
{
    public GetFileException()
    {
	super();
    }
    public GetFileException(String s)
    {
	super(s);
    } 
}


/** WebServices - A Helper Library for Java Clients to the schedd
 * It is meant to be used to simplify communication with the schedd
using 
 * the Stubs generated by Axis from the schedd's WSDL
 */
public class WebServicesHelper {

    /**
     * Files are sent/got from the schedd-spool in these byte-blocks
     */   
    final static int FILE_TRAN_BUFF = 10240000;
    final static int FILE_TRAN_TIME = 4056;


    /** 
     * The most complex (in terms of client-intelligence) of
submitJobHelpers,
     * Since the client would have to fully-form the jobAd to be
submitted
     * 
     * @@param schedd The schedd where the job is to be submitted
     *
     * @@param transaction The transaction you would like to use to
submit 
     * the job. A "null" value, would invoke a new Transaction.
     * NOTE: the transaction will be committed after the job is
submitted 
     * if it's value is "null", else the calling program is expected to 
     * commit the transaction
     *
     * @@param jobAd The fully-formed jobAd, describing the job to be
submitted 
     * 
     * @@param files List of files to be sent to the schedd
     *
     * @@exception JobSubmissionException raised when<br> 
     * 1.the JobAd is null<br> 
     * 2.schedd fails to begin transaction<br> 
     * 3.unable to submit job to the schedd<br> 
     * 4.schedd fails to commit transaction 
     *
     * @@exception SendFileException Unable to perform file-transfer
with schedd
     *
     * @@exception java.io.IOException Unable to access client file
     *
     * @@exception java.rmi.RemoteException Unable to make remote calls
to schedd
     */
    public static void submitJobHelper(CondorScheddPortType schedd, 
				       condor.Transaction transaction, 
				       ClassAdStruct jobAd, 
				       String[] files) 
	throws JobSubmissionException, SendFileException,
	       java.io.IOException, java.rmi.RemoteException {

	int clusterId, jobId;
	
	if(jobAd != null) {
	    clusterId =
Integer.parseInt(getAttributeValue(jobAd,"ClusterId"));
	    jobId = Integer.parseInt(getAttributeValue(jobAd,"ProcId"));
	}
	else {
	    System.err.println("jobAd attribute is NULL");
	    throw new JobSubmissionException("jobAd attribute is NULL");
	}
	
	boolean dontCommit = true;

	if(transaction == null) {
	    dontCommit = false;
	    TransactionAndStatus trans_s =
schedd.beginTransaction(FILE_TRAN_TIME);
	
if(!(trans_s.getStatus()).getCode().equals(StatusCode.SUCCESS)) {
		System.err.println("Failed to begin Transaction");
		throw new JobSubmissionException("Failed to begin
Transaction");
	    }
	    transaction = trans_s.getTransaction();	
	}

	if (files != null) {
		for (int i = 0; i < files.length; i++) {
sendFileHelper(schedd,
files[i], transaction, clusterId, jobId); }
	}

       	RequirementsAndStatus reqs_s = schedd.submit(transaction,
clusterId, jobId, jobAd);
	if(!(reqs_s.getStatus()).getCode().equals(StatusCode.SUCCESS)) {
	    System.err.println("Failed to submit successfully");
	    throw new JobSubmissionException("Failed to submit
successfully");
	}
	Requirements reqs = reqs_s.getRequirements();	
	if(reqs != null) {
	    System.err.println("Requirements not met");
	}
	if(!dontCommit) {
	    Status st = schedd.commitTransaction(transaction);
	    if(!(st.getCode()).equals(StatusCode.SUCCESS)) {
		System.err.println("Failed to commit");
		throw new JobSubmissionException("Failed to commit");
	    }
	}	
    }

public static void submitJobHelper(CondorScheddPortType schedd, String
owner, UniverseType type, String cmd, String args, String requirements,
				       String[] files) 	throws
JobSubmissionException,
SendFileException, java.io.IOException, java.rmi.RemoteException {
	
//	boolean dontCommit = false;
//	boolean nullTransaction = true;
        TransactionAndStatus trans_s =
schedd.beginTransaction(FILE_TRAN_TIME);
        if(!(trans_s.getStatus()).getCode().equals(StatusCode.SUCCESS))
{
		System.err.println("Failed to begin Transaction");
		throw new JobSubmissionException("Failed to begin
Transaction");
        }
        condor.Transaction transaction = trans_s.getTransaction();	

        IntAndStatus clust_s = schedd.newCluster(transaction);
        if(!(clust_s.getStatus()).getCode().equals(StatusCode.SUCCESS))
{
   	   System.err.println("Failed to create new cluster");
       	   throw new JobSubmissionException("Failed to create new
cluster");	
        }	
	int clusterId = (clust_s.getInteger()).intValue();
	IntAndStatus jobid_s = schedd.newJob(transaction, clusterId);
	if(!(jobid_s.getStatus()).getCode().equals(StatusCode.SUCCESS))
{
	   System.err.println("Failed to create new Job");
	   throw new JobSubmissionException("Failed to create new Job");
        }
        int jobId = (jobid_s.getInteger()).intValue();
  
//	if(!dontCommit) {
	    Status st = schedd.commitTransaction(transaction);
	    if(!(st.getCode()).equals(StatusCode.SUCCESS)) {
		System.err.println("Failed to commit");
		throw new JobSubmissionException("Failed to commit");
//	    }
	}

	if(owner == null) {
	    owner = System.getProperty("user.name");
	}

	if(type == null) {
	    type = UniverseType.STANDARD;
	}

	if(args == null) {
	    args = "";
	}

	if(requirements == null) {
	    requirements = "TRUE";
	}

	if(cmd == null) {
	    System.err.println("Illegal Job Template parameter -
CommandName
has to be specified");
	    throw new JobSubmissionException("Illegal Job Template
parameters -
CommandName has to be specified");
	}
	ClassAdStructAndStatus cas_s = 
	    schedd.createJobTemplate(clusterId, jobId, owner, 
				     type, cmd, args, requirements);
	if(!(cas_s.getStatus()).getCode().equals(StatusCode.SUCCESS)) {
	    System.err.println("Failed to create Job Template");
	    throw new JobSubmissionException("Failed to create Job
Template");
	} 
	ClassAdStruct jobAd = cas_s.getClassAd();
    
//	if(nullTransaction)
//	    transaction = null;
        System.out.println("Submitting job: ClusteId=" + clusterId + "
JobId=" + jobId);	    
	submitJobHelper(schedd, transaction, jobAd, files);
    }

 /**
   * A helper function which provides a simple interface to transfer a
   * file to the schedd.
   *
   * @param schedd The schedd where the job is to be submitted
   *
   * @param filenameWithPath Can be the file's pathname (qualified
filename),
   * i.e. the file does not have to exist in the same directory as the
client
   * (but note that the file is placed in a flat directory (i.e. a
directory
   * corresponding to the job) in the schedd). This parameter can also
be just
   * the name of the file without any parent directories, in which case,
it is
   * assumed that the file exists in the same directory as the client.
   *
   * @param transaction The transaction you would like to use to submit
   * the job. A "null" value, would invoke a new Transaction.
   * NOTE: the transaction will be committed after the job is submitted
   * if it's value is "null", else the calling program is expected to
   * commit the transaction
   *
   * @param clusterId cluster-ID of the job
   *
   * @param jobId job-ID of the job
   *
   * @exception SendFileException raised when<br>
   * 1.schedd fails to begin a transaction for the file-transfer<br>
   * 2.there is an error declaring the file<br>
   * 3.there is an error while sending the file as chunks to the
schedd<br>
   * 4.schedd fails to commit the transaction
   *
   * @exception java.rmi.RemoteException Unable to make remote calls to
schedd
   *
   * @exception java.io.IOException Unable to access client file
   *
   * @exception java.io.FileNotFoundException Unable to locate client
file
   */
public static void sendFileHelper(CondorScheddPortType schedd, String
filenameWithPath, Transaction transaction,
int clusterId, int jobId) throws
      SendFileException, java.rmi.RemoteException, java.io.IOException,
java.io.FileNotFoundException {

    boolean dontCommit = true;

    if (transaction == null) {
    System.out.println("Transaction is null");
      dontCommit = false;
      TransactionAndStatus trans_s = schedd.beginTransaction(60);
      if (! (trans_s.getStatus()).getCode().equals(StatusCode.SUCCESS))
{
        System.err.println("Failed to begin Transaction");
        throw new SendFileException("Failed to begin Transaction");
      }
      transaction = trans_s.getTransaction();
    }

System.out.println(filenameWithPath);

    File file = new File(filenameWithPath);
    int fileLen = (int) file.length();
    FileInputStream inputStream = new FileInputStream(file);

    int index = filenameWithPath.lastIndexOf('/');
    System.out.println(index);
    String filename;
    if (index != -1) {
      filename = filenameWithPath.substring(index + 1);
    }
    else {
      filename = filenameWithPath;
    }
    
   System.out.println(filename);

    Status status = schedd.declareFile(transaction, clusterId, jobId,
filename, fileLen, HashType.NOHASH, null);
    if (!status.getCode().equals(StatusCode.SUCCESS)) {
      System.err.println("Error Declaring File" + status.getMessage());
      throw new SendFileException("Error Declaring File");
    }

    int exBuffLen = fileLen % FILE_TRAN_BUFF;
    byte[] exBuffer = new byte[exBuffLen];
    inputStream.read(exBuffer);
    status = schedd.sendFile(transaction, clusterId,
                             jobId, filename, 0, exBuffer);
    if (!status.getCode().equals(StatusCode.SUCCESS)) {
      System.err.println("Error Sending File");
      throw new SendFileException("Error Sending File");
    }

    if (exBuffLen < fileLen) {
      byte[] buffer = new byte[FILE_TRAN_BUFF]; // 4K buffer.
      int offset = exBuffLen;
      while (offset < fileLen) {
        inputStream.read(buffer);
        status = schedd.sendFile(transaction, clusterId,
                                 jobId, filename, offset, buffer);
        if (!status.getCode().equals(StatusCode.SUCCESS)) {
          System.err.println("Error Sending File");
          throw new SendFileException("Error Sending File");
        }
        offset += FILE_TRAN_BUFF;
      }
    }
    inputStream.close();

    if (!dontCommit) {
      Status st = schedd.commitTransaction(transaction);
      if (! (st.getCode()).equals(StatusCode.SUCCESS)) {
        System.err.println("Failed to commit");
        throw new SendFileException("Failed to commit");
      }
    }
  }





     /**
     * To Either modify the value of an existing attribute in a classad
     * or add the new attribute to the classad
     *
     * @@param jobAd classAd to be modified
     *
     * @@param newAttr the attribute to be updated or added
     *
     * @@return the updated classAd
     */
    public static ClassAdStruct setAttributeValue(
                ClassAdStruct jobAd,
                        ClassAdStructAttr newAttr) {
        ClassAdStructAttr[] classAd = jobAd.getItem();
        int len = classAd.length;
        for(int i=0;i<len;i++) {

if((newAttr.getName()).compareToIgnoreCase(classAd[i].getName()) == 0) {
                        classAd[i].setValue(newAttr.getValue());
                        classAd[i].setType(newAttr.getType());
                        return new ClassAdStruct(classAd);
                }
        }
        //add new entry
        ClassAdStructAttr[] newClassAd = new ClassAdStructAttr[len+1];
        for(int i=0;i<len+1;i++)
                newClassAd[i] = new ClassAdStructAttr();

        for(int i=0;i<len;i++) {
                newClassAd[i].setName(classAd[i].getName());
                newClassAd[i].setValue(classAd[i].getValue());
                newClassAd[i].setType(classAd[i].getType());
        }
        newClassAd[len].setName(newAttr.getName());
        newClassAd[len].setValue(newAttr.getValue());
        newClassAd[len].setType(newAttr.getType());
        return new ClassAdStruct(newClassAd);
    }

    /**
     * Getting the value of a certain attribute in a classad
     *
     * @@param classAd The classad being queried
     *
     * @@param AttrName The name of the attribute being queried
     *
     * @@return The value of the attribute
     */
    public static String getAttributeValue(ClassAdStruct jobAd,String
AttrName) {
        ClassAdStructAttr[] classAd = jobAd.getItem();
        int len = classAd.length;
        for(int i=0;i<len;i++) {
            if(AttrName.compareToIgnoreCase(classAd[i].getName()) == 0)
                return classAd[i].getValue();
        }
        return null;
    }

}




This message has been checked for viruses but the contents of an
attachment
may still contain software viruses, which could damage your computer
system:
you are advised to perform your own checks. Email communications with
the
University of Nottingham may be monitored as permitted by UK
legislation.

_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with
a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users

The archives can be found at either
https://lists.cs.wisc.edu/archive/condor-users/
http://www.opencondor.org/spaces/viewmailarchive.action?key=CONDOR