Hi,
I looked through my code and found that there wasnt anything to
actually
send files to the server (whoops!-weird that it wasnt in the IBM
download though).
So I ve added the sendFileHelper{} to the WebServicesHelper so now
I get
a different error:
query collector for the schedd address, please be patient ...
http://128.243.186.210:35501
Submitting job: ClusteId=31 JobId=0
/home/nharmston/test.sh
15
test.sh
Error Declaring File Invalid transaction
Exception in thread "main" SendFileException: Error Declaring File
at WebServicesHelper.sendFileHelper(WebServicesHelper.java:
279)
at WebServicesHelper.submitJobHelper(WebServicesHelper.java:
118)
at WebServicesHelper.submitJobHelper(WebServicesHelper.java:
204)
at wsTest.main(wsTest.java:52)
It says its an invalid transaction but I dont see how this can be? Any
ideas? Code below!!!
Aswell is there a better way of getting the error from Condor via this
interface?
Many Thanks
Nathan
import condor.*;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
class JobSubmissionException extends Exception
{
public JobSubmissionException()
{
super();
}
public JobSubmissionException(String s)
{
super(s);
}
}
class SendFileException extends Exception
{
public SendFileException()
{
super();
}
public SendFileException(String s)
{
super(s);
}
}
class GetFileException extends Exception
{
public GetFileException()
{
super();
}
public GetFileException(String s)
{
super(s);
}
}
/** WebServices - A Helper Library for Java Clients to the schedd
* It is meant to be used to simplify communication with the schedd
using
* the Stubs generated by Axis from the schedd's WSDL
*/
public class WebServicesHelper {
/**
* Files are sent/got from the schedd-spool in these byte-blocks
*/
final static int FILE_TRAN_BUFF = 10240000;
final static int FILE_TRAN_TIME = 4056;
/**
* The most complex (in terms of client-intelligence) of
submitJobHelpers,
* Since the client would have to fully-form the jobAd to be
submitted
*
* @@param schedd The schedd where the job is to be submitted
*
* @@param transaction The transaction you would like to use to
submit
* the job. A "null" value, would invoke a new Transaction.
* NOTE: the transaction will be committed after the job is
submitted
* if it's value is "null", else the calling program is
expected to
* commit the transaction
*
* @@param jobAd The fully-formed jobAd, describing the job to be
submitted
*
* @@param files List of files to be sent to the schedd
*
* @@exception JobSubmissionException raised when<br>
* 1.the JobAd is null<br>
* 2.schedd fails to begin transaction<br>
* 3.unable to submit job to the schedd<br>
* 4.schedd fails to commit transaction
*
* @@exception SendFileException Unable to perform file-transfer
with schedd
*
* @@exception java.io.IOException Unable to access client file
*
* @@exception java.rmi.RemoteException Unable to make remote
calls
to schedd
*/
public static void submitJobHelper(CondorScheddPortType schedd,
condor.Transaction transaction,
ClassAdStruct jobAd,
String[] files)
throws JobSubmissionException, SendFileException,
java.io.IOException, java.rmi.RemoteException {
int clusterId, jobId;
if(jobAd != null) {
clusterId = Integer.parseInt(getAttributeValue
(jobAd,"ClusterId"));
jobId = Integer.parseInt(getAttributeValue(jobAd,"ProcId"));
}
else {
System.err.println("jobAd attribute is NULL");
throw new JobSubmissionException("jobAd attribute is NULL");
}
boolean dontCommit = true;
if(transaction == null) {
dontCommit = false;
TransactionAndStatus trans_s =
schedd.beginTransaction(FILE_TRAN_TIME);
if(!(trans_s.getStatus()).getCode().equals(StatusCode.SUCCESS)) {
System.err.println("Failed to begin Transaction");
throw new JobSubmissionException("Failed to begin Transaction");
}
transaction = trans_s.getTransaction();
}
if (files != null) {
for (int i = 0; i < files.length; i++) { sendFileHelper(schedd,
files[i], transaction, clusterId, jobId); }
}
RequirementsAndStatus reqs_s = schedd.submit(transaction,
clusterId, jobId, jobAd);
if(!(reqs_s.getStatus()).getCode().equals(StatusCode.SUCCESS)) {
System.err.println("Failed to submit successfully");
throw new JobSubmissionException("Failed to submit
successfully");
}
Requirements reqs = reqs_s.getRequirements();
if(reqs != null) {
System.err.println("Requirements not met");
}
if(!dontCommit) {
Status st = schedd.commitTransaction(transaction);
if(!(st.getCode()).equals(StatusCode.SUCCESS)) {
System.err.println("Failed to commit");
throw new JobSubmissionException("Failed to commit");
}
}
}
public static void submitJobHelper(CondorScheddPortType schedd, String
owner, UniverseType type, String cmd, String args, String
requirements,
String[] files) throws JobSubmissionException,
SendFileException, java.io.IOException, java.rmi.RemoteException {
// boolean dontCommit = false;
// boolean nullTransaction = true;
TransactionAndStatus trans_s =
schedd.beginTransaction(FILE_TRAN_TIME);
if(!(trans_s.getStatus()).getCode().equals
(StatusCode.SUCCESS))
{
System.err.println("Failed to begin Transaction");
throw new JobSubmissionException("Failed to begin Transaction");
}
condor.Transaction transaction = trans_s.getTransaction();
IntAndStatus clust_s = schedd.newCluster(transaction);
if(!(clust_s.getStatus()).getCode().equals
(StatusCode.SUCCESS))
{
System.err.println("Failed to create new cluster");
throw new JobSubmissionException("Failed to create new
cluster");
}
int clusterId = (clust_s.getInteger()).intValue();
IntAndStatus jobid_s = schedd.newJob(transaction, clusterId);
if(!(jobid_s.getStatus()).getCode().equals(StatusCode.SUCCESS)) {
System.err.println("Failed to create new Job");
throw new JobSubmissionException("Failed to create new Job");
}
int jobId = (jobid_s.getInteger()).intValue();
// if(!dontCommit) {
Status st = schedd.commitTransaction(transaction);
if(!(st.getCode()).equals(StatusCode.SUCCESS)) {
System.err.println("Failed to commit");
throw new JobSubmissionException("Failed to commit");
// }
}
if(owner == null) {
owner = System.getProperty("user.name");
}
if(type == null) {
type = UniverseType.STANDARD;
}
if(args == null) {
args = "";
}
if(requirements == null) {
requirements = "TRUE";
}
if(cmd == null) {
System.err.println("Illegal Job Template parameter - CommandName
has to be specified");
throw new JobSubmissionException("Illegal Job Template
parameters -
CommandName has to be specified");
}
ClassAdStructAndStatus cas_s =
schedd.createJobTemplate(clusterId, jobId, owner,
type, cmd, args, requirements);
if(!(cas_s.getStatus()).getCode().equals(StatusCode.SUCCESS)) {
System.err.println("Failed to create Job Template");
throw new JobSubmissionException("Failed to create Job
Template");
}
ClassAdStruct jobAd = cas_s.getClassAd();
// if(nullTransaction)
// transaction = null;
System.out.println("Submitting job: ClusteId=" + clusterId + "
JobId=" + jobId);
submitJobHelper(schedd, transaction, jobAd, files);
}
/**
* A helper function which provides a simple interface to transfer a
* file to the schedd.
*
* @param schedd The schedd where the job is to be submitted
*
* @param filenameWithPath Can be the file's pathname (qualified
filename),
* i.e. the file does not have to exist in the same directory as the
client
* (but note that the file is placed in a flat directory (i.e. a
directory
* corresponding to the job) in the schedd). This parameter can also
be just
* the name of the file without any parent directories, in which
case,
it is
* assumed that the file exists in the same directory as the client.
*
* @param transaction The transaction you would like to use to
submit
* the job. A "null" value, would invoke a new Transaction.
* NOTE: the transaction will be committed after the job is
submitted
* if it's value is "null", else the calling program is expected to
* commit the transaction
*
* @param clusterId cluster-ID of the job
*
* @param jobId job-ID of the job
*
* @exception SendFileException raised when<br>
* 1.schedd fails to begin a transaction for the file-transfer<br>
* 2.there is an error declaring the file<br>
* 3.there is an error while sending the file as chunks to the
schedd<br>
* 4.schedd fails to commit the transaction
*
* @exception java.rmi.RemoteException Unable to make remote
calls to
schedd
*
* @exception java.io.IOException Unable to access client file
*
* @exception java.io.FileNotFoundException Unable to locate client
file
*/
public static void sendFileHelper(CondorScheddPortType schedd, String
filenameWithPath, Transaction transaction,
int clusterId, int jobId) throws
SendFileException, java.rmi.RemoteException,
java.io.IOException,
java.io.FileNotFoundException {
boolean dontCommit = true;
if (transaction == null) {
System.out.println("Transaction is null");
dontCommit = false;
TransactionAndStatus trans_s = schedd.beginTransaction(60);
if (! (trans_s.getStatus()).getCode().equals
(StatusCode.SUCCESS))
{
System.err.println("Failed to begin Transaction");
throw new SendFileException("Failed to begin Transaction");
}
transaction = trans_s.getTransaction();
}
System.out.println(filenameWithPath);
File file = new File(filenameWithPath);
int fileLen = (int) file.length();
FileInputStream inputStream = new FileInputStream(file);
int index = filenameWithPath.lastIndexOf('/');
System.out.println(index);
String filename;
if (index != -1) {
filename = filenameWithPath.substring(index + 1);
}
else {
filename = filenameWithPath;
}
System.out.println(filename);
Status status = schedd.declareFile(transaction, clusterId, jobId,
filename, fileLen, HashType.NOHASH, null);
if (!status.getCode().equals(StatusCode.SUCCESS)) {
System.err.println("Error Declaring File" + status.getMessage
());
throw new SendFileException("Error Declaring File");
}
int exBuffLen = fileLen % FILE_TRAN_BUFF;
byte[] exBuffer = new byte[exBuffLen];
inputStream.read(exBuffer);
status = schedd.sendFile(transaction, clusterId,
jobId, filename, 0, exBuffer);
if (!status.getCode().equals(StatusCode.SUCCESS)) {
System.err.println("Error Sending File");
throw new SendFileException("Error Sending File");
}
if (exBuffLen < fileLen) {
byte[] buffer = new byte[FILE_TRAN_BUFF]; // 4K buffer.
int offset = exBuffLen;
while (offset < fileLen) {
inputStream.read(buffer);
status = schedd.sendFile(transaction, clusterId,
jobId, filename, offset, buffer);
if (!status.getCode().equals(StatusCode.SUCCESS)) {
System.err.println("Error Sending File");
throw new SendFileException("Error Sending File");
}
offset += FILE_TRAN_BUFF;
}
}
inputStream.close();
if (!dontCommit) {
Status st = schedd.commitTransaction(transaction);
if (! (st.getCode()).equals(StatusCode.SUCCESS)) {
System.err.println("Failed to commit");
throw new SendFileException("Failed to commit");
}
}
}
/**
* To Either modify the value of an existing attribute in a
classad
* or add the new attribute to the classad
*
* @@param jobAd classAd to be modified
*
* @@param newAttr the attribute to be updated or added
*
* @@return the updated classAd
*/
public static ClassAdStruct setAttributeValue(
ClassAdStruct jobAd,
ClassAdStructAttr newAttr) {
ClassAdStructAttr[] classAd = jobAd.getItem();
int len = classAd.length;
for(int i=0;i<len;i++) {
if((newAttr.getName()).compareToIgnoreCase(classAd[i].getName()) ==
0) {
classAd[i].setValue(newAttr.getValue());
classAd[i].setType(newAttr.getType());
return new ClassAdStruct(classAd);
}
}
//add new entry
ClassAdStructAttr[] newClassAd = new ClassAdStructAttr[len+1];
for(int i=0;i<len+1;i++)
newClassAd[i] = new ClassAdStructAttr();
for(int i=0;i<len;i++) {
newClassAd[i].setName(classAd[i].getName());
newClassAd[i].setValue(classAd[i].getValue());
newClassAd[i].setType(classAd[i].getType());
}
newClassAd[len].setName(newAttr.getName());
newClassAd[len].setValue(newAttr.getValue());
newClassAd[len].setType(newAttr.getType());
return new ClassAdStruct(newClassAd);
}
/**
* Getting the value of a certain attribute in a classad
*
* @@param classAd The classad being queried
*
* @@param AttrName The name of the attribute being queried
*
* @@return The value of the attribute
*/
public static String getAttributeValue(ClassAdStruct jobAd,String
AttrName) {
ClassAdStructAttr[] classAd = jobAd.getItem();
int len = classAd.length;
for(int i=0;i<len;i++) {
if(AttrName.compareToIgnoreCase(classAd[i].getName())
== 0)
return classAd[i].getValue();
}
return null;
}
}
This message has been checked for viruses but the contents of an
attachment
may still contain software viruses, which could damage your
computer system:
you are advised to perform your own checks. Email communications
with the
University of Nottingham may be monitored as permitted by UK
legislation.
_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx
with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users
The archives can be found at either
https://lists.cs.wisc.edu/archive/condor-users/
http://www.opencondor.org/spaces/viewmailarchive.action?key=CONDOR
![[Computer Systems Lab]](https://www-auth.cs.wisc.edu/pics/csl_logo.gif){kind=logo}