[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Problem with WSDL and SOAP usage



On 03/10/2010 04:21 PM, SMITH Michael wrote:
> Thanks Matt, I was hoping you'd jump in. 
> 
>> Matt said:
>> "Some services do that. All sane toolkits let you override, and you
> already know the endpoint if you're issuing a GET for the WSDL
> dynamically."
> 
> I am using netbeans to generate the code from the wsdl and I'm sure you
> are correct about dynamically overriding the endpoint.  I am sure I can
> overcome this issue. I appreciate your depth of understanding.
> 
>> Matt said:
>> "If you go this way be sure to use -format."
> 
> Thanks for the wise tip.  I'll make changes to use -format.
> 
>> Matt said:
>> Is the job being attempted? Is the NumShadowStarts = 0 (condor_q -long
> <jobid> | grep NumShadowStarts)?
> 
> Yes, NumShadowStarts = 5

This means the Schedd is attempting to run the job. You might check the ShadowLog or the StarterLog.slotX for the LastRemoteHost to see why the job is getting booted before executing.


>> Matt said:
>> You said the Schedd is running on a Linux box? If so I'll bet the
> "e:\tmp' isn't accessible from where the condor_schedd is running. If 
>> you need files transferred in for the job you'll need to send them
> along as well. There's a helper library to deal with these things,
> google for 
>> birdbath.jar
>>
>> The job is probably trying to run but failing on those paths. You can
> probably see that in the SchedLog.
> 
> Unless I misunderstand, my netbeans app is running on Windows and I'm
> communicating with the http://localhost:8088 so that means my Schedd is
> on windows, correct?  Dazed and confuzed.

Ahh, ok. I got the impression you were communicating with a remote machine, which might not have access to those files.


> However you bring up and interesting point which raises new questions.
> Should I expect the executable file be transferred without changes to
> the code shown in my original post?  Do I have to doing anything about
> having the output moved back into e:\tmp\uname.out etc... or should the
> code work as is?

Using the SOAP API, you have to do the transfer in and out. It's kinda like using submit -remote.


> Regarding birdbath, I wrote the same submit operation using birdbath and
> it still didn't work, leaving the job in the idle state.  However the
> files issue could be the culprit.

Yeah, definitely take a look at why the job is failing to start. The job is apparently only Idle when you're looking at it.

Best,


matt


> Michael Smith
>  
> 
> -----Original Message-----
> From: condor-users-bounces@xxxxxxxxxxx
> [mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of Matthew Farrellee
> Sent: Wednesday, March 10, 2010 9:33
> To: Condor-Users Mail List
> Subject: Re: [Condor-users] Problem with WSDL and SOAP usage
> 
> Few quick comments inline...
> 
> SMITH Michael wrote:
>> Thank you very much for your help.
>>
>> I tried requestReschedule() this morning to no avail, the job remains 
>> in the idle state and the only way to get it out is to use condor_rm 
>> <job>; condor_rm -forcex <job>.
>>
>> Another issue to mention, by convention the condor web service is 
>> supposed to massage the "SOAP:address location" upon HTTP GET to 
>> report the correct URI (and port), and it doesn't do that.  Leaving 
>> you with the only option to manually edit the "SOAP:address location" 
>> attribute of the WSDL before code generation.  This creates the 
>> problem that the port and address are fixed and cannot be configurable
> and/or dynamic.
>> Unless I am flat out wrong about it, this to me is a serious flaw and 
>> the SOAP/WSDL implementation is crippled.
> 
> Some services do that. All sane toolkits let you override, and you
> already know the endpoint if you're issuing a GET for the WSDL
> dynamically.
> 
> 
>> I'm moving on to write java code that makes calls to condor commands
> via
>> the command line interface, which is btw is going extremely well.
> Doing
>> so mainly because of the trouble I've faced with the SOAP API, and it
>> seems like the problems I've faced are just the tip of the iceberg.
> One
>> hurdle after the other.  Unless someone has a very compelling reason
> to
>> switch back, I can't afford to spend more time on troubleshooting the
>> SOAP interface.
> 
> If you go this way be sure to use -format.
> 
> (more below)
> 
> 
>> Michael Smith
>>  
>>
>> -----Original Message-----
>> From: condor-users-bounces@xxxxxxxxxxx
>> [mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of SMITH Michael
>> Sent: Tuesday, March 9, 2010 16:51
>> To: johnson.raj@xxxxxxxxx; Condor-Users Mail List
>> Subject: Re: [Condor-users] Problem with WSDL and SOAP usage
>>
>> Excellent, thank you so much for the reply.
>>
>> My responses follow:
>> 1. Did that.  Windows can't do -better, linux with -remote simply
> shows
>> same as -ana alone, but with a little extra showing the requirements.
>> 2. Yes compared the SOAP vs. CMD line with a visual diff tool, and
> tried
>> extensively inserting classad lines making a close match as possible.
>> Ultimately failed.
>> 3. Tomorrow, I'll try the requestReschedule() because I consistently
> get
>> "2 match but reject for unknown reasons".  Does that make sense?
>>
>>
>> Michael Smith
>>  
>>
>> -----Original Message-----
>> From: condor-users-bounces@xxxxxxxxxxx
>> [mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of Johnson koil
> Raj
>> Sent: Tuesday, March 9, 2010 10:10
>> To: Condor-Users Mail List
>> Subject: Re: [Condor-users] Problem with WSDL and SOAP usage
>>
>> Hi Smith,
>>
>>        Do the following.
>>
>>       1. condor_q <jobId> -bet -ana
>>
>>       2. Did you compared the compare the JobAds submitted through
> SOAP
>> & Cmd line. If some got missed add that also.
>>
>>       3. If your job matches any machines call requestReschedule()
> after
>> job submission.
>>
>> by
>> Johnson
>>
>>
>> SMITH Michael wrote:
>>> Using 7.2.4 on Windows XP, with Netbeans 6.8 I've generated a jar
> with
>>
>>> the condorSchedd.wsdl and condorCollector.wsdl.  In addition I've 
>>> created a small java app which attempts to submit a job.  For 3 full 
>>> days, I've followed instructions and countless pages on the web 
>>> searching for tidbits of information that can help me.
>>>  
>>> For a test, the job works well using the equivalent job from the 
>>> command line (condor_submit) using the following submit file.
>>>  
>>> // THIS Woks well only when the SCHEDD_ARGS = -p 8088 is not enabled 
>>> universe = vanilla executable = E:\xfer\myuname_$$(OpSys) arguments =
> 
>>> -a output = uname.out error = uname.err log = uname.log
>>>  
>>> requirements = OpSys == "LINUX"
>>> shouldtransferfiles = IF_NEEDED
>>> whentotransferoutput = ON_EXIT
>>>  
>>> queue
>>>  
>>> The problem is the job submits but remains idle forever.  My job is 
>>> submitted on Windows and attemps to run a job on a Linux (Ubuntu
> 9.10)
> 
> Is the job being attempted? Is the NumShadowStarts = 0 (condor_q -long 
> <jobid> | grep NumShadowStarts)?
> 
> 
>>> running Condor 7.2.4, the condor master is the linux box.  This is
> the
>>
>>> equivalent java code that submits well but the job never runs,
> instead
>>
>>> remaining in the idle state.
>>>  
>>> // I must enable the SCHEDD_ARGS = -p 8088
>>>  
>>>             CondorSchedd service = new CondorSchedd();
>>>             CondorScheddPortType port = service.getCondorSchedd();
>>>  
>>>             TransactionAndStatus transaction = 
>>> port.beginTransaction(3000);
>>>             System.out.println("transaction.status := " + 
>>> transaction.getStatus().getCode().value());
>>>             IntAndStatus clusterId =
>>> port.newCluster(transaction.getTransaction());
>>>             System.out.println("clusterId.status := " + 
>>> clusterId.getStatus().getCode().value());
>>>             IntAndStatus jobId =
>>> port.newJob(transaction.getTransaction(), clusterId.getInteger());
>>>             System.out.println("jobId.status := " + 
>>> jobId.getStatus().getCode().value());
>>>             System.out.println("clusterId := " +
>>> clusterId.getInteger() + " jobId := " + jobId.getInteger());
>>>             ClassAdStructAndStatus jobAd = port.createJobTemplate(
>>>                     clusterId.getInteger(),
>>>                     jobId.getInteger(),
>>>                     System.getProperty("user.name"),
>>>                     UniverseType.VANILLA,
>>>                     "e:\\xfer\\myuname_LINUX",
>>>                     "-a", // arguments
>>>                     "(OpSys == \"LINUX\")"
>>>                     );
>>>             System.out.println("classAd.status := " + 
>>> jobAd.getStatus().getCode().value());
>>>              ClassAdStructAttr attr = new ClassAdStructAttr();
>>>             attr.setName("Args");
>>>             attr.setType(ClassAdAttrType.STRING_ATTR);
>>>             attr.setValue("-a");
>>>             jobAd.getClassAd().getValue().getItem().add(attr);
>>>  
>>>             attr = new ClassAdStructAttr();
>>>             attr.setName("Out");
>>>             attr.setType(ClassAdAttrType.STRING_ATTR);
>>>             attr.setValue("e:\\tmp\\uname.out");
>>>             jobAd.getClassAd().getValue().getItem().add(attr);
>>>  
>>>             attr = new ClassAdStructAttr();
>>>             attr.setName("Err");
>>>             attr.setType(ClassAdAttrType.STRING_ATTR);
>>>             attr.setValue("e:\\tmp\\uname.err");
>>>             jobAd.getClassAd().getValue().getItem().add(attr);
>>>  
>>>             attr = new ClassAdStructAttr();
>>>             attr.setName("UserLog");
>>>             attr.setType(ClassAdAttrType.STRING_ATTR);
>>>             attr.setValue("e:\\tmp\\uname.log");
>>>             jobAd.getClassAd().getValue().getItem().add(attr);
>>>  
>>>             attr = new ClassAdStructAttr();
>>>             attr.setName("ShouldTransferFiles");
>>>             attr.setType(ClassAdAttrType.STRING_ATTR);
>>>             attr.setValue("IF_NEEDED");
>>>             jobAd.getClassAd().getValue().getItem().add(attr);
>>>  
>>>             attr = new ClassAdStructAttr();
>>>             attr.setName("WhenToTransferOutput");
>>>             attr.setType(ClassAdAttrType.STRING_ATTR);
>>>             attr.setValue("ON_EXIT");
>>>             jobAd.getClassAd().getValue().getItem().add(attr);
>>>
>>>             RequirementsAndStatus submitResult = port.submit(
>>>                     transaction.getTransaction(),
>>>                     clusterId.getInteger(),
>>>                     jobId.getInteger(),
>>>                     jobAd.getClassAd().getValue()
>>>                     );
>>>  
>>>             System.out.println("result.status := " + 
>>> submitResult.getStatus().getCode().value());
>>>             Status status =
>>> port.commitTransaction(transaction.getTransaction());
>>>             System.out.println("commit status := " + 
>>> status.getMessage().getValue());
>>>  
>>> Any help would be greatly appreciated.  I've tried using the 
>>> birdbath.jar but ran up against a wall when trying to Axis, it's old 
>>> and no longer functional.
> 
> You said the Schedd is running on a Linux box? If so I'll bet the 
> "e:\tmp' isn't accessible from where the condor_schedd is running. If 
> you need files transferred in for the job you'll need to send them along
> 
> as well. There's a helper library to deal with these things, google for 
> birdbath.jar
> 
> The job is probably trying to run but failing on those paths. You can 
> probably see that in the SchedLog.
> 
> 
> Best,
> 
> 
> matt
> 
> 
>>> /Michael Smith/
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with
> a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/