Re: [HTCondor-users] Failed to find UID and GID for user

[Jason Patton <jpatton@xxxxxxxxxxx>]

You should be able to use the python bindings (installed via pip)
without the daemons being on the machine. In the current version of
the bindings, when you "import htcondor" on a machine without a
condor_config, you will get an error about not being able to find the
config. You can do something like "export CONDOR_CONFIG=/dev/null" to
get around this, and I plan to fix this in the next version. You will
have to initialize the htcondor.Collector or htcondor.Schedd objects
with the hostname of the desired daemon.

schedd = htcondor.Schedd('my-submit-machine.domain')

Jason

On Fri, Feb 9, 2018 at 2:59 PM, Alexey Basinov <abasinov@xxxxxxxxxx> wrote:
> Thank you for you response Todd.
> I didn't know that there are bindings for Python. Thank you, I'll check them
> out.
> On a related note, is it possible to use that new Python API remotely ? Is
> it possible to make an api call from a machine that doesn't run any condor
> daemons ? This is the main reason why I started looking into SOAP api. I
> want to decouple my application from the submit node.
>
>
> On Fri, Feb 9, 2018 at 11:11 AM, Todd Tannenbaum <tannenba@xxxxxxxxxxx>
> wrote:
>>
>> On 2/8/2018 5:41 PM, Alexey Basinov wrote:
>> > Greetings,
>> >
>> >
>> > I'm trying to submit a job via SOAP api but I've run into an issue where
>> > condor_schedd.service.submit() api call returns 'Unknwon cluster or job'
>> > error and the jobs don't get submitted.
>> >
>>
>> Hi Alex,
>>
>> I know this is not the answer you are looking for, but especially since
>> you are using Python, I strongly suggest using HTCondor's Python API instead
>> of using the SOAP API. The Python API really became mature with HTCondor
>> v8.6.x, and is so much nicer than SOAP if you are a Python user.
>>
>> The Python API is more complete, more "pythonic", easier to use, and
>> scales better.  Most importantly, the SOAP API support in HTCondor is
>> deprecated and will likely disappear entirely in upcoming versions. The only
>> advantage of the SOAP API over the Python API for Python users is if you are
>> coming from a client where the HTCondor Python client libraries are not
>> available (i.e. something other than an Intel/AMD machine running Linux,
>> Windows, or Mac...).  From the below it looks like you are on Linux...
>>
>> Assuming you installed HTCondor from RPM or DEB, you probably already have
>> the HTCondor package installed.  If not, you can "pip install htcondor".
>>
>> Then documentation on the HTCondor Python API is here:
>>   https://htcondor-python.readthedocs.io/en/latest/index.html
>>
>> By way of example, to submit a job to a schedd on the same machine that
>> runs '/bin/sleep 30' from Python, you could do:
>>
>>   import htcondor
>>   jobToSubmit = htcondor.Submit({"executable": "/bin/sleep", "arguments":
>> "30"})
>>   schedd = htcondor.Schedd()
>>   with schedd.transaction() as txn:
>>     print jobToSubmit.queue(txn)
>>
>> If you really need to use the SOAP API instead of the Python API, I'd be
>> curious to hear why...
>>
>> Hope the above helps,
>> Todd
>>
>>
>>
>> >
>> > I'm using the following Python code:
>> >
>> >
>> > ########Begin Code#########
>> >
>> > uid = pwd.getpwuid(os.getuid())[0]
>> > condor_schedd = Client(url='http://localhost:8080/condorSchedd.wsdl
>> > <http://localhost:8080/condorSchedd.wsdl>', cache=None,
>> > location='http://localhost:8080 <http://localhost:8080/>')
>>
>> >
>> > transaction = condor_schedd.service.beginTransaction(60)
>> > cluster = condor_schedd.service.newCluster(transaction.transaction)
>> > job = condor_schedd.service.newJob(transaction.transaction,
>> > cluster.integer)
>> > jobAd = condor_schedd.service.createJobTemplate(cluster.integer,
>> >            job.integer, uid, 5, "/bin/sleep", "30", "")
>> > result = condor_schedd.service.submit(transaction.transaction,
>> >             cluster.integer, job.integer, jobAd.classAd)
>> > condor_schedd.service.commitTransaction(transaction.transaction)
>> > print result
>> >
>> > ########End Code#########
>> >
>> >
>> > Here's the response that I get from the service.submit() api call:
>> >
>> > (RequirementsAndStatus){
>> >     status =
>> >        (Status){
>> >           code = "UNKNOWNJOB"
>> >           message = "Unknown cluster or job"
>> >        }
>> >   }
>> >
>> >
>> > When job submission fails, I see the following errors/warnings in
>> > /var/log/condor/ScheddLog file:
>> >
>> > 02/07/18 19:51:47 (pid:33466) Not enforcing MAX_JOBS_PER_OWNER for
>> > submit without owner of cluster 157.
>> > 02/07/18 19:51:47 (pid:33466) passwd_cache::cache_uid(): getpwnam("")
>> > failed: user not found
>> > 02/07/18 19:51:47 (pid:33466) (157.0) Failed to find UID and GID for
>> > user . Cannot chown
>> > /var/lib/condor/spool/157/0/cluster157.proc0.subproc0 to user.
>> >
>> >
>> > Here are the settings I set in condor_config.local:
>> >
>> > ALLOW_SOAP = *
>> > ENABLE_SOAP = True
>> > ENABLE_WEB_SERVER = True
>> > WEB_ROOT_DIR=/usr/lib/condor/webservice
>> > USE_SHARED_PORT = FALSE
>> > SCHEDD_ARGS = -p 8080
>> > ALLOW_WRITE = *
>> > HOSTALLOW_WRITE = *
>> > HOSTALLOW_READ = *
>> > QUEUE_ALL_USERS_TRUSTED = TRUE
>> >
>> >
>> > Seems like this issue has came up in the past:
>> >
>> >
>> > https://www-auth.cs.wisc.edu/lists/htcondor-users/2008-February/msg00139.shtml
>> >
>> > <https://www-auth.cs.wisc.edu/lists/htcondor-users/2008-February/msg00139.shtml>
>> >
>> > https://lists.cs.wisc.edu/archive/htcondor-users/2015-October/msg00123.shtml
>> >
>> > <https://lists.cs.wisc.edu/archive/htcondor-users/2015-October/msg00123.shtml>
>> >
>> > https://www-auth.cs.wisc.edu/lists/htcondor-users/2018-January/msg00151.shtml
>> >
>> > <https://www-auth.cs.wisc.edu/lists/htcondor-users/2018-January/msg00151.shtml>
>> >
>> >
>> > Does anyone have any pointers on how to fix this problem ?
>> >
>> > Thank you.
>> >
>> >
>> > _______________________________________________
>> > HTCondor-users mailing list
>> > To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx
>> > with a
>> > subject: Unsubscribe
>> > You can also unsubscribe by visiting
>> > https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>> >
>> > The archives can be found at:
>> > https://lists.cs.wisc.edu/archive/htcondor-users/
>> >
>>
>>
>> --
>> Todd Tannenbaum <tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
>> Center for High Throughput Computing   Department of Computer Sciences
>> HTCondor Technical Lead                1210 W. Dayton St. Rm #4257
>> Phone: (608) 263-7132                  Madison, WI 53706-1685
>
>
>
>
> --
> Alex Basinov
> Systems Engineer team
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/