[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] v8.8/9 Late materialization documentation typo (and bug/error)?



Yes.  late materialization is when you sent the submit file to the schedd *instead* of the job classads â because

creating job classads from a submit description *IS* materialization.

 

You can submit late materialization jobs via the python bindings, but only if you use the Submit object, you canât do it

with the schedd.submit* methods, because those only accept job classads.

 

-tj

 

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Kenyi Paolo HurtadoAnampa
Sent: Thursday, March 14, 2019 12:48 PM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Cc: Brian Bockelman <bbockelm@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] v8.8/9 Late materialization documentation typo (and bug/error)?

 

Hi TJ,

 

 

We did a bit of digging today and the problem seems to be that the Schedd puts some special attributes into the job that it gets from reading the proxy file.  The code that does this in the Schedd is skipped for late materialization.   It will need to be revised to work correctly, but we have a plan for how to revise it.

 

That sounds good, I am indeed interested in this functionality for a CMS production project, thanks!

 

I have a related question in the meantime:

I'm trying to use the condor python bindings (Schedd.submitMany) to submit to condor and populating "x509userproxy, x509userproxysubject, x509userproxyfirstfqan) myself in the proc_ads, so that the client side code handling the X509 attributes are bypassed. This seems to work well, but it is also ignoring the 'max_materialize' attribute in my classads for late materialization too :).


Is there a way to use late materialization with Schedd.submit() / Schedd.submitMany() ?

 

Best Regards,

Kenyi

 

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Kenyi Paolo HurtadoAnampa
Sent: Friday, March 1, 2019 3:14 PM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] v8.8/9 Late materialization documentation typo (and bug/error)?

 

Hi John,

 

I see, thank you!

Just one more question then, is there an estimated time for the late materialization work to be completed (e.g.: in terms of months)? I know this can be a tricky question (so, "we are not sure" is also acceptable.), but I thought it would be worth asking. This is an exciting feature by the way, thank you for implementing it!

 

P.S.- Besides the unit test you pointed out, the knob is also briefly mentioned in the dagman documentation (which is included in Page 86 of the HTCondor PDF documentation).

 

Best Regards,

Kenyi

 

On Fri, Mar 1, 2019 at 4:02 PM John M Knoeller <johnkn@xxxxxxxxxxx> wrote:

Yes.  Late materialization is very incomplete and only works for the simplest jobs.   It does not work with x509 proxies at this time. 

This is one of the reasons that it is not yet documentedâ.

 

The only place I can find the mistyped knob name that you mention in the HTCondor git repo (which includes the docs)  is in a comment

on one of our unit tests.

 

The correct knob name is indeed SCHEDD_ALLOW_LATE_MATERIALIZE, and it defaults to true

in the 8.8 and 8.9 releases. 

 

This knob will be removed entirely once late materialization work is complete. 

For now, the only reason to set the knob is to disable it..

 

-tj

 

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Kenyi Paolo HurtadoAnampa
Sent: Friday, March 1, 2019 2:50 PM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] v8.8/9 Late materialization documentation typo (and bug/error)?

 

Just to update on the late materialization + x509 user proxies.

 

If I change the owner of my proxy file to "condor" (rather than my own username), then the proxy can be read.

This only happens if I try to use late materialization, so it seems like a bug. 

If so, how do I report the issue by the way (or is this mail-list the right way for reporting issues)?.

 

Thanks in advance for the support!

 

Best Regards,

Kenyi 

 

On Fri, Mar 1, 2019 at 3:32 PM Kenyi Paolo HurtadoAnampa <khurtado@xxxxxx> wrote:

Hello,

 

I'm reading the documentation on late materialization available for HTCondor 8.7.4+ and noticed it says the schedd needs the following:

 

SCHEDD_ALLOW_LATE_MATERIALIZATION = true

 

However, the real knob seems to be (at least from my tests):

 

SCHEDD_ALLOW_LATE_MATERIALIZE = true

 

Is this just a typo?

Also, when I try to submit jobs that require a grid proxy to be forwarded using this feature (using "max_materialize = 4" in my submit file), I get an "unable to read proxy file" error that doesn't happen when "SCHEDD_ALLOW_LATE_MATERIALIZE = false". The error is the following:

 

=====================

TriggerEventTypeNumber = 35

...

037 (1532.-01.000) 03/01 21:22:04 Job Materialization Paused

        failed to create ClassAd for Job 1532.0 : Submit:-1:unable to read proxy file

 

        PauseCode 1

...

028 (1532.-01.000) 03/01 21:22:04 Job ad information event triggered.

JOB_Site = "$$(GLIDEIN_Site:Unknown)"

JOB_GLIDEIN_Name = "$$(GLIDEIN_Name:Unknown)"

Reason = "failed to create ClassAd for Job 1532.0 : Submit:-1:unable to read proxy file

"================================

 

My proxy has the typical patter /tmp/x509up_$(id -u $USER) and has the right ownership/permissions. 

If I comment the x509proxy line in my submit file, then I see 4 jobs materialized! But of course, those will stay Idle forever because of the missing proxy.

Any idea on this one?

 

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/