Re: [Condor-users] Running jobs as submitting user in Windows

[Ben Burnett <burnett@xxxxxxxxxxx>]

Hi Alex:

Unless you are explicitly using one of the following variables as 
part of the job requirements, then the "MachAttributes::publish"
message should not affect you.

WindowsMajorVersion
WindowsMinorVersion
WindowsBuildNumber
WindowsServicePackMajorVersion
WindowsServicePackMinorVersion
WindowsProductType

As to what MachAttributes::publish() does: it serializes the machine
class ad so that is can be used to advertise with and match against.
(And, in this case, the publish() part is incorrect: the code is in
the constructor for MachAttributes.)

What bothers me the most, is that you are seeing this fail on a
Windows machine, which *should* never happen... can we take this
offline and try to figure-out why the version information is not
being gathered?

Regards,
-B

> -----Original Message-----
> From: condor-users-bounces@xxxxxxxxxxx [mailto:condor-users-
> bounces@xxxxxxxxxxx] On Behalf Of Alas, Alex [FEDI]
> Sent: Wednesday, December 10, 2008 7:31 PM
> To: Condor-Users Mail List
> Subject: Re: [Condor-users] Running jobs as submitting user in Windows
> 
> Henrik,
> Question of a million bucks? Where you able to fix this problem? Or you
> gave up on it!!! I am experimenting the exact same issue as yours but
> in my case my Central manager is a Windows XP box and I have a pool of
> 3 XP nodes and 16 Windows 2003 nodes. I runs fine on one of XP boxes
> but it simply won't work on the other 2 XP and the rest of 2003 nodes.
> I am pulling my hair here. Same as you when I do condor_status -l, the
> localcredd setting is only set in the XP box that is able to run the
> jobs. I have place several messages on this condor_users mail list but
> nobody seems to reply back
> Thanks for your help in advance,
> Alex
> 
> -----Original Message-----
> From: condor-users-bounces@xxxxxxxxxxx [mailto:condor-users-
> bounces@xxxxxxxxxxx] On Behalf Of Henrik Fällstrand XX
> Sent: Monday, March 10, 2008 1:04 PM
> To: Condor-Users Mail List
> Subject: Re: [Condor-users] Running jobs as submitting user in Windows
> 
> I am now able to submit jobs from all hosts, but they only execute on
> the Master server on which the CREDD daemon is running. On the execute-
> hosts that do not accept new jobs I found this in the StartLog:
> 
> 3/10 17:39:27 MachAttributes::publish: failed to get Windows version
> information 3/10 17:39:48 my_popen: CreateProcess failed 3/10 17:39:48
> Failed to execute C:\condor/bin/condor_starter.std.exe, ignoring
> 
> What does MachAttributes::publish do? Since jobs do not run and I have
> jobs failing due to the requirements, this seems like a possible
> problem? Also, the file condor_starter.std.exe does not exist, so
> naturally it won't be able to run, so the obvious question is, what
> does it do? do I need it, where do I get it?
> 
> /Henrik
> 
> Ps. I added the condor_config and condor_config.local files from the
> execute server here...
> 
> -----Original Message-----
> From: condor-users-bounces@xxxxxxxxxxx [mailto:condor-users-
> bounces@xxxxxxxxxxx] On Behalf Of Henrik Fällstrand XX
> Sent: den 3 mars 2008 16:21
> To: Condor-Users Mail List
> Subject: Re: [Condor-users] Running jobs as submitting user in Windows
> 
> Hi Coop,
> at the moment all machines are running Windows 2000. We'll probably
> upgrade to 2003 server on the master/execute hosts and Vista on the
> submit hosts during summer.
> 
> /Henrik
> 
> -----Original Message-----
> From: condor-users-bounces@xxxxxxxxxxx [mailto:condor-users-
> bounces@xxxxxxxxxxx] On Behalf Of Thompson, Cooper
> Sent: den 3 mars 2008 15:44
> To: Condor-Users Mail List
> Subject: Re: [Condor-users] Running jobs as submitting user in Windows
> 
> What operating systems are the submit/execute machines in your pool
> running?  And what is the requirements expression on your jobs?  I see
> below that E001B78A6A347.mydom.cypoid.com is WINNT50 (Windows 2000).
> Your job requirements also contain OpSys == WINNT50 only.  If your
> execute/submit are anything other than Windows 2000, you'll need to add
> a more flexible requirements expression.
> 
> Also - I have the same problem querying the pool password.  I've fallen
> back to using a scheduled task to run regedit as SYSTEM so I can
> inspect the SECURITY key in the registry to see that the pool password
> is actually stored.  Not very pretty - but it works for testing.
> 
> Coop
> 
> 
> 
> -----Original Message-----
> From: condor-users-bounces@xxxxxxxxxxx [mailto:condor-users-
> bounces@xxxxxxxxxxx] On Behalf Of Henrik Fällstrand XX
> Sent: Monday, March 03, 2008 9:16 AM
> To: Condor-Users Mail List
> Subject: Re: [Condor-users] Running jobs as submitting user in Windows
> 
> 
> OK, here comes...
> 
> I am doing the tests on Condor 7.00 running on Win2000, latest service
> pack.(will probably run on Win2003 Server and Vista in the near future)
> I use the condor_local.config om all machines, I have the
> condor_config.local.credd on the master server. I included the Master,
> Submit and Execute servers condor_config files.
> 
> I tried with/without SEC_DEFAULT_AUTHENTICATION = REQUIRED since there
> was some post that suggested this to solve the HOSTALLOW_WRITE problem.
> 
> I tried setting UID_DOMAIN to nothing, MYDOM and $(FULL_HOSTNAME).
> Setting it to MYDOM made jobs run on the master server, no other
> combinations worked, setting TRUST_UID_DOMAIN did not do any
> difference.
> 
> Regards,
> 
> Henrik
> 
> -----Original Message-----
> From: condor-users-bounces@xxxxxxxxxxx [mailto:condor-users-
> bounces@xxxxxxxxxxx] On Behalf Of Ben Burnett
> Sent: den 29 februari 2008 22:52
> To: 'Condor-Users Mail List'
> Subject: Re: [Condor-users] Running jobs as submitting user in Windows
> 
> Hi Henrik:
> 
> Could you post your configuration files?
> 
> -B
> 
> -----Original Message-----
> From: condor-users-bounces@xxxxxxxxxxx [mailto:condor-users-
> bounces@xxxxxxxxxxx]
> On Behalf Of Henrik Fällstrand XX
> Sent: Friday, February 29, 2008 9:34 AM
> To: Condor-Users Mail List
> Subject: Re: [Condor-users] Running jobs as submitting user in Windows
> 
> I think I was jumping a bit too early there, when I try to set up an
> execute host and a submit host I can still not get jobs to run from
> those hosts. Jobs submitted om the master run OK on the master, not on
> the execute host, and only if I have set UID_DOMAIN = myDomain
> otherwise they won't start there either...
> 
> Also I am not able to see the pool password on remote machines, but
> setting them was OK:
> condor_store_cred query  -c -n E001CC4D61500.myDomain.cypoid.com
> Account: condor_pool@myDomain
> 
> Operation failed.
>     Make sure your HOSTALLOW_WRITE setting includes this host.
> 
> I have included the master and execute servers in the
> HOSTALLOW_ADMINISTRATOR settings on all machines, I even tried with *
> and have set HOSTALLOW_WRITE = *
> 
> //Henrik
> 
> -----Original Message-----
> From: condor-users-bounces@xxxxxxxxxxx [mailto:condor-users-
> bounces@xxxxxxxxxxx]
> On Behalf Of Henrik Fällstrand XX
> Sent: den 26 februari 2008 09:39
> To: Condor-Users Mail List
> Subject: Re: [Condor-users] Running jobs as submitting user in Windows
> 
> Thanx Burnett, this did the trick, jobs get submitted and run as the
> correct user now!!!
> 
> Best Regards,
> Henrik
> 
> -----Original Message-----
> From: condor-users-bounces@xxxxxxxxxxx [mailto:condor-users-
> bounces@xxxxxxxxxxx]
> On Behalf Of burnett@xxxxxxxxxxx
> Sent: den 25 februari 2008 20:54
> To: Condor-Users Mail List
> Subject: Re: [Condor-users] Running jobs as submitting user in Windows
> 
> Hi Henrik:
> 
> Attached is a working local configuration for you.  I've also included
> one of the test jobs I use (it just sleeps for 10 seconds--not very
> exciting, but it will allow you to see that it is running with the
> correct credentials).  Also, since you didn't mention it explicitly in
> your description, I'll just make a quick point of it: It is in my
> experience that the pool password should always be set, even if it is a
> single machine test pool.
> 
> I'd also recommend a fresh start with the new configuration: Stop the
> service; replace the configuration files (you may want to add your
> email and SMTP server in, since I left them blank); clear out the logs
> as well as the contents of the spool directory; and, finally, restart
> condor.
> After that, you should be able to submit the test job successfully.
> 
> Regards,
> -B
> 
> > I've reinstalled the masterserver with condor 7.0 and have this
> > machine as the execute host as well (a 1 machine cluster :-).
> >
> > I added myself with condor_store_cred and try to submit a job as my
> > user, but still get the same result as before. I did a plain
> > installation for a new master server and added the settings as stated
> > in the etc/condor.local.CREDD file.
> >
> > Jobs will still stay queued with the reason that som requirements are
> > not
> > met:
> >
> > Requirements = (Arch == "INTEL") && (OpSys == "WINNT50") && (Disk >=
> > DiskUsage) && ((Memory * 1024) >= ImageSize) && (HasFileTransfer) &&
> > (HasWindowsRunAsOwner && (LocalCredd =?= "E001B78A6A347:9620"))
> >
> > The submitfile states to run the job as me, without this setting jobs
> > will
> > run:
> >
> > Universe   = vanilla
> > Executable = w:\MIN_LSC.cmd
> > getenv     = True
> > run_as_owner = true                      <----------------------
> > Log        = w:\MIN_LSC_$(cluster).$(process).log
> > Output     = w:\MIN_LSC_$(cluster).$(process).out
> > Error      = w:\MIN_LSC_$(cluster).$(process).err
> > Arguments  =
> > Queue
> >
> > Does anyone have a Windows envireonment with a similar working setup?
> > Can you please post a configuration file of this setup?
> >
> > Thank you!!!
> >
> > Best Regards,
> > Henrik
> >
> > -----Original Message-----
> > From: condor-users-bounces@xxxxxxxxxxx
> > [mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of Henrik
> > Fällstrand XX
> > Sent: den 25 februari 2008 10:23
> > To: Condor-Users Mail List
> > Subject: Re: [Condor-users] Running jobs as submitting user in
> Windows
> >
> > No problemo Ben, thanx for the reply! :-) Yes CONDOR_HOST is pointing
> > to the CM or Master. Cooper: I tried to restart the cluster in
> > sequence as you suggested, but unfortionately it did'nt seem to
> help...
> > Since I haven't been stating to use my cluster yet, and version 7 is
> > out I'm thinking of reinstalling and reconfiguring everything from
> > scratch, hopefully solving this issue.
> >
> > Thanx everyone,
> >
> > Regards
> > Henrik
> >
> > -----Original Message-----
> > From: condor-users-bounces@xxxxxxxxxxx
> > [mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of Ben Burnett
> > Sent: den 22 februari 2008 18:22
> > To: 'Condor-Users Mail List'
> > Subject: Re: [Condor-users] Running jobs as submitting user in
> Windows
> >
> > Hi Henrik:
> >
> > Sorry, my message was not intended to be insulting or presumptuous
> :).
> > I just fired it off in the midst of a flurry of work.
> >
> > I assume that CONDOR_HOST points not to the local host, but to the CM
> > (or MASTER), correct?  CREDD_HOST is evaluated to determine what to
> > place in the LocalCredd attribute, so this would be the first place
> to
> > look. The rest appears to be correct.
> >
> > Regards,
> > -B
> >
> > -----Original Message-----
> > From: condor-users-bounces@xxxxxxxxxxx
> > [mailto:condor-users-bounces@xxxxxxxxxxx]
> > On Behalf Of Henrik Fällstrand XX
> > Sent: Friday, February 22, 2008 2:12 AM
> > To: Condor-Users Mail List
> > Subject: Re: [Condor-users] Running jobs as submitting user in
> Windows
> >
> > Hi Ben,
> >
> > Yes, i know, ping works, I can even telnet to that port and get a
> > connection, no problem... What I was suspecting is that there is no
> > "LocalCredd"  entry when I run "condor_status -l E001B78A6A347". Its
> > the only requirement for submitting a job that does not seem to be
> > met, but then again without this i would not get authenticated?? Is
> > there a way of forcing requirements to be reset?
> >
> > Is there some setting with windows authentiaction that I have missed?
> > On the master resp execute hosts i have the settings below. The CREDD
> > daemon only needs to be running on the master host, right?
> >
> > MASTER:
> >
> > ## CREDD logging settings
> > ## Customize these if you wish.
> > CREDD_LOG = $(LOG)/CreddLog
> > CREDD_DEBUG = D_COMMAND
> > MAX_CREDD_LOG = 50000000
> > DAEMON_LIST = $(DAEMON_LIST), CREDD
> > CREDD    = $(SBIN)/condor_credd.exe
> > SEC_CREDD_SESSION_TIMEOUT = 10
> > # Set security settings so that full security to the credd is
> required
> > CREDD.SEC_DEFAULT_AUTHENTICATION =REQUIRED
> > CREDD.SEC_DEFAULT_ENCRYPTION = REQUIRED CREDD.SEC_DEFAULT_INTEGRITY =
> > REQUIRED CREDD.SEC_DEFAULT_NEGOTIATION = REQUIRED # Require PASSWORD
> > auth for password fetching CREDD.SEC_DAEMON_AUTHENTICATION_METHODS =
> > PASSWORD # Only honor password fetch requests to the trusted
> > "condor_pool" user CREDD.ALLOW_DAEMON = condor_pool@$(UID_DOMAIN) #
> > Require NTSSPI for storing credentials
> > CREDD.SEC_DEFAULT_AUTHENTICATION_METHODS = NTSSPI
> >
> > EXECUTE HOSTS:
> >
> > CREDD_HOST  = $(CONDOR_HOST):$(CREDD_PORT) STARTER_ALLOW_RUNAS_OWNER
> =
> > True CREDD_CACHE_LOCALLY = True
> > CRED_STORE_DIR = $(LOCAL_DIR)/cred_dir   (directory is created....)
> >
> > Regards,
> > Henrik
> >
> > Requirements = (Arch == "INTEL") && (OpSys == "WINNT50") && (Disk >=
> > DiskUsage) && ((Memory * 1024) >= ImageSize) && (HasFileTransfer) &&
> > (HasWindowsRunAsOwner && (LocalCredd =?= "E001B78A6A347:9620"))
> >
> > -----Original Message-----
> > From: condor-users-bounces@xxxxxxxxxxx
> > [mailto:condor-users-bounces@xxxxxxxxxxx]
> > On Behalf Of Ben Burnett
> > Sent: den 21 februari 2008 17:08
> > To: 'Condor-Users Mail List'
> > Subject: Re: [Condor-users] Running jobs as submitting user in
> Windows
> >
> >> Do you know what this could be?
> >
> > It's your machine's hostname with the credd port appended to it:
> >
> > E001B78A6A347.mydom.cypoid.com => E001B78A6A347:9620
> >
> > Can you ping your machine using the unqualified name:
> >
> >> ping E001B78A6A347
> >
> > Regards,
> > -B
> >
> > -----Original Message-----
> > From: condor-users-bounces@xxxxxxxxxxx
> > [mailto:condor-users-bounces@xxxxxxxxxxx]
> > On Behalf Of Henrik Fällstrand XX
> > Sent: Thursday, February 21, 2008 9:29 AM
> > To: Condor-Users Mail List
> > Subject: Re: [Condor-users] Running jobs as submitting user in
> Windows
> >
> > Thanks Mike,
> >
> > Seems like I'm almost there, now I still got a problem with jobs not
> > starting at all when using run_as_user. The jobs can not get the
> > correct resources, I'm suspecting LocalCredd =?= "E001B78A6A347:9620"
> > since this is the only requirement value I can't see when running
> condor_status -l.
> > Do you know what this could be?
> > When I set debug logging to ALL_DEBUG = D_SECURITY then the CreddLog
> > shows that all authentication and handshaking succeeds...
> >
> > Thanks for your help!
> > Regards,
> > Henrik
> >
> >
> >
> > C:\condor>bin\condor_q -analyze 348.0
> >
> > -- Submitter: E001B78A6A347.mydom.cypoid.com : <192.168.52.200:1746>
> :
> > E001B78A6A347.mydom.cypoid.com
> >  ID      OWNER            SUBMITTED     RUN_TIME ST PRI SIZE CMD
> > ---
> > 348.000:  Run analysis summary.  Of 6 machines,
> >       6 are rejected by your job's requirements
> >       0 reject your job because of their own requirements
> >       0 match but are serving users with a better priority in the
> pool
> >       0 match but reject the job for unknown reasons
> >       0 match but will not currently preempt their existing job
> >       0 are available to run your job
> >         No successful match recorded.
> >         Last failed match: Thu Feb 21 12:22:06 2008
> >         Reason for last match failure: no match found
> >
> > WARNING:  Be advised:
> >    No resources matched request's constraints
> >    Check the Requirements expression below:
> >
> > Requirements = (Arch == "INTEL") && (OpSys == "WINNT50") && (Disk >=
> > DiskUsage) && ((Memory * 1024) >= ImageSize) && (HasFileTransfer) &&
> > (HasWindowsRunAsOwner && (LocalCredd =?= "E001B78A6A347:9620"))
> >
> > C:\condor>bin\condor_status -l E001B78A6A347 MyType = "Machine"
> > TargetType = "Job"
> > Name = "vm2@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
> > Machine = "E001B78A6A347.mydom.cypoid.com"
> > Rank = 0.000000
> > CpuBusy = ((LoadAvg - CondorLoadAvg) >= 0.500000)
> > COLLECTOR_HOST_STRING = "E001B78A6A347"
> > CondorVersion = "$CondorVersion: 6.8.8 Dec 19 2007 $"
> > CondorPlatform = "$CondorPlatform: INTEL-WINNT50 $"
> > VirtualMachineID = 2
> > VirtualMemory = 1226570
> > Disk = 29504353
> > CondorLoadAvg = 0.000000
> > LoadAvg = 0.000000
> > KeyboardIdle = 0
> > ConsoleIdle = 0
> > Memory = 1023
> > Cpus = 1
> > StartdIpAddr = "<192.168.52.200:1148>"
> > Arch = "INTEL"
> > OpSys = "WINNT50"
> > UidDomain = "mydom"
> > FileSystemDomain = "E001B78A6A347.mydom.cypoid.com"
> > Subnet = "192.168.52"
> > HasIOProxy = TRUE
> > CheckpointPlatform = "WINNT50 INTEL Unknown normal"
> > TotalVirtualMemory = 2453140
> > TotalDisk = 59008707
> > TotalCpus = 2
> > TotalMemory = 2047
> > KFlops = 1270663
> > Mips = 4553
> > LastBenchmark = 1203605961
> > TotalLoadAvg = 0.040000
> > TotalCondorLoadAvg = 0.000000
> > ClockMin = 974
> > ClockDay = 4
> > TotalVirtualMachines = 2
> > HasFileTransfer = TRUE
> > HasPerFileEncryption = TRUE
> > HasReconnect = TRUE
> > HasMPI = TRUE
> > HasTDP = TRUE
> > HasJobDeferral = TRUE
> > HasJICLocalConfig = TRUE
> > HasJICLocalStdin = TRUE
> > HasWindowsRunAsOwner = TRUE
> > StarterAbilityList =
> >
> "HasFileTransfer,HasPerFileEncryption,HasReconnect,HasMPI,HasTDP,HasJo
> > bDeferral, HasJICLocalConfig,HasJICLocalStdin,HasWindowsRunAsOwner"
> > CpuBusyTime = 0
> > CpuIsBusy = FALSE
> > TimeToLive = 2147483647
> > State = "Unclaimed"
> > EnteredCurrentState = 1203605961
> > Activity = "Idle"
> > EnteredCurrentActivity = 1203605961
> > Start = TRUE
> > Requirements = (START) && (IsValidCheckpointPlatform)
> > IsValidCheckpointPlatform = (((TARGET.JobUniverse == 1) == FALSE) ||
> > ((MY.CheckpointPlatform =!=
> > UNDEFINED) && ((TARGET.LastCheckpointPlatform =?= MY.Checkpoint
> > Platform) || (TARGET.NumCkpts == 0)))) MaxJobRetirementTime = 0
> > CurrentRank = 0.000000 MonitorSelfTime = 1203606681
> > MonitorSelfCPUUsage = 0.013019 MonitorSelfImageSize = 56984.000000
> > MonitorSelfResidentSetSize = 24648 MonitorSelfAge = 731
> > MonitorSelfRegisteredSocketCount = 2 DaemonStartTime = 1203605949
> > UpdateSequenceNumber = 3 MyAddress = "<192.168.52.200:1148>"
> > LastHeardFrom = 1203606866
> > UpdatesTotal = 4
> > UpdatesSequenced = 3
> > UpdatesLost = 0
> > UpdatesHistory = "0x00000000000000000000000000000000"
> >
> > _______________________________________________
> > Condor-users mailing list
> > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx
> > with a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> >
> > The archives can be found at:
> > https://lists.cs.wisc.edu/archive/condor-users/
> >
> >
> > _______________________________________________
> > Condor-users mailing list
> > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx
> > with a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> >
> > The archives can be found at:
> > https://lists.cs.wisc.edu/archive/condor-users/
> >
> > _______________________________________________
> > Condor-users mailing list
> > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx
> > with a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> >
> > The archives can be found at:
> > https://lists.cs.wisc.edu/archive/condor-users/
> >
> >
> > _______________________________________________
> > Condor-users mailing list
> > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx
> > with a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> >
> > The archives can be found at:
> > https://lists.cs.wisc.edu/archive/condor-users/
> >
> > _______________________________________________
> > Condor-users mailing list
> > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx
> > with a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> >
> > The archives can be found at:
> > https://lists.cs.wisc.edu/archive/condor-users/
> >
> > _______________________________________________
> > Condor-users mailing list
> > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx
> > with a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> >
> > The archives can be found at:
> > https://lists.cs.wisc.edu/archive/condor-users/
> >
> 
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with
> a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/
> 
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with
> a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/
> 
> 
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with
> a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/
> 
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with
> a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/
> 
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with
> a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with
> a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/