Re: [Condor-users] windows vanilla: does any sort of streaming work?

["Rowe, Thomas" <rowet@xxxxxxxxxx>]

I’m an idiot.  The problem was IO buffering.  My messages are small and apparently never fill up the io buffer before job completion.  If I flush stdout everything works fine.  Thanks for your time.

 

---

From: condor-users-bounces@xxxxxxxxxxx [mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of John (TJ) Knoeller
Sent: Thursday, January 20, 2011 12:08 PM
To: Condor-Users Mail List
Subject: Re: [Condor-users] windows vanilla: does any sort of streaming work?

 

I just tried the second technique using a Windows 7 submit machine running Condor 7.4.4 and a Windows XP execute machine running 7.4.4 and it worked just fine.  Here are the submit and and executable (batch file) I used.   Query.bat
dumps a bunch of information about the execution environment to standard out interspersed with calls to sleep.exe to make sure that I can see the streaming happening. 

What I is see is that the output happens in chunks with 60 or 20 second pauses between chunks, which corresponds to the way that query.bat is producing output. 

-tj

---------------------- query.bat ------------------
@echo USERNAME=%USERNAME%
@echo USERDOMAIN=%USERDOMAIN%
@echo ARGS=$*
@echo PATH=%PATH%
@echo -- environment in 60 sec
sleep 60
set
@echo -- _CONDOR_JOB_AD in 20 sec
sleep 20
type "%_CONDOR_JOB_AD%"
@echo -- _CONDOR_MACHINE_AD in 20 sec
sleep 20
type "%_CONDOR_MACHINE_AD%"
@echo -- subkeys of HKLM\Software in 10 sec
sleep 10
for /F "tokens=*" %%I in ('reg query HKLM\Software') do (
  echo %%~nI
  sleep 2
)
@echo -- end
------------------ end of query.bat ---------------------

------------------ query.sub -------------------
Executable = query.bat
Universe = Vanilla
load_profile = true
stream_output = true
should_transfer_files = true
when_to_transfer_output = ON_EXIT
Requirements = (OpSys == "WINNT51" && Arch == "INTEL")
transfer_input_files = sleep.exe
Error = query.err
Output = query.out
Log = query.log
Queue
---------------------- end of query.sub --------

if you don't have sleep.exe, you can get it from the Windows Server 2003 SDK.
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=9d467a69-57ff-4ae7-96ee-b18c4790cffd

On 1/19/2011 3:29 PM, Rowe, Thomas wrote:

On 01/18/2011 09:57 PM, Rowe, Thomas wrote: 

I’m trying to get messages from a long running job back to the submitter >>machine and the solutions apparent from the manual don’t appear to work. >>This is with Windows XP machines.

I tried using: > echo “the message” | condor_chirp put -mode wat -- >>log_file.txt

I also tried just printing the messages to stdout and modifying the >>classad: stream_output = true \n +TransferOut = true

Neither of these approaches actually stream the messages.  Back on the >>submitter machine the file only shoes up when the job has completely >>finished.

Are these features simply known to be broken on Windows, or is there some >>configuration aspect I am missing?  I did have WantIOProxy enabled.

________________________________________

What version of Condor are you running?  Streaming of stdout wasn't support >on Windows until somewhere around version 7.2.

-Greg

________________________________________

These are fresh installs of 7.4.

_______________________________________________

Condor-users mailing list

To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a

subject: Unsubscribe

You can also unsubscribe by visiting

https://lists.cs.wisc.edu/mailman/listinfo/condor-users

The archives can be found at:

https://lists.cs.wisc.edu/archive/condor-users/