Re: [HTCondor-users] [ExternalEmail] Windows - Condor 8.4.1 - queue command in submit file

[Greg.Hitchen@xxxxxxxx]

Still having issues with linux though:

queue statement in test.sub is:

queue fname from list_files |

where the list_files script is:

#!/bin/bash
ls -1 *.dat

The following is the error message (line 23 is the queue command above - "queue fname from list_files |")

condor_submit test.sub

Submitting job(s)
ERROR: on Line 23 of submit file: not a valid command

ERROR: Failed to parse command file (line 23).

Removing the hash bang line from the script does not help.

Thanks for any advice

Cheers

Greg

-----Original Message-----
From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Greg.Hitchen@xxxxxxxx
Sent: Friday, 20 November 2015 1:09 PM
To: htcondor-users@xxxxxxxxxxx
Subject: Re: [HTCondor-users] [ExternalEmail] Windows - Condor 8.4.1 - queue command in submit file

Aaaarrrggghhh! I knew it was me. :)

DOS batch script needs to be:

@echo off
dir /b *.dat

It was choking on the echoed command itself of "dir /b *.dat"

Hope this helps someone else.

Cheers

Greg

-----Original Message-----
From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Greg.Hitchen@xxxxxxxx
Sent: Friday, 20 November 2015 12:48 PM
To: htcondor-users@xxxxxxxxxxx
Subject: Re: [HTCondor-users] [ExternalEmail] Windows - Condor 8.4.1 - queue command in submit file

Thanks for the reply Klint.

Yes, there are ways around

condor_submit test.sub -queue fname in *.dat

not working in windows. That is not my main concern. I was more after the question in my original
email concerning why the pipe to a script to generate the list of items is not working for me.
I still suspect that I'm doing something wrong, i.e.

queue fname from files.bat |

on windows, or:

queue fname from files.sh |

on linux does not seem to work. The batch script files.bat is a one-liner:

dir /b *.dat

as is the linux script files.sh:

ls -1 *.dat

Are you able to confirm if this works for you?

Thanks

Cheers

Greg


-----Original Message-----
From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Klint Gore
Sent: Friday, 20 November 2015 12:21 PM
To: HTCondor-Users Mail List
Subject: Re: [HTCondor-users] [ExternalEmail] Windows - Condor 8.4.1 - queue command in submit file

There is a handler for using "-" as the input for from so maybe there's a problem.

Can you work around it with something like "for %a in (*.dat) do condor_submit test.sub -queue fname from %a" ?

If you're in a batch file, remember to double the % signs. And appropriate quoting where necessary.

Klint.

-----Original Message-----
From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Greg.Hitchen@xxxxxxxx
Sent: Friday, 20 November 2015 12:33 PM
To: htcondor-users@xxxxxxxxxxx
Subject: Re: [HTCondor-users] [ExternalEmail] Windows - Condor 8.4.1 - queue command in submit file

Crickets chirping, tumbleweeds rolling by, the silence is deafening. :)

Anybody?

I've now tried with the latest 8.4.2 with the same results.

Can someone please tell me what I'm doing wrong?

Or confirm that this either happens or doesn't happen for them?

Thanks

Cheers

Greg

-----Original Message-----
From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Greg.Hitchen@xxxxxxxx
Sent: Friday, 13 November 2015 3:37 PM
To: htcondor-users@xxxxxxxxxxx
Subject: Re: [HTCondor-users] [ExternalEmail] Windows - Condor 8.4.1 - queue command in submit file

OK, I've also tried this on a Ubuntu 14 machine with condor 8.4.1 and get the same results i.e. Queue from <script> | Execute <script> and treat output lines as items doesn't work.

I can do the following (making sure there is no queue command in the submit file) on both windows:

dir/b *.dat | condor_submit test.sub -queue fname from -

and linux:

ls -1 *.dat | condor_submit test.sub -queue fname from -

and for only linux:

condor_submit test.sub -queue fname in *.dat

this does not work for windows (file globbing doesn't work I assume).

Has anyone tried the queue command with a script?

Thanks

Cheers

Greg

-----Original Message-----
From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Greg.Hitchen@xxxxxxxx
Sent: Wednesday, 11 November 2015 4:20 PM
To: htcondor-users@xxxxxxxxxxx
Subject: [ExternalEmail] [HTCondor-users] Windows - Condor 8.4.1 - queue command in submit file

Hi All

I've been fiddling with condor 8.4.1 on a windows submit machine and testing the new features of the "queue" command.

Simple submit file:

universe = vanilla
executable = test.bat
output = test_$(Cluster)_$(Process).out
error = test_$(Cluster)_$(Process).err
log = test_$(Cluster)_$(Process).log
requirements = (Arch == "INTEL" || Arch == "X86_64")
+Shortjob = True
rank = ConsoleIdle
arguments = $(ItemIndex)
transfer_input_files = $(fname), dummy.dll #queue fname in input_0.dat input_1.dat input_2.dat input_3.dat input_4.dat input_5.dat input_6.dat input_7.dat input_8.dat input_9.dat #queue fname matching files *.dat #queue fname from files.txt queue fname from files.bat |


All 3 commented out "queue" statements work fine (and identically as expected),
however the last one (attempting to use a script) does not.

The files.bat script is just the command:
dir /b *.dat

When running condor_submit I get:

WARNING: Empty value(s) for fname at ItemIndex=0!.
ERROR: Can't open "C:\Data\condor_stuff\examples\condor_test>dir /b *.dat"  with flags 00 (Invalid argument)

It appears as though it is not recognizing it as a script and is just trying
to look for a list of items (files) in files.bat?

Further testing with the text file of filenames shows that:
queue fname from files.txt |
gives an error:
ERROR: on Line 21 of submit file: not a valid command
ERROR: Failed to parse command file (line 21).

So it appears as though having the | symbol means something at least.

I'm probably doing something wrong. Can anyone enlighten me? Maybe this isn't implemented
in windows yet?

Thanks

Cheers

Greg


_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/