Mailing List Archives

Public Access

 		[Computer Systems Lab]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] change to condor_submit - user feedback desired! (was Re: multiple condor_submit's - one cluster)

Lauren

It is for the 1st time that I see such a beautifully written, crystal clear, lucid  analysis in defense for usability...

I loved every word I hope that will change the way everyone thinks on this HTCondor forum. This is called UX.

Translation: repect and make happy the future users who never read this thread and never will.

Thanks Lauren

Miha

On Feb 9, 2015 2:05 PM, "Lauren Michael" <lmichael@xxxxxxxx> wrote:
Hi All,

First, I strongly echo Ben's points, especially for keeping the submit file as a record of the exact syntax for future reference by the user (to understand what he/she did). 

For example, the following (#2):
    ls data/*.csv | grep foo | condor_submit -submit_per_line input_line
employs skills and unix familiarity (grep, pipe) that most users I work with largely do not have. To remember and use such a command, they'd end up recording it in a document or perhaps a script. The greater the number of the arguments in the command, the more this type of recording becomes true, in my experience.


Stepping back, I believe there are multiple motivations emerging in this thread, though I'll also point out that I *believe* they are all from "advanced" users of HTCondor and unix (at least for the names I recognize in this thread, probably excluding myself).

Here's an attempt at a summary of desired outcomes listed in this email thread so far:
1. Provide users with an in-file alternative to $(Process) for cases when the user has many similarly-named but non-numbered files, and lacks the know-how/desire/time to convert such files to numbered filenames while maintaining metadata about which file is which.
(not mentioned here yet, but I'm adding it now, as I interact with countless non-advanced users facing this barrier and have otherwise discussed it at length with people like Todd T, motivating a foreach-like option.)
2. Create a simple syntax for executing #1 that doesn't require significant unix/scripting experience.
3. If possible, allow advanced users to also intuitively use the solution in a unix-y and/or scripting way.
4. Minimize performance/latency side effects.


Specifically commenting on syntax:
I also see David's point for not creating a universal name ("file"). Is something *like* the following possible?:

queue foreach species in $(species).data

I'm also in favor of something like the above because I *think* "queue foreach data/*.csv" effectively co-ops the wildcard and would keep the user from specifying files using multiple wildcard instances (say, for sub-directories). For example, what if I wanted to "queue foreach *_data/*.data"?


I am so excited that we're at the point of crowd-sourcing input for such a feature!

-Lauren


Lauren Michael - Research Computing Facilitator, University of Wisconsin - Madison

On Fri, Feb 6, 2015 at 11:58 AM, Ben Cotton <ben.cotton@xxxxxxxxxxxxxxxxxx> wrote:
On Fri, Feb 6, 2015 at 12:49 PM, Brian Bockelman <bbockelm@xxxxxxxxxxx> wrote:
> Honestly, I don't particularly like the stdin-style approaches because, when debugging, it's a little less apparent on how to reproduce because you don't see what users gave you; it is more unix-y though!

I dislike it, as well, but for a different reason. The more you have
to do things outside of the submit file, the more like writing a
script it becomes. If the idea is to make it so that unsophisticated
users can more easily submit this types of workloads, we should avoid
making them write a script.

Also, keeping it in the submit file make it easier to support things
like Windows schedds, remote submissions, etc. in a way more
transparent to the user.

I'm inclined to favor Todd's proprosal #1. The foreach versus for x in
[list] distinction is somewhat irrelevant. The former is more familiar
to csh users, the latter to bash and similar. I actually prefer
foreach slightly just because it simplifies the syntax. But in any
case, so long as it is clear, either works.


Thanks,
BC

--
Ben Cotton
main: 888.292.5320

Cycle Computing
Better Answers. Faster.

http://www.cyclecomputing.com
twitter: @cyclecomputing
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/


_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/