[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Question about POST scripts in a DAG workflow



Hi Nathan,

The syntax you're using to feed arguments into the post script is
correct. Based on the log information you sent, it looks like your OS
is unable to execute the script.py file you've defined. Have you got
the necessary shebang line (ie. #!/usr/bin/env python) at the top of
the file? Does script.py have execute permissions?

POST scripts are run locally on your submit machine. If you try
invoking script.py directly from the command line with some input
arguments, this should reveal the problem fairly quickly.

Mark



On Thu, Dec 3, 2020 at 6:15 PM <email.nsimpson@xxxxxxxxx> wrote:
>
> Hi all,
>
> I have a nested DAG setup as follows:
>
> SUBDAG EXTERNAL dag1 dag1.dag
> SCRIPT POST dag1 script.py arg1 arg2 arg3
> â (repeats with other dags)
>
> where arg1-3 are a set of command-line args I feed to script.py.
>
> I wrote script.py to scrape the dag log files from dag1 and send an automated email. This file runs fine locally, and even is fed the absolute path to the log files as an argument to make sure the script can find them. However, it doesnât work in practice, and I canât see the output or errors from the POST script anywhere in the dagman log files.
>
> My questions are:
> - Is this the right way to feed args to a POST script?
> - Are POST scripts run locally (submit machine) or on a node?
>
> And also, my current dagman out is below for convenience, maybe this has an explanation?
>
> 12/04/20 00:38:41 Running POST script of Node dag1...
> 12/04/20 00:38:41 Warning: mysin has length 0 (ignore if produced by DAGMan; see gittrac #4987, #5031)
> 12/04/20 00:38:41 Create_Process(script.py): child failed with errno 8 (Exec format error) before exec()
> 12/04/20 00:38:41   error: daemonCore->Create_Process() failed; POST script of Job dag1 failed
> 12/04/20 00:38:41 Initializing user log writer for ./dag.dag.nodes.log, (1064990.0.0)
> 12/04/20 00:38:41 Event: ULOG_POST_SCRIPT_TERMINATED for HTCondor Node iteration00 (1064990.0.0) {12/04/20 00:38:41}
> 12/04/20 00:38:41 POST Script of node dag1 failed with status 1
> 12/04/20 00:38:41 POST for Node dag1 returned 1
>
> Best, Nathan
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/



--
Mark Coatsworth
Systems Programmer
Center for High Throughput Computing
Department of Computer Sciences
University of Wisconsin-Madison