[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Tracking resource utilization in condor_submit -interactive



Yeah, I've been using cgroups since the early days, and I think I have at least one or two cgroup-related bugs in the database. ;)

So an example here is that one of our users is running a Xilinx Vivado session to do some interactive simulation and debugging of an FPGA design.  The Vivado executable in his interactive session shows 7:52 worth of CPU time in the ps output column, and there's maybe one extra minute of  time in the adjunct processes.  The "top" output shows the Vivado session at 1.624 gigabytes of resident memory, and a 7.1GB virtual image size.

However:

[pelletm ~]$ condor_q 4558 -af '(remotesyscpu + remoteusercpu)' memoryusage
0.0 undefined
[pelletm  ~]$

Do I maybe have the cgroups configured incorrectly for CentOS/RHEL 7? My BASE_CGROUP config is set to htcondor, which dates back to version 8.0 or whenever that was introduced. Do I need to change that now that cgroups are woven into the kernel and systemd? Maybe the hierarchy is not getting picked up properly?

Thanks!

	-Michael Pelletier.

-----Original Message-----
From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Greg Thain
Sent: Tuesday, February 13, 2018 4:28 PM
To: htcondor-users@xxxxxxxxxxx
Subject: [External] Re: [HTCondor-users] Tracking resource utilization in condor_submit -interactive

On 02/05/2018 12:24 PM, Michael Pelletier wrote:
> We've got a few folks who need interactive shells due to the nature of 
> their workflow, and I've set up appropriate submit descriptions for 
> use with condor_submit -interactive. However, the job attributes such 
> as MemoryUsage and RemoteUserCpu don't track the resource utilization 
> by the condor_ssh_to_job sessions launched by condor_submit (as would 
> be desired under any other circumstance),

Do you have cgroups enabled? With cgroups, the ssh session is placed in a child cgroup of the main job, so the usage should show up in the sleep job.

-greg
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/