You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
54 lines
2.4 KiB
54 lines
2.4 KiB
11 months ago
|
Job Scheduler for Windows
|
||
|
=========================
|
||
|
|
||
|
|
||
|
Original question
|
||
|
-----------------
|
||
|
|
||
|
From: Deb McCaffrey <dmccaff4@asu.edu><br/>
|
||
|
Sent: Monday, December 4, 2023 3:25 PM<br/>
|
||
|
To: people-network@carcc.org<br/>
|
||
|
Subject: [CaRCC people] Scheduling on Windows?
|
||
|
|
||
|
Does anyone know of a way to do job scheduling on Windows? We have some researchers dealing with resource contention on their Windows server. They're just manually communicating their jobs right now.
|
||
|
|
||
|
|
||
|
## Several options:
|
||
|
|
||
|
### 1: HTCondor
|
||
|
|
||
|
"HTCondor offers Windows functionality. You can find more information here: https://htcondor.readthedocs.io/en/latest/platform-specific/microsoft-windows.html" (Carrie Brown)
|
||
|
|
||
|
"If you've got any questions about HTCondor on Windows, feel free to reach out to htcondor-users@cs.wisc.edu. We'd love to hear feedback!" (Brian Lin, OSG)
|
||
|
|
||
|
|
||
|
### 2: HyperShell
|
||
|
|
||
|
"Actually we have something that may work.
|
||
|
At Purdue we've developed a workflow automation tool for high-throughput computing called HyperShell (https://hyper-shell.readthedocs.io, https://github.com/glentner/hyper-shell).
|
||
|
It's been in use for a few years now, is cross-platform, and has a few different modes of use.
|
||
|
To use it as a persistent scheduler, it can have a local SQLite database file on the Windows server, or a remote Postgres instance, and it will schedule tasks on all connect executors.
|
||
|
It is not a resource manager though, however many concurrent tasks you say can run - will, so the workload itself needs to be told not to use more than you intend.
|
||
|
I've used this to orchestrate large numbers of tasks over dozens of Windows Server instances."
|
||
|
(Geoffrey R Lentner <glentner@purdue.edu>)
|
||
|
|
||
|
|
||
|
|
||
|
### Other places to look at:
|
||
|
|
||
|
A handy list of job scheduling software, and some such as Apache Airflow, are open source and run on Windows.
|
||
|
|
||
|
https://en.wikipedia.org/wiki/List_of_job_scheduler_software
|
||
|
|
||
|
It remains to be seen whether any other them have the features your researchers need. (David Paul Reddy)
|
||
|
|
||
|
|
||
|
Considerations for Windows jobs
|
||
|
-------------------------------
|
||
|
|
||
|
"...it's good to get a sense of what sort of jobs researchers running on this Windows server.
|
||
|
|
||
|
* Do they involve a GUI interface and interactive sessions?
|
||
|
* Are they pleasantly parallel tasks with low core requirements or bigger multi-core OpenMP or MPI ones?
|
||
|
* How long do the jobs normally last and how many is each individual trying to run?" (Matthew T. West, Research IT, University of Exeter)
|