Gathered from CaRCC discussion.master
							parent
							
								
									fa6d73a842
								
							
						
					
					
						commit
						1d9a91f698
					
				
				 1 changed files with 53 additions and 0 deletions
			
			
		| @ -0,0 +1,53 @@ | |||||||
|  | Job Scheduler for Windows | ||||||
|  | ========================= | ||||||
|  | 
 | ||||||
|  | 
 | ||||||
|  | Original question | ||||||
|  | ----------------- | ||||||
|  | 
 | ||||||
|  | From: Deb McCaffrey <dmccaff4@asu.edu><br/> | ||||||
|  | Sent: Monday, December 4, 2023 3:25 PM<br/> | ||||||
|  | To: people-network@carcc.org<br/> | ||||||
|  | Subject: [CaRCC people] Scheduling on Windows? | ||||||
|  | 
 | ||||||
|  | Does anyone know of a way to do job scheduling on Windows? We have some researchers dealing with resource contention on their Windows server. They're just manually communicating their jobs right now. | ||||||
|  | 
 | ||||||
|  | 
 | ||||||
|  | ## Several options: | ||||||
|  | 
 | ||||||
|  | ### 1: HTCondor | ||||||
|  | 
 | ||||||
|  | "HTCondor offers Windows functionality. You can find more information here: https://htcondor.readthedocs.io/en/latest/platform-specific/microsoft-windows.html"  (Carrie Brown) | ||||||
|  | 
 | ||||||
|  | "If you've got any questions about HTCondor on Windows, feel free to reach out to htcondor-users@cs.wisc.edu. We'd love to hear feedback!"  (Brian Lin, OSG) | ||||||
|  | 
 | ||||||
|  | 
 | ||||||
|  | ### 2: HyperShell | ||||||
|  | 
 | ||||||
|  | "Actually we have something that may work. | ||||||
|  | At Purdue we've developed a workflow automation tool for high-throughput computing called HyperShell (https://hyper-shell.readthedocs.io, https://github.com/glentner/hyper-shell). | ||||||
|  | It's been in use for a few years now, is cross-platform, and has a few different modes of use. | ||||||
|  | To use it as a persistent scheduler, it can have a local SQLite database file on the Windows server, or a remote Postgres instance, and it will schedule tasks on all connect executors. | ||||||
|  | It is not a resource manager though, however many concurrent tasks you say can run - will, so the workload itself needs to be told not to use more than you intend. | ||||||
|  | I've used this to orchestrate large numbers of tasks over dozens of Windows Server instances." | ||||||
|  | (Geoffrey R Lentner <glentner@purdue.edu>) | ||||||
|  | 
 | ||||||
|  | 
 | ||||||
|  | 
 | ||||||
|  | ### Other places to look at: | ||||||
|  | 
 | ||||||
|  | A handy list of job scheduling software, and some such as Apache Airflow, are open source and run on Windows. | ||||||
|  | 
 | ||||||
|  | https://en.wikipedia.org/wiki/List_of_job_scheduler_software | ||||||
|  | 
 | ||||||
|  | It remains to be seen whether any other them have the features your researchers need. (David Paul Reddy) | ||||||
|  | 
 | ||||||
|  | 
 | ||||||
|  | Considerations for Windows jobs | ||||||
|  | ------------------------------- | ||||||
|  | 
 | ||||||
|  | "...it's good to get a sense of what sort of jobs researchers running on this Windows server. | ||||||
|  | 
 | ||||||
|  |   * Do they involve a GUI interface and interactive sessions? | ||||||
|  |   * Are they pleasantly parallel tasks with low core requirements or bigger multi-core OpenMP or MPI ones? | ||||||
|  |   * How long do the jobs normally last and how many is each individual trying to run?"  (Matthew T. West, Research IT, University of Exeter) | ||||||
					Loading…
					
					
				
		Reference in new issue