WP-hpc-tools

Author	SHA1	Message	Date
Wirawan Purwanto	dfb9db6a60	* Added tool to get a detailed picture of GPU utilization on a SLURM cluster. Tested on Wahab cluster.	2 years ago
Wirawan Purwanto	3aa1688f8e	* Added basic README.	2 years ago
Wirawan Purwanto	4af174ea34	* Containers: Added simplistic tool to dump info about Python inside the container.	2 years ago
Wirawan Purwanto	ce695cd672	* Created initial tool to scan user dirs for the size of their trash folders.	2 years ago
Wirawan Purwanto	c4601a5a30	* (WIP) Minor improvement + documentation.	2 years ago
Wirawan Purwanto	5edd528511	* Saved a sample revised ipython module loader, to be used with "fix3" version of lmod_python_fix.py. Sigs: # -rw------- 1 wpurwant users 291 2020-06-09 20:09 000-odurc-lmod.py.rev3 # 70e84e7eaaf28da297fa342c8318fe2b 000-odurc-lmod.py.rev3	3 years ago
Wirawan Purwanto	4a1b5d0a69	* Saved developmental notebooks to devise/test the "lmod_python_fix.py". These were done on Wahab with legacy (lmod-based) Python suite. Sigs: # -rw-r--r-- 1 wpurwant users 56588 2020-06-09 17:35 debug_ipython_start_1.ipynb # -rw-r--r-- 1 wpurwant users 20395 2020-06-09 17:34 debug_ipython_start_2.ipynb # -rw-r--r-- 1 wpurwant users 21965 2020-06-09 19:20 debug_ipython_start_3.ipynb # -rw-r--r-- 1 wpurwant users 10717 2020-06-09 19:23 debug_ipython_start_4_vanilla.ipynb # -rw-r--r-- 1 wpurwant users 37041 2020-06-09 19:59 debug_ipython_start_5.ipynb # b061fbc819806bc18d48287a9654bcc5 debug_ipython_start_1.ipynb # bc262fc0d3cd290da1f46a242a6fcccb debug_ipython_start_2.ipynb # 5f289c909e967096712b3196a19979b7 debug_ipython_start_3.ipynb # b5f0cd9251e325b0c7e0222ba4cfc037 debug_ipython_start_4_vanilla.ipynb # abc440a885da0872d5bc215adfc3edd7 debug_ipython_start_5.ipynb	3 years ago
Wirawan Purwanto	0ad73f50d6	* Saved README.txt of the lmod_python fix (notes written to Minhao, 2020-06-09).	3 years ago
Wirawan Purwanto	d159d57578	* Third attempt to fix lmod "module" for ipython ("fix3") (dated: 2020-06-09). * Supports both Turing and Wahab. * Allows both addition and deletion of paths from sys.path. * Author's note: Later on, I discovered that these complex steps do not appear to be necessary, somehow the lmod Python commands were able to take care of additions and deletions through the modification of os.environ['PYTHONPATH'] alone, so I am not sure if this fix is absolutely needed. (?) Sigs: # -rw------- 1 wpurwant users 3298 2020-06-09 20:07 lmod_python_fix3.py # 78abc7bae90bf23a42aa40285175ca48 lmod_python_fix3.py	3 years ago
Wirawan Purwanto	232cc5cafb	* Second attempt to fix lmod "module" for ipython ("fix2") (dated: 2020-06-09). WIP: Prevent indefinite lengthening of sys.path but not complete fix yet. Sigs: # -rw------- 1 wpurwant users 1290 2020-06-09 19:28 lmod_python_fix2.py # e218a4ac73762e0380c4bbe2290906b1 lmod_python_fix2.py	3 years ago
Wirawan Purwanto	5c0b516026	* First attempt to fix lmod "module" for ipython ("fix1") (dated: 2020-06-09) Sigs: # -rw------- 1 wpurwant users 915 2020-06-09 17:20 lmod_python_fix1.py # bb42b2bde01f2e235eb205fe555fab9e lmod_python_fix1.py	3 years ago
Wirawan Purwanto	7c02045fa1	* Saved the original lmod_python.py and ~/.ipython/profile_default/startup/000-odurc-lmod.py from Wahab cluster on the "legacy" lmod-based ipython/jupyter session. Last update: 2020-06-09. Sigs: # -rw-r--r-- 1 wpurwant users 179 2020-06-03 15:49 000-odurc-lmod.py # -rw-r--r-- 1 wpurwant users 843 2020-03-17 14:02 lmod_python.py # 8718a2d7a7d6d152701e6c31254f7168 000-odurc-lmod.py # 24a740e83656875bd39f942529378414 lmod_python.py	3 years ago
Wirawan Purwanto	9edb1e040d	* Imported tools to dump information about files in a container. The "dump-info" tool works only for Debian-based distro for now.	3 years ago
Wirawan Purwanto	e828a16a49	* Expanded support to Wahab. Last updated 2021-03-02.	3 years ago
Wirawan Purwanto	cc95430e06	* Added lmod module for OPENIB-based nwchem installation on Wahab.	4 years ago
Wirawan Purwanto	f9d350aa7c	* Added make-based install procedure for nwchem 7.0.0.	4 years ago
Wirawan Purwanto	9e7c5b3312	* Modularize & tidied up the nwbuild05 script.	4 years ago
Wirawan Purwanto	b3f1877f73	* Initial version of nwchem "build05": Using "OPENIB" backend.	4 years ago
Wirawan Purwanto	67c05e7efd	* Added small build notes for nwchem on Wahab.	4 years ago
Wirawan Purwanto	08dfff3ce4	* Modifications for building armci-mpi on the Wahab's install location.	4 years ago
Wirawan Purwanto	8cf081b4b1	* Copied armci-mpi build script from ODU container repo. This is taken from commit #dd7cf94ecdb1c1368fed2458c4f45bd95e1a140c from ODU container repo. No changes were made at all.	4 years ago
Wirawan Purwanto	b97592a8c7	* Copied "build04" nwchem build script from ODU container repo. This is taken from commit #a278c9cb246fc267f93018e607856c1141942d32 from ODU container repo. No changes were made at all.	4 years ago
Wirawan Purwanto	d68c1cc608	* Wahab: Imported comsol-51, last modified 2021-02-11 to accommodate launching from another SLURM job (e.g. OOD virtual desktop interface).	4 years ago
Wirawan Purwanto	0331ea6ed3	* Committed: Original user-invocable "qe-<VERSION>" from Wahab. Signatures: # -rwxr-xr-x 1 wpurwant users 520 2020-03-10 14:39:05 qe-6.3 # -rwxr-xr-x 1 wpurwant users 514 2020-03-10 14:29:54 qe-6.4 # -rwxr-xr-x 1 wpurwant users 531 2020-03-10 14:28:29 qe-6.4-intel # bda38c3efc9bb0f83db351924e06da43 qe-6.3 # 2b2e9ec9c641dd60206776a5a80c75a3 qe-6.4 # 083455403fa47897c4d6e7ce6d591f9a qe-6.4-intel # 8d29ad9caa756b55747e58a06c39aa59576a896b qe-6.3 # 1443cdb47e4340be1106d4e80ba3dc47717d9ca1 qe-6.4 # 1bf39ffc828cdc784658ea510fbda0602d34e237 qe-6.4-intel	4 years ago
Wirawan Purwanto	e9597af157	* Minor typographical improvements.	4 years ago
Wirawan Purwanto	dda0e1b7ad	* ARCHIVE: Imported some tools from my stats work back in 2016. Warning: These tools are obsolete, and may or may not work. Orig-cwd: TURING:/home/wpurwant/hpc-explore/turing-hw/tests20160826 Sigs: # -rwx------ 1 wpurwant users 214 2016-09-13 11:58:21 show-disabled-nodes2.awk # -rwx------ 1 wpurwant users 111 2016-08-26 13:10:14 show-disabled-nodes.sh # -rwx------ 1 wpurwant users 122 2016-08-26 10:25:16 show-node-summary.sh # -rwx------ 1 wpurwant users 153 2016-08-26 12:06:25 show-non-full-nodes.sh # 71c64a7e6cd9df34ca6a9348ebc44ab1 show-disabled-nodes.sh # 8a3570829d132c526cc44adf72212887 show-disabled-nodes2.awk # ebbc0642cd29a1575670c86629d89c72 show-node-summary.sh # 5eee777a8380333654d3a06e3482f38e show-non-full-nodes.sh # dfb36573c6d758bc675f70247ff0befe9ddbe365 show-disabled-nodes.sh # d1995fc91d402c9dcc30adc0b461e2beaa25ab67 show-disabled-nodes2.awk # 3267d169b476396d66f6127e1343bd7c23b37e1f show-node-summary.sh # 18673a78914d9cce47c95d4fd5146805fc227416 show-non-full-nodes.sh	4 years ago
Wirawan Purwanto	9429ad3697	* Added capability to parse & create scratch dirs.	4 years ago
Wirawan Purwanto	7dc03821a8	* g09slurm initial update: correctly parse nprocshared / nproc in link0 command in a case-insensitive manner.	4 years ago
Min Dong	19eacfbf54	* Committed: Original user-invocable "g09slurm" from Wahab and the "g09.slurm" backend script. Signatures: # -rwxr-xr-x 1 wpurwant users 394 2020-09-10 01:06:20 g09.slurm # -rwxr-xr-x 1 wpurwant users 1120 2019-09-16 15:24:20 g09slurm # 387f828b04454678b23b05af61e4f334 g09.slurm # 9ee4bfcfad766165a81239cb13d7ef30 g09slurm # e0dbc8b24aeecbbecf7255a3e3f84dc3fc3a422f g09.slurm # d5a33e554766c41f17f6839242c5c7f4d60e7fe4 g09slurm g09.slurm is originally located at /cm/shared/apps/gaussian/g09revD.01/script/g09/script/g09.slurm	4 years ago
Wirawan Purwanto	19c833c3ff	* sinfo-report-node-stats.sh: Simple tool to report status of compute nodes based on SLURM's "sinfo" output.	4 years ago
Wirawan Purwanto	18d79dd34b	* On custom sacct1.sh, include job (most recent) state as well.	5 years ago
Wirawan Purwanto	d384d0320d	* Added explicitly headless option (--headless). * Increased wait time to 5 minutes.	5 years ago
Wirawan Purwanto	c2a5ae8863	* slurm: Modified fields to print by default.	5 years ago
Wirawan Purwanto	32b82db7a3	* slurm: Added custom sacct wrapper script which contains my preferences.	5 years ago
Wirawan Purwanto	db2ca075ed	* Must re-add SLURM if it is not loaded.	6 years ago
Wirawan Purwanto	a65338a8bf	* Added "wgo" (what's going on) to check the status of processes on login node. Currently supposed to be used for Turing only.	6 years ago
Wirawan Purwanto	1387997010	* Prints an error message and quit in case Jupyter did not start after 2 minutes.	6 years ago
Wirawan Purwanto	ba6c9f53ed	* Added accommodation for Anaconda as well as site-provided python. * Added support for fully headless mode (connect via local browser). * Added some safeguards against job failing to start due to executable not found, etc.	6 years ago
Wirawan Purwanto	1c8a5da492	* Imported launch_jupyter from Turing. Originally furnished by Min Dong, 2019-03-01 14:30 EST.	6 years ago
Wirawan Purwanto	28eb7a0d98	* sq: Customizable squeue wrapper: introduce new defaults / default behavior on squeue.	6 years ago
Wirawan Purwanto	94e0aa9490	* jupyter-anaconda2: A script that will start Jupyter notebook process for Anaconda2 distribution. Note: for now it is a script that has to be submitted in the compute node. I will upgrade this to become a self-submitting script eventually.	6 years ago
Wirawan Purwanto	95034685ff	* interact: Tool to allocate an interactive session on a regular compute node under SLURM.	7 years ago
Wirawan Purwanto	dbce662c5a	* interact-gpu: Tool to allocate an interactive session on a GPU compute node under SLURM.	7 years ago
Wirawan Purwanto	82ea3bc689	* Archived: SGE version of pwscf-5.3 script.	8 years ago
Wirawan Purwanto	9c82c4d465	* Update bash module support: with recent changes on Turing, 'module' seems to be supported out of the box for bash. If 'module' environment is detected, we skip the initiation step.	8 years ago
Wirawan Purwanto	68c0e70d4d	* Added "regular" runsas which runs with more limited memory.	8 years ago
Wirawan Purwanto	27b8ccd6ae	* Added runsas-himem from earlier consultation this year.	8 years ago
Wirawan Purwanto	df6facce86	* pwscf: Ad-hoc fix for Turing after 2016 upgrade. We force using the old (TCL) module system since the new module system (LMOD) always executes itself whenever a bash batch script is executed on Turing right now.	8 years ago
Wirawan Purwanto	739d765f53	* Added convenience for gathering & analyzing CPUs on the cluster. * Documentation update.	8 years ago
Wirawan Purwanto	aa597b907c	* In hoststats subcommand: Also print node status flags if they exist.	8 years ago

1 2

75 Commits (master) All Branches Search

75 Commits (master)

All Branches