wirawan0/WP-hpc-tools
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Commit Graph

  • dfb9db6a60 * Added tool to get a detailed picture of GPU utilization on a SLURM cluster. master Wirawan Purwanto 2023-04-06 16:25:04 -04:00
  • 3aa1688f8e * Added basic README. Wirawan Purwanto 2023-03-01 10:39:12 -05:00
  • 4af174ea34 * Containers: Added simplistic tool to dump info about Python inside the container. Wirawan Purwanto 2023-03-01 09:35:41 -05:00
  • ce695cd672 * Created initial tool to scan user dirs for the size of their trash folders. Wirawan Purwanto 2022-10-03 15:35:30 -04:00
  • c4601a5a30 * (WIP) Minor improvement + documentation. Wirawan Purwanto 2022-09-30 17:22:19 -04:00
  • 5edd528511 * Saved a sample revised ipython module loader, to be used with "fix3" version of lmod_python_fix.py. Wirawan Purwanto 2022-02-22 11:23:54 -05:00
  • 4a1b5d0a69 * Saved developmental notebooks to devise/test the "lmod_python_fix.py". These were done on Wahab with legacy (lmod-based) Python suite. Wirawan Purwanto 2022-02-22 10:44:05 -05:00
  • 0ad73f50d6 * Saved README.txt of the lmod_python fix (notes written to Minhao, 2020-06-09). Wirawan Purwanto 2022-02-22 10:41:11 -05:00
  • d159d57578 * Third attempt to fix lmod "module" for ipython ("fix3") (dated: 2020-06-09). * Supports both Turing and Wahab. * Allows both addition and deletion of paths from sys.path. Wirawan Purwanto 2022-02-22 10:28:14 -05:00
  • 232cc5cafb * Second attempt to fix lmod "module" for ipython ("fix2") (dated: 2020-06-09). WIP: Prevent indefinite lengthening of sys.path but not complete fix yet. Wirawan Purwanto 2022-02-22 10:22:45 -05:00
  • 5c0b516026 * First attempt to fix lmod "module" for ipython ("fix1") (dated: 2020-06-09) Wirawan Purwanto 2022-02-22 10:15:43 -05:00
  • 7c02045fa1 * Saved the original lmod_python.py and ~/.ipython/profile_default/startup/000-odurc-lmod.py from Wahab cluster on the "legacy" lmod-based ipython/jupyter session. Last update: 2020-06-09. Wirawan Purwanto 2022-02-22 09:55:55 -05:00
  • 9edb1e040d * Imported tools to dump information about files in a container. The "dump-info" tool works only for Debian-based distro for now. Wirawan Purwanto 2021-06-29 13:29:56 -04:00
  • e828a16a49 * Expanded support to Wahab. Wirawan Purwanto 2021-06-29 13:25:15 -04:00
  • cc95430e06 * Added lmod module for OPENIB-based nwchem installation on Wahab. Wirawan Purwanto 2021-04-20 00:29:57 -04:00
  • f9d350aa7c * Added make-based install procedure for nwchem 7.0.0. Wirawan Purwanto 2021-04-20 00:29:27 -04:00
  • 9e7c5b3312 * Modularize & tidied up the nwbuild05 script. Wirawan Purwanto 2021-04-20 00:28:39 -04:00
  • b3f1877f73 * Initial version of nwchem "build05": Using "OPENIB" backend. Wirawan Purwanto 2021-04-15 12:41:17 -04:00
  • 67c05e7efd * Added small build notes for nwchem on Wahab. Wirawan Purwanto 2021-04-15 12:40:55 -04:00
  • 08dfff3ce4 * Modifications for building armci-mpi on the Wahab's install location. Wirawan Purwanto 2021-04-15 12:40:01 -04:00
  • 8cf081b4b1 * Copied armci-mpi build script from ODU container repo. Wirawan Purwanto 2021-04-14 12:01:31 -04:00
  • b97592a8c7 * Copied "build04" nwchem build script from ODU container repo. Wirawan Purwanto 2021-04-14 11:36:44 -04:00
  • d68c1cc608 * Wahab: Imported comsol-51, last modified 2021-02-11 to accommodate launching from another SLURM job (e.g. OOD virtual desktop interface). Wirawan Purwanto 2021-02-12 15:16:43 -05:00
  • 0331ea6ed3 * Committed: Original user-invocable "qe-<VERSION>" from Wahab. Wirawan Purwanto 2021-01-28 09:54:38 -05:00
  • e9597af157 * Minor typographical improvements. Wirawan Purwanto 2021-01-28 09:43:50 -05:00
  • dda0e1b7ad * ARCHIVE: Imported some tools from my stats work back in 2016. Wirawan Purwanto 2021-01-26 10:59:44 -05:00
  • 9429ad3697 * Added capability to parse & create scratch dirs. Wirawan Purwanto 2020-12-09 17:37:06 -05:00
  • 7dc03821a8 * g09slurm initial update: correctly parse nprocshared / nproc in link0 command in a case-insensitive manner. Wirawan Purwanto 2020-12-03 14:57:30 -05:00
  • 19eacfbf54 * Committed: Original user-invocable "g09slurm" from Wahab and the "g09.slurm" backend script. sysadmin 2019-09-16 15:24:20 -04:00
  • 19c833c3ff * sinfo-report-node-stats.sh: Simple tool to report status of compute nodes based on SLURM's "sinfo" output. Wirawan Purwanto 2020-06-29 13:30:04 -04:00
  • 18d79dd34b * On custom sacct1.sh, include job (most recent) state as well. Wirawan Purwanto 2020-06-21 17:45:23 -04:00
  • d384d0320d * Added explicitly headless option (--headless). * Increased wait time to 5 minutes. Wirawan Purwanto 2020-01-17 13:58:59 -05:00
  • c2a5ae8863 * slurm: Modified fields to print by default. Wirawan Purwanto 2019-11-25 11:12:25 -05:00
  • 32b82db7a3 * slurm: Added custom sacct wrapper script which contains my preferences. Wirawan Purwanto 2019-07-19 14:58:47 -04:00
  • db2ca075ed * Must re-add SLURM if it is not loaded. Wirawan Purwanto 2019-04-25 17:35:34 -04:00
  • a65338a8bf * Added "wgo" (what's going on) to check the status of processes on login node. Currently supposed to be used for Turing only. Wirawan Purwanto 2019-04-25 17:34:23 -04:00
  • 1387997010 * Prints an error message and quit in case Jupyter did not start after 2 minutes. Wirawan Purwanto 2019-03-27 13:49:19 -04:00
  • ba6c9f53ed * Added accommodation for Anaconda as well as site-provided python. * Added support for fully headless mode (connect via local browser). * Added some safeguards against job failing to start due to executable not found, etc. Wirawan Purwanto 2019-03-27 13:28:33 -04:00
  • 1c8a5da492 * Imported launch_jupyter from Turing. Wirawan Purwanto 2019-03-27 11:24:00 -04:00
  • 28eb7a0d98 * sq: Customizable squeue wrapper: introduce new defaults / default behavior on squeue. Wirawan Purwanto 2018-07-20 11:20:22 -04:00
  • 94e0aa9490 * jupyter-anaconda2: A script that will start Jupyter notebook process for Anaconda2 distribution. Wirawan Purwanto 2018-07-20 09:19:59 -04:00
  • 95034685ff * interact: Tool to allocate an interactive session on a regular compute node under SLURM. Wirawan Purwanto 2018-06-01 12:50:56 -04:00
  • dbce662c5a * interact-gpu: Tool to allocate an interactive session on a GPU compute node under SLURM. Wirawan Purwanto 2018-06-01 12:45:42 -04:00
  • 82ea3bc689 * Archived: SGE version of pwscf-5.3 script. Wirawan Purwanto 2017-05-24 14:37:32 -04:00
  • 9c82c4d465 * Update bash module support: with recent changes on Turing, 'module' seems to be supported out of the box for bash. If 'module' environment is detected, we skip the initiation step. Wirawan Purwanto 2017-05-24 14:28:24 -04:00
  • 68c0e70d4d * Added "regular" runsas which runs with more limited memory. Wirawan Purwanto 2016-11-10 12:48:43 -05:00
  • 27b8ccd6ae * Added runsas-himem from earlier consultation this year. Wirawan Purwanto 2016-11-10 12:45:29 -05:00
  • df6facce86 * pwscf: Ad-hoc fix for Turing after 2016 upgrade. Wirawan Purwanto 2016-11-07 11:57:34 -05:00
  • 739d765f53 * Added convenience for gathering & analyzing CPUs on the cluster. * Documentation update. Wirawan Purwanto 2016-10-31 15:21:10 -04:00
  • aa597b907c * In hoststats subcommand: Also print node status flags if they exist. Wirawan Purwanto 2016-10-20 10:11:18 -04:00
  • 999fe5f571 * Add more info to gather. Wirawan Purwanto 2016-10-20 10:10:31 -04:00
  • cabacb58cb * Also added dump for mount points and disk free for compute nodes. Wirawan Purwanto 2016-09-26 13:04:30 -04:00
  • b6d22cf68b * Added "hoststats" subcommand for summarizing host occupancy statistics irrespective of queue. Wirawan Purwanto 2016-09-20 17:47:24 -04:00
  • ebdc93e80f * Also collect dmesg snapshot. Wirawan Purwanto 2016-09-20 17:46:50 -04:00
  • 879927f16e * Added python workbench hpl_timing, for estimating/analyzing HPL timing. Wirawan Purwanto 2016-09-20 17:45:14 -04:00
  • d6d71364de * Imported initial tools for extracting HPL benchmark results. Wirawan Purwanto 2016-09-20 17:44:39 -04:00
  • 483c6874c0 * Use getopt to handle command-line option. * Include a help command. Wirawan Purwanto 2016-09-14 13:39:55 -04:00
  • 67bc899f4a * Fixes for unhandled/unrecognized command options. * Documentation update. * Added help command. Wirawan Purwanto 2016-09-14 13:38:29 -04:00
  • 850bd34377 * Documentation update. Wirawan Purwanto 2016-09-14 10:29:46 -04:00
  • e382a5eb35 * Minor fix to strip domain name (can be truncated). Wirawan Purwanto 2016-09-14 10:27:45 -04:00
  • f06803ba6c * show-node-status.py: A toolbox to analyze node status returned by SGE. Wirawan Purwanto 2016-09-14 10:16:35 -04:00
  • acfb11e010 * Initial form of documentation. Wirawan Purwanto 2016-09-09 16:50:40 -04:00
  • 04515dcd35 * Allow external qstat-f file for raw node status dump. Wirawan Purwanto 2016-09-09 16:41:15 -04:00
  • 7957b28a05 * show-node-status.py: Initial tool to replace node-slot-status.sh. Wirawan Purwanto 2016-09-09 16:39:28 -04:00
  • 34a7659f3d * show-cluster-usage.py: A tool to summarize the usage of an SGE cluster at a given snapshot in time. Wirawan Purwanto 2016-08-29 19:16:11 -04:00
  • 4f28615bf0 * Added variants of node status to display (still work in progress). Wirawan Purwanto 2016-08-29 13:08:10 -04:00
  • a0ad7c25bc * Added analysis tool to summarize CPUs or group compute nodes based on their CPUs. Wirawan Purwanto 2016-08-29 13:04:25 -04:00
  • 52619c3688 * Added tools to dump compute node info in batch. Wirawan Purwanto 2016-08-26 15:09:36 -04:00
  • f1327c9562 * SGE: Added qconf dump tools. Wirawan Purwanto 2016-08-26 10:00:46 -04:00
  • 79e5b77df2 * bash-module-env.sh: Update was required due to incomplete pre-existing MODULEPATH in some of Turing's compute nodes. Wirawan Purwanto 2016-08-23 13:55:51 -04:00
  • 6f0880c547 * bash-module-env.sh: an effort to facilitate bash support for batch scripts. Wirawan Purwanto 2016-08-23 13:45:21 -04:00
  • bf43a3b0b5 * sge-dump-job-status.sh: Initial version of a tool to dump desirable SGE info from a running job. Wirawan Purwanto 2016-08-08 10:00:32 -04:00
  • 8b99995409 * Added find-run-hosts.sh: swiss-army tool to find hosts where a job run, dump the process trees, etc. Wirawan Purwanto 2016-07-14 00:24:48 -04:00
  • 8ae0841ca6 * pwscf: Initial version of self-submitting script to launch pwscf calculation (version 5.3). Wirawan Purwanto 2016-07-08 23:33:51 -04:00
  • 7f83f897c8 * sge: Added node-slot-status.sh to aggregate slot availability per node type. Wirawan Purwanto 2015-10-28 11:35:49 -04:00