* (WIP) Dumped a Q/A draft from my personal scrap notes.

Raw text, not laid out yet.

python/20210705.run-python-jupyter.md

@@ -0,0 +1,33 @@
+# Question: I have Python code that I would like to run on HPC. Can I run it through the Jupyter notebook? If so, how can I?
+
+<!-- was taken from Wirawan's WORK SCRAP NOTES -->
+
+Yes, but we recommend that you launch the script through the HPC job scheduler, called “SLURM”. Let’s call the script you want to run “SCRI
+PT.py” (include the full path if the script is not the same directory as where you want to run this). Here is a basic SLURM job script that may work for you:
+#!/bin/bash
+#SBATCH -p gpu
+#SBATCH --gres gpu:1
+enable_lmod
+module load container_env tensorflow-gpu/2.4.0
+crun.tensorflow-gpu python3 SCRIPT.py
+Important notes:
+Replace SCRIPT.py with your actual script file name.
+We assume that your workload needs a GPU to run your calculation. If you do not, please remove the #SBATCH lines and replace the words “tensorflow-gpu” with “tensorflow-cpu” is everywhere.
+Create this script using a text editor, save it to a file (say, name this JOB.sh). Then you will submit the script from the UNIX shell interface of the cluster by typing:
+$ sbatch JOB.sh
+(The “$” at the beginning of the line represents the shell prompt--do not type that.) If successful, there will be a message printed “Submitted batch job NNNNNN”, where NNNNNN is an integer called job ID or job number. The output file will be “slurm-NNNNNN.out”.
+Here are a few documentations to help understand this process:
+Documentation on Python on ODU HPC: https://wiki.hpc.odu.edu/Software/Python
+General workflow of using SLURM job scheduler: https://wiki.hpc.odu.edu/slurm#general-workflow-of-using-slurm
+If you prefer video introduction to SLURM, please take a look at this short video to understand what job scheduler is:
+“Working with SLURM”
+https://odumedia.mediaspace.kaltura.com/playlist/dedicated/1_8eqsb16m/1_oy7ls1o0
+-----
+Now, back to your original question: Yes, you can run the script from a notebook. Once you opened a (blank) notebook, you will make this statement in a new cell:
+%load SCRIPT.py
+Press <Shift+Enter> once, the code will be loaded into that same cell. Press <Shift+Enter> once more, the code will execute. But please note that this mode of execution can be awkward, i.e. we have to open a Jupyter session, %load the script in order to run it, and even after that, the maximum time to run the script is under 24 hours. This mode is useful if you plan to interactively test or troubleshoot a script, but less useful when you have a bunch of calculations to do, or calculation that you expect to run for a long time.
+As a closing, I’d like to ask this: Can you share the script with us so I may understand the requirements of your script?
+
+
+The entire training can be found here:
+https://wiki.hpc.odu.edu/Training/HPC-Intro