wirawan0/WP-hpc-tools
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
Files
5edd528511e295aceacdd27d7a415bdc0d6988b3
WP-hpc-tools/lmod/ipython/startup/testbeds/debug_ipython_start_3.ipynb
T
Wirawan Purwanto 4a1b5d0a69 * Saved developmental notebooks to devise/test the "lmod_python_fix.py". 
These were done on Wahab with legacy (lmod-based) Python suite.

  Sigs:
    # -rw-r--r-- 1 wpurwant users 56588 2020-06-09 17:35 debug_ipython_start_1.ipynb
    # -rw-r--r-- 1 wpurwant users 20395 2020-06-09 17:34 debug_ipython_start_2.ipynb
    # -rw-r--r-- 1 wpurwant users 21965 2020-06-09 19:20 debug_ipython_start_3.ipynb
    # -rw-r--r-- 1 wpurwant users 10717 2020-06-09 19:23 debug_ipython_start_4_vanilla.ipynb
    # -rw-r--r-- 1 wpurwant users 37041 2020-06-09 19:59 debug_ipython_start_5.ipynb
    # b061fbc819806bc18d48287a9654bcc5  debug_ipython_start_1.ipynb
    # bc262fc0d3cd290da1f46a242a6fcccb  debug_ipython_start_2.ipynb
    # 5f289c909e967096712b3196a19979b7  debug_ipython_start_3.ipynb
    # b5f0cd9251e325b0c7e0222ba4cfc037  debug_ipython_start_4_vanilla.ipynb
    # abc440a885da0872d5bc215adfc3edd7  debug_ipython_start_5.ipynb
2022-02-22 10:44:05 -05:00

22 KiB
Raw Blame History

Examine & fix jupyter / ipython start file

Date: 2020-06-09

Purpose: Try to examine the outcome of my fix to ~/.ipython/profile_default/startup/000-odurc-lmod.py + the handmade "module" command (mine: lmod_python_fix1.py).

Test-3: restarted the entire Jupyter stuff.

In [1]:
module
Out [1]:
<function env_modules_python.module(command, *arguments)>
In [2]:
module.__code__.co_filename
Out [2]:
'/shared/apps/common/lmod/current/init/env_modules_python.py'
In [3]:
module??
In [4]:
import os

xlibpath = os.environ.get("LMOD_MANUAL_LD_LIBRARY_PATH")
xlibpath
Out [4]:
'/tmp/tmp.JtRbow9G3w'
In [6]:
import sys
sys.path
Out [6]:
['/home/wpurwant/Testbeds/jupyter/wahab_jupyter_start',
 '/shared/apps/common/lmod/scripts/init',
 '',
 '/shared/apps/auto/py-jupyter/1.1.4-gcc-7.3.0-cxdz/lib/python3.7/site-packages',
 '/shared/apps/auto/py-numpy/1.16.3-gcc-7.3.0-mdow/lib/python3.7/site-packages',
 '/shared/apps/auto/py-six/1.12.0-gcc-7.3.0-ay7a/lib/python3.7/site-packages',
 '/shared/apps/auto/py-jinja2/2.10-gcc-7.3.0-usre/lib/python3.7/site-packages',
 '/shared/apps/auto/py-babel/2.6.0-gcc-7.3.0-be4y/lib/python3.7/site-packages',
 '/shared/apps/auto/py-pytz/2018.4-gcc-7.3.0-wqdr/lib/python3.7/site-packages',
 '/shared/apps/auto/py-markupsafe/1.0-gcc-7.3.0-biyo/lib/python3.7/site-packages',
 '/shared/apps/common/python/3.7.3-wqal/lib/python37.zip',
 '/shared/apps/common/python/3.7.3-wqal/lib/python3.7',
 '/shared/apps/common/python/3.7.3-wqal/lib/python3.7/lib-dynload',
 '/home/wpurwant/.local/lib/python3.7/site-packages',
 '/shared/apps/common/python/3.7.3-wqal/lib/python3.7/site-packages',
 '/shared/apps/auto/py-jupyter/1.1.4-gcc-7.3.0-cxdz/lib/python3.7/site-packages/IPython/extensions',
 '/home/wpurwant/.ipython',
 '/home/wpurwant/Testbeds/jupyter/wahab_jupyter_start',
 '/shared/apps/common/lmod/current/init']
In [7]:
os.environ['PYTHONPATH'].split(":")
Out [7]:
['/shared/apps/common/lmod/scripts/init',
 '/shared/apps/auto/py-jupyter/1.1.4-gcc-7.3.0-cxdz/lib/python3.7/site-packages',
 '/shared/apps/auto/py-numpy/1.16.3-gcc-7.3.0-mdow/lib/python3.7/site-packages',
 '/shared/apps/auto/py-six/1.12.0-gcc-7.3.0-ay7a/lib/python3.7/site-packages',
 '/shared/apps/auto/py-jinja2/2.10-gcc-7.3.0-usre/lib/python3.7/site-packages',
 '/shared/apps/auto/py-babel/2.6.0-gcc-7.3.0-be4y/lib/python3.7/site-packages',
 '/shared/apps/auto/py-pytz/2018.4-gcc-7.3.0-wqdr/lib/python3.7/site-packages',
 '/shared/apps/auto/py-markupsafe/1.0-gcc-7.3.0-biyo/lib/python3.7/site-packages']
In [9]:
ls -l $xlibpath
total 0
In [10]:
module("list")
Currently Loaded Modules:
  1) python/3.7.3        5) py-jinja2/2.10         9) py-jupyter/1.1.4
  2) py-markupsafe/1.0   6) py-six/1.12.0         10) texlive/2020
  3) py-pytz/2018.4      7) intel-mkl/2019.4.243
  4) py-babel/2.6.0      8) py-numpy/1.16.3
In [11]:
ls -l $xlibpath
total 0
In [12]:
import sys
sys.path
Out [12]:
['/home/wpurwant/Testbeds/jupyter/wahab_jupyter_start',
 '/shared/apps/common/lmod/scripts/init',
 '',
 '/shared/apps/auto/py-jupyter/1.1.4-gcc-7.3.0-cxdz/lib/python3.7/site-packages',
 '/shared/apps/auto/py-numpy/1.16.3-gcc-7.3.0-mdow/lib/python3.7/site-packages',
 '/shared/apps/auto/py-six/1.12.0-gcc-7.3.0-ay7a/lib/python3.7/site-packages',
 '/shared/apps/auto/py-jinja2/2.10-gcc-7.3.0-usre/lib/python3.7/site-packages',
 '/shared/apps/auto/py-babel/2.6.0-gcc-7.3.0-be4y/lib/python3.7/site-packages',
 '/shared/apps/auto/py-pytz/2018.4-gcc-7.3.0-wqdr/lib/python3.7/site-packages',
 '/shared/apps/auto/py-markupsafe/1.0-gcc-7.3.0-biyo/lib/python3.7/site-packages',
 '/shared/apps/common/python/3.7.3-wqal/lib/python37.zip',
 '/shared/apps/common/python/3.7.3-wqal/lib/python3.7',
 '/shared/apps/common/python/3.7.3-wqal/lib/python3.7/lib-dynload',
 '/home/wpurwant/.local/lib/python3.7/site-packages',
 '/shared/apps/common/python/3.7.3-wqal/lib/python3.7/site-packages',
 '/shared/apps/auto/py-jupyter/1.1.4-gcc-7.3.0-cxdz/lib/python3.7/site-packages/IPython/extensions',
 '/home/wpurwant/.ipython',
 '/home/wpurwant/Testbeds/jupyter/wahab_jupyter_start',
 '/shared/apps/common/lmod/current/init']
In [13]:
ls -l $xlibpath
total 0
In [14]:
module
Out [14]:
<function env_modules_python.module(command, *arguments)>
In [15]:
module("list")
Currently Loaded Modules:
  1) python/3.7.3        5) py-jinja2/2.10         9) py-jupyter/1.1.4
  2) py-markupsafe/1.0   6) py-six/1.12.0         10) texlive/2020
  3) py-pytz/2018.4      7) intel-mkl/2019.4.243
  4) py-babel/2.6.0      8) py-numpy/1.16.3
In [16]:
ls -l $xlibpath
total 0
In [17]:
module("load", "py-pandas")
The following have been reloaded with a version change:
  1) py-pytz/2018.4 => py-pytz/2019.3
In [11]:
ls -l $xlibpath
total 172
lrwxrwxrwx 1 wpurwant users 131 Jun  9 19:04 libiomp5.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/compiler/lib/intel64_lin/libiomp5.so*
lrwxrwxrwx 1 wpurwant users 134 Jun  9 19:04 libiomp5_db.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/compiler/lib/intel64_lin/libiomp5_db.so*
lrwxrwxrwx 1 wpurwant users 136 Jun  9 19:04 libiompstubs5.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/compiler/lib/intel64_lin/libiompstubs5.so*
lrwxrwxrwx 1 wpurwant users 128 Jun  9 19:04 libmkl_avx.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_avx.so*
lrwxrwxrwx 1 wpurwant users 129 Jun  9 19:04 libmkl_avx2.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_avx2.so*
lrwxrwxrwx 1 wpurwant users 131 Jun  9 19:04 libmkl_avx512.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_avx512.so*
lrwxrwxrwx 1 wpurwant users 135 Jun  9 19:04 libmkl_avx512_mic.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_avx512_mic.so*
lrwxrwxrwx 1 wpurwant users 145 Jun  9 19:04 libmkl_blacs_intelmpi_ilp64.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_blacs_intelmpi_ilp64.so*
lrwxrwxrwx 1 wpurwant users 144 Jun  9 19:04 libmkl_blacs_intelmpi_lp64.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_blacs_intelmpi_lp64.so*
lrwxrwxrwx 1 wpurwant users 144 Jun  9 19:04 libmkl_blacs_openmpi_ilp64.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_blacs_openmpi_ilp64.so*
lrwxrwxrwx 1 wpurwant users 143 Jun  9 19:04 libmkl_blacs_openmpi_lp64.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_blacs_openmpi_lp64.so*
lrwxrwxrwx 1 wpurwant users 143 Jun  9 19:04 libmkl_blacs_sgimpt_ilp64.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_blacs_sgimpt_ilp64.so*
lrwxrwxrwx 1 wpurwant users 142 Jun  9 19:04 libmkl_blacs_sgimpt_lp64.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_blacs_sgimpt_lp64.so*
lrwxrwxrwx 1 wpurwant users 134 Jun  9 19:04 libmkl_cdft_core.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_cdft_core.so*
lrwxrwxrwx 1 wpurwant users 129 Jun  9 19:04 libmkl_core.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_core.so*
lrwxrwxrwx 1 wpurwant users 128 Jun  9 19:04 libmkl_def.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_def.so*
lrwxrwxrwx 1 wpurwant users 133 Jun  9 19:04 libmkl_gf_ilp64.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_gf_ilp64.so*
lrwxrwxrwx 1 wpurwant users 132 Jun  9 19:04 libmkl_gf_lp64.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_gf_lp64.so*
lrwxrwxrwx 1 wpurwant users 135 Jun  9 19:04 libmkl_gnu_thread.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_gnu_thread.so*
lrwxrwxrwx 1 wpurwant users 136 Jun  9 19:04 libmkl_intel_ilp64.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_intel_ilp64.so*
lrwxrwxrwx 1 wpurwant users 135 Jun  9 19:04 libmkl_intel_lp64.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_intel_lp64.so*
lrwxrwxrwx 1 wpurwant users 137 Jun  9 19:04 libmkl_intel_thread.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_intel_thread.so*
lrwxrwxrwx 1 wpurwant users 127 Jun  9 19:04 libmkl_mc.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_mc.so*
lrwxrwxrwx 1 wpurwant users 128 Jun  9 19:04 libmkl_mc3.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_mc3.so*
lrwxrwxrwx 1 wpurwant users 135 Jun  9 19:04 libmkl_pgi_thread.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_pgi_thread.so*
lrwxrwxrwx 1 wpurwant users 127 Jun  9 19:04 libmkl_rt.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_rt.so*
lrwxrwxrwx 1 wpurwant users 140 Jun  9 19:04 libmkl_scalapack_ilp64.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_scalapack_ilp64.so*
lrwxrwxrwx 1 wpurwant users 139 Jun  9 19:04 libmkl_scalapack_lp64.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_scalapack_lp64.so*
lrwxrwxrwx 1 wpurwant users 135 Jun  9 19:04 libmkl_sequential.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_sequential.so*
lrwxrwxrwx 1 wpurwant users 135 Jun  9 19:04 libmkl_tbb_thread.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_tbb_thread.so*
lrwxrwxrwx 1 wpurwant users 132 Jun  9 19:04 libmkl_vml_avx.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_vml_avx.so*
lrwxrwxrwx 1 wpurwant users 133 Jun  9 19:04 libmkl_vml_avx2.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_vml_avx2.so*
lrwxrwxrwx 1 wpurwant users 135 Jun  9 19:04 libmkl_vml_avx512.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_vml_avx512.so*
lrwxrwxrwx 1 wpurwant users 139 Jun  9 19:04 libmkl_vml_avx512_mic.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_vml_avx512_mic.so*
lrwxrwxrwx 1 wpurwant users 133 Jun  9 19:04 libmkl_vml_cmpt.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_vml_cmpt.so*
lrwxrwxrwx 1 wpurwant users 132 Jun  9 19:04 libmkl_vml_def.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_vml_def.so*
lrwxrwxrwx 1 wpurwant users 131 Jun  9 19:04 libmkl_vml_mc.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_vml_mc.so*
lrwxrwxrwx 1 wpurwant users 132 Jun  9 19:04 libmkl_vml_mc2.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_vml_mc2.so*
lrwxrwxrwx 1 wpurwant users 132 Jun  9 19:04 libmkl_vml_mc3.so -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin/libmkl_vml_mc3.so*
lrwxrwxrwx 1 wpurwant users  58 Jun  9 19:04 libpython3.7m.so -> /shared/apps/common/python/3.7.3-wqal/lib/libpython3.7m.so*
lrwxrwxrwx 1 wpurwant users  62 Jun  9 19:04 libpython3.7m.so.1.0 -> /shared/apps/common/python/3.7.3-wqal/lib/libpython3.7m.so.1.0*
lrwxrwxrwx 1 wpurwant users  55 Jun  9 19:04 libpython3.so -> /shared/apps/common/python/3.7.3-wqal/lib/libpython3.so*
lrwxrwxrwx 1 wpurwant users 133 Jun  9 19:04 libtbb.so.2 -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/tbb/lib/intel64_lin/gcc4.7/libtbb.so.2*
lrwxrwxrwx 1 wpurwant users 139 Jun  9 19:04 libtbbmalloc.so.2 -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/tbb/lib/intel64_lin/gcc4.7/libtbbmalloc.so.2*
lrwxrwxrwx 1 wpurwant users 145 Jun  9 19:04 libtbbmalloc_proxy.so.2 -> /shared/apps/auto/intel-mkl/2019.4.243-gcc-7.3.0-ntiu/compilers_and_libraries_2019.4.243/linux/tbb/lib/intel64_lin/gcc4.7/libtbbmalloc_proxy.so.2*
In [6]:
import numpy
In [ ]:
Reference in New Issue View Git Blame Copy Permalink