HPC High Performance Computing: 2. Computing environment

Environment variables

By default user common variables printed out by the env command:

test@node009:~$ env
MANPATH=/sNow/easybuild/lmod/lmod/share/man::
EASYBUILD_MODULES_TOOL=Lmod
TERM=xterm
SHELL=/bin/bash
MODULEPATH_ROOT=/sNow/easybuild/modulefiles
SSH_CLIENT=10.80.60.5 58918 22
LMOD_PKG=/sNow/easybuild/lmod/lmod
EASYBUILD_INSTALLPATH=/sNow/easybuild/debian/8.8/Broadwell
LMOD_VERSION=7.4.20
EASYBUILD_PREFIX=/sNow/easybuild/debian/8.8/Broadwell
OS=debian
SSH_TTY=/dev/pts/0
USER=test
LMOD_sys=Linux
OSVERSION=8.8
MAIL=/var/mail/test
PATH=/sNow/snow-tools/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
SLURM_CONF=/etc/slurm/slurm.conf
LMOD_SETTARG_CMD=:
PWD=/homedtic/test
LANG=en_GB.UTF-8
MODULEPATH=/sNow/easybuild/modules/all:/sNow/easybuild/debian/8.8/Broadwell/modules/all:/sNow/easybuild/modulefiles/Linux:/sNow/easybuild/modulefiles/Core:/sNow/easybuild/lmod/lmod/modulefiles/Core
LMOD_CMD=/sNow/easybuild/lmod/lmod/libexec/lmod
SHLVL=1
HOME=/homedtic/test
EASYBUILD_SOURCEPATH=/sNow/easybuild/sources
EASYBUILD_TMP_LOGDIR=/sNow/easybuild/tmp
BASH_ENV=/sNow/easybuild/lmod/lmod/init/bash
PYTHONPATH=/sNow/easybuild/installation/lib/python2.7/site-packages:
LOGNAME=test
SSH_CONNECTION=10.80.60.5 58918 10.80.61.9 22
MODULESHOME=/sNow/easybuild/lmod/lmod
LMOD_FULL_SETTARG_SUPPORT=no
ARCHITECTURE=Broadwell
LMOD_DIR=/sNow/easybuild/lmod/lmod/libexec

From that extensive list, you'll basically work with the following ones:

HOME=/homedtic/test
PATH=/sNow/snow-tools/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
LIBRARY_PATH=/sNow/easybuild/debian/8.8/Broadwell/software/CUDA/8.0.61/lib64
LD_LIBRARY_PATH=/sNow/easybuild/debian/8.8/Broadwell/software/CUDA/8.0.61/lib64

The environment variables are managed by the system and inherited from several places at login time. To add or modify the environment variables, we suggest this way:

Edit the file ~/.bashrc. It is placed on your home directory:

test@node009:~$ 
test@node009:~$ vi .bashrc

and go to the last lines of the file. In the following example, we'll add the environment variables for the Matlab Java JVM and we'll add Matlab to our application path:

# Add the path of matlab's binary to the system PATH
export PATH=$PATH:/soft/MATLAB/R2013b/bin:/sge/matlab

To process the new version of our .bashrc, we have to logout and login again, as .bashrc is processed on login time.

Modules

The Modules package provides for the dynamic modification of the user's environment via modulefiles. Remember that to load the modules you must be logged in a calculation node.

To see the available modules you can execute the module av command. 

test@node009:~$ module av
 
------------------------------------------------ /sNow/easybuild/modules/all ------------------------------------------------
   EasyBuild/3.2.1
 
------------------------------------- /sNow/easybuild/debian/8.8/Broadwell/modules/all --------------------------------------
   Autoconf/2.69-foss-2017a                                        Tcl/8.6.5-foss-2017a
   Automake/1.15-foss-2017a                                        Tk/8.6.5-foss-2017a
   Autotools/20150215-foss-2017a                                   binutils/2.27-GCCcore-6.3.0
   Bison/3.0.4-GCCcore-6.3.0                                       binutils/2.27                 (D)
   Bison/3.0.4                                              (D)    bzip2/1.0.6-foss-2017a
   CUDA/8.0.61                                                     flex/2.6.0
   EasyBuild/3.3.1                                          (D)    flex/2.6.3-GCCcore-6.3.0      (D)
   FFTW/3.3.6-gompi-2017a                                          foss/2017a
   GCC/6.3.0-2.27                                                  gompi/2017a
   GCCcore/6.3.0                                                   help2man/1.47.4-GCCcore-6.3.0
   GMP/6.1.1-foss-2017a                                            hwloc/1.11.5-GCC-6.3.0-2.27
   Java/1.8.0_45                                                   libffi/3.2.1-foss-2017a
   M4/1.4.17-foss-2017a                                            libreadline/6.3-foss-2017a
   M4/1.4.17                                                       libtool/2.4.6-foss-2017a
   M4/1.4.18-GCCcore-6.3.0                                  (D)    mdtest/git-foss-2017a
   OpenBLAS/0.2.19-GCC-6.3.0-2.27-LAPACK-3.7.0                     ncurses/6.0-foss-2017a
   OpenMPI/2.0.2-GCC-6.3.0-2.27                                    numactl/2.0.11-GCC-6.3.0-2.27
   Python/2.7.12-foss-2017a                                        zlib/1.2.8-foss-2017a
   SQLite/3.13.0-foss-2017a                                        zlib/1.2.8
   ScaLAPACK/2.0.2-gompi-2017a-OpenBLAS-0.2.19-LAPACK-3.7.0        zlib/1.2.11-GCCcore-6.3.0     (D)
   Szip/2.1.1-foss-2017a
 
---------------------------------------- /sNow/easybuild/lmod/lmod/modulefiles/Core -----------------------------------------
   lmod/7.4.20    settarg/7.4.20
 
  Where:
   D:  Default Module
 
Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

To load cuda module and to check if is really loaded: 

test@node019:~$ module load CUDA/8.0.61
test@node019:~$ module list
 
Currently Loaded Modules:
  1) CUDA/8.0.61
 

The modules tool is useful to load and unload environment variables without worrying about .bashrc, but shows its best behavior if you want to dinamically change from two different installed versions of a package in the system. If you look carefully, you'll see that we have two different versions of gcc, and default is 4.92:

test@node009:~$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.9/lto-wrapper
Target: x86_64-linux-gnu
Thread model: posix
gcc version 4.9.2 (Debian 4.9.2-10) 

Let’s change the gcc version: 

test@node009:~$ 
test@node009:~$ module load GCC/6.3.0-2.27                                                  
test@node009:~$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/sNow/easybuild/debian/8.8/Broadwell/software/GCCcore/6.3.0/libexec/gcc/x86_64-pc-linux-gnu/6.3.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Thread model: posix
gcc version 6.3.0 (GCC) 
test@node009:~$