+conda

conda/conda.md

+---
+author: Moritz Siegel
+title: Conda - Reproducible Python Environments
+revealOptions:
+transition: 'fade'
+---
+
+
+## Conda & Anaconda & Mamba
+
+- `conda` is the package manager core.
+
+- `miniconda` is the package we installed to provide conda.
+
+- `anaconda` is a proprietary (unfree!) summary of packages.
+
+- `mamba` was a faster sister of conda.
+
+- `micromamba` sister of miniconda.
+
+
+we focus on `conda` via `miniconda`.
+
+
+## Start Conda
+
+First load the **miniconda** package with `module` (or `spack`):
+
+```bash
+zen trainee00@l55:~$ module load miniconda3
+```
+
+Start **conda bash hook** to get the conda functions:
+
+```bash
+zen trainee00@l55:~$ eval "$(conda shell.bash hook)"
+(base) zen trainee00@l55:~$ conda
+usage: conda [-h] [-v] [--no-plugins] [-V] COMMAND ...
+
+conda is a tool for managing and deploying applications,
+environments and packages.
+
+...
+```
+
+Note how the prompt now shows the **base** env.
+
+
+## Define Your Environment
+
+Create a `yaml` file **defining** your environment.
+
+```yaml
+name: myenv
+channels:
+  - conda-forge
+  - defaults
+dependencies:
+  - python=3.10
+  - tensorflow=2.15.0
+```
+
+Use e.g. `nano myenv.yaml` to write this file.
+
+## Create Your Environment
+
+Create your conda env:
+
+```bash
+(base) zen trainee00@l55:~$ conda env create --file myenv.yaml
+Channels:
+ - conda-forge
+ - defaults
+ - bioconda
+Platform: linux-64
+Collecting package metadata (repodata.json): /
+...
+```
+
+Depending of the number of packages this might take some time.
+
+
+## Activate Your Environment
+
+Activate your conda env, note how the **prompt** changes:
+
+```bash
+(base) zen trainee00@l55:~$ conda env create --file myenv.yaml
+(base) zen trainee00@l55:~$ conda activate myenv
+...
+(myenv) zen trainee00@l55:~$ python --version
+Python 3.10.11
+```
+
+The python version shown is the installation from conda.
+
+
+## Submit Your Conda Env To Slurm
+
+Create a slurm batch script starting your conda env and running
+python:
+
+```bash
+#!/bin/bash
+#SBATCH --job-name=slurm_conda_example
+#SBATCH --time=00-00:05:00
+#SBATCH --ntasks=2
+#SBATCH --mem=2GB
+
+module load miniconda3
+eval "$(conda shell.bash hook)"
+conda activate myenv
+python --version
+```
+
+Use e.g. `nano myenv.sh` to write this file.
+
+
+## List All Environments
+
+```bash
+(base) zen trainee00@l55:~$ conda env list
+# conda environments:
+#
+myenv     /home/fs70824/trainee00/.conda/envs/myenv
+base   *  /opt/sw/conda
+py310     /opt/sw/conda/envs/py310
+py311     /opt/sw/conda/envs/py311
+py312     /opt/sw/conda/envs/py312
+py38      /opt/sw/conda/envs/py38
+py39      /opt/sw/conda/envs/py39
+```
+
+The star `*` shows the currently active environment.
+
+
+## Deactivate Your Environment
+
+Deactivate your conda environment to get back to `(base)`:
+
+```bash
+(myenv) zen trainee00@l55:~$ conda deactivate
+(base) zen trainee00@l55:~$ conda deactivate
+zen trainee00@l55:~$
+```
+
+Type `conda deactivate` again to **get out** of conda completely.
+
+
+## Remove Your Environment
+
+If you don't need them anymore please **remove** any old environments:
+
+```bash
+(base) zen trainee00@l55:~$ conda env remove -n myenv
+
+Remove all packages in environment /home/fs70824/trainee00/.conda/env
+s/myenv:
+
+Everything found within the environment (/home/fs70824/trainee00/.con
+da/envs/myenv), including any conda environment configurations and an
+y non-conda files, will be deleted. Do you wish to continue?
+ (y/[n])?
+```
+
+Conda creates **many small files** and might fill up your `$HOME`
+file system quickly.
+
+
+## Exercises
+
+- make a new empty directory `contest` to play around in.
+- load **miniconda3**.
+- write your own `conda.yaml` and install **numpy**.
+- create a new env based on your `conda.yaml`.
+- activate your env.
+- find out what version of python is running there.
+- submit a job to `slurm` printing the python version.
+- deactivate and remove your environment.
+
+
+## Links
+
+- There is a lot about python, conda and jypyter at our **python4hpc** courses:
+
+  [https://vsc.ac.at//research/vsc-research-center/vsc-school-seminar/](https://vsc.ac.at//research/vsc-research-center/vsc-school-seminar/)
+
+- In depth `jupyter` notebooks (`ipynb`) from the python4hpc course:
+
+  [https://gitlab.tuwien.ac.at/vsc-public/training/python4hpc/-/blob/main/D1_02_env_03_conda.ipynb](https://gitlab.tuwien.ac.at/vsc-public/training/python4hpc/-/blob/main/D1_02_env_03_conda.ipynb)
+
+- Some basic usage like this example on our wiki:
+
+  [https://wiki.vsc.ac.at/doku.php?id=doku:python](https://wiki.vsc.ac.at/doku.php?id=doku:python)

conda/makefile

+.PHONY: clean
+
+conda.pdf: conda.md pictures/slide.tex
+	pandoc --to=beamer \
+		--ascii \
+		--pdf-engine=xelatex \
+		--include-in-header=pictures/slide.tex \
+		--variable=aspectratio:169 \
+		--highlight-style=breezedark \
+		--standalone \
+		--output=conda.pdf \
+		conda.md

conda/pictures/slide.tex

+%% template file for markdown > pandoc > latex > pdf
+\usepackage[margins=raggedright]{floatrow}
+\let\oldTexttt\texttt
+\renewcommand{\texttt}[1]{\oldTexttt{\textbf{#1}}} % overwrite \texttt command to use fat \verb
+%\renewcommand{\texttt}[1]{\oldTexttt{\textcolor{vscred}{#1}}} % overwrite \texttt command to use fg color for \verb
+%\renewcommand{\texttt}[1]{\oldTexttt{\colorbox{shadecolor}{#1}}} % overwrite \texttt command to use bg color for \verb
+\definecolor{vscred}{RGB}{204,51,0}
+\usecolortheme[named=vscred]{structure}
+\setbeamertemplate{footline}[text line]{%       
+  \parbox{%
+    \linewidth}{%
+    \includegraphics[height=0.4cm]{pictures/footer.png}
+    \includegraphics[height=0.4cm]{pictures/footer.png}
+    \includegraphics[height=0.4cm]{pictures/footer.png}
+    \hfill\usebeamercolor[vscred]{}\large
+    \insertframenumber/\inserttotalframenumber
+    \vspace{1em}
+  }
+}
+\logo{\includegraphics[height=1cm]{pictures/logo.png}\vspace{44em}~} % adjust position via vspace

conda/readme.md

+---
+author: Moritz Siegel
+title: Conda - Reproducible Python Environments
+revealOptions:
+transition: 'fade'
+---
+
+
+## Conda & Anaconda & Mamba
+
+- `conda` is the package manager core.
+
+- `miniconda` is the package we installed to provide conda.
+
+- `anaconda` is a proprietary (unfree!) summary of packages.
+
+- `mamba` was a faster sister of conda.
+
+- `micromamba` sister of miniconda.
+
+
+we focus on `conda` via `miniconda`.
+
+
+## Start Conda
+
+First load the **miniconda** package with `module` (or `spack`):
+
+```bash
+zen trainee00@l55:~$ module load miniconda3
+```
+
+Start **conda bash hook** to get the conda functions:
+
+```bash
+zen trainee00@l55:~$ eval "$(conda shell.bash hook)"
+(base) zen trainee00@l55:~$ conda
+usage: conda [-h] [-v] [--no-plugins] [-V] COMMAND ...
+
+conda is a tool for managing and deploying applications,
+environments and packages.
+
+...
+```
+
+Note how the prompt now shows the **base** env.
+
+
+## Define Your Environment
+
+Create a `yaml` file **defining** your environment.
+
+```yaml
+name: myenv
+channels:
+  - conda-forge
+  - defaults
+dependencies:
+  - python=3.10
+  - tensorflow=2.15.0
+```
+
+Use e.g. `nano myenv.yaml` to write this file.
+
+## Create Your Environment
+
+Create your conda env:
+
+```bash
+(base) zen trainee00@l55:~$ conda env create --file myenv.yaml
+Channels:
+ - conda-forge
+ - defaults
+ - bioconda
+Platform: linux-64
+Collecting package metadata (repodata.json): /
+...
+```
+
+Depending of the number of packages this might take some time.
+
+
+## Activate Your Environment
+
+Activate your conda env, note how the **prompt** changes:
+
+```bash
+(base) zen trainee00@l55:~$ conda env create --file myenv.yaml
+(base) zen trainee00@l55:~$ conda activate myenv
+...
+(myenv) zen trainee00@l55:~$ python --version
+Python 3.10.11
+```
+
+The python version shown is the installation from conda.
+
+
+## Submit Your Conda Env To Slurm
+
+Create a slurm batch script starting your conda env and running
+python:
+
+```bash
+#!/bin/bash
+#SBATCH --job-name=slurm_conda_example
+#SBATCH --time=00-00:05:00
+#SBATCH --ntasks=2
+#SBATCH --mem=2GB
+
+module load miniconda3
+eval "$(conda shell.bash hook)"
+conda activate myenv
+python --version
+```
+
+Use e.g. `nano myenv.sh` to write this file.
+
+
+## List All Environments
+
+```bash
+(base) zen trainee00@l55:~$ conda env list
+# conda environments:
+#
+myenv     /home/fs70824/trainee00/.conda/envs/myenv
+base   *  /opt/sw/conda
+py310     /opt/sw/conda/envs/py310
+py311     /opt/sw/conda/envs/py311
+py312     /opt/sw/conda/envs/py312
+py38      /opt/sw/conda/envs/py38
+py39      /opt/sw/conda/envs/py39
+```
+
+The star `*` shows the currently active environment.
+
+
+## Deactivate Your Environment
+
+Deactivate your conda environment to get back to `(base)`:
+
+```bash
+(myenv) zen trainee00@l55:~$ conda deactivate
+(base) zen trainee00@l55:~$ conda deactivate
+zen trainee00@l55:~$
+```
+
+Type `conda deactivate` again to **get out** of conda completely.
+
+
+## Remove Your Environment
+
+If you don't need them anymore please **remove** any old environments:
+
+```bash
+(base) zen trainee00@l55:~$ conda env remove -n myenv
+
+Remove all packages in environment /home/fs70824/trainee00/.conda/env
+s/myenv:
+
+Everything found within the environment (/home/fs70824/trainee00/.con
+da/envs/myenv), including any conda environment configurations and an
+y non-conda files, will be deleted. Do you wish to continue?
+ (y/[n])?
+```
+
+Conda creates **many small files** and might fill up your `$HOME`
+file system quickly.
+
+
+## Exercises
+
+- make a new empty directory `contest` to play around in.
+- load **miniconda3**.
+- write your own `conda.yaml` and install **numpy**.
+- create a new env based on your `conda.yaml`.
+- activate your env.
+- find out what version of python is running there.
+- submit a job to `slurm` printing the python version.
+- deactivate and remove your environment.
+
+
+## Links
+
+- There is a lot about python, conda and jypyter at our **python4hpc** courses:
+
+  [https://vsc.ac.at//research/vsc-research-center/vsc-school-seminar/](https://vsc.ac.at//research/vsc-research-center/vsc-school-seminar/)
+
+- In depth `jupyter` notebooks (`ipynb`) from the python4hpc course:
+
+  [https://gitlab.tuwien.ac.at/vsc-public/training/python4hpc/-/blob/main/D1_02_env_03_conda.ipynb](https://gitlab.tuwien.ac.at/vsc-public/training/python4hpc/-/blob/main/D1_02_env_03_conda.ipynb)
+
+- Some basic usage like this example on our wiki:
+
+  [https://wiki.vsc.ac.at/doku.php?id=doku:python](https://wiki.vsc.ac.at/doku.php?id=doku:python)