Updated to hub-2.1.1 and added pyspark & all-spark images. Updated README

README.md

@@ -2,17 +2,17 @@
 
 ## '.def' file
 
-In order to create an image file suitable for our VSC JupyterHub instance you first have to create a '.def' file that tells singularity how to build the actual image. This can be made from scratch or by e.g. using a previously built docker image. 
+In order to create an image file suitable for our VSC JupyterHub instance you first have to create a '.def' file that tells singularity how to build the actual image. This can be made from scratch or by e.g. using a previously built docker image.
 
 If you want to use a docker image in moste cases the following definition is enough. You just have to replace `<docker-image-tag>` with the right tag you want to base your image on. In the `post` step we want to add the batchspawner package in order to be able for the image to work together with JupyterHub. If you have a different python installation in your image you need to adjust the path to the `pip` executable as well.
 
-
+```singularity
     BootStrap: docker
     From: <docker-image-tag>
 
     %post
     /opt/conda/bin/pip install batchspawner==1.1.0
-
+```
 
 ## '.meta' file
 
@@ -20,6 +20,7 @@ Our Jupyterhub uses self-defined '.meta' files to automatically add the existing
 
 For this mechanism to work you just have to place a `.meta` file with the same name as the `.def` file in the same directory.
 
+```json
     {
         "title": "Base Notebook ({group})",
         "description": "Base image of the jupyter docker core stacks containing python3",
@@ -32,14 +33,15 @@ For this mechanism to work you just have to place a `.meta` file with the same n
         "required-groups": [],
         "order": 0
     }
+```
 
 The `.meta` files contents must conform to JSON specifications. It can contain fields as listed above in the examples.
 
 The `title` field may use the `{group}` tag which automatically gets filled with the folder name the image is stored in (empty if its in the root directory of the image share).
 
-Most of the other fields are pretty self explanatory: the `description` field can contain a small description of whats in the image. This description will be displayed in the UI upon selecting the image frmo the drop down. Similar to this field the `link` will also be displayed there. 
+Most of the other fields are pretty self explanatory: the `description` field can contain a small description of whats in the image. This description will be displayed in the UI upon selecting the image frmo the drop down. Similar to this field the `link` will also be displayed there.
 
-`version` should correspond to the contained version of the image. 
+`version` should correspond to the contained version of the image.
 
 `required-caps` determines in which cluster configurations the image can be used and `exclude-caps` tells the system that the image cannot be used if one of the caps is present. Possible required caps are:
 
@@ -51,16 +53,15 @@ Most of the other fields are pretty self explanatory: the `description` field ca
 
 `order` is a simple integer number to override the lexicographical order within a folder
 
-
 ## Building
 
-Once checked in and added to `.gitlab-ci.yml` (just copy an example job and adjust), the image can be built by going to the gitlab page of the repository (https://git.zid.tuwien.ac.at/vsc/vsc-jupyterhub-notebooks/-/pipelines) and running the added image job. 
+Once checked in and added to `.gitlab-ci.yml` (just copy an example job and adjust), the image can be built by going to the gitlab page of the [repository](https://gitlab.tuwien.ac.at/vsc/vsc-jupyterhub-notebooks) and running the added image job.
 
-After a sucessful build the image will be automatically copied to 
+After a sucessful build the image will be automatically copied to
 
-- VSC3 Shares: USIF/jupyter-notebook/<image-tag>
-- VSC4 Shares: usif/jupyter-notebook/<image-tag>
+- VSC3 Shares: USIF/jupyter-notebook/\<image-tag\>
+- VSC4 Shares: usif/jupyter-notebook/\<image-tag\>
 
 and should be available from the dropdown of the UI.
 
-Note: In case the built image doesn't show up immediately log out and login again to trigger the loading of existing meta files. You can also check the `admin` section in JupyterHub where all images that have been read in are listed.
\ No newline at end of file
+Note: In case the built image doesn't show up immediately log out and login again to trigger the loading of existing meta files. You can also check the `admin` section in JupyterHub where all images that have been read in are listed.

default/all-spark.def

+BootStrap: docker
+From: jupyter/all-spark-notebook:hub-2.1.1
+
+%files
+requirements.txt /home/jovyan/requirements.txt
+install-packages.sh /home/jovyan/install-packages.sh
+
+%post
+/usr/bin/bash /home/jovyan/install-packages.sh
+rm -f /home/jovyan/requirements.txt
+rm -f /home/jovyan/install-packages.sh

default/all-spark.meta

+{
+    "title": "All-Spark Notebook ({group})",
+    "description": "Based on the 'PySpark' image with the addition of IRKernel, rcurl, sparklyr, ggplot2 & spylon-kernel",
+    "link": "https://github.com/jupyter/docker-stacks",
+    "version": "hub-2.1.1",
+    "author": "VSC",
+    "contact": "service@vsc.ac.at",
+    "required-caps": [],
+    "exclude-caps": ["gpu"],
+    "required-groups": [],
+    "order": 4
+}
\ No newline at end of file

default/datascience.def

 BootStrap: docker
-From: jupyter/datascience-notebook:hub-2.1.0
+From: jupyter/datascience-notebook:hub-2.1.1
 
 %files
 requirements.txt /home/jovyan/requirements.txt

default/datascience.meta

@@ -2,7 +2,7 @@
     "title": "Datascience Notebook ({group})",
     "description": "The Datascience image contains everything from 'SciPy' & 'R' as well as an Julia environment.",
     "link": "https://github.com/jupyter/docker-stacks",
-    "version": "hub-2.1.0",
+    "version": "hub-2.1.1",
     "author": "VSC",
     "contact": "service@vsc.ac.at",
     "required-caps": [],

default/minimal.def

 BootStrap: docker
-From: jupyter/minimal-notebook:hub-2.1.0
+From: jupyter/minimal-notebook:hub-2.1.1
 
 %files
 requirements.txt /home/jovyan/requirements.txt

default/minimal.meta

@@ -2,7 +2,7 @@
     "title": "Minimal Notebook ({group})",
     "description": "Minimal image contains everything from 'base' as well as TeX Live",
     "link": "https://github.com/jupyter/docker-stacks",
-    "version": "hub-2.1.0",
+    "version": "hub-2.1.1",
     "author": "VSC",
     "contact": "service@vsc.ac.at",
     "required-caps": [],

default/pyspark.def

+BootStrap: docker
+From: jupyter/pyspark-notebook:hub-2.1.1
+
+%files
+requirements.txt /home/jovyan/requirements.txt
+install-packages.sh /home/jovyan/install-packages.sh
+
+%post
+/usr/bin/bash /home/jovyan/install-packages.sh
+rm -f /home/jovyan/requirements.txt
+rm -f /home/jovyan/install-packages.sh

default/pyspark.meta

+{
+    "title": "PySpark Notebook ({group})",
+    "description": "The PySpark image is based on the 'SciPy' image and contains Apache Spark, Hadoop binaries and the pyarrow library in addition",
+    "link": "https://github.com/jupyter/docker-stacks",
+    "version": "hub-2.1.1",
+    "author": "VSC",
+    "contact": "service@vsc.ac.at",
+    "required-caps": [],
+    "exclude-caps": ["gpu"],
+    "required-groups": [],
+    "order": 4
+}
\ No newline at end of file

default/r.def

 BootStrap: docker
-From: jupyter/r-notebook:hub-2.1.0
+From: jupyter/r-notebook:hub-2.1.1
 
 %files
 requirements.txt /home/jovyan/requirements.txt

default/r.meta

@@ -2,7 +2,7 @@
     "title": "R Notebook ({group})",
     "description": "R image contains everything from 'minimal' as well as the R interpreter and base environment.",
     "link": "https://github.com/jupyter/docker-stacks",
-    "version": "hub-2.1.0",
+    "version": "hub-2.1.1",
     "author": "VSC",
     "contact": "service@vsc.ac.at",
     "required-caps": [],

default/scipy.def

 BootStrap: docker
-From: jupyter/scipy-notebook:hub-2.1.0
+From: jupyter/scipy-notebook:hub-2.1.1
 
 %files
 requirements.txt /home/jovyan/requirements.txt

default/scipy.meta

@@ -2,7 +2,7 @@
     "title": "SciPy Notebook ({group})",
     "description": "The SciPy image contains everything from 'minimal' and popular packages from the scientific Python ecosystem.",
     "link": "https://github.com/jupyter/docker-stacks",
-    "version": "hub-2.1.0",
+    "version": "hub-2.1.1",
     "author": "VSC",
     "contact": "service@vsc.ac.at",
     "required-caps": [],

default/tensorflow.def

 BootStrap: docker
-From: jupyter/tensorflow-notebook:hub-2.1.0
+From: jupyter/tensorflow-notebook:hub-2.1.1
 
 %files
 requirements.txt /home/jovyan/requirements.txt

default/tensorflow.meta

@@ -2,7 +2,7 @@
     "title": "Tensorflow Notebook ({group})",
     "description": "The Tensorflow image contains everything from 'SciPy' as well as the tensorflow machine learning library.",
     "link": "https://github.com/jupyter/docker-stacks",
-    "version": "hub-2.1.0",
+    "version": "hub-2.1.1",
     "author": "VSC",
     "contact": "service@vsc.ac.at",
     "required-caps": [],