Add Jupyter (#27)

* Started adding jupyter

* Add docker-entrypoint.sh

* Can start jupyter but permission issues

* Finalize setu

* Update README

* Update versions

* Update README

Author: tschaffter

.env.example

@@ -1,15 +1,23 @@
-# RStudio
-USER=rstudio
+# NLP Sandbox Notebooks image version
+IMAGE_VERSION=0.2.0
+
+# User created when the container starts.
+# Those are the credentials to log in RStudio
+USERNAME=nlp
 PASSWORD=changeme
 USERID=1000
 GROUPID=1000
 ROOT=FALSE
 
-# Synapse
-SYNAPSE_TOKEN=yourtoken
+# Notebook IDE ("rstudio" or "jupyter")
+IDE=rstudio
+
+# Port on the host that connects to the IDE
+HOST_RSTUDIO_PORT=8787
+HOST_JUPYTER_PORT=8888
 
-# Port on the host that connects to RStudio
-HOST_PORT=80
+# Synapse settings
+SYNAPSE_TOKEN=
 
-# User-specified settings made visible to RStudio (APP_*)
+# User-specified variables visible to RStudio (must start with "APP_")
 APP_NLPSANDBOX_SCHEMAS_VERSION=1.2.0

Dockerfile

@@ -21,3 +21,11 @@ RUN install2.r --error renv \
     && R -e "renv::restore(lockfile='/tmp/renv.lock')" \
     && rm -rf /tmp/downloaded_packages/ /tmp/*.rds /tmp/renv.lock \
     && R -e "extrafont::font_import(prompt=FALSE)"
+
+WORKDIR /
+COPY docker-entrypoint.sh .
+RUN chmod +x docker-entrypoint.sh
+
+ENTRYPOINT ["/docker-entrypoint.sh"]
+CMD ["rstudio"]
+# CMD ["jupyter"]

README.md

@@ -10,33 +10,45 @@
 
 ## Introduction
 
-This repository provides Rmd notebooks that enable you to use R and Python
-together to interact with the NLP Sandbox ecosystem. For example, use example
-notebooks included with this repository to perform analysis using data stored in
-a local or remote instance of the NLP Sandbox Data Node. See the Section
-[Notebooks](#Notebooks) below for the list of notebooks included with this
-repository.
+This repository provides a portable development environment that includes
+RStudio and Jupyter. This environment aims to support NLP Sandbox users who
+develop or apply tools that intereact with the NLP Sandbox ecosystem.
 
-The Docker image [nlpsandbox/notebooks] offered by this project is based on the
-image [sagebionetworks/rstudio].
+For example, try the example notebooks included with this repository to perform
+analysis using data stored in a local or remote instance of the NLP Sandbox Data
+Node. See the Section [Notebooks](#Notebooks) for the list of example notebooks
+included with this repository.
+
+The Docker image [nlpsandbox/notebooks] that comes with this project is based on
+the image [sagebionetworks/rstudio].
 
 ## Requirements
 
-- [Docker Engine] >=19.03.0
+- [Docker Engine] >= 19.03.0
 - [Synapse.org] user account
 
 ## Specification
 
-- Notebooks version: 0.1.0
+- Notebooks version: 0.2.0
 - NLP Sandbox schemas version: 1.2.0
+- RStudio version: 4.1.0
+- JupyterLab version: 3.2.1
 
 ## Notebooks
 
-Rmd Notebook | Description
+Example notebooks for RStudio:
+
+Notebook | Description
+-------- | -----------
+[notebook.Rmd](notebooks/rstudio/notebook.Rmd)         | Default RStudio notebook.
+[r-and-python.Rmd](notebooks/rstudio/r-and-python.Rmd) | Shows how to use R and Python together.
+[data-node.Rmd](notebooks/rstudio/data-node.Rmd)       | Interact with an NLP Sandbox Data Node instance.
+
+Example notebooks for Jupiter:
+
+Notebook | Description
 -------- | -----------
-[notebook.Rmd](notebooks/notebook.Rmd)         | Default RStudio notebook.
-[r-and-python.Rmd](notebooks/r-and-python.Rmd) | Shows how to use R and Python together.
-[data-node.Rmd](notebooks/data-node.Rmd)       | Interact with an NLP Sandbox Data Node instance.
+[hello-world.ipynb](notebooks/jupyter/hello-world.ipynb) | Hello World notebook.
 
 > Important: Please make sure when you write your own notebooks that no
 > sensitive information ends up being publicly available. Please check with the
@@ -45,51 +57,61 @@ Rmd Notebook | Description
 
 ## Usage
 
-Create and edit the configuration file.
+Create the configuration file.
 
     cp .env.example .env
 
-You can either pull the docker image [nlpsandbox/notebooks] or build it using
+Specify in the config file what IDE should be used.
+
+    # Notebook IDE ("rstudio" or "jupyter")
+    IDE=rstudio
+
+You can either pull the Docker image [nlpsandbox/notebooks] or build it using
 the information included in `docker-compose.yml`. To pull the image and start
-RStudio:
+the IDE:
 
     docker compose pull
     docker compose up
 
-If you want to build the image and start RStudio, for instance after customizing
-`Dockerfile`:
+If you want to build the image before starting the IDE, for instance after
+customizing `Dockerfile`:
 
     docker compose up --build
 
-RStudio is now available at http://localhost. On the login page, enter the
-default username (`rstudio`) and the password specified in `.env`.
+To access the IDE, open your browser and follow the instructions below.
 
-In RStudio, open the RStudio project provided by this repository by clicking on
-the button `Project: (None)` > `Open Project...` on the top-right corner, then
-select the file `nlpsandbox/nlpsandbox.Rproj`. You should now be able to open
-and run the notebook `r-and-python.Rmd`. This will confirm that you can
-successfully use R and Python togehter.
+- RStudio: Navigate to http://localhost:8787
+- Jupyter: Navigate to the link displayed in the terminal when Jupyter started.
+  You will need to replace the hostname by "localhost". E.g.
+  http://localhost:8888/lab?token=4b08e8fc4469db3b4de1bb4d0ca20ef3684e5705bd193855
+
+### Stop the IDE
 
-To stop RStudio, enter `Ctrl+C` followed by `docker compose down`.  If running
+To stop the IDE, enter `Ctrl+C` followed by `docker compose down`.  If running
 in detached mode, you will only need to enter `docker compose down`.
 
 Tips:
 
 - Add `-d` or `--detach` as in `docker compose up -d` to run in the background.
 - If the command `docker compose` is missing, try `docker-compose`.
 
-## Configuring the CI/CD workflow
+## Open the RStudio project
+
+In RStudio, open the RStudio project provided by this repository by clicking on
+the button `Project: (None)` > `Open Project...` on the top-right corner, then
+select the file `nlpsandbox/nlpsandbox.Rproj`. You should now be able to open
+and run the notebook `r-and-python.Rmd`. This will confirm that you can
+successfully use R and Python togehter.
 
-The [CI/CD workflow] of this repository performs the following actions:
+## Configuring the CI/CD workflow
 
-- Generate HTML notebooks from R notebook and publishes them to GitHub Pages.
-- Build the Docker image [nlpsandbox/notebooks] and push it to Docker Hub.
+The [CI/CD workflow] of this repository builds the Docker image
+[nlpsandbox/notebooks] and pushes it to Docker Hub.
 
-If you decided to fork this repository or use it as a template, you will need to
+If you decide to use this repository as a GitHub template, you will need to
 update the environment variables defined at the top of the [CI/CD workflow]. You
-also need to create the following [GitHub Secrets]:
+also need to create the following [GitHub Secrets].
 
-- `RSTUDIO_PASSWORD`: "changeme"
 - `DOCKERHUB_USERNAME`: Your Docker Hub username.
 - `DOCKERHUB_TOKEN`: A personal access token (PAT) that has the permissions to
   push the image.

conda/nlpsandbox/requirements.txt

@@ -1,4 +1,5 @@
 challengeutils==4.0.1
+jupyterlab==3.2.1
 nlpsandbox-client==4.0.0
 pandas==1.2.4
 synapseclient==2.3.1

docker-compose.yml

@@ -2,19 +2,24 @@ version: "3.8"
 
 services:
   rstudio:
-    image: nlpsandbox/notebooks:0.1.0
+    image: nlpsandbox/notebooks:${IMAGE_VERSION}
     build:
       context: .
-    container_name: nlpsandbox-notebooks
-    command: rstudio
     env_file:
       - .env
+    environment:
+      # Prevents the local $USER value to be picked up. Instead use the user
+      # value defined in .env
+      - USER=${USERNAME}
+    container_name: nlpsandbox-notebooks
+    command: ${IDE}
     volumes:
-      - .:/home/rstudio/nlpsandbox:rw
+      - .:/home/${USERNAME}/nlpsandbox:rw
     # networks:
     #   - data-node
     ports:
-      - "${HOST_PORT}:8787"
+      - "${HOST_RSTUDIO_PORT}:8787"
+      - "${HOST_JUPYTER_PORT}:8888"
 
 # networks:
 #   data-node:

docker-entrypoint.sh

@@ -0,0 +1,17 @@
+#!/usr/bin/env bash
+
+set -e
+
+if [ "$1" = 'rstudio' ]; then
+    # RStudio userconf.sh script runs RStudio as $USER
+    exec /init
+elif [ "$1" = 'jupyter' ]; then
+    # /init takes care of creating $USER. It also starts RStudio.
+    exec /init gosu $USER /opt/miniconda/envs/nlpsandbox/bin/jupyter-lab \
+        --notebook-dir /home/${USER}/nlpsandbox \
+        --port=8888 \
+        --no-browser \
+        --ip=0.0.0.0
+fi
+
+exec /init gosu $USER "$@"

notebooks/jupyter/.gitignore

@@ -0,0 +1 @@
+.ipynb_checkpoints/

notebooks/jupyter/hello-world.ipynb

@@ -0,0 +1,44 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Hello World\n"
+     ]
+    }
+   ],
+   "source": [
+    "print('Hello World')"
+   ]
+  }
+ ],
+ "metadata": {
+  "interpreter": {
+   "hash": "93e7f2dcfb01f6655e7a9697237dee78452e308af71dd1cd67063cb7230fb558"
+  },
+  "kernelspec": {
+   "display_name": "Python 3.9.4 64-bit ('phi-deidentifier': conda)",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.4"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}