Skip to content

Commit a671e52

Browse files
minrkminrk
authored
Merge pull request #23 from willingc/user-docs
[WIP] Edit README to be more user friendly
2 parents cc3162f + c89e4ec commit a671e52

File tree

3 files changed

+189
-93
lines changed

3 files changed

+189
-93
lines changed

‎README.md‎

Lines changed: 161 additions & 85 deletions
Original file line numberDiff line numberDiff line change
@@ -1,102 +1,171 @@
1+
**[Technical Overview](#technical-overview)** |
2+
**[Prerequisites](#prerequisites)** |
3+
**[Authenticator setup](#authenticator-setup)** |
4+
**[Build the JupyterHub Docker image](#build-the-jupyterhub-docker-image)** |
5+
**[Spawner: Prepare the Jupyter Notebook Image](#spawner-prepare-the-jupyter-notebook-image)** |
6+
**[Run JupyterHub](#run-jupyterhub)** |
7+
**[Behind the scenes](#behind-the-scenes)** |
8+
**[FAQ](#faq)**
9+
110
# jupyterhub-deploy-docker
211

3-
This repository provides a reference deployment of [JupyterHub](https://github.com/jupyter/jupyterhub), a multi-user [Jupyter Notebook](http://jupyter.org/) environment, on a **single host** using [Docker](https://docs.docker.com).
12+
**jupyterhub-deploy-docker** provides a reference
13+
deployment of [JupyterHub](https://github.com/jupyter/jupyterhub), a
14+
multi-user [Jupyter Notebook](http://jupyter.org/) environment, on a
15+
**single host** using [Docker](https://docs.docker.com).
416

5-
This deployment:
17+
Possible **use cases** include:
618

7-
* Runs the [JupyterHub components](https://jupyterhub.readthedocs.org/en/latest/getting-started.html#overview) in a Docker container on the host
8-
* Uses [DockerSpawner](https://github.com/jupyter/dockerspawner) to spawn single-user Jupyter Notebook servers in separate Docker containers on the same host
9-
* Persists JupyterHub data in a Docker volume on the host
10-
* Persists user notebook directories in Docker volumes on the host
11-
* Uses [OAuthenticator](https://github.com/jupyter/oauthenticator) and [GitHub OAuth](https://developer.github.com/v3/oauth/) to authenticate users
19+
* Creating a JupyterHub demo environment that you can spin up relatively
20+
quickly.
21+
* Providing a multi-user Jupyter Notebook environment for small classes,
22+
teams, or departments.
1223

13-
![JupyterHub single host Docker deployment](internal/jupyterhub-docker.png)
24+
**Disclaimer:** This deployment is **NOT** intended for a production
25+
environment.
26+
27+
28+
## Technical Overview
29+
30+
Key components of this reference deployment are:
31+
32+
* **Host**: Runs the [JupyterHub components](https://jupyterhub.readthedocs.org/en/latest/getting-started.html#overview)
33+
in a Docker container on the host.
34+
35+
* **Authenticator**: Uses [OAuthenticator](https://github.com/jupyter/oauthenticator)
36+
and [GitHub OAuth](https://developer.github.com/v3/oauth/) to
37+
authenticate users.
1438

15-
## Use Cases
39+
* **Spawner**:Uses [DockerSpawner](https://github.com/jupyter/dockerspawner)
40+
to spawn single-user Jupyter Notebook servers in separate Docker
41+
containers on the same host.
1642

17-
Possible use cases for this deployment may include, but are not limited to:
43+
* **Persistence of Hub data**: Persists JupyterHub data in a Docker
44+
volume on the host.
1845

19-
* A JupyterHub demo environment that you can spin up relatively quickly.
20-
* A multi-user Jupyter Notebook environment for small classes, teams, or departments.
46+
* **Persistence of user notebook directories**: Persists user notebook
47+
directories in Docker volumes on the host.
2148

22-
## Disclaimer
49+
![JupyterHub single host Docker deployment](internal/jupyterhub-docker.png)
2350

24-
This deployment is **NOT** intended for a production environment.
2551

2652
## Prerequisites
2753

28-
* This deployment uses Docker for all the things, via [Docker Compose](https://docs.docker.com/compose/overview/).
29-
It requires [Docker Engine](https://docs.docker.com/engine) 1.12.0 or higher.
30-
See the [installation instructions](https://docs.docker.com/engine/installation/) for your environment.
31-
* This example configures JupyterHub for HTTPS connections (the default).
32-
As such, you must provide TLS certificate chain and key files to the JupyterHub server.
33-
If you do not have your own certificate chain and key, you can either
34-
[create self-signed versions](https://jupyter-notebook.readthedocs.org/en/latest/public_server.html#using-ssl-for-encrypted-communication),
35-
or obtain real ones from [Let's Encrypt](https://letsencrypt.org)
36-
(see the [letsencrypt example](examples/letsencrypt/README.md) for instructions).
54+
### Docker
55+
56+
This deployment uses Docker, via [Docker Compose](https://docs.docker.com/compose/overview/), for all the things.
57+
[Docker Engine](https://docs.docker.com/engine) 1.12.0 or higher is
58+
required.
59+
60+
1. Use [Docker's installation instructions](https://docs.docker.com/engine/installation/)
61+
to set up Docker for your environment.
62+
63+
2. To verify your docker installation, whether running docker as a local
64+
installation or using [docker-machine](./docs/docker-machine.md),
65+
enter these commands:
66+
67+
```bash
68+
docker version
69+
docker ps
70+
```
71+
72+
### HTTPS and SSL/TLS certificate
3773

38-
From here on, we'll assume you are set up with docker,
39-
via a local installation or [docker-machine](./docs/docker-machine.md).
40-
At this point,
74+
This deployment configures JupyterHub to use HTTPS. You must provide a
75+
certificate and key file in the JupyterHub configuration. To configure:
4176

42-
docker ps
77+
1. Obtain the domain name that you wish to use for JupyterHub, for
78+
example, `myfavoritesite.com` or `jupiterplanet.org`.
4379

44-
should work.
80+
1. If you do not have an existing certificate and key, you can:
4581

82+
- obtain one from [Let's Encrypt](https://letsencrypt.org) using
83+
the [certbot](https://certbot.eff.org) client,
84+
- use the helper script in this repo's [letsencrypt example](examples/letsencrypt/README.md), or
85+
- [create a self-signed certificate](https://jupyter-notebook.readthedocs.org/en/latest/public_server.html#using-ssl-for-encrypted-communication).
4686

47-
## Setup GitHub Authentication
87+
1. Copy the certificate and key files to a
88+
directory named `secrets` in this repository's root directory. These will be
89+
added to the JupyterHub Docker image at build time. For example, create a
90+
`secrets` directory in the root of this repo and copy the certificate and
91+
key files (`jupyterhub.crt` and `jupyterhub.key`) to this directory:
92+
93+
```bash
94+
mkdir -p secrets
95+
cp jupyterhub.crt jupyterhub.key secrets/
96+
```
97+
98+
99+
## Authenticator setup
48100

49101
This deployment uses GitHub OAuth to authenticate users.
50-
It requires that you create a [GitHub application](https://github.com/settings/applications/new).
51-
You will need to specify an OAuth callback URL in the following form:
52102

53-
```
54-
https://<myhost.mydomain>/hub/oauth_callback
55-
```
103+
It requires that you create and register a [GitHub OAuth application](https://github.com/settings/applications/new)
104+
by filling out a form on the GitHub site:
56105

57-
You must pass the secrets that GitHub provides for your application to JupyterHub at runtime.
58-
You can do this by setting the `GITHUB_CLIENT_ID`, `GITHUB_CLIENT_SECRET`,
59-
and `OAUTH_CALLBACK_URL` environment variables when you run the JupyterHub container,
60-
or you can add them to the `.env` file in the root directory of this repository. For example,
106+
![GitHub OAuth application form](docs/oauth-form.png)
61107

62-
```
63-
GITHUB_CLIENT_ID=<github_client_id>
64-
GITHUB_CLIENT_SECRET=<github_client_secret>
65-
OAUTH_CALLBACK_URL=https://<myhost.mydomain>/hub/oauth_callback
66-
```
108+
In this form, you will specify the OAuth application's callback URL in
109+
this format: `https://<myhost.mydomain>/hub/oauth_callback`.
67110

68-
**Note:** The `.env` file is a special file that Docker Compose uses to lookup environment variables.
69-
If you choose to place the GitHub secrets in this file,
70-
you should ensure that this file remains private
71-
(e.g., do not commit the secrets to source control).
111+
After you submit the GitHub form, GitHub registers your OAuth application and
112+
assigns a unique Client ID and Client Secret. The Client Secret should be
113+
kept private.
72114

73-
## Build the JupyterHub Docker image
115+
At JupyterHub's runtime, you must pass the GitHub OAuth Client ID, Client
116+
Secret and OAuth callback url. You can do this by either:
74117

75-
Configure JupyterHub and build it into a Docker image.
118+
- setting the `GITHUB_CLIENT_ID`, `GITHUB_CLIENT_SECRET`, and
119+
`OAUTH_CALLBACK_URL` environment variables when you run the
120+
JupyterHub container, or
121+
- add them to the `.env` file in the root directory of this repository.
122+
For example, update the following lines in the `.env` file:
76123

77-
1. Copy the TLS certificate chain and key files for the JupyterHub server to a directory named `secrets` within this repository directory. These will be added to the JupyterHub Docker image at build time. If you do not have a certificate chain and key, you can either [create self-signed versions](https://jupyter-notebook.readthedocs.org/en/latest/public_server.html#using-ssl-for-encrypted-communication), or obtain real ones from [Let's Encrypt](https://letsencrypt.org) (see the [letsencrypt example](examples/letsencrypt/README.md) for instructions).
124+
`.env` file
125+
```
126+
GITHUB_CLIENT_ID=<github_client_id>
127+
GITHUB_CLIENT_SECRET=<github_client_secret>
128+
OAUTH_CALLBACK_URL=https://<myhost.mydomain>/hub/oauth_callback
129+
```
78130

79-
```
80-
mkdir -p secrets
81-
cp jupyterhub.crt jupyterhub.key secrets/
82-
```
131+
**Note:** The `.env` file is a special file that Docker Compose uses
132+
to lookup environment variables. If you choose to place the GitHub
133+
OAuth application settings in this file, you should make sure that the
134+
file remains private (be careful to not commit the `.env` file with
135+
these secrets to source control).
83136

84-
1. Create a `userlist` file with a list of authorized users. At a minimum, this file should contain a single admin user. The username should be a GitHub username. For example:
85137

138+
## Build the JupyterHub Docker image
139+
140+
Finish configuring JupyterHub and then build the hub's Docker image. (We'll
141+
build the Jupyter Notebook image in the next section.)
142+
143+
1. Configure `userlist`: Create a `userlist` file of authorized JupyterHub
144+
users. The list should contain GitHub usernames, and this file should
145+
designate at least one `admin` user. For instance, the example file below
146+
contains three users, `jtyberg`, `jenny`, and `guido`, and one designated
147+
administrator, `jtyberg`:
148+
149+
`userlist` file
86150
```
87151
jtyberg admin
152+
jenny
153+
guido
88154
```
89155

90-
The admin user will have the ability to add more users in the JupyterHub admin console.
156+
The admin user will have the ability to add more users through JupyterHub's
157+
admin console.
91158

92-
1. Use [docker-compose](https://docs.docker.com/compose/reference/) to build the
93-
JupyterHub Docker image on the active Docker machine host:
159+
1. Use [docker-compose](https://docs.docker.com/compose/reference/) to build
160+
the JupyterHub Docker image on the active Docker machine host by running
161+
the `make build` command:
94162

95-
```
96-
make build
97-
```
163+
```bash
164+
make build
165+
```
98166

99-
## Prepare the Jupyter Notebook Image
167+
168+
## Spawner: Prepare the Jupyter Notebook Image
100169

101170
You can configure JupyterHub to spawn Notebook servers from any Docker image, as
102171
long as the image's `ENTRYPOINT` and/or `CMD` starts a single-user instance of
@@ -126,36 +195,48 @@ Notebook server that is compatible with JupyterHub).
126195

127196
You can pull the image using the following command:
128197

129-
```
198+
```bash
130199
make notebook_image
131200
```
132201

202+
133203
## Run JupyterHub
134204

135205
Run the JupyterHub container on the host.
136206

137207
To run the JupyterHub container in detached mode:
138208

139-
```
209+
```bash
140210
docker-compose up -d
141211
```
142212

143213
Once the container is running, you should be able to access the JupyterHub console at
144214

215+
**file**
145216
```
146217
https://myhost.mydomain
147218
```
148219

149220
To bring down the JupyterHub container:
150221

151-
```
222+
```bash
152223
docker-compose down
153224
```
154225

226+
---
227+
155228
## Behind the scenes
156229

157230
`make build` does a few things behind the scenes, to set up the environment for JupyterHub:
158231

232+
### Create a JupyterHub Data Volume
233+
234+
Create a Docker volume to persist JupyterHub data. This volume will reside on the host machine. Using a volume allows user lists, cookies, etc., to persist across JupyterHub container restarts.
235+
236+
```bash
237+
docker volume create --name jupyterhub-data
238+
```
239+
159240
### Create a Docker Network
160241

161242
Create a Docker network for inter-container communication. The benefits of using a Docker network are:
@@ -165,25 +246,19 @@ Create a Docker network for inter-container communication. The benefits of usin
165246

166247
Here we create a Docker network named `jupyterhub-network`. Later, we will configure the JupyterHub and single-user Jupyter Notebook containers to run attached to this network.
167248

168-
```
249+
```bash
169250
docker network create jupyterhub-network
170251
```
171252

172-
### Create a JupyterHub Data Volume
173-
174-
Create a Docker volume to persist JupyterHub data. This volume will reside on the host machine. Using a volume allows user lists, cookies, etc., to persist across JupyterHub container restarts.
175-
176-
```
177-
docker volume create --name jupyterhub-data
178-
```
253+
---
179254

180255
## FAQ
181256

182257
### How can I view the logs for JupyterHub or users' Notebook servers?
183258

184259
Use `docker logs <container>`. For example, to view the logs of the `jupyterhub` container
185260

186-
```
261+
```bash
187262
docker logs jupyterhub
188263
```
189264

@@ -197,14 +272,15 @@ JupyterHub reads the Notebook image name from `jupyterhub_config.py`, which
197272
reads the Notebook image name from the `DOCKER_NOTEBOOK_IMAGE` environment
198273
variable:
199274

200-
```
275+
```python
201276
# DockerSpawner setting in jupyterhub_config.py
202277
c.DockerSpawner.container_image = os.environ['DOCKER_NOTEBOOK_IMAGE']
203278
```
204279

205280
By default, the`DOCKER_NOTEBOOK_IMAGE` environment variable is set in the
206281
`.env` file.
207282

283+
**file**
208284
```
209285
# Setting in the .env file
210286
DOCKER_NOTEBOOK_IMAGE=jupyter/scipy-notebook:2d878db5cbff
@@ -216,7 +292,7 @@ by setting the `DOCKER_NOTEBOOK_IMAGE` variable to a different Notebook
216292
image in the environment where you launch JupyterHub. For example, the
217293
following setting would be used to spawn single-user `pyspark` notebook servers:
218294

219-
```
295+
```bash
220296
export DOCKER_NOTEBOOK_IMAGE=jupyterhub/pyspark-notebook:2d878db5cbff
221297

222298
docker-compose up -d
@@ -244,25 +320,25 @@ There are multiple ways to [backup and restore](https://docs.docker.com/engine/u
244320

245321
Suppose you have the following running containers:
246322

247-
```
248-
docker ps --format "table {{.ID}}\t{{.Image}}\t{{.Names}}"
323+
```bash
324+
docker ps --format "table {{.ID}}\t{{.Image}}\t{{.Names}}"
249325

250-
CONTAINER ID IMAGE NAMES
251-
bc02dd6bb91b jupyter/minimal-notebook jupyter-jtyberg
252-
7b48a0b33389 jupyterhub jupyterhub
326+
CONTAINER ID IMAGE NAMES
327+
bc02dd6bb91b jupyter/minimal-notebook jupyter-jtyberg
328+
7b48a0b33389 jupyterhub jupyterhub
253329
```
254330

255331
In this deployment, the user's notebook directories (`/home/jovyan/work`) are backed by Docker volumes.
256332

257-
```
258-
docker inspect -f '{{ .Mounts }}' jupyter-jtyberg
333+
```bash
334+
docker inspect -f '{{ .Mounts }}' jupyter-jtyberg
259335

260-
[{jtyberg /var/lib/docker/volumes/jtyberg/_data /home/jovyan/work local rw true rprivate}]
336+
[{jtyberg /var/lib/docker/volumes/jtyberg/_data /home/jovyan/work local rw true rprivate}]
261337
```
262338

263339
We can backup the user's notebook directory by running a separate container that mounts the user's volume and creates a tarball of the directory.
264340

265-
```
341+
```bash
266342
docker run --rm \
267343
-u root \
268344
-v /tmp:/backups \

‎docs/oauth-form.png‎

244 KB
Loading

0 commit comments

Comments
 (0)