How to configure Jupyter notebook in Docker container

Jupyter notebook is configured under the docker container, mainly for writing python code, more specifically for deep learning development.

The most efficient way to use Jupyter Web is to deploy it on the cloud. Whether it is a CPU cloud server or a GPU cloud server, it can be started and used quickly.

The emergence of docker has made deployment and use much more convenient.

- Install Docker

Docker is divided into Docker CE and Docker EE. Generally, Docker CE (community version) is used.

Docker can be installed on Linux (Ubuntu, CentOS), MacOS, Windows or Raspberry Pi. It is generally used under Linux, and I personally like the Ubuntu system. So let's introduce how to install docker under ubutnu.

First remove any old versions that may exist on your machine:

$ sudo apt-get remove docker docker-engine docker.io

Installing optional kernel modules

To reduce the installation size of the kernel package, starting from Ubuntu 14.04, some kernel modules have been moved to optional kernel module packages (linux-image-extra-*). A normally installed system should include the optional kernel module packages, while some trimmed systems may have them omitted. The AUFS kernel driver is part of the optional kernel module. As the recommended Docker storage layer driver, it is generally recommended to install the optional kernel module package to use AUFS.

$ sudo apt-get update
$ sudo apt-get install linux-image-extra-$(uname -r) linux-image-extra-virtual

Docker CE on Ubuntu 16.04 and above uses the overlay2 storage layer driver by default, and no manual configuration is required.

Certificate and key preparation

Because the apt source uses HTTPS to ensure that the software is not tampered with during the download process. Therefore, we first need to add the package to be transferred using HTTPS along with the CA certificate.

$ sudo apt-get update
$ sudo apt-get install apt-transport-https ca-certificates curl software-properties-common

Due to domestic network issues, it is strongly recommended to use domestic sources. Please see the official sources in the comments.

$ curl -fsSL https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu/gpg | sudo apt-key add -
# Official source# $ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Finally, add the Docker software source:

$ sudo add-apt-repository \
 "deb [arch=amd64] https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu \
 $(lsb_release -cs) \
 stable"
# Official source# $ sudo add-apt-repository \
# "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
# $(lsb_release -cs) \
# stable

The above command will add the stable version of Docker CE APT mirror source. If you need a test or daily build version of Docker CE, please change stable to test or nightly.

Install Docker CE

$ sudo apt-get update
$ sudo apt-get install docker-ce

Start Docker CE

$ sudo systemctl enable docker
$ sudo systemctl start docker

For Ubuntu 14.04, use the following command to start:

$ sudo service docker start

Create a docker user group

For security reasons, the root user is generally not used directly on Linux systems. Therefore, a better approach is to add users who need to use docker to the docker user group.

$ sudo groupadd docker
$ sudo usermod -aG docker $USER

Test whether Docker is installed correctly

$ docker run hello-world

Pull the hello-world image for testing,

Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
d1725b59e92d: Pull complete
Digest: sha256:0add3ace90ecb4adbf7777e9aacf18357296e799f81cabc9fde470971e499788
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
 (amd64)
 3. The Docker daemon creates a new container from that image which runs the
 executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
 to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

If the above information can be output normally, it means the installation is successful.

Image Accelerator

Sometimes it is difficult to pull images from Docker Hub in China. In this case, you can configure an image accelerator. Docker officially and many domestic cloud service providers provide domestic accelerator services such as Alibaba Cloud, Qiniu Cloud, etc.

Ubuntu 14.04 system:

Edit /etc/default/docker and configure the accelerator address in DOCKER_OPTS:

DOCKER_OPTS="--registry-mirror=https://registry.docker-cn.com"

Then restart the service:

$ sudo service docker restart

Ubuntu 16.04+:

Please write the following content in /etc/docker/daemon.json (if the file does not exist, please create a new one):

{
 "registry-mirrors": [
 "https://registry.docker-cn.com"
 ]
}

Then restart the service:

$ sudo systemctl daemon-reload
$ sudo systemctl restart docker

Check whether the accelerator is effective

Run the docker info command on the command line. If you see the following output, the configuration is successful.

Registry Mirrors:
 https://registry.docker-cn.com/

- Common Docker commands

First of all, before using the docker command, you need to distinguish the concepts of image and container. Recommended reference links

The same image starts multiple Docker containers. After these containers are started, they are active and isolated from each other. If you want to keep the existing system environment after operating on a container, you need to submit and save it.

Start command: docker run

For example, the following command prints "Hello World" and then terminates the container.

$ docker run ubuntu:18.04 /bin/echo 'Hello world'
Hello world

This is almost indistinguishable from executing /bin/echo 'hello world' locally.

The following command starts a bash terminal, allowing user interaction.

$ docker run -it ubuntu:16.04 /bin/bash
root@af8bae53bdd3:/#

The -i option keeps the container's standard input open, and the -t option lets Docker allocate a pseudo-terminal (pseudo-tty) and bind it to the container's standard input.

The status of the container mainly includes:

created: has been created (can be listed using the docker ps -a command) but has not yet been started (cannot be listed using the docker ps command)

running: running

paused: The container process is paused

restarting: The container process is in the process of restarting

exited: The stopped state in the figure above means that the container has been running before but is now stopped (to be distinguished from the created state, which refers to a newly created container that has not yet been run). You can use the start command to put it back into the running state.

destroyed: The container has been deleted and no longer exists

View the image pulled by the current system:

$docker images

View all started (Up status) containers in the current system

$ docker container ls

or

$ docker ps

View all containers in the current system

$ docker container ls -a

or

$ docker ps -a

To terminate a container:

$ docker container stop (id or name)

Or exit the terminal using the exit command or Ctrl+d to stop the container.

Enter the container:

$docker attach (id or name)

or

$docker exec (id or name)

$docker attach exits from this stdin, which will cause the container to stop.

$docker exec exits from this stdin, which will not cause the container to stop.

It is recommended to use $docker exec

Delete a container: $docker container rm (id or name)

Clean up all containers in the terminated state $ docker container prune

- Install jupyter

First enter the container:

$ docker run -i -t ubuntu:16.04 /bin/bash

This process is basically the same as installing Jupyter on the Ubuntu system, but the Ubuntu in the container is a minimalist environment and the python-dev package is not installed.

#Update apt-get environment apt-get update

#Install the python dev package apt-get install python-dev

#Install jupyter
pip install jupyter

By default, jupyter can only be accessed through a local address. You need to loosen the configuration to allow jupyter to be accessed remotely. When enabling remote access, you need to set a password. Jupyter's configuration file only supports encrypted ciphertext passwords.

#Generate jupyter configuration file, this will generate the configuration file .jupyter/jupyter_notebook_config.py
jupyter notebook --generate-config

#Use ipython to generate a password In [1]: from notebook.auth import passwd
In [2]: passwd()
Enter password: 
Verify password: 
Out[2]: 'sha1:******'

#Modify the following parameters in the configuration file .jupyter/jupyter_notebook_config.py c.NotebookApp.ip='*' #Bind all addresses c.NotebookApp.password = u'The password just generated'
c.NotebookApp.open_browser = False #Whether to automatically open in the browser after startup c.NotebookApp.port =8888 #Specify an access port, the default is 8888, pay attention to the corresponding mapped docker port

After the configuration is complete, you can start Jupyter with the jupyter notebook command. If you use the root user directly in the container, the command to start Jupyter is jupyter notebook --allow-root.

The final command is:

docker run -it --name jupytertest -p 8888:8888 -v ~/mnt:/mnt jupyter-ubuntu:v1 su root -c 'jupyter notebook --allow-root'

-p is port mapping, -v is path mount mapping.

After successful startup, use http://ubuntu-ip:8888 to access.

Supplement: Create a remote jupyter on docker

Use command inside docker container on remote server

jupyter notebook --port=8888 --allow-root

You can run jupyter.

One thing to note is that if the port of the current docker container is 8888, you can omit --port=8888. If the port of the current docker container is not 8888, you need to specify the port to be consistent with the port of the current docker container when running jupyter.

If you forget the port of the current docker container, you can use the command outside docker

docker ps

Check.

After running jupyter, a bunch of log-like things will be output on the interface. as follows:

Keep the current interface and record the token (the part outlined in red).

After running, enter the remote server IP in the local browser: the port number of the docker container running jupyter. For example: 192.168.0.101:8888.

The login interface is as follows:

If you don't want to set a password, you can log in directly using the token.

If you want to set a password, you can use token to set the password. Next time you log in, you can use the password directly instead of the token.

The above is my personal experience. I hope it can give you a reference. I also hope that you will support 123WORDPRESS.COM. If there are any mistakes or incomplete considerations, please feel free to correct me.