How to use Docker-compose to deploy Django applications offline

Our intranet environment needs to deploy an application similar to CMS, which has functions such as CRUD of some tables, data export, and personnel authority management. Thinking that Django is good at this kind of work and the development workload is not large, I chose Django as the development basis. The development function is relatively simple, and almost all of the above functions are achieved using plug-ins such as xadmin. But there is a problem that we cannot get around, that is, deployment to an intranet environment, where tools such as pip cannot be used. Fortunately, there is a yum server available in the intranet, so we decided to install Docker on the intranet server, and then copy the container of the development environment to the production environment for deployment. Here are the main steps:

1. Install Docker-ce for the development environment
2. Install Docker-compose for the development environment
3. Configuring the development environment
4. Save Container
5. Install Docker-ce and docker-compose for production environment
6. Send the container file and run

Note: My development environment here is Ubuntu 18.04 and the production environment is Centos 7.2. If you are using another environment, please check the differences yourself and use the commands that are appropriate for your system.

Install Docker-ce for the development environment

Docker and Docker-compose are the key points of this deployment. I will try to reduce the Django application part. Docker is responsible for the underlying part of container virtualization. Docker-compose is a container orchestration tool. With it, we don't need to write shells to connect containers. Let's install Docker-ce first. Here we mainly refer to the official documentation of Docker. If what I wrote is not detailed enough or is outdated, you can go to the official website to view more authoritative and updated documents.

Uninstall old versions

Before installation, you need to uninstall the old version of Docker. If you are using a new system, you can ignore this step.

$ sudo apt remove docker docker-engine docker.io containerd runc

Install the apt repository used

Update apt package index

$ sudo apt update

Allow apt to access the repository via https

$ sudo apt install \
  apt-transport-https \
  ca-certificates \
  curl \
  gnupg-agent \
  software-properties-common

Add Docker's official GPG key

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Add Docker repository

$ sudo add-apt-repository \
  "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) \
  stable"

Install Docker-ce

After making the above preparations, installing Docker-ce is easy. If you are familiar with Ubuntu, you can install it quickly.

$ sudo apt update
$ sudo apt install -y docker-ce

Once the installation is complete, start the docker service and enable it to launch every time at system boot.

$ sudo systemctl start docker
$ sudo systemctl enable docker

Install Docker-compose for the development environment

After Docker-ce is installed, Docker-compose is easy to use. If you are on a Linux or other platform, you can directly download the compiled binary file of Docker-compose and use it.

After downloading, modify the permissions and add executable

$ sudo chmod +x /usr/local/bin/docker-compose

Finally, you can check the version number of Docker-compose to verify whether it is successfully installed.

$ docker-compose --version
docker-compose version 1.24.0-rc1, build 0f3d4dda

Configuring the development environment

The development environment here is the Django environment. For the convenience of demonstration, I try to use a newly created Django project.

Create a new Django project

Create a new Django project and first create a top-level folder to put the project files in this folder. The directory structure is roughly as follows:

--project
 --Dockerfile
 --docker-compose.yml
 --mysite
  --manage.py
  --requirements.txt

First create a project folder

$ mkdir project

Then create a new Django project, or you can copy an existing project.

$ django-admin.py startproject mysite

Generate requirements.txt file

In the previous step, there is already a Django project called mysite. Suppose we put requirements.txt in this folder, and the content is as follows:

$ cat requirements.txt

defusedxml==0.5.0
diff-match-patch==20181111
Django==2.1.7
django-crispy-forms==1.7.2
django-formtools==2.1
django-import-export==1.2.0
django-reversion==3.0.3
et-xmlfile==1.0.1
future==0.15.2
httplib2==0.9.2
jdcal==1.4
odfpy==1.4.0
openpyxl==2.6.0
pytz==2018.9
PyYAML==3.13
six==1.10.0
tablib==0.12.1
unicodecsv==0.14.1
xadmin==0.6.1
xlrd==1.2.0
xlwt==1.3.0
mysqlclient==1.4.2

Of course, these are the dependencies my project needs, your dependencies may be different from mine.

Create a new Dockerfile

Now that we have the project and its dependency files, the next step is to create a Docker image of the running environment for our Django project. First, create a Dockerfile to build the Docker image. Create a new Dockerfile in the project folder with the following content:

$ cat Dockerfile

FROM python:3.6.8
ENV PYTHONUNBUFFERED 1

RUN mkdir /config
ADD /mysite/requirements.txt /config/
RUN pip install -r /config/requirements.txt
RUN mkdir /src
WORKDIR /src/mysite

Let me briefly explain this file.

FROM python:3.6.8

The base image I use here is python: 3.6.8. Its base image is Ubuntu, which I am more familiar with. If you are more familiar with alpine, you can also use alpine, which is much smaller.

ENV PYTHONUNBUFFERED 1

You can create environment variables for any operating system using the Env keyword.

ENV PYTHONUNBUFFERED 1

For example, if you're using it to store your Django keys, you might write:

ENV DJANGO_SECRET_KEY l!fafmjcqyn+j+zz1@2@wt$o8w8k(_dhgub%41l#k3zi2m-b%m

Use it in your code like this:

import os
SECRET_KEY = os.environ['DJANGO_SECRET_KEY']

As the name suggests, RUN runs commands in the container. Here, the RUN command creates two folders, /config and /src, and installs Python's dependency environment.

RUN mkdir /config
RUN mkdir /src
RUN pip install -r /config/requirements.txt

ADD

ADD /mysite/requirements.txt /config/

Add local files to the container's WORKDIR

WORKDIR /src/mysite

It specifies the default path for all subsequent commands to be run in the container. The commands to be run can be seen in the docker-compose file later.

Create a new docker-compose script

Docker-compose can be used to manage multiple containers. The work of manually adding massive parameters to run containers and connecting containers can now be done by Docker-compose. The content of my docker-compose.yml is as follows:

$ cat docker-compose.yml

version: '3'
services:
  db:
   image:mysql:5.7
   container_name: mysite_db
   ports:
    - "3306:3306"
   environment:
    MYSQL_ROOT_PASSWORD: mysite
    MYSQL_DATABASE: mysite
    LANG: C.UTF-8
  web:
   build: .
   container_name: mysite_web
   command: bash -c "python manage.py makemigrations && python manage.py migrate && python manage.py runserver 0.0.0.0:8000"
   depends_on:
    -db
   volumes:
    - ./mysite:/src
   restart: always
   ports:
    - "8002:8000"

Let me briefly explain this docker-compose file.

version: '3'

Refers to the version of docker-compose. Different versions support slightly different configuration items. Services managed by services. In our example, there are two services: db and web. I will explain the configuration items in the two services separately:

image directly uses the existing image from docker hub or local. This one uses MySQL5.7. container_name specifies the name of the container. ports specifies the port mapping of the container to the host. The port in front is the host port, and the port in the back is the container port. environment specifies the environment in which the current service is running. For details of the environment, refer to the description of the current image. We can configure the ones that are supported. In this case we specify the MySQL root password, the default database, and the database character set. web: build compiles the image. Here, the Dockerfile in the current folder is used. command is the command executed after the container is started. depends_on is the service that the current container depends on. That is, the service in the dependency must be started successfully before the current service can be started. volumes is the volume to be mounted by the current container. The front part refers to the host directory, and the back part is the container directory. restart specifies the restart strategy of the container. In the current case, it will keep restarting if an error occurs. Here, the container's port 8000 is mapped to the host's port 8002, and the web service is accessed from port 8002.

Configuring the Django Project

Now modify the settings.py file of the mysite project according to the current container environment.

$ vim mysite/mysite/settings.py

Find the ALLOW_HOSTS section in the file and add "web" to it, with the following content:

ALLOW_HOSTS = [
  ...
  'web'
]

Then modify the DATABASES part in settings.py and change the parameters to the parameters of the MySQL service db. The content is as follows:

DATABASES = {
  'default': {
    'ENGINE': 'django.db.backends.mysql',
    'NAME': 'mysite',
    'USER': 'root',
    'PASSWORD': 'mysite',
    'HOST': 'db'
  }
}

The MySQL connection parameters here are defined in the environment of the db part of the docker-compose.yml file. It is worth pointing out that the parameter HOST value is db. After docker-compose starts the containers, these containers will be connected. The containers can ping each other using the service name, just like using the domain name, so the "HOST" here can be directly filled in with "db".

Build the project using Docker-compose

After the above efforts, we are basically ready. We can build our image. There are two services here. DB only needs to download or use the local image at runtime, and web also needs to be built using Dockerfile.

$ docker-compose build

After a while of downloading or building, you can see the information that the image was successfully built.

Run the project and test it

After the build is complete, we have an image of the web service. We now use docker-compose to start the service.

$ docker-compose up -d

This process may also take a while, depending on your network speed. It will download the MySQL image, construct a container based on the db and web images, and run the container. After completion, you can use docker-compose ps and docker-compose images to view the containers and images we generated

$ docker-compose ps

Name Command State Ports       
---------------------------------------------------------------------------------------
mysite_db docker-entrypoint.sh mysqld Up 0.0.0.0:3306->3306/tcp, 33060/tcp
mysite_web bash -c python manage.py m ... Up 0.0.0.0:8002->8000/tcp

$ docker-compose images

Container Repository Tag Image Id Size
--------------------------------------------------------
mysite_db mysql 5.7 e47e309f72c8 355 MB
mysite_web mysite_web latest 3989acbcc3c9 938 MB

You can also use docker-compose to stop and start services. For more specific usage, please refer to the official documentation.

$ docker-compose start
Starting db ... done
Starting web ... done

$ docker-compose stop
Stopping mysite_web ... done
Stopping mysite_db ... done

You can see that the order of stopping and starting the services here is regular. When starting, the dependent service starts first and then the service that depends on it is started. When starting the service, it is just the opposite. After the service is running normally, you can access the browser to test whether the service is started normally.

Save Container

If everything is normal with the service, we need to save the current container and prepare for deployment to the new platform. Note: Use save to save the image. Save includes information such as the connection status between containers. If you use export to export the image to the production environment, you cannot use docker-compose to restore the service.

$ docker save -o mysql.tar mysql:5.7
$ docker save -o mysite.tar mysite_web:latest

When the above command is executed successfully, two tar files will be generated in the current directory, together with the Dockerfile and docker-compose.yml files in the project directory, ready to be migrated to the production machine.

Install Docker-ce and docker-compose for production environment

Since the production environment is CentOS, you can directly use yum to install

$ sudo yum install docker-ce

After successful installation, deploy docker-compose to the production server by referring to the development environment.

Send the container file and run

Use scp or other tools to send mysql.tar, mysite.tar, Docker-compose.yml, and project folders to the production server, and find a suitable folder to store these files, keeping the original directory structure. Let's first restore the two images to the production server

$ docker load -i mysql.tar
$ docker load -i mysite_web.tar

Wait for a while for the execution to complete, and you can see that the current server already has these two images.

REPOSITORY TAG IMAGE ID CREATED SIZE
mysite_web latest 3989acbcc3c9 2 days ago 983MB
mysql 5.7 e47e309f72c8 3 weeks ago 372MB

Before executing the build container, we need to make a simple modification to docker-compose.yml. You may have noticed that the production server has no internet, so we cannot build the image anymore. We also copied the image of the development environment as it is, so this time the web service can be run from the image. The content is as follows:

version: '3'
services:
  db:
   ...
  web:
   image: mysite_web:latest
   ...

Just delete the build item in the web and add an image item, and the content will be the image we copied. Later we can build the container and start the service.

$ docker-compose up -d

result

Name Command State Ports       
----------------------------------------------------------------------------------------
mysite_web bash -c python manage.py m ... Up 0.0.0.0:8002->8000/tcp      
mysite_db docker-entrypoint.sh mysqld Up 0.0.0.0:3306->3306/tcp, 33060/tcp

Open the browser again to see if it starts normally.

postscript

There are many more uses for docker-compose, and I will make more in-depth introductions in other directions in future projects. Thank you for viewing my work, I hope it helps you a little.

Reference Documentation

Get Docker CE for Ubuntu
Install Docker Compose

The above is the full content of this article. I hope it will be helpful for everyone’s study. I also hope that everyone will support 123WORDPRESS.COM.