Detailed explanation of Docker+Jenkins+Gitlab+Django application deployment practice

Detailed explanation of Docker+Jenkins+Gitlab+Django application deployment practice

1. Background

In the context of rapid updates and iterations of Internet applications, traditional manual work or simple scripts can no longer adapt to this change. Devops provides us with a good solution. The application of CI/CD can greatly facilitate our daily work. Automated and fast continuous integration/continuous delivery brings us faster application development, better stability and stronger reliability.

2. Topological Environment

2.1 Architecture Topology

As shown in the example above, the following process topology is briefly described:

  • When the developer pushes the local code to gitlab-server, the webhook automatically triggers Jenkins to build the application
  • Deploy the application on the docker host git clone the source code from gitlabserver and start the application
  • LB can be placed at the front end for high availability
  • Database connection cloud database
  • The logs can be stored in log and delivered to elk later to realize log visualization
  • Build completion email notification to relevant personnel (testing or opening)

2.2 System software version

name Version
Linux CentOS 7.3 64-bit
Docker 1.13
Django 2.0

3. Installation and Deployment

3.1 Jenkins installation and deployment

For Jenkins installation and deployment, please refer to: Jenkins Notes

After the installation is complete, add the Docker target server

Configuring the outgoing mail server

3.2 Docker installation and deployment

For more information on Docker installation and deployment and Dockerfile writing, please refer to: Docker Container Detailed Explanation

3.3 Gitlab installation and deployment

GitLab can be installed on a public Linux server by running some commands. If there is no public network, you need to manually modify external_url 'http://自己的內網IP' in the /etc/gitlab/gitlab.rb file

yum install -y libsemanage-static libsemanage-devel policycoreutils openss
h-server openssh-clients postfix
systemctl enable postfix && systemctl start postfix

wget https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el7/gitlab-ce-8.0.
0-ce.0.el7.x86_64.rpm
rpm -i gitlab-ce-8.0.0-ce.0.el7.x86_64.rpm
# Get the public IP
PUBLICIP=$(curl http://ipv4.icanhazip.com)
# Modify sed -i "s/gitlab-server/${PUBLICIP}/g" /etc/gitlab/gitlab.rb

gitlab-ctl reconfigure
gitlab-ctl restart

echo "Username:root"
echo "Password:5iveL!fe"

3.4 Configuration Release Process

Jenkins creates a free-style software project

Use parameterized builds to facilitate subsequent deployment of Docker imported mapping source ports and releases

The source code comes from the Django project of gitlab

Use webhook to connect gitlab and jenkins

Jenkins installation plugin:

Generate a random token value

Configure the GitLab webhook URL generated by jenkins to gitlab

When the developer pushes the code locally, Jenkins will automatically trigger the project build. There is a git pull code written in the Dockerfile. Again, there is no need to distribute the code from Jenkins to the Docker host. Jenkins is used as a trigger for Docker builds.

Configure the email after the build is completed

Email template, email type selection:

Content type selection: HTML

Fill in the email subject:構建通知:${BUILD_STATUS} - ${PROJECT_NAME} - Build # ${BUILD_NUMBER} !

Build the notification template:

<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>${ENV, var="JOB_NAME"}-${BUILD_NUMBER} build log</title>
</head>

<body leftmargin="8" marginwidth="0" topmargin="8" marginheight="4"
 offset="0">
 <table width="95%" cellpadding="0" cellspacing="0"
 style="font-size: 11pt; font-family: Tahoma, Arial, Helvetica, sans-serif">
 <tr>
  <td>(This email is automatically sent by the program, please do not reply!)</td>
 </tr>
 <tr>
  <td><h2>
   <font color="#0000FF">Build Result - ${BUILD_STATUS}</font>
  </h2></td>
 </tr>
 <tr>
  <td><br />
  <b><font color="#0B610B">Build Information</font></b>
  <hr size="2" width="100%" align="center" /></td>
 </tr>
 <tr>
  <td>
  <ul>
   <li>Project name: ${PROJECT_NAME}</li>
   <li>Build number: ${BUILD_NUMBER} build</li>
   <li>SVN version: ${SVN_REVISION}</li>
   <li>Trigger reason: ${CAUSE}</li>
   <li>Build log: <a href="${BUILD_URL}console">${BUILD_URL}console</a></li>
   <li>Build URL: <a href="${BUILD_URL}">${BUILD_URL}</a></li>
   <li>Working directory: <a href="${PROJECT_URL}ws">${PROJECT_URL}ws</a></li>
   <li>Project URL: <a href="${PROJECT_URL}">${PROJECT_URL}</a></li>
  </ul>
  </td>
 </tr>
 <tr>
  <td><b><font color="#0B610B">Changes Since Last
   Successful Build:
  <hr size="2" width="100%" align="center" /></td>
 </tr>
 <tr>
  <td>
  <ul>
   <li>Historical changes: <a href="${PROJECT_URL}changes">${PROJECT_URL}changes</a></li>
  </ul> ${CHANGES_SINCE_LAST_SUCCESS,reverse=true, format="Changes for Build #%n:<br />%c<br />",showPaths=true,changesFormat="<pre>[%a]<br />%m</pre>",pathFormat=" %p"}
  </td>
 </tr>
 <tr>
  <td><b>Failed Test Results</b>
  <hr size="2" width="100%" align="center" /></td>
 </tr>
 <tr>
  <td><pre
   $FAILED_TESTS
  <br /></td>
 </tr>
 <tr>
  <td><b><font color="#0B610B">Build log (last 100 lines):</font></b>
  <hr size="2" width="100%" align="center" /></td>
 </tr>
 <!-- <tr>
  <td>Test Logs (if test has ran):
  href="${PROJECT_URL}ws/TestResult/archive_logs/Log-Build-${BUILD_NUMBER}.zip">${PROJECT_URL}/ws/TestResult/archive_logs/Log-Build-${BUILD_NUMBER}.zip</a>
  <br />
  <br />
  </td>
 </tr> -->
 <tr>
  <td><textarea cols="80" rows="30" readonly="readonly"
   style="font-family: Courier New">${BUILD_LOG, maxLines=100}</textarea>
  </td>
 </tr>
 </table>
</body>
</html>

The trigger type can be filled in according to your needs. Here, fill in always to send an email regardless of success or failure.

View files in the remote Docker server

Django deployment has used conda to package the project's Python 3.6 environment package to create a Docker image.

Previously, a pure Python 3.6 system was used. During each build, pip was used to install the modules in requirements.txt. However, due to the seldom changes in the environment over time, pip installation was time-consuming each time. Therefore, conda was used to customize the packaged Python environment to reduce the environment deployment time. The local disk can also be mounted in the environment through the -v parameter of docker image creation. Each time, a local conda can be built to complete the rapid environment deployment.

View Dockerfile

FROM 87a69025db6a
MAINTAINER kaliarch

# Define the working directory ENV WORK_DIR /work/ in docker
# Create a working directory in docker RUN mkdir $WORK_DIR
# Define the mapping port EXPOSE 80

WORKDIR $WORK_DIR 
RUN git clone http://123.xxxx.xxxxx.245/Devops/go2cloud.git

# Add startup service script ADD *.sh ${WORK_DIR}

CMD `which bash` /work/start_all.sh && tail -f /work/logs/server-$(date +%F).log

View the Django startup script

#!/bin/bash

BASEPATH=$(cd `dirname $0`;pwd)

PY_CMD=/python3/bin/python

# Service entry file #MAIN_APP=${BASEPATH}/go2cloud/manage.py 
# Migration script entry file SCRIPTS_APP=${BASEPATH}/go2cloud/scripts/migrate_task_schdule.py
# Delete the script entry file DELETE_APP=${BASEPATH}/go2cloud/scripts/delete_transfer_server.py

# Log directory LOG_DIR=${BASEPATH}/logs/
[ ! -d ${LOG_DIR} ] && mkdir ${LOG_DIR}

# Start the service #nohup ${PY_CMD} -u ${MAIN_APP} runserver 0.0.0.0:80 >> ${LOG_DIR}server-$(date +%F).log 2>&1 &
# Start the script migration scheduling script echo "---------$0 $(date) excute----------" >> ${LOG_DIR}task-script-$(date +%F).log
nohup ${PY_CMD} -u ${SCRIPTS_APP} >> ${LOG_DIR}script-$(date +%F).log 2>&1 &

# Start the migration and deletion script echo "---------$0 $(date) excute----------" >> ${LOG_DIR}delete-script-$(date +%F).log
nohup ${PY_CMD} -u ${DELETE_APP} >> ${LOG_DIR}delete-script-$(date +%F).log 2>&1 &

View the Jenkins deployment script

#!/bin/bash

release=$1
port=$2

BASEPATH=$(cd `dirname $0`;pwd)

# Build go2cloud-platform image cd /dockerwork
docker build -t go2cloud-platform-mini:$release .

IMGNAME=$(docker images|awk -v release=$release '{if($1=="go2cloud-platform-mini" && $2==release) print $3}')

echo $IMGNAME
# Start the container docker run -d -p ${port}:80 -v /testlog/:/work/logs ${IMGNAME}

Use the -v parameter to store logs persistently on the Docker host

4. Test display

4.1 Test Build

Manually build tests

4.2 View log

4.3 View Docker Container

4.4 Test the app

5. Reflection and Improvement

  • The database currently connected is a database built on a cloud server. Later, the database also uses docker, and multiple groups use docker-compose for unified deployment and management.
  • Later, you can use the public cloud k8s cluster for convenient testing
  • Currently, the logs generated by the docker container are on the docker host. Later, they can be stored on cos and then delivered to the elk cluster for log visualization processing.
  • Manage images in a unified way and create a local image repository
  • Gitlab adds code review and combines it with automatic testing

The above is the full content of this article. I hope it will be helpful for everyone’s study. I also hope that everyone will support 123WORDPRESS.COM.

You may also be interested in:
  • Docker+gitlab+jenkins builds automated deployment from scratch
  • Setting up GitLab+Jenkins continuous integration environment under centos (installing Jenkins)
  • Docker Gitlab+Jenkins+Harbor builds a persistent platform operation
  • jenkins+gitlab+nginx deployment of front-end application
  • Jenkins integrates Gitlab to realize the whole process record of automated deployment

<<:  MySQL deduplication methods

>>:  How to generate Vue user interface by dragging and dropping

Recommend

7 native JS error types you should know

Table of contents Overview 1. RangeError 2. Refer...

mysql5.7.18.zip Installation-free version configuration tutorial (windows)

This is the installation tutorial of mysql5.7.18....

Using CSS3 to achieve transition and animation effects

Why should we use CSS animation to replace JS ani...

Simple steps to create a MySQL container with Docker

Preface We have already installed Docker and have...

VMwarea virtual machine installation win7 operating system tutorial diagram

The installation process of VMwarea will not be d...

Front-end JavaScript thoroughly understands function currying

Table of contents 1. What is currying 2. Uses of ...

Install Zookeeper under Docker (standalone and cluster)

After starting Docker, let's take a look at t...

Summary of MySQL common functions

Preface: The MySQL database provides a wide range...

MySQL log settings and viewing methods

MySQL has the following logs: Error log: -log-err...

The problem and solution of using docker storage and causing Exit to cause files to fail to upload to the server

1. Problem description The storage installed in t...

Example method of deploying react project on nginx

Test project: react-demo Clone your react-demo pr...

WeChat applet wxs date and time processing implementation example

Table of contents 1. Timestamp to date 2. Convert...

Docker Stack deployment method steps for web cluster

Docker is becoming more and more mature and its f...

About VSCode formatting JS automatically adding or removing semicolons

introduction It is okay to add or not add a semic...

An example of the execution order between the href jump and onclick of the html hyperlink a tag

The execution relationship between the href jump ...