Using nginx + fastcgi to implement image recognition server

background

A specific device is used to perform inference on the deep learning model. The machine only provides a C++ encapsulated API for loading, starting and inferring the model. Caffe is still used for model training. The model needs to be converted into a format supported by the device. The conversion of the model will not be introduced here. In order to make the model reasoning into a service, we had to use C++ to build an HTTP service, so that the user could post a picture through the HTTP service, the server would start the model reasoning, implement the model prediction, and return the result to the client.

Overall framework

The service content of the short service is to pre-process the received images and then perform model inference. What needs to be done now is to introduce HTTP service

Preliminary research

For a C++ novice, the initial research is of course to search first. The results of http server c++ search processing are also varied. Some teach you how to implement an http server, some use a third-party library, and some directly deal with a bunch of code. . . I saw this on stackoverflow:

why

not try NGINX with fcgi-function mapping?

Implementation steps

Nginx is a magic tool for proxy and is often used for load balancing. As long as the content of my client is sent to nginx, and then nginx forwards the data to fcgi-related applications, all I need to do is combine fcgi with my reasoning program.

nginx

Simply put, nginx is the middleman. The client sends the request to the middleman, who goes to the source of goods to pick up the goods and then responds to the customer:

The customer tells nginx that I want to buy ** product, and nginx goes to the corresponding service provider to get the corresponding service and returns it to the customer.

What we need now is to implement the fcgi part, so what is fcgi?

cgi

Common Gateway Interface (CGI) is an important Internet technology that allows a client to request data from a web browser to a program executed on a network server. CGI describes a standard for transferring data between a server and a request processing program.

The standard input and output here correspond to some environment variables, which mainly include three categories: request-related environment variables, server-related environment variables, and client-related environment variables.

fastcgi

FastCGI is actually CGI with some extended functions added. It is an improvement of CGI. It is also a standard for describing data transmission between client and Web server programs.

FastCGI aims to reduce the overhead of interaction between Web servers and CGI programs, allowing Web servers to handle more Web requests simultaneously. Unlike CGI, which creates a new process for each Web request, FastCGI uses persistent processes to handle a series of Web requests, which are managed by the FastCGI process manager instead of the Web server.

Why do we say that it reduces the cost of interaction? This depends on the difference between the two treatment methods!

The workflow of cgi:

Whenever a client sends a new request, a CGI child process must first be created. After CGI processes the request, as many CGI child processes as there are connections will be started. When the number of requests is large, a large amount of system resources will be occupied.

fastcgi

Fastcgi uses continuous processes to handle a series of requests. These processes are managed by the fastcgi process manager. The specific process is as follows:

It can also be compared like this:

cgi is selling egg-filled pancakes. When customers want to eat, he starts to light the fire, beat the eggs, spread the pancakes, and then turns off the fire. Then wait for the next customer

fastcgi is the old version of a breakfast shop. It employs a group of waiters to prepare meals that need to be cooked on the spot. The boss only needs to arrange the orders, and the waiters are responsible for serving porridge and pancakes.

Specific steps

• Building a C++ development environment
• Build nginx
• Install fastcgi
• Install fastcgi's process manager spawn-cgi
• Write and run programs
• Compile and run

If you want to do your job well, you must first sharpen your tools and build the environment first!

By reading a lot of blog content, I found the simplest installation steps. Many of them are by downloading the source code and then compiling it through make. However, for these more commonly used libraries, they are already integrated in the software package.

C++ Development Environment Installation

apt-get install build-essential

nginx

apt-get install nginx

fastcgi

sudo apt-get install libfcgi-dev

spawn-fcgi

apt-get install spawn-fcgi

Write and run programs

#include <iostream>
#include "fcgio.h"
 
using namespace std;
 
int main(void) {
 // Backup the stdio streambufs
 streambuf * cin_streambuf = cin.rdbuf();
 streambuf * cout_streambuf = cout.rdbuf();
 streambuf *cerr_streambuf = cerr.rdbuf();
 
 FCGX_Request request;
 
 FCGX_Init();
 FCGX_InitRequest(&request, 0, 0);
 
 while (FCGX_Accept_r(&request) == 0) {
  fcgi_streambuf cin_fcgi_streambuf(request.in);
  fcgi_streambuf cout_fcgi_streambuf(request.out);
  fcgi_streambuf cerr_fcgi_streambuf(request.err);
 
  cin.rdbuf(&cin_fcgi_streambuf);
  cout.rdbuf(&cout_fcgi_streambuf);
  cerr.rdbuf(&cerr_fcgi_streambuf);
 
  cout << "Content-type: text/html\r\n"
    << "\r\n"
    << "<html>\n"
    << " <head>\n"
    << " <title>Hello, World!</title>\n"
    << " </head>\n"
    << " <body>\n"
    << " <h1>Hello, World!</h1>\n"
    << " </body>\n"
    << "</html>\n";
 
 }
 cin.rdbuf(cin_streambuf);
 cout.rdbuf(cout_streambuf);
 cerr.rdbuf(cerr_streambuf);
 return 0;

Compiling the Program

g++ cgi.cpp -o cgidemo -lfcgi

Modify the nginx configuration file

vi /usr/local/nginx/conf/nginx.conf 

Start nginx

nginx -c /usr/local/nginx/conf/nginx.conf

Verify that nginx is started normally through the browser http://*******:80

Start the spwan-cgi process

spawn-fcgi -a 127.0.0.1 -C 20 -p 7070 ./cgidemo

Summarize

The above is the full content of this article. I hope that the content of this article will have certain reference learning value for your study or work. If you have any questions, you can leave a message to communicate. Thank you for your support for 123WORDPRESS.COM.