Detailed explanation of the pitfalls of nginx proxy socket.io service

Nginx proxies two socket.io servers. The working mode of socket.io is polling upgraded to websocket

Phenomenon

When requesting services through nginx, a large number of 400 errors occurred. Sometimes they could be upgraded to websocket, and sometimes they would keep reporting errors. But when accessing directly through ip+端口, 100% success.

analyze

sid

SID is the key to our problem. When initially creating a connection (polling mode is simulating a long connection), the client will initiate a request like this:

https://***/?EIO=3&transport=polling&t=1540820717277-0

After receiving the message, the server will create an object, bind it to the connection, and return a sid (session id) to mark the session. What does a session mean? A session is a series of interactions, and these interactions are related. In our scenario, when the next http request comes, I need to find the object that was previously bound to the theoretical long connection (there is no websocket here, so it is theoretical). We know that HTTP requests are stateless and each request is independent, so socket.io introduces SID to do this. After receiving the request, the server will generate a sid and look at the response:

Each subsequent request will need to carry this sid, including the one for establishing a websocket request. Therefore, sid is the key to polling and upgrading polling to websocket. The request after this looks like:

https://***/?EIO=3&transport=polling&t=1540820717314-1&sid=EoGaL3fRQlpTOaLp5eST

or

wss://***/?EIO=3&transport=websocket&t=1540820717314-1&sid=EoGaL3fRQlpTOaLp5eST

So the question is, what happens if the SID included in the request is not generated by the server? The server will not recognize it and return a 400 to you, and tell you

invalid sid

This is the problem we encountered. The default load balancing strategy of nginx is polling, so the request may be hit on a machine that is not the one that generated the SID. At this time, we will receive a 400. If we are lucky, it may also be hit on the original machine. If we are luckier, we can even persist until the websocket connection is established.

solve

Here are two solutions

1. Nginx uses ip_hash for load balancing, which ensures that all requests from a client go to one server.
2. Do not use polling mode, only use websocket

Both options have their pros and cons. The second one is obvious, old browsers and clients that don't support websockets won't work. The first problem is more hidden. Imagine what would happen if you added or removed machines. At this time, the model of the ip_hash strategy would change, and all previous connections would become invalid. However, for microservices, scaling is a very frequent operation (especially when the product is in the development stage), and this lossy scaling is most likely unacceptable.

In summary, it is recommended to use websocket directly. After all, the old versions that do not support websocket account for a small proportion, and it will take less time than polling first.

The above is the full content of this article. I hope it will be helpful for everyone’s study. I also hope that everyone will support 123WORDPRESS.COM.