How to use shell to perform batch operations on multiple servers

SSH protocol

Before talking about public key login, let's talk about the SSH protocol.

SSH is a network protocol. When we say ssh, we usually refer to its implementation, namely OpenSSH. In the shell, it is the ssh command.

SSH

Secure Shell (SSH) is an encrypted network transmission protocol that can provide a secure transmission environment for network services in an insecure network. SSH establishes a secure tunnel in the network to connect the SSH client and server.

The principle of SSH is similar to that of HTTPS, both of which are application layer protocols based on TCP and asymmetric encryption. The difference between it and HTTPS is that HTTPS uses digital certificates and digital certificate authentication centers to prevent man-in-the-middle attacks, while the public key of the ssh server is not notarized and its identity can only be manually determined through its public key fingerprint.

As shown in the figure below, when we use ssh to log in to a server for the first time, ssh will prompt us to verify the server's public key fingerprint.

When we verify that this public key fingerprint is the server we want to log in, the server's public key will be added to ~/.ssh/known_hosts. When logging in again, ssh will skip the public key verification stage after detecting that it is an authenticated server.

Connection process

I have also introduced the concept of communication encryption in my previous article, see: Let’s talk about encryption again - understanding and using RSA asymmetric encryption. As for the SSH protocol connection process, please refer to: Protocol Basics: Secure Shell Protocol.

To sum up, the following steps are mainly included:

• TCP three-way handshake
• SSH protocol version negotiation
• Public key exchange between client and server
• Encryption algorithm negotiation
• The client uses symmetric encryption key authentication
• Secure communication between client and server

I used tcpdump + wireshark to capture packets and check the SSH connection process, as shown in the following figure:

I have to sigh again that tcpdump + wireshark is a real tool for learning network protocols.

ssh tool

ssh

As a tool, ssh is divided into server and client. On the server side, it is sshd, which generally occupies port 22. We usually use its client, the general usage is ssh user@host, and then according to the ssh prompt, we enter the password and log in to the server.

Its functions are very powerful, as you can see from its supported parameters.

ssh [-1246AaCfGgKkMNnqsTtVvXxYy] [-b bind_address] [-c cipher_spec] [-D [bind_address:]port] [-E log_file] [-e escape_char] [-F configfile] [-I pkcs11] [-i identity_file] [-J [user@]host[:port]] [-L address] [-l login_name] [-m mac_spec] [-O ctl_cmd] [-o option] [-p port] [-Q query_option] [-R address] [-S ctl_path] [-W host:port] [-w local_tun[:remote_tun]] [user@]hostname [command]

After introducing the SSH protocol and ssh command, we finally talk about public key login.

Public key login

Once you understand the principles of asymmetric encryption, logging in with a public key will be very simple. Since the public and private keys are a unique pair, as long as the client ensures the security of its own private key, the server can completely determine the authenticity of the client through the public key. Therefore, to implement public key login, we must first generate a public and private key pair.

Generate a key pair using the ssh-keygen command. To make the steps more complete, I temporarily save them to the working directory. By default, they will be saved to the ~/.ssh directory.

~ ssh-keygen

Generating public/private rsa key pair.

Enter file in which to save the key (/Users/zbs/.ssh/id_rsa): ./test

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in ./test.

Your public key has been saved in ./test.pub.

The key fingerprint is:

SHA256:xxxxx/B17z/xxxxxx [email protected]

The key's randomart image is:

+---[RSA 2048]----+

| o+*..EO* |

| .... |

|oo+ .o++.o|

+----[SHA256]-----+

~ ls ./test*

./test ./test.pub

Put the contents of the private key file ./test into the client's ~/.ssh/id_rsa, then try to log in to the server using the password, and put the contents of the public key file ./test.pub into the server's ~/.ssh/authorized_keys.

When you log in again, ssh will automatically use your own private key for authentication, thus avoiding the need to enter a password.

Batch Operations

Public key login helps us avoid the trouble of entering a password every time we log in to the server. It also solves the problem of synchronous blocking of each login session, so that we can use ssh's ssh user@host command method to execute commands directly on the server.

At the same time, when we have an IP list, it is possible to use a for loop to traverse the IP list and execute commands in batches on multiple servers.

Multi-server file merging

A few days ago, I helped a colleague search for logs on multiple servers. I needed to aggregate the logs found on multiple servers onto the same machine for statistical analysis. I use pssh to log in to multiple servers. Since the log volume is too large, it is not practical to output the results to the terminal and then copy them. If redirection is used, the results will be redirected to each server.

scp

At this time, you can use scp. scp and ssh are commands from the same family and are also secure transmission protocols implemented based on the SSH protocol. As long as the public keys of each server are kept, password-free operation can be achieved just like the ssh command.

A common usage of scp is scp src dst, where the remote path can be expressed as user@host:/path. In the case of batch login, you can use commands such as grep to input the result files into a file first, and then use the scp command to copy it to the same server.

To avoid file name conflicts between servers, you can use uuidgen | xargs -I {} scp result.log root@ip:/result/{} to copy the results of each server to different files, and then use cat to merge the files in the result folder into one.

nc

Of course, in most cases, our servers do not save each other's public keys, but the nc command can perfectly solve this problem.

The -k option of nc allows the nc server to keep the connection open after the file transfer is completed. In this way, we use nc -k -4l port > result.log to start an nc server, and then use grep xxx info.log | nc ip port to merge the result data.

summary

The various tools introduced in this article are still small-scale development. It is always good to know more tools. If you are doing operation and maintenance work, you still need to rely on the OPS platform to integrate more functions and achieve complete automation.

The above is the details of how to use shell to perform batch operations on multiple servers. For more information about using shell to perform batch operations on multiple servers, please pay attention to other related articles on 123WORDPRESS.COM!