VPS builds offline download server (post-network disk era)

motivation

Due to learning needs, I purchased a vps service from a foreign server manufacturer (I won’t tell you which manufacturer it was). However, even if used as a ladder, 1T of traffic per month will never be used up. I finally feel that I have enough nutrition and want to find a movie to watch.
Unfortunately, the speed of Baidu Netdisk is really touching. A one-year Baidu Netdisk super membership is extremely expensive, and Baidu's offline download is not really offline. It just pulls the files on its server to you. If you want to find some scarce resources, Baidu Netdisk is useless.

Hey, can I use vps to build my own offline download server? This way you can utilize excess bandwidth resources and increase download speeds, isn’t that great! With this question in mind, I started building it on a lonely night.

Construction process

Machine configuration distribution: CentOS 6.9
RAM: 512M
CPU: Single core Bandwidth: 1000Mbps

The configuration of this machine is not high, but it is more than enough to be used as a download server.

Tool Selection

After some searching, I chose transmission + vsftpd + nginx + aria2c as the deployment tools.

• transmission, transmission-daemon: As a client for torrents and magnets, and as a core tool for offline downloads.
• vsftpd: used to build ftp to manage file downloads (poor vps only has 10G of disk space T_T)
• nginx: High-performance nginx is used as a static file server.
• aria2c: A multi-threaded downloader that downloads our offline files at high speed from the file list displayed by nginx.

Deployment Practice

1. Install transmission, modify configuration and start service

Install transmission and transmission-daemon

$ wget http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
$ rpm -ivh epel-release-6-8.noarch.rpm
$ yum -y upgrade
$ yum -y install transmission transmission-daemon

Modify the configuration and start the transmission service. The settings.json file under the path /var/lib/transmission/.config/transmission is the configuration file of transmission. There are a lot of configurations in it. We need to modify the following configuration so that we can use remote software to control our transmission

"rpc-authentication-required": true,
"rpc-enabled": true,
"rpc-password": "Remote call password",
"rpc-whitelist-enabled": false,
"rpc-username": "Remote call username",

Start the transmission service

$ sudo service transmission-daemon start

So we can access the transmission service through http://host_ip:9091. After a test, the speed is still good, with a peak speed of 30MB/s

In addition, we can also install Transmission-Remote-GUI to control transmission without visiting the page. It can be downloaded from the app store under Linux (Mint). The operation method is similar to the UI interface of uTorrent:

2. vsftpd builds ftp server

Building an ftp server is not used to download files. The speed of downloading files through ftp is very slow. Therefore, the role of the ftp server is to manage files, generally speaking, it is a deletion operation.

Install vsftpd

$ yum install -y vsftpd

We use Linux users to log in, not virtual users (because it is troublesome to install the database)
Because vsftpd does not allow root login by default, we need to add a user

Adding Users

$ useradd ftpuser
$ passwd ftpuser

Change the home directory of ftpuser to the download directory of transmission

$ usermod -d /var/lib/transmission/Downloads ftpuser
$ rm -rf /home/ftpuser

Modify the vsftpd configuration file (/etc/vsftpd/vsftpd.conf)
The configuration files are as follows:

# generate using cat vsftpd.conf | grep -v "#"
anonymous_enable=NO
local_enable=YES
write_enable=YES
local_umask=022
dirmessage_enable=YES
xferlog_enable=YES
connect_from_port_20=YES
xferlog_std_format=YES
ftpd_banner=Welcome to Private FTP service.
chroot_list_enable=NO
listen_ipv6=YES

pam_service_name=vsftpd
userlist_enable=YES
tcp_wrappers=YES
local_root=/var/lib/transmission/Downloads

The configuration chroot_list_enable=NO indicates that the user is not allowed to switch out of the home directory.

Start vsftpd

sudo service vsftpd start

In this way, we can use ftp to manage the files downloaded by transmission.

3. Build a static file server with nginx

Use nginx as a static resource server to provide a display of downloaded resources. There are a lot of tutorials on the Internet for installing nginx. As a follow-up fill.

Mainly talk about configuration

Modify the nginx configuration file (/etc/nginx/conf.d/default.conf)

autoindex on;
autoindex_exact_size on;
autoindex_localtime on;

server {
  listen 8090;
  listen [::]:8090;
  server_name localhost;

  charset utf-8;
  location / {
    root /var/lib/transmission/Downloads;
    index index.html index.htm;
  }

Start the nginx service and you can get the file list through port 8090.

4. aria2c download files

aria2c is used to download files from the server and install it on your own machine.

Mainly talk about how to accelerate downloading through aria2c multi-threading

There are several parameters in aria2c related to multi-threaded downloading

Parameter x: --max-connection-per-server=<NUM>, the maximum number of connections per server, the default is 1
Parameter s: --split=<N>, refers to the number of download connections for each file, the default is 5, -s is limited by -x, if s defaults to x greater than 5, x will also be limited to s

So for large files, increasing the x and s parameters appropriately can increase the download speed.

Reference Links:

How To Install Transmission BitTorrent Client on CentOS 6
Building FTP server under Linux