Implementation of Vue large file upload and breakpoint resumable upload

2 solutions for file upload

Based on file stream (form-data)

The client converts the file to base64

Large file upload

Get the file object and convert it into an ArrayBuffer object
Create slices
Send Request
After all slices are sent successfully

Resume download

Code Summary

One More Thing

FormData
FormData.append()

2 solutions for file upload

Based on file stream (form-data)

The upload component of the element-ui framework is based on file stream by default.

Data format: form-data;
Data transferred: file file stream information; filename file name

The client converts the file to base64

After converting to a base64 string through fileRead.readAsDataURL(file), it must be compiled with encodeURIComponent before sending. The sent data is processed by qs.stringify, and the request header adds "Content-Type": "application/x-www-form-urlencoded"

Large file upload

First, build the page with the help of element-ui. Because you want to customize an upload implementation, the auto-upload of the el-upload component must be set to false; action is a required parameter and you don't need to fill in a value here.

<template>
  <div id="app">
    <!-- Upload component -->
    <el-upload action drag :auto-upload="false" :show-file-list="false" :on-change="handleChange">
      <i class="el-icon-upload"></i>
      <div class="el-upload__text">Drag files here, or <em>click to upload</em></div>
      <div class="el-upload__tip" slot="tip">Video size does not exceed 200M</div>
    </el-upload>

    <!-- Progress Display-->
    <div class="progress-box">
      <span>Upload progress: {{ percent.toFixed() }}%</span>
      <el-button type="primary" size="mini" @click="handleClickBtn">{{ upload | btnTextFilter}}</el-button>
    </div>

    <!-- Display the successfully uploaded video-->
    <div v-if="videoUrl">
      <video :src="videoUrl" controls />
    </div>
  </div>
</template>

Get the file object and convert it into an ArrayBuffer object

Convert to ArrayBuffer because SparkMD5 library will be used to generate hash values and name files later.

async handleChange(file) {
  const fileObj = file.raw
  try{
    const buffer = await this.fileToBuffer(fileObj)
    console.log(buffer)
  }catch(e){
    console.log(e)
  }
}

The print buffer result is as follows

Note: Both the before-upload function and the on-change function have file as parameters, but the file in on-change is not a File object. To obtain a File object, you need to use file.raw. The FileReader class is used here to convert the File object to an ArrayBuffer object. Because it is an asynchronous process, it is encapsulated with Promise:

// Convert the File object to an ArrayBuffer 
fileToBuffer(file) {
  return new Promise((resolve, reject) => {
    const fr = new FileReader()
    fr.onload = e => {
      resolve(e.target.result)
    }
    fr.readAsArrayBuffer(file)
    fr.onerror = () => {
      reject(new Error('Error in converting file format'))
    }
  })
}

Create slices

A file can be divided into several parts by fixed size or fixed number. In order to avoid the error caused by the IEEE754 binary floating point arithmetic standard used by js, I decided to cut the file in a fixed size way and set the size of each slice to 2M, that is, 2M = 21024KB = 21024*1024B = 2097152B. Blob.slice() is used to cut files

// Slice the file into fixed size (2M). Note that multiple constants are declared here const chunkSize = 2097152,
  chunkList = [], // Array to hold all slices chunkListLength = Math.ceil(fileObj.size / chunkSize), // Calculate the total number of slices suffix = /\.([0-9A-z]+)$/.exec(fileObj.name)[1] // File suffix // Generate hash value based on file content const spark = new SparkMD5.ArrayBuffer()
spark.append(buffer)
const hash = spark.end()

// Generate slices. The backend requires the passed parameters to be byte data chunks (chunk) and the file name of each data chunk (fileName)
let curChunk = 0 // Initial position when slicing for (let i = 0; i < chunkListLength; i++) {
  const item = {
    chunk: fileObj.slice(curChunk, curChunk + chunkSize),
    fileName: `${hash}_${i}.${suffix}` // The file name is named according to hash_1.jpg}
  curChunk += chunkSize
  chunkList.push(item)
}
console.log(chunkList)

After selecting a file, you will get a print result like the following:

Send Request

Sending requests can be parallel or serial, here choose serial sending. A new request is created for each slice. In order to achieve breakpoint resuming, we encapsulate the request into the function fn, use an array requestList to save the request set, and then encapsulate a send function for request sending. In this way, once the pause button is pressed, the upload can be easily terminated. The code is as follows:

sendRequest() {
  const requestList = [] // request collection this.chunkList.forEach(item => {
    const fn = () => {
      const formData = new FormData()
      formData.append('chunk', item.chunk)
      formData.append('filename', item.fileName)
      return axios({
        url: '/single3',
        method: 'post',
        headers: { 'Content-Type': 'multipart/form-data' },
        data: formData
      }).then(res => {
        if (res.data.code === 0) { // Successif (this.percentCount === 0) {
            this.percentCount = 100 / this.chunkList.length
          }
          this.percent += this.percentCount // change progress}
      })
    }
    requestList.push(fn)
  })
  
  let i = 0 // Record the number of requests sent const send = async () => {
    // if ('pause') return
    if (i >= requestList.length) {
      //Send completed return
    } 
    await requestList[i]()
    i++
    send()
  }
  send() // Send request},

The axios part can also be written directly in the following form:

axios.post('/single3', formData, {
  headers: { 'Content-Type': 'multipart/form-data' }
})

After all slices are sent successfully

According to the backend interface, another get request is sent and the hash value of the file is passed to the server. We define a complete method to implement it. Here, it is assumed that the file sent is a video file.

const complete = () => {
  axios({
    url: '/merge',
    method: 'get',
    params: { hash: this.hash }
  }).then(res => {
    if (res.data.code === 0) { // Request sent successfully this.videoUrl = res.data.path
    }
  })
}

In this way, you can browse the sent video on the page after the file is sent successfully.

Resume download

First, the pause button text is processed. A filter is used. If the upload value is true, "Pause" is displayed, otherwise "Continue" is displayed:

filters:
  btnTextFilter(val) {
    return val ? 'Pause' : 'Continue'
  }
}

When the pause button is pressed, the handleClickBtn method is triggered

handleClickBtn() {
  this.upload = !this.upload
  // If not paused, continue uploading if (this.upload) this.sendRequest()
}

Add if (!this.upload) return at the beginning of the send method to send the slice, so that as long as the upload variable is false, the upload will not continue. In order to continue sending after the pause, you need to delete the slice from the chunkList array after each successful sending of a slice this.chunkList.splice(index, 1)

Code Summary

<template>
  <div id="app">
    <!-- Upload component -->
    <el-upload action drag :auto-upload="false" :show-file-list="false" :on-change="handleChange">
      <i class="el-icon-upload"></i>
      <div class="el-upload__text">Drag files here, or <em>click to upload</em></div>
      <div class="el-upload__tip" slot="tip">Video size does not exceed 200M</div>
    </el-upload>

    <!-- Progress Display-->
    <div class="progress-box">
      <span>Upload progress: {{ percent.toFixed() }}%</span>
      <el-button type="primary" size="mini" @click="handleClickBtn">{{ upload | btnTextFilter}}</el-button>
    </div>

    <!-- Display the successfully uploaded video-->
    <div v-if="videoUrl">
      <video :src="videoUrl" controls />
    </div>
  </div>
</template>

<script>
  import SparkMD5 from "spark-md5"
  import axios from "axios"
  
  export default {
    name: 'App3',
    filters:
      btnTextFilter(val) {
        return val ? 'Pause' : 'Continue'
      }
    },
    data() {
      return {
        percent: 0,
        videoUrl: '',
        upload: true,
        percentCount: 0
      }
    },
    methods: {
      async handleChange(file) {
        if (!file) return
        this.percent = 0
        this.videoUrl = ''
        // Get the file and convert it into an ArrayBuffer object const fileObj = file.raw
        let buffer
        try {
          buffer = await this.fileToBuffer(fileObj)
        } catch (e) {
          console.log(e)
        }
        
        // Slice the file into fixed size (2M). Note that multiple constants are declared here const chunkSize = 2097152,
          chunkList = [], // Array to hold all slices chunkListLength = Math.ceil(fileObj.size / chunkSize), // Calculate the total number of slices suffix = /\.([0-9A-z]+)$/.exec(fileObj.name)[1] // File suffix // Generate hash value based on file content const spark = new SparkMD5.ArrayBuffer()
        spark.append(buffer)
        const hash = spark.end()

        // Generate slices. The backend requires the passed parameters to be byte data chunks (chunk) and the file name of each data chunk (fileName)
        let curChunk = 0 // Initial position when slicing for (let i = 0; i < chunkListLength; i++) {
          const item = {
            chunk: fileObj.slice(curChunk, curChunk + chunkSize),
            fileName: `${hash}_${i}.${suffix}` // The file name is named according to hash_1.jpg}
          curChunk += chunkSize
          chunkList.push(item)
        }
        this.chunkList = chunkList // sendRequest needs to use this.hash = hash // sendRequest needs to use this.sendRequest()
      },
      
      // Send request sendRequest() {
        const requestList = [] // request collection this.chunkList.forEach((item, index) => {
          const fn = () => {
            const formData = new FormData()
            formData.append('chunk', item.chunk)
            formData.append('filename', item.fileName)
            return axios({
              url: '/single3',
              method: 'post',
              headers: { 'Content-Type': 'multipart/form-data' },
              data: formData
            }).then(res => {
              if (res.data.code === 0) { // Successif (this.percentCount === 0) { // Avoid deleting the slice after the upload is successful and changing the length of chunkList to affect the value of percentCountthis.percentCount = 100 / this.chunkList.length
                }
                this.percent += this.percentCount // Change progress this.chunkList.splice(index, 1) // Once the upload is successful, delete this chunk to facilitate breakpoint resuming}
            })
          }
          requestList.push(fn)
        })
        
        let i = 0 // Record the number of requests sent // After all file slices are sent, you need to request the '/merge' interface and pass the file hash to the server const complete = () => {
          axios({
            url: '/merge',
            method: 'get',
            params: { hash: this.hash }
          }).then(res => {
            if (res.data.code === 0) { // Request sent successfully this.videoUrl = res.data.path
            }
          })
        }
        const send = async () => {
          if (!this.upload) return
          if (i >= requestList.length) {
            // Sending completed()
            return
          } 
          await requestList[i]()
          i++
          send()
        }
        send() // Send request},
      
      // Press the pause button handleClickBtn() {
        this.upload = !this.upload
        // If not paused, continue uploading if (this.upload) this.sendRequest()
      },
      
      // Convert the File object to an ArrayBuffer 
      fileToBuffer(file) {
        return new Promise((resolve, reject) => {
          const fr = new FileReader()
          fr.onload = e => {
            resolve(e.target.result)
          }
          fr.readAsArrayBuffer(file)
          fr.onerror = () => {
            reject(new Error('Error in converting file format'))
          }
        })
      }
    }
  }
</script>

<style scoped>
  .progress-box {
    box-sizing: border-box;
    width: 360px;
    display: flex;
    justify-content: space-between;
    align-items: center;
    margin-top: 10px;
    padding: 8px 10px;
    background-color: #ecf5ff;
    font-size: 14px;
    border-radius: 4px;
  }
</style>

The effect is as follows:

One More Thing

FormData

FormData is used to send data here. If the encoding type is set to "multipart/form-data", it will use the same format as the form.

FormData.append()

A new value will be added to an existing key in the FormData object, or the key will be added if it does not exist. This method can pass three parameters, formData.append(name, value, filename), where filename is an optional parameter and is the file name passed to the server. When a Blob or File is used as the second parameter, the default file name of the Blob object is "blob". The default file name for a File object is the name of the file.

This is the end of this article about the implementation of Vue large file upload and breakpoint resume. For more related Vue large file upload and breakpoint resume content, please search 123WORDPRESS.COM's previous articles or continue to browse the following related articles. I hope everyone will support 123WORDPRESS.COM in the future!

You may also be interested in:

Vue+element+oss realizes front-end fragment upload and breakpoint resume
Based on vue-simple-uploader, encapsulate the global upload plug-in function of file segment upload, instant upload and breakpoint resume

<<: How to solve the problem of character set when logging in to Linux

>>: mysql-8.0.15-winx64 decompression version installation tutorial and three ways to exit