The ultimate solution for writing bash scripts with nodejs

Preface

I've been learning bash script syntax recently, but if you're not familiar with bash syntax, it's very easy to make mistakes, for example: showing undefined variables. Even if a variable is not defined in the shell, it can still be used, but the result may not be what you expect. For example:

#! /bin/bash

# Here we are judging whether the variable var is equal to the string abc, but the variable var is not declared if [ "$var" = "abc" ] 
then
   # If the if judgment is true, print "not abc" in the console
   echo " not abc" 
else
   # If the if judgment is false, print "abc" in the console
   echo " abc "
fi

The result is that abc is printed, but the problem is that this script should report an error. It is an error that the variable is not assigned a value.

To remedy these errors, we learned to add the following to the beginning of the script: set -u

This command means that the script adds it to the head, and if it encounters a non-existent variable, it will report an error and stop execution.

If you run it again, you will get the following message: test.sh: 3: test.sh: num: parameter not set

Imagine again that you originally wanted to delete: rm -rf $dir/* and then dir is empty, what happens? rm -rf is the deletion command. If $dir is empty, it is equivalent to executing rm -rf /*, which deletes all files and folders. . . Then, your system is gone. Is this the legendary deletion of the database and running away?

If it is a node or browser environment, we will definitely get an error if we directly use var === 'abc'. In other words, many JavaScript programming experiences cannot be reused in bash. It would be great if they could be reused.

Later I started to explore, it would be great if I could use node scripts instead of bash. After a day of tossing and turning, I gradually discovered a magic tool, Google's zx library. Don't worry, I won't introduce this library yet. Let's first look at how the mainstream uses node to write bash scripts, and you will know why it is a magic tool.

Executing bash scripts with node: a reluctant solution: child_process API

For example, the exec command in the child_process API

const { exec } = require("child_process");

exec("ls -la", (error, stdout, stderr) => {
    if (error) {
        console.log(`error: ${error.message}`);
        return;
    }
    if (stderr) {
        console.log(`stderr: ${stderr}`);
        return;
    }
    console.log(`stdout: ${stdout}`);
});

It should be noted here that, first of all, exec is asynchronous, but many of our bash script commands are synchronous.

Also note: the error object is different from stderr. error when the child_process module cannot execute the command, the object is not empty. For example, if a file is not found, the error object is not null. However, if the command runs successfully and writes messages to the standard error stream, then that stderr object will not be null.

Of course we can use the synchronous exec command, execSync

// Import exec command from child_process module const { execSync } = require("child_process");

// Synchronously create a folder named hello execSync("mkdir hello");

Let's briefly introduce other APIs of child_process that can execute bash commands.

• spawn: Start a child process to execute a command
• exec: Start a subprocess to execute a command. Unlike spawn, it has a callback function to know the status of the subprocess.
• execFile: Start a child process to execute an executable file
• fork: Similar to spawn, except that it requires specifying the JavaScript file that the child process needs to execute

The difference between exec and ececFile is that exec is suitable for executing commands, while eexecFile is suitable for executing files.

Node executes bash script: Advanced solution shelljs

const shell = require('shelljs');
 
# Delete file command shell.rm('-rf', 'out/Release');
// Copy file command shell.cp('-R', 'stuff/', 'out/Release');
 
# Switch to the lib directory, list the files ending with .js in the directory, and replace the file contents (sed -i is a command to replace text)
shell.cd('lib');
shell.ls('*.js').forEach(function (file) {
  shell.sed('-i', 'BUILD_VERSION', 'v0.1.2', file);
  shell.sed('-i', /^.*REMOVE_THIS_LINE.*$/, '', file);
  shell.sed('-i', /.*REPLACE_LINE_WITH_MACRO.*\n/, shell.cat('macro.js'), file);
});
shell.cd('..');
 
# Unless otherwise specified, execute the given commands synchronously. In synchronous mode, this will return a ShellString
# (Compatible with ShellJS v0.6.x, it returns an object of the form { code:..., stdout:..., stderr:... }).
# Otherwise, this returns the subprocess object, and the callback receives arguments (code, stdout, stderr).
if (shell.exec('git commit -am "Auto-commit"').code !== 0) {
  shell.echo('Error: Git commit failed');
  shell.exit(1);
}

Judging from the above code, shelljs is really a very good solution for writing bash scripts in nodejs. If your node environment cannot be upgraded at will, I think shelljs is indeed sufficient.

Next, let’s take a look at today’s protagonist, zx, which has started at 17.4k.

zx library

Official website: www.npmjs.com/package/zx

Let’s see how to use it first

#!/usr/bin/env zx

await $`cat package.json | grep name`

let branch = await $`git branch --show-current`
await $`dep deploy --branch=${branch}`

await Promise.all([
  $`sleep 1; echo 1`,
  $`sleep 2; echo 2`,
  $`sleep 3; echo 3`,
])

let name = 'foo bar'
await $`mkdir /tmp/${name}

What do you think? It is just writing Linux commands. You can ignore a lot of bash syntax and just use js directly. And its advantages are more than that. Some of its features are quite interesting:

1. Support ts, automatically compile .ts to .mjs file. The .mjs file is the file ending that supports es6 module in the high version of node. That is, this file can directly import the module without escaping with other tools.

2. Comes with support for pipeline operation pipe method

3. It comes with a fetch library for network requests, a chalk library for printing colored fonts, and a nothrow method for error handling. If a bash command fails, you can wrap it in this method to ignore the error.

Complete Chinese document (please forgive my poor translation skills)

#!/usr/bin/env zx

await $`cat package.json | grep name`

let branch = await $`git branch --show-current`
await $`dep deploy --branch=${branch}`

await Promise.all([
  $`sleep 1; echo 1`,
  $`sleep 2; echo 2`,
  $`sleep 3; echo 3`,
])

let name = 'foo bar'
await $`mkdir /tmp/${name}

Bash is great, but when writing scripts, people usually choose a more convenient programming language. JavaScript is a perfect choice, but the standard Node.js library requires a few extra steps before it can be used. zx is based on child_process, escaping arguments and providing sensible defaults.

Install

npm i -g zx

Required environment

Node.js >= 14.8.0

Write your script in a file with a .mjs extension to be able to use await at the top level.

Add the following shebang to the beginning of your zx script:

#!/usr/bin/env zx
Now you will be able to run your script like this:

chmod +x ./script.mjs
./script.mjs

Or via the zx executable:

zx ./script.mjs

All functions ($, cd, fetch, etc.) can be used directly without any imports.

$`command`

Use the spawn function in the child_process package to execute the given string and return a ProcessPromise.

let count = parseInt(await $`ls -1 | wc -l`)
console.log(`Files count: ${count}`)

For example, to upload files in parallel:

If the executed program returns a non-zero exit code, ProcessOutput will be thrown.

try {
  await $`exit 1`
} catch (p) {
  console.log(`Exit code: ${p.exitCode}`)
  console.log(`Error: ${p.stderr}`)
}

ProcessPromise, the following is the interface definition of promise typescript

class ProcessPromise<T> extends Promise<T> {
  readonly stdin: Writable
  readonly stdout: Readable
  readonly stderr: Readable
  readonly exitCode: Promise<number>
  pipe(dest): ProcessPromise<T>
}

The pipe() method can be used to redirect standard output:

await $`cat file.txt`.pipe(process.stdout)

Read more about pipelines github.com/google/zx/b…

Typescript interface definition of ProcessOutput

class ProcessOutput {
  readonly stdout: string
  readonly stderr: string
  readonly exitCode: number
  toString(): string
}

function:

cd()

Change the Current Working Directory

cd('/tmp')
await $`pwd` // outputs /tmp

fetch()

The node-fetch package.

let resp = await fetch('http://wttr.in')
if (resp.ok) {
  console.log(await resp.text())
}

question()

The readline Package

let bear = await question('What kind of bear is best? ')
let token = await question('Choose env variable: ', {
  choices: Object.keys(process.env)
})

In the second parameter, you can specify an array of options for tab autocompletion

The following is the interface definition

function question(query?: string, options?: QuestionOptions): Promise<string>
type QuestionOptions = { choices: string[] }

sleep()

Based on setTimeout function

await sleep(1000)

nothrow()

Change the behavior of $ to not throw an exception if the exit code is not 0.

ts interface definition

function nothrow<P>(p: P): P

await nothrow($`grep something from-file`)
// Inside the pipeline:

await $`find ./examples -type f -print0`
  .pipe(nothrow($`xargs -0 grep something`))
  .pipe($`wc -l`)

The following packages do not need to be imported and can be used directly

chalk

console.log(chalk.blue('Hello world!'))

fs

Similar to the following usage

import { promises as fs } from 'fs'
let content = await fs.readFile('./package.json')

os

await $`cd ${os.homedir()} && mkdir example`

Configuration:

$.shell

Specifies bash to use.

$.shell = '/usr/bin/bash'

$.quote

Specifies the function used to escape special characters during command substitution

The default package used is shq.

Notice:

The two variables __filename & __dirname are in commonjs. We use es6 modules ending in .mjs.

In ESM modules, Node.js does not provide __filename and __dirname global variables. Since such global variables are very convenient in scripts, zx provides these for use in .mjs files (when using the zx executable)

Require is also a module import method in commonjs.

In the ESM module, there is no require() function defined. zx provides a require() function so it can be used with imports in .mjs files (when using the zx executable)

Passing environment variables

process.env.FOO = 'bar'
await $`echo $FOO`

Passing Arrays

If an array of values ​​is passed as an argument to $, the items of the array will be escaped individually and concatenated by spaces.

Example:

let files = [1,2,3]
await $`tar cz ${files}`

$ and other functions can be used by explicitly importing

#!/usr/bin/env node
import {$} from 'zx'
await $`date`

zx can compile .ts scripts to .mjs and execute them

zx-examples/typescript.ts

Summarize

This is the end of this article about writing bash scripts with nodejs. For more information about writing bash scripts with nodejs, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!