How to use macros in JavaScript

In languages, macros are often used to implement DSLs. Through macros, developers can customize the format of some languages, such as implementing JSX syntax. Now that WASM has been implemented, it is not impossible to write web pages in other languages. For example, the Rust language has a powerful macro function, which means that the Rust-based Yew framework does not need to implement something like Babel, but can implement JSX-like syntax by relying on the language itself. An example of a Yew component that supports a JSX-like syntax.

impl Component for MyComponent {
    // ...

    fn view(&self) -> Html {
        let onclick = self.link.callback(|_| Msg::Click);
        html! {
            <button onclick=onclick>{ self.props.button_text }</button>
        }
    }
}

Limitations of JavaScript Macros

Unlike Rust, JavaScript itself does not support macros, so the entire tool chain does not consider macros. Therefore, you can write a macro that recognizes custom syntax, but since the supporting tool chain does not support it, such as the most common VSCode and Typescript, you will get a syntax error. Similarly, the parser used by Babel itself does not support extended syntax unless you fork another Babel. Therefore babel-plugin-macros does not support custom syntax. However, with the help of template string functions, we can take a detour and at least gain the ability to partially customize the syntax tree. A GraphQL example that supports writing GraphQL directly in JavaScript.

import { gql } from 'graphql.macro';

const query = gql`
  query User {
    user(id: 5) {
      lastName
      ...UserEntry1
    }
  }
`;

// Will be converted to ↓ ↓ ↓ ↓ ↓ ↓ at compile time

const query = {
  "kind": "Document",
  "definitions": [{
    ...

Why use macros instead of Babel plugins?

The capabilities of Babel plugins are indeed far greater than macros, and in some cases you really have to use plugins. One thing that's better about macros than Babel plugins is that the idea of ​​macros is to use them out of the box. Developers who use React must have heard of the famous Create-React-App, which encapsulates various underlying details for you, so developers can focus on writing code. But the problem with CRA is that it is too tightly encapsulated. If you need to customize the Babel plugin at all, you basically need to execute yarn react-script eject to expose all the underlying details. As for macros, you only need to add a babel-plugin-macros plug-in in the project's Babel configuration, then any custom Babel macros can be perfectly supported, instead of having to download various plug-ins like plug-ins. CRA has built-in babel-plugin-macros, you can use any Babel macros in your CRA project.

How to write a macro?

introduce

A macro is very much like a Babel plugin, so it is very helpful to know how to write a Babel plugin in advance. Babel officially has a manual on how to write a Babel plugin from scratch. Now that we know how to write Babel plugins, let's first use an example of using macros to explain how Babel identifies macros in files. Is it some special syntax or just bad use of the $ symbol?

import preval from 'preval.macro'

const one = preval `module.exports = 1 + 2 - 1 - 1`

This is a very common macro. Its function is to execute the JavaScript code in the string during compilation, and then replace the execution result with the corresponding place. The above code will be expanded to:

import preval from 'preval.macro'

const one = 1

From the usage point of view, the only thing related to identifying macros is the *.macro character, which is indeed how Babel identifies macros. In fact, not only for the *.macro form, Babel considers libraries whose names match the regular expression /[./]macro(\.c?js)?$/ to be Babel macros. Some examples of these matching expressions:

'my.macro'
'my.macro.js'
'my.macro.cjs'
'my/macro'
'my/macro.js'
'my/macro.cjs'

write

Next, we will simply write an importURL macro, which is used to import some libraries through URLs, and pre-pull the code of these libraries during compilation, process it, and then import it into the file. I know some Webpack plugins already support importing libraries from urls, but this is also a good example to learn how to write macros, just for fun! And how to make synchronous requests in NodeJS! :)

Prepare

First, create a folder named importURL and execute npm init -y to quickly create a project. People who use macros in the project need to install babel-plugin-macros. Similarly, those who write macros also need to install this plug-in. Before writing, we also need to install some other libraries in advance to assist us in writing macros. Before development, we need to:

• Change the name in package.json to import-url.macro, which conforms to the format that Babel recognizes macros.
• We need to use the helper methods provided by Babel to create macros. Execute yarn add babel-plugin-macros
• yarn add fs-extra, a library that is easier to use to replace the Nodefs module
• yarn add find-root, in the process of writing macros, we need to find the working directory of the processed file according to its path, so as to write it into the cache. This is a packaged library

Example

Our goal is to convert the following code into

import importURL from 'importurl.macros';

const React = importURL('https://unpkg.com/[email protected]/umd/react.development.js');

// Compile to import importURL from 'importurl.macros';

const React = require('../cache/pkg1.js');

We will parse the first parameter of the importURL function of the code as the address of the remote library, and then pull the code content through the Get request synchronously during compilation. Then write it to the .chache in the top-level folder of the project, and replace the corresponding importURL statement with a require(...) statement. The path... uses the relative path in the .cache file of the importURL file, so that webpack can find the corresponding code when it is finally packaged.

start

Let's first take a look at what the final code looks like

import { execSync } from 'child_process';
import findRoot from 'find-root';
import path from 'path';
import fse from 'fs-extra';

import { createMacro } from 'babel-plugin-macros';

const syncGet = (url) => {
  const data = execSync(`curl -L ${url}`).toString();
  if (data === '') {
    throw new Error('empty data');
  }
  return data;
}

let count = 0;
export const genUniqueName = () => `pkg${++count}.js`;

module.exports = createMacro((ctx) => {
  const {
    references, // All references to macros in the file babel: {
      types: t,
    }
  } = ctx;
  // Babel will set the currently processed file path to ctx.state.filename
  const workspacePath = findRoot(ctx.state.filename);
  // Calculate the cache folder const cacheDirPath = path.join(workspacePath, '.cache');
  //
  const calls = references.default.map(path => path.findParent(path => path.node.type === 'CallExpression' ));
  calls.forEach(nodePath => {
    // Determine the type of astNode if (nodePath.node.type === 'CallExpression') {
      // Make sure the first argument of the function is a pure string if (nodePath.node.arguments[0]?.type === 'StringLiteral') {
        // Get a parameter as the address of the remote library const url = nodePath.node.arguments[0].value;
        // Pull codes according to url const codes = syncGet(url);
        // Generate a unique package name to prevent conflicts const pkgName = genUniqueName();
        // Determine the final file path to be written const cahceFilename = path.join(cacheDirPath, pkgName);
        //Write the content through the fse library, outputFileSync will automatically create a non-existent folder fse.outputFileSync(cahceFilename, codes);
        // Calculate the relative path const relativeFilename = path.relative(ctx.state.filename, cahceFilename);
        // Final calculation to replace importURL statement nodePath.replaceWith(t.stringLiteral(`require('${relativeFilename}')`))
      }
    }
  });
});

Create a macro

We create a macro through the createMacro function. createMacro accepts the function we write as a parameter to generate a macro, but we don’t actually care what the return value of createMacro is, because our code will eventually be replaced by itself and will not be executed during runtime. The first argument to the function we wrote is some state that Babel passed to us, and we can take a quick look at what its type is.

function createMacro(handler: MacroHandler, options?: Options): any;
interface MacroParams {
      references: { default: Babel.NodePath[] } & References;
      state: Babel.PluginPass;
      babel: typeof Babel;
      config?: { [key: string]: any };
  }
export interface PluginPass {
    file: BabelFile;
    key: string;
    opts: PluginOptions;
    cwd: string;
    filename: string;
    [key: string]: unknown;
}

Visualizing the AST

We can use astexplorer to observe the syntax tree of the code we are going to process. For the following code

import importURL from 'importurl.macros';

const React = importURL('https://unpkg.com/[email protected]/umd/react.development.js');

The following syntax tree will be generated

The syntax tree nodes marked in red are what Babel will pass to us through ctx.references, so we need to use the .findParent() method to find the parent node CallExpresstion to get the parameters under the arguments property and get the URL address of the remote library.

Synchronous Request

One difficulty here is that Babel does not support asynchronous transformations. All transformation operations are synchronous, so the request must also be a synchronous request. I would have thought this would be a simple thing to do and Node would provide an option like sync: true. But no, Node does not support any synchronous requests, unless you choose to use the following weird way

const syncGet = (url) => {
  const data = execSync(`curl -L ${url}`).toString();
  if (data === '') {
    throw new Error('empty data');
  }
  return data;
}

ending

After getting the code, we write the code to the file path calculated at the beginning. The purpose of using fs-extra here is that if fs-extra encounters a non-existent folder when writing, it will not throw an error directly like fs, but automatically create the corresponding file. After writing is completed, we create a string node through the auxiliary method stringLiteral provided by Babel, and then replace our importURL(...), and our entire conversion process is completed.

at last

This macro has some defects, and interested students can continue to improve it:

There is no library to identify the same URL and reuse it, but I think these are enough for the purpose of how to write a macro.

genUniqueName will calculate duplicate package names across files. The correct algorithm should be to calculate the hash value based on the URL as the unique package name

This is the end of this article about how to use macros in JavaScript. For more information about using macros in JavaScript, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!