Example of making XML online editor using js

Preface

I haven’t updated my blog for more than a year because "Mount & Blade 2" was released during the epidemic, and then I went to write game MODs.

I spent about 7 months writing game MODs in C#, working until very late every night. During this period, I learned PR because of introducing this game MOD, and then became a UP host on Bilibili.

Later on, I had some other ideas and company business adjustments, and I was too lazy to write a blog, and before I knew it, more than a year had passed.

There are still gains:

• For example, when this MOD was discontinued, its download volume ranked first on both the Chinese site and the 3DM MOD site, and it was far ahead of the second place. If you have friends who play the "Mount and Blade 2" MOD, you should be able to guess who I am.
• For example, he gained more than 5,000 fans on Bilibili, but he stuttered at the beginning and he still stuttered at the end. But because of my own editing, the viewing effect is quite good.
• For example, I have a deep understanding of how troublesome it is to be a UP and an anchor. Even with my poor data, I am actually ahead of many UP hosts on Bilibili. Among the UP hosts, most of them are not the top UP hosts, but the UP hosts whose videos have 0 views. You can take a look at the latest videos on Bilibili. After flipping through dozens of pages, there are only 0 views, which is extremely spectacular.
• Interesting life experiences have increased

Okay, let’s get back to the topic.

Now the MOD is basically discontinued, and the UP owner is too lazy to continue working on it seriously.

Here we mainly talk about technology-related things, that is, a pure front-end implementation, an XML online editor for writing MODs.

It is a VSCode-style editor that can automatically learn the game MOD file generation constraint rules to help us implement code prompts and code verification.

What's more important is that it can directly modify files on your computer.

This is the final product code repository: https://gitee.com/vvjiang/mod-xml-editor

And a picture of the finished product:

Technologies covered in this blog:

• CodeMirror
• react-codemirror2
• xmldom
• FileReader
• IndexDB
• Web Workers
• File System Access

Let’s start from the beginning.

The need for online XML editors

When making a MOD for Mount & Blade 2, you need to write XML files frequently.

Because the data configuration of Mount & Blade 2 is saved in the form of XML, and after the MOD is loaded, the MOD's XML is used to overwrite the official XML.

Usually when we work on MOD data, we refer to the official XML and write the XML file ourselves.

But this will lead to a problem. XML has no code hints and code verification, and it is difficult to detect a wrong character.

Or sometimes when the game is updated, its XML rules may change.

The official will not issue a notification to tell you about these changes, so if you still use the previous elements and attributes, it means you are writing them wrong.

The result of writing incorrectly is often that the game crashes directly when loading the MOD, and it won’t give you any prompts. You can only slowly look for the bug.

As Mount & Blade 2 is a large-scale game, it takes a long time to start up each time, which means that the testing process for testing whether the MOD data is configured correctly will be very long.

Oh my god, there have been so many nights when I broke down the moment the game crashed.

So later I thought about making an XML online editor to solve this problem.

Technology Pre-research

Visual Programming

Actually, I didn't have the idea of ​​making this XML editor at the beginning, because it looks difficult to use. Instead, I wanted to implement it through visual programming by dragging and dropping elements and attributes.

You know what, I actually made a preliminary plan, but after dragging and dropping the configuration of a large XML thing countless times, I gradually lost my temper and gave up this plan.

VSCODE Plugin

I want to see if there is any VSCode plug-in that can provide code hints. There is one that uses XSD for code verification, which seems to be provided by IBM.

But unfortunately it has been abandoned and can no longer be used, so give up this plan.

Online Editor

The reason why we used an online editor to do this was because in March and April the company wanted to make something that could edit the Java project environment XML configuration files online.

Then I tried to make one and learned about CodeMirror .

CodeMirror supports XML code hints by configuring tags itself, but does not support XML code verification, so you need to do XML code verification yourself.

And because we usually use xsd to verify xml, we also need to convert xsd into CodeMirror 's tags configuration.

Whether it is Baidu, Google, or Github, I can't find the corresponding solution, so I can only write the code myself to implement it.

During this process, I gained a deeper understanding of CodeMirror , xsd , and htmllint , and finally completed the project.

Because this is the code of the previous company, I will not release it here.

In short, it was during this process that I learned about CodeMirror and came up with the idea of ​​using CodeMirror to make an online editor for MOD.

Original form: simple online XML editor

Okay, without further ado, just pick up the keyboard and start working.

The initial form did not have a file tree on the left, only a simple editor and a rule learning pop-up box.

There are three technologies involved:

CodeMirror

FileReader

xmldom

Using CodeMirror as an editor

CodeMirror mainly uses react-codemirror2, a packaged version of react. Anyway, just read the documentation and demo to configure it yourself.

The only difficulty is that many of the CodeMirror configuration introductions on the Internet are copied and reproduced, and some are still wrong, which is outrageous.

In short, if you want to play around, it’s best to read the official documentation (https://codemirror.net/) and the demos on the documentation, and then study it yourself. If you copy other people’s configurations, the water is very deep and you can’t control it.

Here I'll post a piece of configuration code for the editor component I encapsulated. It's definitely usable and has most of the editor's functions OK, but it's only suitable for editing XML.

The comments in it are quite detailed, including commonly used code folding and code formatting. I am too lazy to explain them one by one. You can refer to the official website to see for yourself.

I won’t post some of the referenced codes here. If you are interested, you can go to the code repository mentioned above to have a look.

import { useEffect } from 'react'
import { Controlled as ControlledCodeMirror } from 'react-codemirror2'
import CodeMirror from 'codemirror'
import 'codemirror/lib/codemirror.css'
import 'codemirror/theme/ayu-dark.css'
import 'codemirror/mode/xml/xml.js'
// Cursor line code highlight import 'codemirror/addon/selection/active-line'
// Folding code import 'codemirror/addon/fold/foldgutter.css'
import 'codemirror/addon/fold/foldcode.js'
import 'codemirror/addon/fold/xml-fold.js'
import 'codemirror/addon/fold/foldgutter.js'
import 'codemirror/addon/fold/comment-fold.js'
// Code hint completion and import 'codemirror/addon/hint/xml-hint.js'
import 'codemirror/addon/hint/show-hint.css'
import './hint.css'
import 'codemirror/addon/hint/show-hint.js'
// Code verification import 'codemirror/addon/lint/lint'
import 'codemirror/addon/lint/lint.css'
import CodeMirrorRegisterXmlLint from './xml-lint'
// Automatically type the closing tag when typing > import 'codemirror/addon/edit/closetag.js'
// Comments import 'codemirror/addon/comment/comment.js'

// Used to adjust the theme style of codeMirror import style from './index.less'

// Register Xml code verification CodeMirrorRegisterXmlLint(CodeMirror)

// Formatting related CodeMirror.extendMode("xml", {
commentStart: "<!--",
commentEnd: "-->",
newlineAfterToken: function (type, content, textAfter, state) {
    return (type === "tag" && />$/.test(content) && state.context) ||
    /^</.test(textAfter);
}
});

// Format the specified range CodeMirror.defineExtension("autoFormatRange", function (from, to) {
var cm = this;
var outer = cm.getMode(), text = cm.getRange(from, to).split("\n");
var state = CodeMirror.copyState(outer, cm.getTokenAt(from).state);
var tabSize = cm.getOption("tabSize");

var out = "", lines = 0, atSol = from.ch === 0;
function newline() {
    out += "\n";
    atSol = true;
    ++lines;
}

for (var i = 0; i < text.length; ++i) {
    var stream = new CodeMirror.StringStream(text[i], tabSize);
    while (!stream.eol()) {
    var inner = CodeMirror.innerMode(outer, state);
    var style = outer.token(stream, state), cur = stream.current();
    stream.start = stream.pos;
    if (!atSol || /\S/.test(cur)) {
        out += cur;
        atSol = false;
    }
    if (!atSol && inner.mode.newlineAfterToken &&
        inner.mode.newlineAfterToken(style, cur, stream.string.slice(stream.pos) || text[i + 1] || "", inner.state))
        newline();
    }
    if (!stream.pos && outer.blankLine) outer.blankLine(state);
    if (!atSol && i < text.length - 1) newline();
}

cm.operation(function () {
    cm.replaceRange(out, from, to);
    for (var cur = from.line + 1, end = from.line + lines; cur <= end; ++cur)
    cm.indentLine(cur, "smart");
    cm.setSelection(from, cm.getCursor(false));
});
});

// Xml editor component function XmlEditor(props) {
const { tags, value, onChange, onErrors, onGetEditor, onSave } = props

useEffect(() => {
    // Every time tags changes, the validation rules will be changed again CodeMirrorRegisterXmlLint(CodeMirror, tags, onErrors)
}, [onErrors, tags])

// Start tag function completeAfter(cm, pred) {
    if (!pred || pred()) setTimeout(function () {
    if (!cm.state.completionActive)
        cm.showHint({
        completeSingle: false
        });
    }, 100);
    return CodeMirror.Pass;
}

// End tag function completeIfAfterLt(cm) {
    return completeAfter(cm, function () {
    var cur = cm.getCursor();
    return cm.getRange(CodeMirror.Pos(cur.line, cur.ch - 1), cur) === "<";
    });
}

// Attributes and attribute values ​​function completeIfInTag(cm) {
    return completeAfter(cm, function () {
    var tok = cm.getTokenAt(cm.getCursor());
    if (tok.type === "string" && (!/['"]/.test(tok.string.charAt(tok.string.length - 1)) || tok.string.length === 1)) return false;
    var inner = CodeMirror.innerMode(cm.getMode(), tok.state).state;
    return inner.tagName;
    });
}

return (
    <div className={style.editor} >
    <ControlledCodeMirror
        value={value}
        options={{
        mode: {
            name: 'xml',
            // Whether to add the length of the tag when the xml attribute wraps multilineTagIndentPastTag: false
        },
        indentUnit: 2, // How many spaces are the default indent for line breaks theme: 'ayu-dark', // Editor theme lineNumbers: true, // Whether to display line numbers autofocus: true, // Automatically get focus styleActiveLine: true, // Highlight the cursor line code autoCloseTags: true, // Automatically type the end element when entering > toggleComment: true, // Open comments // Fold code begin
        lineWrapping: true,
        foldGutter: true,
        gutters: ['CodeMirror-linenumbers', 'CodeMirror-foldgutter', 'CodeMirror-lint-markers'],
        //Folding code end
        extraKeys: {
            // Code hint "'<'": completeAfter,
            "'/'": completeIfAfterLt,
            "' '": completeIfInTag,
            "'='": completeIfInTag,
            // Comment function "Ctrl-/": (cm) => {
            cm.toggleComment()
            },
            // Save function "Ctrl-S": (cm) => {
            onSave()
            },
            // Format "Shift-Alt-F": (cm) => {
            const totalLines = cm.lineCount();
            cm.autoFormatRange({ line: 0, ch: 0 }, { line: totalLines })
            },
            // Tab is automatically converted to space "Tab": (cm) => {
            if (cm.somethingSelected()) {// Overall indentation after selection cm.indentSelection('add')
            } else {
                cm.replaceSelection(Array(cm.getOption("indentUnit") + 1).join(" "), "end", "+input")
            }
            }
        },
        // Code hint hintOptions: { schemaInfo: tags, matchInMiddle: true },
        lint: true
        }}
        editorDidMount={onGetEditor}
        onBeforeChange={onChange}
    />
    </div>
)
}

export default XmlEditor

Learn XML and extract tags rules

When we use CodeMirror to make a simple editor, we need to use tags if we want to provide an XML code prompt.

Obviously, different games have different XML rules, and the XML rules may change when the game is updated.

Therefore, we must ensure that there is a mechanism to continuously learn these XML rules, so here I have made a pop-up window for learning XML file rules to do this.

Click on the constraint rule in the upper left corner of the editor -> Add constraint rule

A pop-up window like this will pop up:

Read the XML files in the specified folder through FileReader, and then use xmldom to parse the text of these XML files in turn to generate document objects.

Then analyze these document objects to get the final tags rules.

This step only requires some understanding of XML, which is actually quite basic, so I won’t explain it.

In short, now we have completed its initial form. Every time you use it, you need to copy the content of the XML file you edited into this online editor. After editing, copy the completed text into the original XML file and save it to overwrite it.

Evolution: Online XML editor with tree-like file structure and full file validation

The above editor actually has a very narrow usage scenario and can only be used when writing a new XML.

A MOD often consists of dozens, hundreds, or even thousands of files. It is impossible to paste them one by one into the editor for verification.

So we need to load all the XML files of the MOD in this editor and perform a code verification.

There are two technologies involved:

• FileReader
• Web Workers

Left file tree

The file tree on the left is completed using Ant Design's Tree component, so I won't go into details about the configuration here.

When you click the Open Folder button

Also use FileReader to read files in the MOD folder.

However, FileReader obtains an array of files. To generate the tree structure on the left, we need to manually parse the path of each XML file and generate a tree structure based on it.

Full file verification function

The moment the folder is opened, we need to perform a code verification on all XML files. If the verification is incorrect, we need to mark all related files and a series of folders of its parent and ancestor levels in red on the folder on the left.

This function seems simple, but it actually has a lot of pitfalls, because the amount of verification calculation is actually not small, especially when your MOD has hundreds or thousands of files, it is very easy to cause your js to be blocked and the page to become unresponsive.

Here I use Web Worker to open a new thread to handle the verification process and return the result to me after the verification is completed.

In this process, I also learned more about the use of Web Workers.

I have always thought that it was a new Worker (a js file) and it feels difficult to use it in combination with react's modular development.

But in fact, now you can use Web Workers very conveniently by configuring worker-loader in webpack.

First, our worker code can be written as follows:

import { lintFileTree } from '@/utils/files'

onmessage = ({ data }) => {
lintFileTree(data.fileTree, data.currentTags).then(content => {
    postMessage(content)
})
}

Then when we use this Worker, we can do it as follows

import { useWebWorkerFromWorker } from 'react-webworker-hook'
import lintFileTreeWorker from '@/utils/webWorker/lintFileTree.webworker'

const worker4LintFileTree = new lintFileTreeWorker()

const [lintedFileTree, startLintFileTree] = useWebWorkerFromWorker(worker4LintFileTree)

Then you use useEffect to depend on this lintedFileTree, and do something if it changes, so it is as easy to write as using useState.

Non-recursive tree traversal

You can see that many of the things we use above are related to trees, such as traversing the file tree to verify the code.

Or after we switch a constraint rule, we also need to traverse the entire file tree for rechecking.

During the traversal process, I used recursion to traverse the entire tree. The disadvantage of this is that the memory cannot be released during recursion, so later I changed the algorithm and used a non-recursive method to traverse the entire tree.

IndexDB saves file contents

Because our MOD files are large and contain a lot of content, they may occupy a lot of memory, and it is impossible to keep these file contents in the memory all the time.

So I read the file contents and put them into IndexDB one by one, and only display the contents of the currently edited file.

The file content is retrieved from IndexDB again only when needed, such as when verifying the entire file or switching files.

Ultimate evolution: Break through the browser sandbox restrictions and achieve the addition, deletion and modification of local files on the computer

Through the previous operations, we have finally completed a basically usable online XML editor.

However, it has a fatal flaw, which is that it is limited by the browser sandbox environment. After we modify the file, we cannot save it directly to the computer, but must rely on manually copying the modified code to the corresponding file one by one.

This operation is cumbersome and complicated, which means that the functions of our editor may only be used to assist in code writing and batch verification.

I thought this was all I could do before, but then I happened to read a post on Zhihu and found that the Chrome86+ version has an additional functional API: FileSystemAccess.

In addition, unless it is a local localhost environment, this API can only be called in an https environment. That is to say, if you are on an http website, you cannot call it even if you are using Chrome86+ or Edge86+.

This API allows us to directly operate files on the local computer, instead of only being able to read like FileReader, or only being able to operate within the browser sandbox like FileSystem.

Through FileSystemAccess, we can not only read and modify files in the folder, but also add and delete files.

So I used this API to completely replace all the points where FileReader was used before, and realized the addition and deletion of folders and files by right-clicking on the file tree. (File renaming is not supported here, but we can actually simulate renaming by deleting and then adding, but I'm too lazy to do it)

At the same time, after pressing the Save button or the save shortcut key Ctrl+S, you can directly save the file.

The following is a component code that uses FileSystemAccess to open a folder:

    import React from 'react'

    // Customized open folder component const FileInput = (props) => {
    const { children, onChange } = props
    const handleClick = async () => {
        const dirHandle = await window.showDirectoryPicker()
        dirHandle.requestPermission({ mode : "readwrite" })
        onChange(dirHandle)
    }
    return <span onClick={handleClick}>
        {children}
    </span>
    }

    export default FileInput

As long as the element wrapped by this component (such as a button) is clicked, showDirectoryPicker will be called immediately to request to open the folder.

After opening the folder, request the folder write permission through the obtained folder handle, and then pass this folder handle to the outside to obtain the file tree structure.

The operation here is flawed, because when requesting to open a folder, the browser will pop up a box to ask the user for permission to read the folder.

After opening, a second box will pop up to obtain write permission, which means that two boxes will pop up when opening a folder.

But I can only request all permissions at once through this method, otherwise it is not a good idea to request permissions again when you want to save.

However, the advantages outweigh the disadvantages. This API not only realizes the addition, deletion and modification of files, but also eliminates the use of IndexDB.

Because we can get the corresponding file content through the file handle at any time, there is no need to save the file content in IndexDB.

More features and details

The above is just an overview of the core technical functions. In fact, this editor has many more details.

For example, the panel for adjusting tags rules, the buttons on the toolbar, the simple encapsulation of dva, and when analyzing XML, if the attribute value is a number, no prompt will be given but it will be ignored directly, because numbers often do not make much sense and the enumeration value is too large.

There are so many of these, but their applications are relatively basic, so I don’t want to go into details, otherwise this blog will become very long and it will be difficult to highlight the core ideas.

Shortcomings and Summary

The shortcomings here are more due to laziness. For example, the folder and file renaming function mentioned earlier, and the custom rules for adjusting tags rules do not support modification and deletion.

It's doable, I'm just too lazy to do it.

This thing took me several months to work on. It's not that I wrote about it every night. I wrote about it when I had inspiration, or when I found something that could be improved.

In total, I worked on this every night for about two or three weeks, and then as it became more complete and usable, I became more and more lazy about it.

Because the remaining operations are not very important and can be completed with just a little imagination, there are not many challenging parts.

But overall, the usability of this thing is still very strong.

It can not only be used to assist in the writing of XML files for a series of games such as "Mount & Blade 2", "The Great Cultivation Simulator", and "Civilization 6", but can also be used for XML configurations that do not have XSD rules and are too complex. It can even learn your custom XML rules.

This is the end of this article about using js to create an XML online editor. For more information about using js to create an XML online editor, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!