Description

This PR adds support for injected languages using tree-sitter as described in #16 and in the tree-sitter documentation. Languages can contain injected languages which are described in a language's injections.scm file. Some examples of injected languages are:

• HTML contains CSS and Javascript in style and script tags
• Javascript contains Regex literals
• PHP contains HTML between the <php tags
• C++ can contain rawstring literals of arbitrary languages

Details

This PR is a rework of the TreeSitterClient class. Specifically it:

• Adds a layers array and primaryLayer property. layers contains all language layers in the document, and primaryLayer is the ID of the document's primary language.
• Each layer is a LanguageLayer object. These objects represent an injected language-range(s) combination. Each layer can have one or more range associated with it depending on if it should be parsed as one document or multiple.
• When editing:
  • Each layer's ranges are updated using a similar algorithm to tree-sitter, and edits are applied to make use of the incremental parsing tree-sitter gives.
  • Each layer is checked for any injections, inserting any new injected layers and keeping track of any 'untouched' layers to remove.
  • Any layers that were not touched are removed.

The highlight query algorithm is largely the same, but keeps track of any ranges not used by any injected layers, and only queries the primary layer for those ranges so as not to override any injected highlights.

Related Issues

• Closes ✨ Injected Languages #16

Screenshots

Before, other languages were detected but parsed and highlighted as normal text.

With Injected languages, in this case CSS and JS embedded in HTML and a second layer of Regex embedded in JS embedded in HTML: