translate alias of named symbols

milahu commented

2023-03-04 14:24:03 +01:00

(Migrated from github.com)

support tree-sitter grammars like

module.exports = grammar({
  rules: {
    a: $ => "aaa",
    c: $ => alias($.a, $.b), // alias of the named symbol a
  }
}

tested on tree-sitter-make (result)

not sure if the tokens-logic is needed (this.tokenNewNames etc)

support tree-sitter grammars like ```js module.exports = grammar({ rules: { a: $ => "aaa", c: $ => alias($.a, $.b), // alias of the named symbol a } } ``` tested on [tree-sitter-make](https://github.com/alemuller/tree-sitter-make) ([result](https://github.com/milahu/lezer-parser-import-tree-sitter-scanner/tree/master/test/cases/make/lezer-parser-make/src)) not sure if the tokens-logic is needed (`this.tokenNewNames` etc)

marijnh commented

2023-03-06 13:14:59 +01:00

(Migrated from github.com)

I do not understand what this does. Can you give an example of what kind of output you are producing for a renamed rule?

milahu commented

2023-03-06 14:57:54 +01:00

(Migrated from github.com)

the inner symbol is hidden by adding a underscore-prefix "_" to the tree-sitter symbol name

i added a test case in my fork: test-alias-of-named-symbol

tree-sitter grammar

module.exports = grammar({
  name: "test_alias_of_named_symbol",

  rules: {
    document: $ => repeat(choice($.word, $.parens_word)),
    word: $ => /[a-z]+/,
    parens_word: $ => seq("(", alias($.word, $.word_in_parens), ")"),
  },
});

tree-sitter test

================================================================================
hello node
================================================================================

a (b) c

--------------------------------------------------------------------------------

(document
  (word)
  (parens_word
    (word_in_parens))
  (word))

lezer grammar

@top Document {
  (Word | ParensWord)*
}

ParensWord {
  "(" WordInParens { word1 } ")"
}

@skip { token_1 }

@tokens {
  token_1 {
    $[ \t\r\n]
  }
  Word { word1 }
  word1 {
    $[a-z]+
  }
}

lezer test

# test
a (b) c
==>
Document (
  Word,
  ParensWord (
    WordInParens
  ),
  Word
)

not sure if the tokens-logic is needed

yes, see test: Word is a token

the inner symbol is hidden by adding a underscore-prefix "_" to the tree-sitter symbol name i added a test case in my fork: [test-alias-of-named-symbol](https://github.com/milahu/lezer-parser-import-tree-sitter-scanner/tree/11ab876d314a1e3fd41e4145c9b28d85e045ac1f/test/cases/test-alias-of-named-symbol) [tree-sitter grammar](https://github.com/milahu/lezer-parser-import-tree-sitter-scanner/blob/11ab876d314a1e3fd41e4145c9b28d85e045ac1f/test/cases/test-alias-of-named-symbol/tree-sitter-test-alias-of-named-symbol/grammar.js) ```js module.exports = grammar({ name: "test_alias_of_named_symbol", rules: { document: $ => repeat(choice($.word, $.parens_word)), word: $ => /[a-z]+/, parens_word: $ => seq("(", alias($.word, $.word_in_parens), ")"), }, }); ``` [tree-sitter test](https://github.com/milahu/lezer-parser-import-tree-sitter-scanner/blob/11ab876d314a1e3fd41e4145c9b28d85e045ac1f/test/cases/test-alias-of-named-symbol/tree-sitter-test-alias-of-named-symbol/test/corpus/hello.txt) ``` ================================================================================ hello node ================================================================================ a (b) c -------------------------------------------------------------------------------- (document (word) (parens_word (word_in_parens)) (word)) ``` [lezer grammar](https://github.com/milahu/lezer-parser-import-tree-sitter-scanner/blob/11ab876d314a1e3fd41e4145c9b28d85e045ac1f/test/cases/test-alias-of-named-symbol/lezer-parser-test-alias-of-named-symbol/src/grammar.lezer) ``` @top Document { (Word | ParensWord)* } ParensWord { "(" WordInParens { word1 } ")" } @skip { token_1 } @tokens { token_1 { $[ \t\r\n] } Word { word1 } word1 { $[a-z]+ } } ``` [lezer test](https://github.com/milahu/lezer-parser-import-tree-sitter-scanner/blob/8102a4d960b88703195c650c2ad1e0ccfa77b33e/test/cases/test-alias-of-named-symbol/lezer-parser-test-alias-of-named-symbol/test/hello.txt) ``` # test a (b) c ==> Document ( Word, ParensWord ( WordInParens ), Word ) ``` > not sure if the tokens-logic is needed yes, see test: Word is a token

marijnh commented

2023-03-07 08:29:18 +01:00

(Migrated from github.com)

I see, it renames the original token in order to be able to wrap it. I can see this introducing new issues (of ambiguous tokens), and my approach so far was that this kind of renaming or wrapping is something the user will have to address manually.

Again, I don't want to spend much time maintaining this, so I don't want to complicate it with further features or spend time reviewing patches. If you want to move this tool forward, I suggest you work on a fork instead. I'm going to decline this patch.

I see, it renames the original token in order to be able to wrap it. I can see this introducing new issues (of ambiguous tokens), and my approach so far was that this kind of renaming or wrapping is something the user will have to address manually. Again, I don't want to spend much time maintaining this, so I don't want to complicate it with further features or spend time reviewing patches. If you want to move this tool forward, I suggest you work on a fork instead. I'm going to decline this patch.

Pull request closed

This pull request cannot be reopened because the branch was deleted.

Rows
Columns

translate alias of named symbols #8

Pull request closed