lezer/generator
1
0
Fork 0
Code Issues Pull requests Activity

fix: stringify numeric separated literals with a leading zero #15

Closed
deemp wants to merge 1 commit from main into main
pull from: main
merge into: lezer:main
Conversation 1 Commits 1 Files changed 1 +1 -1
deemp commented 2023-09-27 20:51:02 +02:00 (Migrated from github.com)
Copy link

More examples of invalid numbers:
https://github.com/microsoft/TypeScript/issues/39563

For this grammar

@top Program {
    repro
}

repro { var | tok }

@tokens {
    var { ![\n] }
}

tok { @specialize<var, "0_2"> }

I now get

// This file was generated by lezer-generator. You probably shouldn't edit it.
import {LRParser} from "@lezer/lr"
const spec_var = {__proto__:null,"0_2":10}
export const parser = LRParser.deserialize({
  version: 14,
  states: "bOQOPOOOOOO'#C_'#C_QOOOOO",
  stateData: "Y~OSPOTPO~O",
  goto: "WSPPPTRQO",
  nodeNames: "⚠ Program",
  maxTerm: 5,
  skippedNodes: [0],
  repeatNodeCount: 0,
  tokenData: "j~RSOY_Z;'S_;'S;=`d<%lO_~dOS~~gP;=`<%l_",
  tokenizers: [0],
  topRules: {"Program":[0,1]},
  specialized: [{term: 4, get: (value) => spec_var[value] || -1}],
  tokenPrec: 0
})

notice "0_2"

Before, 0_2:10 produced a syntax error.

More examples of invalid numbers: https://github.com/microsoft/TypeScript/issues/39563 For this grammar ```grammar @top Program { repro } repro { var | tok } @tokens { var { ![\n] } } tok { @specialize<var, "0_2"> } ``` I now get ```console // This file was generated by lezer-generator. You probably shouldn't edit it. import {LRParser} from "@lezer/lr" const spec_var = {__proto__:null,"0_2":10} export const parser = LRParser.deserialize({ version: 14, states: "bOQOPOOOOOO'#C_'#C_QOOOOO", stateData: "Y~OSPOTPO~O", goto: "WSPPPTRQO", nodeNames: "⚠ Program", maxTerm: 5, skippedNodes: [0], repeatNodeCount: 0, tokenData: "j~RSOY_Z;'S_;'S;=`d<%lO_~dOS~~gP;=`<%l_", tokenizers: [0], topRules: {"Program":[0,1]}, specialized: [{term: 4, get: (value) => spec_var[value] || -1}], tokenPrec: 0 }) ``` notice `"0_2"` Before, `0_2:10` produced a syntax error.
marijnh commented 2023-09-28 07:53:00 +02:00 (Migrated from github.com)
Copy link

This would also go wrong for other strings (say 1a). Attached patch solves the issue in a somewhat different way.

This would also go wrong for other strings (say `1a`). Attached patch solves the issue in a somewhat different way.
👍 1

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lezer/generator!15
No description provided.