Parse more variations of bits and bytes #6

Closed

tiwilliam wants to merge 2 commits from parse-binary-better into main

tiwilliam commented

2022-08-13 23:51:45 +02:00

(Migrated from github.com)

This adds new tokens for bits and bytes. We now support these three variations of bits and bytes syntax:

Regular bit style: b'0101'
Unquoted bit style (MySQL): 0b0101
Byte style (BigQuery): b'abcd'

I've also added a few tests for these new parsers.

This work was triggered to resolve this syntax parsing issue for BigQuery byte strings:
Screenshot 2022-08-13 at 23 46 18

PostgreSQL 14 Reference: https://www.postgresql.org/docs/14/functions-bitstring.html
MySQL 8.0 Reference: https://dev.mysql.com/doc/refman/8.0/en/bit-value-literals.html
BigQuery Example: https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#byte_length

This adds new tokens for bits and bytes. We now support these three variations of bits and bytes syntax: * Regular bit style: b'0101' * Unquoted bit style (MySQL): 0b0101 * Byte style (BigQuery): b'abcd' I've also added a few tests for these new parsers. This work was triggered to resolve this syntax parsing issue for BigQuery byte strings: <img width="335" alt="Screenshot 2022-08-13 at 23 46 18" src="https://user-images.githubusercontent.com/190349/184511762-f2eb8fcd-2183-440e-8dd2-1c5f88a2e8dc.png"> PostgreSQL 14 Reference: https://www.postgresql.org/docs/14/functions-bitstring.html MySQL 8.0 Reference: https://dev.mysql.com/doc/refman/8.0/en/bit-value-literals.html BigQuery Example: https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#byte_length

marijnh (Migrated from github.com) reviewed

2022-08-14 08:54:45 +02:00

src/tokens.ts

It looks like the 0b format was parsed by default before and now requires an option to be enabled. Is that intentional?

Same question for no longer accepting 0B.

It looks like the `0b` format was parsed by default before and now requires an option to be enabled. Is that intentional? Same question for no longer accepting `0B`.

tiwilliam (Migrated from github.com) reviewed

2022-08-14 09:47:07 +02:00

src/tokens.ts

You are correct, I have found 0b to be supported in MySQL only, hence only enabling that route for those parsers since PostgreSQL for example would parse it as a integer and then get a syntax error at b. But I’m open to switch the default if we want to be backwards-compatible.

0B is removed because it’s not valid according to MySQL docs.

You are correct, I have found 0b to be supported in MySQL only, hence only enabling that route for those parsers since PostgreSQL for example would parse it as a integer and then get a syntax error at `b`. But I’m open to switch the default if we want to be backwards-compatible. 0B is removed because it’s not valid according to MySQL docs.

marijnh commented

2022-08-14 11:17:58 +02:00

(Migrated from github.com)

Thanks, merged as cc88732 (after reindenting the new test file to use 2 spaces).

Thanks, merged as cc88732 (after reindenting the new test file to use 2 spaces).

tiwilliam commented

2022-08-14 11:25:34 +02:00

(Migrated from github.com)

Thank you for prompt feedback and maintaining CodeMirror! ❤️

Pull request closed

This pull request cannot be reopened because the branch was deleted.

No reviewers

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

codemirror/lang-sql!6

No description provided.

Rows
Columns