7c358cd5e6
* Modernize Whitespace token: basic lexer * Modernize Whitespace token: bibtex lexer * Modernize Whitespace token: boa lexer * Modernize Whitespace token: capnproto lexer + new example * Modernize Whitespace token: cddl lexer * Modernize Whitespace token: chapel lexer * Modernize Whitespace token: c_like lexer * Modernize Whitespace token: configs lexer * Modernize Whitespace token: console lexer * Modernize Whitespace token: crystal lexer * Modernize Whitespace token: csound lexer * Modernize Whitespace token: css lexer * Revert a change in basic lexer
25 lines
406 B
Text
25 lines
406 B
Text
---input---
|
|
'a'
|
|
'я'
|
|
'\u{1234}'
|
|
'
|
|
'
|
|
'abc'
|
|
|
|
---tokens---
|
|
"'a'" Literal.String.Char
|
|
'\n' Text.Whitespace
|
|
|
|
"'я'" Literal.String.Char
|
|
'\n' Text.Whitespace
|
|
|
|
"'\\u{1234}'" Literal.String.Char
|
|
'\n' Text.Whitespace
|
|
|
|
"'\n'" Literal.String.Char
|
|
'\n' Text.Whitespace
|
|
|
|
"'" Error
|
|
'abc' Name
|
|
"'" Error
|
|
'\n' Text.Whitespace
|