7c358cd5e6
* Modernize Whitespace token: basic lexer * Modernize Whitespace token: bibtex lexer * Modernize Whitespace token: boa lexer * Modernize Whitespace token: capnproto lexer + new example * Modernize Whitespace token: cddl lexer * Modernize Whitespace token: chapel lexer * Modernize Whitespace token: c_like lexer * Modernize Whitespace token: configs lexer * Modernize Whitespace token: console lexer * Modernize Whitespace token: crystal lexer * Modernize Whitespace token: csound lexer * Modernize Whitespace token: css lexer * Revert a change in basic lexer
56 lines
1.5 KiB
Text
56 lines
1.5 KiB
Text
---input---
|
|
"A#{ (3..5).group_by { |x| x/2}.map do |k,v| "#{k}" end.join }" + "Z"
|
|
|
|
---tokens---
|
|
'"' Literal.String.Double
|
|
'A' Literal.String.Double
|
|
'#{' Literal.String.Interpol
|
|
' ' Text.Whitespace
|
|
'(' Punctuation
|
|
'3' Literal.Number.Integer
|
|
'..' Operator
|
|
'5' Literal.Number.Integer
|
|
')' Punctuation
|
|
'.' Operator
|
|
'group_by' Name
|
|
' ' Text.Whitespace
|
|
'{' Literal.String.Interpol
|
|
' ' Text.Whitespace
|
|
'|' Operator
|
|
'x' Name
|
|
'|' Operator
|
|
' ' Text.Whitespace
|
|
'x' Name
|
|
'/' Operator
|
|
'2' Literal.Number.Integer
|
|
'}' Literal.String.Interpol
|
|
'.' Operator
|
|
'map' Name
|
|
' ' Text.Whitespace
|
|
'do' Keyword
|
|
' ' Text.Whitespace
|
|
'|' Operator
|
|
'k' Name
|
|
',' Punctuation
|
|
'v' Name
|
|
'|' Operator
|
|
' ' Text.Whitespace
|
|
'"' Literal.String.Double
|
|
'#{' Literal.String.Interpol
|
|
'k' Name
|
|
'}' Literal.String.Interpol
|
|
'"' Literal.String.Double
|
|
' ' Text.Whitespace
|
|
'end' Keyword
|
|
'.' Operator
|
|
'join' Name
|
|
' ' Text.Whitespace
|
|
'}' Literal.String.Interpol
|
|
'"' Literal.String.Double
|
|
' ' Text.Whitespace
|
|
'+' Operator
|
|
' ' Text.Whitespace
|
|
'"' Literal.String.Double
|
|
'Z' Literal.String.Double
|
|
'"' Literal.String.Double
|
|
'\n' Text.Whitespace
|