diff options
| author | Joel Klinghed <the_jk@spawned.biz> | 2025-09-29 09:39:49 +0200 |
|---|---|---|
| committer | Joel Klinghed <the_jk@spawned.biz> | 2025-09-29 09:50:47 +0200 |
| commit | d196d51e07f50f3510c43ad375c5559b58860023 (patch) | |
| tree | 3432b8e99e306d0ece9f29ddad1e2945f88a1481 /data/java-8/tokens.grammar | |
| parent | 1e9e51dae1c01bab7562911b958c47528b8011c8 (diff) | |
java: Add tokens support for Java 21
Some new keywords, I opted to modify java-8 grammar to use the new
names, even if they are not going to match anything. Makes the
tokenizer easier to write.
Diffstat (limited to 'data/java-8/tokens.grammar')
| -rw-r--r-- | data/java-8/tokens.grammar | 12 |
1 files changed, 12 insertions, 0 deletions
diff --git a/data/java-8/tokens.grammar b/data/java-8/tokens.grammar index 3521ac0..3941b94 100644 --- a/data/java-8/tokens.grammar +++ b/data/java-8/tokens.grammar @@ -43,7 +43,13 @@ Identifier: IdentifierChars: JavaLetter {JavaLetterOrDigit} +# Java 8 only has reserved keywords, but use modern names +# here to make a shared tokenizer simpler. Keyword: + ReservedKeyword + ContextualKeyword + +ReservedKeyword: abstract continue for @@ -95,14 +101,20 @@ Keyword: super while +ContextualKeyword: + Literal: IntegerLiteral FloatingPointLiteral BooleanLiteral CharacterLiteral StringLiteral + TextBlock NullLiteral +# Java 8 doesn't have TextBlock, but add it as newer grammers have it +TextBlock: + IntegerLiteral: DecimalIntegerLiteral HexIntegerLiteral |