Phase 3: Grammar-Kit parser, PSI and ParserDefinition

Adds src/main/grammar/Jai.bnf, the generated parser/PSI in src/main/gen, a
JaiParserDefinition, and the Tier 3 corpus parse gate (406/714 files clean).

Two things were not obvious:

- Grammar-Kit mints its own token instances from the .bnf 'tokens' block, which
  are different objects from the ones JaiLexer emits, so every rule silently
  failed to match. Fixed with tokenTypeFactory -> JaiTokenTypes.byName.
- A backslash inside an identifier is a continuation in the compiler's lexer
  (Jai_Lexer/module.jai:444): 'left\_margin' is one identifier. JaiLexer now
  does the same; the Tier 0 round-trip still holds because the token span
  covers the backslash and the skipped spaces.

Constructs that BNF alone cannot express live in JaiParserUtil: directive-name
tests (#ident is one token), '==' before '{' for the switch form, procedure
header vs parenthesised expression, and the declaration lookahead.
This commit is contained in:
hgranthorner
2026-08-04 12:05:42 -04:00
parent 746647d27d
commit 37809b8aa7
169 changed files with 8328 additions and 7 deletions

View File

@@ -52,3 +52,9 @@ Key docs:
| `AGENTS.md` | how to work in this repo; current state |
| `docs/JAI_LANGUAGE_REFERENCE.md` | the language, transcribed from the compiler |
| `docs/BUILD_PLAN.md` | phased plan and testing strategy |
Local IntelliJ references are available in `~/programming/thirdparty`:
- `fortran-plugin` — a simple language plugin to use as reference.
- `intellij-elixir` — a more complex plugin to use as reference.
- `intellij-sdk-docs` — local versions of the IntelliJ SDK documentation.