4 Commits

Author SHA1 Message Date
hgranthorner
e3ff13f9f3 Tier 2 golden trees, and #scope_file is a statement not a modifier
isDeclarationAhead used to skip *any* directive, so '#scope_file' was absorbed as
a modifier of the declaration after it. That misreports scope, since a scope
directive governs everything that follows it. Only #as, #overlay, #add_context
and #no_reset actually prefix a declaration in the corpus; the rest are
statements now, which the regenerated Directives.txt shows.

Golden fixtures cover the reference doc's gotcha list: declaration forms,
procedures (named/multiple/variadic/polymorphic/foreign returns), structs, enums,
unions, array and pointer types, the 'if x == { case }' switch form, for
modifiers, ifx, and directives including a here-string.

DebugParseTest is now inert by default and documented as the scratch loop.

generateParser wipes its output through a Delete task rather than a doFirst,
because a lambda added from a Kotlin build script captures the script object and
breaks the configuration cache ('./jaigradle check' failed to store it).

check and verifyPlugin (IU-253/261/262) both green.
2026-08-04 12:48:30 -04:00
hgranthorner
dc53033898 Build: regenerate the parser on every build, and always from scratch
Two staleness traps cost real debugging time, and both made grammar edits look
like they had no effect:

- compileJava/compileKotlin read src/main/gen with no task dependency, so an
  edit to Jai.bnf could be compiled against the previous generated parser. The
  compiled JaiParser.class was three minutes older than its source while the
  tests reported green. They now depend on generateParser.
- purgeOldFiles is not enough. Grammar-Kit rewrites only the files whose rule
  changed, so JaiDeclaration kept a getArgumentList() its regenerated Impl no
  longer had and the build failed inside generated code. generateParser now
  deletes src/main/gen first.

Grammar: named return defaults use a private rule rather than 'initializer'.
Giving the default its own PSI element made the enclosing procLiteralExpr end
early, so '-> a: int = 1 { }' silently lost its body.

Corpus parse 90.9% (649/714), ratchet at 645.
2026-08-04 12:37:46 -04:00
hgranthorner
37809b8aa7 Phase 3: Grammar-Kit parser, PSI and ParserDefinition
Adds src/main/grammar/Jai.bnf, the generated parser/PSI in src/main/gen, a
JaiParserDefinition, and the Tier 3 corpus parse gate (406/714 files clean).

Two things were not obvious:

- Grammar-Kit mints its own token instances from the .bnf 'tokens' block, which
  are different objects from the ones JaiLexer emits, so every rule silently
  failed to match. Fixed with tokenTypeFactory -> JaiTokenTypes.byName.
- A backslash inside an identifier is a continuation in the compiler's lexer
  (Jai_Lexer/module.jai:444): 'left\_margin' is one identifier. JaiLexer now
  does the same; the Tier 0 round-trip still holds because the token span
  covers the backslash and the skipped spaces.

Constructs that BNF alone cannot express live in JaiParserUtil: directive-name
tests (#ident is one token), '==' before '{' for the switch form, procedure
header vs parenthesised expression, and the declaration lookahead.
2026-08-04 12:05:42 -04:00
hgranthorner
cb5a1d9555 initial commit 2026-08-04 10:12:15 -04:00