Commit Graph

2 Commits

Author SHA1 Message Date
hgranthorner
840a503e40 Parser: raise corpus parse rate from 57% to 69%
Grammar gains: array types, directive flags, declaration-level directive
modifiers, parameter markers, backticked declaration names, notes after a
block-valued declaration, 'if cond then stmt', 'ifx c else v', '#module_parameters'
two-group form, '(.*)' prefix dereference, and '#asm' bodies consumed opaquely
(they are x86-64 assembly, not Jai).

Return items no longer take a default value: allowing one made
'f: (K) -> u32 = null' swallow the enclosing parameter's default and read the
rest of the parameter list as extra return values.
2026-08-04 12:13:26 -04:00
hgranthorner
37809b8aa7 Phase 3: Grammar-Kit parser, PSI and ParserDefinition
Adds src/main/grammar/Jai.bnf, the generated parser/PSI in src/main/gen, a
JaiParserDefinition, and the Tier 3 corpus parse gate (406/714 files clean).

Two things were not obvious:

- Grammar-Kit mints its own token instances from the .bnf 'tokens' block, which
  are different objects from the ones JaiLexer emits, so every rule silently
  failed to match. Fixed with tokenTypeFactory -> JaiTokenTypes.byName.
- A backslash inside an identifier is a continuation in the compiler's lexer
  (Jai_Lexer/module.jai:444): 'left\_margin' is one identifier. JaiLexer now
  does the same; the Tier 0 round-trip still holds because the token span
  covers the backslash and the skipped spaces.

Constructs that BNF alone cannot express live in JaiParserUtil: directive-name
tests (#ident is one token), '==' before '{' for the switch form, procedure
header vs parenthesised expression, and the declaration lookahead.
2026-08-04 12:05:42 -04:00