grant/intellijai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Phase 3: Grammar-Kit parser, PSI and ParserDefinition

Browse Source 
Adds src/main/grammar/Jai.bnf, the generated parser/PSI in src/main/gen, a
JaiParserDefinition, and the Tier 3 corpus parse gate (406/714 files clean).

Two things were not obvious:

- Grammar-Kit mints its own token instances from the .bnf 'tokens' block, which
  are different objects from the ones JaiLexer emits, so every rule silently
  failed to match. Fixed with tokenTypeFactory -> JaiTokenTypes.byName.
- A backslash inside an identifier is a continuation in the compiler's lexer
  (Jai_Lexer/module.jai:444): 'left\_margin' is one identifier. JaiLexer now
  does the same; the Tier 0 round-trip still holds because the token span
  covers the backslash and the skipped spaces.

Constructs that BNF alone cannot express live in JaiParserUtil: directive-name
tests (#ident is one token), '==' before '{' for the switch form, procedure
header vs parenthesised expression, and the declaration lookahead.
This commit is contained in:
hgranthorner
2026-08-04 12:05:42 -04:00
parent 746647d27d
commit 37809b8aa7
169 changed files with 8328 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
src/test/testData/parsing/Expressions.jai Normal file
View File

@@ -0,0 +1,2 @@
x = a + b * c;
foo(x, y + 1);

37
src/test/testData/parsing/Expressions.txt Normal file
View File

@@ -0,0 +1,37 @@
Jai File(0,29)
JaiStatementImpl(STATEMENT)(0,14)
JaiExpressionStatementImpl(EXPRESSION_STATEMENT)(0,14)
JaiAssignExprImpl(ASSIGN_EXPR)(0,13)
JaiRefExprImpl(REF_EXPR)(0,1)
PsiElement(Jai:IDENT)('x')(0,1)
PsiElement(Jai:=)('=')(2,3)
JaiAddExprImpl(ADD_EXPR)(4,13)
JaiRefExprImpl(REF_EXPR)(4,5)
PsiElement(Jai:IDENT)('a')(4,5)
PsiElement(Jai:+)('+')(6,7)
JaiMulExprImpl(MUL_EXPR)(8,13)
JaiRefExprImpl(REF_EXPR)(8,9)
PsiElement(Jai:IDENT)('b')(8,9)
PsiElement(Jai:*)('*')(10,11)
JaiRefExprImpl(REF_EXPR)(12,13)
PsiElement(Jai:IDENT)('c')(12,13)
PsiElement(Jai:;)(';')(13,14)
JaiStatementImpl(STATEMENT)(15,29)
JaiExpressionStatementImpl(EXPRESSION_STATEMENT)(15,29)
JaiCallExprImpl(CALL_EXPR)(15,28)
JaiRefExprImpl(REF_EXPR)(15,18)
PsiElement(Jai:IDENT)('foo')(15,18)
PsiElement(Jai:()('(')(18,19)
JaiArgumentImpl(ARGUMENT)(19,20)
JaiRefExprImpl(REF_EXPR)(19,20)
PsiElement(Jai:IDENT)('x')(19,20)
PsiElement(Jai:,)(',')(20,21)
JaiArgumentImpl(ARGUMENT)(22,27)
JaiAddExprImpl(ADD_EXPR)(22,27)
JaiRefExprImpl(REF_EXPR)(22,23)
PsiElement(Jai:IDENT)('y')(22,23)
PsiElement(Jai:+)('+')(24,25)
JaiLiteralExprImpl(LITERAL_EXPR)(26,27)
PsiElement(Jai:NUMBER)('1')(26,27)
PsiElement(Jai:))(')')(27,28)
PsiElement(Jai:;)(';')(28,29)

4
src/test/testData/parsing/SimpleDeclarations.jai Normal file
View File

@@ -0,0 +1,4 @@
a : float = 37.0;
b := 10;
c :: 42;
return a + b;

41
src/test/testData/parsing/SimpleDeclarations.txt Normal file
View File

@@ -0,0 +1,41 @@
Jai File(0,49)
JaiStatementImpl(STATEMENT)(0,17)
JaiDeclarationImpl(DECLARATION)(0,17)
JaiDeclNamesImpl(DECL_NAMES)(0,1)
PsiElement(Jai:IDENT)('a')(0,1)
PsiElement(Jai::)(':')(2,3)
JaiRefExprImpl(REF_EXPR)(4,9)
PsiElement(Jai:IDENT)('float')(4,9)
PsiElement(Jai:=)('=')(10,11)
JaiInitializerImpl(INITIALIZER)(12,16)
JaiLiteralExprImpl(LITERAL_EXPR)(12,16)
PsiElement(Jai:NUMBER)('37.0')(12,16)
PsiElement(Jai:;)(';')(16,17)
JaiStatementImpl(STATEMENT)(18,26)
JaiDeclarationImpl(DECLARATION)(18,26)
JaiDeclNamesImpl(DECL_NAMES)(18,19)
PsiElement(Jai:IDENT)('b')(18,19)
PsiElement(Jai::=)(':=')(20,22)
JaiInitializerImpl(INITIALIZER)(23,25)
JaiLiteralExprImpl(LITERAL_EXPR)(23,25)
PsiElement(Jai:NUMBER)('10')(23,25)
PsiElement(Jai:;)(';')(25,26)
JaiStatementImpl(STATEMENT)(27,35)
JaiDeclarationImpl(DECLARATION)(27,35)
JaiDeclNamesImpl(DECL_NAMES)(27,28)
PsiElement(Jai:IDENT)('c')(27,28)
PsiElement(Jai:::)('::')(29,31)
JaiInitializerImpl(INITIALIZER)(32,34)
JaiLiteralExprImpl(LITERAL_EXPR)(32,34)
PsiElement(Jai:NUMBER)('42')(32,34)
PsiElement(Jai:;)(';')(34,35)
JaiStatementImpl(STATEMENT)(36,49)
JaiReturnStatementImpl(RETURN_STATEMENT)(36,49)
PsiElement(Jai:return)('return')(36,42)
JaiAddExprImpl(ADD_EXPR)(43,48)
JaiRefExprImpl(REF_EXPR)(43,44)
PsiElement(Jai:IDENT)('a')(43,44)
PsiElement(Jai:+)('+')(45,46)
JaiRefExprImpl(REF_EXPR)(47,48)
PsiElement(Jai:IDENT)('b')(47,48)
PsiElement(Jai:;)(';')(48,49)