Phase 3: Grammar-Kit parser, PSI and ParserDefinition

Adds src/main/grammar/Jai.bnf, the generated parser/PSI in src/main/gen, a
JaiParserDefinition, and the Tier 3 corpus parse gate (406/714 files clean).

Two things were not obvious:

- Grammar-Kit mints its own token instances from the .bnf 'tokens' block, which
  are different objects from the ones JaiLexer emits, so every rule silently
  failed to match. Fixed with tokenTypeFactory -> JaiTokenTypes.byName.
- A backslash inside an identifier is a continuation in the compiler's lexer
  (Jai_Lexer/module.jai:444): 'left\_margin' is one identifier. JaiLexer now
  does the same; the Tier 0 round-trip still holds because the token span
  covers the backslash and the skipped spaces.

Constructs that BNF alone cannot express live in JaiParserUtil: directive-name
tests (#ident is one token), '==' before '{' for the switch form, procedure
header vs parenthesised expression, and the declaration lookahead.
This commit is contained in:
hgranthorner
2026-08-04 12:05:42 -04:00
parent 746647d27d
commit 37809b8aa7
169 changed files with 8328 additions and 7 deletions

View File

@@ -0,0 +1,36 @@
package dev.hgh.jai.psi
import com.intellij.lang.ASTNode
import com.intellij.lang.ParserDefinition
import com.intellij.lang.PsiParser
import com.intellij.lexer.Lexer
import com.intellij.openapi.project.Project
import com.intellij.psi.FileViewProvider
import com.intellij.psi.PsiElement
import com.intellij.psi.PsiFile
import com.intellij.psi.tree.IFileElementType
import com.intellij.psi.tree.TokenSet
import dev.hgh.jai.JaiLanguage
import dev.hgh.jai.lexer.JaiLexer
import dev.hgh.jai.lexer.JaiTokenTypes
import dev.hgh.jai.parser.JaiParser
class JaiParserDefinition : ParserDefinition {
override fun createLexer(project: Project?): Lexer = JaiLexer()
override fun createParser(project: Project?): PsiParser = JaiParser()
override fun getFileNodeType(): IFileElementType = FILE
override fun getCommentTokens(): TokenSet = JaiTokenTypes.COMMENTS
override fun getStringLiteralElements(): TokenSet = JaiTokenTypes.STRINGS
override fun createElement(node: ASTNode?): PsiElement = JaiTypes.Factory.createElement(node)
override fun createFile(viewProvider: FileViewProvider): PsiFile = JaiFile(viewProvider)
companion object {
val FILE = IFileElementType(JaiLanguage)
}
}