Implement Jai formatter

AGENTS.md

@@ -61,7 +61,7 @@ Last verified state (all green, `./jaigradle check` and `verifyPlugin` too):
 ```text
 dev.hgh.HarnessSmokeTest                       tests=2
 dev.hgh.jai.JaiFileTypeTest                    tests=4
-dev.hgh.jai.completion.JaiCompletionTest      tests=19
+dev.hgh.jai.completion.JaiCompletionTest      tests=20
 dev.hgh.jai.editor.JaiEditorSupportTest       tests=7
 dev.hgh.jai.editor.JaiFoldingBuilderTest      tests=3
 dev.hgh.jai.findusages.JaiFindUsagesTest      tests=2
@@ -73,11 +73,12 @@ dev.hgh.jai.parser.JaiParserGoldenTest         tests=5   <- Tier 2 golden trees
 dev.hgh.jai.parser.JaiParserLongTailTest       tests=1   <- focused parser regressions
 dev.hgh.jai.parser.DebugParseTest              tests=2   <- scratch harness, inert
 dev.hgh.jai.refactoring.JaiRenameTest          tests=3
-dev.hgh.jai.reference.JaiReferenceTest         tests=9
+dev.hgh.jai.reference.JaiReferenceTest         tests=10
 dev.hgh.jai.structure.JaiStructureViewTest     tests=3
+dev.hgh.jai.formatter.JaiFormatterTest          tests=4
 dev.hgh.jai.settings.JaiProjectSettingsTest    tests=3
 dev.hgh.jai.settings.JaiConfiguredRootTest    tests=6
--> total 93, failures+errors 0
+-> total 99, failures+errors 0
 ```
 
 The corpus gates report what they actually did; check both lines are still there:
@@ -129,6 +130,10 @@ Tier 3: parsed 714 files, 714 clean (100.0%), 0 with errors, 0 PsiErrorElements
   sources. Headless tests cover state round-tripping, the Settings panel,
   custom-root `#import`/`#load` navigation and completion, and cross-root
   find-usages.
+- **Phase 6** — PSI-aware formatter with operator/punctuation spacing,
+  block indentation, opaque `#asm` preservation, and a corpus-wide idempotence
+  gate. Headless formatter tests cover registration, representative formatting,
+  opaque/directive token preservation, and all 714 corpus files.
 
 ### Lexer design facts worth knowing before touching it
 
@@ -197,16 +202,14 @@ Tier 3: parsed 714 files, 714 clean (100.0%), 0 with errors, 0 PsiErrorElements
 
 ### In progress — pick up here
 
-1. **Phases 6+** — formatter, inspections, compiler integration, and other
+1. **Phases 7+** — inspections, compiler integration, and other optional work
-   optional work remain. `#asm` bodies are intentionally consumed opaquely, so
+   remain. `#asm` bodies are intentionally consumed opaquely, so nothing inside
-   nothing inside them has PSI yet.
+   them has PSI yet.
 
 ### Next planned increment
 
-- Decide whether to continue with the optional formatter, inspections, and
+- Decide whether to continue with inspections and compiler integration phases
-  compiler integration phases (6–8).
+  (7–8).
-- If formatter work begins, add the corpus-wide idempotence gate described in
-  `docs/BUILD_PLAN.md` before changing formatting behavior.
 
 ### Open questions for the user (unanswered)
 

docs/BUILD_PLAN.md

@@ -171,12 +171,14 @@ Each phase has a machine-checkable gate. Do not advance without a green gate.
 | **4** | Structure view, folding, `#import`/`#load` reference resolution + go-to-definition | Tier 4 fixture tests — **done** |
 | **5** | Completion (keywords, directives, module names, visible declarations, parameters, module aliases, incomplete-expression recovery), rename, find-usages | Tier 4 fixture tests — **done** (84-test suite) |
 | **5a** | Configurable Jai module/import roots and indexed external search scope | Tier 4 settings, resolution, completion, navigation, and find-usages tests — **done** (93-test suite) |
-| **6** | Formatter, code style settings | Formatter round-trip: formatting the corpus is idempotent |
+| **6** | Formatter, code style settings | Formatter round-trip — **done**, formatting all 714 corpus files is idempotent |
 | **7** | Inspections (e.g. `#must` misuse), quick fixes, live templates | Tier 4 + `verifyPlugin` |
 | **8** | Optional: run-configuration to invoke the `jai` compiler, parse its error output | Integration test against `~/.local/jai/bin` |
 
 Phase 6's idempotence check (format twice, assert no change) is another
-corpus-scale invariant that needs no human judgment.
+corpus-scale invariant that needs no human judgment. `JaiFormatterTest` runs this
+check across all 714 files and also covers formatter registration, representative
+spacing/indentation, directive flags, strings, and opaque `#asm` bodies.
 
 ---
 
@@ -185,8 +187,8 @@ corpus-scale invariant that needs no human judgment.
 Phase 5 completion now covers keywords, compiler directives,
 `#import`/`#load` module and file paths, visible same-file or imported
 top-level declarations, procedure parameters, and members of imported module
-aliases. Struct-field completion remains a possible follow-up before or after
+aliases. Struct-field completion remains a possible follow-up, but it is not
-Phase 6, but it is not currently a committed milestone.
+currently a committed milestone.
 
 ## 5. Risks and how the plan handles them
 
@@ -216,6 +218,6 @@ Phase 6, but it is not currently a committed milestone.
    setup, works today), or install a system JDK via Homebrew? I'd default to the
    wrapper.
 
-Phases 0–5a are implemented and verified headlessly. The next decision is
+Phases 0–6 are implemented and verified headlessly. The next decision is
-whether to continue with the optional formatter, inspections, and compiler
+whether to continue with the optional inspections and compiler integration work
-integration work in Phases 6–8.
+in Phases 7–8.

src/main/kotlin/dev/hgh/jai/formatter/JaiFormattingModelBuilder.kt

@@ -0,0 +1,283 @@
+package dev.hgh.jai.formatter
+
+import com.intellij.formatting.Alignment
+import com.intellij.formatting.Block
+import com.intellij.formatting.ChildAttributes
+import com.intellij.formatting.FormattingContext
+import com.intellij.formatting.FormattingModel
+import com.intellij.formatting.FormattingModelBuilder
+import com.intellij.formatting.FormattingModelProvider
+import com.intellij.formatting.Indent
+import com.intellij.formatting.Spacing
+import com.intellij.formatting.SpacingBuilder
+import com.intellij.formatting.Wrap
+import com.intellij.lang.ASTNode
+import com.intellij.psi.TokenType
+import com.intellij.psi.codeStyle.CodeStyleSettings
+import com.intellij.psi.formatter.common.AbstractBlock
+import com.intellij.psi.tree.IElementType
+import com.intellij.psi.tree.TokenSet
+import dev.hgh.jai.JaiLanguage
+import dev.hgh.jai.lexer.JaiTokenTypes
+import dev.hgh.jai.psi.JaiDirectiveExpr
+import dev.hgh.jai.psi.JaiExpressionStatement
+import dev.hgh.jai.psi.JaiPrefixExpr
+import dev.hgh.jai.psi.JaiStatement
+import dev.hgh.jai.psi.JaiTypes
+
+/** Basic PSI-aware formatter for Jai source files. */
+class JaiFormattingModelBuilder : FormattingModelBuilder {
+    override fun createModel(formattingContext: FormattingContext): FormattingModel {
+        val settings = formattingContext.codeStyleSettings
+        val spacingBuilder = createSpacingBuilder(settings)
+        val block = JaiFormattingBlock(formattingContext.node, null, null, spacingBuilder)
+        return FormattingModelProvider.createFormattingModelForPsiFile(
+            formattingContext.containingFile,
+            block,
+            settings,
+        )
+    }
+
+    private companion object {
+        private val BINARY_OPERATORS =
+            TokenSet.create(
+                JaiTokenTypes.PLUS,
+                JaiTokenTypes.MINUS,
+                JaiTokenTypes.STAR,
+                JaiTokenTypes.SLASH,
+                JaiTokenTypes.PERCENT,
+                JaiTokenTypes.PLUS_EQ,
+                JaiTokenTypes.MINUS_EQ,
+                JaiTokenTypes.STAR_EQ,
+                JaiTokenTypes.SLASH_EQ,
+                JaiTokenTypes.PERCENT_EQ,
+                JaiTokenTypes.EQ,
+                JaiTokenTypes.EQ_EQ,
+                JaiTokenTypes.EQ_EQ_EQ,
+                JaiTokenTypes.NOT_EQ,
+                JaiTokenTypes.LT,
+                JaiTokenTypes.GT,
+                JaiTokenTypes.LT_EQ,
+                JaiTokenTypes.GT_EQ,
+                JaiTokenTypes.AND_AND,
+                JaiTokenTypes.OR_OR,
+                JaiTokenTypes.AND_AND_EQ,
+                JaiTokenTypes.OR_OR_EQ,
+                JaiTokenTypes.AND,
+                JaiTokenTypes.OR,
+                JaiTokenTypes.XOR,
+                JaiTokenTypes.AND_EQ,
+                JaiTokenTypes.OR_EQ,
+                JaiTokenTypes.XOR_EQ,
+                JaiTokenTypes.SHL,
+                JaiTokenTypes.SHR,
+                JaiTokenTypes.ROL,
+                JaiTokenTypes.ROR,
+                JaiTokenTypes.SHL_EQ,
+                JaiTokenTypes.SHR_EQ,
+                JaiTokenTypes.ROL_EQ,
+                JaiTokenTypes.ROR_EQ,
+                JaiTokenTypes.RIGHT_ARROW,
+                JaiTokenTypes.QUICK_LAMBDA,
+            )
+
+        private fun createSpacingBuilder(settings: CodeStyleSettings): SpacingBuilder =
+            SpacingBuilder(settings, JaiLanguage)
+                .before(JaiTokenTypes.COMMA)
+                .none()
+                .after(JaiTokenTypes.COMMA)
+                .spaces(1)
+                .before(JaiTokenTypes.DOUBLE_COMMA)
+                .none()
+                .after(JaiTokenTypes.DOUBLE_COMMA)
+                .spaces(1)
+                .before(JaiTokenTypes.COLON)
+                .none()
+                .after(JaiTokenTypes.COLON)
+                .spaces(1)
+                .around(JaiTokenTypes.COLON_COLON)
+                .spaces(1)
+                .around(JaiTokenTypes.COLON_EQ)
+                .spaces(1)
+                .around(BINARY_OPERATORS)
+                .spaces(1)
+                .around(JaiTokenTypes.DOT)
+                .none()
+                .around(JaiTokenTypes.POSTFIX_DEREFERENCE)
+                .none()
+                .before(JaiTokenTypes.LBRACE)
+                .spaces(1)
+                .before(JaiTypes.BLOCK)
+                .spaces(1)
+                .withinPair(JaiTokenTypes.LPAREN, JaiTokenTypes.RPAREN)
+                .none()
+                .withinPair(JaiTokenTypes.LBRACKET, JaiTokenTypes.RBRACKET)
+                .none()
+                .withinPair(JaiTokenTypes.BEGIN_STRUCT_LITERAL, JaiTokenTypes.RBRACE)
+                .none()
+                .withinPair(JaiTokenTypes.BEGIN_ARRAY_LITERAL, JaiTokenTypes.RBRACKET)
+                .none()
+    }
+}
+
+private class JaiFormattingBlock(
+    node: ASTNode,
+    wrap: Wrap?,
+    alignment: Alignment?,
+    private val spacingBuilder: SpacingBuilder,
+) : AbstractBlock(node, wrap, alignment) {
+    private val astNode: ASTNode = node
+
+    override fun buildChildren(): List<Block> {
+        if (isOpaqueAsm(myNode)) return emptyList()
+
+        val blocks = mutableListOf<Block>()
+        var child = myNode.firstChildNode
+        while (child != null) {
+            if (child.elementType != TokenType.WHITE_SPACE && child.textLength > 0) {
+                blocks += JaiFormattingBlock(child, null, null, spacingBuilder)
+            }
+            child = child.treeNext
+        }
+        return blocks
+    }
+
+    override fun getIndent(): Indent? {
+        val parent = myNode.treeParent ?: return Indent.getNoneIndent()
+        return if (parent.elementType == JaiTypes.BLOCK &&
+            myNode.elementType != JaiTokenTypes.LBRACE &&
+            myNode.elementType != JaiTokenTypes.RBRACE
+        ) {
+            Indent.getNormalIndent()
+        } else {
+            Indent.getNoneIndent()
+        }
+    }
+
+    override fun getChildAttributes(newChildIndex: Int): ChildAttributes =
+        if (myNode.elementType == JaiTypes.BLOCK) {
+            ChildAttributes(Indent.getNormalIndent(), null)
+        } else {
+            ChildAttributes(Indent.getNoneIndent(), null)
+        }
+
+    override fun getSpacing(
+        child1: Block?,
+        child2: Block,
+    ): Spacing? {
+        val firstNode = nodeOf(child1)
+        val secondNode = nodeOf(child2)
+        val firstType = firstNode?.elementType
+        val secondType = secondNode?.elementType
+
+        if (myNode.elementType == JaiTypes.BLOCK &&
+            firstType == JaiTokenTypes.LBRACE &&
+            secondType == JaiTokenTypes.RBRACE
+        ) {
+            return NO_SPACE
+        }
+
+        if (firstType == JaiTokenTypes.LINE_COMMENT) return LINE_COMMENT_END
+        if (secondType == JaiTokenTypes.LINE_COMMENT) return COMMENT_START
+
+        if (firstType == JaiTokenTypes.COLON_COLON || secondType == JaiTokenTypes.COLON_COLON ||
+            firstType == JaiTokenTypes.COLON_EQ || secondType == JaiTokenTypes.COLON_EQ
+        ) {
+            return ONE_SPACE
+        }
+
+        if (secondType == JaiTokenTypes.COLON) return NO_SPACE
+        if (firstType == JaiTokenTypes.COLON) return ONE_SPACE
+
+        if (firstType == JaiTokenTypes.COMMA &&
+            (
+                isDirectiveFlagComma(firstNode) ||
+                    (myNode.elementType == JaiTypes.DECLARATION && isUsingFlagComma(firstNode)) ||
+                    (myNode.elementType == JaiTypes.PARAMETER && isUsingFlagComma(firstNode))
+            )
+        ) {
+            return NO_SPACE
+        }
+
+        if ((myNode.psi is JaiPrefixExpr || myNode.elementType == JaiTypes.PREFIX_EXPR) &&
+            isPrefixOperator(firstType)
+        ) {
+            return NO_SPACE
+        }
+
+        if (secondType == JaiTypes.STATEMENT && isBodyStatementParent(myNode.elementType)) {
+            return ONE_SPACE
+        }
+
+        if (firstType == JaiTokenTypes.DOT || secondType == JaiTokenTypes.DOT ||
+            firstType == JaiTokenTypes.POSTFIX_DEREFERENCE || secondType == JaiTokenTypes.POSTFIX_DEREFERENCE
+        ) {
+            return NO_SPACE
+        }
+
+        return spacingBuilder.getSpacing(this, child1, child2)
+    }
+
+    override fun isLeaf(): Boolean = isOpaqueAsm(myNode) || myNode.firstChildNode == null
+
+    private fun isBodyStatementParent(type: IElementType): Boolean =
+        type == JaiTypes.IF_STATEMENT ||
+            type == JaiTypes.WHILE_STATEMENT ||
+            type == JaiTypes.FOR_STATEMENT ||
+            type == JaiTypes.ELSE_CLAUSE ||
+            type == JaiTypes.CASE_CLAUSE ||
+            type == JaiTypes.DEFER_STATEMENT ||
+            type == JaiTypes.PUSH_CONTEXT_STATEMENT
+
+    private fun isUsingFlagComma(comma: ASTNode?): Boolean {
+        if (comma == null || comma.treeNext?.elementType != JaiTokenTypes.IDENT) return false
+        return comma.treePrev?.elementType == JaiTokenTypes.KW_USING
+    }
+
+    private fun isPrefixOperator(type: IElementType?): Boolean =
+        type == JaiTokenTypes.PLUS ||
+            type == JaiTokenTypes.MINUS ||
+            type == JaiTokenTypes.STAR ||
+            type == JaiTokenTypes.NOT ||
+            type == JaiTokenTypes.TILDE ||
+            type == JaiTokenTypes.DOLLAR ||
+            type == JaiTokenTypes.DOUBLE_DOLLAR ||
+            type == JaiTokenTypes.SHL ||
+            type == JaiTokenTypes.SHR
+
+    private fun nodeOf(block: Block?): ASTNode? = (block as? JaiFormattingBlock)?.astNode
+
+    private fun isDirectiveFlagComma(comma: ASTNode?): Boolean {
+        if (comma == null) return false
+        if (comma.treeNext?.elementType != JaiTokenTypes.IDENT) return false
+
+        val previous = comma.treePrev ?: return false
+        if (previous.elementType == TokenType.WHITE_SPACE) return false
+        if (previous.elementType == JaiTokenTypes.DIRECTIVE ||
+            previous.elementType == JaiTokenTypes.KW_CAST ||
+            previous.elementType == JaiTokenTypes.KW_XX ||
+            previous.elementType == JaiTokenTypes.KW_PUSH_CONTEXT
+        ) {
+            return true
+        }
+        if (previous.elementType != JaiTokenTypes.IDENT) return false
+
+        val previousComma = previous.treePrev ?: return false
+        return previousComma.elementType != TokenType.WHITE_SPACE &&
+            previousComma.elementType == JaiTokenTypes.COMMA &&
+            isDirectiveFlagComma(previousComma)
+    }
+
+    private fun isOpaqueAsm(node: ASTNode): Boolean {
+        val psi = node.psi
+        return (psi is JaiDirectiveExpr || psi is JaiExpressionStatement || psi is JaiStatement) &&
+            psi.text.trimStart().startsWith("#asm")
+    }
+
+    private companion object {
+        val NO_SPACE: Spacing = Spacing.createSpacing(0, 0, 0, false, 0)
+        val ONE_SPACE: Spacing = Spacing.createSpacing(1, 1, 0, false, 0)
+        val COMMENT_START: Spacing = Spacing.createSpacing(1, 1, 0, true, 0)
+        val LINE_COMMENT_END: Spacing = Spacing.createSpacing(0, 0, 1, true, 0)
+    }
+}

src/main/resources/META-INF/plugin.xml

@@ -22,6 +22,7 @@
             extensions="jai"/>
 
         <lang.parserDefinition language="Jai" implementationClass="dev.hgh.jai.psi.JaiParserDefinition"/>
+        <lang.formatter language="Jai" implementationClass="dev.hgh.jai.formatter.JaiFormattingModelBuilder"/>
 
         <projectService serviceImplementation="dev.hgh.jai.settings.JaiProjectSettings"/>
         <projectConfigurable

src/test/kotlin/dev/hgh/jai/formatter/JaiFormatterTest.kt

@@ -0,0 +1,126 @@
+package dev.hgh.jai.formatter
+
+import com.intellij.lang.LanguageFormatting
+import com.intellij.openapi.command.WriteCommandAction
+import com.intellij.openapi.util.Computable
+import com.intellij.psi.PsiFileFactory
+import com.intellij.psi.codeStyle.CodeStyleManager
+import com.intellij.testFramework.fixtures.BasePlatformTestCase
+import dev.hgh.jai.JaiLanguage
+import java.io.File
+
+class JaiFormatterTest : BasePlatformTestCase() {
+    fun testFormatterIsRegistered() {
+        val builder = LanguageFormatting.INSTANCE.forLanguage(JaiLanguage)
+
+        assertNotNull("expected a formatter for Jai", builder)
+        assertTrue("expected JaiFormattingModelBuilder, got $builder", builder is JaiFormattingModelBuilder)
+    }
+
+    fun testFormatsSpacingAndIndentation() {
+        myFixture.configureByText(
+            "format.jai",
+            """
+            main::()->int{
+            value:=1+2;
+            negative:=-69;
+            if value>0{
+            return value;
+            }
+            }
+            """.trimIndent(),
+        )
+
+        WriteCommandAction.runWriteCommandAction(project) {
+            CodeStyleManager.getInstance(project).reformatText(myFixture.file, 0, myFixture.file.textLength)
+        }
+        myFixture.checkResult(
+            """
+            main :: () -> int {
+                value := 1 + 2;
+                negative := -69;
+                if value > 0 {
+                    return value;
+                }
+            }
+            """.trimIndent(),
+        )
+    }
+
+    fun testPreservesDirectiveFlagsOpaqueAssemblyAndStringText() {
+        myFixture.configureByText(
+            "opaque.jai",
+            """
+            #import,file "library.jai";
+            main::(){
+            value:=cast,trunc(u32) input;
+            #asm { mov rax, rbx; }
+            text:="  keep  ";
+            }
+            """.trimIndent(),
+        )
+
+        WriteCommandAction.runWriteCommandAction(project) {
+            CodeStyleManager.getInstance(project).reformatText(myFixture.file, 0, myFixture.file.textLength)
+        }
+        myFixture.checkResult(
+            """
+            #import,file "library.jai";
+            main :: () {
+                value := cast,trunc(u32) input;
+                #asm { mov rax, rbx; }
+                text := "  keep  ";
+            }
+            """.trimIndent(),
+        )
+    }
+
+    fun testCorpusFormattingIsIdempotent() {
+        val corpusDir = File(System.getProperty("user.home"), ".local/jai")
+        if (!corpusDir.exists()) {
+            println("Skipping formatter corpus test: $corpusDir does not exist")
+            return
+        }
+
+        val jaiFiles =
+            corpusDir
+                .walkTopDown()
+                .filter { it.isFile && it.extension == "jai" }
+                .sorted()
+                .toList()
+        assertTrue("Corpus files must be found in $corpusDir", jaiFiles.isNotEmpty())
+
+        val nonIdempotent = mutableListOf<String>()
+        jaiFiles.forEach { file ->
+            val normalizedText = file.readText().replace("\r\n", "\n").replace('\r', '\n')
+            val psiFile = PsiFileFactory.getInstance(project).createFileFromText(file.name, JaiLanguage, normalizedText)
+            val once = reformatPsiFile(psiFile)
+            val reparsed = PsiFileFactory.getInstance(project).createFileFromText(file.name, JaiLanguage, once)
+            val hasParseErrors =
+                com.intellij.psi.util.PsiTreeUtil
+                    .findChildrenOfType(reparsed, com.intellij.psi.PsiErrorElement::class.java)
+                    .isNotEmpty()
+            val twice = reformatPsiFile(psiFile)
+            if (hasParseErrors || once != twice) {
+                nonIdempotent += file.relativePath(corpusDir)
+            }
+        }
+
+        println("Formatter corpus: checked ${jaiFiles.size} files, non-idempotent=${nonIdempotent.size}")
+        assertTrue(
+            "Formatter is not idempotent for ${nonIdempotent.take(20)}",
+            nonIdempotent.isEmpty(),
+        )
+    }
+
+    private fun reformatPsiFile(psiFile: com.intellij.psi.PsiFile): String =
+        WriteCommandAction.runWriteCommandAction(
+            project,
+            Computable {
+                CodeStyleManager.getInstance(project).reformatRange(psiFile, 0, psiFile.textLength)
+                psiFile.text
+            },
+        )
+
+    private fun File.relativePath(base: File): String = absolutePath.removePrefix(base.absolutePath + "/")
+}