diff --git a/AGENTS.md b/AGENTS.md index 0604454..820d735 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -61,7 +61,7 @@ Last verified state (all green, `./jaigradle check` and `verifyPlugin` too): ```text dev.hgh.HarnessSmokeTest tests=2 dev.hgh.jai.JaiFileTypeTest tests=4 -dev.hgh.jai.completion.JaiCompletionTest tests=6 +dev.hgh.jai.completion.JaiCompletionTest tests=19 dev.hgh.jai.editor.JaiEditorSupportTest tests=7 dev.hgh.jai.editor.JaiFoldingBuilderTest tests=3 dev.hgh.jai.findusages.JaiFindUsagesTest tests=2 @@ -76,8 +76,8 @@ dev.hgh.jai.refactoring.JaiRenameTest tests=3 dev.hgh.jai.reference.JaiReferenceTest tests=9 dev.hgh.jai.structure.JaiStructureViewTest tests=3 dev.hgh.jai.settings.JaiProjectSettingsTest tests=3 -dev.hgh.jai.settings.JaiConfiguredRootTest tests=4 --> total 78, failures+errors 0 +dev.hgh.jai.settings.JaiConfiguredRootTest tests=6 +-> total 93, failures+errors 0 ``` The corpus gates report what they actually did; check both lines are still there: @@ -120,10 +120,10 @@ Tier 3: parsed 714 files, 714 clean (100.0%), 0 with errors, 0 PsiErrorElements `#import`/`#load` path references with standard go-to-definition resolution. Headless tests cover source-ordered declarations, nested structure members, folding ranges, local `#load`, and Jai module directories. -- **Phase 5** — basic completion for keywords, directives, and module/file - paths; same-file and imported symbol navigation; rename; and find-usages for - declaration names. Symbol-aware identifier/member completion remains future - work. +- **Phase 5** — completion for keywords, directives, module/file paths, visible + same-file or imported declarations, procedure parameters, and module aliases; + same-file and imported symbol navigation; rename; and find-usages for + declaration names. Struct-field completion remains future work. - **Phase 5a** — persistent project settings for Jai module/import roots, shared root-aware resolution and completion, and indexed external Jai library sources. Headless tests cover state round-tripping, the Settings panel, @@ -190,11 +190,10 @@ Tier 3: parsed 714 files, 714 clean (100.0%), 0 with errors, 0 PsiErrorElements ### Known code-insight gaps -- Completion currently offers keywords, compiler directives, and module/file - paths, but not declarations, imported symbols, parameters, struct fields, or - module-qualified members. -- Module-qualified members, procedure parameters, and struct fields also do - not yet have symbol resolution. +- Completion currently offers keywords, compiler directives, module/file paths, + visible same-file or imported declarations, procedure parameters, and members + of imported aliases, but not struct fields. +- Struct fields do not yet have symbol resolution or completion. ### In progress — pick up here diff --git a/CHANGELOG.md b/CHANGELOG.md index 6c285e5..06cb647 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,7 +20,9 @@ - Folding for blocks, nested comments, and multi-line here-strings. - Go-to-definition references for `#import` and `#load`, including project-relative files and the local Jai module layout (`Name.jai` or `Name/module.jai`). -- Basic completion for Jai keywords, compiler directives, and module/file paths. +- Completion for Jai keywords, compiler directives, module/file paths, visible + same-file or imported declarations, procedure parameters, and members of + imported module aliases. - Project-level Jai module/import root settings with ordered custom roots. - Go-to-definition for procedure and type symbols in the same file and through loaded, imported, or configured external Jai files. @@ -33,7 +35,5 @@ not have inner PSI. - `#asm` bodies remain intentionally opaque; assembly instructions do not have inner PSI or structure entries. -- Symbol-aware completion for declarations and imported symbols is not yet - implemented. -- Module-qualified members, procedure parameters, and struct fields do not yet - have symbol resolution or completion. +- Struct fields and other type-qualified members do not yet have symbol + resolution or completion. diff --git a/README.md b/README.md index a92cf3b..09bb581 100644 --- a/README.md +++ b/README.md @@ -17,17 +17,18 @@ never from online material (see `AGENTS.md`). - Comment / uncomment (`//` and nesting `/* */`) - Brace matching, including the Jai-only `.{` struct-literal and `.[` array-literal openers -- Basic completion for keywords, directives, and Jai module/file paths - (symbol-aware identifier completion is not implemented yet) +- Completion for keywords, directives, Jai module/file paths, visible + same-file or imported declarations, procedure parameters, and members of + imported module aliases - Project settings for ordered Jai module/import roots - Go-to-definition for procedures and types in the same file and through `#import`/`#load`, including configured external roots - Indexed external Jai roots so find-usages can cross project/library boundaries - Rename and find-usages for declaration names and their resolved references -Remaining code-insight gaps include symbol-aware completion for declarations -and imported symbols, plus module-qualified members, parameters, and struct -fields. Formatting, inspections, and compiler integration are planned later. +Remaining code-insight gaps include completion and symbol resolution for struct +fields and other type-qualified members. Formatting, inspections, and compiler +integration are planned later. See `docs/BUILD_PLAN.md` for the phase plan. ## Install it locally diff --git a/docs/BUILD_PLAN.md b/docs/BUILD_PLAN.md index 2e0de93..477e03d 100644 --- a/docs/BUILD_PLAN.md +++ b/docs/BUILD_PLAN.md @@ -169,8 +169,8 @@ Each phase has a machine-checkable gate. Do not advance without a green gate. | **2** | `SyntaxHighlighter`, color settings page, commenter, brace matcher | Tier 1 golden dumps; highlighter maps every token type | | **3** | `.bnf` grammar, generated parser + PSI, `ParserDefinition` | Tier 2 golden trees; Tier 3 corpus parse ≥ target — **done, 100.0% (714/714)** | | **4** | Structure view, folding, `#import`/`#load` reference resolution + go-to-definition | Tier 4 fixture tests — **done** | -| **5** | Completion (keywords, directives, module names), rename, find-usages | Tier 4 fixture tests — **done** (71-test suite) | -| **5a** | Configurable Jai module/import roots and indexed external search scope | Tier 4 settings, resolution, completion, navigation, and find-usages tests — **done** (78-test suite) | +| **5** | Completion (keywords, directives, module names, visible declarations, parameters, module aliases, incomplete-expression recovery), rename, find-usages | Tier 4 fixture tests — **done** (84-test suite) | +| **5a** | Configurable Jai module/import roots and indexed external search scope | Tier 4 settings, resolution, completion, navigation, and find-usages tests — **done** (93-test suite) | | **6** | Formatter, code style settings | Formatter round-trip: formatting the corpus is idempotent | | **7** | Inspections (e.g. `#must` misuse), quick fixes, live templates | Tier 4 + `verifyPlugin` | | **8** | Optional: run-configuration to invoke the `jai` compiler, parse its error output | Integration test against `~/.local/jai/bin` | @@ -182,11 +182,11 @@ corpus-scale invariant that needs no human judgment. ### Completion follow-up (not yet scheduled) -Phase 5 completion is intentionally limited to keywords, compiler directives, -and `#import`/`#load` module and file paths. It does not yet offer declarations, -imported symbols, parameters, struct fields, or module-qualified members. -Symbol-aware completion is a possible follow-up before or after Phase 6, but it -is not currently a committed milestone. +Phase 5 completion now covers keywords, compiler directives, +`#import`/`#load` module and file paths, visible same-file or imported +top-level declarations, procedure parameters, and members of imported module +aliases. Struct-field completion remains a possible follow-up before or after +Phase 6, but it is not currently a committed milestone. ## 5. Risks and how the plan handles them diff --git a/src/main/kotlin/dev/hgh/jai/completion/JaiCompletionContributor.kt b/src/main/kotlin/dev/hgh/jai/completion/JaiCompletionContributor.kt index 4456cbe..8d49c9d 100644 --- a/src/main/kotlin/dev/hgh/jai/completion/JaiCompletionContributor.kt +++ b/src/main/kotlin/dev/hgh/jai/completion/JaiCompletionContributor.kt @@ -6,6 +6,7 @@ import com.intellij.codeInsight.completion.CompletionProvider import com.intellij.codeInsight.completion.CompletionResultSet import com.intellij.codeInsight.completion.CompletionType import com.intellij.codeInsight.lookup.LookupElementBuilder +import com.intellij.openapi.diagnostic.Logger import com.intellij.openapi.project.DumbAware import com.intellij.openapi.vfs.VirtualFile import com.intellij.patterns.PlatformPatterns @@ -17,8 +18,11 @@ import dev.hgh.jai.lexer.JaiTokenTypes import dev.hgh.jai.module.JaiModuleResolver import dev.hgh.jai.module.JaiPathMode import dev.hgh.jai.psi.JaiDirectiveExpr +import dev.hgh.jai.reference.JaiSymbolResolver /** Directives and their adjacent comma flags before an import/load string. */ +private const val COMPLETION_DEBUG_PROPERTY = "jai.completion.debug" +private val LOG = Logger.getInstance("dev.hgh.jai.completion") private val DIRECTIVE_WITH_FLAGS = Regex("""#(import|load)((?:\s*,\s*[A-Za-z_][A-Za-z0-9_]*)*)\s*$""") @@ -43,6 +47,12 @@ private class JaiCompletionProvider : CompletionProvider() ) { val completionContext = JaiCompletionContext(parameters) val moduleDirective = completionContext.moduleDirective + debug( + "request file=${parameters.originalFile.virtualFile?.path ?: parameters.originalFile.name} " + + "offset=${parameters.offset} token=${completionContext.debugToken} " + + "prefix='${completionContext.prefix}' module=$moduleDirective " + + "member=${completionContext.memberContext}", + ) when { moduleDirective != null -> { addModuleNames(parameters, moduleDirective, completionContext.prefix, resultSet) @@ -52,12 +62,18 @@ private class JaiCompletionProvider : CompletionProvider() return } + completionContext.memberContext != null -> { + val member = completionContext.memberContext!! + addMemberSymbols(parameters, member, resultSet) + } + completionContext.isDirectivePrefix -> { addDirectives(completionContext.prefix, resultSet) } else -> { addKeywords(completionContext.prefix, resultSet) + addSymbols(parameters, completionContext.prefix, resultSet) } } } @@ -100,6 +116,53 @@ private class JaiCompletionProvider : CompletionProvider() } } + private fun addSymbols( + parameters: CompletionParameters, + prefix: String, + resultSet: CompletionResultSet, + ) { + val candidates = JaiSymbolResolver.completionCandidates(parameters.originalFile, parameters.position) + debug("unqualified candidates=${candidates.size} names=${candidates.take(40).joinToString { it.name }}") + candidates + .filter { it.name.startsWith(prefix) } + .forEach { candidate -> + resultSet.addElement( + LookupElementBuilder + .create(candidate.name) + .withPsiElement(candidate.element) + .withTypeText(candidate.kind), + ) + } + } + + private fun addMemberSymbols( + parameters: CompletionParameters, + member: JaiCompletionContext.MemberContext, + resultSet: CompletionResultSet, + ) { + val candidates = + JaiSymbolResolver.memberCompletionCandidates( + parameters.originalFile, + parameters.position, + member.qualifier, + ) + debug("member qualifier=${member.qualifier} candidates=${candidates.size} names=${candidates.take(40).joinToString { it.name }}") + candidates + .filter { it.name.startsWith(member.prefix) } + .forEach { candidate -> + resultSet.addElement( + LookupElementBuilder + .create(candidate.name) + .withPsiElement(candidate.element) + .withTypeText(candidate.kind), + ) + } + } + + private fun debug(message: String) { + if (java.lang.Boolean.getBoolean(COMPLETION_DEBUG_PROPERTY)) LOG.info(message) + } + private fun addModuleNames( parameters: CompletionParameters, directive: String, @@ -149,6 +212,23 @@ private class JaiCompletionContext( val isDirectivePrefix: Boolean get() = token?.type === JaiTokenTypes.DIRECTIVE || token?.type === JaiTokenTypes.HASH + val debugToken: String + get() = token?.let { "${it.type}@${it.start}..${it.end}" } ?: "none" + + data class MemberContext( + val qualifier: String, + val prefix: String, + ) + + val memberContext: MemberContext? + get() = + MEMBER_PREFIX.find(text.substring(0, offset))?.let { match -> + MemberContext( + qualifier = match.groupValues[1], + prefix = match.groupValues.getOrNull(2).orEmpty(), + ) + } + val prefix: String get() { val current = token ?: return "" @@ -210,6 +290,11 @@ private class JaiCompletionContext( val end: Int, ) + private companion object { + val MEMBER_PREFIX = + Regex("""(?:^|[^A-Za-z0-9_`])(`?[A-Za-z_][A-Za-z0-9_]*`?)\s*\.\s*(`?[A-Za-z_][A-Za-z0-9_]*`?)?$""") + } + private fun tokenAtCaret( text: String, offset: Int, diff --git a/src/main/kotlin/dev/hgh/jai/reference/JaiReferenceContributor.kt b/src/main/kotlin/dev/hgh/jai/reference/JaiReferenceContributor.kt index 6f32bbd..a141164 100644 --- a/src/main/kotlin/dev/hgh/jai/reference/JaiReferenceContributor.kt +++ b/src/main/kotlin/dev/hgh/jai/reference/JaiReferenceContributor.kt @@ -3,22 +3,16 @@ package dev.hgh.jai.reference import com.intellij.openapi.util.TextRange import com.intellij.patterns.PlatformPatterns import com.intellij.psi.PsiElement -import com.intellij.psi.PsiFile import com.intellij.psi.PsiManager import com.intellij.psi.PsiReference import com.intellij.psi.PsiReferenceBase import com.intellij.psi.PsiReferenceContributor import com.intellij.psi.PsiReferenceProvider import com.intellij.psi.PsiReferenceRegistrar -import com.intellij.psi.PsiReferenceService -import com.intellij.psi.util.PsiTreeUtil import com.intellij.util.ProcessingContext import dev.hgh.jai.lexer.JaiTokenTypes import dev.hgh.jai.module.JaiModuleResolver import dev.hgh.jai.module.JaiModuleTarget -import dev.hgh.jai.psi.JaiBlock -import dev.hgh.jai.psi.JaiDeclName -import dev.hgh.jai.psi.JaiDeclaration import dev.hgh.jai.psi.JaiDirectiveExpr import dev.hgh.jai.psi.JaiLiteralExpr import dev.hgh.jai.psi.JaiPsiElementFactory @@ -115,109 +109,6 @@ private class JaiSymbolReference( override fun getVariants(): Array = emptyArray() } -private object JaiSymbolResolver { - private data class Candidate( - val name: JaiDeclName, - val scope: JaiBlock?, - ) - - fun resolve(reference: JaiRefExpr): JaiDeclName? { - val file = reference.containingFile ?: return null - resolveInFile(file, reference)?.let { return it } - return resolveImported(file, reference.text, linkedSetOf()) - } - - private fun resolveInFile( - file: PsiFile, - reference: JaiRefExpr, - ): JaiDeclName? { - val candidates = - PsiTreeUtil - .findChildrenOfType(file, JaiDeclaration::class.java) - .flatMap { declaration -> - declaration.declNames.declNameList.map { name -> - Candidate(name, PsiTreeUtil.getParentOfType(name, JaiBlock::class.java)) - } - }.filter { candidate -> - symbolName(candidate.name) == reference.text && isVisible(candidate, reference) - }.sortedWith( - compareByDescending { scopeDepth(it.scope) } - .thenByDescending { - it.name.textRange.startOffset <= reference.textRange.startOffset - }.thenBy { - kotlin.math.abs(it.name.textRange.startOffset - reference.textRange.startOffset) - }, - ) - return candidates.firstOrNull()?.name - } - - private fun resolveImported( - file: PsiFile, - referenceName: String, - visited: MutableSet, - ): JaiDeclName? { - val fileKey = file.virtualFile?.path ?: file.name - if (!visited.add(fileKey)) return null - - for (importedFile in importedFiles(file)) { - topLevelDeclaration(importedFile, referenceName)?.let { return it } - resolveImported(importedFile, referenceName, visited)?.let { return it } - } - return null - } - - private fun topLevelDeclaration( - file: PsiFile, - referenceName: String, - ): JaiDeclName? = - PsiTreeUtil - .findChildrenOfType(file, JaiDeclaration::class.java) - .asSequence() - .flatMap { declaration -> declaration.declNames.declNameList.asSequence() } - .firstOrNull { - PsiTreeUtil.getParentOfType(it, JaiBlock::class.java) == null && - symbolName(it) == referenceName - } - - private fun importedFiles(file: PsiFile): List = - PsiTreeUtil - .findChildrenOfType(file, JaiLiteralExpr::class.java) - .mapNotNull { literal -> - val directive = literal.parent as? JaiDirectiveExpr ?: return@mapNotNull null - val name = directive.directiveName() ?: return@mapNotNull null - if (name != "#import" && name != "#load") return@mapNotNull null - PsiReferenceService - .getService() - .getContributedReferences(literal) - .firstOrNull() - ?.resolve() as? PsiFile - }.distinctBy { it.virtualFile?.path ?: it.name } - - private fun isVisible( - candidate: Candidate, - reference: JaiRefExpr, - ): Boolean = candidate.scope == null || PsiTreeUtil.isAncestor(candidate.scope, reference, false) - - private fun scopeDepth(scope: JaiBlock?): Int { - var current: PsiElement? = scope - var depth = 0 - while (current != null) { - if (current is JaiBlock) depth++ - current = current.parent - } - return depth - } - - private fun symbolName(name: JaiDeclName): String { - val text = name.text - return if (text.length >= 2 && text.first() == '`' && text.last() == '`') { - text.substring(1, text.length - 1) - } else { - text - } - } -} - private class JaiModuleReference( private val sourceElement: PsiElement, rangeInElement: TextRange, diff --git a/src/main/kotlin/dev/hgh/jai/reference/JaiSymbolResolver.kt b/src/main/kotlin/dev/hgh/jai/reference/JaiSymbolResolver.kt new file mode 100644 index 0000000..18c51c7 --- /dev/null +++ b/src/main/kotlin/dev/hgh/jai/reference/JaiSymbolResolver.kt @@ -0,0 +1,410 @@ +package dev.hgh.jai.reference + +import com.intellij.openapi.diagnostic.Logger +import com.intellij.psi.PsiElement +import com.intellij.psi.PsiFile +import com.intellij.psi.PsiManager +import com.intellij.psi.PsiReferenceService +import com.intellij.psi.util.PsiTreeUtil +import dev.hgh.jai.lexer.JaiTokenTypes +import dev.hgh.jai.module.JaiModuleResolver +import dev.hgh.jai.module.JaiModuleTarget +import dev.hgh.jai.psi.JaiBlock +import dev.hgh.jai.psi.JaiDeclName +import dev.hgh.jai.psi.JaiDeclaration +import dev.hgh.jai.psi.JaiDirectiveExpr +import dev.hgh.jai.psi.JaiForName +import dev.hgh.jai.psi.JaiForStatement +import dev.hgh.jai.psi.JaiInitializer +import dev.hgh.jai.psi.JaiLiteralExpr +import dev.hgh.jai.psi.JaiParamName +import dev.hgh.jai.psi.JaiProcLiteralExpr +import dev.hgh.jai.psi.JaiRefExpr +import dev.hgh.jai.settings.JaiProjectSettings + +private const val COMPLETION_DEBUG_PROPERTY = "jai.completion.debug" +private val LOG = Logger.getInstance("dev.hgh.jai.reference.JaiSymbolResolver") +private val RECOVERABLE_IMPORT = + Regex("""(?m)(#(?:import|load))((?:\s*,\s*[A-Za-z_][A-Za-z0-9_]*)*)\s*"((?:\\.|[^"\\])*)"""") +private val RECOVERABLE_PROCEDURE = + Regex("""(?s)(?:^|[;}\n])\s*[A-Za-z_][A-Za-z0-9_]*\s*::\s*\([^{}]*\)\s*(?:->[^\{;}]*)?\{""") +private val RECOVERABLE_LOCAL_DECLARATION = + Regex("""(?m)(?:^|[;{}])\s*([A-Za-z_][A-Za-z0-9_]*)\s*(?::=|:(?!=))""") + +/** A declaration-like PSI element that can be inserted by identifier completion. */ +internal data class JaiSymbolCandidate( + val element: PsiElement, + val name: String, + val kind: String = "symbol", +) + +/** Shared symbol lookup for references and symbol-aware completion. */ +internal object JaiSymbolResolver { + private data class ScopedSymbol( + val element: PsiElement, + val name: String, + val scope: PsiElement?, + val fallbackScope: PsiElement?, + val visibleFrom: Int, + ) + + private data class ImportBinding( + val file: PsiFile, + val scope: PsiElement?, + val visibleFrom: Int, + val boundName: String?, + val importsIntoScope: Boolean, + val excludedNames: Set, + ) + + fun resolve(reference: JaiRefExpr): PsiElement? { + val file = reference.containingFile ?: return null + return completionCandidates(file, reference) + .firstOrNull { it.name == reference.text } + ?.element + } + + /** Returns visible local declarations followed by visible unqualified imports. */ + fun completionCandidates( + file: PsiFile, + position: PsiElement, + ): List { + val candidates = linkedMapOf() + + val localSymbols = localSymbols(file) + val recoveredLocals = recoverLocalSymbols(file, position) + localSymbols + .filter { isVisible(it, position) && !isInside(it.element, position) } + .sortedWith( + compareByDescending { scopeDepth(it.scope ?: it.fallbackScope) } + .thenBy { it.visibleFrom } + .thenBy { it.element.textOffset }, + ).forEach { symbol -> + candidates.putIfAbsent( + symbol.name, + JaiSymbolCandidate(symbol.element, symbol.name), + ) + } + + recoveredLocals.forEach { symbol -> + candidates.putIfAbsent(symbol.name, symbol) + } + importedCandidates(file, position).forEach { symbol -> + candidates.putIfAbsent(symbol.name, symbol) + } + val result = candidates.values.toList() + debug( + "file=${file.virtualFile?.path ?: file.name} roots=${JaiProjectSettings.getInstance(file.project).rootPaths()} " + + "locals=${localSymbols.joinToString { + "${it.name}@${it.element.textOffset}/scope=${it.scope?.textRange}/fallback=${it.fallbackScope?.textRange}" + }} recovered=${recoveredLocals.joinToString { it.name }} " + + "imports=${importBindings(file).joinToString { it.file.virtualFile?.path ?: it.file.name }} " + + "candidates=${result.take(40).joinToString { it.name }}", + ) + return result + } + + /** Returns members of a module bound to [qualifier] in the visible lexical scope. */ + fun memberCompletionCandidates( + file: PsiFile, + position: PsiElement, + qualifier: String, + ): List { + val binding = + importBindings(file) + .filter { it.boundName == normalizeName(qualifier) && isVisible(it, position) } + .sortedWith( + compareByDescending { scopeDepth(it.scope) } + .thenByDescending { it.visibleFrom }, + ).firstOrNull() + ?: return emptyList() + + return topLevelDeclarations(binding.file) + .asSequence() + .filter { symbolName(it) !in binding.excludedNames } + .mapNotNull { name -> + symbolName(name)?.let { JaiSymbolCandidate(name, it, "member") } + }.distinctBy { it.name } + .toList() + } + + private fun localSymbols(file: PsiFile): List { + val declarations = + PsiTreeUtil + .findChildrenOfType(file, JaiDeclaration::class.java) + .flatMap { declaration -> + declaration.declNames.declNameList.mapNotNull { name -> + symbolName(name)?.let { + ScopedSymbol( + element = name, + name = it, + scope = PsiTreeUtil.getParentOfType(name, JaiBlock::class.java), + fallbackScope = PsiTreeUtil.getParentOfType(name, JaiProcLiteralExpr::class.java), + visibleFrom = name.textOffset, + ) + } + } + } + + val parameters = + PsiTreeUtil + .findChildrenOfType(file, JaiParamName::class.java) + .mapNotNull { name -> + val symbol = symbolName(name) ?: return@mapNotNull null + val procedure = PsiTreeUtil.getParentOfType(name, JaiProcLiteralExpr::class.java) + ScopedSymbol( + element = name, + name = symbol, + // Keep the procedure itself as the parameter scope. The body block is a + // nested scope, so a local declaration correctly shadows a parameter. + scope = procedure, + fallbackScope = null, + visibleFrom = procedure?.textOffset ?: name.textOffset, + ) + } + + val forNames = + PsiTreeUtil + .findChildrenOfType(file, JaiForName::class.java) + .mapNotNull { name -> + val symbol = symbolName(name) ?: return@mapNotNull null + ScopedSymbol( + element = name, + name = symbol, + scope = PsiTreeUtil.getParentOfType(name, JaiForStatement::class.java), + fallbackScope = PsiTreeUtil.getParentOfType(name, JaiProcLiteralExpr::class.java), + visibleFrom = name.textOffset, + ) + } + + return declarations + parameters + forNames + } + + private fun importedCandidates( + file: PsiFile, + position: PsiElement, + ): List = + importBindings(file) + .asSequence() + .filter { (it.boundName == null || it.importsIntoScope) && isVisible(it, position) } + .flatMap { binding -> + topLevelDeclarations(binding.file) + .asSequence() + .filter { symbolName(it) !in binding.excludedNames } + .mapNotNull { name -> + symbolName(name)?.let { JaiSymbolCandidate(name, it, "import") } + } + }.distinctBy { it.name } + .toList() + + /** Recovers simple locals when an unfinished statement prevents the procedure PSI from closing. */ + private fun recoverLocalSymbols( + file: PsiFile, + position: PsiElement, + ): List { + val caretOffset = position.textOffset.coerceIn(0, file.textLength) + val beforeCaret = file.text.substring(0, caretOffset) + val procedure = RECOVERABLE_PROCEDURE.findAll(beforeCaret).lastOrNull() ?: return emptyList() + val bodyStart = procedure.range.last + 1 + if (bodyStart >= beforeCaret.length) return emptyList() + + return RECOVERABLE_LOCAL_DECLARATION + .findAll(beforeCaret.substring(bodyStart)) + .map { match -> JaiSymbolCandidate(position, match.groupValues[1], "local") } + .distinctBy { it.name } + .toList() + } + + private fun importBindings(file: PsiFile): List { + val parsed = + PsiTreeUtil + .findChildrenOfType(file, JaiLiteralExpr::class.java) + .mapNotNull { literal -> + val directive = literal.parent as? JaiDirectiveExpr ?: return@mapNotNull null + val name = directive.directiveName() ?: return@mapNotNull null + if (name != "#import" && name != "#load") return@mapNotNull null + + val declaration = boundDeclaration(literal) + val boundName = + declaration + ?.declNames + ?.declNameList + ?.firstOrNull() + ?.let(::symbolName) + val declarationPrefix = declaration?.let { prefixBeforeNames(it) }.orEmpty() + val importsIntoScope = declaration == null || declarationPrefix.trimStart().startsWith("using") + val excludedNames = excludedNames(declarationPrefix) + val target = + PsiReferenceService + .getService() + .getContributedReferences(literal) + .firstOrNull() + ?.resolve() as? PsiFile + ?: return@mapNotNull null + + ImportBinding( + file = target, + scope = PsiTreeUtil.getParentOfType(literal, JaiBlock::class.java), + visibleFrom = literal.textOffset, + boundName = boundName, + importsIntoScope = importsIntoScope, + excludedNames = excludedNames, + ) + } + + // An unfinished statement immediately before the caret can make Grammar-Kit retain the + // file's top-level declarations but omit later directive PSI. Recover bare imports from + // the source text so completion still works while the user is typing. + val recovered = if (parsed.isEmpty()) recoverImportBindings(file) else emptyList() + return (parsed + recovered).distinctBy { binding -> + listOf( + binding.file.virtualFile?.path ?: binding.file.name, + binding.scope?.textRange?.startOffset, + binding.boundName, + binding.visibleFrom, + ) + } + } + + private fun recoverImportBindings(file: PsiFile): List { + val sourceFile = file.virtualFile ?: return emptyList() + return RECOVERABLE_IMPORT + .findAll(file.text) + .mapNotNull { match -> + val directive = match.groupValues[1] + val flags = + Regex("""[A-Za-z_][A-Za-z0-9_]*""") + .findAll(match.groupValues[2]) + .map { it.value } + .toSet() + if ("string" in flags) return@mapNotNull null + val path = unescapeImportPath(match.groupValues[3]) + val target = + JaiModuleResolver.resolve( + file.project, + sourceFile, + JaiModuleTarget(directive, path, flags), + ) ?: return@mapNotNull null + val targetPsi = PsiManager.getInstance(file.project).findFile(target) ?: return@mapNotNull null + ImportBinding( + file = targetPsi, + scope = null, + visibleFrom = match.range.first, + boundName = null, + importsIntoScope = true, + excludedNames = emptySet(), + ) + }.toList() + } + + private fun unescapeImportPath(text: String): String = + buildString(text.length) { + var index = 0 + while (index < text.length) { + if (text[index] == '\\' && index + 1 < text.length) { + append(text[index + 1]) + index += 2 + } else { + append(text[index++]) + } + } + } + + /** The declaration that binds an import, not an enclosing procedure declaration. */ + private fun boundDeclaration(literal: JaiLiteralExpr): JaiDeclaration? { + val initializer = PsiTreeUtil.getParentOfType(literal, JaiInitializer::class.java) ?: return null + return PsiTreeUtil.getParentOfType(initializer, JaiDeclaration::class.java) + } + + private fun prefixBeforeNames(declaration: JaiDeclaration): String { + val start = declaration.textRange.startOffset + val end = declaration.declNames.textRange.startOffset + return declaration.containingFile.text.substring(start, end) + } + + private fun excludedNames(prefix: String): Set { + val body = Regex("""except\s*\(([^)]*)\)""").find(prefix)?.groupValues?.get(1) ?: return emptySet() + return body + .split(',') + .map { normalizeName(it.trim()) } + .filter(String::isNotEmpty) + .toSet() + } + + private fun topLevelDeclarations(file: PsiFile): List = + PsiTreeUtil + .findChildrenOfType(file, JaiDeclaration::class.java) + .asSequence() + .flatMap { declaration -> declaration.declNames.declNameList.asSequence() } + .filter { PsiTreeUtil.getParentOfType(it, JaiBlock::class.java) == null } + .sortedBy { it.textOffset } + .toList() + + private fun isVisible( + symbol: ScopedSymbol, + position: PsiElement, + ): Boolean { + val inScope = + when { + symbol.scope == null && symbol.fallbackScope == null -> true + symbol.scope?.textRange?.contains(position.textOffset) == true -> true + symbol.fallbackScope?.textRange?.contains(position.textOffset) == true -> true + else -> false + } + if (!inScope) return false + // Block-local declarations and loop names are not visible before their declaration. Keep + // top-level declarations available for forward references, which Jai permits. + if (symbol.scope != null && symbol.visibleFrom > position.textOffset) return false + return true + } + + private fun isVisible( + binding: ImportBinding, + position: PsiElement, + ): Boolean = + // File-scope imports are declarations for the whole file even when the directive is + // written below its first use (a common Jai layout). Block-local imports remain ordered. + (binding.scope == null || binding.scope.textRange.contains(position.textOffset)) && + (binding.scope == null || binding.visibleFrom <= position.textOffset) + + private fun isInside( + element: PsiElement, + position: PsiElement, + ): Boolean = element == position || element.textRange.contains(position.textOffset) + + private fun scopeDepth(scope: PsiElement?): Int { + var current = scope + var depth = 0 + while (current != null) { + if (current is JaiBlock || current is JaiForStatement || current is JaiProcLiteralExpr) depth++ + current = current.parent + } + return depth + } + + private fun debug(message: String) { + if (java.lang.Boolean.getBoolean(COMPLETION_DEBUG_PROPERTY)) LOG.info(message) + } + + private fun symbolName(element: PsiElement): String? { + val text = element.text.trim() + if (text.isEmpty()) return null + val name = + when (element) { + is JaiParamName -> text.removePrefix("$$").removePrefix("$") + else -> text + } + return normalizeName(name).takeIf(String::isNotEmpty) + } + + private fun normalizeName(name: String): String { + val trimmed = name.trim() + return if (trimmed.length >= 2 && trimmed.first() == '`' && trimmed.last() == '`') { + trimmed.substring(1, trimmed.length - 1) + } else { + trimmed + } + } +} + +private fun JaiDirectiveExpr.directiveName(): String? = node.findChildByType(JaiTokenTypes.DIRECTIVE)?.text diff --git a/src/main/kotlin/dev/hgh/jai/settings/JaiProjectSettings.kt b/src/main/kotlin/dev/hgh/jai/settings/JaiProjectSettings.kt index 7f74a94..805d2ec 100644 --- a/src/main/kotlin/dev/hgh/jai/settings/JaiProjectSettings.kt +++ b/src/main/kotlin/dev/hgh/jai/settings/JaiProjectSettings.kt @@ -1,6 +1,7 @@ package dev.hgh.jai.settings import com.intellij.openapi.application.ApplicationManager +import com.intellij.openapi.components.PathMacroManager import com.intellij.openapi.components.PersistentStateComponent import com.intellij.openapi.components.State import com.intellij.openapi.components.Storage @@ -62,7 +63,7 @@ class JaiProjectSettings( val trimmed = rawPath.trim() if (trimmed.isEmpty()) return null - val expanded = + var expanded = when { trimmed == "~" -> { System.getProperty("user.home") @@ -76,6 +77,20 @@ class JaiProjectSettings( trimmed } } + // Workspace files may persist project-relative roots using IntelliJ path macros. The + // settings UI normally supplies absolute paths, but loading a project saved elsewhere + // must expand these before the resolver tries to find the VFS root. + expanded = PathMacroManager.getInstance(project).expandPath(expanded) + expanded = + expanded + .replace("\$USER_HOME\$", System.getProperty("user.home")) + .replace("\$USER_HOME", System.getProperty("user.home")) + .let { value -> + val basePath = project.basePath ?: return@let value + value + .replace("\$PROJECT_DIR\$", basePath) + .replace("\$PROJECT_DIR", basePath) + } val file = File(expanded) val absolute = if (file.isAbsolute) { diff --git a/src/test/kotlin/dev/hgh/jai/completion/JaiCompletionTest.kt b/src/test/kotlin/dev/hgh/jai/completion/JaiCompletionTest.kt index 9b52107..df088cf 100644 --- a/src/test/kotlin/dev/hgh/jai/completion/JaiCompletionTest.kt +++ b/src/test/kotlin/dev/hgh/jai/completion/JaiCompletionTest.kt @@ -54,6 +54,254 @@ class JaiCompletionTest : BasePlatformTestCase() { myFixture.checkResult("#load \"other.jai\";") } + fun testCompletesVisibleDeclarationsInSameFile() { + myFixture.configureByText( + "symbols.jai", + """ + helper :: () {} + main :: () { hel(); } + """.trimIndent(), + ) + + myFixture.completeBasic() + myFixture.checkResult( + """ + helper :: () {} + main :: () { helper(); } + """.trimIndent(), + ) + } + + fun testCompletesLocalDeclarationsInProcedure() { + myFixture.configureByText( + "locals.jai", + """ + main :: () { + local_value := 1; + print(local); + } + """.trimIndent(), + ) + + myFixture.completeBasic() + assertEquals( + "completion should insert the local declaration", + """ + main :: () { + local_value := 1; + print(local_value); + } + """.trimIndent(), + myFixture.file.text, + ) + } + + fun testCompletesSymbolsFromImportedModule() { + myFixture.configureByText( + "module-consumer.jai", + """ + #import "Basic"; + main :: () { allo(); } + """.trimIndent(), + ) + + myFixture.complete(CompletionType.BASIC, 1) + val strings = myFixture.lookupElementStrings + assertNotNull("expected symbols from the imported Basic module", strings) + assertTrue("expected alloc in $strings", strings!!.contains("alloc")) + } + + fun testCompletesSymbolsFromLoadedFile() { + myFixture.addFileToProject("library.jai", "Helper :: () {}") + myFixture.configureByText( + "consumer.jai", + """ + #load "library.jai"; + main :: () { Hel(); } + """.trimIndent(), + ) + myFixture.completeBasic() + assertEquals( + "completion should insert the loaded symbol", + """ + #load "library.jai"; + main :: () { Helper(); } + """.trimIndent(), + myFixture.file.text, + ) + } + + fun testCompletesLocalDeclarationsWithEmptyPrefix() { + myFixture.configureByText( + "empty-prefix.jai", + """ + main :: () { + local_value := 1; + print(); + } + """.trimIndent(), + ) + + myFixture.complete(CompletionType.BASIC, 1) + assertTrue( + "local declarations should be offered with an empty prefix", + myFixture.lookupElementStrings.orEmpty().contains("local_value"), + ) + } + + fun testCompletesProcedureParameters() { + myFixture.configureByText( + "parameters.jai", + """ + main :: (local_value: int) { + print(local); + } + """.trimIndent(), + ) + + myFixture.completeBasic() + myFixture.checkResult( + """ + main :: (local_value: int) { + print(local_value); + } + """.trimIndent(), + ) + } + + fun testCompletesLocalDeclarationBeforeUnterminatedExpression() { + myFixture.configureByText( + "unterminated-local.jai", + """ + main :: () { + foobazz := 1; + fo + } + """.trimIndent(), + ) + + myFixture.complete(CompletionType.BASIC, 1) + assertTrue( + "local declaration should survive an unterminated completion expression: ${myFixture.lookupElementStrings}", + myFixture.lookupElementStrings.orEmpty().contains("foobazz"), + ) + } + + fun testCompletesLoadedSymbolsBeforeUnterminatedExpression() { + myFixture.addFileToProject("library.jai", "loaded_value :: 1;") + myFixture.configureByText( + "unterminated-import.jai", + """ + main :: () { + local_value := 1; + lo + } + #load "library.jai"; + """.trimIndent(), + ) + + myFixture.complete(CompletionType.BASIC, 1) + val strings = myFixture.lookupElementStrings.orEmpty() + assertTrue("local symbol should be offered: $strings", "local_value" in strings) + assertTrue("loaded symbol should be offered: $strings", "loaded_value" in strings) + } + + fun testDoesNotLeakDeclarationsFromTransitiveImports() { + myFixture.addFileToProject("dependency.jai", "transitive_value :: 1;") + myFixture.addFileToProject( + "library.jai", + """ + #import,file "dependency.jai"; + direct_value :: 1; + """.trimIndent(), + ) + myFixture.configureByText( + "transitive-consumer.jai", + """ + #load "library.jai"; + main :: () { tra; } + """.trimIndent(), + ) + + myFixture.complete(CompletionType.BASIC, 1) + assertFalse( + "completion should not recursively expose every imported file: ${myFixture.lookupElementStrings}", + myFixture.lookupElementStrings.orEmpty().contains("transitive_value"), + ) + } + + fun testCompletesMembersOfAnImportedAlias() { + myFixture.addFileToProject("library.jai", "Helper :: () {}\nother_value :: 1;") + myFixture.configureByText( + "consumer.jai", + """ + Library :: #import,file "library.jai"; + main :: () { Library.Hel(); } + """.trimIndent(), + ) + + myFixture.completeBasic() + myFixture.checkResult( + """ + Library :: #import,file "library.jai"; + main :: () { Library.Helper(); } + """.trimIndent(), + ) + } + + fun testCompletesImportedAliasMembersWithEmptyPrefix() { + myFixture.addFileToProject("library.jai", "Helper :: () {}") + myFixture.configureByText( + "consumer.jai", + """ + Library :: #import,file "library.jai"; + main :: () { Library.(); } + """.trimIndent(), + ) + + myFixture.complete(CompletionType.BASIC, 1) + assertTrue( + "module members should be offered after a dot with an empty prefix", + myFixture.lookupElementStrings.orEmpty().contains("Helper"), + ) + } + + fun testBoundImportDoesNotLeakMembersIntoUnqualifiedCompletion() { + myFixture.addFileToProject("library.jai", "Helper :: () {}") + myFixture.configureByText( + "consumer.jai", + """ + Library :: #import,file "library.jai"; + main :: () { Hel(); } + """.trimIndent(), + ) + + myFixture.complete(CompletionType.BASIC, 1) + assertFalse( + "bound module members should require the module qualifier", + myFixture.lookupElementStrings.orEmpty().contains("Helper"), + ) + } + + fun testCompletesFileScopeImportsDeclaredAfterUse() { + myFixture.addFileToProject("library.jai", "Helper :: () {}") + myFixture.configureByText( + "consumer.jai", + """ + main :: () { Hel(); } + #import,file "library.jai"; + """.trimIndent(), + ) + + myFixture.completeBasic() + myFixture.checkResult( + """ + main :: () { Helper(); } + #import,file "library.jai"; + """.trimIndent(), + ) + } + private fun completeStrings(): List? { myFixture.complete(CompletionType.BASIC, 1) return myFixture.lookupElementStrings diff --git a/src/test/kotlin/dev/hgh/jai/settings/JaiConfiguredRootTest.kt b/src/test/kotlin/dev/hgh/jai/settings/JaiConfiguredRootTest.kt index 236537e..0acf120 100644 --- a/src/test/kotlin/dev/hgh/jai/settings/JaiConfiguredRootTest.kt +++ b/src/test/kotlin/dev/hgh/jai/settings/JaiConfiguredRootTest.kt @@ -1,5 +1,6 @@ package dev.hgh.jai.settings +import com.intellij.codeInsight.completion.CompletionType import com.intellij.openapi.vfs.LocalFileSystem import com.intellij.psi.PsiFile import com.intellij.psi.PsiManager @@ -86,6 +87,49 @@ class JaiConfiguredRootTest : BasePlatformTestCase() { myFixture.checkResult("#load \"external.jai\";") } + fun testConfiguredRootSuppliesImportedSymbolCompletion() { + configureRoot("Custom/module.jai" to "Helper :: () {}") + myFixture.configureByText( + "consumer.jai", + """ + #import "Custom"; + main :: () { Hel(); } + """.trimIndent(), + ) + + myFixture.completeBasic() + myFixture.checkResult( + """ + #import "Custom"; + main :: () { Helper(); } + """.trimIndent(), + ) + } + + fun testProbeStyleWorkspaceRootsSupplyLocalAndImportedCompletions() { + val settings = JaiProjectSettings.getInstance(project) + val jaiModules = Path.of(System.getProperty("user.home"), ".local", "jai", "modules") + val probeModules = Path.of(System.getProperty("user.home"), "programming", "probe", "modules") + settings.setRootPaths(listOf(jaiModules.toString(), probeModules.toString())) + myFixture.configureByText( + "main.jai", + """ + Foo :: struct { bar: int; } + main :: () { + foobazz := Foo.{}; + fo; + } + #import "Foo"; + #import "Basic"; + """.trimIndent(), + ) + + myFixture.complete(CompletionType.BASIC, 1) + val strings = myFixture.lookupElementStrings.orEmpty() + assertTrue("local variable should be offered: $strings", "foobazz" in strings) + assertTrue("Foo module symbol should be offered: $strings", "foo" in strings) + } + fun testReferencesSearchIncludesProjectUsagesForConfiguredExternalDeclaration() { val module = configureRoot("Custom/module.jai" to "Helper :: () {}") val source =