improved completion

This commit is contained in:
hgranthorner
2026-08-05 16:32:40 -04:00
parent ede8522716
commit 98e25c5468
10 changed files with 832 additions and 139 deletions

View File

@@ -61,7 +61,7 @@ Last verified state (all green, `./jaigradle check` and `verifyPlugin` too):
```text ```text
dev.hgh.HarnessSmokeTest tests=2 dev.hgh.HarnessSmokeTest tests=2
dev.hgh.jai.JaiFileTypeTest tests=4 dev.hgh.jai.JaiFileTypeTest tests=4
dev.hgh.jai.completion.JaiCompletionTest tests=6 dev.hgh.jai.completion.JaiCompletionTest tests=19
dev.hgh.jai.editor.JaiEditorSupportTest tests=7 dev.hgh.jai.editor.JaiEditorSupportTest tests=7
dev.hgh.jai.editor.JaiFoldingBuilderTest tests=3 dev.hgh.jai.editor.JaiFoldingBuilderTest tests=3
dev.hgh.jai.findusages.JaiFindUsagesTest tests=2 dev.hgh.jai.findusages.JaiFindUsagesTest tests=2
@@ -76,8 +76,8 @@ dev.hgh.jai.refactoring.JaiRenameTest tests=3
dev.hgh.jai.reference.JaiReferenceTest tests=9 dev.hgh.jai.reference.JaiReferenceTest tests=9
dev.hgh.jai.structure.JaiStructureViewTest tests=3 dev.hgh.jai.structure.JaiStructureViewTest tests=3
dev.hgh.jai.settings.JaiProjectSettingsTest tests=3 dev.hgh.jai.settings.JaiProjectSettingsTest tests=3
dev.hgh.jai.settings.JaiConfiguredRootTest tests=4 dev.hgh.jai.settings.JaiConfiguredRootTest tests=6
-> total 78, failures+errors 0 -> total 93, failures+errors 0
``` ```
The corpus gates report what they actually did; check both lines are still there: The corpus gates report what they actually did; check both lines are still there:
@@ -120,10 +120,10 @@ Tier 3: parsed 714 files, 714 clean (100.0%), 0 with errors, 0 PsiErrorElements
`#import`/`#load` path references with standard go-to-definition resolution. `#import`/`#load` path references with standard go-to-definition resolution.
Headless tests cover source-ordered declarations, nested structure members, Headless tests cover source-ordered declarations, nested structure members,
folding ranges, local `#load`, and Jai module directories. folding ranges, local `#load`, and Jai module directories.
- **Phase 5** — basic completion for keywords, directives, and module/file - **Phase 5** — completion for keywords, directives, module/file paths, visible
paths; same-file and imported symbol navigation; rename; and find-usages for same-file or imported declarations, procedure parameters, and module aliases;
declaration names. Symbol-aware identifier/member completion remains future same-file and imported symbol navigation; rename; and find-usages for
work. declaration names. Struct-field completion remains future work.
- **Phase 5a** — persistent project settings for Jai module/import roots, - **Phase 5a** — persistent project settings for Jai module/import roots,
shared root-aware resolution and completion, and indexed external Jai library shared root-aware resolution and completion, and indexed external Jai library
sources. Headless tests cover state round-tripping, the Settings panel, sources. Headless tests cover state round-tripping, the Settings panel,
@@ -190,11 +190,10 @@ Tier 3: parsed 714 files, 714 clean (100.0%), 0 with errors, 0 PsiErrorElements
### Known code-insight gaps ### Known code-insight gaps
- Completion currently offers keywords, compiler directives, and module/file - Completion currently offers keywords, compiler directives, module/file paths,
paths, but not declarations, imported symbols, parameters, struct fields, or visible same-file or imported declarations, procedure parameters, and members
module-qualified members. of imported aliases, but not struct fields.
- Module-qualified members, procedure parameters, and struct fields also do - Struct fields do not yet have symbol resolution or completion.
not yet have symbol resolution.
### In progress — pick up here ### In progress — pick up here

View File

@@ -20,7 +20,9 @@
- Folding for blocks, nested comments, and multi-line here-strings. - Folding for blocks, nested comments, and multi-line here-strings.
- Go-to-definition references for `#import` and `#load`, including project-relative - Go-to-definition references for `#import` and `#load`, including project-relative
files and the local Jai module layout (`Name.jai` or `Name/module.jai`). files and the local Jai module layout (`Name.jai` or `Name/module.jai`).
- Basic completion for Jai keywords, compiler directives, and module/file paths. - Completion for Jai keywords, compiler directives, module/file paths, visible
same-file or imported declarations, procedure parameters, and members of
imported module aliases.
- Project-level Jai module/import root settings with ordered custom roots. - Project-level Jai module/import root settings with ordered custom roots.
- Go-to-definition for procedure and type symbols in the same file and through - Go-to-definition for procedure and type symbols in the same file and through
loaded, imported, or configured external Jai files. loaded, imported, or configured external Jai files.
@@ -33,7 +35,5 @@
not have inner PSI. not have inner PSI.
- `#asm` bodies remain intentionally opaque; assembly instructions do not have - `#asm` bodies remain intentionally opaque; assembly instructions do not have
inner PSI or structure entries. inner PSI or structure entries.
- Symbol-aware completion for declarations and imported symbols is not yet - Struct fields and other type-qualified members do not yet have symbol
implemented. resolution or completion.
- Module-qualified members, procedure parameters, and struct fields do not yet
have symbol resolution or completion.

View File

@@ -17,17 +17,18 @@ never from online material (see `AGENTS.md`).
- Comment / uncomment (`//` and nesting `/* */`) - Comment / uncomment (`//` and nesting `/* */`)
- Brace matching, including the Jai-only `.{` struct-literal and `.[` - Brace matching, including the Jai-only `.{` struct-literal and `.[`
array-literal openers array-literal openers
- Basic completion for keywords, directives, and Jai module/file paths - Completion for keywords, directives, Jai module/file paths, visible
(symbol-aware identifier completion is not implemented yet) same-file or imported declarations, procedure parameters, and members of
imported module aliases
- Project settings for ordered Jai module/import roots - Project settings for ordered Jai module/import roots
- Go-to-definition for procedures and types in the same file and through - Go-to-definition for procedures and types in the same file and through
`#import`/`#load`, including configured external roots `#import`/`#load`, including configured external roots
- Indexed external Jai roots so find-usages can cross project/library boundaries - Indexed external Jai roots so find-usages can cross project/library boundaries
- Rename and find-usages for declaration names and their resolved references - Rename and find-usages for declaration names and their resolved references
Remaining code-insight gaps include symbol-aware completion for declarations Remaining code-insight gaps include completion and symbol resolution for struct
and imported symbols, plus module-qualified members, parameters, and struct fields and other type-qualified members. Formatting, inspections, and compiler
fields. Formatting, inspections, and compiler integration are planned later. integration are planned later.
See `docs/BUILD_PLAN.md` for the phase plan. See `docs/BUILD_PLAN.md` for the phase plan.
## Install it locally ## Install it locally

View File

@@ -169,8 +169,8 @@ Each phase has a machine-checkable gate. Do not advance without a green gate.
| **2** | `SyntaxHighlighter`, color settings page, commenter, brace matcher | Tier 1 golden dumps; highlighter maps every token type | | **2** | `SyntaxHighlighter`, color settings page, commenter, brace matcher | Tier 1 golden dumps; highlighter maps every token type |
| **3** | `.bnf` grammar, generated parser + PSI, `ParserDefinition` | Tier 2 golden trees; Tier 3 corpus parse ≥ target — **done, 100.0% (714/714)** | | **3** | `.bnf` grammar, generated parser + PSI, `ParserDefinition` | Tier 2 golden trees; Tier 3 corpus parse ≥ target — **done, 100.0% (714/714)** |
| **4** | Structure view, folding, `#import`/`#load` reference resolution + go-to-definition | Tier 4 fixture tests — **done** | | **4** | Structure view, folding, `#import`/`#load` reference resolution + go-to-definition | Tier 4 fixture tests — **done** |
| **5** | Completion (keywords, directives, module names), rename, find-usages | Tier 4 fixture tests — **done** (71-test suite) | | **5** | Completion (keywords, directives, module names, visible declarations, parameters, module aliases, incomplete-expression recovery), rename, find-usages | Tier 4 fixture tests — **done** (84-test suite) |
| **5a** | Configurable Jai module/import roots and indexed external search scope | Tier 4 settings, resolution, completion, navigation, and find-usages tests — **done** (78-test suite) | | **5a** | Configurable Jai module/import roots and indexed external search scope | Tier 4 settings, resolution, completion, navigation, and find-usages tests — **done** (93-test suite) |
| **6** | Formatter, code style settings | Formatter round-trip: formatting the corpus is idempotent | | **6** | Formatter, code style settings | Formatter round-trip: formatting the corpus is idempotent |
| **7** | Inspections (e.g. `#must` misuse), quick fixes, live templates | Tier 4 + `verifyPlugin` | | **7** | Inspections (e.g. `#must` misuse), quick fixes, live templates | Tier 4 + `verifyPlugin` |
| **8** | Optional: run-configuration to invoke the `jai` compiler, parse its error output | Integration test against `~/.local/jai/bin` | | **8** | Optional: run-configuration to invoke the `jai` compiler, parse its error output | Integration test against `~/.local/jai/bin` |
@@ -182,11 +182,11 @@ corpus-scale invariant that needs no human judgment.
### Completion follow-up (not yet scheduled) ### Completion follow-up (not yet scheduled)
Phase 5 completion is intentionally limited to keywords, compiler directives, Phase 5 completion now covers keywords, compiler directives,
and `#import`/`#load` module and file paths. It does not yet offer declarations, `#import`/`#load` module and file paths, visible same-file or imported
imported symbols, parameters, struct fields, or module-qualified members. top-level declarations, procedure parameters, and members of imported module
Symbol-aware completion is a possible follow-up before or after Phase 6, but it aliases. Struct-field completion remains a possible follow-up before or after
is not currently a committed milestone. Phase 6, but it is not currently a committed milestone.
## 5. Risks and how the plan handles them ## 5. Risks and how the plan handles them

View File

@@ -6,6 +6,7 @@ import com.intellij.codeInsight.completion.CompletionProvider
import com.intellij.codeInsight.completion.CompletionResultSet import com.intellij.codeInsight.completion.CompletionResultSet
import com.intellij.codeInsight.completion.CompletionType import com.intellij.codeInsight.completion.CompletionType
import com.intellij.codeInsight.lookup.LookupElementBuilder import com.intellij.codeInsight.lookup.LookupElementBuilder
import com.intellij.openapi.diagnostic.Logger
import com.intellij.openapi.project.DumbAware import com.intellij.openapi.project.DumbAware
import com.intellij.openapi.vfs.VirtualFile import com.intellij.openapi.vfs.VirtualFile
import com.intellij.patterns.PlatformPatterns import com.intellij.patterns.PlatformPatterns
@@ -17,8 +18,11 @@ import dev.hgh.jai.lexer.JaiTokenTypes
import dev.hgh.jai.module.JaiModuleResolver import dev.hgh.jai.module.JaiModuleResolver
import dev.hgh.jai.module.JaiPathMode import dev.hgh.jai.module.JaiPathMode
import dev.hgh.jai.psi.JaiDirectiveExpr import dev.hgh.jai.psi.JaiDirectiveExpr
import dev.hgh.jai.reference.JaiSymbolResolver
/** Directives and their adjacent comma flags before an import/load string. */ /** Directives and their adjacent comma flags before an import/load string. */
private const val COMPLETION_DEBUG_PROPERTY = "jai.completion.debug"
private val LOG = Logger.getInstance("dev.hgh.jai.completion")
private val DIRECTIVE_WITH_FLAGS = private val DIRECTIVE_WITH_FLAGS =
Regex("""#(import|load)((?:\s*,\s*[A-Za-z_][A-Za-z0-9_]*)*)\s*$""") Regex("""#(import|load)((?:\s*,\s*[A-Za-z_][A-Za-z0-9_]*)*)\s*$""")
@@ -43,6 +47,12 @@ private class JaiCompletionProvider : CompletionProvider<CompletionParameters>()
) { ) {
val completionContext = JaiCompletionContext(parameters) val completionContext = JaiCompletionContext(parameters)
val moduleDirective = completionContext.moduleDirective val moduleDirective = completionContext.moduleDirective
debug(
"request file=${parameters.originalFile.virtualFile?.path ?: parameters.originalFile.name} " +
"offset=${parameters.offset} token=${completionContext.debugToken} " +
"prefix='${completionContext.prefix}' module=$moduleDirective " +
"member=${completionContext.memberContext}",
)
when { when {
moduleDirective != null -> { moduleDirective != null -> {
addModuleNames(parameters, moduleDirective, completionContext.prefix, resultSet) addModuleNames(parameters, moduleDirective, completionContext.prefix, resultSet)
@@ -52,12 +62,18 @@ private class JaiCompletionProvider : CompletionProvider<CompletionParameters>()
return return
} }
completionContext.memberContext != null -> {
val member = completionContext.memberContext!!
addMemberSymbols(parameters, member, resultSet)
}
completionContext.isDirectivePrefix -> { completionContext.isDirectivePrefix -> {
addDirectives(completionContext.prefix, resultSet) addDirectives(completionContext.prefix, resultSet)
} }
else -> { else -> {
addKeywords(completionContext.prefix, resultSet) addKeywords(completionContext.prefix, resultSet)
addSymbols(parameters, completionContext.prefix, resultSet)
} }
} }
} }
@@ -100,6 +116,53 @@ private class JaiCompletionProvider : CompletionProvider<CompletionParameters>()
} }
} }
private fun addSymbols(
parameters: CompletionParameters,
prefix: String,
resultSet: CompletionResultSet,
) {
val candidates = JaiSymbolResolver.completionCandidates(parameters.originalFile, parameters.position)
debug("unqualified candidates=${candidates.size} names=${candidates.take(40).joinToString { it.name }}")
candidates
.filter { it.name.startsWith(prefix) }
.forEach { candidate ->
resultSet.addElement(
LookupElementBuilder
.create(candidate.name)
.withPsiElement(candidate.element)
.withTypeText(candidate.kind),
)
}
}
private fun addMemberSymbols(
parameters: CompletionParameters,
member: JaiCompletionContext.MemberContext,
resultSet: CompletionResultSet,
) {
val candidates =
JaiSymbolResolver.memberCompletionCandidates(
parameters.originalFile,
parameters.position,
member.qualifier,
)
debug("member qualifier=${member.qualifier} candidates=${candidates.size} names=${candidates.take(40).joinToString { it.name }}")
candidates
.filter { it.name.startsWith(member.prefix) }
.forEach { candidate ->
resultSet.addElement(
LookupElementBuilder
.create(candidate.name)
.withPsiElement(candidate.element)
.withTypeText(candidate.kind),
)
}
}
private fun debug(message: String) {
if (java.lang.Boolean.getBoolean(COMPLETION_DEBUG_PROPERTY)) LOG.info(message)
}
private fun addModuleNames( private fun addModuleNames(
parameters: CompletionParameters, parameters: CompletionParameters,
directive: String, directive: String,
@@ -149,6 +212,23 @@ private class JaiCompletionContext(
val isDirectivePrefix: Boolean val isDirectivePrefix: Boolean
get() = token?.type === JaiTokenTypes.DIRECTIVE || token?.type === JaiTokenTypes.HASH get() = token?.type === JaiTokenTypes.DIRECTIVE || token?.type === JaiTokenTypes.HASH
val debugToken: String
get() = token?.let { "${it.type}@${it.start}..${it.end}" } ?: "none"
data class MemberContext(
val qualifier: String,
val prefix: String,
)
val memberContext: MemberContext?
get() =
MEMBER_PREFIX.find(text.substring(0, offset))?.let { match ->
MemberContext(
qualifier = match.groupValues[1],
prefix = match.groupValues.getOrNull(2).orEmpty(),
)
}
val prefix: String val prefix: String
get() { get() {
val current = token ?: return "" val current = token ?: return ""
@@ -210,6 +290,11 @@ private class JaiCompletionContext(
val end: Int, val end: Int,
) )
private companion object {
val MEMBER_PREFIX =
Regex("""(?:^|[^A-Za-z0-9_`])(`?[A-Za-z_][A-Za-z0-9_]*`?)\s*\.\s*(`?[A-Za-z_][A-Za-z0-9_]*`?)?$""")
}
private fun tokenAtCaret( private fun tokenAtCaret(
text: String, text: String,
offset: Int, offset: Int,

View File

@@ -3,22 +3,16 @@ package dev.hgh.jai.reference
import com.intellij.openapi.util.TextRange import com.intellij.openapi.util.TextRange
import com.intellij.patterns.PlatformPatterns import com.intellij.patterns.PlatformPatterns
import com.intellij.psi.PsiElement import com.intellij.psi.PsiElement
import com.intellij.psi.PsiFile
import com.intellij.psi.PsiManager import com.intellij.psi.PsiManager
import com.intellij.psi.PsiReference import com.intellij.psi.PsiReference
import com.intellij.psi.PsiReferenceBase import com.intellij.psi.PsiReferenceBase
import com.intellij.psi.PsiReferenceContributor import com.intellij.psi.PsiReferenceContributor
import com.intellij.psi.PsiReferenceProvider import com.intellij.psi.PsiReferenceProvider
import com.intellij.psi.PsiReferenceRegistrar import com.intellij.psi.PsiReferenceRegistrar
import com.intellij.psi.PsiReferenceService
import com.intellij.psi.util.PsiTreeUtil
import com.intellij.util.ProcessingContext import com.intellij.util.ProcessingContext
import dev.hgh.jai.lexer.JaiTokenTypes import dev.hgh.jai.lexer.JaiTokenTypes
import dev.hgh.jai.module.JaiModuleResolver import dev.hgh.jai.module.JaiModuleResolver
import dev.hgh.jai.module.JaiModuleTarget import dev.hgh.jai.module.JaiModuleTarget
import dev.hgh.jai.psi.JaiBlock
import dev.hgh.jai.psi.JaiDeclName
import dev.hgh.jai.psi.JaiDeclaration
import dev.hgh.jai.psi.JaiDirectiveExpr import dev.hgh.jai.psi.JaiDirectiveExpr
import dev.hgh.jai.psi.JaiLiteralExpr import dev.hgh.jai.psi.JaiLiteralExpr
import dev.hgh.jai.psi.JaiPsiElementFactory import dev.hgh.jai.psi.JaiPsiElementFactory
@@ -115,109 +109,6 @@ private class JaiSymbolReference(
override fun getVariants(): Array<Any> = emptyArray() override fun getVariants(): Array<Any> = emptyArray()
} }
private object JaiSymbolResolver {
private data class Candidate(
val name: JaiDeclName,
val scope: JaiBlock?,
)
fun resolve(reference: JaiRefExpr): JaiDeclName? {
val file = reference.containingFile ?: return null
resolveInFile(file, reference)?.let { return it }
return resolveImported(file, reference.text, linkedSetOf())
}
private fun resolveInFile(
file: PsiFile,
reference: JaiRefExpr,
): JaiDeclName? {
val candidates =
PsiTreeUtil
.findChildrenOfType(file, JaiDeclaration::class.java)
.flatMap { declaration ->
declaration.declNames.declNameList.map { name ->
Candidate(name, PsiTreeUtil.getParentOfType(name, JaiBlock::class.java))
}
}.filter { candidate ->
symbolName(candidate.name) == reference.text && isVisible(candidate, reference)
}.sortedWith(
compareByDescending<Candidate> { scopeDepth(it.scope) }
.thenByDescending {
it.name.textRange.startOffset <= reference.textRange.startOffset
}.thenBy {
kotlin.math.abs(it.name.textRange.startOffset - reference.textRange.startOffset)
},
)
return candidates.firstOrNull()?.name
}
private fun resolveImported(
file: PsiFile,
referenceName: String,
visited: MutableSet<String>,
): JaiDeclName? {
val fileKey = file.virtualFile?.path ?: file.name
if (!visited.add(fileKey)) return null
for (importedFile in importedFiles(file)) {
topLevelDeclaration(importedFile, referenceName)?.let { return it }
resolveImported(importedFile, referenceName, visited)?.let { return it }
}
return null
}
private fun topLevelDeclaration(
file: PsiFile,
referenceName: String,
): JaiDeclName? =
PsiTreeUtil
.findChildrenOfType(file, JaiDeclaration::class.java)
.asSequence()
.flatMap { declaration -> declaration.declNames.declNameList.asSequence() }
.firstOrNull {
PsiTreeUtil.getParentOfType(it, JaiBlock::class.java) == null &&
symbolName(it) == referenceName
}
private fun importedFiles(file: PsiFile): List<PsiFile> =
PsiTreeUtil
.findChildrenOfType(file, JaiLiteralExpr::class.java)
.mapNotNull { literal ->
val directive = literal.parent as? JaiDirectiveExpr ?: return@mapNotNull null
val name = directive.directiveName() ?: return@mapNotNull null
if (name != "#import" && name != "#load") return@mapNotNull null
PsiReferenceService
.getService()
.getContributedReferences(literal)
.firstOrNull()
?.resolve() as? PsiFile
}.distinctBy { it.virtualFile?.path ?: it.name }
private fun isVisible(
candidate: Candidate,
reference: JaiRefExpr,
): Boolean = candidate.scope == null || PsiTreeUtil.isAncestor(candidate.scope, reference, false)
private fun scopeDepth(scope: JaiBlock?): Int {
var current: PsiElement? = scope
var depth = 0
while (current != null) {
if (current is JaiBlock) depth++
current = current.parent
}
return depth
}
private fun symbolName(name: JaiDeclName): String {
val text = name.text
return if (text.length >= 2 && text.first() == '`' && text.last() == '`') {
text.substring(1, text.length - 1)
} else {
text
}
}
}
private class JaiModuleReference( private class JaiModuleReference(
private val sourceElement: PsiElement, private val sourceElement: PsiElement,
rangeInElement: TextRange, rangeInElement: TextRange,

View File

@@ -0,0 +1,410 @@
package dev.hgh.jai.reference
import com.intellij.openapi.diagnostic.Logger
import com.intellij.psi.PsiElement
import com.intellij.psi.PsiFile
import com.intellij.psi.PsiManager
import com.intellij.psi.PsiReferenceService
import com.intellij.psi.util.PsiTreeUtil
import dev.hgh.jai.lexer.JaiTokenTypes
import dev.hgh.jai.module.JaiModuleResolver
import dev.hgh.jai.module.JaiModuleTarget
import dev.hgh.jai.psi.JaiBlock
import dev.hgh.jai.psi.JaiDeclName
import dev.hgh.jai.psi.JaiDeclaration
import dev.hgh.jai.psi.JaiDirectiveExpr
import dev.hgh.jai.psi.JaiForName
import dev.hgh.jai.psi.JaiForStatement
import dev.hgh.jai.psi.JaiInitializer
import dev.hgh.jai.psi.JaiLiteralExpr
import dev.hgh.jai.psi.JaiParamName
import dev.hgh.jai.psi.JaiProcLiteralExpr
import dev.hgh.jai.psi.JaiRefExpr
import dev.hgh.jai.settings.JaiProjectSettings
private const val COMPLETION_DEBUG_PROPERTY = "jai.completion.debug"
private val LOG = Logger.getInstance("dev.hgh.jai.reference.JaiSymbolResolver")
private val RECOVERABLE_IMPORT =
Regex("""(?m)(#(?:import|load))((?:\s*,\s*[A-Za-z_][A-Za-z0-9_]*)*)\s*"((?:\\.|[^"\\])*)"""")
private val RECOVERABLE_PROCEDURE =
Regex("""(?s)(?:^|[;}\n])\s*[A-Za-z_][A-Za-z0-9_]*\s*::\s*\([^{}]*\)\s*(?:->[^\{;}]*)?\{""")
private val RECOVERABLE_LOCAL_DECLARATION =
Regex("""(?m)(?:^|[;{}])\s*([A-Za-z_][A-Za-z0-9_]*)\s*(?::=|:(?!=))""")
/** A declaration-like PSI element that can be inserted by identifier completion. */
internal data class JaiSymbolCandidate(
val element: PsiElement,
val name: String,
val kind: String = "symbol",
)
/** Shared symbol lookup for references and symbol-aware completion. */
internal object JaiSymbolResolver {
private data class ScopedSymbol(
val element: PsiElement,
val name: String,
val scope: PsiElement?,
val fallbackScope: PsiElement?,
val visibleFrom: Int,
)
private data class ImportBinding(
val file: PsiFile,
val scope: PsiElement?,
val visibleFrom: Int,
val boundName: String?,
val importsIntoScope: Boolean,
val excludedNames: Set<String>,
)
fun resolve(reference: JaiRefExpr): PsiElement? {
val file = reference.containingFile ?: return null
return completionCandidates(file, reference)
.firstOrNull { it.name == reference.text }
?.element
}
/** Returns visible local declarations followed by visible unqualified imports. */
fun completionCandidates(
file: PsiFile,
position: PsiElement,
): List<JaiSymbolCandidate> {
val candidates = linkedMapOf<String, JaiSymbolCandidate>()
val localSymbols = localSymbols(file)
val recoveredLocals = recoverLocalSymbols(file, position)
localSymbols
.filter { isVisible(it, position) && !isInside(it.element, position) }
.sortedWith(
compareByDescending<ScopedSymbol> { scopeDepth(it.scope ?: it.fallbackScope) }
.thenBy { it.visibleFrom }
.thenBy { it.element.textOffset },
).forEach { symbol ->
candidates.putIfAbsent(
symbol.name,
JaiSymbolCandidate(symbol.element, symbol.name),
)
}
recoveredLocals.forEach { symbol ->
candidates.putIfAbsent(symbol.name, symbol)
}
importedCandidates(file, position).forEach { symbol ->
candidates.putIfAbsent(symbol.name, symbol)
}
val result = candidates.values.toList()
debug(
"file=${file.virtualFile?.path ?: file.name} roots=${JaiProjectSettings.getInstance(file.project).rootPaths()} " +
"locals=${localSymbols.joinToString {
"${it.name}@${it.element.textOffset}/scope=${it.scope?.textRange}/fallback=${it.fallbackScope?.textRange}"
}} recovered=${recoveredLocals.joinToString { it.name }} " +
"imports=${importBindings(file).joinToString { it.file.virtualFile?.path ?: it.file.name }} " +
"candidates=${result.take(40).joinToString { it.name }}",
)
return result
}
/** Returns members of a module bound to [qualifier] in the visible lexical scope. */
fun memberCompletionCandidates(
file: PsiFile,
position: PsiElement,
qualifier: String,
): List<JaiSymbolCandidate> {
val binding =
importBindings(file)
.filter { it.boundName == normalizeName(qualifier) && isVisible(it, position) }
.sortedWith(
compareByDescending<ImportBinding> { scopeDepth(it.scope) }
.thenByDescending { it.visibleFrom },
).firstOrNull()
?: return emptyList()
return topLevelDeclarations(binding.file)
.asSequence()
.filter { symbolName(it) !in binding.excludedNames }
.mapNotNull { name ->
symbolName(name)?.let { JaiSymbolCandidate(name, it, "member") }
}.distinctBy { it.name }
.toList()
}
private fun localSymbols(file: PsiFile): List<ScopedSymbol> {
val declarations =
PsiTreeUtil
.findChildrenOfType(file, JaiDeclaration::class.java)
.flatMap { declaration ->
declaration.declNames.declNameList.mapNotNull { name ->
symbolName(name)?.let {
ScopedSymbol(
element = name,
name = it,
scope = PsiTreeUtil.getParentOfType(name, JaiBlock::class.java),
fallbackScope = PsiTreeUtil.getParentOfType(name, JaiProcLiteralExpr::class.java),
visibleFrom = name.textOffset,
)
}
}
}
val parameters =
PsiTreeUtil
.findChildrenOfType(file, JaiParamName::class.java)
.mapNotNull { name ->
val symbol = symbolName(name) ?: return@mapNotNull null
val procedure = PsiTreeUtil.getParentOfType(name, JaiProcLiteralExpr::class.java)
ScopedSymbol(
element = name,
name = symbol,
// Keep the procedure itself as the parameter scope. The body block is a
// nested scope, so a local declaration correctly shadows a parameter.
scope = procedure,
fallbackScope = null,
visibleFrom = procedure?.textOffset ?: name.textOffset,
)
}
val forNames =
PsiTreeUtil
.findChildrenOfType(file, JaiForName::class.java)
.mapNotNull { name ->
val symbol = symbolName(name) ?: return@mapNotNull null
ScopedSymbol(
element = name,
name = symbol,
scope = PsiTreeUtil.getParentOfType(name, JaiForStatement::class.java),
fallbackScope = PsiTreeUtil.getParentOfType(name, JaiProcLiteralExpr::class.java),
visibleFrom = name.textOffset,
)
}
return declarations + parameters + forNames
}
private fun importedCandidates(
file: PsiFile,
position: PsiElement,
): List<JaiSymbolCandidate> =
importBindings(file)
.asSequence()
.filter { (it.boundName == null || it.importsIntoScope) && isVisible(it, position) }
.flatMap { binding ->
topLevelDeclarations(binding.file)
.asSequence()
.filter { symbolName(it) !in binding.excludedNames }
.mapNotNull { name ->
symbolName(name)?.let { JaiSymbolCandidate(name, it, "import") }
}
}.distinctBy { it.name }
.toList()
/** Recovers simple locals when an unfinished statement prevents the procedure PSI from closing. */
private fun recoverLocalSymbols(
file: PsiFile,
position: PsiElement,
): List<JaiSymbolCandidate> {
val caretOffset = position.textOffset.coerceIn(0, file.textLength)
val beforeCaret = file.text.substring(0, caretOffset)
val procedure = RECOVERABLE_PROCEDURE.findAll(beforeCaret).lastOrNull() ?: return emptyList()
val bodyStart = procedure.range.last + 1
if (bodyStart >= beforeCaret.length) return emptyList()
return RECOVERABLE_LOCAL_DECLARATION
.findAll(beforeCaret.substring(bodyStart))
.map { match -> JaiSymbolCandidate(position, match.groupValues[1], "local") }
.distinctBy { it.name }
.toList()
}
private fun importBindings(file: PsiFile): List<ImportBinding> {
val parsed =
PsiTreeUtil
.findChildrenOfType(file, JaiLiteralExpr::class.java)
.mapNotNull { literal ->
val directive = literal.parent as? JaiDirectiveExpr ?: return@mapNotNull null
val name = directive.directiveName() ?: return@mapNotNull null
if (name != "#import" && name != "#load") return@mapNotNull null
val declaration = boundDeclaration(literal)
val boundName =
declaration
?.declNames
?.declNameList
?.firstOrNull()
?.let(::symbolName)
val declarationPrefix = declaration?.let { prefixBeforeNames(it) }.orEmpty()
val importsIntoScope = declaration == null || declarationPrefix.trimStart().startsWith("using")
val excludedNames = excludedNames(declarationPrefix)
val target =
PsiReferenceService
.getService()
.getContributedReferences(literal)
.firstOrNull()
?.resolve() as? PsiFile
?: return@mapNotNull null
ImportBinding(
file = target,
scope = PsiTreeUtil.getParentOfType(literal, JaiBlock::class.java),
visibleFrom = literal.textOffset,
boundName = boundName,
importsIntoScope = importsIntoScope,
excludedNames = excludedNames,
)
}
// An unfinished statement immediately before the caret can make Grammar-Kit retain the
// file's top-level declarations but omit later directive PSI. Recover bare imports from
// the source text so completion still works while the user is typing.
val recovered = if (parsed.isEmpty()) recoverImportBindings(file) else emptyList()
return (parsed + recovered).distinctBy { binding ->
listOf(
binding.file.virtualFile?.path ?: binding.file.name,
binding.scope?.textRange?.startOffset,
binding.boundName,
binding.visibleFrom,
)
}
}
private fun recoverImportBindings(file: PsiFile): List<ImportBinding> {
val sourceFile = file.virtualFile ?: return emptyList()
return RECOVERABLE_IMPORT
.findAll(file.text)
.mapNotNull { match ->
val directive = match.groupValues[1]
val flags =
Regex("""[A-Za-z_][A-Za-z0-9_]*""")
.findAll(match.groupValues[2])
.map { it.value }
.toSet()
if ("string" in flags) return@mapNotNull null
val path = unescapeImportPath(match.groupValues[3])
val target =
JaiModuleResolver.resolve(
file.project,
sourceFile,
JaiModuleTarget(directive, path, flags),
) ?: return@mapNotNull null
val targetPsi = PsiManager.getInstance(file.project).findFile(target) ?: return@mapNotNull null
ImportBinding(
file = targetPsi,
scope = null,
visibleFrom = match.range.first,
boundName = null,
importsIntoScope = true,
excludedNames = emptySet(),
)
}.toList()
}
private fun unescapeImportPath(text: String): String =
buildString(text.length) {
var index = 0
while (index < text.length) {
if (text[index] == '\\' && index + 1 < text.length) {
append(text[index + 1])
index += 2
} else {
append(text[index++])
}
}
}
/** The declaration that binds an import, not an enclosing procedure declaration. */
private fun boundDeclaration(literal: JaiLiteralExpr): JaiDeclaration? {
val initializer = PsiTreeUtil.getParentOfType(literal, JaiInitializer::class.java) ?: return null
return PsiTreeUtil.getParentOfType(initializer, JaiDeclaration::class.java)
}
private fun prefixBeforeNames(declaration: JaiDeclaration): String {
val start = declaration.textRange.startOffset
val end = declaration.declNames.textRange.startOffset
return declaration.containingFile.text.substring(start, end)
}
private fun excludedNames(prefix: String): Set<String> {
val body = Regex("""except\s*\(([^)]*)\)""").find(prefix)?.groupValues?.get(1) ?: return emptySet()
return body
.split(',')
.map { normalizeName(it.trim()) }
.filter(String::isNotEmpty)
.toSet()
}
private fun topLevelDeclarations(file: PsiFile): List<JaiDeclName> =
PsiTreeUtil
.findChildrenOfType(file, JaiDeclaration::class.java)
.asSequence()
.flatMap { declaration -> declaration.declNames.declNameList.asSequence() }
.filter { PsiTreeUtil.getParentOfType(it, JaiBlock::class.java) == null }
.sortedBy { it.textOffset }
.toList()
private fun isVisible(
symbol: ScopedSymbol,
position: PsiElement,
): Boolean {
val inScope =
when {
symbol.scope == null && symbol.fallbackScope == null -> true
symbol.scope?.textRange?.contains(position.textOffset) == true -> true
symbol.fallbackScope?.textRange?.contains(position.textOffset) == true -> true
else -> false
}
if (!inScope) return false
// Block-local declarations and loop names are not visible before their declaration. Keep
// top-level declarations available for forward references, which Jai permits.
if (symbol.scope != null && symbol.visibleFrom > position.textOffset) return false
return true
}
private fun isVisible(
binding: ImportBinding,
position: PsiElement,
): Boolean =
// File-scope imports are declarations for the whole file even when the directive is
// written below its first use (a common Jai layout). Block-local imports remain ordered.
(binding.scope == null || binding.scope.textRange.contains(position.textOffset)) &&
(binding.scope == null || binding.visibleFrom <= position.textOffset)
private fun isInside(
element: PsiElement,
position: PsiElement,
): Boolean = element == position || element.textRange.contains(position.textOffset)
private fun scopeDepth(scope: PsiElement?): Int {
var current = scope
var depth = 0
while (current != null) {
if (current is JaiBlock || current is JaiForStatement || current is JaiProcLiteralExpr) depth++
current = current.parent
}
return depth
}
private fun debug(message: String) {
if (java.lang.Boolean.getBoolean(COMPLETION_DEBUG_PROPERTY)) LOG.info(message)
}
private fun symbolName(element: PsiElement): String? {
val text = element.text.trim()
if (text.isEmpty()) return null
val name =
when (element) {
is JaiParamName -> text.removePrefix("$$").removePrefix("$")
else -> text
}
return normalizeName(name).takeIf(String::isNotEmpty)
}
private fun normalizeName(name: String): String {
val trimmed = name.trim()
return if (trimmed.length >= 2 && trimmed.first() == '`' && trimmed.last() == '`') {
trimmed.substring(1, trimmed.length - 1)
} else {
trimmed
}
}
}
private fun JaiDirectiveExpr.directiveName(): String? = node.findChildByType(JaiTokenTypes.DIRECTIVE)?.text

View File

@@ -1,6 +1,7 @@
package dev.hgh.jai.settings package dev.hgh.jai.settings
import com.intellij.openapi.application.ApplicationManager import com.intellij.openapi.application.ApplicationManager
import com.intellij.openapi.components.PathMacroManager
import com.intellij.openapi.components.PersistentStateComponent import com.intellij.openapi.components.PersistentStateComponent
import com.intellij.openapi.components.State import com.intellij.openapi.components.State
import com.intellij.openapi.components.Storage import com.intellij.openapi.components.Storage
@@ -62,7 +63,7 @@ class JaiProjectSettings(
val trimmed = rawPath.trim() val trimmed = rawPath.trim()
if (trimmed.isEmpty()) return null if (trimmed.isEmpty()) return null
val expanded = var expanded =
when { when {
trimmed == "~" -> { trimmed == "~" -> {
System.getProperty("user.home") System.getProperty("user.home")
@@ -76,6 +77,20 @@ class JaiProjectSettings(
trimmed trimmed
} }
} }
// Workspace files may persist project-relative roots using IntelliJ path macros. The
// settings UI normally supplies absolute paths, but loading a project saved elsewhere
// must expand these before the resolver tries to find the VFS root.
expanded = PathMacroManager.getInstance(project).expandPath(expanded)
expanded =
expanded
.replace("\$USER_HOME\$", System.getProperty("user.home"))
.replace("\$USER_HOME", System.getProperty("user.home"))
.let { value ->
val basePath = project.basePath ?: return@let value
value
.replace("\$PROJECT_DIR\$", basePath)
.replace("\$PROJECT_DIR", basePath)
}
val file = File(expanded) val file = File(expanded)
val absolute = val absolute =
if (file.isAbsolute) { if (file.isAbsolute) {

View File

@@ -54,6 +54,254 @@ class JaiCompletionTest : BasePlatformTestCase() {
myFixture.checkResult("#load \"other.jai\";") myFixture.checkResult("#load \"other.jai\";")
} }
fun testCompletesVisibleDeclarationsInSameFile() {
myFixture.configureByText(
"symbols.jai",
"""
helper :: () {}
main :: () { hel<caret>(); }
""".trimIndent(),
)
myFixture.completeBasic()
myFixture.checkResult(
"""
helper :: () {}
main :: () { helper(); }
""".trimIndent(),
)
}
fun testCompletesLocalDeclarationsInProcedure() {
myFixture.configureByText(
"locals.jai",
"""
main :: () {
local_value := 1;
print(local<caret>);
}
""".trimIndent(),
)
myFixture.completeBasic()
assertEquals(
"completion should insert the local declaration",
"""
main :: () {
local_value := 1;
print(local_value);
}
""".trimIndent(),
myFixture.file.text,
)
}
fun testCompletesSymbolsFromImportedModule() {
myFixture.configureByText(
"module-consumer.jai",
"""
#import "Basic";
main :: () { allo<caret>(); }
""".trimIndent(),
)
myFixture.complete(CompletionType.BASIC, 1)
val strings = myFixture.lookupElementStrings
assertNotNull("expected symbols from the imported Basic module", strings)
assertTrue("expected alloc in $strings", strings!!.contains("alloc"))
}
fun testCompletesSymbolsFromLoadedFile() {
myFixture.addFileToProject("library.jai", "Helper :: () {}")
myFixture.configureByText(
"consumer.jai",
"""
#load "library.jai";
main :: () { Hel<caret>(); }
""".trimIndent(),
)
myFixture.completeBasic()
assertEquals(
"completion should insert the loaded symbol",
"""
#load "library.jai";
main :: () { Helper(); }
""".trimIndent(),
myFixture.file.text,
)
}
fun testCompletesLocalDeclarationsWithEmptyPrefix() {
myFixture.configureByText(
"empty-prefix.jai",
"""
main :: () {
local_value := 1;
print(<caret>);
}
""".trimIndent(),
)
myFixture.complete(CompletionType.BASIC, 1)
assertTrue(
"local declarations should be offered with an empty prefix",
myFixture.lookupElementStrings.orEmpty().contains("local_value"),
)
}
fun testCompletesProcedureParameters() {
myFixture.configureByText(
"parameters.jai",
"""
main :: (local_value: int) {
print(local<caret>);
}
""".trimIndent(),
)
myFixture.completeBasic()
myFixture.checkResult(
"""
main :: (local_value: int) {
print(local_value);
}
""".trimIndent(),
)
}
fun testCompletesLocalDeclarationBeforeUnterminatedExpression() {
myFixture.configureByText(
"unterminated-local.jai",
"""
main :: () {
foobazz := 1;
fo<caret>
}
""".trimIndent(),
)
myFixture.complete(CompletionType.BASIC, 1)
assertTrue(
"local declaration should survive an unterminated completion expression: ${myFixture.lookupElementStrings}",
myFixture.lookupElementStrings.orEmpty().contains("foobazz"),
)
}
fun testCompletesLoadedSymbolsBeforeUnterminatedExpression() {
myFixture.addFileToProject("library.jai", "loaded_value :: 1;")
myFixture.configureByText(
"unterminated-import.jai",
"""
main :: () {
local_value := 1;
lo<caret>
}
#load "library.jai";
""".trimIndent(),
)
myFixture.complete(CompletionType.BASIC, 1)
val strings = myFixture.lookupElementStrings.orEmpty()
assertTrue("local symbol should be offered: $strings", "local_value" in strings)
assertTrue("loaded symbol should be offered: $strings", "loaded_value" in strings)
}
fun testDoesNotLeakDeclarationsFromTransitiveImports() {
myFixture.addFileToProject("dependency.jai", "transitive_value :: 1;")
myFixture.addFileToProject(
"library.jai",
"""
#import,file "dependency.jai";
direct_value :: 1;
""".trimIndent(),
)
myFixture.configureByText(
"transitive-consumer.jai",
"""
#load "library.jai";
main :: () { tra<caret>; }
""".trimIndent(),
)
myFixture.complete(CompletionType.BASIC, 1)
assertFalse(
"completion should not recursively expose every imported file: ${myFixture.lookupElementStrings}",
myFixture.lookupElementStrings.orEmpty().contains("transitive_value"),
)
}
fun testCompletesMembersOfAnImportedAlias() {
myFixture.addFileToProject("library.jai", "Helper :: () {}\nother_value :: 1;")
myFixture.configureByText(
"consumer.jai",
"""
Library :: #import,file "library.jai";
main :: () { Library.Hel<caret>(); }
""".trimIndent(),
)
myFixture.completeBasic()
myFixture.checkResult(
"""
Library :: #import,file "library.jai";
main :: () { Library.Helper(); }
""".trimIndent(),
)
}
fun testCompletesImportedAliasMembersWithEmptyPrefix() {
myFixture.addFileToProject("library.jai", "Helper :: () {}")
myFixture.configureByText(
"consumer.jai",
"""
Library :: #import,file "library.jai";
main :: () { Library.<caret>(); }
""".trimIndent(),
)
myFixture.complete(CompletionType.BASIC, 1)
assertTrue(
"module members should be offered after a dot with an empty prefix",
myFixture.lookupElementStrings.orEmpty().contains("Helper"),
)
}
fun testBoundImportDoesNotLeakMembersIntoUnqualifiedCompletion() {
myFixture.addFileToProject("library.jai", "Helper :: () {}")
myFixture.configureByText(
"consumer.jai",
"""
Library :: #import,file "library.jai";
main :: () { Hel<caret>(); }
""".trimIndent(),
)
myFixture.complete(CompletionType.BASIC, 1)
assertFalse(
"bound module members should require the module qualifier",
myFixture.lookupElementStrings.orEmpty().contains("Helper"),
)
}
fun testCompletesFileScopeImportsDeclaredAfterUse() {
myFixture.addFileToProject("library.jai", "Helper :: () {}")
myFixture.configureByText(
"consumer.jai",
"""
main :: () { Hel<caret>(); }
#import,file "library.jai";
""".trimIndent(),
)
myFixture.completeBasic()
myFixture.checkResult(
"""
main :: () { Helper(); }
#import,file "library.jai";
""".trimIndent(),
)
}
private fun completeStrings(): List<String>? { private fun completeStrings(): List<String>? {
myFixture.complete(CompletionType.BASIC, 1) myFixture.complete(CompletionType.BASIC, 1)
return myFixture.lookupElementStrings return myFixture.lookupElementStrings

View File

@@ -1,5 +1,6 @@
package dev.hgh.jai.settings package dev.hgh.jai.settings
import com.intellij.codeInsight.completion.CompletionType
import com.intellij.openapi.vfs.LocalFileSystem import com.intellij.openapi.vfs.LocalFileSystem
import com.intellij.psi.PsiFile import com.intellij.psi.PsiFile
import com.intellij.psi.PsiManager import com.intellij.psi.PsiManager
@@ -86,6 +87,49 @@ class JaiConfiguredRootTest : BasePlatformTestCase() {
myFixture.checkResult("#load \"external.jai\";") myFixture.checkResult("#load \"external.jai\";")
} }
fun testConfiguredRootSuppliesImportedSymbolCompletion() {
configureRoot("Custom/module.jai" to "Helper :: () {}")
myFixture.configureByText(
"consumer.jai",
"""
#import "Custom";
main :: () { Hel<caret>(); }
""".trimIndent(),
)
myFixture.completeBasic()
myFixture.checkResult(
"""
#import "Custom";
main :: () { Helper(); }
""".trimIndent(),
)
}
fun testProbeStyleWorkspaceRootsSupplyLocalAndImportedCompletions() {
val settings = JaiProjectSettings.getInstance(project)
val jaiModules = Path.of(System.getProperty("user.home"), ".local", "jai", "modules")
val probeModules = Path.of(System.getProperty("user.home"), "programming", "probe", "modules")
settings.setRootPaths(listOf(jaiModules.toString(), probeModules.toString()))
myFixture.configureByText(
"main.jai",
"""
Foo :: struct { bar: int; }
main :: () {
foobazz := Foo.{};
fo<caret>;
}
#import "Foo";
#import "Basic";
""".trimIndent(),
)
myFixture.complete(CompletionType.BASIC, 1)
val strings = myFixture.lookupElementStrings.orEmpty()
assertTrue("local variable should be offered: $strings", "foobazz" in strings)
assertTrue("Foo module symbol should be offered: $strings", "foo" in strings)
}
fun testReferencesSearchIncludesProjectUsagesForConfiguredExternalDeclaration() { fun testReferencesSearchIncludesProjectUsagesForConfiguredExternalDeclaration() {
val module = configureRoot("Custom/module.jai" to "Helper :: () {}") val module = configureRoot("Custom/module.jai" to "Helper :: () {}")
val source = val source =