grant/intellijai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

initial commit

Browse Source
This commit is contained in:
hgranthorner
2026-08-04 10:12:15 -04:00
commit cb5a1d9555
26 changed files with 1792 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

210
docs/BUILD_PLAN.md Normal file
View File

@@ -0,0 +1,210 @@
# Build Plan — Jai Language Plugin for IntelliJ
**Design constraint:** every milestone must be verifiable by an agent from the
command line. No manual IDE launch, no human eyeballing. This drives the
architecture below, not just the CI config.
Companion doc: `docs/JAI_LANGUAGE_REFERENCE.md` (language facts, derived from
`~/.local/jai`).
---
## 0. Verified starting conditions
Checked empirically before writing this plan:
| Fact | Status |
| --- | --- |
| Scaffold is the bare JetBrains template (demo tool window) | confirmed |
| Gradle 9.6.1, IntelliJ Platform 2025.3.5, Kotlin 2.3.20, IPGP 2.18.1 | confirmed |
| `./gradlew help` succeeds | confirmed |
| **No JDK on `PATH`; `JAVA_HOME` unset** | ⚠️ **blocker** |
| Adoptium JDK 21.0.12 present at `~/.gradle/jdks/…` | confirmed working |
| `org.gradle.java.home` in `gradle.properties` does **not** fix it | confirmed — `gradlew` needs a JVM to launch itself |
| Wrapper script that auto-discovers the JDK **does** fix it | confirmed, `BUILD SUCCESSFUL` |
| `generateLexer` / `generateParser` available via bundled `org.jetbrains.intellij.platform.grammarkit` | confirmed by `gradlew tasks --all` |
| Not a git repository | ⚠️ blocker for safe agent iteration |
| Jai corpus: **714 `.jai` files, 16.4 MB** | confirmed |
The 714-file corpus is the single most important asset for this project — see §3.
---
## 1. Phase 0 — Make the project agent-operable (do this first)
Nothing else is testable until this is done.
1. **`./jaigradle`** — committed wrapper that resolves a JDK then delegates to
`./gradlew`. Verified working. Every later instruction uses this, so an agent
never has to think about `JAVA_HOME`.
2. **`git init`** + `.gitignore` for `build/`, `.gradle/`, `.intellijPlatform/`,
and generated sources. Agents need a clean diff and a rollback point.
3. **`AGENTS.md`** — the contract for future agents: the one command to run
(`./jaigradle check`), where the corpus lives, where generated code lives and
that it must never be hand-edited, and a pointer to the language reference.
4. **Strip the template** — delete `MyToolWindowFactory`, the demo tool window
registration, and the message bundle stub; rewrite `plugin.xml` metadata.
5. **Prove the harness** — add one trivial passing test and confirm
`./jaigradle test` runs it headlessly. This validates the whole loop before
any real work.
**Exit gate:** `./jaigradle check` green, from a shell with no `JAVA_HOME`.
---
## 2. Architecture decisions (chosen for testability)
### 2.1 Hand-written lexer, not JFlex — recommended
Standard practice is JFlex, but two Jai features argue against it:
- **Here-strings have a dynamic terminator.** `#string DONE … DONE` where the
terminator is an arbitrary identifier captured at runtime. JFlex is a DFA
generator; it cannot match a token captured earlier. This is only expressible
as hand-written Java inside a JFlex action — i.e. the hard part isn't in the
grammar anyway.
- **Nested block comments** need a depth counter, again custom action code.
Add the practical argument: the lexer is the component agents will iterate on
most, and a hand-written `LexerBase` removes a codegen round-trip from that
loop. It is also directly unit-testable with plain JUnit.
I'd write `JaiLexer extends LexerBase` by hand, with `JaiTokenTypes` as the
shared token holder. If this turns out badly we can retreat to JFlex later —
the token set and tests stay valid either way, so the decision is reversible.
### 2.2 Grammar-Kit BNF for parser + PSI — recommended
Here codegen genuinely pays: Grammar-Kit generates the PSI class hierarchy,
which is a large amount of boilerplate. It's the standard path, and I verified
the tasks run headlessly from our existing plugin (no extra dependency — the ID
is `org.jetbrains.intellij.platform.grammarkit`, bundled since IPGP 2.12.0).
The `.bnf` references the hand-written lexer's token types. Generated sources go
to a dedicated `src/main/gen` root, git-tracked so diffs are reviewable, and
regenerated by `./jaigradle generateParser`.
Caveat: Grammar-Kit does not support two-pass generation, so no method mixins —
use the `mixin`/`extends` attributes instead.
### 2.3 Sequencing: ship a useful plugin before touching the parser
Syntax highlighting needs only the lexer. That's most of the perceived value and
it de-risks the schedule. The parser is Phase 3, not Phase 1.
---
## 3. The testing strategy
This is the part that answers "agents test it without me booting an IDE."
### Tier 0 — Corpus invariants (highest value, no IDE, plain JUnit)
Run the lexer across all 714 real `.jai` files and assert:
1. **Round-trip:** concatenating every token's text reproduces the source file
**byte for byte.** An IntelliJ lexer must tile the input with no gaps or
overlaps; violating this corrupts the editor. Over 16 MB of real code this one
assertion catches the overwhelming majority of lexer bugs.
2. **No `BAD_CHARACTER`** tokens anywhere in the distribution.
3. **Progress:** every `advance()` strictly increases the offset — catches
infinite loops, which otherwise hang the IDE rather than failing visibly.
This is the flagship gate. It's fast, deterministic, needs no IntelliJ fixture,
and it is *real-world* coverage rather than toy snippets. Failures should report
file, line, and column.
### Tier 1 — Golden token dumps (`LexerTestCase`)
Hand-picked adversarial snippets, one per gotcha in the reference doc §14:
nested comments, here-strings, `---` vs `--` vs `->` vs `-=`, `,,`, `.{`/`.[`/
`.IDENT`, `#char "a"` (and that `'` is *not* a delimiter), `==` before `{`,
hex/binary/hexfloat/underscored numerics, backticked identifiers.
Golden files make regressions obvious in a diff.
### Tier 2 — Parsing tests (`ParsingTestCase`)
Fixture `.jai` files with expected PSI trees in `.txt`. Useful property for
agents: **`ParsingTestCase` writes the expected file automatically if missing**,
so adding a case is cheap — but the generated tree must be reviewed before
committing, or the test asserts nothing.
### Tier 3 — Corpus parse gate
Parse all 714 files, assert **zero `PsiErrorElement`**. This is the acceptance
criterion for "the grammar is correct," and no human inspection can match it.
Expect to allow a small, explicit, shrinking allowlist of known-unsupported
files rather than blocking the phase on 100% — but the allowlist must be
committed and visible so it can't quietly grow.
### Tier 4 — Code-insight tests (`BasePlatformTestCase`, headless)
`myFixture` drives annotators, folding, brace matching, commenter, completion,
formatter, and rename entirely headlessly. Highlighting is verified with
`checkHighlighting` against `<warning>`-annotated fixtures — this is how you
check colors without looking at a screen.
### Tier 5 — Packaging and compatibility
`./jaigradle verifyPlugin` (Plugin Verifier — catches API misuse and
compatibility breaks) and `buildPlugin` (produces the installable ZIP).
### Single entry point
`./jaigradle check` runs Tiers 0–4. Agents run one command; `runIde` exists for
you but is never required of an agent.
---
## 4. Phased milestones
Each phase has a machine-checkable gate. Do not advance without a green gate.
| Phase | Deliverable | Gate |
| --- | --- | --- |
| **0** | Env fix, git, AGENTS.md, template stripped | `./jaigradle check` green with no `JAVA_HOME` |
| **1** | File type, icon, `JaiTokenTypes`, hand-written lexer | Tier 0 corpus invariants pass on all 714 files |
| **2** | `SyntaxHighlighter`, color settings page, commenter, brace matcher | Tier 1 golden dumps; highlighter maps every token type |
| **3** | `.bnf` grammar, generated parser + PSI, `ParserDefinition` | Tier 2 golden trees; Tier 3 corpus parse ≥ target |
| **4** | Structure view, folding, `#import`/`#load` reference resolution + go-to-definition | Tier 4 fixture tests |
| **5** | Completion (keywords, directives, module names), rename, find-usages | Tier 4 fixture tests |
| **6** | Formatter, code style settings | Formatter round-trip: formatting the corpus is idempotent |
| **7** | Inspections (e.g. `#must` misuse), quick fixes, live templates | Tier 4 + `verifyPlugin` |
| **8** | Optional: run-configuration to invoke the `jai` compiler, parse its error output | Integration test against `~/.local/jai/bin` |
Phase 6's idempotence check (format twice, assert no change) is another
corpus-scale invariant that needs no human judgment.
---
## 5. Risks and how the plan handles them
| Risk | Mitigation |
| --- | --- |
| **Environment is the real blocker** — no JDK on PATH | Phase 0 wrapper script, already verified |
| Grammar-Kit codegen needing the IDE UI | Verified false — tasks run from CLI |
| Here-strings breaking a DFA lexer | Drove the hand-written lexer decision |
| Nested comments / `'` / `.{` mis-lexed | Explicitly enumerated as Tier 1 cases |
| Grammar churn silently breaking the parser | Tier 3 corpus gate |
| Generated sources hand-edited then lost | Committed to `src/main/gen`, AGENTS.md forbids editing, regeneration is one command |
| Slow feedback discouraging test runs | Tier 0 is plain JUnit with no fixture boot; keep it under a few seconds |
| Corpus files using features we never support | Explicit committed allowlist, not a silently loosened assertion |
---
## 6. Open questions for you
1. **Scope/ambition** — stop at solid syntax highlighting + navigation (Phases
0–4), or go all the way to formatter and inspections (0–7)?
2. **Compiler integration** (Phase 8) — worth it? It's the only phase needing
the actual `jai` binary, and it's the least testable.
3. **Target IDE** — IntelliJ IDEA only (current setting), or all JetBrains IDEs?
The latter is mostly a `plugin.xml` dependency change, cheapest decided now.
4. **JDK policy** — rely on the Gradle-provisioned JDK via the wrapper (zero
setup, works today), or install a system JDK via Homebrew? I'd default to the
wrapper.
My recommendation: approve Phases 0–4 now, decide on 5–8 once the parser gate is
green and we can see how the grammar behaved against real code.

618
docs/JAI_LANGUAGE_REFERENCE.md Normal file
View File

@@ -0,0 +1,618 @@
# Jai Language Reference (for plugin implementors)
**Provenance:** Everything here was derived from the local Jai distribution at
`~/.local/jai` — primarily `modules/Jai_Lexer/module.jai` (the compiler's own
lexer, authoritative for tokens/keywords), plus `how_to/` and `modules/`.
**No online sources were used; online Jai material is outdated.**
Anything marked ⚠️ is inferred rather than directly confirmed in the lexer.
Re-verify against: `~/.local/jai/modules/Jai_Lexer/module.jai`
---
## 1. Source layout
- Extension: `.jai`
- Encoding: UTF-8 assumed; the language imposes no encoding on `string`.
- Line endings: `\n` or `\r\n` (here-strings normalize to `\n`).
- No zero-terminated strings; no preprocessor in the C sense.
---
## 2. Comments
```jai
// line comment to end of line
/* block comment
/* THESE NEST — track depth, do not stop at the first */
still inside */
```
**Nested block comments are real.** `Jai_Lexer/module.jai:1436` maintains
`comment_depth`, incrementing on `/*` and decrementing on `*/`.
A lexer that stops at the first `*/` is wrong.
Note the source comment at line 1433: the lexer will treat `/////*` as a nested
open-comment (the author flags this as possibly undesirable). Match the simple
depth-counting behavior.
---
## 3. Keywords
Exact list, from `check_for_keyword` (`module.jai:758`). These are the *only*
identifiers promoted to keyword tokens.
| Len | Keywords |
| ----- | ---------- |
| 2 | `if` `xx` |
| 3 | `ifx` `for` |
| 4 | `then` `else` `null` `case` `enum` `true` `cast` |
| 5 | `while` `break` `using` `defer` `false` `union` |
| 6 | `return` `struct` `remove` `inline` |
| 7 | `size_of` `type_of` `code_of` `context` |
| 8 | `continue` `operator` |
| 9 | `type_info` `no_inline` `interface` |
| 10 | `enum_flags` |
| 11 | `is_constant` |
| 12 | `push_context` |
| 14 | `initializer_of` |
Flat list:
```text
if xx ifx for then else null case enum true cast while break using defer false
union return struct remove inline size_of type_of code_of context continue
operator type_info no_inline interface enum_flags is_constant push_context
initializer_of
```
Notes:
- `xx` is the auto-cast operator (`KEYWORD_AUTO_CAST`), not an identifier.
- `remove` is a loop-body statement for removing the current element.
- `then` is used by `ifx` (`ifx cond then a else b`).
- `interface` appears in type restrictions: `(x: $T/interface Matchable)`.
- `struct`, `union`, `enum`, `enum_flags` are all type-constructor keywords.
**Not keywords** (contrary to what one might assume): `int`, `float`, `bool`,
`string`, `s8`..`s64`, `u8`..`u64`, `float32`, `float64`, `void`, `Type`, `Any`,
`Code`. These are ordinary identifiers resolving to built-in types. Highlight
them as *built-in types*, in a distinct style from keywords, and do not let a
parser depend on them being reserved.
### Built-in type names (identifiers, highlight separately)
```text
s8 s16 s32 s64 u8 u16 u32 u64 int
float float32 float64 bool string void
Type Any Code
```
`int` is an alias for `s64`; `float` is an alias for `float32`.
---
## 4. Operators and punctuation
From `Token_Type` (`module.jai:17`). Multi-character tokens must be matched
longest-first.
### Arithmetic / assignment
```text
+ - * / %
+= -= *= /= %=
```
### Comparison / logical
```text
== != < > <= >= && || !
&&= ||=
```
`ISEQUAL_FOR_SWITCH_STATEMENT` is a distinct token: `==` immediately followed by
a `{` block introduces the switch form (`if x == { case ...; }`). The lexer
distinguishes it; a highlighter can ignore the distinction, a parser cannot.
### Bitwise
```text
& | ^ ~
&= |= ^=
<< >> (SHIFT_LEFT, SHIFT_RIGHT)
<<= >>=
<<< >>> (ROTATE_LEFT, ROTATE_RIGHT — disambiguated by the parser, not the lexer)
<<<= >>>=
```
### Distinctive Jai tokens
| Token | Name | Meaning |
| ------- | ------ | --------- |
| `->` | `RIGHT_ARROW` | procedure return type |
| `..` | `DOUBLE_DOT` | inclusive range, `for 0..7` |
| `$` | | polymorphic type/value capture |
| `$$` | `DOUBLE_DOLLAR` | optionally-constant parameter |
| `---` | `TRIPLE_MINUS` | "do not initialize", `x: T = ---;` |
| `--` | `DOUBLE_MINUS` | distinct token (⚠️ not a decrement operator in normal code) |
| `,,` | `DOUBLE_COMMA` | inline context modification, `join(a,, allocator=temp)` |
| `.{` | `BEGIN_STRUCT_LITERAL` | struct literal |
| `.[` | `BEGIN_ARRAY_LITERAL` | array literal |
| `=>` | `QUICK_LAMBDA` | quick lambda, `x => x.count` |
| `.*` | `POSTFIX_DEREFERENCE` | pointer dereference, `ptr.*` |
| `*` | `POINTER_DEREFERENCE` | prefix: address-of AND pointer-type marker |
| `===` | `TRIPLE_EQUALS` | `#asm` register pinning |
| `` ` `` | backtick | see §9 |
| `:` `::` `:=` | | declaration forms, see §6 |
**Critical `*` semantics — inverted vs C.** `*T` is "pointer to T" *and* `*value`
is "address of value". Dereference is postfix `.*`. There is no prefix `*`
dereference. This matters for any expression parser.
`-` lexing (`module.jai:1680`): `->` then `---` then `--` then `-=` then `-`.
---
## 5. Literals
### Integers
```jai
123
1_000_000 // underscores allowed anywhere as separators
0xfade_deaf // hex
0b101101101 // binary
```
Default type of an integer literal is `s64`, but literals implicitly convert to
any type they fit in.
### Floats
```jai
37.0
1.111
0h7fbf_ffff // IEEE-754 bits in hex ("hexfloat"), 16 hex digits max
0h8000_0000_0000_0000 // 64-bit negative zero
```
Default is `float32` unless precision demands `float64`.
`Value_Flags` tracks `HEX`, `BINARY`, `FLOAT`, `REQUIRES_FLOAT64`, `OVERFLOWED`.
### Strings
```jai
"Hello, Sailor!" // escapes: \n \t \" \\ etc.
```
Strings are `{count: s64, data: *u8}` views — **not** zero-terminated, and
subscripting yields `u8` (there is no character type).
### Here-strings (`#string`)
```jai
THE_STRING :: #string DONE
Anything at all, including "quotes" and \n literally.
DONE
```
Syntax: `#string <IDENT>` then a newline, then raw text, terminated by a line
that **starts with** `<IDENT>`. The terminator identifier is arbitrary (`DONE`
is only convention). Flagged as `Value_Flags.HERE_STRING`.
There is a `#string` variant used with imports: `#import,string #string DONE`.
For an IntelliJ lexer this needs a dedicated state with the terminator captured,
much like heredocs in shell/Perl.
### Character literals
```jai
#char "a" // yields a u8
```
`#char` is a directive, not a `'x'` literal form. **There is no single-quote
character literal in Jai.** Do not lex `'` as a string delimiter.
### Notes (`@`)
```jai
foo :: () { } @PrintLike @Deprecated
x: int; @Cleanup
```
`@Identifier` is a `NOTE` token. Common in the wild: `@Cleanup`, `@Incomplete`,
`@Speed`, `@Robustness`, `@Temporary`, `@ToDo`, `@Hack`, `@Copypasta`, `@test`,
`@PrintLike`, `@NoProfile`. Treat as metadata/annotation for highlighting.
`Note_Flags.IS_SYSTEM_LEVEL` distinguishes compiler-known notes.
---
## 6. Declarations
The universal declaration form is `name : type = value`, with parts omissible:
```jai
a : float = 37.0; // explicit type + value
b : float; // explicit type, default-initialized to zero
c := 111.0; // type inferred (: and = merge into :=)
d : float : -123.45; // CONSTANT (second colon instead of =)
e :: 42; // constant, type inferred
f : T = ---; // explicitly UNINITIALIZED (no zeroing)
x, y, z: float; // compound declaration
```
**Everything is zero-initialized by default** unless `= ---` is used.
`::` (constant) is how procedures, structs, and enums are declared — they are
just constant values:
```jai
main :: () { }
Person :: struct { }
Fruit :: enum u32 { }
```
This is the single most important structural fact for a parser: there is no
`func`/`fn`/`class` keyword. A top-level declaration is
`IDENT :: <struct|enum|enum_flags|union|(params)...|expr>`.
---
## 7. Procedures
```jai
name :: (a: int, b: float) -> ReturnType { ... }
name :: (a: int) -> named: ReturnType { ... } // named return value
name :: (a: int) -> (out: [] T) { ... } // parenthesized
name :: () -> A, B { ... } // multiple returns
name :: (x: int, y := 10) { ... } // default argument
name :: (fmt: string, args: ..Any) { ... } // variadic
```
Calls support named arguments: `f(flavor = "chocolate", num_scoops = 3)`.
### Quick lambdas
```jai
quick_sort(to_sort, x => x.count);
quick_sort(to_sort, (a, b) => ifx a.count != b.count then b.count-a.count else 0);
count1 :: x => x.count;
```
### Operator overloading
```jai
operator + :: (x: Complex, y: Complex) -> Complex { ... }
operator == :: (a: T, b: T) -> bool #symmetric { ... }
operator [] :: (a: Bit_Array, index: int) -> bool { ... }
```
`operator` is a keyword; the token following it is the operator symbol, then
`::` and the procedure. `OPERATOR_ARRAY_SUBSCRIPT` / `OPERATOR_ASSIGNMENT_TO_ARRAY_SUBSCRIPT`
exist as parser-internal token types for `[]`.
### Polymorphism
```jai
square :: (x: $T) -> T { return x*x; } // capture T from argument type
array_add :: (array: *[..] $T, item: T) { } // capture inside a compound type
multiplier :: ($T: Type) { } // $ = must be compile-time constant
divider :: (x: int, $$ y: int) -> int { } // $$ = optionally constant
discuss :: (x: $T/interface Matchable) { } // type restriction via interface
proc :: (x: $T/SomeStruct) { } // ⚠️ restriction by type
Holder :: struct ($T: Type, $N: s64) { } // polymorphic struct
```
`$` on a parameter type captures it; `$` on a parameter name requires the
*value* be compile-time constant. `/` after a polymorphic capture introduces a
restriction.
---
## 8. Types
```jai
[8] int // fixed-size array
[] float // array view {count, data}
[..] int // resizable/dynamic array
*T // pointer to T
[9] u8 // e.g. Phone_Number :: [9] u8;
```
### Structs
```jai
Rectangle :: struct {
x0, y0: float;
color_name: string;
temperature := -10.0; // default value
info: Ice_Cream_Info;
info.flavor = "chocolate"; // override a sub-struct default, inside the struct body
}
```
Struct bodies may contain assignment statements that set nested defaults.
Layout is declaration order, always, with no compiler reordering.
```jai
Video_File :: struct {
#as using base: Document; // #as enables implicit cast to Document
// using imports Document's names
}
```
### Enums
```jai
Fruits :: enum u32 {
BANANA :: 5; // explicit value
APPLE; // auto-increments
}
Fruits :: enum u32 #specified { BANANA :: 1; } // values locked for serialization
Flags :: enum_flags u8 { A :: 0x1; B :: 0x2; }
```
**Unary dot:** enum values can be written `.APRICOT` when the type is inferred.
This means a leading `.` followed by an identifier is a valid *expression*, and
must not be confused with member access. Same for `.{` and `.[`.
### Unions
```jai
union { a: int; b: float; }
```
---
## 9. Control flow
```jai
if cond { } else if cond { } else { }
if cond single_statement;
ifx cond then a else b // expression form
if value == { // switch form
case .BANANA; print("...");
case .APPLE; x := 1; print("..."); // each case is its own scope
case; print("default"); // valueless case = default
}
```
Cases **do not** fall through. To fall through, end the block with `#through`.
`#complete` on the `if` asserts all enum cases are handled.
```jai
while cond { }
break; continue;
```
### For loops — many forms
```jai
for 1..7 { print("%", it); } // range; implicit `it`
for i: 1..7 { } // named index
for a..b { } // runtime bounds
for < b..a { } // REVERSED (note the `<`)
for array { print("%", it); } // iterate values; `it` and `it_index`
for < numbers { } // reversed iteration
for value, i: numbers { } // named value and index
for * teas { } // iterate BY POINTER (`it` is a pointer)
for outer: numbers { for inner: numbers { } } // named for nesting
for numbers if (it & 1) == 0 remove it; // `remove` current element
```
Implicit loop variables are `it` (value) and `it_index` (index). They are
ordinary identifiers, but worth special highlighting.
`for <` and `for *` are modifier tokens between `for` and the iteration
expression — a parser must accept `for`, then optional `<` and/or `*`.
### Custom iteration (`for_expansion`)
```jai
for_expansion :: (holder: Holder, body: Code, flags: For_Flags) #expand {
for slot_index: 0..holder.count-1 {
`it_index := slot_index; // backtick exports the name to the caller
`it := holder.values[slot_index];
#insert body;
}
}
for :positive_vibes_only holder { } // ⚠️ select a named expansion
```
### Backticked identifiers
`` `name `` marks an identifier that a macro exports into the caller's scope.
`ident_is_backticked` is a flag on the token. Only `defer`, `return`,
`push_context`, and overloaded operator functions may be backticked among
keywords (`module.jai:1537`).
### defer
```jai
defer x += 1; // runs on ANY exit from the enclosing scope, incl. break
defer free(ptr);
```
### Context
```jai
new_context := context;
new_context.allocator = temp;
push_context new_context { some_function(); }
join(strings,, allocator=temp); // ,, = inline context change for one call
```
---
## 10. Casting
```jai
cast(u8) b
cast,no_check(u8) c // modifier on cast, suppresses range check
xx value // auto-cast (infer target type)
xx,no_check value // ⚠️ modifier form
```
`cast` takes a parenthesized type and an optional `,modifier` between the
keyword and the paren.
---
## 11. Compiler directives (`#`)
Directives are `#` immediately followed by an identifier. Full list observed in
the distribution, ordered by frequency (counts from `modules/`):
**Very common**
`#foreign` `#c_call` `#type` `#cpp_method` `#if` `#import` `#char` `#overlay`
`#elsewhere` `#as` `#run` `#through` `#scope_file` `#load` `#no_context`
`#library` `#type_info_none` `#expand` `#align` `#asm` `#assert` `#scope_module`
`#scope_export` `#caller_location` `#string`
**Common**
`#define` `#cpp_return_type_is_non_pod` `#no_padding` `#no_aoc`
`#bake_arguments` `#insert` `#complete` `#compiler` `#no_abc` `#include`
`#Context` `#code` `#module_parameters` `#endif` `#add_context` `#modify`
`#location` `#deprecated` `#ifdef` `#no_debug` `#symmetric` `#filepath`
`#compile_time` `#intrinsic`
**Occasional**
`#type_info_procedures_are_void_pointers` `#program_export` `#placeholder`
`#procedure_of_call` `#pragma` `#version` `#this` `#bytes`
`#type_info_no_size_complaint` `#discard` `#file` `#else` `#run_and_insert`
`#undef` `#place` `#bake` `#bake_constants` `#no_reset` `#poke_name`
`#specified` `#no_alias` `#dump` `#placeholders`
> ⚠️ Some of these (`#define`, `#include`, `#ifdef`, `#endif`, `#undef`,
> `#pragma`, `#elsewhere`) appear predominantly inside **C-binding files
> generated by `Bindings_Generator`**, which embed C preprocessor text. Do not
> assume they are core language directives.
>
> ⚠️ **`#must` does not exist in this distribution** — a full-tree grep of
> `~/.local/jai` returns zero hits. Older online documentation mentions it. This
> is exactly the kind of drift the "don't search online" instruction guards
> against. Do not add it to the keyword list.
### Semantics of the important ones
| Directive | Meaning |
| ----------- | --------- |
| `#import "Basic"` | import a module; also `#import,string`, `#import,file`, `#import,dir` |
| `#load "file.jai"` | textually include another file into this scope |
| `#run expr` | execute at compile time |
| `#if cond { }` / `#else` | static (compile-time) conditional; can appear at top level |
| `#assert cond` | compile-time assertion |
| `#scope_file` / `#scope_module` / `#scope_export` | change visibility of everything below, until the next scope directive |
| `#expand` | mark a procedure as a macro |
| `#insert code` | splice a `Code` value in |
| `#code expr` | produce a `Code` value |
| `#through` | fall through to next `case` |
| `#complete` | require exhaustive `case` coverage |
| `#specified` | lock enum values for forward compatibility |
| `#as` | allow implicit cast from this struct member |
| `#char "a"` | character byte literal |
| `#string ID` | here-string |
| `#asm { }` | inline x86-64 assembly block |
| `#foreign` / `#library` / `#c_call` | FFI |
| `#caller_location` / `#location()` / `#file` / `#filepath` | source location introspection |
| `#no_abc` / `#no_aoc` | disable array-bounds-check / arithmetic-overflow-check |
| `#place` | overlay a struct member at another member's offset |
| `#module_parameters` | parameterize a module |
| `#modify` | compile-time hook to inspect/alter polymorph resolution |
| `#deprecated "msg"` | deprecation warning |
| `#body_text` / `#poke_name` / `#dump` | metaprogramming utilities |
`#scope_file` etc. are *statement-position* directives that affect everything
following them in the file — relevant if the plugin does symbol visibility.
---
## 12. Modules and imports
```jai
#import "Basic";
#import "Math";
Sort :: #import "Sort"; // bind a module to a name
#load "other_file.jai";
#import "Foo"(PARAM = 3); // ⚠️ module parameters
```
Module search path: `~/.local/jai/modules/`. A module is either
`Name.jai` or a directory `Name/module.jai`. Both forms exist in the
distribution (e.g. `Bit_Array.jai` vs `Basic/module.jai`).
This is the resolution rule a "go to definition" / import-completion feature
must implement.
---
## 13. Highlighting recommendations (IntelliJ token groups)
| Group | Contents |
| ------- | ---------- |
| Keyword | §3 list |
| Built-in type | §3 built-in type names |
| Directive | `#ident` |
| Note / annotation | `@Ident` |
| Number | int/hex/binary/float/hexfloat, incl. `_` separators |
| String | `"..."` |
| Here-string | `#string ID ... ID` (own token, own lexer state) |
| Line comment | `//...` |
| Block comment | `/*...*/` **nesting** |
| Operator | §4 |
| Loop variable | `it`, `it_index` |
| Polymorph | `$T`, `$$x` |
| Backtick ident | `` `name `` |
| Uninitialized | `---` |
---
## 14. Gotchas that break naive implementations
1. **Nested block comments.** Must count depth.
2. **No `'c'` char literal.** `'` is not a string delimiter. Use `#char "c"`.
3. **`*` is address-of (prefix) and pointer-type; `.*` is dereference.** Inverted from C.
4. **`.{`, `.[`, `.IDENT`** — a leading `.` is not always member access.
5. **Here-strings** need a lexer state carrying the terminator identifier.
6. **`--` vs `---` vs `->` vs `-=`** — longest-match order matters.
7. **`,,`** is one token, not two commas.
8. **`==` before `{`** is a distinct token (switch form).
9. **Primitive type names are not keywords** — don't reserve them.
10. **`#must` is not in this version.** Don't trust old online docs.
11. **`for` takes modifiers** (`<`, `*`) before the iterable.
12. **Declarations have no introducer keyword.** `Foo :: struct {}` and
`foo :: () {}` and `foo :: 3;` are all the same syntactic shape.
13. **`remove` is a keyword**, valid only in loop bodies.
14. **`then` is a keyword**, used only by `ifx`.
15. **Struct bodies can contain assignments**, not just declarations.
---
## 15. Primary sources to re-read
| Topic | Path (under `~/.local/jai`) |
| ------- | ------------------------------ |
| Tokens, keywords, lexing | `modules/Jai_Lexer/module.jai` ← authoritative |
| Basics, declarations | `how_to/001_first.jai`, `002_number_types.jai` |
| Arrays / strings | `how_to/004_arrays.jai`, `005_strings.jai` |
| Structs / literals | `how_to/006_structs.jai`, `007_struct_literals.jai` |
| Types as values | `how_to/008_types.jai` |
| Enums | `how_to/013_enums.jai`, `014_enum_unary_dot.jai` |
| Loops | `how_to/019_looping.jai`, `730_for_expansions.jai` |
| if / ifx / switch | `how_to/022_if.jai`, `025_ifx.jai`, `027_if_case.jai` |
| Imports / scopes | `how_to/040_import_and_load/`, `080_scopes.jai`, `151_file_and_global_scopes/` |
| using | `how_to/042_using.jai`, `044_using_advanced/` |
| Operator overloading | `how_to/093_operator_overloading.jai`, `094_array_operators.jai` |
| Polymorphism | `how_to/100_`, `110_`, `115_`, `120_`, `160_type_restrictions.jai` |
| Context / `,,` | `how_to/011_context.jai`, `225_comma_comma.jai` |
| Metaprogramming | `how_to/450_basic_metaprogram/`, `600_insert.jai`, `630_compiler_get_nodes.jai` |
| Inline asm | `how_to/900_inline_assembly.jai` |
| Compiler API | `modules/Compiler/` |