grant/intellijai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

initial commit

Browse Source
This commit is contained in:
hgranthorner
2026-08-04 10:12:15 -04:00
commit cb5a1d9555
26 changed files with 1792 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

618
docs/JAI_LANGUAGE_REFERENCE.md Normal file
View File

@@ -0,0 +1,618 @@
# Jai Language Reference (for plugin implementors)
**Provenance:** Everything here was derived from the local Jai distribution at
`~/.local/jai` — primarily `modules/Jai_Lexer/module.jai` (the compiler's own
lexer, authoritative for tokens/keywords), plus `how_to/` and `modules/`.
**No online sources were used; online Jai material is outdated.**
Anything marked ⚠️ is inferred rather than directly confirmed in the lexer.
Re-verify against: `~/.local/jai/modules/Jai_Lexer/module.jai`
---
## 1. Source layout
- Extension: `.jai`
- Encoding: UTF-8 assumed; the language imposes no encoding on `string`.
- Line endings: `\n` or `\r\n` (here-strings normalize to `\n`).
- No zero-terminated strings; no preprocessor in the C sense.
---
## 2. Comments
```jai
// line comment to end of line
/* block comment
/* THESE NEST — track depth, do not stop at the first */
still inside */
```
**Nested block comments are real.** `Jai_Lexer/module.jai:1436` maintains
`comment_depth`, incrementing on `/*` and decrementing on `*/`.
A lexer that stops at the first `*/` is wrong.
Note the source comment at line 1433: the lexer will treat `/////*` as a nested
open-comment (the author flags this as possibly undesirable). Match the simple
depth-counting behavior.
---
## 3. Keywords
Exact list, from `check_for_keyword` (`module.jai:758`). These are the *only*
identifiers promoted to keyword tokens.
| Len | Keywords |
| ----- | ---------- |
| 2 | `if` `xx` |
| 3 | `ifx` `for` |
| 4 | `then` `else` `null` `case` `enum` `true` `cast` |
| 5 | `while` `break` `using` `defer` `false` `union` |
| 6 | `return` `struct` `remove` `inline` |
| 7 | `size_of` `type_of` `code_of` `context` |
| 8 | `continue` `operator` |
| 9 | `type_info` `no_inline` `interface` |
| 10 | `enum_flags` |
| 11 | `is_constant` |
| 12 | `push_context` |
| 14 | `initializer_of` |
Flat list:
```text
if xx ifx for then else null case enum true cast while break using defer false
union return struct remove inline size_of type_of code_of context continue
operator type_info no_inline interface enum_flags is_constant push_context
initializer_of
```
Notes:
- `xx` is the auto-cast operator (`KEYWORD_AUTO_CAST`), not an identifier.
- `remove` is a loop-body statement for removing the current element.
- `then` is used by `ifx` (`ifx cond then a else b`).
- `interface` appears in type restrictions: `(x: $T/interface Matchable)`.
- `struct`, `union`, `enum`, `enum_flags` are all type-constructor keywords.
**Not keywords** (contrary to what one might assume): `int`, `float`, `bool`,
`string`, `s8`..`s64`, `u8`..`u64`, `float32`, `float64`, `void`, `Type`, `Any`,
`Code`. These are ordinary identifiers resolving to built-in types. Highlight
them as *built-in types*, in a distinct style from keywords, and do not let a
parser depend on them being reserved.
### Built-in type names (identifiers, highlight separately)
```text
s8 s16 s32 s64 u8 u16 u32 u64 int
float float32 float64 bool string void
Type Any Code
```
`int` is an alias for `s64`; `float` is an alias for `float32`.
---
## 4. Operators and punctuation
From `Token_Type` (`module.jai:17`). Multi-character tokens must be matched
longest-first.
### Arithmetic / assignment
```text
+ - * / %
+= -= *= /= %=
```
### Comparison / logical
```text
== != < > <= >= && || !
&&= ||=
```
`ISEQUAL_FOR_SWITCH_STATEMENT` is a distinct token: `==` immediately followed by
a `{` block introduces the switch form (`if x == { case ...; }`). The lexer
distinguishes it; a highlighter can ignore the distinction, a parser cannot.
### Bitwise
```text
& | ^ ~
&= |= ^=
<< >> (SHIFT_LEFT, SHIFT_RIGHT)
<<= >>=
<<< >>> (ROTATE_LEFT, ROTATE_RIGHT — disambiguated by the parser, not the lexer)
<<<= >>>=
```
### Distinctive Jai tokens
| Token | Name | Meaning |
| ------- | ------ | --------- |
| `->` | `RIGHT_ARROW` | procedure return type |
| `..` | `DOUBLE_DOT` | inclusive range, `for 0..7` |
| `$` | | polymorphic type/value capture |
| `$$` | `DOUBLE_DOLLAR` | optionally-constant parameter |
| `---` | `TRIPLE_MINUS` | "do not initialize", `x: T = ---;` |
| `--` | `DOUBLE_MINUS` | distinct token (⚠️ not a decrement operator in normal code) |
| `,,` | `DOUBLE_COMMA` | inline context modification, `join(a,, allocator=temp)` |
| `.{` | `BEGIN_STRUCT_LITERAL` | struct literal |
| `.[` | `BEGIN_ARRAY_LITERAL` | array literal |
| `=>` | `QUICK_LAMBDA` | quick lambda, `x => x.count` |
| `.*` | `POSTFIX_DEREFERENCE` | pointer dereference, `ptr.*` |
| `*` | `POINTER_DEREFERENCE` | prefix: address-of AND pointer-type marker |
| `===` | `TRIPLE_EQUALS` | `#asm` register pinning |
| `` ` `` | backtick | see §9 |
| `:` `::` `:=` | | declaration forms, see §6 |
**Critical `*` semantics — inverted vs C.** `*T` is "pointer to T" *and* `*value`
is "address of value". Dereference is postfix `.*`. There is no prefix `*`
dereference. This matters for any expression parser.
`-` lexing (`module.jai:1680`): `->` then `---` then `--` then `-=` then `-`.
---
## 5. Literals
### Integers
```jai
123
1_000_000 // underscores allowed anywhere as separators
0xfade_deaf // hex
0b101101101 // binary
```
Default type of an integer literal is `s64`, but literals implicitly convert to
any type they fit in.
### Floats
```jai
37.0
1.111
0h7fbf_ffff // IEEE-754 bits in hex ("hexfloat"), 16 hex digits max
0h8000_0000_0000_0000 // 64-bit negative zero
```
Default is `float32` unless precision demands `float64`.
`Value_Flags` tracks `HEX`, `BINARY`, `FLOAT`, `REQUIRES_FLOAT64`, `OVERFLOWED`.
### Strings
```jai
"Hello, Sailor!" // escapes: \n \t \" \\ etc.
```
Strings are `{count: s64, data: *u8}` views — **not** zero-terminated, and
subscripting yields `u8` (there is no character type).
### Here-strings (`#string`)
```jai
THE_STRING :: #string DONE
Anything at all, including "quotes" and \n literally.
DONE
```
Syntax: `#string <IDENT>` then a newline, then raw text, terminated by a line
that **starts with** `<IDENT>`. The terminator identifier is arbitrary (`DONE`
is only convention). Flagged as `Value_Flags.HERE_STRING`.
There is a `#string` variant used with imports: `#import,string #string DONE`.
For an IntelliJ lexer this needs a dedicated state with the terminator captured,
much like heredocs in shell/Perl.
### Character literals
```jai
#char "a" // yields a u8
```
`#char` is a directive, not a `'x'` literal form. **There is no single-quote
character literal in Jai.** Do not lex `'` as a string delimiter.
### Notes (`@`)
```jai
foo :: () { } @PrintLike @Deprecated
x: int; @Cleanup
```
`@Identifier` is a `NOTE` token. Common in the wild: `@Cleanup`, `@Incomplete`,
`@Speed`, `@Robustness`, `@Temporary`, `@ToDo`, `@Hack`, `@Copypasta`, `@test`,
`@PrintLike`, `@NoProfile`. Treat as metadata/annotation for highlighting.
`Note_Flags.IS_SYSTEM_LEVEL` distinguishes compiler-known notes.
---
## 6. Declarations
The universal declaration form is `name : type = value`, with parts omissible:
```jai
a : float = 37.0; // explicit type + value
b : float; // explicit type, default-initialized to zero
c := 111.0; // type inferred (: and = merge into :=)
d : float : -123.45; // CONSTANT (second colon instead of =)
e :: 42; // constant, type inferred
f : T = ---; // explicitly UNINITIALIZED (no zeroing)
x, y, z: float; // compound declaration
```
**Everything is zero-initialized by default** unless `= ---` is used.
`::` (constant) is how procedures, structs, and enums are declared — they are
just constant values:
```jai
main :: () { }
Person :: struct { }
Fruit :: enum u32 { }
```
This is the single most important structural fact for a parser: there is no
`func`/`fn`/`class` keyword. A top-level declaration is
`IDENT :: <struct|enum|enum_flags|union|(params)...|expr>`.
---
## 7. Procedures
```jai
name :: (a: int, b: float) -> ReturnType { ... }
name :: (a: int) -> named: ReturnType { ... } // named return value
name :: (a: int) -> (out: [] T) { ... } // parenthesized
name :: () -> A, B { ... } // multiple returns
name :: (x: int, y := 10) { ... } // default argument
name :: (fmt: string, args: ..Any) { ... } // variadic
```
Calls support named arguments: `f(flavor = "chocolate", num_scoops = 3)`.
### Quick lambdas
```jai
quick_sort(to_sort, x => x.count);
quick_sort(to_sort, (a, b) => ifx a.count != b.count then b.count-a.count else 0);
count1 :: x => x.count;
```
### Operator overloading
```jai
operator + :: (x: Complex, y: Complex) -> Complex { ... }
operator == :: (a: T, b: T) -> bool #symmetric { ... }
operator [] :: (a: Bit_Array, index: int) -> bool { ... }
```
`operator` is a keyword; the token following it is the operator symbol, then
`::` and the procedure. `OPERATOR_ARRAY_SUBSCRIPT` / `OPERATOR_ASSIGNMENT_TO_ARRAY_SUBSCRIPT`
exist as parser-internal token types for `[]`.
### Polymorphism
```jai
square :: (x: $T) -> T { return x*x; } // capture T from argument type
array_add :: (array: *[..] $T, item: T) { } // capture inside a compound type
multiplier :: ($T: Type) { } // $ = must be compile-time constant
divider :: (x: int, $$ y: int) -> int { } // $$ = optionally constant
discuss :: (x: $T/interface Matchable) { } // type restriction via interface
proc :: (x: $T/SomeStruct) { } // ⚠️ restriction by type
Holder :: struct ($T: Type, $N: s64) { } // polymorphic struct
```
`$` on a parameter type captures it; `$` on a parameter name requires the
*value* be compile-time constant. `/` after a polymorphic capture introduces a
restriction.
---
## 8. Types
```jai
[8] int // fixed-size array
[] float // array view {count, data}
[..] int // resizable/dynamic array
*T // pointer to T
[9] u8 // e.g. Phone_Number :: [9] u8;
```
### Structs
```jai
Rectangle :: struct {
x0, y0: float;
color_name: string;
temperature := -10.0; // default value
info: Ice_Cream_Info;
info.flavor = "chocolate"; // override a sub-struct default, inside the struct body
}
```
Struct bodies may contain assignment statements that set nested defaults.
Layout is declaration order, always, with no compiler reordering.
```jai
Video_File :: struct {
#as using base: Document; // #as enables implicit cast to Document
// using imports Document's names
}
```
### Enums
```jai
Fruits :: enum u32 {
BANANA :: 5; // explicit value
APPLE; // auto-increments
}
Fruits :: enum u32 #specified { BANANA :: 1; } // values locked for serialization
Flags :: enum_flags u8 { A :: 0x1; B :: 0x2; }
```
**Unary dot:** enum values can be written `.APRICOT` when the type is inferred.
This means a leading `.` followed by an identifier is a valid *expression*, and
must not be confused with member access. Same for `.{` and `.[`.
### Unions
```jai
union { a: int; b: float; }
```
---
## 9. Control flow
```jai
if cond { } else if cond { } else { }
if cond single_statement;
ifx cond then a else b // expression form
if value == { // switch form
case .BANANA; print("...");
case .APPLE; x := 1; print("..."); // each case is its own scope
case; print("default"); // valueless case = default
}
```
Cases **do not** fall through. To fall through, end the block with `#through`.
`#complete` on the `if` asserts all enum cases are handled.
```jai
while cond { }
break; continue;
```
### For loops — many forms
```jai
for 1..7 { print("%", it); } // range; implicit `it`
for i: 1..7 { } // named index
for a..b { } // runtime bounds
for < b..a { } // REVERSED (note the `<`)
for array { print("%", it); } // iterate values; `it` and `it_index`
for < numbers { } // reversed iteration
for value, i: numbers { } // named value and index
for * teas { } // iterate BY POINTER (`it` is a pointer)
for outer: numbers { for inner: numbers { } } // named for nesting
for numbers if (it & 1) == 0 remove it; // `remove` current element
```
Implicit loop variables are `it` (value) and `it_index` (index). They are
ordinary identifiers, but worth special highlighting.
`for <` and `for *` are modifier tokens between `for` and the iteration
expression — a parser must accept `for`, then optional `<` and/or `*`.
### Custom iteration (`for_expansion`)
```jai
for_expansion :: (holder: Holder, body: Code, flags: For_Flags) #expand {
for slot_index: 0..holder.count-1 {
`it_index := slot_index; // backtick exports the name to the caller
`it := holder.values[slot_index];
#insert body;
}
}
for :positive_vibes_only holder { } // ⚠️ select a named expansion
```
### Backticked identifiers
`` `name `` marks an identifier that a macro exports into the caller's scope.
`ident_is_backticked` is a flag on the token. Only `defer`, `return`,
`push_context`, and overloaded operator functions may be backticked among
keywords (`module.jai:1537`).
### defer
```jai
defer x += 1; // runs on ANY exit from the enclosing scope, incl. break
defer free(ptr);
```
### Context
```jai
new_context := context;
new_context.allocator = temp;
push_context new_context { some_function(); }
join(strings,, allocator=temp); // ,, = inline context change for one call
```
---
## 10. Casting
```jai
cast(u8) b
cast,no_check(u8) c // modifier on cast, suppresses range check
xx value // auto-cast (infer target type)
xx,no_check value // ⚠️ modifier form
```
`cast` takes a parenthesized type and an optional `,modifier` between the
keyword and the paren.
---
## 11. Compiler directives (`#`)
Directives are `#` immediately followed by an identifier. Full list observed in
the distribution, ordered by frequency (counts from `modules/`):
**Very common**
`#foreign` `#c_call` `#type` `#cpp_method` `#if` `#import` `#char` `#overlay`
`#elsewhere` `#as` `#run` `#through` `#scope_file` `#load` `#no_context`
`#library` `#type_info_none` `#expand` `#align` `#asm` `#assert` `#scope_module`
`#scope_export` `#caller_location` `#string`
**Common**
`#define` `#cpp_return_type_is_non_pod` `#no_padding` `#no_aoc`
`#bake_arguments` `#insert` `#complete` `#compiler` `#no_abc` `#include`
`#Context` `#code` `#module_parameters` `#endif` `#add_context` `#modify`
`#location` `#deprecated` `#ifdef` `#no_debug` `#symmetric` `#filepath`
`#compile_time` `#intrinsic`
**Occasional**
`#type_info_procedures_are_void_pointers` `#program_export` `#placeholder`
`#procedure_of_call` `#pragma` `#version` `#this` `#bytes`
`#type_info_no_size_complaint` `#discard` `#file` `#else` `#run_and_insert`
`#undef` `#place` `#bake` `#bake_constants` `#no_reset` `#poke_name`
`#specified` `#no_alias` `#dump` `#placeholders`
> ⚠️ Some of these (`#define`, `#include`, `#ifdef`, `#endif`, `#undef`,
> `#pragma`, `#elsewhere`) appear predominantly inside **C-binding files
> generated by `Bindings_Generator`**, which embed C preprocessor text. Do not
> assume they are core language directives.
>
> ⚠️ **`#must` does not exist in this distribution** — a full-tree grep of
> `~/.local/jai` returns zero hits. Older online documentation mentions it. This
> is exactly the kind of drift the "don't search online" instruction guards
> against. Do not add it to the keyword list.
### Semantics of the important ones
| Directive | Meaning |
| ----------- | --------- |
| `#import "Basic"` | import a module; also `#import,string`, `#import,file`, `#import,dir` |
| `#load "file.jai"` | textually include another file into this scope |
| `#run expr` | execute at compile time |
| `#if cond { }` / `#else` | static (compile-time) conditional; can appear at top level |
| `#assert cond` | compile-time assertion |
| `#scope_file` / `#scope_module` / `#scope_export` | change visibility of everything below, until the next scope directive |
| `#expand` | mark a procedure as a macro |
| `#insert code` | splice a `Code` value in |
| `#code expr` | produce a `Code` value |
| `#through` | fall through to next `case` |
| `#complete` | require exhaustive `case` coverage |
| `#specified` | lock enum values for forward compatibility |
| `#as` | allow implicit cast from this struct member |
| `#char "a"` | character byte literal |
| `#string ID` | here-string |
| `#asm { }` | inline x86-64 assembly block |
| `#foreign` / `#library` / `#c_call` | FFI |
| `#caller_location` / `#location()` / `#file` / `#filepath` | source location introspection |
| `#no_abc` / `#no_aoc` | disable array-bounds-check / arithmetic-overflow-check |
| `#place` | overlay a struct member at another member's offset |
| `#module_parameters` | parameterize a module |
| `#modify` | compile-time hook to inspect/alter polymorph resolution |
| `#deprecated "msg"` | deprecation warning |
| `#body_text` / `#poke_name` / `#dump` | metaprogramming utilities |
`#scope_file` etc. are *statement-position* directives that affect everything
following them in the file — relevant if the plugin does symbol visibility.
---
## 12. Modules and imports
```jai
#import "Basic";
#import "Math";
Sort :: #import "Sort"; // bind a module to a name
#load "other_file.jai";
#import "Foo"(PARAM = 3); // ⚠️ module parameters
```
Module search path: `~/.local/jai/modules/`. A module is either
`Name.jai` or a directory `Name/module.jai`. Both forms exist in the
distribution (e.g. `Bit_Array.jai` vs `Basic/module.jai`).
This is the resolution rule a "go to definition" / import-completion feature
must implement.
---
## 13. Highlighting recommendations (IntelliJ token groups)
| Group | Contents |
| ------- | ---------- |
| Keyword | §3 list |
| Built-in type | §3 built-in type names |
| Directive | `#ident` |
| Note / annotation | `@Ident` |
| Number | int/hex/binary/float/hexfloat, incl. `_` separators |
| String | `"..."` |
| Here-string | `#string ID ... ID` (own token, own lexer state) |
| Line comment | `//...` |
| Block comment | `/*...*/` **nesting** |
| Operator | §4 |
| Loop variable | `it`, `it_index` |
| Polymorph | `$T`, `$$x` |
| Backtick ident | `` `name `` |
| Uninitialized | `---` |
---
## 14. Gotchas that break naive implementations
1. **Nested block comments.** Must count depth.
2. **No `'c'` char literal.** `'` is not a string delimiter. Use `#char "c"`.
3. **`*` is address-of (prefix) and pointer-type; `.*` is dereference.** Inverted from C.
4. **`.{`, `.[`, `.IDENT`** — a leading `.` is not always member access.
5. **Here-strings** need a lexer state carrying the terminator identifier.
6. **`--` vs `---` vs `->` vs `-=`** — longest-match order matters.
7. **`,,`** is one token, not two commas.
8. **`==` before `{`** is a distinct token (switch form).
9. **Primitive type names are not keywords** — don't reserve them.
10. **`#must` is not in this version.** Don't trust old online docs.
11. **`for` takes modifiers** (`<`, `*`) before the iterable.
12. **Declarations have no introducer keyword.** `Foo :: struct {}` and
`foo :: () {}` and `foo :: 3;` are all the same syntactic shape.
13. **`remove` is a keyword**, valid only in loop bodies.
14. **`then` is a keyword**, used only by `ifx`.
15. **Struct bodies can contain assignments**, not just declarations.
---
## 15. Primary sources to re-read
| Topic | Path (under `~/.local/jai`) |
| ------- | ------------------------------ |
| Tokens, keywords, lexing | `modules/Jai_Lexer/module.jai` ← authoritative |
| Basics, declarations | `how_to/001_first.jai`, `002_number_types.jai` |
| Arrays / strings | `how_to/004_arrays.jai`, `005_strings.jai` |
| Structs / literals | `how_to/006_structs.jai`, `007_struct_literals.jai` |
| Types as values | `how_to/008_types.jai` |
| Enums | `how_to/013_enums.jai`, `014_enum_unary_dot.jai` |
| Loops | `how_to/019_looping.jai`, `730_for_expansions.jai` |
| if / ifx / switch | `how_to/022_if.jai`, `025_ifx.jai`, `027_if_case.jai` |
| Imports / scopes | `how_to/040_import_and_load/`, `080_scopes.jai`, `151_file_and_global_scopes/` |
| using | `how_to/042_using.jai`, `044_using_advanced/` |
| Operator overloading | `how_to/093_operator_overloading.jai`, `094_array_operators.jai` |
| Polymorphism | `how_to/100_`, `110_`, `115_`, `120_`, `160_type_restrictions.jai` |
| Context / `,,` | `how_to/011_context.jai`, `225_comma_comma.jai` |
| Metaprogramming | `how_to/450_basic_metaprogram/`, `600_insert.jai`, `630_compiler_get_nodes.jai` |
| Inline asm | `how_to/900_inline_assembly.jai` |
| Compiler API | `modules/Compiler/` |