# PBXA64

PBXA64 is a C11-based x86-64 compiler prototype targeting Windows COFF objects and PE32+ console executables. The repository contains a generic MGC frontend/core, an x64 backend, assembler, COFF/PE writer, linker scaffolding, PILOT frontend work, and a dedicated early PowerBASIC frontend path.

## Current state

This tree is still a **prototype**, not a complete PowerBASIC-compatible compiler. The important change in this package is the **B24 build/optimizer pass**: release identity is updated to `0.65.0-B24`, the source-release layout is restored to a buildable CMake tree, normal optimizer runs avoid unnecessary pass-counter scans, and the enabled CTest lane is green at `107 / 107` on the Linux validation host. B19/B20 hardening is retained: runtime trap stubs, known WinAPI import synthesis, malformed-assembler hard errors, optimizer pipeline controls, pass-manager/MemorySSA scaffolding, and runtime-effect metadata.

Implemented in this package:

B24 build/optimizer highlights:

- Consistent `0.65.0-B24` version/release identity.
- Restored the release archive from a flat B23 file set into a single rooted source tree with physical `src/`, `include/`, `tests/`, `docs/`, `tools/`, and `samples/` paths.
- Full enabled validation lane: `107 / 107` CTest tests passed in the reconstructed B24 tree.
- Added `PBXA64_ENABLE_COMPILER_CACHE` for automatic `sccache`/`ccache` compiler-launcher use when available.
- Added `fast-dev` and `release-fast` presets for shorter local iteration builds.
- Replaced the stale hand-written unity source with CMake-managed unity groups for safe source clusters.
- Reduced optimizer overhead by collecting expensive per-pass instruction/change counters only when optimizer stats are requested.
- Fixed legacy PB codegen/assembler/linker issues surfaced by the tests: `jz`/`jnz`, standalone `rep` prefixes, `movsb`, `stosq`, `leave`, explicit `shr` counts, default WinAPI import hints, and a bad `__pb_memcmp` runtime call.
- Added POSIX host-build fallbacks for file and network runtime shims while preserving Win32/WinSock behavior on Windows.

Retained B19 hardening highlights:

- Linker default-import resolver for known WinAPI symbols, with `--nodefaultlibs` / `/NODEFAULTLIB` opt-out.
- Deterministic trap stubs for incomplete runtime helpers instead of silent zero-return behavior.
- Runtime-builtin effect/foldability/availability metadata for optimizer and diagnostic consumers.
- Optimizer pass-scope metadata, preserved-analysis masks, pass timing, and text pipeline parsing.
- First-generation pass-manager and MemorySSA-lite APIs.
- Assembler hard error for malformed `mov`, plus `.zero` and `.p2align` directive support.

- Canonical CMake build with `mgc64_core`, `mgc64`, and `pbxa64` targets.
- CMake presets for debug, release, and ASAN/UBSAN builds.
- CTest integration for PB lexer, PB parser/semantic smoke tests, optimizer tests, assembler/linker tests, MGC e2e tests, PB e2e tests, and direct PILOT e2e link tests.
- `.pb`, `.bas`, `.pbi`, `.pilot`, `.pil`, and `.pcode` input routing through the PB/PILOT frontend path.
- PB lexer handling for newlines, comments, suffix/operator context, `&H`, `&O`, `&B`, `##`, `$$`, and line continuation.
- PB parser and AST for a useful early console subset:
  - `#COMPILE`, `#DIM ALL`, `#INCLUDE` preprocessing,
  - `DECLARE FUNCTION/SUB ... LIB ... ALIAS ...`,
  - multiline DECLARE parameter lists with `_` continuation,
  - `FUNCTION PBMAIN() AS LONG`,
  - `LOCAL` / `DIM` / `STATIC`,
  - assignment including `FUNCTION = expr`,
  - integer literals and string literals,
  - arithmetic and comparison expressions,
  - `PRINT`,
  - `IF / ELSE / END IF`,
  - `FOR / NEXT`,
  - `WHILE / WEND`,
  - `CALL` and external function calls.
- PB semantic checks for `#DIM ALL`, duplicate locals, arrays requiring indices, string/numeric assignment compatibility, external call presence, argument counts, return target handling, and basic type categories.
- PB string/WSTRING support for this milestone:
  - local `STRING` variables backed by fixed local ASCIIZ buffers,
  - string literal assignment,
  - string-to-string assignment,
  - string concatenation with `+`,
  - `LEN(expr)`,
  - `STRPTR(expr)`,
  - string equality/inequality and relational comparison,
  - scalar `WSTRING` locals/statics/globals,
  - WSTRING assignment from narrow literals/STRING values and WSTRING-to-STRING narrowing for the current output path,
  - WSTRING `LEN`, `PRINT`, `STRPTR`, comparison, concatenation, and scalar WSTRING function parameters/returns,
  - dynamic `STRING` and `WSTRING` fields inside `TYPE` / UDT values, including scalar locals, fixed UDT arrays, dynamic UDT arrays, indexed UDT-array field access, whole-UDT assignment, and `REDIM PRESERVE` deep-copy handling.
- PB dynamic `STRING`/`WSTRING` fields inside UDTs for this milestone:
  - dynamic descriptor-field initialization for stack-local UDTs, fixed UDT arrays, and dynamic UDT arrays after `REDIM`,
  - deep-copy assignment for UDTs containing `STRING` / `WSTRING` fields,
  - recursive field cleanup for non-main routines and old dynamic UDT-array blocks,
  - `REDIM PRESERVE` deep-copy handling for supported dynamic UDT arrays containing descriptor-string fields.
- PB numeric arrays:
  - fixed arrays: `LOCAL a(upper) AS LONG`, `LOCAL a(lower TO upper) AS LONG`,
  - dynamic-style stack-backed arrays: `LOCAL a() AS LONG` + `REDIM a(lower TO upper) AS LONG`,
  - `LBOUND(a)` and `UBOUND(a)`,
  - element assignment `a(i) = expr`,
  - element reads `expr = a(i)`.
- PB file I/O subset:
  - `OPEN "file" FOR OUTPUT AS #n`,
  - `OPEN ... FOR APPEND AS #n` with seek-to-end lowering,
  - `OPEN ... FOR INPUT AS #n`,
  - `PRINT #n, expr`,
  - `INPUT #n, var`,
  - CR/LF-aware `LINE INPUT #n, var`,
  - `GET #n, pos, target`, `PUT #n, pos, expr`, `SEEK #n, pos`, `LOF(n)`, `EOF(n)`,
  - `CLOSE #n`.
- BYREF/VARPTR support for numeric locals, string locals, and array elements in the supported subset.
- Direct PB x64 assembly emission for the subset above.
- PB console output runtime path via imported WinAPI calls: `GetStdHandle`, `WriteFile`, `ExitProcess`, and `wsprintfA`.
- String helper imports: `lstrlenA` and `lstrcpynA`; STRING comparison/search lowering now uses `memcmp` with explicit lengths.
- External call lowering that stages first four Windows x64 arguments into registers and additional arguments onto stack slots.
- PB `DECLARE ... ALIAS "#ordinal"` import-by-ordinal support; the linker emits ordinal import thunks while generated assembly relocates against the PB public declare name.
- PB-generated PE32+ console executables with import tables for the runtime-visible functions and declared external symbols.
- PB-generated PE32+ DLLs with `IMAGE_FILE_DLL`, minimal `__pb_dll_main`, and `.edata` export tables for `EXPORT` routines.
- Multiple `#COMPILE EXE` / `#COMPILE DLL` directives are preserved and each output path is produced during link.
- Embedded PILOT `#PCODE` lowering for `T:`, `A:`, `M:`, `Y:`, `N:`, `C:`, `J:`, `U:`, `E:`, `R:`, `S:`, `D:`, `GR:`, `SO:`, and `*LABEL` forms.
- PILOT text, numeric, wildcard, and comma-list matching lowered through the PB string/numeric subset, including `$MATCH`, `$LEFT`, `$RIGHT`, and `#POS` state variable.
- Raw `#PCODE`/`#ENDP` PILOT block extraction with preserved source bytes, plus V11 lowering for conditional suffix commands (`TY:`, `TN:`, `JY:`, `JN:`, `UY:`), parenthesized command conditions such as `T(#score>=0):`, and `U:*LABEL` subroutine call/`E:` return semantics.
- C11-portable PILOT parser string duplication and explicit AST destruction.
- Fixed compiler/linker object lifetime so the linker no longer retains pointers to stack-scoped compilation units.
- Working `-S` mode that emits assembly instead of reporting success silently.
- Real CFG lowering for `if`, `while`, `for`, `break`, and `continue` in the MGC IR builder.

## B24 additions

```text
release identity: 0.65.0-B24
validation: 107 / 107 enabled CTest tests passed
build: restored source tree, fast-dev/release-fast presets, compiler cache option, CMake-managed unity groups
optimizer: pass-counter IR scans gated behind print_stats
assembler: jz/jnz aliases, standalone rep prefixes, movsb/stosq, leave, explicit shr count
linker/runtime: expanded default WinAPI import hints and corrected __imp_memcmp runtime call
host portability: POSIX file and socket fallbacks for non-Windows validation builds
```

## V26 additions

```text
ON ERROR / ERROR / ERR / ERL / ERRCLEAR / RESUME
DATA / READ / RESTORE
ARRAY ASSIGN / REVERSE / SORT / COPY / SCAN
FILECOPY / RMDIR / GETATTR / SETATTR / RESET / FLUSH / SETEOF / REMAIN$
MIN / MAX / IIF / CHOOSE / SWITCH / ROUND
BUILD$ / CLIP$ / RETAIN$ / SHRINK$ / FORMAT$
PRINT USING syntax, LIKE, #ASM blocks, LINE graphics scaffold
WSTRING helper routing expanded for the V24/V25 string surface
```

## Build

```sh
cmake --preset fast-dev
cmake --build --preset fast-dev -j2

# full test lane
cmake -S . -B build/tests-debug -G Ninja -DCMAKE_BUILD_TYPE=Debug -DPBXA64_BUILD_TESTS=ON
cmake --build build/tests-debug -j2
ctest --test-dir build/tests-debug --output-on-failure
```

Sanitizer build:

```sh
cmake --preset asan-ubsan
cmake --build --preset asan-ubsan -j2
ASAN_OPTIONS=detect_stack_use_after_return=1:halt_on_error=1:detect_leaks=1 ctest --test-dir build/asan-ubsan --output-on-failure
```

## Usage examples

Emit assembly from PB:

```sh
build/debug/pbxa64 -S tests/e2e/examples_pb/real_usable.pb -o real_usable.asm
```

Compile/link a PB PE32+ console executable:

```sh
build/debug/pbxa64 tests/e2e/examples_pb/real_usable.pb -o real_usable.exe
```

Compile an include-based PB sample:

```sh
build/debug/pbxa64 -S tests/e2e/examples_pb/with_include.pb -o with_include.asm
```

Compile a standalone PILOT file through the V11 direct PILOT path:

```sh
build/debug/pbxa64 examples/pilot/v11_core_demo.pilot -o v11_core_demo.exe
```

Compile the V13 PB parity sample:

```sh
build/debug/pbxa64 tests/e2e/examples_pb/v13_pb_parity.pb -o v13_pb_parity.exe
```

Compile the V14 scalar WSTRING sample:

```sh
build/debug/pbxa64 tests/e2e/examples_pb/v14_wstring_scalar.pb -o v14_wstring_scalar.exe
```

Compile the V15 dynamic UDT STRING/WSTRING sample:

```sh
build/debug/pbxa64 tests/e2e/examples_pb/v15_udt_dynamic_strings.pb -o v15_udt_dynamic_strings.exe
```

Compile the V18 Unicode input sample:

```sh
build/debug/pbxa64 tests/e2e/examples_pb/v18_unicode_input.pb -o v18_unicode_input.exe
```

Compile the V17 WSTRING I/O/ABI sample:

```sh
build/debug/pbxa64 tests/e2e/examples_pb/v17_wstring_io_abi.pb -o v17_wstring_io_abi.exe
```

Compile the V16 WSTRING WinAPI/file-I/O sample:

```sh
build/debug/pbxa64 tests/e2e/examples_pb/v16_wstring_winapi_fileio.pb -o v16_wstring_winapi_fileio.exe
```


Compile the V13 restored embedded/direct PILOT samples:

```sh
build/debug/pbxa64 tests/e2e/examples_pb/v13_pilot_restored.pb -o v13_pilot_restored.exe
build/debug/pbxa64 examples/pilot/v13_restored_core.pilot -o v13_restored_core.exe
```

Emit assembly from MGC:

```sh
build/debug/pbxa64 -S examples/test_basic.mgc -o test_basic.asm
```

Dump PB tokens or AST:

```sh
build/debug/pbxa64 --dump-tokens tests/e2e/examples_pb/hello.pb
build/debug/pbxa64 --dump-ast tests/e2e/examples_pb/strings_arrays_winapi.pb
```

## Example PB subset program

```basic
#COMPILE EXE "strings_arrays_winapi.exe"
#DIM ALL

DECLARE FUNCTION MessageBoxA LIB "user32.dll" ALIAS "MessageBoxA" _
    (BYVAL hWnd AS QUAD, BYVAL lpText AS ASCIIZ PTR, BYVAL lpCaption AS ASCIIZ PTR, BYVAL uType AS LONG) AS LONG

FUNCTION PBMAIN() AS LONG
    LOCAL s$ AS STRING
    LOCAL t$ AS STRING
    LOCAL a(1 TO 3) AS LONG

    s$ = "Hello"
    t$ = s$ + ", PBXA64"
    a(1) = LEN(t$)
    a(2) = 42
    a(3) = a(1) + a(2)

    PRINT t$
    PRINT a(3)
    CALL MessageBoxA(0, STRPTR(t$), "PBXA64", 0)
    FUNCTION = 0
END FUNCTION
```

## Architecture

```text
src/
  common/      shared support types
  frontend/    MGC, PB, and PILOT lexing/parsing pieces
  middle/      semantic bridge and IR
  backend/     x64 selection, allocation, encoding
  assembler/   textual/machine assembly and COFF object output
  linker/      object aggregation and PE executable output
  runtime/     early CRT/builtin scaffolding
  driver/      CLI, compiler pipeline, output orchestration
```

## Known incomplete areas

The PB frontend is now useful for small console programs, string/array smoke tests, and basic WinAPI import experiments, but the project still needs substantial work before claiming broad PowerBASIC compatibility:

- Full dynamic PB string runtime: V18 adds UTF-8 redirected WSTRING input decoding plus UTF-16LE/BOM file-input decoding for WSTRING targets; V17 adds redirected WSTRING console output fallback, encoding-aware file metadata, and wider WSTRING built-in lowering; V15 adds recursive ownership/lifetime handling for dynamic string fields inside UDTs. Complete Unicode policy parity, full `STR$`/`VAL`/`FORMAT$` compatibility, and the rest of the historical string library remain incomplete.
- Full dynamic-array runtime: V15 covers one-dimensional dynamic UDT arrays with dynamic string fields and `REDIM PRESERVE` deep-copy lowering; multidimensional descriptors, arrays inside TYPE fields, and full PowerBASIC array parity remain incomplete.
- Full `TYPE` / `UNION` parity: the current implementation supports basic `TYPE`/`UNION` fields, numeric fields, dynamic/fixed STRING and WSTRING fields, nested TYPE fields, member access, and selected UDT-array paths, but still lacks complete packed/aligned Windows ABI structure layout, arrays in TYPE fields, complex structure parameters, and struct returns.
- Full file I/O: the current implementation supports `OPEN`/`PRINT #`/`INPUT #`/CR-LF-aware `LINE INPUT #`/`GET`/`PUT`/`SEEK`/`LOF`/`EOF`/`CLOSE` for the basic subset; random/binary mode parity, record layouts, file attributes, and richer error handling remain incomplete.
- Full `DECLARE` lowering: V17 adds a first external `VARARG` path and V16 supports external `BYVAL WSTRING` as `LPWSTR`, but pointer arithmetic, complex callbacks, richer type checking, and structure parameters remain incomplete.
- Full Windows x64 ABI hardening: unwind metadata, nonvolatile register handling across larger generated functions, struct returns, complete varargs coverage, and callback runtime execution coverage.
- PE/COFF completeness: V12+ emits `.bss` for supported uninitialized numeric storage, but resources, icons/version info, DLL output, `.pdata`/`.xdata`, and richer section/data handling remain incomplete.
- Diagnostics: precise source snippets/carets and snapshot tests for invalid PB programs.
- Optimizer passes behind `-O0..-O2`.
- Broad PowerBASIC compatibility fixture suite.

See `PATCH_NOTES_V18.md` for the V18 Unicode input decoding pass, `PATCH_NOTES_V17.md` for the V17 WSTRING I/O/ABI hardening pass, `PATCH_NOTES_V16.md` for the V16 WSTRING WinAPI/file-I/O hardening pass, `PATCH_NOTES_V15.md` for the V15 dynamic UDT string-field lifecycle expansion, `PATCH_NOTES_V14.md` for the V14 scalar-WSTRING compiler expansion, `PATCH_NOTES_V13.md` for the V13 PB/PILOT parity expansion, `PATCH_NOTES_V12.md` for the V12 PB runtime/compiler expansion, `PATCH_NOTES_V11.md` for the V11 PILOT core-semantics pass, `PATCH_NOTES_V10.md` for the V10 PILOT lowering milestone, and `PATCH_NOTES_V9.md` for the String64 corrective pass.


## V19 file-I/O runtime additions

V19 adds encoding-aware `LINE INPUT #` into `STRING` targets, baseline `OPEN ... FOR RANDOM AS #n LEN = expr`, RANDOM-aware `GET`/`PUT`/`SEEK`/`LOC`, packed-size numeric BINARY `GET`/`PUT`, and Wine-backed runtime harness scripts. PILOT, scalar WSTRING, WSTRING WinAPI/file-I/O, Unicode WSTRING input, and dynamic STRING/WSTRING UDT lifecycle regression guards remain active.

## V20 dynamic UDT file serialization additions

V20 removes the V19 `GET #` / `PUT #` rejection for `TYPE` values containing dynamic `STRING` / `WSTRING` fields. Such records now use a descriptor-safe serializer: scalar/fixed fields are written as field bytes, while dynamic descriptor strings are written as `QUAD length` plus payload bytes and reconstructed on `GET #`. Numeric `INPUT #` also uses the encoding-aware line decoder before `_atoi64` / `atof`, so UTF-16LE/AUTO numeric input shares the Unicode file-I/O path.


## V21 core commands and preprocessor additions

V21 adds a dedicated PB preprocessor pass before parsing. It supports `#IF/#ELSEIF/#ELSE/#ENDIF`, `$IF/$ELSEIF/$ELSE/$ENDIF`, `#DEFINE/#UNDEF`, `$DEFINE/$UNDEF`, `%DEF(name)`, and simple `NOT`/`AND`/`OR` directive expressions.

Core PB command coverage was expanded and regression-tested for `FREEFILE`, `HEX$`, `RND/RANDOMIZE`, `SIN/COS/SQR`, `DATE$/TIME$/TIMER`, `SLEEP`, `SWAP`, `INCR/DECR`, `WRITE #`, `KILL/NAME/MKDIR/CHDIR/DIR$`, `FUNCPTR`, `^`, `&`, `XOR`, `EQV`, and `IMP`. Existing V20 file serialization, WSTRING, UDT dynamic-string lifecycle, and PILOT guards remain active.

V21 also introduces the `PbMacroExpansionContext` / `PbMacroDefinition` scaffold that V22 can extend into a real `MACRO` / `MACROFUNCTION` expansion preprocessor.


## V22 macro-preprocessor release

V22 adds real macro expansion (`#DEFINE` replacements, single-line and multi-line `MACRO`, first-pass `MACROFUNCTION` with `MACROTEMP`), fixes double preprocessing, and adds correct `#INCLUDE ONCE` handling. See `PATCH_NOTES_V22.md`. V23 implements `#COMPILE DLL`; GUI/WINDOWS output remains explicitly unsupported.


## V23 DLL-output release

V23 adds `#COMPILE DLL`, exported `FUNCTION`/`SUB` routines, `.edata` export-table generation, DLL PE header flags, and multi-output fan-out for multiple `#COMPILE` directives. See `PATCH_NOTES_V23.md` and `docs/DLL_OUTPUT_V23.md`.


## V24 string-completion release

V24 adds a broad string-helper pass on top of V23: `REPEAT$`, `STRDELETE$`, `STRINSERT$`, `STRREVERSE$`, `BIN$`, `OCT$`, `LTRIM$`, `RTRIM$`, `REPLACE$`, `TALLY`, `VERIFY`, `SCAN$`, `COMMAND$`, `ENVIRON$`, `CURDIR$`, `TAB$`, `PEEK$`, `INPUT$`, `INKEY$`, and `WAITKEY$`. See `PATCH_NOTES_V24.md`. V23 DLL output and multi-output fan-out are preserved.


## V24.1 string/memory command extension

V24.1 adds `POKE`, `POKE$`, numeric `PEEK`, `INPUT$(n, #file)`, `NUL$`, `DEC$`, `MKI$`/`MKL$`/`MKQ$`/`MKS$`/`MKD$`, `CVI`/`CVL`/`CVQ`/`CVS`/`CVD`, `JOIN$`, `SPLIT ... INTO`, and `MID$` assignment. See `PATCH_NOTES_V24_1.md`.


## V25 Memory/String Command Pass

V25 adds POKE/POKE$, numeric PEEK, INPUT$(n,#file), NUL$/DEC$, binary MK*/CV* helpers, JOIN$, SPLIT ... INTO, and MID$ assignment.
