Add macros and the rest of a professional RISC-V assembler

The RISC-V front end already encodes instructions and a handful of data directives. It does not assemble the files you actually write. Those are GNU as: .macro / .endm, \name, numbered local labels, .option, .pushsection. r-test/fp.S is the concrete target.

This document tells you how to add that, in order. Do not expand macros inside the ANTLR grammar. The encoder already advances address in parser actions; a .macro body that is a lines rule will emit bytes at definition time, or fight the address, or both.

The one-line version

Parse once to collect directives, expand macros / conditionals / includes into plain source, then parse again to encode. New language features that invent text belong in a preprocessor. New language features that invent bytes (.align, .globl, .option) belong in the grammar and RISCVEncoder.

source .S  →  preprocessor  →  ANTLR parse + encode  →  bin / ELF

Dialect for everything new is GNU as. Keep the NASM %define / %include that already work so testbench/define.s does not break. Do not add a second full NASM %macro unless a later day needs it.

What already exists

Do not reinvent these. They are incomplete, not absent.

DefineListener.map and DefineListener.lines are static. A second file in the same JVM inherits the first file's defines. Make them instance state when you touch that class.

The grammar rule named macro is not a macro. It is a bucket for define, ifdef, include, and data:

macro : define | ifdef | include | dotbyte | dothalf | dotword | dotdword | dotstring ;

Leave that name alone until the preprocessor owns define/ifdef/include. Then the rule can shrink to data directives.

Why the preprocessor, not the grammar

RISCVAssemblerParser members hold address and every instruction does address+=encoder.encodeType...(...). A .macro ADD rd, rs whose body is parsed as lines will encode add while you are still defining ADD. The invocation then has nothing to expand, or encodes a second time.

preProcess already does the right shape: walk the text, produce new text, then the real assemble() encodes. Grow that into a package hk.quantr.assembler.riscv.preprocess rather than piling more static maps into AssemblerLib.

Assembler.main must call the preprocessor before the encode parse. Today preProcess is used from tests (TestMacro, TestMissingMacro) and is easy to skip from the CLI path. Grep preProcess( and make every assemble entry go through it.

Files you will touch

src/main/java/hk/quantr/assembler/antlr/RISCVAssemblerLexer.g4
src/main/java/hk/quantr/assembler/antlr/RISCVAssemblerParser.g4
src/main/java/hk/quantr/assembler/AssemblerLib.java
src/main/java/hk/quantr/assembler/Assembler.java
src/main/java/hk/quantr/assembler/riscv/listener/DefineListener.java
src/main/java/hk/quantr/assembler/riscv/listener/IncludeListener.java
src/main/java/hk/quantr/assembler/riscv/RISCVEncoder.java
src/main/java/hk/quantr/assembler/riscv/preprocess/   (new)
src/test/java/hk/quantr/assembler/riscv/TestGasMacro.java  (new)

After any .g4 change: mvn -DskipTests compile so ANTLR regenerates into target/generated-sources/antlr4.

Phase 1: real macros

This is the feature the request named. Do it first and stop to test.

Tokens

In RISCVAssemblerLexer.g4, next to DEFINE / INCLUDE:

DOTMACRO    :   '.macro';
DOTENDM     :   '.endm';
DOTEXITM    :   '.exitm';

Do not add a parser rule that treats the body as lines. The preprocessor reads these as lines of text.

Data

A definition is a name, a list of formal parameters, and the raw body (lines between .macro and .endm, not encoded). An invocation is a name used as an opcode with comma-separated arguments.

GNU as substitution:

GAS also allows .macro NAME arg1=default. Defaults can wait until the required-argument form works.

Algorithm

1. Walk the source line by line. # and ; comments strip for this walk, but keep the original line text for error reporting.
2. On .macro NAME [formals...], slurp until .endm. Nested .macro inside a body is stored, not executed. Missing .endm is an error that names the opening line.
3. On a line whose first identifier is a defined macro, split arguments on commas that are not inside (...) or quotes, bind formals, substitute, splice the expansion in place of the invocation, and re-scan from there so macros can call macros.
4. Recursion depth: cap at something like 100. The error must name the invocation site and the definition site.
5. Unknown arity: error, do not encode a truncated body.

Invocations look like instructions (SECTION name, T fld, LDD fs0, d_one). The encode parse must not see those names. After expansion, SECTION is gone and the body (la, call, ...) remains.

What not to put in the grammar

Do not add NAME args as a generic instruction alternative. That would steal real opcodes (add, ld) if someone names a macro after one. Expansion happens first; the parser only ever sees real instructions.

First golden test

r-test/fp.S macros, smallest useful subset:

	.macro	E
	la	a3, 7b
	call	test_end
	.endm

	.macro	LDD freg, sym
	la	a0, \sym
	c.fld	\freg, 0(a0)
	.endm

A file that does LDD fs0, scratch then E must expand to la / c.fld / la / call. Compare bytes to riscv64-elf-as (phase 1 can ignore .option and .pushsection by not using those lines yet).

TestGasMacro outline:

String src = Files.readString(Path.of("src/test/resources/macro_ldd.s"));
String expanded = Preprocessor.expand(src, "rv64");
byte[] ours = assembleRv64(expanded);
byte[] gas = gasAssemble(src);   // riscv64-elf-as -march=rv64imafdc
assertArrayEquals(gas, ours);

gasAssemble writes a temp .s, runs riscv64-elf-as -o t.o, then riscv64-elf-objcopy -O binary t.o t.bin (or read .text from the ELF). Same idea as FullTest versus gas.

Phase 1 is done when that test passes and testbench/define.s still works.

Phase 2: conditionals and repeats

%ifdef is already in the grammar and does nothing. Evaluating it in the parser would still encode the false branch, because both lines alternatives are walked. Evaluate in the preprocessor, then delete the ifdef parser rule or leave it as a no-op that never fires on expanded text.

Add GAS forms. These are what fp.S does not use yet but every real tree has:

.if  expr
.ifdef name
.ifndef name
.else
.endif
.rept N
.endr

.if uses the same expression evaluator as immediates (CalculatorLibrary.cal). A name in .ifdef is defined if it is in the %define / .equ table or is a .macro.

.rept N splices the body N times, then re-scans. That is also the correct implementation of times / %times (the empty branch in preProcess).

.irp reg, t0, t1, t2 can wait until .rept works. It is the same loop with a formal rebound each iteration.

False branches must not define macros and must not invoke them. Skip lines until the matching .else / .endif, tracking nest depth. A dangling .endif is an error.

Phase 3: symbols and local labels

Without this, expanded fp.S still will not assemble.

.equ / .set

Same table as %define. .set may redefine; %define today forbids it and exits. Pick one policy and document it: .set overwrites, %define of an existing name is an error (keep today's behaviour).

Lexer: DOTEQU : '.equ'; DOTSET : '.set';

The encode parse needs these names in immediates. Either the preprocessor substitutes them (like %define already does with replaceAll("\\b"+name+"\\b")) or CalculatorLibrary consults the table. Substitution is simpler and matches the current %define path.

Numbered local labels

GNU as: 1: through 19: (you only need 0–9 to start), referenced as 1b (nearest backward) and 1f (nearest forward). fp.S uses 7: / 7b and 9: / 9b.

This cannot be only a preprocessor rewrite of the current file, because 1f depends on a label that appears later. Do it in the encoder:

1. Lexer: allow label to be [0-9]+ COLON, and immediates / jump targets to be [0-9]+ [bf].
2. First pass, or a collected list: each N: at address A is pushed on a per-digit list.
3. When encoding jal / beq / la that uses 7b, take the last 7: whose address is <= current address; 7f takes the next one after.

If you stay strictly one-pass, 7f is unknown when you see it. The encoder already has labels on the parser (ArrayList<Label>). Either two-pass (walk once for labels, once to encode) or record a fixup and patch the immediate when the forward label is defined. Two-pass is less clever and matches how gas works. The current address+= one-pass is why forward regular labels are already shaky; this is the moment to make a label pass explicit if you have to fight it.

# comments

LINE_COMMENT : [;#] ~[\r\n]* ;

GAS also treats /* */ as comments. Not required for fp.S. # is.

Phase 4: sections and options

Needed for a real object, and for fp.S after macros expand.

Sections

Replace the catch-all

section : DOT sectionName=IDENTIFIER IDENTIFIER? ;

with explicit directives so .macro / .equ / .option are not eaten as section names (today . plus an identifier is a section).

.text
.data
.rodata
.bss
.align  N
.globl  name
.asciz  "string"
.ascii  "string"
.pushsection name
.popsection

.byte already exists. Add .asciz (NUL-terminated) distinct from .string if .string stays identifier-only; or teach .string to take DOUBLE_QUOTATION ... DOUBLE_QUOTATION. fp.S uses .asciz.

RISCVEncoder must keep a current section and a stack for .pushsection / .popsection. Bytes go into that section's buffer, not one flat out. ELF output (-f elf) already exists; point it at those buffers instead of a single blob. bin output can concatenate .text then .data the way a raw image expects, or refuse and require -f elf. Pick one and test it.

.align N on RISC-V gas is power-of-two (.align 3 means 8 bytes). Pad with zeros or nop/c.nop in .text. Wrong interpretation here will fail every gas comparison.

.option

.option rvc
.option norvc
.option push
.option pop

LDD in fp.S wraps c.fld in .option push / rvc / pop. Without this, either you always accept compressed (current behaviour) or you reject c.fld when someone writes .option norvc. Store a stack of flags on the encoder. encodeType* for compressed opcodes checks rvcEnabled.

Phase 5: professional tooling

Do this after the language works. None of it changes what bytes mean.

• Include path. preProcessIncludeFile opens "testbench/" + to. Resolve against the including file's directory, then each -I from Assembler.main. Add the CLI option next to -a / -o.
• Listing of expansions. -l already writes listing. After preprocess, each encoded line should carry the invocation that produced it, so a failure in an expanded T fld names T and the .macro T line.
• Errors. Invocation site plus definition site, for macros, includes, and missing .endm. Stop calling System.exit from DefineListener / preProcess; throw or record on MessageHandler and let main set the exit code. Tests cannot survive an exit.
• Instance state. DefineListener.map and friends must not be static.

How to test, every phase

Always compare to GNU as. That is the project's existing contract (FullTest / gas vs quantr).

# gas
riscv64-elf-as -march=rv64imafdc -o gas.o t.s
riscv64-elf-objcopy -O binary --only-section=.text gas.o gas.bin

# ours, after you wire preProcess into main
java -jar target/assembler-*-jar-with-dependencies.jar -a rv64 -f bin -o ours.bin t.s

cmp gas.bin ours.bin

Keep a table of fixtures under src/test/resources/riscv/:

testbench/define.s and testbench/macro.s stay green.

Acceptance

Phase 1–4 together: this jar assembles r-test/fp.S (or a copy with the # comments and .equ values it already has) and the .text bytes match riscv64-elf-as -march=rv64imafdc. That file uses every phase: macros with arguments, # comments, 7: / 7b, .option push/rvc/pop, .pushsection, .asciz.

Until that works, do not start ia32 macros, a C preprocessor, or a relocating linker. Those are different jobs.

Out of scope

• Rewriting the encoder into a full relocating linker (relocs, -shared)
• ia32 / NASM %macro / %endmacro
• Running cpp for #include / #ifdef / #define (GAS # is a comment, not cpp, unless you pass -x assembler-with-cpp)
• Changing instruction encodings or the disassembler

2026/08/30 by admin