Compile-Time Metaprogramming

On this page

Metaprogramming allows programs to participate in generating or transforming other programs. Compile-time metaprogramming shifts some work to the build phase, but different mechanisms operate on different objects: some handle preprocessor tokens, some handle syntax data, some compute constants, and others modify bytecode.

When understanding these concepts, start by asking three questions: Does the metaprogram receive syntax or checked values? At what stage does it execute? Is the artifact code, data, or additional behavior?

First distinguish input and artifact: generate syntax, evaluate, weave behavior Token / Syntax Data Macro Expansion New syntax → checks Constant Expressions Constant evaluation Values / Static Data Program + Matching Rules Weaving Program + new behavior Transformations occur at build time, but this does not mean the generated behavior or data has no runtime cost.

Why Regular Functions Aren't Enough

Regular functions are suitable for abstracting runtime calculations; they typically do not need to know the names of the caller's fields and cannot arbitrarily generate new declarations. When the requirement is "generate serialization code for each field of a struct" or "construct a new type based on type parameters," capabilities beyond regular function calls are needed.

However, metaprogramming increases expansion, diagnostic, and build complexity. If regular functions, generics, or composition can clearly express the requirement, they are usually easier to debug. The choice should be based on what information needs to be manipulated, not on which mechanism looks more powerful.

C Macros: Preprocessor Token Expansion

The C preprocessor replaces according to preprocessor token rules; it is not arbitrary character replacement, nor does it understand the types of subsequent complete expressions. It supports uses such as conditional compilation, stringification, and token pasting.

The x, y, and z statements below belong inside a function body; this is a local macro-expansion example.

#define PICK_MAX(a, b) ((a) > (b) ? (a) : (b))

int x = 3;
int y = 2;
int z = PICK_MAX(x++, y); /* x is evaluated twice, final x=5, z=4 */

Parentheses solve some operator precedence issues, but not repeated evaluation: the comparison uses x++ once, and the selected branch executes x++ again. If arguments contain function calls, side effects may also be duplicated. For fixed-type comparisons, regular inline functions can provide single-parameter evaluation and type checking. See GCC Macro Pitfalls for more expansion boundaries.

Compare "Expanded Twice" vs. "Argument Evaluated Once" at the Same Call Site

Save the following as evaluate-once.c. The two tests start from x=3; the macro results in x=5, result 4; the function results in x=4, result 3. This example does not modify x multiple times in the same unsorted expression, avoiding undefined behavior to explain repeated evaluation.

#include <stdio.h>
#define PICK_MAX(a, b) ((a) > (b) ? (a) : (b))
static inline int max_int(int a, int b) { return a > b ? a : b; }
int main(void) {
    int x = 3, y = 2;
    int by_macro = PICK_MAX(x++, y);
    if (x != 5 || by_macro != 4) return 1;
    x = 3;
    int by_function = max_int(x++, y);
    if (x != 4 || by_function != 3) return 1;
    printf("macro=%d function=%d\n", by_macro, by_function);
    return 0;
}

Execute cc -Wall -Wextra evaluate-once.c -o evaluate-once, output macro=4 function=3. Using cc -E to view expansion also shows where x++ appears in the two branches. Inline optimization can eliminate function call overhead, but function arguments are still evaluated according to language semantics and do not become macro-style repeated evaluation. GCC Side Effect Duplication

Single evaluation does not equal a fixed evaluation order between multiple arguments. If the second argument also reads/writes x, you must still comply with C's evaluation order rules; if a clear sequence is needed, calculate temporary values with independent statements first, then call the function. Replacing this fixed-type macro with a function solves the argument copying problem, not automatically sorting all expressions.

Why can x increment twice despite correct parentheses?

Track each x++ read and its old/new values, not only the result. When initial x>y, comparison reads once and the selected branch reads again; when x≤y that branch does not execute. The function evaluates one argument value then reuses it. The contrast respects C conditional sequencing rather than exploiting undefined behavior.

Preparing the visual
Why can x increment twice despite correct parentheses?

The macro copies unevaluated expressions; a selected x++ branch executes again after comparison. The function evaluates the argument once and reuses its value. Unselected branches do not execute.

Lisp Macros: Input is Unevaluated Code Forms

Lisp code forms can be operated on as data, such as lists. Macros receive forms that are not evaluated according to regular function argument rules, produce new code forms, and then the language processes the expansion result.

(defmacro unless-demo (condition &body body)
  `(if (not ,condition) (progn ,@body)))

Here condition and body are placed into the generated if form; the runtime actions they express are not executed during macro expansion. Quasi-quotation, backquote, and concatenation help construct code, but macro authors still need to consider variable capture, evaluation count, expansion environment, and error locations. Code being data does not mean scope and semantic issues disappear.

Rust Procedural Macros: TokenStream is Not a Typed AST

Rust procedural macros consume and produce TokenStream at compile time. Macros can parse into syntax structures using libraries like syn, but this does not mean obtaining the internal AST after rustc completes type inference, nor can it generally query the type deduced for a specific expression.

Procedural macros are mainly divided into three types: function-like, derive, and attribute, with different insertion or replacement rules for expansion. For example, derive generates additional entries, often used to generate impl; attribute can replace the annotated entry. Interfaces and rules are in the Rust Reference.

Suppose generating a serialization implementation for struct Record<T> { value: T }; the macro must preserve generic parameters and where constraints, generate valid paths and bounds, and then subsequent compilation stages check if the implementation is type-correct. "The macro knows field syntax" is different from "The macro proves field types satisfy all constraints."

Hygiene: Who Generated Names Point To After Expansion

Hygiene solves the accidental capture of generated names and caller names. Rust macro_rules! has its mixed-position hygiene rules; procedural macros cannot be generalized as "naturally fully hygienic." Procedural macro authors need to handle name conflicts and path resolution, such as using appropriate absolute paths, preserving spans, and avoiding generating auxiliary names that are prone to conflicts.

Hygiene does not automatically guarantee single evaluation, correct borrowing, or good error messages. These properties of macro-generated code still need to be designed separately.

Can generated tmp capture the caller’s tmp?

Two tmp spellings may originate in different declaration environments. Caller C and generated M stay visible, with separate spliced and helper uses. Disabling hygiene captures only when the caller also uses tmp; renaming avoids this instance, not all macro collisions. Hygiene maintains binding provenance rather than merely choosing an unusual name.

Preparing the visual
Can generated tmp capture the caller’s tmp?

Hygiene preserves the spliced expression’s binding while generated uses refer to the helper declaration. Nearest-name lookup on expanded text can capture the caller variable.

Compile-Time Evaluation: Computing a Value

Rust's const fn can execute in constant contexts that meet conditions; it can also execute at runtime at regular call sites. You cannot assume every call happens at compile time just based on the const in the function declaration.

const fn square_table() -> [u32; 4] {
    let mut out = [0; 4];
    let mut i = 0;
    while i < 4 {
        out[i] = (i * i) as u32;
        i += 1;
    }
    out
}

const SQUARES: [u32; 4] = square_table();

fn main() {
    assert_eq!(SQUARES, [0, 1, 4, 9]);
}

Constant contexts require obtaining a valid constant result; the language restricts executable operations. Array out-of-bounds become compile errors when evaluation is mandatory; allowed syntax and reference behaviors are specified by language and version. Rust Constant Evaluation

C++ constexpr, consteval, and Zig comptime have different specific capabilities and should not be uniformly described as the same interpreter. Do not confuse const evaluation with the permissions of build scripts or procedural macros: the latter two may access the build environment and file system.

Pre-computation shifts computation time to the build phase and may place results into artifacts. It may also increase binary size, compile memory, or cache pressure; "compile-time execution" does not mean all costs are zero.

Same const fn, Call Context Determines Which Stage Exposes Errors

Below, define the maximum valid payload as 1024, with a fixed header occupying 8 bytes. The initialization of constant PACKET_SIZE must be evaluated at compile time; the length from args in main belongs to regular runtime input.

const fn packet_size(payload: usize) -> usize {
    assert!(payload <= 1024, "payload too large");
    payload + 8
}

const PACKET_SIZE: usize = packet_size(16);

fn main() {
    assert_eq!(PACKET_SIZE, 24);
    let runtime_input = std::env::args().count();
    assert_eq!(packet_size(runtime_input), runtime_input + 8);
}

Compiling directly and running without extra arguments will pass. If the constant initialization is changed to packet_size(1025), the compiler must reject this constant; if 1025 is only passed in the regular path of main, the program will panic when executing the assertion. The const fn body does not become two different logics; what changes is whether the current context requires a valid constant result. Constant Context

Usage LocationStage ObligationCannot Infer
const initializationMust complete under constant evaluation rulesData has no artifact space cost
Regular runtime callProduces value or failure according to regular execution semanticsCompiler does not perform constant folding at all
Build script reading fileRuns independent program at build timeFile content automatically becomes a declared dependency
Procedural macro generating expressionProduces syntax to be compiledExpression has completed type checking

At which stage does the same const fn fail?

Separate phase obligations from optimization opportunities. Const initialization requires valid compile-time evaluation; an ordinary path has runtime behavior only when executed. The same packet_size body remains while context and input change. Even if an optimizer precomputes an ordinary call, it cannot indiscriminately replace its semantics with const-context diagnostics.

Preparing the visual
At which stage does the same const fn fail?

A const initializer must yield a valid constant. An ordinary call is not required constant evaluation merely because the function is const; an unexecuted call has no runtime panic.

Templates and Generics: Parameters Can Also Determine Program Structure

Generic type parameters participate in checking; some languages also support value parameters, type-level computation, or code instantiation based on parameters. Monomorphization allows different instances to obtain specialized implementations, but increases the number of instances and compilation costs. Type erasure and dictionary passing use different runtime representations and calling conventions.

These can be combined with macros: macros first generate implementations with generics, and the compiler then checks and generates code according to strategies. Calling all generics "macros" or calling all metaprogramming "AST replacement" obscures stage differences.

AOP Weaving: Select Join Points in Program and Add Behavior

AspectJ describes a set of join points via pointcut and specifies behavior to execute via advice. call and execution match different objects; you cannot uniformly draw method execution join points as inserting AST before the call point.

AspectJ supports source code compilation and weaving into existing class files; artifacts can be bytecode with woven behavior; there is also load-time weaving. It is not limited to "the compiler only modifies Java AST." AspectJ Compiler and Weaver

Early weaving can avoid certain proxy boundaries, but logging, timing, and checks in advice still execute at runtime, and dynamic join point conditions may also have costs. Whether self-calls are intercepted depends on the mechanism, matching rules, and weaving scope; you cannot use "covers all calls, zero runtime overhead" as a universal conclusion.

Reproducible generation requires all output-affecting inputs Explicit Inputs Schema + version + options Deterministic Generation Fixed field order and naming rules Generated Code Parse, type-check, compile Hidden inputs: directory, environment, time, traversal order Model any output-affecting input explicitly, or remove its influence. Generation success is not semantic correctness; test the generated interface after compilation.

Build Cache Must See the Generator's Real Dependencies

For example, generating a serialization implementation for Record from a schema; field order comes from the schema, naming rules from the generator version, and output mode from configuration switches. Listing only the generator source files as dependencies may lead to using old outputs after modifying the schema; listing only the schema as a dependency may fail to rebuild after upgrading naming rules.

Conversely, if the generator unconditionally adds the current time on every run, even if all business inputs remain unchanged, the generation result will change, filling caches and code reviews with noise. First determine if time belongs to the artifact's semantics; if it's just logging, leave it in the build log.

ChangeReasonable ExpectationNeed to Verify
New fieldCorresponding generation declaration changesWhether input dependencies cause the generator to rerun
Upgrade generator rulesSame schema generates according to new rulesWhether tool version enters the cache key
No business input changeGenerated content remains stableOrder, random numbers, and implicit environment
Generate illegal type combinationsCompilation diagnostics point to domain inputWhether source locations or generation manifests are preserved

Compile-time computation also has scale boundaries: if N generic instances each generate B bytes, code space may increase with the number of instances, but linker folding, optimization, and sharing strategies will change the final size. Look at artifacts and build resources; do not assume compile-time metaprogramming definitely saves total cost just because fewer lines of source code were written.

Engineering Review Quick Reference

RequirementMechanism to ConsiderCheck
Small amount of repeated calculationFunctions, GenericsIs it already sufficient to express
Generate implementation by fieldderive / Procedural MacrosGenerics, paths, error spans, checkability of generated code
Pre-generate constant tableConstant EvaluationAllowed operations, compilation cost, artifact size
Generate specialized structures by type and valueTemplates or Type-Level MechanismsInstance bloat, constraint diagnostics
Cross-cutting behaviorWeaving or ProxiesJoin point semantics, coverage, runtime side effects

Verification should not only look at normal expansion, but also at error inputs, same-name variables, parameters with side effects, generic boundaries, and whether incremental builds after modification invalidate correctness. Generated results should have traceable locations and stable order, avoiding implicit inputs from the build environment causing unreproducible artifacts.

References

Keywords: metaprogramming, macro, TokenStream, hygiene, const fn, constexpr, templates, AspectJ, weaving