---
title: DSL Design
url: https://doc.liz6.com/en/compilers/08-compilation-technology-applications/02-dsl-design
locale: en
area: compilers
tags:
- DSL
- domain-specific language
- internal DSL
- external DSL
- fluent API
- builder pattern
- operator overloading DSL
- expression template
- macro DSL
- proc macro
- transpile
- code generation
- embedded DSL
- language design
- Serde
- diesel
- compilers
- compilation-technology-applications
date: 2026-06-30
modified: 2026-07-16
description: Crafting refined vocabulary and syntax for a specific domain—from internal DSLs (extensions of the host syntax with free type checking) to external DSLs (independent parser + codegen). The choice depends on whether domain concepts can be expressed naturally in the host language.
---

# DSL Design

> Crafting refined vocabulary and syntax for a specific domain—from internal DSLs (extensions of the host syntax with free type checking) to external DSLs (independent parser + codegen). The choice depends on whether domain concepts can be expressed naturally in the host language.

## Overview

General-Purpose Languages (GPLs) can write anything, but expressing certain domains in a GPL creates significant noise. A Domain-Specific Language (DSL) crafts a refined vocabulary and syntax for a specific domain—SQL for data manipulation, regular expressions for text matching, Terraform for describing infrastructure. For compilation techniques, there are two fundamentally different implementation paths for DSLs: **Internal DSLs** (extensions of the host language's syntax, no new parser by the compiler) and **External DSLs** (independent syntax + parser + code generation). This article covers the mechanisms of both paths, criteria for selection, and the effort required for the compiler frontend when designing an external DSL from scratch.

## Internal DSLs: Extensions of the Host Language

Internal DSLs parasitize the host language—leveraging the host language's syntactic features (operator overloading, chained calls, macros, trailing closures) to make code read like a specially designed language. There is no new parser; compilation is handled by the host compiler.

### Fluent API (Chained Calls)

```rust
// A fluent DSL for building SQL queries
let results = query::select(&[name, age])
    .from(users)
    .where_(
        condition::eq(department, "Engineering")
            .and(condition::gt(salary, 50000)))
    .order_by(name, Desc)
    .execute(db)?;
```

Mechanism: Each method (`select`, `from`, `where_`, `order_by`) returns a `QueryBuilder`, carrying accumulated AST fragments; `.execute()` translates the AST into an SQL string and executes it. The host compiler performs type checking at compile time—if the field type in `where_` does not match the value type, the host compiler reports an error. **A core advantage of internal DSLs: type checking is free.**

### Operator Overloading DSL

Leverage operator overloading to make expressions look more mathematical:

```cpp
// C++ Eigen (Linear Algebra DSL)
MatrixXd A = B * C + D.transpose();
VectorXd x = A.lu().solve(b);
```

Under the hood, each `*`/`+`/`.transpose()` is a C++ function call, but it does not perform actual matrix multiplication—it returns an **expression template**, which is a compile-time AST. When the assignment `x = ...` occurs, the expression template is expanded into optimal loop code (no temporary matrix allocation at compile time). **Zero runtime overhead—this is the key distinction between operator overloading DSLs and "writing an interpreter".**

### Macro DSL

Rust's procedural macros allow translating custom syntax into the host language's AST at compile time:

```rust
#[derive(Serialize)]          ← DSL: Declares serialization rules
struct User {
    name: String,
    age: u32,
}
```

`#[derive(Serialize)]` expands at compile time into `impl Serialize for User`, generating field-by-field serialization code for the `serialize()` method. `serde` and `diesel` are two representative examples of Rust macro DSLs—`serde`'s DSL translates `struct` definitions into serialization logic, while `diesel`'s DSL maps schema definitions to database tables at compile time.

Advantages of Macro DSLs: **Syntax is fully expanded at compile time**—incorrect DSL usage (e.g., a field type that is not serializable) results in a compile-time error rather than a runtime error.

## External DSLs: Independent Language + Full Compilation Pipeline

When a domain's syntax cannot be embedded in the host language (e.g., regular expressions, SQL, Terraform HCL), an **external DSL** is required—it has its own lexical/syntactic/semantic rules and compiles into a target representation (bytecode, machine code, or translated back to the host language).

### External DSL Compilation Pipeline

```
Source Code (DSL Syntax) → Lexical Analysis → Syntax Analysis (typically Pratt/LR) → Semantic Analysis → Code Generation
                                         ↓
                                  AST (DSL-specific nodes)
```

It is **exactly the same** as a general compiler, except:
- The language is smaller (dozens of grammar rules rather than hundreds).
- The target is usually not machine code, but source code in another language (transpilation) or bytecode.
- Error messages are the user interface of this DSL, **more important than error messages in general compilers**—users working in a specific domain should receive error messages using domain terminology.

### Code Generation Targets: Transpile vs. Compile

- **Transpile**: DSL source code → Source code in another language. Terraform HCL → Internal plan JSON; Protobuf `.proto` → `.go`/`.rs`/`.py`. Advantages: Output is readable and debuggable; downstream compilers perform optimizations.
- **Compile**: DSL source code → IR → Machine code/Bytecode. Terra (a mix of Lua and low-level languages) takes this path—certain parts of the DSL are JIT-compiled to machine code. Advantages: Best performance; Cost: Complex implementation (requires IR → backend).

The choice mainly depends on "whether the DSL users prioritize performance or debuggability."

### Selecting an External DSL: When is it Worth Starting a New Language?

Ask three questions:
1. **Can domain vocabulary be expressed naturally in the host syntax?** SQL's `SELECT...FROM...WHERE` can be made close to natural using a fluent API in a general language, but regular expressions like `^a.*b$` cannot be made close to natural in general syntax.
2. **Is the usage frequency sufficient?** A configuration file format used a few times does not justify writing a parser; thousands of queries within a system do.
3. **Do error messages require domain knowledge?** "Unmatched parenthesis at position 15" from a regex compiler and "SyntaxError: unexpected token" from a general compiler are not of the same quality of experience—DSLs need to understand domain concepts to report good errors.

Criterion: **If a concept "always requires comments to explain intent" in the host language, it is a signal for a DSL.**

## Progressive Evolution: From Internal DSL to External DSL

Many successful DSLs start as internal DSLs. When the limitations of the host syntax become unacceptable, an external parser is added:

```
Phase 1: Fluent API + String Templates                    (Internal DSL)
Phase 2: Macro DSL (Compile-time AST construction + Host type checking)   (Internal DSL, Compile-time)
Phase 3: External Parser + Transpilation to Host Language               (External DSL, Transpilation)
```

Serde evolved from Phase 1 (manual `Serialize` impl) to Phase 2 (procedural macro `#[derive(Serialize)]`), and currently stopping at Phase 2 is entirely sufficient. SQL stops at Phase 1–2 in most ORMs, but the SQL parser inside database engines is Phase 3.

## References

- **Martin Fowler**: "Domain-Specific Languages" — DSL design patterns and selection guide
- **Holden**: "Build Your Own Lisp" — Writing a Lisp DSL interpreter from scratch, including parser + codegen
- **Serde**: https://serde.rs — The quintessential example of a Rust macro DSL
- **Terra**: https://terralang.org — Representative of low-level DSLs (JIT compilation)

*Keywords: DSL, domain-specific language, internal DSL, external DSL, fluent API, builder pattern, operator overloading DSL, expression template, macro DSL, proc macro, transpile, code generation, embedded DSL, language design, Serde, diesel*