---
title: LSP and Language Servers
url: https://doc.liz6.com/en/compilers/08-compilation-technology-applications/04-lsp-and-language-servers
locale: en
area: compilers
tags:
- compilers
- compilation-technology-applications
date: 2026-06-30
modified: 2026-07-16
description: IDE completion, jump-to-definition, hover types, refactoring—these code intelligence features all come from the language server, which is essentially a compiler frontend + incremental updates + error-tolerant parsing. LSP exposes lexical/symbol table capabilities to the editor via JSON-RPC.
---
# LSP and Language Servers
> IDE completion, jump-to-definition, hover types, refactoring—these code intelligence features all come from the language server, which is essentially a compiler frontend + incremental updates + error-tolerant parsing. LSP exposes lexical/symbol table capabilities to the editor via JSON-RPC.
## Overview
IDE code intelligence—completion, jump-to-definition, hover type display, refactoring—traditionally required each IDE to write a separate analysis logic for each language it supported (M × N). LSP (Language Server Protocol) solves this using the same strategy as MCP: **provide one language server process per language, with the IDE communicating via JSON-RPC**. For compilation technology, LSP is a direct consumer of compiler frontend data (lexical/syntax/semantic/symbol table)—a language server is essentially a compiler frontend + an incremental update engine. This article covers the core request types of the LSP protocol, how language servers use the compiler frontend techniques from previous chapters to build indexes and respond, and why "being fast enough" is harder to achieve than "being precise enough."
## LSP is not a compiler; it's a compiler frontend with a different goal
What a compiler wants: produce machine code from source code; correctness is paramount; batch processing across a set of files.
What a language server wants: update completion/diagnostics/highlighting in <100ms for every keystroke, provide reasonable results for **incomplete/erroneous code**, and only recompute changed parts.
The latter requirement means a language server cannot simply run the entire compiler pipeline—it must be **incremental** and **tolerant of syntax errors** (source code will inevitably have syntax errors while the user is still typing).
## Core Request Types: All Queries on Symbol Tables + AST
Every LSP request is a query on compiler frontend data structures:
| Request | Corresponding Compiler Frontend | What is Queried |
|------|---------------|-----------|
| `textDocument/completion` | Symbol Table + Type System | What names are visible in the current scope? What are their types and documentation? |
| `textDocument/definition` | Symbol Table | On which line is the definition of this identifier? |
| `textDocument/references` | Symbol Table (Reverse Lookup) | Where is this definition referenced? |
| `textDocument/hover` | Symbol Table + Type System | What is the type of this identifier? What does its documentation comment say? |
| `textDocument/signatureHelp` | Type System | What are the parameter lists and overloads of the current function? Which parameter is currently active? |
| `textDocument/rename` | Symbol Table (All References) | Change the name at all reference sites of this definition to the new name. |
| `textDocument/publishDiagnostics` | Semantic Analysis (Type Checking) | What type errors/unused variables exist in this file? |
**All capabilities stem from the [Symbol Table](/compilers/03-semantic-analysis/01-symbol-tables-and-scopes.md)**. `definition` is a lookup in the symbol table, `references` is the use list of a symbol, `completion` is an enumeration of the symbol table in the current scope, and `hover` is the type and doc string of a symbol. If the symbol table does not maintain a reverse lookup index for "where this name is referenced," `references` and `rename` cannot be efficient—building the symbol table requires bidirectional indexes (from definition to reference, and from reference to definition).
## Indexing: Knowing What Each File Exports Before Opening It
In large projects with thousands of files, when a user opens `main.rs`, the language server cannot parse the entire project. It needs a **persistent index**:
rust-analyzer uses Salsa (an incremental computation framework) to maintain this index. Any change only recomputes affected files, while unaffected files are retrieved directly from the cache. This is the key to keeping IDE response times under <100ms—not full recomputation, but incremental updates.
## Error-Tolerant Parsing: Producing Results on Syntactically Invalid Input
When a user is typing (`let x = some_struct.`), the source code is inevitably incomplete—the parser sees the dot after `some_struct`, expects a field name, but encounters EOF. A compiler would stop here with an error. A language server **cannot stop; it must recover from the error and provide completion for what might follow the `.`**.
This requires the error recovery mechanism described in [AST Design and Error Recovery](/compilers/02-syntax-analysis/03-ast-design-and-error-recovery.md) to be robust: upon seeing an incomplete expression, the parser should still construct an AST node for it (marked as an error). Higher-level semantic analysis can then perform partial type inference on this "AST with errors"—extracting the field list from the type information of `some_struct` and returning it as completion candidates.
```
User Input: let x = some_struct.
Parser Output: Expr::Field { object: "some_struct", field: }
Semantic Analysis: Look up type of some_struct in symbol table → StructFoo { a: i32, b: String }
Completion: [a: i32, b: String]
```
Without error recovery, there is no completion—these two are **a pair** in a language server. The only distinction is whether "the current token is a complete expression" or "the current token is a syntax error." The latter requires the AST node produced by the parser to mark "the field is missing," allowing semantic analysis to know that the user expects completion at that position.
## Semantic Tokens: Another Way to Implement Syntax Highlighting
Traditional highlighting uses regex matching (like Tree-sitter's `highlights.scm`), but regex doesn't understand semantics—`foo` is a variable definition in `let foo = ...`, a function call in `foo()`, and a function declaration in `fn foo()`. LSP's `textDocument/semanticTokens/full` assigns a **semantic category** (variable, function, keyword, type, comment...) to **each token**, and the IDE colors them based on these categories.
The language server determines the semantic category for each token on top of the AST + symbol table—this requires the tokens produced by the lexer to carry sufficient information (is it an identifier? Does reverse lookup in the symbol table reveal it as a variable, function, or type?), or for semantic analysis to annotate them additionally.
## Architecture: Single Process, Multi-threading, Incremental Scheduling
The typical architecture of industrial language servers (rust-analyzer, clangd):
Debounce and cancellation are lifelines for performance—when a user types quickly, `on_type` frequency is ~100ms per character. If every character triggered a full file re-parse, the language server would 100% freeze. Debounce merges "N consecutive inputs" into "parse after the last one," and cancellation discards outdated intermediate results.
## References
- **LSP 3.17 specification**: https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/
- **rust-analyzer**: https://github.com/rust-lang/rust-analyzer — Architecture documentation and source code for the Rust LSP server
- **Clangd**: https://clangd.llvm.org — C/C++ LSP server based on the Clang frontend
*Keywords: LSP, Language Server Protocol, JSON-RPC, completion, go-to-definition, find-references, hover, diagnostics, semantic tokens, incremental parsing, index, debounce, cancellation, error-tolerant parsing, symbol table reverse lookup, rust-analyzer, clangd*