malloc Internals: ptmalloc2

Coverage: ptmalloc2 (glibc) → arenas/bins → mmap threshold → tcmalloc / jemalloc comparison → debugging methods Applicable to: glibc 2.x

Overview

glibc's malloc is based on ptmalloc2 (pthread malloc v2), which is a multi-threaded improvement of dlmalloc. The core idea is that each thread has its own allocation arena to reduce lock contention; small allocations are taken from bins (free lists), while large allocations use mmap directly. Understanding the internal mechanisms is crucial for troubleshooting "slow malloc", "memory fragmentation", and "RSS significantly larger than actual usage".

Arena: Thread-Independent Heap

dlmalloc (single-threaded): 1 heap, 1 lock
ptmalloc2 (multi-threaded): multiple arenas

arena (allocation area):
  Main arena (main_arena): extends using brk/sbrk (located in the [heap] region)
  Sub-arena:             allocated using mmap (located in the mmap region, anonymous)

When a thread allocates:
  1. Tries to use the arena it used last (cached in TSD)
  2. If that arena is held by another thread → tries the next arena
  3. If all arenas have contention → creates a new arena (up to 8×CPU count)

Strategy: trade off the number of arenas for lock-free parallelism → suitable for programs with dense multi-threaded allocations
Drawback: too many arenas → memory fragmentation (each arena has its own top chunk)

Bins: Free Lists

ptmalloc2 uses bins to manage released free blocks:

Fastbins (LIFO, singly linked list, no merging):
  Used for very small releases (16~64 bytes, depending on MAX_FAST)
  On free: push to fastbin
  On alloc: pop from fastbin (if size matches, use it; otherwise → merge)

Small bins (FIFO, circular doubly linked list):
  Size: 32~1024 bytes (64 bins, 8-byte steps per bin)
  Adjacent free blocks are automatically merged

Large bins (FIFO + best fit):
  Size: 1024+ bytes (63 bins, variable-width steps per bin)
  Best fit: find the smallest free block ≥ size

Unsorted bin (cache layer):
  Newly freed blocks go here first (LIFO)
  On alloc, traverse → exact match? return it : place in corresponding small/large bin

Allocation Flow

flowchart TD
    START["malloc(size)"]

    START --> THRESH{"size &lt; 128KB<br/>(M_MMAP_THRESHOLD)?"}

    THRESH -->|"Yes → bins path"| FAST{"size &lt; MAX_FAST<br/>(~64B)?"}
    THRESH -->|"No → large alloc"| MMAP["mmap(anonymous)<br/>independent mapping<br/><br/>on free: munmap<br/>fork-friendly: no COW"]

    FAST -->|"Yes"| FB["Check fastbins<br/>(LIFO, by size index)"]
    FAST -->|"No"| SMALL{"size &lt; 1024?"}

    SMALL -->|"Yes"| SB["Check small bins<br/>(FIFO, doubly linked list)"]
    SMALL -->|"No"| LARGE["① Check unsorted bin<br/>② Match? return : else put in large bin<br/>③ Check large bin (best fit)<br/>④ If insufficient → slice from top chunk"]

    FB --> OK1["✅ Return"]
    SB --> OK2["✅ Return"]
    LARGE --> OK3["✅ Return"]
    MMAP --> OK4["✅ Return"]

    FB -.->|"Miss"| SMALL
    SB -.->|"Miss"| LARGE
    LARGE -.->|"Top chunk also insufficient"| FALLBACK["Expand arena<br/>sbrk (main) / mmap (sub)<br/>retry alloc"]
    FALLBACK -.->|"Still fails"| FAIL["❌ ENOMEM<br/>Return NULL"]

    classDef start fill:#e3f2fd,stroke:#1565c0
    classDef decision fill:#fff3e0,stroke:#ef6c00
    classDef path fill:#f3e5f5,stroke:#7b1fa2
    classDef ok fill:#e8f5e9,stroke:#2e7d32
    classDef fail fill:#ffebee,stroke:#c62828
    class START start
    class THRESH,FAST,SMALL decision
    class FB,SB,LARGE,MMAP,FALLBACK path
    class OK1,OK2,OK3,OK4 ok
    class FAIL fail

Top Chunk

Each arena has one top chunk (remaining space at the top of the heap):
  If all bins are insufficient → slice from top chunk → expand top chunk (sbrk/mmap)
  If the freed block is adjacent to the top chunk? → merge back into top chunk

mmap Threshold

// Large allocation strategy:
#define DEFAULT_MMAP_THRESHOLD (128 * 1024)  // 128KB

// Dynamic adjustment: each time a large block is freed → threshold may decrease
//   → mallopt(M_MMAP_THRESHOLD, value) can fix it

// Why use mmap for large allocations?
//   1. munmap immediately on free → return to kernel → reduce RSS
//   2. Independent mapping → does not fragment the heap
//   3. Fork-friendly: child process modifies its own (not COW)

Multi-threading Pitfalls

False Sharing

Two threads frequently allocate/release → may share the same arena
  → arena's mutex → contention → lock overhead

Mitigation:
  malloc_trim(0): force all arenas to release free space to the kernel
  MALLOC_ARENA_MAX: environment variable to limit the maximum number of arenas

Memory Fragmentation

Frequent allocation + release of different sizes → free blocks in bins cannot be reused:
  Example: alloc 512B → free → alloc 1024B → 512B block too small → slice from top chunk
  → heap grows gradually but actual usage is small → high RSS

Detection:
  malloc_stats() or malloc_info(0, stderr) → check system bytes vs in use

tcmalloc / jemalloc Comparison

Switching to a Non-glibc Allocator

# Runtime replacement (without recompilation):
LD_PRELOAD=/usr/lib/libtcmalloc.so ./my_program
LD_PRELOAD=/usr/lib/libjemalloc.so ./my_program

Debugging and Tuning

# malloc statistics
MALLOC_TRACE=/tmp/mtrace.log ./my_program  # Record all malloc/free
mtrace ./my_program /tmp/mtrace.log          # Analyze leaks

# Runtime parameters
MALLOC_ARENA_MAX=2 ./my_program             # Limit number of arenas
MALLOC_MMAP_THRESHOLD_=65536 ./my_program   # Lower mmap threshold

# valgrind check
valgrind --leak-check=full ./my_program

# ASAN (compile-time)
gcc -fsanitize=address -g -o prog prog.c

# View memory allocation statistics (glibc)
#include <malloc.h>
malloc_info(0, stderr);   # Detailed statistics in XML format
malloc_stats();           # Simple stderr statistics

# /proc observation
cat /proc/<pid>/status | grep Vm

References

• glibc Source Code: malloc/malloc.c (~5000 lines, complete implementation of ptmalloc2)
• Documentation: man mallopt, man malloc_info
• LWN: "The design of glibc malloc", "Hoard, tcmalloc, and jemalloc"

Keywords: ptmalloc2, arena, bins, fastbin, mmap threshold, tcmalloc, jemalloc, malloc_stats, ASAN