Loading and Startup: _start → main

Coverage: ELF loading process → _start → _libc_start_main → main → exit → auxv → AT* vectors → init/fini arrays → static vs dynamic linking Applicable to: glibc (x86-64 / ARM64), musl comparison

Overview

The entry point of a C program is not main—it is glibc's _start. Before main is called, the kernel has already done a lot of work (ELF parsing, stack layout, auxv passing), and glibc has completed its own initialization (TLS, vDSO parsing, stdio buffering, __attribute__((constructor))). Understanding this path is crucial for debugging "crashes before reaching main."

Kernel Side: ELF Loading

flowchart TD
    START["execve() → load_elf_binary()"]

    START --> VALIDATE["🔍 Validate ELF header<br/>magic + class + machine + ABI"]

    VALIDATE --> PHDR["Read program header table"]

    PHDR --> LOAD["Iterate over each PT_LOAD"]
    LOAD --> MAP["elf_map() → mmap(segment)<br/>Map code and data into memory"]

    MAP --> INTERP{"PT_INTERP<br/>Present?"}

    INTERP -->|"Yes (Dynamic Linking)"| LD["Load ld.so<br/>Set ld.so entry address"]
    LD --> RET_LD["Kernel returns to ld.so entry<br/>(not the program's _start)"]

    INTERP -->|"No (Static Linking)"| ENTRY["Set e_entry"]
    ENTRY --> RET_START["Kernel returns to program's _start"]

    classDef start fill:#e3f2fd,stroke:#1565c0
    classDef step fill:#f3e5f5,stroke:#7b1fa2
    classDef decision fill:#fff3e0,stroke:#ef6c00
    classDef done fill:#e8f5e9,stroke:#2e7d32
    class START start
    class VALIDATE,PHDR,LOAD,MAP step
    class INTERP decision
    class LD,ENTRY,RET_LD,RET_START done

Dynamic Linking: ld.so Execution

flowchart TD
    START["Kernel returns to ld.so entry"] --> BOOT["① ld.so bootstrap<br/>ld.so itself is dynamically linked<br/>First, self-relocate with restrictions"]

    BOOT --> DYNAMIC["② Read main program's .dynamic"]
    DYNAMIC --> NEEDED["DT_NEEDED → libc.so.6, libfoo.so, ..."]
    DYNAMIC --> DEBUG["DT_DEBUG → Set debug structure<br/>(for gdb reading)"]

    NEEDED --> DEPS["③ _dl_map_object_deps()<br/>Topological sort → mmap .so files one by one"]

    DEPS --> RELOC["④ _dl_relocate_object()<br/>Symbol resolution + relocation<br/>Fill GOT, resolve IFUNC"]

    RELOC --> INIT["⑤ Call library init functions<br/>DT_INIT / DT_INIT_ARRAY<br/>→ __libc_init_first()"]

    INIT --> START_["⑥ Call main program entry<br/>e_entry → _start"]

    classDef start fill:#e3f2fd,stroke:#1565c0
    classDef step fill:#f3e5f5,stroke:#7b1fa2
    classDef done fill:#e8f5e9,stroke:#2e7d32
    class START start
    class BOOT,DYNAMIC,NEEDED,DEBUG,DEPS,RELOC,INIT step
    class START_ done

User Space: _start

# glibc sysdeps/x86_64/start.S
_start:
    xorl  %ebp, %ebp          # Mark outermost frame (ebp=0 → end of backtrace)
    movq  %rdx, %r9           # rdx=destructor function (rtld_fini)
    popq  %rsi                # argc
    movq  %rsp, %rdx          # argv
    andq  $-16, %rsp          # 16-byte stack alignment (ABI requirement)
    pushq %rax                # Alignment filler
    pushq %rsp                # stack_end (for __libc_stack_end)
    xorl  %r8d, %r8d          # 8th argument (glibc internal flag)
    call __libc_start_main    # → libc init → main
    hlt                       # Never reached (main returns via exit)

__libc_start_main

// csu/libc-start.c (simplified)
int __libc_start_main(
    int (*main)(int, char **, char **),  // main function
    int argc,
    char **argv,
    void (*init)(void),                   // __libc_csu_init
    void (*fini)(void),                   // __libc_csu_fini
    void (*rtld_fini)(void),              // ld.so cleanup function
    void *stack_end)
{
    // 1. Set up TLS (Thread Local Storage)
    __pthread_initialize_minimal();

    // 2. Set up stack protection (canary)
    __stack_chk_guard = _dl_setup_stack_chk_guard();

    // 3. Read auxv → Initialize global variables:
    //    __environ = envp;
    //    __libc_stack_end = stack_end;
    //    _dl_phdr = AT_PHDR;
    //    _dl_phnum = AT_PHNUM;
    //    __page_size = AT_PAGESZ;

    // 4. Call init → __libc_csu_init()
    //    → Iterate .init_array → Execute __attribute__((constructor)) functions
    //    → Including: _init() (if present)
    (*init)(argc, argv, envp);

    // 5. Call main
    result = (*main)(argc, argv, __environ);

    // 6. main returns → exit(result)
    exit(result);
}

auxv (Auxiliary Vector)

// Kernel builds this during execve, placed on the stack after envp
// Format: { type, value } pairs, terminated by AT_NULL

// Key AT_* types:
AT_PHDR:       Address of program header (used by dl_iterate_phdr)
AT_PHENT:      Program header entry size
AT_PHNUM:      Number of program headers
AT_PAGESZ:     Page size (4KB / 16KB / 64KB)
AT_BASE:       Load base address of ld.so
AT_ENTRY:      Address of _start
AT_UID / AT_EUID / AT_GID / AT_EGID: Process credentials
AT_SECURE:     Whether secure mode is needed (setuid → 1)
AT_SYSINFO_EHDR: vDSO address
AT_RANDOM:     16 bytes of random data (used for ASLR, stack canary)
AT_PLATFORM:   CPU platform string (e.g., "x86_64")

Reading auxv

#include <sys/auxv.h>
unsigned long hwcap = getauxval(AT_HWCAP);  // CPU feature flags

// Manual iteration:
#include <elf.h>
extern char **environ;  // auxv follows envp

# LD_SHOW_AUXV: Make ld.so print auxv
LD_SHOW_AUXV=1 /bin/true

Static Linking Comparison

Static Linking:
  - No PT_INTERP → Kernel jumps directly to _start
  - _start still calls __libc_start_main (linked against libc.a)
  - But no dynamic linking → No PLT/GOT → No lazy binding
  - Entire libc linked in → Huge binary (~800KB for hello world)
  - But extremely fast startup (no symbol resolution, no relocation)

Static linking with musl is very popular (in the container world):
  - musl requires only ~100 lines of code from _start → __libc_start_main → main
  - The startup path is ~10x shorter than glibc's

Debugging

# Trace the startup process
strace -e trace=mmap,open,read /bin/true

# ld.so debug output
LD_DEBUG=all /bin/true 2>&1 | head -100

# View auxv
LD_SHOW_AUXV=1 /bin/true
cat /proc/self/auxv | xxd

# View init_array
readelf -x .init_array /bin/ls

# Static analysis of startup path
gdb /bin/ls
(gdb) break _start
(gdb) run

References

glibc source: sysdeps/x86_64/start.S, csu/libc-start.c, elf/dl-start.c, elf/rtld.c
musl source: crt/crt1.c, ldso/dlstart.c — 10x simpler than glibc, recommended for learning
Kernel source: fs/binfmt_elf.c — load_elf_binary
LWN: "How programs get run", "The init and fini arrays"

Keywords: _start, __libc_start_main, auxv, AT_PHDR, vDSO, .init_array, constructor, TLS, ld.so bootstrap, PT_INTERP