Buffered I/O (stdio)

Coverage: setvbuf three strategies → fread/fwrite buffering mechanism → user-space buffering vs kernel page cache → line buffering and terminals → custom buffering Applicable to: glibc stdio, musl

Overview

fread/fwrite do not directly invoke the read/write system calls—they maintain buffers in user space. This is why fwrite(buf, 1, 1, fp) does not trigger a system call (until fflush is called or the buffer is full). Understanding this layer is fundamental to troubleshooting issues like "why isn't printf output appearing" or "why is fwrite faster than write."

setvbuf: Three Buffering Modes

#include <stdio.h>

// _IONBF: Unbuffered → each fwrite triggers a syscall directly
setvbuf(fp, NULL, _IONBF, 0);

// _IOLBF: Line-buffered → flush on '\n' or when buffer is full
setvbuf(fp, NULL, _IOLBF, BUFSIZ);  // BUFSIZ = 8192 (glibc)

// _IOFBF: Fully buffered → flush only when buffer is full
setvbuf(fp, NULL, _IOFBF, BUFSIZ);

// Default strategies:
//   stderr:      Unbuffered (_IONBF)
//   tty (stdout): Line-buffered (_IOLBF)
//   Regular files: Fully buffered (_IOFBF), size = BUFSIZ

FILE Internal Structure (glibc Simplified)

// glibc: libio/fileops.c, libio/genops.c
struct _IO_FILE {
    int     _flags;         // _IO_UNBUFFERED, _IO_LINE_BUF, ...
    char    *_IO_read_ptr;  // Current read pointer
    char    *_IO_read_end;  // End of read buffer (buffer + bytes_read)
    char    *_IO_write_ptr; // Current write pointer
    char    *_IO_write_end; // End of write buffer (buffer + BUFSIZ)
    char    *_IO_buf_base;  // Start of buffer
    char    *_IO_buf_end;   // End of buffer

    int     _fileno;        // Corresponding file descriptor
    // ... locks, wide char, fallback, ...
};

// fwrite(buf, 1, n, fp):
//   1. If n <= (fp->_IO_write_end - fp->_IO_write_ptr):
//      → memcpy to buffer → _IO_write_ptr += n → return n
//      (Very fast! Purely user-space, no syscall)
//   2. Otherwise: flush first (write(fd, buffer, ...)) → then memcpy
//   3. If line-buffered and buf contains '\n': flush up to '\n' position

User-Space Buffering vs Kernel Page Cache

fwrite writing 1 byte × 1000 times:
  Fully buffered mode:    1 write syscall (flush buffer) → ~1 kernel call
  Unbuffered mode:        1000 write syscalls → ~1000 kernel calls
  Performance difference: ~100x

Kernel write writing 1 byte × 1000 times to a file:
  Page cache may coalesce (if within the same page) → 1 disk IO
  But each write() incurs syscall overhead (~200ns each)

Typical combination:
  fwrite: User-space batching (reduces number of syscalls)
  page cache: Kernel-level batching (reduces number of disk IOs)

Pitfalls of Line Buffering

// Terminal output (stdout is line-buffered by default):
printf("hello");        // Enters buffer, no output (no '\n')
fprintf(stderr, "!");   // stderr is unbuffered → immediately outputs "!"
printf(" world\n");     // Enters buffer + '\n' → flush → outputs "hello world"

// Output: "!hello world" (not the expected order!)

// Fix:
printf("hello");
fflush(stdout);         // Force flush
fprintf(stderr, "!");
printf(" world\n");

// Or: Disable stdout buffering from the start:
setvbuf(stdout, NULL, _IONBF, 0);

fork and Buffering

// Classic pitfall:
printf("before fork\n");  // Data in buffer but no '\n'? → But '\n' is present → flush
fork();
// OK: flushed before fork

printf("before fork");    // Data in buffer, no '\n'
fork();
// Child process copies the entire FILE structure (including buffer!)
// → Both parent and child will output "before fork" on subsequent flushes
// → Output appears twice!

// Rule: Always fflush(NULL) before fork, or use line buffering to ensure flushing

Custom Buffering Strategies

// Provide your own buffer (instead of glibc's default BUFSIZ allocation):
char mybuf[4096];
setvbuf(fp, mybuf, _IOFBF, sizeof(mybuf));

// Note: The lifetime of mybuf must be ≥ the lifetime of fp
// (glibc does not copy the buffer, it only stores the pointer)

References

• man: setvbuf(3), fflush(3), fwrite(3)
• glibc source: libio/fileops.c, libio/genops.c
• LWN: "The GNU C Library's stdio implementation"

Keywords: setvbuf, _IONBF, _IOLBF, _IOFBF, fflush, BUFSIZ, FILE, fork buffer, line buffering