Buffered I/O (stdio)

stdio provides formatted I/O, buffering, stream state, and position management via FILE objects. Buffering can coalesce many small operations, but it does not guarantee that every fwrite avoids entering the kernel, nor does it guarantee that each API call corresponds to exactly one system call when unbuffered. To determine where data has reached, you must distinguish between library interfaces, low-level I/O, and persistence layers.

FILE and File Descriptors

An fd is an integer index in the process descriptor table, while a FILE also maintains a buffer cursor, EOF/error state, direction, and locks. fread may return directly from a prefetched user buffer, or it may fill the buffer or perform larger low-level reads; fwrite similarly can take the buffered path or write directly.

The _IO_FILE fields in glibc aid in reading the implementation but are not a portable interface across different libc implementations and should not be modified in application code. A FILE does not necessarily correspond to a regular file; for example, memory streams may have different underlying operations.

Three Buffering Strategies

The common Linux libc defaults to line buffering for terminal stdout, fully buffered for regular file output, and unbuffered for stderr. After redirection to a file or pipe, stdout behavior may change; thus, it is not contradictory for the same program to output promptly in a terminal but delay output in a pipeline.

setvbuf should be called after opening the stream and before performing other operations on it, and its return value should be checked. BUFSIZ is a macro provided by the implementation; not all FILE buffers are guaranteed to be the same size, and you should not hardcode a specific glibc value as a portable constant. setvbuf

FILE *fp = tmpfile();
if (fp == NULL) return 1;
char storage[4096];
if (setvbuf(fp, storage, _IOFBF, sizeof storage) != 0) {
    fclose(fp);
    return 1;
}
/* Use fp; fclose(fp) must be called before storage goes out of scope. */
if (fclose(fp) == EOF) return 1;

This is a snippet within a function body and requires <stdio.h>. Custom buffers must remain alive until the stream is closed; passing a function-local array to a FILE that will be returned leaves a dangling reference.

fwrite, fflush, and fsync Guarantees Differ

fwrite returning the full count may only indicate that the data was accepted by the library; fflush succeeding means the corresponding user buffer has been submitted via a low-level write operation, but it does not guarantee persistence against power loss. For regular files requiring specific persistence guarantees, you typically call fflush first, then fsync on fileno(fp), checking errors for each separately. fflush(3)

fsync operates on kernel file state and cannot flush out data still sitting in the FILE buffer. Crash consistency for newly created or renamed files may also require syncing the directory containing them, which differs from syncing individual file content. fsync(2)

The kernel cache also does not guarantee that any 1000 write calls will be merged into exactly one physical I/O operation. Writeback policies, filesystems, block sizes, and device queues all play a role. Avoid hardcoding numbers like "fixed 100x speedup" or "system calls definitely take 200 ns."

Why can't fsync flush data inside FILE?

Append three bytes, then call fsync directly: the kernel doesn't know about these three bytes yet, so the sync guarantee won't cover them. Call fflush first, then fsync, and the three layers of accounting align. The underlying file retains committed content; don't depict sync as "moving and clearing" data from the kernel, because the file cache may still hold these bytes.

Correctly Interpreting Return Values and Errors

fread/fwrite return the number of elements completed; the count equals the byte count only when size=1. To handle trailing bytes byte-by-byte, using fread(buf, 1, capacity, fp) more clearly expresses how many bytes were obtained.

A short read may be EOF or an error; distinguish using feof and ferror. The EOF flag is set only after an attempt to read discovers the end, so while (!feof(fp)) is not a correct read loop condition. fread(3)

The following function copies two streams opened by the caller and flushes the output on successful completion; it does not close the streams and provides no persistence guarantees:

#include <stdio.h>

int copy_stream(FILE *in, FILE *out) {
    unsigned char buf[8192];
    size_t n;
    while ((n = fread(buf, 1, sizeof buf, in)) != 0) {
        if (fwrite(buf, 1, n, out) != n) return -1;
    }
    if (ferror(in)) return -1;
    return fflush(out) == 0 ? 0 : -1;
}

Write errors may occur at the time of fwrite, or they may be deferred until fflush or fclose. Even if an error is reported on close, the caller cannot continue to treat the original FILE as a valid object to retry fclose. Whether to retry after clearerr on an error stream should be designed based on the cause and position semantics, not blindly loop infinitely.

Errors May Still Appear During Flush After Write Acceptance

Linux's /dev/full returns ENOSPC for writes, which can be used to demonstrate how the library buffer defers errors to fflush without filling a real disk. The following is limited to glibc/Linux: place three characters in a fully buffered stream, then explicitly flush. full(4)

#include <errno.h>
#include <stdio.h>
int main(void) {
    FILE *fp = fopen("/dev/full", "w");
    if (fp == NULL) return 1;
    char buffer[1024];
    if (setvbuf(fp, buffer, _IOFBF, sizeof buffer) != 0) {
        fclose(fp); return 1;
    }
    int accepted = fputc('a', fp) != EOF;
    accepted = (fputc('b', fp) != EOF) && accepted;
    accepted = (fputc('c', fp) != EOF) && accepted;
    errno = 0;
    int flushed = fflush(fp);
    int saved = errno;
    int marked = ferror(fp) != 0;
    /* Regardless of the return value after close, do not access fp or call fclose again. */
    int closed = fclose(fp);
    printf("accepted=%d flush_failed=%d ENOSPC=%d error_mark=%d close=%d\n",
           accepted, flushed == EOF, saved == ENOSPC, marked, closed);
    return !(accepted && flushed == EOF && saved == ENOSPC && marked);
}

Save as delayed-error.c, compile, and run. The first four boolean results should be 1. The close result is shown separately and is not required to repeat the same flush error. You cannot ignore previous failures by only looking at the fclose return value at the end, nor can you declare data written successfully just because three fputc calls succeeded.

Mixing Reads, Writes, and Low-level fd Operations

Switching an update stream (e.g., r+, w+) from output to input requires fflush or a positioning operation; switching from input to output usually requires a positioning operation, unless the previous input encountered EOF. Use fseek/fsetpos etc. to explicitly synchronize direction and position. fopen(3)

stdio's lookahead can cause the underlying fd offset to advance ahead of the application's consumption position. Calling read directly on fileno(fp) may bypass unconsumed buffers; mixing write may also alter output order. Prefer unified I/O layers on a single object; if mixing is necessary, adhere to POSIX active handle coordination rules.

fflush(stdin) is not a universal method for clearing keyboard input. POSIX defines buffer discarding behavior for seekable input streams, but pipes and terminals are not in the same category.

Checking Position, Direction, and EOF in the Same Update Stream

Save as update-stream.c; tmpfile creates a binary update stream. Write abcdef, then read the first three bytes; use a positioning operation to coordinate the switch from read to write, replace d at the current position with X, and finally read back abcXef.

#include <stdio.h>
#include <string.h>
int main(void) {
    FILE *fp = tmpfile();
    if (fp == NULL) return 1;
    int failed = 1;
    char data[7] = {0};
    if (fwrite("abcdef", 1, 6, fp) != 6) goto done;
    if (fseek(fp, 0, SEEK_SET) != 0) goto done;
    if (fread(data, 1, 3, fp) != 3 || memcmp(data, "abc", 3) != 0) goto done;
    if (fseek(fp, 0, SEEK_CUR) != 0) goto done;
    if (fputc('X', fp) == EOF) goto done;
    if (fseek(fp, 0, SEEK_SET) != 0) goto done;
    if (fread(data, 1, 6, fp) != 6 || memcmp(data, "abcXef", 6) != 0) goto done;
    if (feof(fp) || ferror(fp)) goto done;
    if (fgetc(fp) != EOF || !feof(fp) || ferror(fp)) goto done;
    clearerr(fp);
    if (feof(fp) || ferror(fp)) goto done;
    /* Clearing the flag does not move the file position; reading again will still hit EOF. */
    if (fgetc(fp) != EOF || !feof(fp)) goto done;
    if (fseek(fp, 0, SEEK_SET) != 0 || feof(fp)) goto done;
    if (fgetc(fp) != 'a') goto done;
    failed = 0;
    puts("update and EOF transitions passed");
done:
    if (fclose(fp) == EOF) failed = 1;
    return failed;
}

Compile and run: cc -Wall -Wextra update-stream.c -o update-stream, then ./update-stream. When exactly 6 bytes are read, the application's request is satisfied, and no additional read has occurred to discover the end; the next fgetc sets EOF. clearerr only clears the state, not the file position, while a successful fseek both positions and clears EOF. fseek(3), Stream State

The goto here uniformly closes the FILE, avoiding resource leaks in each check branch. It does not automatically recover from failed I/O; if you want to report specific reasons, save the error information before jumping.

fork Copies Unflushed User Buffers

After fork, parent and child processes each have copies of FILE states and user buffers, while the underlying open file descriptors may be shared. If both sides flush normally via exit, the same unflushed content may be written twice.

Even if printf("before fork\n") contains a newline, stdout redirected to a fully buffered context may not have flushed yet. Single-threaded programs can explicitly fflush at appropriate points and check the result; you cannot infer safety across all output environments based on "having a newline."

After fork in multithreaded programs, the child process is still subject to async-signal-safety restrictions before exec and cannot arbitrarily call stdio to tidy up lock states. The exec failure path typically uses appropriate low-level reporting and _exit, avoiding repeated flushing of inherited buffers. _exit does not flush stdio, unlike normal exit.

Order, Atomicity, and Performance

Writing to stdout and stderr separately, even if both are flushed, does not guarantee a unified global display order in all external collectors. For strict ordering, use the same output channel or a logging protocol with sequence numbers.

stdio single calls usually have stream locks, but a log composed of multiple calls does not automatically become a transaction. When multiple processes write to a pipe, you must also consider the actual length of low-level write and PIPE_BUF; do not treat "one fprintf" as "one atomic pipe message."

Using strace to observe the number of low-level write calls can verify batching effects; evaluate buffer sizes using actual throughput, latency, and error behavior. Non-blocking network event loops usually manage fds and application buffers directly; see epoll and Event-Driven Programming for details.

Keywords: stdio, FILE, setvbuf, fread, fwrite, fflush, fsync, buffering, EOF, fork, active handle