NFS (Network File System)

Coverage: NFSv3/v4 protocol → SUNRPC layer → client/server architecture → caching (attribute/delegation) → pNFS → Kerberos security → nfsd Kernel version: 2.6 ~ 6.x

Overview

NFS is the most traditional network file system for Unix/Linux. NFSv4 (2003) is a major rewrite of v3: it introduced state (locking + delegation), compound RPC operations, a pseudo-filesystem, and a better security model.

The Linux NFS implementation is divided into two independent modules:

• nfsd (server, fs/nfsd/): Exports local file systems to the network
• nfs (client, fs/nfs/): Mounts remote file systems

SUNRPC Layer

NFS uses SUNRPC (ONC RPC) as its transport layer:

// net/sunrpc/
// RPC client: sends requests → waits for replies
struct rpc_clnt {
    struct rpc_program *cl_program;    // NFS (100003) or other programs
    struct rpc_version *cl_versions;   // NFSv3 (3) or NFSv4 (4)
    struct rpc_xprt    *cl_xprt;       // Transport: TCP/UDP/RDMA
};

// RPC Transport:
//   TCP (default): Reliable, built-in flow control
//   UDP:       Lightweight but risk of packet loss (v3 only)
//   RDMA:      High performance (InfiniBand/RoCE)

NFS Client

Mounting and Pseudo-Filesystem

NFSv3 mount:
  mount -t nfs server:/export/path /mnt
    → MOUNT protocol (portmapper) to obtain initial filehandle
    → Subsequent operations use NFS protocol (port 2049)

NFSv4 mount:
  mount -t nfs4 server:/ /mnt
    → Pseudo-filesystem: Server exports the root; client traverses to reach sub-exports
    → No auxiliary MOUNT protocol required
    → fsid=0 in /etc/exports marks the root export

Attribute Cache

// NFS client caches file attributes (stat, permissions, etc.)
// Attribute cache times are controlled by acregmin/acregmax/acdirmin/acdirmax

// close-to-open consistency:
//   This is NFS's default consistency model:
//   1. open() → forces re-validation of attributes (GETATTR)
//   2. Files not opened: attributes are reused within the cache period
//   3. close() → writes back all modifications
//   4. Another client opens the same file → sees the complete modifications

// NFSv4 delegation:
//   The server can "delegate" read/write rights of a file to the client
//   Client has exclusive read/write → local cache of attributes + data → zero RPCs!
//   Another client wants to access → Server first revokes the delegation → First client flushes back to server
//   Similar to the lease mechanism in distributed systems

Page Cache and Writeback

NFS client page cache:
  Read: page cache hit → no RPC initiated → served locally
  Write: write to page cache first → asynchronous WRITE RPC → background writeback

NFS COMMIT:
  fsync → COMMIT RPC → Server ensures data is written to stable storage
  → Server may use NFS write delegation to accelerate

NFSv4.1+ pNFS (Parallel NFS):
  Metadata goes through MDS (Metadata Server)
  Data goes directly through DS (Data Server)
  → Bypasses the bandwidth bottleneck of the MDS

NFS Server (nfsd)

// fs/nfsd/
// Kernel NFS server: nfsd kernel thread pool

// Exporting file systems:
//   /etc/exports:
//     /export *(rw,sync,no_subtree_check)
//   exportfs -a → notifies the kernel

// nfsd threads:
//   One nfsd kernel thread per CPU
//   nfsd loop: fetch RPC request → execute NFS operation → return reply
//   Thread count: /proc/fs/nfsd/threads

// NFSv4 state:
//   Server maintains client state (open files, locks, delegations)
//   State storage: in /var/lib/nfs/nfsdcltrack (or filesystem-based for v4.2+)
//   Client crash → state is automatically reclaimed after the lease time

Security Model

NFSv3: 
  Based on AUTH_SYS (UNIX credentials: uid/gid/aux gids)
  → Trusts the uid/gid provided by the client (insecure!)
  → root_squash mitigates this (maps root to nobody)
  → sec=sys (default), sec=krb5 (optional)

NFSv4:
  Supports Kerberos 5 (RPCSEC_GSS)
    sec=krb5:    Authentication only
    sec=krb5i:   Authentication + Integrity (prevents tampering, signing)
    sec=krb5p:   Authentication + Integrity + Privacy (encrypts data)

  Mandatory requirement: NFSv4.2+ servers must support sec=krb5

TASK_KILLABLE and NFS

// NFS is one of the primary motivations for introducing TASK_KILLABLE
// Classic problem: NFS is mounted but the server is unreachable
//   → Process accesses NFS mount point → enters TASK_UNINTERRUPTIBLE (D state)
//   → SIGKILL is ineffective (D state does not respond to signals!)
//   → Only a reboot works
//
// TASK_KILLABLE solution (5.x+):
//   Some wait paths use wait_event_killable()
//   → Checks fatal_signal_pending() while waiting
//   → SIGKILL can interrupt the wait
//   Not all NFS paths have been changed to KILLABLE

Debugging

# NFS client statistics
cat /proc/self/mountstats  # Detailed RPC statistics
nfsstat -c                  # Client

# NFS server statistics
nfsstat -s                  # Server

# View NFS mount options
mount | grep nfs
cat /proc/mounts | grep nfs

# Trace NFS RPC
rpcdebug -m nfs -s all      # Enable NFS client debugging
rpcdebug -m nfsd -s all     # Enable NFS server debugging

References and Further Reading

• Kernel Documentation: Documentation/filesystems/nfs/
• RFCs: RFC 7530 (NFSv4), RFC 5661 (NFSv4.1/pNFS), RFC 7862 (NFSv4.2)
• Source Code:
  • fs/nfs/ — Client implementation
  • fs/nfsd/ — Server implementation
  • net/sunrpc/ — RPC layer
  • include/linux/sunrpc/ — RPC type definitions

Keywords: NFSv4, SUNRPC, delegation, close-to-open, pNFS, nfsd, TASK_KILLABLE, Kerberos, RPCSEC_GSS