---
title: Network Protocol Stack Overview
url: https://doc.liz6.com/en/linux-kernel/06-network-subsystem/01-network-protocol-stack-overview
locale: en
area: linux-kernel
tags:
- linux-kernel
- network-subsystem
date: 2026-06-30
modified: 2026-07-16
description: 'Coverage: sk_buff → NAPI → softirq (NET_RX/NET_TX) → protocol stack layers → socket API → net_device → multi-queue Kernel version: 2.6 ~ 6.x'
---

# Network Protocol Stack Overview

> Coverage: sk_buff → NAPI → softirq (NET_RX/NET_TX) → protocol stack layers → socket API → net_device → multi-queue
> Kernel version: 2.6 ~ 6.x

## Overview

The Linux network stack is the most widely used TCP/IP implementation in the world. From NIC DMA packet reception to the user-space `read()` return, packets go through 10+ layers of processing. This article traces this complete path, focusing on core data structures and design decisions.

## sk_buff: Packet Descriptor

```c
// include/linux/skbuff.h
struct sk_buff {
    struct sk_buff      *next, *prev;  // Doubly linked list (scheduling queue)
    struct sock         *sk;           // Associated socket

    ktime_t             tstamp;        // Timestamp (SO_TIMESTAMPING)
    struct net_device   *dev;          // Input/output device

    // Data pointers (linear data + non-linear frags)
    unsigned char       *head, *data;
    unsigned int        len, data_len;
    sk_buff_data_t      tail, end;

    // Protocol header pointers (set during layer-by-layer parsing)
    __be16              protocol;      // L3 (ETH_P_IP, ETH_P_IPV6)
    unsigned char       *transport_header;  // L4 (TCP/UDP)
    unsigned char       *network_header;    // L3 (IP header)
    unsigned char       *mac_header;        // L2 (Ethernet)

    // Segmentation/Reassembly
    unsigned int        gso_size, gso_segs;  // GSO: Large packets → hardware segmentation
    unsigned short      nr_frags;            // Number of non-contiguous pages
};
```

The core design of `sk_buff` is the four-pointer layout of `head/data/tail/end`: data resides in the buffer allocated from `head` to `end`, while `data` to `tail` represents valid data. The regions `head` to `data` and `tail` to `end` are headroom and tailroom, respectively. As the protocol stack adds each layer's header, the `data` pointer moves backward, reusing the existing headroom—no copying is required.

## NAPI: Hybrid Interrupt + Polling

```c
// net/core/dev.c
// Problem: High-speed NICs (10Gbps+) receive millions of packets per second → one interrupt per packet → interrupt storm
// NAPI: First packet triggers an interrupt notification → interrupts disabled → polling for packet reception in softirq

// NIC Driver:
//   1. Packet received → disable RX interrupt → napi_schedule(&napi)
//   2. NET_RX_SOFTIRQ triggered → napi_poll()
//      → Batch receive packets (one budget, typically 300 packets)
//      → After processing a batch → if more packets remain → continue polling
//      → If no more packets → enable RX interrupt → return to step 1
```

## Complete Packet Reception Path

```mermaid
flowchart TD
    DMA["NIC DMA<br/>→ Ring Buffer (RX ring)"]
    DMA -->|"Hardware Interrupt"| IRQ["Driver Interrupt Handler<br/>napi_schedule()"]
    IRQ -->|"NET_RX_SOFTIRQ"| POLL["napi_poll()<br/>Batch receive, up to 300 packets"]

    POLL -->|"Per Packet"| NETIF["__netif_receive_skb()"]

    NETIF --> XDP["XDP hook (bpf)<br/>Earliest processing point"]

    NETIF --> RXH{"RX Handler<br/>List Check"}
    RXH -->|"tcpdump"| AF_PACKET["AF_PACKET<br/>Copy to packet socket"]
    RXH -->|"bridge/bonding/VLAN"| REDIR["Possible Redirection"]

    NETIF --> DELIVER["deliver_skb()"]

    DELIVER --> IP_RCV["ip_rcv()<br/>L3 (IPv4)"]

    IP_RCV --> IP_LOCAL["ip_local_deliver()"]

    IP_LOCAL --> NF{"netfilter<br/>PREROUTING"}
    NF -->|"FORWARD"| FWD["Forward to other interfaces"]
    NF -->|"INPUT"| IP_FINISH["ip_local_deliver_finish()"]

    IP_FINISH --> TCP_RCV["tcp_v4_rcv()<br/>L4 (TCP)"]

    TCP_RCV --> TCP_DO["tcp_v4_do_rcv()"]

    TCP_DO --> STATE{"Connection State?"}
    STATE -->|"ESTABLISHED"| FAST["tcp_rcv_established()<br/>Fast path ⚡"]
    STATE -->|"LISTEN"| SLOW["tcp_rcv_state_process()"]
    SLOW --> QUEUE["Data enqueued to receive queue"]

    FAST --> QUEUE
    QUEUE --> WAKE["sk_data_ready()<br/>Wake up blocked read() ✅"]

    classDef hw fill:#fff3e0,stroke:#ef6c00
    classDef irq fill:#ffebee,stroke:#c62828
    classDef l2 fill:#e3f2fd,stroke:#1565c0
    classDef l3 fill:#f3e5f5,stroke:#7b1fa2
    classDef l4 fill:#e8f5e9,stroke:#2e7d32
    classDef decision fill:#fff8e1,stroke:#f9a825
    class DMA,IRQ hw
    class POLL,NETIF,XDP,RXH,AF_PACKET,REDIR l2
    class IP_RCV,IP_LOCAL,IP_FINISH,DELIVER,NF,FWD l3
    class TCP_RCV,TCP_DO,FAST,SLOW,QUEUE,WAKE l4
    class STATE decision
```

## Multi-Queue and RSS

```
RSS (Receive Side Scaling):
  NIC uses hash(src_ip, dst_ip, src_port, dst_port) % N
  → N receive queues → N CPUs

  NAPI's napi_struct is per-queue:
    → Each queue has its own NAPI → Parallel packet reception
    → Packets from the same flow always go to the same queue → TCP ordering preserved

XPS (Transmit Packet Steering):
  Multi-queue on the transmit side: similar concept but for TX
```

## net_device: Network Device Abstraction

```c
// include/linux/netdevice.h
struct net_device {
    char                name[IFNAMSIZ];    // eth0, wlan0
    const struct net_device_ops *netdev_ops;
    // → ndo_open, ndo_stop, ndo_start_xmit, ndo_set_rx_mode, ...

    struct net_device_stats stats;         // Statistics filled by driver
    struct list_head    qdisc_list;        // Queuing disciplines (tc)

    unsigned int        num_rx_queues;
    unsigned int        num_tx_queues;
    struct netdev_rx_queue  *_rx;          // RX queue array (per-queue NAPI)
};
```

## Debugging

```bash
# Packet receive/transmit statistics
cat /proc/net/dev
ip -s link

# Softirq statistics (NET_RX/NET_TX)
cat /proc/softirqs | grep NET

# NAPI statistics (per-queue polling)
ethtool -S eth0 | grep -E 'rx_packets|rx_bytes|tx_'

# Drop reasons (drop counter)
dropwatch -l kas
```

## References

- **Source Code**: `net/core/dev.c`, `include/linux/skbuff.h`, `include/linux/netdevice.h`
- **Kernel Documentation**: `Documentation/networking/`
- **LWN**: "The Linux networking stack", "NAPI design"

*Keywords: sk_buff, NAPI, NET_RX_SOFTIRQ, multi-queue, RSS, net_device, XDP*