Switching, Bridging, and VXLAN

From STP loop prevention to VLAN isolation to VXLAN Layer 3 tunnels—the evolution of Layer 2 networks is the history of "broadcast domains getting smaller and tunnels reaching further." Understanding MAC learning and STP convergence is understanding why data center networks look the way they do.

Overview

Switches automatically build forwarding paths through MAC learning, allowing Ethernet to scale to thousands of nodes without manual configuration. The Linux bridge implements the same functionality in software—containers and virtual machines connect to the bridge via veth pairs, and the bridge handles L2 forwarding. STP/RSTP prevents broadcast storms caused by loops, while VXLAN overlays a virtual L2 network on top of IP, extending the limit of 4094 VLANs to 16 million. These are the foundations of L2 virtualization in data centers and home networks.

MAC Learning: Distributed Hash Map

Each switch maintains a MAC address table (Filtering Database):

Learning and Forwarding happen simultaneously—when a frame is received, the source is learned first (updating the MAC table), then the destination is looked up. If dest == source port → do not forward (filter, prevent loops).

Linux Bridge

// net/bridge/
// The bridge is essentially a virtual switch, storing MAC→port mappings in an FDB (Forwarding Database)

struct net_bridge_fdb_entry {
    unsigned char       addr[ETH_ALEN];     // MAC
    struct net_bridge_port *dst;            // Egress port
    struct hlist_node   fdb_node;           // Hash table node
    struct hlist_node   vlan_node;          // Per-VLAN list node
    unsigned long       updated;            // Jiffies of last refresh
    unsigned long       used;               // Jiffies of last use
    u16                 vlan_id;
    // flags: sticky, added_by_user, added_by_external_learn, offloaded
};

Differences between bridge and physical switches:

1. Physical switch: MAC table is in hardware ASIC → per-port ultra-fast lookup
2. Linux bridge: MAC table is in a hash table (software) → ~10x slower than hardware, but sufficient for software switches (<10Gbps)
3. Linux bridge allows setting ageing_time:

# MAC aging time (default 300s):
cat /sys/class/net/br0/bridge/ageing_time
echo 60 > /sys/class/net/br0/bridge/ageing_time   # 1 min

# View FDB:
bridge fdb show br br0
bridge fdb add <MAC> dev eth0 master static       # Static entry (does not age)

Hairpin and Bridge-NF

bridge-nf (netfilter on bridge): iptables can run on frames forwarded by the bridge. Key paths (net/bridge/br_netfilter.c):

• NF_BR_PRE_ROUTING: Frame just arrived at bridge port
• NF_BR_FORWARD: Frame forwarding from one port to another
• NF_BR_POST_ROUTING: Frame about to leave bridge port

This allows using iptables on the bridge for transparent firewalls/filtering/logging, but with a performance overhead.

STP (802.1D)

Loop problem: Two switches connected by two cables → no mechanism → broadcast frames loop infinitely → broadcast storm → entire network paralysis.

STP Core: Forward on a spanning tree, block redundant links for standby

Election:
  1. Root Bridge: Lowest Bridge ID (priority + MAC)
  2. Each non-Root bridge: Select one Root Port (shortest path to root)
  3. Each segment: Select one Designated Port (responsible for forwarding on this segment)
  4. Remaining ports → Blocked (do not forward, only receive BPDUs)

BPDU (Bridge Protocol Data Unit):
  EtherType: 0x0026 (LLC) → DSAP/SSAP=0x42 (STP)
  Sent every 2 seconds (Hello timer)

STP BPDU Content:
  Protocol ID: 0
  Version: 0 (STP) or 2 (RSTP)
  Type: 0 (config BPDU) or 0x80 (TCN)
  Flags: TC (Topology Change) + TCA (Topology Change Ack)
  Root ID: priority (2B) + MAC (6B)
  Root Path Cost: (4B)
  Bridge ID: priority + MAC
  Port ID: priority (4 bits) + port number (12 bits)
  Message Age / Max Age / Hello Time / Forward Delay

Convergence Time:
  STP: 30-50s (Listening→Learning→Forwarding, 15s per stage)
  RSTP (802.1w): 1-3s (Proposal/Agreement mechanism)

VXLAN (RFC 7348)

Core Requirement: Data Center L2 Extension

VLANs have only a 12-bit VID → 4094 isolated domains. Large data centers need more (multi-tenant isolation). VXLAN provides a 24-bit VNI → 16 million isolated domains.

Encapsulation

VXLAN Encapsulation:
  [Outer Eth][Outer IP][Outer UDP dport=4789][VXLAN Hdr 8B][Inner Eth][Inner IP][Inner Payload]

VXLAN Header:
  Flags (1 byte):  bit 3 = I (VNI valid), bits 0-2/4-7 = reserved
  Reserved (3 bytes): Zero
  VNI (3 bytes): Virtual Network Identifier — 24-bit identifier
  Reserved (1 byte): Zero

Why use UDP:
  - ECMP: Routers can perform multi-path load balancing based on UDP src port (entropy)
  - NAT traversal: UDP can traverse NAT (unlike GRE/IPIP which require special handling)
  - Firewall friendly: dst port=4789 is fixed → easy to set ACLs

VTEP

VTEP (VXLAN Tunnel Endpoint):
  Encapsulation/Decapsulation point (Linux: vxlan device)

  Creation: ip link add vxlan100 type vxlan id 100 remote 10.0.0.2 dstport 4789 dev eth0

  Forwarding Logic:
    1. Local VM sends frame → bridge looks up MAC → target MAC not local?
    2. If MAC is on another VTEP → lookup FDB: MAC→remote VTEP IP
    3. Encapsulate: Add VXLAN header + outer IP/UDP → route to remote VTEP IP
    4. Remote: vxlan device receives → decapsulate → strip outer headers → bridge processing

  ARP Suppression (reduce VXLAN flooding):
    Multicast/Broadcast in VXLAN must be replicated to all VTEPs → bandwidth waste
    → ARP proxy at VTEP: Reply to ARP locally (no need to flood to network)
    → L2-population: Learn MAC→VTEP mapping from control plane (e.g., Open vSwitch)

References

• IEEE: 802.1D, 802.1w, 802.1Q
• RFC: 7348 (VXLAN), 7348 Appendix A (ARP suppression recommendations)
• Source Code: net/bridge/ (bridge + STP), drivers/net/vxlan/ (VXLAN driver)
• LWN: "VXLAN and overlay networks in the kernel"

Keywords: MAC learning, FDB, Linux bridge, STP, RSTP, BPDU, VXLAN, VTEP, VNI, ARP suppression, overlay network