--- title: WebRTC url: https://doc.liz6.com/en/networking/10-real-time-communication/01-WebRTC locale: en area: networking tags: - WebRTC - signaling - SDP - ICE - Trickle ICE - STUN - TURN - DTLS-SRTP - SCTP - Data Channel - GCC - simulcast - SVC - SFU - MCU - mesh - Opus - networking - real-time-communication date: 2026-06-30 modified: 2026-07-16 description: Direct audio/video transmission between browsers—WebRTC uses SDP to negotiate media parameters, ICE to punch through NATs, and DTLS-SRTP for encrypted transmission. P2P’s "no server" nature is a performance advantage and a debugging nightmare. --- # WebRTC > Direct audio/video transmission between browsers—WebRTC uses SDP to negotiate media parameters, ICE to punch through NATs, and DTLS-SRTP for encrypted transmission. P2P’s "no server" nature is a performance advantage and a debugging nightmare. ## Overview WebRTC (2011) is a browser-native real-time communication standard that enables video calls, screen sharing, and P2P data transfer without plugins. It combines several protocols: SDP for media negotiation, ICE for NAT traversal, DTLS-SRTP for media encryption, and SCTP for data channels. WebRTC connection establishment is fully asynchronous—candidates arrive and are validated one by one—while Trickle ICE significantly reduces connection time. ## Architecture Stack WebRTC Architecture Stack: From Browser API to Underlying Transport Browser JS API(RTCPeerConnection, RTCDataChannel) WebRTC C++ stack(libwebrtc, Google) SDP Media Negotiation (offer/answer) ICE NAT Traversal (STUN + TURN) DTLS Key Exchange → Derive SRTP Keys SRTP Encrypted RTP Stream SCTP Data Channel UDP (Preferred) TCP (Fallback) These five protocols form the "transport layer"—SDP/ICE handles connection negotiation, DTLS/SRTP manages media security, and SCTP handles data channels; all run over UDP, falling back to TCP only if UDP fails. ## Signaling: The Intentionally Blank Spot WebRTC **does not specify signaling**—how endpoints exchange SDP and ICE candidates is entirely up to you to implement via WebSocket/HTTP/any channel. This is intentional: the media path must be P2P, but "how two people who aren't connected yet meet" inherently requires a third party reachable by both. Signaling Facilitates Connection, Media/Data Goes Direct P2P Signaling Server (Self-hosted, Any Protocol) Exchange SDP offer/answer + ICE candidate Peer A Peer B (After Negotiation) SRTP Media / SCTP Data —— Direct P2P Signaling only handles "how endpoints find each other"; the protocol is not specified (WebSocket/HTTP both work, standard doesn't care); Once offer/answer and candidates are exchanged, media and data bypass the server entirely for direct connection. Signaling must deliver the offer/answer first before ICE has candidates to connect to—this is the "chicken-and-egg" problem of WebRTC. ## SDP Negotiation ``` Caller → Callee (Offer): v=0 o=- 1234567890 2 IN IP4 192.168.1.100 s=- t=0 0 a=group:BUNDLE audio video data ← Multiplex multiple streams on one transport m=audio 49170 UDP/TLS/RTP/SAVPF 111 103 c=IN IP4 192.168.1.100 a=rtpmap:111 opus/48000/2 a=rtcp-mux ← RTP and RTCP on the same port a=ice-ufrag:8sdf7g a=ice-pwd:asd88fg a=fingerprint:sha-256 AB:CD:... ← DTLS certificate hash Callee → Caller (Answer): m=audio 49180 UDP/TLS/RTP/SAVPF 111 ← Selected port + codec ... → Negotiation Complete → ICE Connectivity Checks Begin ``` ## ICE Each peer collects three types of candidates: 1. **host**: Local IP:port 2. **srflx** (server reflexive): Public IP:port returned by STUN server 3. **relay** (relayed): Relay address allocated by TURN server For each pair (local candidate + remote candidate), perform a connectivity check (STUN binding) → Select the best available pair (priority: host > srflx > relay, i.e., "prefer direct connection over relay"). **STUN vs TURN Division of Labor**: STUN simply "asks for your public mapping," does not relay media, and has near-zero cost; but when both ends are behind symmetric NATs and hole punching fails, you must fall back to a **TURN relay**—all media is forwarded through it, incurring high bandwidth costs. This is the only unavoidable server cost in deployment. Production experience: about 10-20% of connections ultimately fall back to TURN. **Trickle ICE**: No need to wait for all candidates to be collected before sending an offer. Candidates are sent to the peer via signaling as they are discovered, and connectivity checks are performed concurrently, compressing connection time from "waiting for the slowest candidate" to "first available pair ready"—this is the key to WebRTC's fast connection establishment. ## DTLS-SRTP DTLS handshake (TLS 1.2-like over UDP) → After handshake completion, both parties derive SRTP keys from DTLS (no additional exchange): DTLS Handshake: From ClientHello to Deriving SRTP Keys ClientHello ServerHello Certificate ServerKey- Exchange ClientKey- Exchange ChangeCipher- Spec Finished DTLS-SRTP Extension: Negotiate which SRTP cipher suite to use Both parties derive SRTP keys from DTLS master_secret SRTP keys = SRTP_KDF(DTLS master_secret) SRTP keys are no longer transmitted separately—derived directly from the master secret generated by this DTLS handshake, naturally bound to this handshake, saving an extra round of key exchange. ## Data Channel SCTP (Stream Control Transmission Protocol) layered over DTLS: ``` Compared to TCP: - Multiple streams (no HOL blocking) - Preserves message boundaries - Configurable: reliable (reliable) / partially reliable (timed reliability) Suitable for: File transfer, game state, chat ``` ## Congestion Control and Simulcast Real-time media cannot rely on packet loss + retransmission like TCP—the deadline for a single frame is only tens of milliseconds; waiting for retransmission will inevitably miss it. WebRTC uses **GCC (Google Congestion Control)**: it predicts congestion based on the **delay gradient** of RTP packet arrivals (queue growing → reduce bitrate early), rather than waiting for actual packet loss, and the sender dynamically adjusts the encoding bitrate accordingly. **Simulcast / SVC** solves heterogeneous bandwidth issues in multi-person conferences: the same camera stream is **encoded in multiple tiers simultaneously** (e.g., 180p/360p/720p), and a middle-tier server delivers the appropriate tier to each receiver based on their network—this is the prerequisite for the SFU below to "select tiers per user." ## SFU / MCU / Mesh — Multi-person Topology WebRTC is inherently P2P, but P2P does not scale for multi-person scenarios, requiring server introduction. Three topologies (with the same distribution trade-offs as in [Streaming Protocols](/networking/10-real-time-communication/02-streaming-protocols.md)): Multi-person Topology Trade-offs: Mesh / SFU / MCU Mesh N people connect pairwise → Each person uploads N-1 streams Only usable for 3-4 people, no server SFU Selective forwarding: Server only forwards, does not decode; each person uploads 1 stream, downloads as needed (with simulcast) → Mainstream solution, scalable, low server CPU (no transcoding) MCU Mixing: Server decodes all streams + composes one stream to send → Saves receiver downstream/compute, but server CPU is very expensive SFU only forwards, does not decode; it is the balance point between scalability and server cost, the mainstream solution for multi-person conferences; Mesh is only suitable for very small scales, MCU uses server compute to replace client lightness but has the highest cost. **Why WebRTC is hard to deploy on CDN like HLS**: Each connection is stateful (ICE/DTLS/SRTP context + real-time congestion feedback), not a cacheable file. Scaling can only be done via SFU cascading, not edge caching—this is the fundamental cost paid for "sub-second real-time." ## References - **webrtc.org**: webrtc.googlesource.com - **Tools**: `chrome://webrtc-internals` *Keywords: WebRTC, signaling, SDP, ICE, Trickle ICE, STUN, TURN, DTLS-SRTP, SCTP, Data Channel, GCC, simulcast, SVC, SFU, MCU, mesh, Opus*