WebRTC

WebRTC is a set of APIs and protocols that enable browsers to collaborate with other real-time communication endpoints for audio, video, and application data. It can connect directly, relay via TURN, or connect to media servers such as SFUs/MCUs; "not going through a server" is not a prerequisite for using WebRTC.

It is easier to troubleshoot by breaking a call down into stages: the application first locates the peer and obtains authorization, negotiates media and transport parameters, verifies available paths and endpoint keys, and then continuously encodes, transmits, and plays media. Successful signaling, successful ICE, receiving media, and successful playback are distinct milestones.

Protocol Stack: DTLS Handshake Does Not Mean All Media is Encapsulated in DTLS

Media uses the secure form of RTP/RTCP, which is SRTP/SRTCP. The DTLS handshake negotiates the SRTP protection configuration and exports direction keys and salts via the exporter. Media packets are then protected in SRTP format; not every RTP packet is wrapped in a DTLS application-data record. Application data channels use SCTP over DTLS. DTLS-SRTP RFC 5764, Data Channels RFC 8831

The exporter has defined labels, contexts, and output organization; it cannot be implemented as arbitrarily executing SRTP_KDF(secret) on some DTLS secret. Nor can the DTLS 1.2 ClientKeyExchange/ChangeCipherSpec timing be treated as a universal handshake for all versions.

SDP is a session description format carrying information such as media type, codec, direction, ICE credentials, and DTLS fingerprints. It does not directly transport video frames; ICE is a candidate path checking and selection mechanism, not another layer of IP headers around every business packet.

Signaling and Negotiation: Control state to accept subsequent candidates

WebRTC does not mandate that applications use a specific general-purpose signaling transport; HTTPS, WebSocket, or existing session services can be used. However, signaling still requires identity authentication, room permissions, message ordering, and session association. If an attacker can replace the peer description or certificate fingerprint, "media is encrypted" alone is not sufficient to prove communication with the correct party. Security Architecture RFC 8827

The basic offer/answer process can be understood in four state changes: A sets a local offer and passes it to B; B sets the remote offer, generates, and sets a local answer; A receives and sets the remote answer. Candidate collection can overlap with these asynchronous steps. If a peer candidate arrives before the applicable remote description, the application must queue it and add it to the correct media section/ICE generation once the description becomes effective.

Both parties initiating offers simultaneously causes negotiation collisions; the application needs a consistent conflict resolution strategy, such as distinguishing the yielding party and performing the corresponding rollback. Not all setRemoteDescription failures can be attributed to network issues. WebRTC API and Negotiation

ICE: Candidate Addresses Are Not Proof of Connectivity

Host candidates come from the local network, server-reflexive candidates reflect the mapping observed by STUN, relay candidates come from TURN allocation, and peer-reflexive candidates can be discovered during checks. Browsers may use mDNS names to represent certain host addresses for privacy, so candidates in logs are not always bare IPs.

Both ends pair compatible candidates and verify reachability via STUN checks with ICE credentials. The controlling agent nominates and selects candidate pairs according to rules. Receiving an external IP or obtaining a TURN allocation does not mean that candidate has become the active path. ICE RFC 8445

Trickle ICE allows candidates to be sent and checked as they are collected, reducing the wait time for full collection. It requires handling candidate end-of-candidates signals and associating them with each ICE generation; old candidates cannot be unconditionally added to a new session after a restart. Trickle ICE RFC 8838

TURN can accept client connections via UDP, TCP, or TLS; the choice depends on support and network conditions. "If UDP fails, make SRTP run over any TCP socket" is not a universal fallback interface exposed by browsers. TURN control connections, allocated addresses, and peer media paths should be observed separately; see Ports and NAT Traversal for basic mechanisms.

When connectivity degrades, a brief disconnected state does not necessarily mean permanent failure; persistent failure may require an ICE restart to renegotiate paths with new ICE credentials. Calling restart locally without passing the new description to the peer will not complete recovery.

Trust Boundaries: TURN, SFU, and Frame-Level Encryption

TURN relays typically only forward encrypted data between endpoints and do not hold the SRTP content keys between these two endpoints. A standard SFU acts as the WebRTC peer for each client, terminating the corresponding DTLS-SRTP, and then selects media streams to forward to other endpoints. "SFU does not decode video" means it usually does not reconstruct pixels and re-encode; it does not mean it is inherently unable to obtain plaintext media.

If the requirement is that the SFU also cannot interpret media content, end-to-end protection must be added at the media frame level, addressing member key distribution, join/leave events, and key rotation. SFrame defines the corresponding frame protection constructs; it does not automatically provide complete group member management, nor does it hide all routing metadata. SFrame RFC 9605

Latency is a Budget: Sometimes Retransmission is Worth It, Sometimes There is No Time

Real-time media can use packet loss feedback, RTX retransmission, FEC, or packet loss concealment; retransmission is not prohibited. The choice depends on how much time remains until the playback deadline for the missing packet, and the network and queuing costs of retransmission. A 30 fps frame interval is about 33 ms, but this does not mean the total deadline from sending to playback is only 33 ms. WebRTC RTP Usage RFC 8834

For example, capture and encoding take 35 ms, send queuing takes 10 ms, one-way network latency is 25 ms, jitter buffer is 60 ms, and decoding and display take 25 ms, totaling a budget of 155 ms. The jitter buffer absorbs network variations but increases latency; reducing it to 20 ms saves budget but makes late packets more likely to miss playback.

stages_ms = {'capture_encode': 35, 'send_queue': 10, 'network': 25,
             'jitter_buffer': 60, 'decode_display': 25}
assert sum(stages_ms.values()) == 155

def retry_fits(slack, feedback_to_sender, retransmit_to_receiver, decode):
    return feedback_to_sender + retransmit_to_receiver + decode < slack
assert retry_fits(80, 25, 25, 10)
assert not retry_fits(40, 25, 25, 10)
print('Retransmission availability window under given playback budget passed')

Congestion control constrains sending rates based on feedback, the encoder adjusts bitrate, resolution, or frame rate, and the pacer controls sending rhythm. GCC is a common implementation architecture, not a WebRTC specification requirement that all browsers must use the same algorithm. An expanded send queue consumes latency budget, even if average throughput looks sufficient.

When can real-time data afford a retransmission?

With50ms network roundtrip and10ms decode, reducing slack80→40 turns useful recovery into a late arrival. This is remaining budget after detecting loss, not the frame interval.

$$t_{feedback}+t_{retransmit}+t_{decode}<t_{playout}-t_{now}$$

Multi-party Topology: Where Does the Saved Upstream Traffic Go?

In Mesh, everyone sends to the other N−1 people; in SFU, the sender uploads one or more encoding layers, and the server replicates them to receivers based on subscription relationships; MCU typically decodes, composes, and re-encodes, allowing receivers to accept the composed result. The scale limit depends on bitrate, devices, and layout; there is no protocol-defined "Mesh supports at most four people."

Assume six people each send 1.5 Mbit/s video, and everyone watches the other five, excluding audio and encapsulation: In Mesh, each person's upstream is 7.5 Mbit/s; in a single-layer SFU, each person's upstream is 1.5 Mbit/s, but the SFU's total downstream is still 45 Mbit/s. SFU reduces duplicate uploads from clients but does not magically eliminate all reception copies.

n, rate = 6, 1.5
assert n * (n - 1) // 2 == 15
assert (n - 1) * rate == 7.5
assert n * rate == 9
assert n * (n - 1) * rate == 45
# Upstream for three independent simulcast encodings is the sum, not just the highest one.
assert sum([0.2, 0.6, 1.5]) == 2.3
print('Mesh, SFU, and multi-bitrate upload traffic accounting passed')

Simulcast generates multiple encoding representations simultaneously; SVC uses scalable encoding layers and their dependencies; the two cannot be directly equated to "three independent video files." SFU layer selection must also consider keyframes and dependencies; it cannot arbitrarily discard base data required by upper layers. Simulcast RFC 8853

WebRTC media connections hold congestion, key, and transport states, making them unsuitable for direct use as HLS static segment caching; scaling can adopt a combination of regional media nodes, selective subscription, distribution layers, and protocol transcoding, and should not be limited to "only SFU cascading."

Why does SFU egress grow while client uplink shrinks?

With six watching everyone, SFU receives6 uploads and sends30 copies. One subscription each reduces egress to6 copies without changing per-client upload. Savings belong to specific links in this replication structure.

DataChannel: Ordering and Reliability Are Two Dimensions

DataChannel preserves message boundaries, can be configured as ordered, and supports partial reliability policies like maxRetransmits or maxPacketLifeTime; these two limits cannot be set simultaneously. Unordered does not mean unreliable, and reliable does not mean all channels share a single application message order. Data Channels

File transfer requires reliability and application-level chunking/checksum recovery; game position updates may care more about new states and allow dropping old packets. Multiple SCTP streams can isolate some ordering constraints, but they still share congestion resources; large messages, implementation interleaving support, and send buffers will affect other streams. It is incorrect to write "multiple streams mean no head-of-line blocking."

Apply backpressure during sending based on bufferedAmount, low-watermark events, and the negotiated maximum message size. send() accepting data only means it has entered the sending process, not that the peer has persisted the business data; if business confirmation is needed, establish message IDs and acknowledgment rules separately.

Determining Stalls from getStats

First, associate statistics objects with the same transport, candidate pair, and SSRC, then compare time differences; cumulative counters cannot be directly treated as current bitrate. When objects change, counters reset, or fields are unavailable, re-establish the baseline. WebRTC Stats

For example, the bytesSent of the same outbound-rtp object increases from 1,000,000 to 1,500,000 within two seconds, corresponding to a 2 Mbit/s rate for this statistic. It does not automatically include the network card rate including every layer of IP/UDP/TURN overhead. On the receiving side, jitterBufferDelay / jitterBufferEmittedCount can be used to calculate the cumulative average queuing time; when observing interval changes, take the difference of the two cumulative quantities and then divide.

These pieces of evidence break down "WebRTC not working" into verifiable stages. Room IDs, negotiation generations, and timestamps in application logs should correspond to statistics, rather than just saving a final connectionState.