MQTT

MQTT transports application messages using a publish/subscribe model: devices publish to topics on the Broker, and the Broker distributes messages to clients based on subscription filters. Transmission relies on a reliable, ordered, bidirectional connection, typically TCP or WebSocket; it is not the same packet format running directly over arbitrary unreliable UDP.

Using a temperature sensor as an example, periodic readings can tolerate occasional loss of a single message, whereas "open the valve" commands require confirmation of execution. MQTT QoS manages protocol message delivery, while business commands require command IDs, validity periods, and execution acknowledgments. These two aspects should be designed separately. MQTT 5.0

Topic: Names, Filters, and Permissions Are Handled Separately

Topic Names are case-sensitive, with / delimiting levels; leading/trailing slashes and empty levels are significant. PUBLISH does not use + or # wildcards; in subscription Filters, + matches a single level, and # must be at the end and covers zero or more levels. Ordinary top-level wildcards do not match system topics starting with $.

Naming should consider both routing and authorization. For example, tenant/a/device/17/state clearly defines tenant and device scope, but a client declaring this string does not prove it belongs to tenant a; the Broker's authentication and ACL must constrain the publishable and subscribable scope. Mosquitto Access Control Configuration

QoS: Each Hop Manages Its Own Packet State

Delivery is negotiated/executed separately from Publisher to Broker and from Broker to Subscriber. Outbound QoS is constrained by the original publish QoS, the maximum allowed by the subscription, and server capabilities; you cannot infer that every subscriber receives QoS 2 just because the source used QoS 2.

QoS 2 does not require the business database and MQTT state to share a transaction. If the application changes inventory but crashes before recording the consumption result, how to handle recovery depends on the client, application commit boundaries, and message deduplication. For device commands, event IDs and execution results should be reliably associated; Packet Identifiers can be reused after the exchange is complete and cannot serve as permanent deduplication keys.

In MQTT 5, PUBACK/PUBREC with a reason code indicating rejection cannot be treated as normal success. A successful acknowledgment from the Broker to the publisher usually only indicates the result of that hop's processing, proving neither that all subscribers have received the message nor that the valve has actuated.

What to Resend on Reconnection

MQTT 5 protocol retransmissions primarily target QoS 1/2 PUBLISH and PUBREL messages that are unacknowledged in a resumed session; you should not arbitrarily add a timeout-based periodic retransmission algorithm for still-active connections. When the protocol retransmits PUBLISH, the DUP flag indicates potential duplication, but a new publish with DUP=0 may also carry the same business event content. MQTT 5 §4.4

Even if the network is unreachable for a long time, the session has expired, or the administrator has deleted the state, the protocol does not guarantee eventual delivery. QoS semantics must be combined with state retention and error results; "no duplication, no loss" cannot be used as an unconditional operational promise.

Packet Length: Two-Byte Fixed Header Is Only the Minimum Case

The first byte of an MQTT packet contains the type and fixed flags, followed by Remaining Length, which uses a one-to-four-byte variable-length integer to represent the length of the rest; if it exceeds 127, it no longer occupies just one byte. Topic string length is encoded in bytes, and Packet Identifier is two bytes, appearing only in corresponding message types/QoS.

Below is the construction of an MQTT 5 QoS 1 PUBLISH: topic home/temp, Packet Identifier 7, no properties, payload 23.5. The first byte is 0x32, Remaining Length is 18, and the total packet is 20 B; this does not yet include TCP/TLS/IP overhead.

import struct

def vbi(n):
    if not 0 <= n <= 268435455:
        raise ValueError('Out of MQTT variable integer range')
    result = bytearray()
    while True:
        digit = n % 128
        n //= 128
        result.append(digit | (0x80 if n else 0))
        if not n:
            return bytes(result)

assert vbi(127) == b'\x7f'
assert vbi(128) == b'\x80\x01'
assert vbi(16384) == b'\x80\x80\x01'
assert vbi(268435455) == b'\xff\xff\xff\x7f'
topic = b'home/temp'
variable_and_payload = struct.pack('!H', len(topic)) + topic + b'\x00\x07' + b'\x00' + b'23.5'
packet = b'\x32' + vbi(len(variable_and_payload)) + variable_and_payload
assert len(variable_and_payload) == 18 and len(packet) == 20
assert packet.hex() == '32120009686f6d652f74656d7000070032332e35'
print('MQTT 5 PUBLISH encoding and Remaining Length boundary passed')

Topic Alias values are fixed at two bytes; MQTT 5 property identifiers and property lengths incur additional overhead; it is not a "1/2 byte alias". Alias mappings are managed per connection and direction; after reconnection, you cannot assume the Broker remembers old aliases. The first mapping establishment still requires the topic name; subsequent messages can omit the repeated name within allowed limits.

Session, Retained Value, and Will Lifecycles

Clean Start and Session Expiry

MQTT 5 uses Clean Start to decide whether to start a new session upon connection, and Session Expiry Interval to decide how long to save it after disconnection. Clean Start=1 can be paired with a non-zero expiry time, indicating clearing the old session this time but retaining the new session afterwards; Clean Start=0 does not guarantee successful restoration; check Session Present in CONNACK.

Resuming a session should use a stable Client ID and restore the client's own incomplete delivery state. Two devices using the same Client ID simultaneously may knock each other offline, causing continuous reconnection; randomly generating IDs for each connection would destroy the subscriptions and queue relationships you originally hoped to restore.

MQTT 3.1.1's Clean Session and MQTT 5's two fields cannot be directly mixed. 3.1.1 also allows the server to accept zero-length Client IDs under specified conditions; it is not "only 5.0 allows server-assigned IDs". MQTT 5 has the Assigned Client Identifier property to return the assigned value to the client. MQTT 3.1.1

Retained: A Snapshot for Latecomers

Messages published with RETAIN can become the retained value for that topic, which subsequent subscribers can obtain according to their subscription options; new values replace old ones. You cannot rely on retained messages to replay every historical temperature reading. A zero-length payload with RETAIN is used to clear the retained message for that topic, but current subscribers may still receive this publish.

Retained state does not belong to a specific subscribing client's session; clearing that client's session does not equal clearing retained values for all topics. MQTT 5's Message Expiry can also limit message lifespan, so "the server once retained it" does not mean new subscribers can always obtain it.

Will: Offline Inference Is Not Immediate Fault Detection

Clients register their Will in CONNECT. After conditions such as abnormal termination are triggered, the Broker publishes according to rules; normal disconnection usually cancels the will, and MQTT 5 also defines a DISCONNECT reason code to explicitly request publishing the Will.

Will Delay can allow time for brief reconnections; if the session resumes in time, erroneous offline notifications can be avoided. If the session ends earlier, you cannot assume the will will definitely wait for the full configured delay. Online status messages should carry timestamps or generation numbers, allowing subscribers to identify stale notifications, rather than treating a retained offline message as an eternal fact.

If the Session Is Still Alive, Are Commands and Retained Values Still Valid?

At the 20-second mark, the session is still alive, but the 10-second command has already expired; you cannot execute it just because the subscription was restored. If Session Expiry is shortened to 15 seconds, the Will does not need to wait the full 30 seconds; retained messages still exist according to their own expiry.

Keep Alive, Message Expiry, and Slow Subscribers

Keep Alive is the interval constraint for the client to send MQTT control packets; having existing PUBLISH or other control packets satisfies the activity requirement; it is not sending PINGREQ unconditionally every K seconds. PINGREQ/PINGRESP are used when there is no other traffic; the server closes the connection according to rules if no control packets are received within approximately 1.5 times the effective Keep Alive; a value of 0 disables this mechanism. MQTT 5 servers can also specify the value to use via Server Keep Alive.

Session Expiry, Message Expiry, and Will Delay each manage different objects. Saving a session for one day does not mean a control command valid for ten seconds can continue to execute after the device comes back online the next day. Device applications should also check command validity periods and not rely solely on transmission queue configurations.

Receive Maximum limits the number of parallel deliveries for corresponding QoS 1/2; it does not limit all QoS 0 traffic, nor is it a total byte memory cap. Maximum Packet Size controls single packet size; Broker queues, client buffers, disk persistence, and publish speeds still need their own constraints.

Assuming a backlog of 50 messages per second, each actually occupying 1 KiB, ten minutes offline amounts to approximately 29.3 MiB, not including index and protocol state overhead. Setting a very long lifespan for a session without limiting the queue will not automatically become safe offline storage.

Shared Subscription and Business Request/Response

$share/workers/jobs/+ indicates a subscription for the shared group workers, where one member of the group receives the corresponding distribution, suitable for workload sharing. Which specific member receives it is determined by the server policy; the protocol does not guarantee round-robin, fair processing duration, or business completion order across all workers.

MQTT 5's Response Topic and Correlation Data can help applications implement request/response, but they do not automatically provide synchronous RPC timeouts, authorization, or transactions. The requester must first prepare to receive responses, assign business IDs to requests, and consider the branch "operation completed but response lost" after timeout.

From Connection Success to Business Validity

Broker authentication solves who can connect, topic ACL solves what can be published or subscribed to, and payload schema with business state solves whether the message can be executed. Client ID is a session identifier and should not directly serve as device identity without trusted binding.

Persistence and queue configurations in implementations like Mosquitto affect restart recovery and resource limits; protocol QoS does not automatically guarantee that the disk has been persisted according to your disaster recovery standards before each acknowledgment. Mosquitto Configuration

When troubleshooting, record Client ID, connection generation, topic, business event ID, QoS, reason code, and Session Present. If a publish has no response, first check connection and flow control; if publish acknowledgment is received but there is no business result, continue to check subscription matching, permissions, expiry, and consumption submission, rather than simply changing QoS from 1 to 2.