]> Bytedance

yuanjinghao@bytedance.com

Bytedance

xiaolei.shawn@bytedance.com

Akamai Technologies

mbishop@evequefou.be

Web and Internet Transport QUIC QUIC congestion control careful resume This draft defines a new transport frame which enables consenting endpoints to share congestion control state about the network connection for various purposes. It also allows an endpoint's own congestion control state to be echoed back to it by a peer for consideration at the beginning of a future connection. About This Document Status information for this document may be found at . Discussion of this document takes place on the QUIC Working Group mailing list (), which is archived at . Subscribe at . Source for this draft and an issue tracker can be found at .

Introduction All Internet transports are required to either use a congestion control algorithm, or to constrain their rate of transmission . Most congestion control algorithms take time to ramp up the sending rate, called the "Slow-Start phase." This defines a time in which a sender intentionally uses less capacity than might be available so as to avoid or limit overshoot of the available capacity for the path. This is because any overshoot can have detrimental effects both for the current flow and for any other flows with which it shares network bottlenecks. At the same time, using less capacity than is available necessarily limits performance early in the connection. Careful Resume is a mechanism whereby remembered congestion control parameters can be validated as potentially applicable to a new connection, probed, and ultimately used to grow the congestion window more rapidly than slow-start would otherwise permit. One optimization approach is to use historical congestion information to provide the congestion algorithm with reliable input to help it exit the slow start phase. The most direct and reliable information can be samples collected during the congestion algorithm, such as the congestion window size and probe bandwidth. While clients often connect to a manageable number of servers and can retain such state, servers typically service orders of magnitude more clients and cannot feasibly retain such information. Further, servers are often deployed with many instances and attempting to coordinate the sharing of this information between them may prove impractical. Thus, for a server to implement Careful Resume, some external means of recalling its previous state is useful. This document specifies a mechanism which allows a QUIC endpoint to periodically export its congestion control state, optionally in an integrity-protected manner. This exported state is sent to the peer in a CONGESTION_DATA frame. When establishing a subsequent connection, an endpoint with persistent storage can include this data in a CONGESTION_DATA_RECALL frame in its 0-RTT or 1-RTT packets, assisting its peer to recall the information necessary to perform Careful Resume. This mechanism is comparable to HTTP cookies, , but for transport information. This data may also be useful for application-layer purposes, such as adaptive-bit-rate video. The exported information is readable by the peer and can be exposed to the application through local interfaces.

Peer Visibility The peer's viewpoint of a connection can be useful for debugging and as additional information to be considered by on-path entities such as congestion controllers and application-layer protocols. Therefore, this extension deliberately does not encrypt the data reported to the peer. Instead, the data is provided in cleartext with an optional integrity tag. If a server wishes to recall information about past connections without sharing that data with the client, this information can already be encoded in address validation tokens without requiring the cooperation of the client; see .

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. This document also uses terminology defined in and , in particular the frame layout notation from .

Transport Parameter Desire and willingness to receive the frames defined in this specification is indicated by means of the following QUIC transport parameter: The support_congestion_data value is a variable-length integer that encodes these three one-bit flags:

CONSUME (0x01):

This indicates that the sender is interested in receiving CONGESTION_DATA frames for its own uses during the current connection, independent of the receiver's ability to reuse the data in the future.

CACHE (0x02):

This indicates that the sender is willing to receive CONGESTION_DATA frames and potentially return the contents in a CONGESTION_DATA_RECALL frame on a subsequent connection.

CONSIDER (0x04):

This indicates that the sender is willing to have values it may have provided on a previous connection returned to it in a CONGESTION_DATA_RECALL frame.

All other bits MUST be set to zero when sending and MUST be ignored on receipt. The default for this parameter is 0, which indicates that the endpoint does not support CONGESTION_DATA or CONGESTION_DATA_RECALL frames. A value other than 0 indicates that the endpoint supports the indicated frame types and is willing to receive such frames on this connection. An endpoint MUST NOT send CONGESTION_DATA or CONGESTION_DATA_RECALL frames until it has received the support_congestion_data transport parameter with a non-zero value during the handshake (or during a previous handshake if 0-RTT is used). An endpoint MUST NOT send CONGESTION_DATA frames to a peer which did not set the CONSUME or CACHE flags. An endpoint MUST NOT send CONGESTION_DATA_RECALL frames to a peer which did not set the CONSIDER flag. An endpoint that receives a frame for which it has not indicated support via the transport parameter MUST terminate the connection with an error of type PROTOCOL_VIOLATION.

Network Statistics Each network statistic is structured as a TLV: This structure includes the following fields:

Type:

Indicates the statistic being offered, encoded as a variable-length integer.

Length:

The length of the Value field in bytes, encoded as a variable-length integer.

Value:

A Type-specific value carrying the payload of the indicated statistic.

This specification defines a number of initial statistics, but additional statistics can be added by registering a value in the appropriate registry (see ). An implementation MUST ignore any statistics it cannot understand, but MAY decline to return protected statistics to a peer if it cannot verify that it is willing to share the contained information. A receiver may get a message with multiple occurrences of a particular TLV value. If the values are identical, the reciever SHOULD ignore them. If they differ, and one of the values is protected by an integrity tag, the receiver SHOULD treat this as an attack and close the connection. If none of the instances are integrity-protected, the receiver MAY ignore them, use only one of the instances, or close the connection as it determines to be most appropriate. In the sub-section below, only the names are used; the numeric value that appears in the protocol is defined in .

Timestamp The Timestamp statistic indicates the time at which the sender generated this frame. This can be used on future connections to determine whether the recalled statistics are recent enough to be useful.

Path Tuple The Path Tuple statistic encodes an identifier of the path on which these statistics were generated. Knowing the connection addresses as seen from the peer's perspective can be useful for a number of scenarios (e.g., ). The reciever MAY use this to to compare similarity of the previous endpoints to those of a new connection will when deciding if returned statistics might be applicable to a new connection. The structure of the value is: It contains the following fields:

Local IP:

The sender's IP address on the associated transport path, encoded as either 4 or 16 bytes depending on IP version.

Local Port:

The sender's port number on the associate transport path, encoded as a two-byte integer.

Remote IP:

The receiver's IP address as observed by the sender on the associated transport path, encoded as either 4 or 16 bytes depending on IP version.

Remote Port:

The receiver's port number as observed by the sender on the associated transport path

The IP version being used can be inferred from the length of the payload.

Slow Start Status The Slow Start Status statistic indicates whether the sender's congestion controller is in the Slow Start phase. The value is a single byte, set to 0x00 if the sender is not in Slow Start and 0x01 if the sender is in Slow Start.

Network Type The Network Type statistic indicates the sender's understanding of its network access medium, encoded as a single byte value. Note that this is purely advisory, since applications will only be aware of the local network at best. The defined values are at .

Maximum Congestion Window The Maximum Congestion Window statistic indicates the maximum congestion window (CWD) sampled within the observation period, measured in bytes. It is encoded as a variable-lenth integer. CWD is a key metric in congestion control algorithms, as it represents the amount of unacknowledged data that a sender can have in flight on the network. A larger CWD generally allows for a higher sending rate.

Maximum In-Flight Data The Maximum In-Flight Data statistic indicates the maximum in-flight data sampled within the observation period, measured in bytes. It is encoded as a variable-length integer. This represents the highest number of bytes sent by the sender that have not yet been acknowledged by the receiver during the measurement period. This metric provides insight into the actual amount of data in transit at any given time, which can be useful for diagnosing network performance issues.

Smoothed RTT The Smoothed RTT statistic indicates the most recent smoothed Round-Trip Time (RTT) within the observation period, measured in milliseconds. It is encoded as a variable-length integer. It is calculated as defined in . RTT is a key metric for congestion control, estimating the time it takes for a packet to travel from the sender to the receiver and back. The smoothed RTT calculation accounts for both the latest RTT and a historical average, helping to dampen the effect of short-term network fluctuations.

Minimum RTT The Minimum RTT statistic indicates the minimum RTT sampled within the observation period, measured in milliseconds. It is encoded as a variable-length integer, This metric provides a baseline for the best-case network latency observed during the measurement period. A low minimum RTT can indicate a stable and efficient network path, while a high one might suggest persistent latency issues.

RTT Variance The RTT Variance statistic indicates the most recent RTT deviation within the observation period, measured in milliseconds. It is encoded as a variable-length integer. It is calculated as defined in . This metric quantifies the variability of the RTT, providing insight into network jitter and stability. A low RTT variance suggests a consistent network path, while a high value indicates significant fluctuations in network latency.

Latest Bandwidth The Latest Bandwidth statistic indicates the current raw throughput of the connection, measured in kilobits per second (kbps). It is encoded as a variable-length integer. This metric represents the instantaneous sending capacity as perceived by the sender and is a crucial input for congestion control algorithms.

Maximum Bandwidth The Maximum Bandwidth statistic indicates the maximum raw throughput sampled within the observation period, measured in kbps. It is encoded as a variable-length integer. This metric provides a view of the peak network capacity observed during the measurement period, which can be useful for understanding the best possible performance on the current network path.

Throughput The Throughput statistic indicates the useful throughput for data, excluding retransmissions, within the observation period, measured in kbps. It is encoded as a variable-length integer. This metric is a measure of the effective data rate delivered to the receiver's application layer. It isolates the useful data rate, providing a more accurate measure of application-level performance than the raw sending rate, which includes retransmitted data.

Send Rate The Send Rate statistic indicates the sending rate for all data, including retransmissions, within the observation period, measured in kbps. It is encoded as a variable-length integer. This metric provides a measure of the total data rate at which the sender is transmitting data. It is useful for understanding the sender's total load on the network.

Receive Rate The Receive Rate statistic indicates the receiving rate within the observation period in kbps. It is encoded as a variable-length integer. This metric measures the total rate at which the receiver is acknowledging data, including both new data and retransmissions. It is useful for understanding the receiver's perspective on the data flow and can be used to compare against the sender's rate to identify potential bottlenecks.

Input Rate The Input Rate statistic indicates the input bitrate from the application layer within the observation period in kbps. It is encoded as a variable-length integer. This metric represents the rate at which data is being provided to the receiving application. It gives an end-to-end view of the application data flow.

Loss Rate The Loss Rate statistic indicates the arithmetic mean of the packet loss rate samples within the observation period. The value is expressed as a percentage at 0.1% resolution within a range of 0 to 1000 inclusive. It is encoded as a variable-length integer. This metric provides a clear measure of the quality of the network path, as it quantifies the proportion of packets that are sent but not received. A high loss rate often indicates network congestion or instability.

Buffer Length The Buffer Length statistic indicates the current amount of data cached by the QUIC connection layer buffer when the observation frame is generated in bytes. It is encoded as a variable-length integer. This metric reflects the amount of data the sender is holding in its buffer before transmission, which can be an important indicator of the sender's ability to keep up with the application's sending rate and can also be a sign of network congestion.

Frame Types

CONGESTION_DATA Frames CONGESTION_DATA frames (type TBD1) provide a list of Network Statistics values which the sender chooses to share about the state of the network connection from its viewpoint. CONGESTION_DATA frames contain the following fields:

Protected Count:

A variable-length integer representing the number of Network Statistics in the Protected Network Statistics field.

Protected Network Statistics:

A sequence of Network Statistics objects whose length is given by the Protected Count.

Integrity Tag:

A message integrity check, as described in . This field is absent if Protected Count is zero. While this document provides some examples, the format of the check MUST be treated as opaque by the receiver.

Unprotected Count:

A variable-length integer representing the number of Network Statistics in the Unprotected Network Statistics field.

Unprotected Network Statistics:

A sequence of Network Statistics objects whose length is given by Unprotected Count.

CONGESTION_DATA frames are not retransmittable, though a loss event might trigger the generation of a new CONGESTION_DATA frame; see . CONGESTION_DATA frames can be sent at any point in the connection after 0-RTT or 1-RTT keys have been established, though useful data will likely not be available until at least one round-trip has occurred. If a CONGESTION_DATA frame is received in an Initial or Handshake packet, it MUST be treated as a connection error of type PROTOCOL_VIOLATION.

Sending Network Statistics If an endpoint wishes to receive CONGESTION_DATA_RECALL frames on future connections with the peer and the peer has set the CACHE flag, the endpoint MAY send CONGESTION_DATA frames containing the values it wishes to recall in future connections in the Protected Network Statistics field. It MAY send additional CONGESTION_DATA frames when these values have changed significantly and it wishes to update the stored values, or when a previous CONGESTION_DATA frame is declared lost. If the peer has set the CONSUME flag, an endpoint SHOULD send CONGESTION_DATA frames periodically throughout the connection's lifetime. However, an implementation SHOULD NOT send additional CONGESTION_DATA frames if the connection has been idle since the last such frame was sent. In addition to any values the endpoint placed in Protected Network Statistics, the endpoint includes such other values as it is willing to provide in the Unprotected Network Statistics field. If an endpoint sends multiple CONGESTION_DATA frames, it is not required to include the same set of network statistics in each frame. For example, some statistics are more useful sent at a regular frequency, while others only need to be sent if they have changed significantly from the last value known to have been received. However, as the server does not control which CONGESTION_DATA will be cached, it SHOULD include the same Protected Network Statistics fields in each frame.

Integrity Tag The integrity tag is calculated over the Protected Count and Protected Network Statistics field by the sender. This field is a variable-length set of bytes, whose format is known only to the sender. The purpose of this field is to provide suitable assurance to the sender that, when the statistics are later sent back to it through the CONGESTION_DATA_RECALL frame, that they have not been modified. This is often called a "message authentication code" (MAC). To emphasize that only a portion of a message is protected, this document does not use that term. The algorithm for generating and verifying an integrity tag MAY depend on the ordering of the Protected fields although some implementations may perform a simple canonicalization by sorting the statistics by type identifier. Because of this, receivers SHOULD NOT modify the content or ordering of any of the Protected statistics in any way, unless they have out-of-band knowledge that it is safe to do so. If the server has a nonce or other private material, it can hash that with the incoming Protected fields and use that as the outgoing Integrity tag. This can be either a simple hash of both parts, or the HMAC keyed hash can be used. Being able to change algorithms without large-scale protocol modifications is important. Servers may wish to use a fixed-number of leading bytes to indicate the algorithm they are using. It is also a best practice to generate new private data periodically, while still allowing old messages to be validated. To handle this, it is a good idea to use a fixed number of secondary bytes to act as a key or nonce identifier. A sample implementation is provided in .

CONGESTION_DATA_RECALL Frames CONGESTION_DATA_RECALL frames (type TBD2) contain a list of Network Statistics values which the sender received from the recipient during a previous connection. This frame SHOULD be sent as early as possible in the connection once 0-RTT or 1-RTT keys are available. While the frame MAY be sent at any point in the connection, if it arrives after the recipient has exited slow-start the values it contains will likely not be useful. CONGESTION_DATA_RECALL frames contain the following fields:

Protected Count:

A variable-length integer representing the number of Network Statistics in the Protected Network Statistics field, received in a CONGESTION_DATA frame from the peer on a previous connection.

Protected Network Statistics:

A sequence of Network Statistics objects whose length is given by Protected Count, received in a CONGESTION_DATA frame from the peer on a previous connection.

Integrity Tag:

The integrity tag, received in a CONGESTION_DATA frame from the peer on a previous connection. See .

If a CONGESTION_DATA_RECALL frame is received in an Initial or Handshake packet, it MUST be treated as a connection error of type PROTOCOL_VIOLATION.

Recalling Network Statistics Upon receipt of a CONGESTION_DATA_RECALL frame, an endpoint computes the expected Integrity Tag value as in . If the Integrity Tag value does not match, the frame is ignored. If the tag is acceptable, the endpoint takes the network statistics contained in the frame and incorporates them into its congestion control strategy. For example, it might exit the Reconnaissance Phase of Careful Resume . The specifics of how this is done are outside the scope of this extension. Endpoints MUST NOT process more than one CONGESTION_DATA_RECALL frame on a connection. Subsequent CONGESTION_DATA_RECALL frames MUST be ignored without processing, regardless of whether the first frame was valid.

Security Considerations Clients choosing to return network statistics to a server provide a potential tracking mechanism. However, this tracking mechanism provides no additional capabilities to a server beyond those already enabled by the address validation tokens defined in . While address validation tokens are opaque and can contain any data the server might wish to recall, the statistics being transported by this mechanism are visible to the clients. Clients can inspect the values to ensure that nothing objectionable is being saved; implementations MAY choose not to send CONGESTION_DATA_RECALL packets which contain statistics they cannot interpret. Clients SHOULD NOT send CONGESTION_DATA_RECALL packets on connections where they would not have sent an Address Validation token if one were available. A client MAY also decide not to send the packet if the length of the integrity tag does not correspond to a digest length and a few additional bytes. This is admittedly inelegant. and could be avoided if the format of the tag were publicly defined, and an IANA registry for tag algorithms defined. Clients SHOULD discard stored network statistics when other potential tracking mechanisms (e.g. HTTP Cookies) are cleared by the user.

IANA Considerations IANA is requested to take the following actions, replacing "ThisRFC" with the RFC number when assigned.

New QUIC Transport Parameters Entry Add a new entry to the "QUIC Transport Parameters" registry with the following values:

Field Name	Value
Value	TBD
Parameter Name	support_congestion_data
Status	permanent
Specification	ThisRFC
Date	TBD
Change Controller	IETF
Contact	quic@ietf.org
Notes	None

New QUIC Network Type Registry A new registry "QUIC Network Type" is created with the following fields:

Value:

Numeric value

Meaning:

Brief textual description

Reference:

A pointer to the defining document

The registration policy for this registry is "Specification Required" as described in . The initial value of the registry is as follows:

Value	Meaning	Reference
0x00	Reserved	ThisRFC
0x01	Wire/Ethernet	ThisRFC
0x02	Reserved	ThisRFC
0x03	WLAN	ThisRFC
0x04	2G Mobile	ThisRFC
0x05	3G Mobile	ThisRFC
0x06	4G Mobile	ThisRFC
0x07	5G Mobile	ThisRFC

New QUIC Network Statistics Registry A new "QUIC Network Statistics" registry is created. It follows the registration policies defined in . In addition to the fields described in that section, permanent registrations MUST include the following fields:

Type:

The type of statistic, as described in ThisRFC, Section .

Name:

A short name for the field.

The initial value of the table is:

Type	Name
0xc8	Timestamp
0xca	Path Tuple
0xcb	Slow Start Status
0xcc	Network Type
0xcd	Maximum Congestion Window
0xce	Maximum In-Flight Data
0xcf	Smoothed RTT
0xd0	Minimum RTT
0xd1	RTT Variance
0xd2	Latest Bandwidth
0xd3	Maximum Bandwidth
0xd4	Throughput
0xd5	Send Rate
0xd6	Receive Rate
0xd7	Input Rate
0xd8	Loss Rate
0xd9	Buffer Length

These fields are permanent, and therefore all have the following values for the common fields:

Field Name	Value
Status	permanent
Specification	ThisRFC
Date	TBD
Change Controller	IETF
Contact	quic@ietf.org
Notes	None

References Normative References This document defines the core of the QUIC transport protocol. QUIC provides applications with flow-controlled streams for structured communication, low-latency connection establishment, and network path migration. QUIC includes security measures that ensure confidentiality, integrity, and availability in a range of deployment circumstances. Accompanying documents describe the integration of TLS for key negotiation, loss detection, and an exemplary congestion control algorithm. The User Datagram Protocol (UDP) provides a minimal message-passing transport that has no inherent congestion control mechanisms. This document provides guidelines on the use of UDP for the designers of applications, tunnels, and other protocols that use UDP. Congestion control guidelines are a primary focus, but the document also provides guidance on other topics, including message sizes, reliability, checksums, middlebox traversal, the use of Explicit Congestion Notification (ECN), Differentiated Services Code Points (DSCPs), and ports. Because congestion control is critical to the stable operation of the Internet, applications and other protocols that choose to use UDP as an Internet transport must employ mechanisms to prevent congestion collapse and to establish some degree of fairness with concurrent traffic. They may also need to implement additional mechanisms, depending on how they use UDP. Some guidance is also applicable to the design of other protocols (e.g., protocols layered directly on IP or via IP-based tunnels), especially when these protocols do not themselves provide congestion control. This document obsoletes RFC 5405 and adds guidelines for multicast UDP usage. In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This document describes how Transport Layer Security (TLS) is used to secure QUIC. This document describes loss detection and congestion control mechanisms for QUIC. Many protocols make use of points of extensibility that use constants to identify various protocol parameters. To ensure that the values in these fields do not have conflicting uses and to promote interoperability, their allocations are often coordinated by a central record keeper. For IETF protocols, that role is filled by the Internet Assigned Numbers Authority (IANA). To make assignments in a given registry prudently, guidance describing the conditions under which new values should be assigned, as well as when and how modifications to existing values can be made, is needed. This document defines a framework for the documentation of these guidelines by specification authors, in order to assure that the provided guidance for the IANA Considerations is clear and addresses the various issues that are likely in the operation of a registry. This is the third edition of this document; it obsoletes RFC 5226. This document defines the core of the QUIC transport protocol. QUIC provides applications with flow-controlled streams for structured communication, low-latency connection establishment, and network path migration. QUIC includes security measures that ensure confidentiality, integrity, and availability in a range of deployment circumstances. Accompanying documents describe the integration of TLS for key negotiation, loss detection, and an exemplary congestion control algorithm. Informative References Thales Alenia Space Orange University of Aberdeen University of Aberdeen Private Octopus Inc. This document specifies a cautious method for Internet transports that enables fast startup of congestion control for a wide range of connections, known as "Careful Resume". It reuses a set of computed congestion control parameters that are based on previously observed path characteristics between the same pair of transport endpoints. These parameters are saved, allowing them to be later used to modify the congestion control behaviour of a subsequent connection. It describes assumptions and defines requirements for how a sender utilises these parameters to provide opportunities for a connection to more rapidly get up to speed and rapidly utilise available capacity. It discusses how use of this method impacts the capacity at a shared network bottleneck and the safe response that is needed after any indication that the new rate is inappropriate. This document defines the HTTP Cookie and Set-Cookie header fields. These header fields can be used by HTTP servers to store state (called cookies) at HTTP user agents, letting the servers maintain a stateful session over the mostly stateless HTTP protocol. Although cookies have many historical infelicities that degrade their security and privacy, the Cookie and Set-Cookie header fields are widely used on the Internet. This document obsoletes RFC 2965. [STANDARDS-TRACK] Session Traversal Utilities for NAT (STUN) is a protocol that serves as a tool for other protocols in dealing with Network Address Translator (NAT) traversal. It can be used by an endpoint to determine the IP address and port allocated to it by a NAT. It can also be used to check connectivity between two endpoints, and as a keep-alive protocol to maintain NAT bindings. STUN works with many existing NATs, and does not require any special behavior from them. STUN is not a NAT traversal solution by itself. Rather, it is a tool to be used in the context of a NAT traversal solution. This is an important change from the previous version of this specification (RFC 3489), which presented STUN as a complete solution. This document obsoletes RFC 3489. [STANDARDS-TRACK] This document describes HMAC, a mechanism for message authentication using cryptographic hash functions. HMAC can be used with any iterative cryptographic hash function, e.g., MD5, SHA-1, in combination with a secret shared key. The cryptographic strength of HMAC depends on the properties of the underlying hash function. This memo provides information for the Internet community. This memo does not specify an Internet standard of any kind

Sample Integrity Tag implementation This section is not normative. We define an Integrity Tag format that consists of a one-byte algorithm identifier, two bytes of private nonce, and a digest. Based on the choice of algorithm, this is a 32-byte SHA256 digest. The entire tag is therefore 35 bytes long. The digest will be computed over the nonce, five bytes of 0x01, and the wire-format of the protected fields. In this example, we will use our third nonce, which is the ASCII values of "quic-new-frame": The Network Statistics values are: Which have the following wire representation: The value for the Integrity tag is represented by the following psuedo-code:

Acknowledgments TODO acknowledge.