]> Google LLC

asedeno@google.com

Transport Multiplexed Application Substrate over QUIC Encryption quic http datagram VPN proxy tunnels masque http-ng layer-2 ethernet This document describes how to proxy Ethernet frames in HTTP. This protocol is similar to IP proxying in HTTP, but for Layer 2 instead of Layer 3. More specifically, this document defines a protocol that allows an HTTP client to create Layer 2 Ethernet tunnel through an HTTP server to an attached physical or virtual Ethernet segment. About This Document The latest revision of this draft can be found at . Status information for this document may be found at . Discussion of this document takes place on the MASQUE Working Group mailing list (), which is archived at . Subscribe at . Source for this draft and an issue tracker can be found at .

Introduction HTTP provides the CONNECT method (see ) for creating a TCP tunnel to a destination, a similar mechanism for UDP , and an additional mechanism for IP . However, these mechanisms can't carry layer 2 frames without further encapsulation inside of IP, for instance with EtherIP , GUE or L2TP , which imposes an additional MTU cost. This document describes a protocol for exchanging Ethernet frames with an HTTP server. Either participant in the HTTP connection can then relay Ethernet frames to and from a local or virtual interface, allowing the bridging of two Ethernet broadcast domains to establish a Layer 2 VPN. This can simplify connectivity to network-connected appliances that are configured to only interact with peers on the same Ethernet broadcast domain. This protocol supports all existing versions of HTTP by using HTTP Datagrams . When using HTTP/2 or HTTP/3 , it uses HTTP Extended CONNECT as described in and . When using HTTP/1.x , it uses HTTP Upgrade as defined in . This protocol necessarily involves additional framing overhead. When possible, users should use higher-level proxying protocols, such as connect-ip or connect-udp.

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. In this document, we use the term "Ethernet proxy" to refer to the HTTP server that responds to the Ethernet proxying request. The term "client" is used in the HTTP sense; the client constructs the Ethernet proxying request. If there are HTTP intermediaries (as defined in ) between the client and the Ethernet proxy, those are referred to as "intermediaries" in this document. The term "Ethernet proxying endpoints" refers to both the client and the Ethernet proxy. This document uses terminology from . Where this document defines protocol types, the definition format uses the notation from . This specification uses the variable-length integer encoding from . Variable-length integer values do not need to be encoded in the minimum number of bytes necessary. Note that, when the HTTP version in use does not support multiplexing streams (such as HTTP/1.1), any reference to "stream" in this document represents the entire connection.

Configuration of Clients Clients are configured to use Ethernet proxying over HTTP via a URI Template . The URI Templates used by this protocol do not require any variables; implementations or extensions MAY specify their own. URI Templates specified for this protocol MAY use the well-known location registered by this document. Examples are shown below: An implementation that supports connecting to different Ethernet segments might add a "vlan-identifier" variable to specify which segment to connect to. The optionality of variables needs to be considered when defining the template so that variables are either self-identifying or possible to exclude in the syntax. How valid values for such variables are communicated to the client is not a part of this protocol. Hypothetical examples are shown below:

Tunnelling Ethernet over HTTP To allow negotiation of a tunnel for Ethernet over HTTP, this document defines the "connect-ethernet" HTTP upgrade token. The resulting Ethernet tunnels use the Capsule Protocol (see ) with HTTP Datagrams in the format defined in . To initiate an Ethernet tunnel associated with a single HTTP stream, a client issues a request containing the "connect-ethernet" upgrade token. By virtue of the definition of the Capsule Protocol (see ), Ethernet proxying requests do not carry any message content. Similarly, successful Ethernet proxying responses also do not carry any message content. Ethernet proxying over HTTP MUST be operated over TLS or QUIC encryption, or another equivalent encryption protocol, to provide confidentiality, integrity, and authentication.

Ethernet Proxy Handling Upon receiving an Ethernet proxying request:

• If the recipient is configured to use another HTTP proxy, it will act as an intermediary by forwarding the request to another HTTP server. Note that such intermediaries may need to re-encode the request if they forward it using a version of HTTP that is different from the one used to receive it, as the request encoding differs by version (see below).
• Otherwise, the recipient will act as an Ethernet proxy. The Ethernet proxy can choose to reject the Ethernet proxying request or establish an Ethernet tunnel.

The lifetime of the Ethernet tunnel is tied to the Ethernet proxying request stream. A successful response (as defined in Sections and ) indicates that the Ethernet proxy has established an Ethernet tunnel and is willing to proxy Ethernet frames. Any response other than a successful response indicates that the request has failed; thus, the client MUST abort the request.

HTTP/1.1 Request When using HTTP/1.1 , an Ethernet proxying request will meet the following requirements:

• The method SHALL be "GET".
• The request SHALL include a single Host header field containing the host and optional port of the Ethernet proxy.
• The request SHALL include a Connection header field with value "Upgrade" (note that this requirement is case-insensitive as per ).
• The request SHALL include an Upgrade header field with value "connect-ethernet".

An Ethernet proxying request that does not conform to these restrictions is malformed. The recipient of such a malformed request MUST respond with an error and SHOULD use the 400 (Bad Request) status code. For example, if the client is configured with the URI Template "https://example.org/.well-known/masque/ethernet/" and wishes to open an Ethernet tunnel, it could send the following request.

Example HTTP/1.1 Request

HTTP/1.1 Response The server indicates success by replying with a response that conforms to the following requirements:

• The HTTP status code on the response SHALL be 101 (Switching Protocols).
• The response SHALL include a Connection header field with value "Upgrade" (note that this requirement is case-insensitive as per ).
• The response SHALL include a single Upgrade header field with value "connect-ethernet".
• The response SHALL meet the requirements of HTTP responses that start the Capsule Protocol; see .

If any of these requirements are not met, the client MUST treat this proxying attempt as failed and close the connection. For example, the server could respond with:

Example HTTP/1.1 Response

HTTP/2 and HTTP/3 Requests When using HTTP/2 or HTTP/3 , Ethernet proxying requests use HTTP Extended CONNECT. This requires that servers send an HTTP Setting as specified in and and that requests use HTTP pseudo-header fields with the following requirements:

• The :method pseudo-header field SHALL be "CONNECT".
• The :protocol pseudo-header field SHALL be "connect-ethernet".
• The :authority pseudo-header field SHALL contain the authority of the IP proxy.
• The :path and :scheme pseudo-header fields SHALL NOT be empty. Their values SHALL contain the scheme and path from the configured URL; see .

An Ethernet proxying request that does not conform to these restrictions is malformed; see and . For example, if the client is configured with the URI Template "https://example.org/.well-known/masque/ethernet/" and wishes to open an Ethernet tunnel, it could send the following request.

Example HTTP/2 or HTTP/3 Request

HTTP/2 and HTTP/3 Responses The server indicates success by replying with a response that conforms to the following requirements:

• The HTTP status code on the response SHALL be in the 2xx (Successful) range.
• The response SHALL meet the requirements of HTTP responses that start the Capsule Protocol; see .

If any of these requirements are not met, the client MUST treat this proxying attempt as failed and abort the request. As an example, any status code in the 3xx range will be treated as a failure and cause the client to abort the request. For example, the server could respond with:

Example HTTP/2 or HTTP/3 Response

Context Identifiers The mechanism for proxying Ethernet in HTTP defined in this document allows future extensions to exchange HTTP Datagrams that carry different semantics from Ethernet frames. Some of these extensions could augment Ethernet payloads with additional data or compress Ethernet frame header fields, while others could exchange data that is completely separate from Ethernet payloads. In order to accomplish this, all HTTP Datagrams associated with Ethernet proxying requests streams start with a Context ID field; see . Context IDs are 62-bit integers (0-2⁶²-1). Context IDs are encoded as variable-length integers; see . The Context ID value of 0 is reserved for Ethernet payloads, while non-zero values are dynamically allocated. Non-zero even-numbered Context-IDs are client allocated, and odd-numbered Context IDs are proxy-allocated. The Context ID namespace is tied to a given HTTP request; it is possible for a Context ID with the same numeric value to be simultaneously allocated in distinct requests, potentially with different semantics. Context IDs MUST NOT be re-allocated within a given HTTP request but MAY be allocated in any order. The Context ID allocation restrictions to the use of even-numbered and odd-numbered Context IDs exist in order to avoid the need for synchronization between endpoints. However, once a Context ID has been allocated, those restrictions do not apply to the use of the Context ID; it can be used by either the client or the Ethernet proxy, independent of which endpoint initially allocated it. Registration is the action by which an endpoint informs its peer of the semantics and format of a given Context ID. This document does not define how registration occurs. Future extensions MAY use HTTP header fields or capsules to register Context IDs. Depending on the method being used, it is possible for datagrams to be received with Context IDs that have not yet been registered. For instance, this can be due to reordering of the packet containing the datagram and the packet containing the registration message during transmission.

HTTP Datagram Payload Format When associated with Ethernet proxying request streams, the HTTP Datagram Payload field of HTTP Datagrams (see ) has the format defined in . Note that when HTTP Datagrams are encoded using QUIC DATAGRAM frames, the Context ID field defined below directly follows the Quarter Stream ID field which is at the start of the QUIC DATAGRAM frame payload.

Ethernet Proxying HTTP Datagram Format

The Ethernet Proxying HTTP Datagram Payload contains the following fields:

Context ID:

A variable-length integer that contains the value of the Context ID. If an HTTP/3 datagram which carries an unknown Context ID is received, the receiver SHALL either drop that datagram silently or buffer it temporarily (on the order of a round trip) while awaiting the registration of the corresponding Context ID.

Payload:

The payload of the datagram, whose semantics depend on value of the previous field. Note that this field can be empty.

Ethernet frames are encoded using HTTP Datagrams with the Context ID set to zero. When the Context ID is set to zero, the Payload field contains a full Layer 2 Ethernet Frame (from the MAC destination field until the last byte of the Frame check sequence field), as defined by IEEE 802.3, with support for optional IEEE 802.1Q tagging (see ).

Ethernet Frame Handling This document defines a tunnelling mechanism that is conceptually an Ethernet link. Implementations might need to handle some of the responsibilities of an Ethernet switch or bridge if they do not delegate them to another implementation such as a kernel. Those responsibilities are beyond the scope of this document, and include, but are not limited to, the handling of broadcast packets and multicast groups, or the local termination of PAUSE frames. If an Ethernet proxying endpoint fails to deliver a frame to an underlying Ethernet segment, the endpoint MUST drop the frame.

Examples Ethernet proxying in HTTP enables the bridging of Ethernet broadcast domains. These examples are provided to help illustrate some of the ways in which Ethernet proxying can be used.

Remote Access L2VPN The following example shows a point to point VPN setup where a client appears to be connected to a remote Layer 2 network.

L2VPN Tunnel Setup HOST 1 | +--------------------+ L2 | Layer 2 | | Client |<--Layer 2 Tunnel---| Proxy +------------+---> HOST 2 | +--------------------+ | Broadcast | +--------+ +--------+ Domain +---> HOST 3 ]]>

In this case, the client connects to the Ethernet proxy and immediately can start sending ethernet frames to the attached broadcast domain.

VPN Full-Tunnel Example

Site-to-Site L2VPN The following example shows a site-to-site VPN setup where a client joins a locally attached broadcast domain to a remote broadcast domain through the Proxy.

Site-to-site L2VPN Example HOST 1 | Broadcast Broadcast | HOST B <---+ Domain Domain +---> HOST 2 | | HOST C <---+ +---> HOST 3 ]]>

In this case, the client connects to the Ethernet proxy and immediately can start relaying Ethernet frames from its attached broadcast domain to the proxy. The difference between this example and is limited to what the Client is doing with the the tunnel; the exchange between the Client and the Proxy is the same as in above.

Performance Considerations When the protocol running inside the tunnel uses congestion control (e.g., or ), the proxied traffic will incur at least two nested congestion controllers. Implementers will benefit from reading the guidance in . By default the tunneling of Ethernet frames MUST NOT assume that the carried Ethernet frames contain congestion controlled traffic. Optimizations for traffic flows carried within the Ethernet Frames MAY be done in cases where the content of the Ethernet Frames have been identified to be congestion controlled traffic. Some implementations might find it benefitial to maintain a small buffer of frames to be sent through the tunnel to smooth out short term variations and bursts in tunnel capacity. Any such buffer MUST be limited, and when exceeded any Ethernet frames that would otherwise be buffered MUST be dropped. When the protocol running inside the tunnel uses loss recovery (e.g., or ) and the outer HTTP connection runs over TCP, the proxied traffic will incur at least two nested loss recovery mechanisms. This can reduce performance, as both can sometimes independently retransmit the same data. To avoid this, Ethernet proxying SHOULD be performed over HTTP/3 to allow leveraging the QUIC DATAGRAM frame.

MTU and Frame Ordering Considerations When using HTTP/3 with the QUIC Datagram extension , Ethernet frames can be transmitted in QUIC DATAGRAM frames. Since DATAGRAM frames cannot be fragmented, they can only carry Ethernet frames up to a given length determined by the QUIC connection configuration and the Path MTU (PMTU). Implementations MAY rely on 's use of to probe and discover the PMTU over the connection's lifetime, and adjust any associated interface MTU as needed. Furthermore, the UDP packets carrying these frames could be reordered by the network. When using HTTP/1.1 or HTTP/2, and when using HTTP/3 without the QUIC Datagram extension , Ethernet frames are transmitted in DATAGRAM capsules as defined in . DATAGRAM capsules are transmitted reliably over an underlying stream, maintaining frame order, though they could be split across multiple QUIC or TCP packets. The trade-off between supporting a larger MTU and avoiding fragmentation should be considered when deciding what mode(s) to operate in. Implementations SHOULD NOT intentionally reorder Ethernet frames, but are not required to provide guaranteed in-order delivery. If in-order delivery of Ethernet frames is required, DATAGRAM capsules can be used.

IEEE 802.1Q VLAN tagging While the protocol as described can proxy Ethernet frames with IEEE 802.1Q VLAN tags, it is RECOMMENDED that individual VLANs be proxied in separate connections, and VLAN tags be stripped and applied by the Ethernet proxying endpoints as needed.

Security Considerations There are risks in allowing arbitrary clients to establish a tunnel to a Layer 2 network. Bad actors could abuse this capability to attack hosts on that network that they would otherwise be unable to reach. HTTP servers that support Ethernet proxying SHOULD restrict its use to authenticated users. Depending on the deployment, possible authentication mechanisms include mutual TLS between IP proxying endpoints, HTTP-based authentication via the HTTP Authorization header , or even bearer tokens. Proxies can enforce policies for authenticated users to further constrain client behavior or deal with possible abuse. For example, proxies can rate limit individual clients that send an excessively large amount of traffic through the proxy. Users of this protocol may send arbitrary Ethernet frames through the tunnel, including frames with arbitrary source MAC addresses. This could allow impersonation of other hosts, poisoning of ARP , NDP and CAM (Content Addressable Memory) tables, and cause a denial of service to other hosts on the network. These are the same attacks available to an arbitrary client with physical access to the network. An implementation that is intended for point-to-site connections might limit clients to a single source MAC address, or Ethernet proxying endpoints might be configured to limit forwarding to pre-configured MAC addresses, though such filtering is outside the scope of this protocol. Dynamic signalling or negotiation of MAC address filtering is left to future extensions. This protocol is agnostic to where on the Ethernet segment a gateway for higher-level routing might be located. A client may connect via an Ethernet proxy and discover an existing gateway on the Ethernet segment, supply a new gateway to the Ethernet segment, both, or neither.

IANA Considerations

HTTP Upgrade Token This document will request IANA to register "connect-ethernet" in the HTTP Upgrade Token Registry maintained at <>.

Value:

connect-ethernet

Description:

Proxying of Ethernet Payloads

Expected Version Tokens:

None

References:

This document

Updates to the MASQUE URI Suffixes Registry This document will request IANA to register "ethernet" in the MASQUE URI Suffixes Registry maintained at <>, created by . New MASQUE URI Suffixes

Path Segment	Description	Reference
ethernet	Ethernet Proxying	This Document

References Normative References The Hypertext Transfer Protocol (HTTP) is a stateless application-level protocol for distributed, collaborative, hypertext information systems. This document specifies the HTTP/1.1 message syntax, message parsing, connection management, and related security concerns. This document obsoletes portions of RFC 7230. This specification describes an optimized expression of the semantics of the Hypertext Transfer Protocol (HTTP), referred to as HTTP version 2 (HTTP/2). HTTP/2 enables a more efficient use of network resources and a reduced latency by introducing field compression and allowing multiple concurrent exchanges on the same connection. This document obsoletes RFCs 7540 and 8740. The QUIC transport protocol has several features that are desirable in a transport for HTTP, such as stream multiplexing, per-stream flow control, and low-latency connection establishment. This document describes a mapping of HTTP semantics over QUIC. This document also identifies HTTP/2 features that are subsumed by QUIC and describes how HTTP/2 extensions can be ported to HTTP/3. The Hypertext Transfer Protocol (HTTP) is a stateless application-level protocol for distributed, collaborative, hypertext information systems. This document describes the overall architecture of HTTP, establishes common terminology, and defines aspects of the protocol that are shared by all versions. In this definition are core protocol elements, extensibility mechanisms, and the "http" and "https" Uniform Resource Identifier (URI) schemes. This document updates RFC 3864 and obsoletes RFCs 2818, 7231, 7232, 7233, 7235, 7538, 7615, 7694, and portions of 7230. This document specifies the Transmission Control Protocol (TCP). TCP is an important transport-layer protocol in the Internet protocol stack, and it has continuously evolved over decades of use and growth of the Internet. Over this time, a number of changes have been made to TCP as it was specified in RFC 793, though these have only been documented in a piecemeal fashion. This document collects and brings those changes together with the protocol specification from RFC 793. This document obsoletes RFC 793, as well as RFCs 879, 2873, 6093, 6429, 6528, and 6691 that updated parts of RFC 793. It updates RFCs 1011 and 1122, and it should be considered as a replacement for the portions of those documents dealing with TCP requirements. It also updates RFC 5961 by adding a small clarification in reset handling while in the SYN-RECEIVED state. The TCP header control bits from RFC 793 have also been updated based on RFC 3168. This document describes how to proxy UDP in HTTP, similar to how the HTTP CONNECT method allows proxying TCP in HTTP. More specifically, this document defines a protocol that allows an HTTP client to create a tunnel for UDP communications through an HTTP server that acts as a proxy. This document describes how to proxy IP packets in HTTP. This protocol is similar to UDP proxying in HTTP but allows transmitting arbitrary IP packets. More specifically, this document defines a protocol that allows an HTTP client to create an IP tunnel through an HTTP server that acts as an IP proxy. This document updates RFC 9298. This document describes the Layer Two Tunneling Protocol (L2TP). [STANDARDS-TRACK] This document describes "version 3" of the Layer Two Tunneling Protocol (L2TPv3). L2TPv3 defines the base control protocol and encapsulation for tunneling multiple Layer 2 connections between two IP nodes. Additional documents detail the specifics for each data link type being emulated. [STANDARDS-TRACK] This document describes HTTP Datagrams, a convention for conveying multiplexed, potentially unreliable datagrams inside an HTTP connection. In HTTP/3, HTTP Datagrams can be sent unreliably using the QUIC DATAGRAM extension. When the QUIC DATAGRAM frame is unavailable or undesirable, HTTP Datagrams can be sent using the Capsule Protocol, which is a more general convention for conveying data in HTTP connections. HTTP Datagrams and the Capsule Protocol are intended for use by HTTP extensions, not applications. This document defines a mechanism for running the WebSocket Protocol (RFC 6455) over a single stream of an HTTP/2 connection. The mechanism for running the WebSocket Protocol over a single stream of an HTTP/2 connection is equally applicable to HTTP/3, but the HTTP-version-specific details need to be specified. This document describes how the mechanism is adapted for HTTP/3. In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This document defines the core of the QUIC transport protocol. QUIC provides applications with flow-controlled streams for structured communication, low-latency connection establishment, and network path migration. QUIC includes security measures that ensure confidentiality, integrity, and availability in a range of deployment circumstances. Accompanying documents describe the integration of TLS for key negotiation, loss detection, and an exemplary congestion control algorithm. A URI Template is a compact sequence of characters for describing a range of Uniform Resource Identifiers through variable expansion. This specification defines the URI Template syntax and the process for expanding a URI Template into a URI reference, along with guidelines for the use of URI Templates on the Internet. [STANDARDS-TRACK] This memo defines a path prefix for "well-known locations", "/.well-known/", in selected Uniform Resource Identifier (URI) schemes. In doing so, it obsoletes RFC 5785 and updates the URI schemes defined in RFC 7230 to reserve that space. It also updates RFC 7595 to track URI schemes that support well-known URIs in their registry. This document defines an extension to the QUIC transport protocol to add support for sending and receiving unreliable datagrams over a QUIC connection. This document specifies Datagram Packetization Layer Path MTU Discovery (DPLPMTUD). This is a robust method for Path MTU Discovery (PMTUD) for datagram Packetization Layers (PLs). It allows a PL, or a datagram application that uses a PL, to discover whether a network path can support the current size of datagram. This can be used to detect and reduce the message size when a sender encounters a packet black hole. It can also probe a network path to discover whether the maximum packet size can be increased. This provides functionality for datagram transports that is equivalent to the PLPMTUD specification for TCP, specified in RFC 4821, which it updates. It also updates the UDP Usage Guidelines to refer to this method for use with UDP datagrams and updates SCTP. The document provides implementation notes for incorporating Datagram PMTUD into IETF datagram transports or applications that use datagram transports. This specification updates RFC 4960, RFC 4821, RFC 6951, RFC 8085, and RFC 8261. The purpose of this RFC is to present a method of Converting Protocol Addresses (e.g., IP addresses) to Local Network Addresses (e.g., Ethernet addresses). This is an issue of general concern in the ARPA Internet Community at this time. The method proposed here is presented for your consideration and comment. This is not the specification of an Internet Standard. This document specifies the Neighbor Discovery protocol for IP Version 6. IPv6 nodes on the same link use Neighbor Discovery to discover each other's presence, to determine each other's link-layer addresses, to find routers, and to maintain reachability information about the paths to active neighbors. [STANDARDS-TRACK] Informative References Quantonium Independent Microsoft This specification describes Generic UDP Encapsulation (GUE), which is a scheme for using UDP to encapsulate packets of different IP protocols for transport across layer 3 networks. By encapsulating packets in UDP, specialized capabilities in networking hardware for efficient handling of UDP packets can be leveraged. GUE specifies basic encapsulation methods upon which higher level constructs, such as tunnels and overlay networks for network virtualization, can be constructed. GUE is extensible by allowing optional data fields as part of the encapsulation, and is generic in that it can encapsulate packets of various IP protocols. The User Datagram Protocol (UDP) provides a minimal message-passing transport that has no inherent congestion control mechanisms. This document provides guidelines on the use of UDP for the designers of applications, tunnels, and other protocols that use UDP. Congestion control guidelines are a primary focus, but the document also provides guidance on other topics, including message sizes, reliability, checksums, middlebox traversal, the use of Explicit Congestion Notification (ECN), Differentiated Services Code Points (DSCPs), and ports. Because congestion control is critical to the stable operation of the Internet, applications and other protocols that choose to use UDP as an Internet transport must employ mechanisms to prevent congestion collapse and to establish some degree of fairness with concurrent traffic. They may also need to implement additional mechanisms, depending on how they use UDP. Some guidance is also applicable to the design of other protocols (e.g., protocols layered directly on IP or via IP-based tunnels), especially when these protocols do not themselves provide congestion control. This document obsoletes RFC 5405 and adds guidelines for multicast UDP usage.

Acknowledgments Much of the initial version of this draft borrows heavily from . The author would like to thank Alexander Chernyakhovsky and David Schinazi for their advice while preparing this document, and Etienne Dechamps for useful discussion on the subject material. Additionally, Mirja Kühlewind, Magnus Westerlund, and Martin Thompson provided valuable feedback on the document.