]> Red Hat

United States of America bcodding@redhat.com

Red Hat

United States of America smayhew@redhat.com

Oracle Corporation

United States of America chuck.lever@oracle.com

Transport Network File System Version 4 This document specifies a protocol for conveying Remote Procedure (RPC) messages via QUIC version 1 connections. It requires no revision to application RPC protocols or the RPC protocol itself. Note This note is to be removed before publishing as an RFC. Discussion of this draft occurs on the NFSv4 working group mailing list, archived at . Working Group information is available at . Submit suggestions and changes as pull requests at . Instructions are on that page.

Introduction The QUIC version 1 protocol is a secure, reliable connection-oriented network transport described in . Its features include integrated transport layer security, multiple independent streams over each connection, fast reconnecting, and advanced packet loss recovery and congestion avoidance mechanisms. Open Network Computing Remote Procedure Call (often shortened to "RPC") is a Remote Procedure Call protocol that runs over a variety of network transports . RPC implementations so far use UDP , TCP , or RDMA . This document specifies how to transport RPC messages over QUIC version 1.

Motivation For a New RPC Transport Viewed at a moderate distance, RPC over QUIC provides a similar feature set as RPC over TCP with TLS (as described in ). However, a closer look reveals some essential benefits of using QUIC transports:

• Even though the QUIC protocol utilizes the same set of encryption algorithms as TLSv1.3, the QUIC record protocol encrypts nearly the entire transport layer header and authenticates each IP packet. Advanced traffic analysis which was possible with TLS on TCP is no longer possible. QUIC protects against transport packet spoofing and downgrade attacks better than TLS on TCP.
• Because many real IP networks are oversubscribed, packet loss due to momentary link or switch saturation continues to be likely even on well-maintained data center-quality network fabrics. The QUIC protocol utilizes packet loss recovery and congestion avoidance features that are lacking in TCP. Because TCP protocol design has ossified, it is unlikely to gain these improvements. QUIC is more extensible than TCP, meaning future improvements in this area can be designed and deployed without application disruption.
• Further, because QUIC handles packet loss on a per-stream rather than a per-connection basis, spreading RPC traffic across multiple streams enables workloads to continue largely unperturbed while packet recovery proceeds.
• The QUIC protocol is designed to facilitate secure and automatic transit of firewalls. Firewall transparency is a foundational feature of NFSv4 (which is built on RPC).

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

RPC-over-QUIC Framework RPC is first and foremost a message-passing protocol. This section covers the implementaion details of exchanging RPC messages over QUICv1. Readers should already be familiar with the fundamentals of ONC RPC (see ).

Establishing a Connection When a network host wishes to send RPC requests to a remote service via QUICv1, it must first find an established QUICv1 connection, or establish a new one. For the purpose of explanation, the peer that initiates QUICv1 connection establishment is referred to as an "RPC client" peer. The peer that passively accepts the connection is referred to a an "RPC server" peer. QUICv1 connections are not defined by the classic 5-tuple (IP proto, source address, source port, destination address, and destination port). Rather, each connection is defined by its connection ID. Thus, if the IP address of either peer should change, or a NAT/PAT binding and the source UDP port changes, the receiver can still recognize an ingress QUICv1 packet to belong to an established connection. As a result, due to network conditions or administrative actions, an RPC-over-QUICv1 connection can be replaced (a reconnect event) or migrated (a failover event) without interrupting the operation of an upper layer protocol such as RPC-over-QUICv1. A more complete discussion can be found in .

RPC Service Discovery Because all QUICv1 traffic goes to a single port and is demultiplexed on the receiving peer, the usual precursor step of the RPC client sending an rpcbind query to the RPC server before an RPC-over-QUICv1 connection is to be established is unneeded. Notice that we now have to invent a new way to perform proper RPC service discovery. The historical method of discovering an RPC service is to send an rpcbind query to a well-known port number (port 111). The result of an rpcbind query contains the IP address and port number of the requested RPC service. For QUICv1, the port number is part of the QUICv1 connection, and is the same for all QUICv1 traffic directed to that network host.

Transport Layer Security As part of establishing a QUICv1 connection, the two connecting peers authenticate to each other and choose encryption parameters to establish a confidential channel of communication. All traffic on one QUICv1 connection is thus bound to the authentication identity that was used during connection establishment. RPC-over-QUICv1 provides peer authentication and encryption services using a framework based on Transport Layer Security (TLS). Ergo, RPC-over-QUICv1 inherently fulfills many of the requirements of . The details of QUIC's use of TLS are specified in . In particular:

• With QUICv1, security at the transport layer is always enabled. Thus, there is no need or use for the STARTTLS mechanism described in .
• The discussion in about the opportunistic use of TLS does not apply to RPC-over-QUICv1.
• The peer authentication requirements in do apply to RPC-over-QUICv1.
• The PKIX Extended Key Usage values defined in are also valid for use with RPC-over-QUICv1.
• The ALPN defined in is also used for RPC-over-QUICv1.

QUIC Streams RPC-over-QUICv1 connections are mediated entirely by each peer's RPC layer and are not visible to RPC applications. An RPC client establishes a QUICv1 connection whenever there are application RPC requests to be executed. QUICv1 provides a "stream" abstraction, described in . A QUICv1 connection carries one or more streams. Once a QUICv1 connection has been established, either connection peer may create a stream. Typically, the RPC client peer creates the first stream on a connection. Unless explicitly specified, when RPC upper layer protocol specifications refer to a "connection", for RPC-over-QUICv1, this is a stream. As an example, an NFSv4.1 BIND_CONN_TO_SESSION operation binds to a QUICv1 stream. As another example, to signify the loss of an RPC request, an NFS server closes the QUICv1 stream that received that request, but it does close not the encompassing QUICv1 connection. In terms of TI-RPC semantic labels, a QUICv1 stream behaves as a "tpi_cots_ord" transport: connection-oriented and in order.

RPC Message Framing RPC-over-QUICv1 uses only bidirectional streams. When a connection peer creates a QUICv1 stream, that peer's stream endpoint is referred to as a "Requester", and MUST emit only RPC Call messages on that stream. The other endpoint is referred to as a "Responder", and MUST emit only RPC Reply messages on that stream. Receivers MUST silently discard RPC messages whose direction field does not match its Requester/Responder role. Requesters and Responders match RPC Calls to RPC Replies using the XID carried in each RPC message. Responders MUST send RPC Replies on the same stream on which they received the matching RPC Call. Because each QUICv1 stream is an ordered-byte stream, an RPC-with-QUICv1 stream carries only a sequence of complete RPC messages. Although data from multiple streams can be interleaved on a single QUICv1 connection, RPC messages MUST NOT be interleaved on one stream. Just as with RPC on a TCP socket, each RPC message is an ordered sequence of one or more records on a single stream. Such RPC records bear no relationship to QUIC stream frames; in fact, stream frames as defined in are not visible to RPC endpoints. Each RPC record begins with a four-octet record marker. A record marker contains the count of octets in the record in its lower 31 bits, and a flag that indicates whether the record is the last record in the RPC message in the highest order bit. See for a comparison with TCP record markers.

Stream Count Given the above definition of RPC message framing on QUICv1 streams, it is possible for a Requester to create a stream, send one RPC Call, receive one RPC Reply, then destroy the stream. That is a degenerate case that might be used for simple RPC-based protocols like rpcbind. For protocols that carry a significant workload, this style of stream allocation would generate needless overhead. Moreover, stream identifiers cannot be re-used on one QUICv1 connection, so eventually a QUICv1 connection can no longer create a new stream for each RPC XIDs. And finally, a connection peer may advertise a max_streams value that is significantly lower than 2 ^ 60. Instead, RPC clients may create as many streams as is convenient to their design, but should reuse streams on each connection efficiently. For example, an RPC client could allocate a handful of streams per CPU core to reduce contention for the streams and their associated data structures. Or, an RPC client could create a set of streams whose count is the same as the number of slots in an NFSv4 session. Servers that implement RPC-over-QUICv1 must be mindful that each additional stream amounts to incremental overhead. RPC servers MAY deny the creation of new streams if an RPC client already has many active streams. RPC clients need to be prepared for this.

RPC Authentication Flavors Streams in a QUIC connection may use different RPC authentication flavors. One stream might use RPC_AUTH_UNIX, while at the same time, another might use RPCSEC_GSS. GSS mutual (peer) authentication occurs only after a QUIC connection has been established. It is a separate process, and is unchanged by the use of QUIC. Additionally, authentication of RPCSEC_GSS users is unchanged by the use of QUIC. RPCSEC_GSS can optionally perform per-RPC integrity or confidentiality protection. When operating within a QUIC connection, these GSS services become largely redundant. An RPC implementation capable of concurrently using QUIC and RPCSEC_GSS MUST use Generic Security Service Application Program Interface (GSS-API) channel binding, as defined in , to determine when an underlying transport already provides a sufficient degree of confidentiality. RPC-over-QUIC implementations MUST provide the "tls-exporter" channel binding type, as defined in .

Implementation Status This section is to be removed before publishing as an RFC. This section records the status of known implementations of the protocol defined by this specification at the time of posting of this Internet-Draft, and is based on a proposal described in . The description of implementations in this section is intended to assist the IETF in its decision processes in progressing drafts to RFCs. Please note that the listing of any individual implementation here does not imply endorsement by the IETF. Furthermore, no effort has been spent to verify the information presented here that was supplied by IETF contributors. This is not intended as, and must not be construed to be, a catalog of available implementations or their features. Readers are advised to note that other implementations may exist. There are no known implementations of RPC-over-QUICv1 as described in this document.

Security Considerations Readers should refer to the discussion of QUIC's transport layer security in .

IANA Considerations RFC Editor: In the following subsections, please replace RFC-TBD with the RFC number assigned to this document. Furthermore, please remove this Editor's Note before this document is published.

Netids for RPC-over-QUIC Each new RPC transport is assigned one or more RPC "netid" strings. These strings are an rpcbind string naming the underlying transport protocol, appropriate message framing, and the format of service addresses and ports, among other things. This document requests that IANA allocate the following "Netid" registry strings in the "ONC RPC Netid" registry, as defined in : These netids MUST be used for any transport satisfying the requirements described in this document. The "quic" netid is to be used when IPv4 addressing is employed by the underlying transport, and "quic6" for IPv6 addressing. IANA should use this document (RFC-TBD) as the reference for the new entries.

ALPN Identifier for SunRPC on QUICv1 RPC-over-QUICv1 utilizes the same ALPN string as RPC-with-TLS does, as defined in : This document requests that a reference to (RFC-TBD) be added to the SunRPC protocol entry in the "TLS Application-Layer Protocol Negotiation (ALPN) Protocol IDs" registry.

References Normative References This document describes the binding protocols used in conjunction with the ONC Remote Procedure Call (ONC RPC Version 2) protocols. [STANDARDS-TRACK] The concept of channel binding allows applications to establish that the two end-points of a secure channel at one network layer are the same as at a higher layer by binding authentication at the higher layer to the channel at the lower layer. This allows applications to delegate session protection to lower layers, which has various performance benefits. This document discusses and formalizes the concept of channel binding to secure channels. [STANDARDS-TRACK] This document describes the Open Network Computing (ONC) Remote Procedure Call (RPC) version 2 protocol as it is currently deployed and accepted. This document obsoletes RFC 1831. [STANDARDS-TRACK] This document lists IANA Considerations for Remote Procedure Call (RPC) Network Identifiers (netids) and RPC Universal Network Addresses (uaddrs). This document updates, but does not replace, RFC 1833. [STANDARDS-TRACK] This document defines the core of the QUIC transport protocol. QUIC provides applications with flow-controlled streams for structured communication, low-latency connection establishment, and network path migration. QUIC includes security measures that ensure confidentiality, integrity, and availability in a range of deployment circumstances. Accompanying documents describe the integration of TLS for key negotiation, loss detection, and an exemplary congestion control algorithm. This document describes how Transport Layer Security (TLS) is used to secure QUIC. This document defines a channel binding type, tls-exporter, that is compatible with TLS 1.3 in accordance with RFC 5056, "On the Use of Channel Bindings to Secure Channels". Furthermore, it updates the default channel binding to the new binding for versions of TLS greater than 1.2. This document updates RFCs 5801, 5802, 5929, and 7677. This document describes a mechanism that, through the use of opportunistic Transport Layer Security (TLS), enables encryption of Remote Procedure Call (RPC) transactions while they are in transit. The proposed mechanism interoperates with Open Network Computing (ONC) RPC implementations that do not support it. This document updates RFC 5531. In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. Informative References This document describes a simple process that allows authors of Internet-Drafts to record the status of known implementations by including an Implementation Status section. This will allow reviewers and working groups to assign due consideration to documents that have the benefit of running code, which may serve as evidence of valuable experimentation and feedback that have made the implemented protocols more mature. This process is not mandatory. Authors of Internet-Drafts are encouraged to consider using the process for their documents, and working groups are invited to think about applying the process to all of their protocol specifications. This document obsoletes RFC 6982, advancing it to a Best Current Practice. This document specifies a protocol for conveying Remote Procedure Call (RPC) messages on physical transports capable of Remote Direct Memory Access (RDMA). This protocol is referred to as the RPC-over- RDMA version 1 protocol in this document. It requires no revision to application RPC protocols or the RPC protocol itself. This document obsoletes RFC 5666. This document describes the Network File System (NFS) version 4 minor version 1, including features retained from the base protocol (NFS version 4 minor version 0, which is specified in RFC 7530) and protocol extensions made subsequently. The later minor version has no dependencies on NFS version 4 minor version 0, and is considered a separate protocol. This document obsoletes RFC 5661. It substantially revises the treatment of features relating to multi-server namespace, superseding the description of those features appearing in RFC 5661.

Acknowledgments The authors express their deepest appreciation for the contributions of J. Bruce Fields who was an original author of this document. In addition, we are indebted to Lars Eggert and the QUIC working group for the creation of the QUIC transport protocol. The editor is grateful to Bill Baker, Greg Marsden, Richard Scheffenegger, and Martin Thomson for their input and support. Special thanks to Area Director Gorry Fairhurst, NFSV4 Working Group Chairs Brian Pawlowski and Christopher Inacio, and NFSV4 Working Group Secretary Thomas Haynes for their guidance and oversight.