]> Nederlandse Publieke Omroep

Netherlands james.ietf@gmail.com

Tencent America LLC

United States of America spencerdawkins.ietf@gmail.com

applications MOQ Mailing List Internet-Draft QUIC This document describes use cases that have been discussed in the IETF community under the banner of "Media Over QUIC", provides analysis about those use cases, recommends a subset of use cases that cover live media ingest, syndication, and streaming for further exploration, and describes requirements that should guide the design of protocols to satisfy these use cases. Note to Readers RFC Editor: please remove this section before publication Source code and issues for this draft can be found at https://github.com/fiestajetsam/draft-gruessing-moq-requirements. Discussion of this draft should take place on the IETF Media Over QUIC (MoQ) mailing list, at https://www.ietf.org/mailman/listinfo/moq.

Introduction This document describes use cases that have been discussed in the IETF community under the banner of "Media Over QUIC", provides analysis about those use cases, recommends a subset of use cases that cover live media ingest, syndication, and streaming for further exploration, and describes requirements that should guide the design of protocols to satisfy these use cases.

For The Impatient Reader

• Our proposal is to focus on live media use cases, as described in , rather than on interactive media use cases or on-demand use cases.
• The reasoning behind this proposal can be found in .
• The requirements for protocol work to satisfy the proposed use cases can be found in .

Most of the rest of this document provides background for these sections.

Why QUIC For Media? It is not the purpose of this document to argue against proposals for work on media applications that do not involve QUIC. Such proposals are simply out of scope for this document. When work on the QUIC protocol () was chartered (), the key goals for QUIC were:

• Minimizing connection establishment and overall transport latency for applications, starting with HTTP,
• Providing multiplexing without head-of-line blocking,
• Requiring only changes to path endpoints to enable deployment,
• Enabling multipath and forward error correction extensions, and
• Providing always-secure transport, using TLS 1.3 by default.

These goals were chosen with HTTP () in mind. While work on "QUIC version 1" (version codepoint 0x00000001) was underway, protocol designers considered potential advantages of the QUIC protocol for other applications. In addition to the key goals for HTTP applications, these advantages were immediately apparent for at least some media applications:

• QUIC endpoints can create bidirectional or unidirectional ordered byte streams.
• QUIC will automatically handle congestion control, packet loss, and reordering for stream data.
• QUIC streams allow multiple media streams to share congestion and flow control without otherwise blocking each other.
• QUIC streams also allow partial reliability, since either the sender or receiver can terminate the stream early without affecting the overall connection.
• With the DATAGRAM extension (), further partially reliable models are possible, and applications can send congestion controlled datagrams below the MTU size.
• QUIC connections are established using an ALPN.
• QUIC endpoints can choose and change their connection ID.
• QUIC endpoints can migrate IP address without breaking the connection.
• Because QUIC is encapsulated in UDP, QUIC implementations can run in user space, rather than in kernel space, as TCP typically does. This allows more room for extensible APIs between application and transport, allowing more rapid implementation and deployment of new congestion control, retransmission, and prioritization mechanisms.
• QUIC is supported in browsers via HTTP/3 or WebTransport.
• With WebTransport, it is possible to write libraries or applications in JavaScript.

The specific advantages of interest may vary from use case to use case, but these advantages justify further investigation of "Media Over QUIC".

Terminology

The Many Meanings of "Media Over QUIC" Protocol developers have been considering the implications of the QUIC protocol () for media transport for several years, resulting in a large number of possible meanings of the term "Media Over QUIC", or "MOQ". As of this writing, "Media Over QUIC" has had at least these meanings:

• any kind of media carried directly over the QUIC protocol, as a QUIC payload
• any kind of media carried indirectly over the QUIC protocol, as an RTP payload ()
• any kind of media carried indirectly over the QUIC protocol, as an HTTP/3 payload
• any kind of media carried indirectly over the QUIC protocol, as a WebTransport payload
• the encapsulation of any Media Transport Protocol in a QUIC payload
• an IETF mailing list (), which was requested "... for discussion of video ingest and distribution protocols that use QUIC as the underlying transport", although discussion of other Media Over QUIC proposals have also been discussed there.

There may be IETF participants using other meanings as well. As of this writing, the second bullet ("any kind of media carried indirectly over the QUIC protocol, as an RTP payload"), seems to be in scope for the IETF AVTCORE working group, and was discussed at some length at the February 2022 AVTCORE working group meeting , although no drafts in this space have yet been adopted by the AVTCORE working group.

Latency Requirement Categories }The "Operational Considerations for Streaming Media" document () described a range of latencies of interest to streaming media providers, as

• ultra low-latency (less than 1 second)
• low-latency live (less than 10 seconds)
• non-low-latency live (10 seconds to a few minutes)
• on-demand (hours or more)

Because the IETF Media Over QUIC community now expresses interest in interactive media and live media } use cases will have requirements that are significantly less than the "streaming media"-defined "ultra-low latency". Within this document, we are using

• near real-time (less than 50 ms)
• Ull-200 (less than 200 ms)

Perhaps obviously, these last two latency bands are the shortened form of "ultra-low latency - 50 ms" and "ultra-low-latency - 200 ms". Perhaps less obviously, bikeshedding on better names and more useful values is welcomed.

Prior and Existing Specifications Several draft specifications have been proposed which either encapsulate existing Media Transport Protocols in QUIC (), make use of RTP, RTCP, and SDP () or define their own new Media Transport Protocol on top of QUIC. Some have already seen deployment into the wild (e.g. , ) where as others are unconfirmed. Whilst most just focus on defining wire format, defines an architecture using a pub/sub model for both producers and consumers.

Comparison of Existing Specifications

• Some use QUIC Datagram frames, while others use QUIC streams.
• All drafts take differing approaches to flow/stream identification and management. Some address congestion control and others just defer this to QUIC to handle.
• Some drafts specify ALPN identification, while others do not.

Use Cases Informing This Proposal Our goal in this section is to understand the range of use cases that have been proposed for "Media Over QUIC". Although some of the use cases described in this section came out of "RTP over QUIC" proposals, they are worth considering in the broader "Media Over QUIC" context, and may be especially relevant to MOQ, depending on whether "RTP over QUIC" requires major changes to RTP and RTCP, in order to meet the requirements arising out of the corresponding use cases. An early draft in the "media over QUIC" space, , defined several key use cases. Some of the following use cases have been inspired by that document, and others have come from discussions with the wider MOQ community (among other places, a side meeting at IETF 112). For each use case in this section, we also define

• the number of senders or receiver in a given session transmitting distinct streams,
• whether a session has bi-direction flows of media from senders and receivers, and
• the expected lowest latency requirements using the definitions specified in .

It is likely that we should add other characteristics, as we come to understand them.

Interactive Media The use cases described in this section have one particular attribute in common - the target latency for these cases are on the order of one or two RTTs. In order to meet those targets, it is not possible to rely on protocol mechanisms that require multiple RTTs to function effectively. For example,

• When the target latency is on the order of one RTT, it makes sense to use FEC and codec-level packet loss concealment , rather than selectively retransmitting only lost packets. These mechanisms use more bytes, but do not require multiple RTTs in order to recover from packet loss.
• When the target latency is on the order of one RTT, it is impossible to use congestion control schemes like BBR , since BBR has probing mechanisms that rely on temporarily inducing delay and amortizing the consequences of that over multiple RTTs.

This may help to explain why these use cases often rely on protocols such as RTP , which provide low-level control of packetization and transmission.

Gaming

Attribute	Value
Senders/Receivers	One to One
Bi-directional	Yes
Latency	Ull-50

Where media is received, and user inputs are sent by the client. This may also include the client receiving other types of signalling, such as triggers for haptic feedback. This may also carry media from the client such as microphone audio for in-game chat with other players.

Remote Desktop

Attribute	Value
Senders/Receivers	One to One
Bi-directional	Yes
Latency	Ull-50

Where media is received, and user inputs are sent by the client. Latency requirements with this usecase are marginally different than the gaming use case. This may also include signalling and/or transmitting of files or devices connected to the user's computer.

Video Conferencing/Telephony

Attribute	Value
Senders/Receivers	Many to Many
Bi-directional	Yes
Latency	Ull-50 to Ull-200

Where media is both sent and received; This may include audio from both microphone(s) or other inputs, or may include "screen sharing" or inclusion of other content such as slide, document, or video presentation. This may be done as client/server, or peer to peer with a many to many relationship of both senders and receivers. The target for latency may be as large as Ull-200 for some media types such as audio, but other media types in this use case have much more stringent latency targets.

Hybrid Interactive and Live Media For the video conferencing/telephony use case, there can be additional scenarios where the audience greatly outnumbers the concurrent active participants, but any member of the audience could participate. As this has a much larger total number of participants - as many as Live Media Streaming , but with the bi-directionality of confercing, this should be considered a "hybrid".

Live Media The use cases in this section, unlike the use cases described in , still have "humans in the loop", but these humans expect media to be "responsive", where the responsiveness is more on the order of 5 to 10 RTTs. This allows the use of protocol mechanisms that require more than one or two RTTs - as noted in , end-to-end recovery from packet loss and congestion avoidance are two such protocol mechanisms that can be used with Live Media. To illustrate the difference, the responsiveness expected with videoconferencing is much greater than watching a video, even if the video is being produced "live" and sent to a platform for syndication and distribution.

Live Media Ingest

Attribute	Value
Senders/Receivers	One to One
Bi-directional	No
Latency	Ull-200 to Ultra-Low

Where media is received from a source for onwards handling into a distribution platform. The media may comprise of multiple audio and/or video sources. Bitrates may either be static or set dynamically by signalling of connection inforation (bandwidth, latency) based on data sent by the receiver.

Live Media Syndication

Attribute	Value
Senders/Receivers	One to One
Bi-directional	No
Latency	Ull-200 to Ultra-Low

Where media is sent onwards to another platform for further distribution. The media may be compressed down to a bitrate lower than source, but larger than final distribution output. Streams may be redundant with failover mechanisms in place.

Live Media Streaming

Attribute	Value
Senders/Receivers	One to Many
Bi-directional	No
Latency	Ull-200 to Ultra-Low

Where media is received from a live broadcast or stream. This may comprise of multiple audio or video outputs with different codecs or bitrates. This may also include other types of media essence such as subtitles or timing signalling information (e.g. markers to indicate change of behaviour in client such as advertisement breaks). The use of "live rewind" where a window of media behind the live edge can be made available for clients to playback, either because the local player falls behind edge or because the viewer wishes to play back from a point in the past.

On-Demand Media Finally, the "On-Demand" use cases described in this section do not have a tight linkage between ingest and streaming, allowing significant transcoding, processing, insertion of video clips in a news article, etc. The latency constraints for the use cases in this section may be dominated by the time required for whatever actions are required before media are available for streaming.

On-Demand Ingest

Attribute	Value
Senders/Receivers	One to Many
Bi-directional	No
Latency	On Demand

Where media is ingested and processed for a system to later serve it to clients as on-demand media. This media provided from a pre-recorded source, or captured from live output, but in either case, this media is not immediately passed to viewers, but is stored for "on-demand" retrieval, and may be transcoded upon ingest.

On-Demand Media Streaming

Attribute	Value
Senders/Receivers	One to Many
Bi-directional	No
Latency	On Demand

Where media is received from a non-live, typically pre-recorded source. This may feature additional outputs, bitrates, codecs, and media types described in the live media streaming use case.

Proposed Scope for "Media Over QUIC" Our proposal is that "Media Over QUIC" discussions focus first on the use cases described in , which are Live Media Ingest (), Syndication (), and Streaming (). Our reasoning for this suggestion follows. Each of the above use cases in fit into one of three classifications of solutions.

Analysis for Interactive Use Cases The first group, Interactive Media, as described in , and covering gaming (), screen sharing (), and general video conferencing (), are largely covered by RTP, often in conjunction with WebRTC , and related protocols today. Whilst there may be benefit in these use cases having a QUIC based protocol it may be more appropriate given the size of existing deployments to extend the RTP protocols and specifications.

Analysis for Live Media Use Cases The second group of classifications, in , covering Live Media Ingest (), Live Media Syndication (), and Live Media Streaming () are likely the use cases that will benefit most from this work. Existing ingest and streaming protocols such as HLS and DASH are reaching limits towards how low they can reduce latency in live streaming and for scenarios where low-bitrate audio streams are used, these protocols add a significant amount of overhead compared to the media bitstream itself. For this reason, we suggest that work on "Media Over QUIC" protocols target these use cases at this time.

Analysis for On-Demand Use Cases The third group, , covering On-Demand Media Ingest () and On-Demand Media streaming () is unlikely to benefit from work in this space. Without the same "Live Media" latency requirements that would motivate deployment of new protocols, existing protocols such as HLS and DASH are probably "good enough" to meet the needs of these use cases. This does not mean that existing protocols in this space are perfect. Segmented protocols such as HLS and DASH were developed to overcome the deficiencies of TCP, as used in HTTP/1.1 and HTTP/2 , and do not make full use of the possible congestion window along the path from sender to receiver. Other protocols in this space have their own deficiencies. For example, RTSP does not have easy ways to add support for new media codecs. Our expectation is that these use cases will not drive work in the "Media Over QUIC" space, but as new protocols come into being, they may very well be taken up for these use cases as well.

Requirements for Protocol Work TODO: Quite a lot, really ...

IANA Considerations This document makes no requests of IANA.

Security Considerations As this document is intended to guide discussion and consensus, it introduces no security considerations of its own.

Informative References This memorandum describes RTP, the real-time transport protocol. RTP provides end-to-end network transport functions suitable for applications transmitting real-time data, such as audio, video or simulation data, over multicast or unicast network services. RTP does not address resource reservation and does not guarantee quality-of- service for real-time services. The data transport is augmented by a control protocol (RTCP) to allow monitoring of the data delivery in a manner scalable to large multicast networks, and to provide minimal control and identification functionality. RTP and RTCP are designed to be independent of the underlying transport and network layers. The protocol supports the use of RTP-level translators and mixers. Most of the text in this memorandum is identical to RFC 1889 which it obsoletes. There are no changes in the packet formats on the wire, only changes to the rules and algorithms governing how the protocol is used. The biggest change is an enhancement to the scalable timer algorithm for calculating when to send RTCP packets in order to minimize transmission in excess of the intended rate when many participants join a session simultaneously. [STANDARDS-TRACK] This document describes a framework for using Forward Error Correction (FEC) codes with applications in public and private IP networks to provide protection against packet loss. The framework supports applying FEC to arbitrary packet flows over unreliable transport and is primarily intended for real-time, or streaming, media. This framework can be used to define Content Delivery Protocols that provide FEC for streaming media delivery or other packet flows. Content Delivery Protocols defined using this framework can support any FEC scheme (and associated FEC codes) that is compliant with various requirements defined in this document. Thus, Content Delivery Protocols can be defined that are not specific to a particular FEC scheme, and FEC schemes can be defined that are not specific to a particular Content Delivery Protocol. [STANDARDS-TRACK] This document defines the Opus interactive speech and audio codec. Opus is designed to handle a wide range of interactive audio applications, including Voice over IP, videoconferencing, in-game chat, and even live, distributed music performances. It scales from low bitrate narrowband speech at 6 kbit/s to very high quality stereo music at 510 kbit/s. Opus uses both Linear Prediction (LP) and the Modified Discrete Cosine Transform (MDCT) to achieve good compression of both speech and music. [STANDARDS-TRACK] The Hypertext Transfer Protocol (HTTP) is a stateless application-level protocol for distributed, collaborative, hypertext information systems. This document provides an overview of HTTP architecture and its associated terminology, defines the "http" and "https" Uniform Resource Identifier (URI) schemes, defines the HTTP/1.1 message syntax and parsing requirements, and describes related security concerns for implementations. This specification describes an optimized expression of the semantics of the Hypertext Transfer Protocol (HTTP), referred to as HTTP version 2 (HTTP/2). HTTP/2 enables a more efficient use of network resources and a reduced perception of latency by introducing header field compression and allowing multiple concurrent exchanges on the same connection. It also introduces unsolicited push of representations from servers to clients. This specification is an alternative to, but does not obsolete, the HTTP/1.1 message syntax. HTTP's existing semantics remain unchanged. This memorandum defines the Real-Time Streaming Protocol (RTSP) version 2.0, which obsoletes RTSP version 1.0 defined in RFC 2326. RTSP is an application-layer protocol for the setup and control of the delivery of data with real-time properties. RTSP provides an extensible framework to enable controlled, on-demand delivery of real-time data, such as audio and video. Sources of data can include both live data feeds and stored clips. This protocol is intended to control multiple data delivery sessions; provide a means for choosing delivery channels such as UDP, multicast UDP, and TCP; and provide a means for choosing delivery mechanisms based upon RTP (RFC 3550). This document describes a protocol for transferring unbounded streams of multimedia data. It specifies the data format of the files and the actions to be taken by the server (sender) and the clients (receivers) of the streams. It describes version 7 of this protocol. This document defines the core of the QUIC transport protocol. QUIC provides applications with flow-controlled streams for structured communication, low-latency connection establishment, and network path migration. QUIC includes security measures that ensure confidentiality, integrity, and availability in a range of deployment circumstances. Accompanying documents describe the integration of TLS for key negotiation, loss detection, and an exemplary congestion control algorithm. Google Google Google Google Google This document specifies the BBR congestion control algorithm. BBR ("Bottleneck Bandwidth and Round-trip propagation time") uses recent measurements of a transport connection's delivery rate, round-trip time, and packet loss rate to build an explicit model of the network path. BBR then uses this model to control both how fast it sends data and the maximum volume of data it allows in flight in the network at any time. Relative to loss-based congestion control algorithms such as Reno [RFC5681] or CUBIC [RFC8312], BBR offers substantially higher throughput for bottlenecks with shallow buffers or random losses, and substantially lower queueing delays for bottlenecks with deep buffers (avoiding "bufferbloat"). BBR can be implemented in any transport protocol that supports packet-delivery acknowledgment. Thus far, open source implementations are available for TCP [RFC793] and QUIC [RFC9000]. This document specifies version 2 of the BBR algorithm, also sometimes referred to as BBRv2 or bbr2. Facebook Facebook Facebook Facebook RUSH is an application-level protocol for ingesting live video. This document describes the protocol and how it maps onto QUIC. Discussion Venues This note is to be removed before publishing as an RFC. Discussion of this document takes place on the mailing list (), which is archived at . Source for this draft and an issue tracker can be found at https://github.com/afrind/draft-rush. Twitch This document defines the core behavior for Warp, a segmented live media transport protocol. Warp maps live media to QUIC streams based on the underlying media encoding. Media is prioritized to reduce latency when encountering congestion. Haivision Network Video GmbH Haivision Network Video GmbH This document presents an approach to tunnelling SRT live streams over QUIC datagrams. QUIC [RFC9000] is a UDP-based transport protocol providing TLS encryption, stream multiplexing, and connection migration. It was designed to become a faster alternative to the TCP protocol [RFC7323]. An Unreliable Datagram Extension to QUIC [QUIC-DATAGRAM] adds support for sending and receiving unreliable datagrams over a QUIC connection, but transfers the responsibility for multiplexing different kinds of datagrams, or flows of datagrams, to an application protocol. SRT [SRTRFC] is a UDP-based transport protocol. Essentially, it can operate over any unreliable datagram transport. SRT provides loss recovery and stream multiplexing mechanisms. In its live streaming configuration SRT provides an end-to-end latency-aware mechanism for packet loss recovery. If SRT fails to recover a packet loss within a specified latency, then the packet is dropped to avoid blocking playback of further packets. The Datagram Extension to QUIC could be used as an underlying transport instead of UDP. This way QUIC would provide TLS-level security, connection migration, and potentially multi-path support. It would be easier for existing network facilities to process, route, and load balance the unified QUIC traffic. SRT on its side would provide end-to-end latency tracking and latency-aware loss recovery. Technical University Munich Technical University Munich This document specifies a minimal mapping for encapsulating RTP and RTCP packets within QUIC. It also discusses how to leverage state from the QUIC implementation in the endpoints to reduce the exchange of RTCP packets and how to implement congestion control. Discussion Venues This note is to be removed before publishing as an RFC. Source for this draft and an issue tracker can be found at https://github.com/mengelbart/rtp-over-quic-draft. cisco Cisco This specification outlines the design for a media delivery protocol over QUIC. It aims at supporting multiple application classes with varying latency requirements including ultra low latency applications such as interactive communication and gaming. It is based on a publish/subscribe metaphor where entities publish and subscribe to data that is sent through, and received from, relays in the cloud. The information subscribed to is named such that this forms an overlay information centric network. The relays allow for efficient large scale deployments. Technische Universitaet Muenchen Huawei University of Glasgow CALLSTATS I/O Oy QUIC is a UDP-based protocol for congestion controlled reliable data transfer, while RTP serves carrying (conversational) real-time media over UDP. This draft discusses design aspects and issues of carrying RTP over QUIC. Akamai Technologies, Inc. Networked Media Tencent America LLC This document provides an overview of operational networking issues that pertain to quality of experience when streaming video and other high-bitrate media over the Internet. Apple Inc. Apple Inc. Google LLC This document defines an extension to the QUIC transport protocol to add support for sending and receiving unreliable datagrams over a QUIC connection. Akamai The QUIC transport protocol has several features that are desirable in a transport for HTTP, such as stream multiplexing, per-stream flow control, and low-latency connection establishment. This document describes a mapping of HTTP semantics over QUIC. This document also identifies HTTP/2 features that are subsumed by QUIC and describes how HTTP/2 extensions can be ported to HTTP/3. Google The WebTransport Protocol Framework enables clients constrained by the Web security model to communicate with a remote server using a secure multiplexed transport. It consists of a set of individual protocols that are safe to expose to untrusted applications, combined with a model that allows them to be used interchangeably. This document defines the overall requirements on the protocols used in WebTransport, as well as the common features of the protocols, support for some of which may be optional. n.d. n.d. n.d.

Acknowledgements The authors would like to thank the many authors of the specifications referenced in for their work. The authors would also like to thank Alan Frindell, Luke Curley, and Maxim Sharabayko for text contributions to this draft. James Gruessing would also like to thank Francesco Illy and Nicholas Book for their part in providing the needed motivation.