MoQ Media Interop

Protocol Operation

Track Names The publisher selects a namespace of their choosing, and sends an ANNOUNCE message for this namespace. Within the publisher namespace the publisher will offer media tracks named as videoX and audioX where X will be an integer starting at 0. So in case the publisher issues 2 audio tracks and 1 video track, the track names available will be video0, audio0, and audio1. The subscriber will consider all of those tracks belonging to the same namespace as part of the same synchronization group (timestamps aligned to the same timeline).

Mapping Tracks to MoQT Object Model For the video track, the publisher begins a new group at the start of each IDR (so object 0 will be always an IDR Keyframe), and each group contains a single subgroup. Each object has the format described in . For the audio track, the publisher begins a new group with each audio object, and each group contains a single subgroup. Each object has the format described in . TODO: Datagram forwarding preference could be used, but has problems if audio frame does not fit in a single UDP payload.

Timestamps To avoid using fractional numbers and having to deal with rounding errors, timestamps will be expressed with two integers: - timestamp numerator (ex: PTS, DTS, duration) - timebase To convert a timestamp into seconds you just need to: timestamp(s) = timestamp numerator / timebase Example: PTS = 11, timebase = 30 PTS(s) = 11/30 = 0.366666

Object Format

MOQT Media object

Media Type This value indicates what kind of media payload will follow

Code	Value
0x0	Video H264 in AVCC with LOC packager
0x1	Audio Opus bitsream
0x2	UTF-8 text
0x3	Audio AAC-LC in MPEG4

Media payload Is where media related information is carried, and it is specifed by Media type

Video H264 in AVCC with LOC packager format

MOQT Media video h264 loc

Seq ID Monotonically increasing counter for this media track

PTS Timestamp Indicates PTS in timebase TODO: Varint does NOT accept easily negative, so it could be challenging to encode at start (priming)

DTS Timestamp Not needed if B frames are NOT used, in that case should be same value as PTS. TODO: Varint does NOT accept easily negative, so it could be challenging to encode at start (priming)

Timebase Units used in PTS, DTS, and duration.

Duration Duration in timebase. It will be 0 if not set

Wall Clock EPOCH time in ms when this frame started being captured. It will be 0 if not set

Metadata Size Size in bytes of the metadata section It can be 0 if no metadata is sent

Metadata Extradata needed to decode this stream This will be AVCDecoderConfigurationRecord as described in [ISO14496-15:2019] section 5.3.3.1, with field lengthSizeMinusOne = 3 (So length = 4). If any other size length is indicated (in AVCDecoderConfigurationRecord) we should error with “Protocol violation” Any change in encoding parameters MUST send a new AVCDecoderConfigurationRecord

Payload H264 with bitstream AVC1 format as described in [ISO14496-15:2019] section 5.3. Using 4bytes size field length.

Audio Opus bitsream

MOQT Media audio Opus LOC

Seq Id Monotonically increasing counter for this media track

PTS Timestamp Indicates PTS in timebase TODO: Varint does NOT accept easily negative, so it could be challenging to encode at start (priming)

Timebase Units used in PTS, DTS, and duration

Sample Freq Sample frequency used in the original signal (before encoding)

Num Channels Number of channels in the original signal (before encoding)

Duration Duration in timebase. It will be 0 if not set

Wallclock EPOCH time in ms when this frame started being captured. It will be 0 if not set

Payload Opus packets, as described in - section 3

UTF-8 Text

MOQT UTF-8 Text

Seq Id Monotonically increasing counter for this media track

Payload Text packets in UTF-8, as described in

Audio AAC-LC in MPEG4 bitstream

MOQT Media audio AAC-LC MPEG4 LOC

Seq Id Monotonically increasing counter for this media track

PTS Timestamp Indicates PTS in timebase TODO: Varint does NOT accept easily negative, so it could be challenging to encode at start (priming)

Timebase Units used in PTS, DTS, and duration

Sample Freq Sample frequency used in the original signal (before encoding)

Num Channels Number of channels in the original signal (before encoding)

Duration Duration in timebase. It will be 0 if not set

Wallclock EPOCH time in ms when this frame started being captured. It will be 0 if not set

Payload AAC frame (syntax element raw_data_block()), as described in section 4.4.2.1 of [ISO14496-3:2009].