Extensible YANG Model for Network Telemetry Messages

Introduction Nowadays network operators are using machine and human readable YANG to model their configurations and obtain YANG modelled operational data from their networks. Network operators organize their data in a Data Mesh where a Message Broker such as Apache Kafka or Apache Pulsar facilitates the exchange of messages among data processing components. Today, subscribing to a YANG datastore, publishing a YANG modeled notifications message from the network and viewing the data in a time series database, manual labor is needed to perform data transformation to make a Message Broker and its data processing components with YANG notifications interoperable. Even though YANG is intended to ease data management, this promise has not yet been fulfilled for Network Telemetry. An Architecture for YANG-Push to Message Broker Integration defined an architecture for integrating YANG-Push with Message Brokers for a Data Mesh architecture. How the notification messages at a YANG-Push Receiver is being transformed to the Message Broker is being described in Section , however the produced message format left unspecified. The message could be published as it was received from the network to their organization's Message Broker. However, this approach is insufficient for correct human and automated understanding of the data generated by the network. This insufficiency stems from not presenting a holistic picture along with the data generated by the network. In particular, when a data consumer in the data mesh consumes a YANG message from their organization's Message Broker, they cannot answer simple questions such as:

Which network operating system collected the data?
To which network platform belongs the network node?
What is the subscribed xpath, sub-tree filter and its schema reference?
When did a data collector received the data?
What additional metadata is necessary for a consumer to make sense of the data?

describes the content of a Data Manifest and how it is being mapped to the collected Network Telemetry data. The "ietf-telemetry-message" YANG module defined in this document makes use of the platform-details grouping defined in for the network node and the data collector. This document defines a standard YANG envelope message to carry with the collected Network Telemetry notifications the provenance and metadata information for a YANG data exchange between between Message Broker producers and consumers as described in Section 4.5 and 4.6 . The described YANG model fascilitates JSON and CBOR serialization.

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Terminology The terms "Subscriber", "Publisher", and "Receiver" are used as defined in . The term "Network Telemetry" is used as defined in . This document uses the term export and collection to distinguish between the data export and collection. The term "Message Broker" is used as defined in . The term "Data Manifest" is used as defined in . The term provenance is used in general to describe the origin, history, and authenticity of an asset which then is described in a manifest. In addition, this document reuses the terms "Notification Metadata" and "Notification Envelope" defined in for the use in Message Broker environment: Notification Metadata: Additional data describing the context of a notification that is sent in each message, e.g. which node generated the messsage or at which time the notification was published. Notification Envelope: YANG structure encapsulating the payload of a notification, allowing the inclusion of metadata.

YANG Modules This document defines two YANG modules, an extensible YANG module for Network Telemetry messages defined in and a YANG-Push extension defined in . The extensible YANG module for Network Telemetry messages defines an envelope message schema which adds two provenance and two metadata categories to the collected Network Telemetry data. The YANG-Push extension adds YANG-Push specific subscription metadata to the Network Telemetry protocol provenance of the envelope.

Network Node Manifest:: The OPTIONAL "network-node-manifest" container in contains the Data Manifest about the network node that exported Network Telemetry data. It adds information such as the node name, address, and software version. Each leaf is OPTIONAL.

Network Telemetry Protocol Metadata:: The "telemetry-message-metadata" container in contains the Network Telemetry session information between the collector and the network node. It adds fields such as the Network Telemetry protocol name, the IP addresses and ports of that transport session and the time the event was exported at the network node and received at the data collector. Apart from the collection-timestamp, session-protocol and remote-address leafs all other leafs are OPTIONAL. These Network Telemetry session informations allow a network operator to troubleshoot whether the metrics were obtained through NETCONF or RESTCONF <GET> operations or YANG-Push and measure delay and loss related issues. Moreover, this document also defines with an extension specific to YANG-Push that includes YANG-Push subscription information relevant to the Network Telemetry session information.

Data Collection Manifest:: The OPTIONAL "data-collection-manifest" container in contains the Data Manifest of the data collector which collected the Network Telemetry data. The data type is the same as the previous category but specific to the collector node.

Network Operator Metadata:: The OPTIONAL "labels" list in the "network-operator-metadata" container in contains the operator specific metadata. Some operators enrich the collected data with specific information. For instance: type of the network node (provider or provider edge node) or which operational unit the node is operated by. For this purpose the document defines a generic metadata map with key/values that can be used freely by the network operator.

Thomas Graf "; description "This YANG module defines an extensible message schema to be used at data collection to transform Network Telemetry messages towards external systems such as Message Brokers. Copyright (c) 2025 IETF Trust and the persons identified as authors of the code. All rights reserved. Redistribution and use in source and binary forms, with or without modification, is permitted pursuant to, and subject to the license terms contained in, the Revised BSD License set forth in Section 4.c of the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info). This version of this YANG module is part of RFC XXXX; see the RFC itself for full legal notices."; revision 2025-10-19 { description "Initial revision."; reference "RFC XXXX"; } sx:structure "message" { description "Telemetry message used within the Data Mesh"; container network-node-manifest { if-feature "network-node-manifest"; description "Contains the Data Manifest about the network node that exported Network Telemetry data."; uses p-mf:platform-details; } container telemetry-message-metadata { description "contains the session information about the session between the collector and the network node."; leaf node-export-timestamp { type yang:date-and-time; description "Timestamp when the Network Telemetry data has been exported from network element. This can be obtained in YANG-Push from event-time defined in draft-ietf-netconf-notif-envelope or in IPFIX from the export time in the message header as defined in RFC 7011 or in BMP from the timestamp in The per-peer header as defined in RFC 7854."; } leaf collection-timestamp { type yang:date-and-time; mandatory true; description "Timestamp when the data collector collected the Network Telemetry data from the network element."; } leaf notification-event { type telemetry-notification-event-type; mandatory true; description "Describes wherever the notification was received and forwarded from the network node or generated or removed from the data collector state cache."; } leaf session-protocol { type telemetry-session-protocol-type; mandatory true; description "Session protocol used to collect the Network Telemetry data from the network node."; } leaf export-address { type inet:host; mandatory true; description "Network node IP address from where the Network Telemetry data was exported from."; } leaf export-port { type inet:port-number; description "Network node transport port number from where the Network Telemetry data was exported."; } leaf collection-address { type inet:host; description "Data collector IP address at which the Network Telemetry data was collected."; } leaf collection-port { type inet:port-number; description "Data collector transport port number at which the Network Telemetry data was collected."; } } container data-collection-manifest { if-feature "data-collection-manifest"; description "Contains the Data Manifest of the data collector which collected the Network Telemetry data."; uses p-mf:platform-details; } container network-operator-metadata { description "Network operator specific metadata added by the Network Telemetry data collection."; list labels { key "name"; description "Abritrary labels assigned by the data collector."; leaf name { type string { length "1..max"; } description "Label name."; } choice value { mandatory true; description "label value"; choice string-choice { description "String value"; leaf string-value { type string; description "String value"; } } choice anydata-choice { description "YANG anydata value"; anydata anydata-values { description "anydata yang"; } } } } } anydata payload { description "Message or notification received from network element."; } } feature network-node-manifest { description "This feature indicates the network node manifest support."; } feature data-collection-manifest { description "This feature indicates the data collection manifest support."; } identity session-protocol { description "Base identity to represent session protocols."; } identity yang-push { base session-protocol; description "YANG-Push in RFC 8640 or RFC 8641 or RFC 8650."; reference "RFC 8640, RFC 8641, RFC 8650: YANG-Push Events and Notifications for Datastores."; } identity netconf { base session-protocol; description "NETCONF GET RPC as described in RFC 6241."; reference "RFC 6241: NETCONF GET RPC."; } identity restconf { base session-protocol; description "RESTCONF HTTP GET as described in RFC 8040."; reference "RFC 8040: HTTP GET."; } typedef telemetry-notification-event-type { type enumeration { enum "log" { description "Collector is reporting the event as it arrived from the network element."; } enum "update" { description "Collector has updated an entry inside its local cache. This could be triggered by an event from the network for instance interface operational status changed or an internal event in the collector, such as a timer triggered to referesh old enteries."; } enum "delete" { description "Collector has deleted an entry from its local cache."; } } description "Type of event reported by the collector."; } typedef telemetry-session-protocol-type { type identityref { base session-protocol; } description "Network Telemetry protocol used to deliver the notification between the network node and the data collector."; } } ]]>

Thomas Graf "; description "Adds YANG-Push specific subscription metadata to the data collection protocol provenance of the ietf-telemetry-message envelope. The key words 'MUST', 'MUST NOT', 'REQUIRED', 'SHALL', 'SHALL NOT', 'SHOULD', 'SHOULD NOT', 'RECOMMENDED', 'NOT RECOMMENDED', 'MAY', and 'OPTIONAL' in this document are to be interpreted as described in BCP 14 (RFC 2119) (RFC 8174) when, and only when, they appear in all capitals, as shown here. Copyright (c) 2025 IETF Trust and the persons identified as authors of the code. All rights reserved. Redistribution and use in source and binary forms, with or without modification, is permitted pursuant to, and subject to the license terms contained in, the Revised BSD License set forth in Section 4.c of the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info). This version of this YANG module is part of RFC XXXX; see the RFC itself for full legal notices."; revision 2025-10-19 { description "Initial revision."; reference "RFC XXXX"; } sx:augment-structure "/tm:message/tm:telemetry-message-metadata" { description "Augments telemetry-message-metadata with YANG-Push specific subscription metadata"; container yang-push-subscription { config false; description "YANG-Push specific subscription metadata"; leaf id { type sn:subscription-id; description "This references the affected subscription."; } choice filter-spec { description "The content filter specification for this request."; anydata subtree-filter { description "Event stream evaluation criteria or the parameter identifies the port of the target datastore encoded in the syntax of a subtree filter as defined in RFC 6241, Section 6."; reference "RFC 6241: Network Configuration Protocol (NETCONF), Section 6."; } leaf xpath-filter { type yang:xpath1.0; description "Event stream evaluation criteria or porting of the target datastore encoded in the syntax of an XPath 1.0 expression"; reference "XML Path Language (XPath) Version 1.0 (https://www.w3.org/TR/1999/REC-xpath-19991116) RFC 7950: The YANG 1.1 Data Modeling Language, Section 10"; } } choice target { description "Identifies the source of information against which a subscription is being applied as well as specifics on the subset of information desired from that source."; case stream { leaf stream { type string; description "Indicates the event stream to be considered for this subscription."; } } case datastore { leaf datastore { type identityref { base ds:datastore; } description "Datastore from which to retrieve data."; } } } leaf transport { type sn:transport; description "For a configured subscription, this leaf specifies the transport used to deliver messages destined for all receivers of that subscription."; } leaf encoding { type sn:encoding; description "The type of encoding for notification messages. For a dynamic subscription, if not included as part of an 'establish-subscription' RPC, the encoding will be populated with the encoding used by that RPC. For a configured subscription, if not explicitly configured, the encoding will be the default encoding for an underlying transport."; } leaf purpose { type string; description "Open text allowing a configuring entity to embed the originator or other specifics of this subscription."; } choice update-trigger { description "Defines necessary conditions for sending an event record to the subscriber."; case periodic { container periodic { presence "indicates a periodic subscription"; description "The publisher is requested to notify periodically the current values of the datastore as defined by the selection filter."; leaf period { type yp:centiseconds; description "Duration of time which should occur between periodic push updates, in one hundredths of a second."; } leaf anchor-time { type yang:date-and-time; description "Designates a timestamp before or after which a series of periodic push updates are determined. The next update will take place at a whole multiple interval from the anchor time. For example, for an anchor time is set for the top of a particular minute and a period interval of a minute, updates will be sent at the top of every minute this subscription is active."; } } } case on-change { container on-change { presence "indicates an on-change subscription"; description "The publisher is requested to notify changes in values in the datastore subset as defined by a selection filter."; leaf dampening-period { type yp:centiseconds; default "0"; description "Specifies the minimum interval between the assembly of successive update records for a single receiver of a subscription. Whenever subscribed objects change, and a dampening period interval (which may be zero) has elapsed since the previous update record creation for a receiver, then any subscribed objects and properties which have changed since the previous update record will have their current values marshalled and placed into a new update record."; } leaf sync-on-start { type boolean; default "true"; description "When this object is set to false, it restricts an on-change subscription from sending push-update notifications. When false, pushing a full selection per the terms of the selection filter MUST NOT be done for this subscription. Only updates about changes, i.e. only push-change-update notifications are sent. When true (default behavior), in order to facilitate a receiver's synchronization, a full update is sent when the subscription starts using a push-update notification. After that, push-change-update notifications are exclusively sent unless the publisher chooses to resync the subscription via a new push-update notification."; } } } } list module { key "name"; config false; description "List of yang-push-module-version grouping defined in draft-ietf-netconf-yang-notifications-versioning. The revision is not configurable."; leaf name { type yang:yang-identifier; config false; description "This references the YANG module name, section 7.1 of RFC 7950."; } leaf revision { type rev:revision-date; config false; description "This references the YANG module revision, section 7.1.2 of RFC 7950, of the sent notification message."; } leaf version { type ysver:version; description "This references the YANG module semantic version, draft-ietf-netmod-yang-semver, of the sent notification \t\t\t\t message."; } } leaf yang-library-content-id { type string; config false; description "Contains the YANG library content identifier, RFC 8525, information."; } } } } ]]>

Use Cases

Data Chain Troubleshooting In a distributed system like a data chain described in , a single system component could introduce a new issue after a software upgrade of a system component . Thanks to the network node and the data collection manifest, the data engineer is able to identify wherever a software upgrade has been performed on a network node or on a data collector and its impact on the telemetry messages. Such an upgrade could have a potential impact on the YANG schema of the exported YANG-Push Notification which is identifable through the "yang-library-content-id" and "revision" and "version" defined in and transformed at the data collectors as part of Network Telemetry protocol metadata or the YANG schema id when being produced to the message broker as described in .

Data Product Service Level Objective For Service Disruption Detection in Network Anomaly Detection, operational Network Telemetry data needs to to be consumeable as a YANG data consumer within the defined time to ensure a holistic view of the network. To measure the delay and loss between the data export on the network node, the data collectors and the YANG data consumer, sequence numbers and timestamps are required. With "hostname" and "sequence-number" defined in each message is uniquely identifiably and with "event-time" and "observation time" defined in section 3.4 and 3.5 of and "collection-time" defined in this document and the timestamp when the data arrived at the YANG data consumer, the loss and delay for a given network node and message can be deducted. Metadata introduced in this document help a data engineer to determine a common denominator and identify the system component causing the delay and loss. Certain collection protocol types such as polling or push based and subscriptions types, such as on-change or periodical, have different data delay characteristics. Push based on-change subscriptions are expected to export messages almost instantly where polling based subscriptions observe changes depending on their polling interval much later. With the Network Telemetry protocol metadata introduced by this document, based on the update-trigger informations, a YANG data consumer can define for a given YANG-Push subscription id a lower service Level objective than for another. Therefore adjust the detection to be more real-time than without this information.

IANA Considerations This document registers the following two namespace URIs in the IETF XML Registry:

URI: urn:ietf:params:xml:ns:yang:ietf-telemetry-message
Registrant Contact: The IESG.
XML: N/A; the requested URI is an XML namespace.

URI: urn:ietf:params:xml:ns:yang:ietf-yang-push-telemetry-message
Registrant Contact: The IESG.
XML: N/A; the requested URI is an XML namespace.

This document registers the following two YANG modules in the YANG Module Names registry:

Name: ietf-telemetry-message
Namespace: urn:ietf:params:xml:ns:yang:ietf-telemetry-message
Prefix: tm
Reference: RFC XXXX

Name: ietf-yang-push-telemetry-message
Namespace: urn:ietf:params:xml:ns:yang:ietf-yang-push-telemetry-message
Prefix: yptm
Reference: RFC XXXX

Security Considerations This section is modeled after the template described in . The "ietf-telemetry-message" and "ietf-yang-push-telemetry-message" YANG modules defines two data models that are designed to be accessed via YANG-based management protocols, such as NETCONF and RESTCONF . These protocols have to use a secure transport layer (e.g., SSH , TLS , and QUIC ) and have to use mutual authentication. The Network Configuration Access Control Model (NACM) provides the means to restrict access for particular NETCONF or RESTCONF users to a preconfigured subset of all available NETCONF or RESTCONF protocol operations and content. There are a number of data nodes defined in this YANG module that are writable/creatable/deletable (i.e., "config true", which is the default). All writable data nodes are likely to be reasonably sensitive or vulnerable in some network environments. Write operations (e.g., edit-config) and delete operations to these data nodes without proper protection or authentication can have a negative effect on network operations. The following subtrees and data nodes have particular sensitivities/vulnerabilities: "There are no particularly sensitive writable data nodes." Some of the readable data nodes in this YANG module may be considered sensitive or vulnerable in some network environments. It is thus important to control read access (e.g., via get, get-config, or notification) to these data nodes. Specifically, the following subtrees and data nodes have particular sensitivities/ vulnerabilities: "There are no particularly sensitive readable data nodes."

Implementation status This section provides pointers to existing open source implementations of this draft. Note to the RFC-editor: Please remove this before publishing.

Netgauze An open source Network Telemetry data collection implemented "ietf-telemetry-message" and "ietf-yang-push-telemetry-message" . The open source code can be accessed here: . provides an example of a JSON encoded, , Network Telemetry message.

References Normative References Informative References Data Mesh O'Reilly Media Apache Pulsar Apache Software Foundation Apache Kafka Apache Software Foundation Netgauze open source Network Telemetry Data Collection

Acknowledgements The authors would like to thank Rob Wilton, Alex Huang Feng, Benoit Claise, Leonardo Rodoni, Reshad Rahman, Dan Voyer, Paolo Lucente and Martin Björklund for their review and valuable comments.