Huawei

Townsend Street, 4th Floor George's Court Dublin Ireland zhuoyao.lin1@huawei-partners.com

Huawei

benoit.claise@huawei.com

Telefonica

Ronda de la Comunicacion, S/N Madrid 28050 Spain ignacio.dominguezmartinez@telefonica.com

OPS NETCONF This document augments the ietf-yang-library to provide the augmented-by list. It facilitates the process of obtaining all dependencies between YANG modules, by querying the network management server's YANG library. Source for this draft and an issue tracker can be found at .

The YANG library specifies a YANG module that provides module information such as submodule list and deviation list to help a client listing all datastores supported by a network management server and the schema that is used by each of these datastores. According to Section 4.2.8 and 5.6.3 in , both augmentations and deviations define additional nodes internal or external to the module, which are the reverse dependencies of a YANG module. Reverse dependencies and the include, import as defined in Section 5.1.1 of are both crucial information for understanding all dependencies of a YANG module. However, currently it is difficult to obtain the YANG schema tree without obtaining and parsing all YAG modules from a management server. The deviation list defined in YANG library enables client to obtain the module reverse dependency without having to get and parse all YANG modules. However, the augmentation list is not defined in it. Since both augmentation and deviation work as YANG module dependencies, it is reasonable to document them the same way in the YANG library. Having both augmentation and deviation directly available in the YANG library provides an easy and light-weight solution for determining the reverse dependencies. Therefore, this document proposes a YANG module that augments the YANG library to include the YANG module augmentation information.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. The terminology from is used in this document The term "client" is used as defined in for NETCONF and for RESTCONF. Tree diagrams in this document use the notation defined in .

When using a YANG module, it is necessary to make sure that all its dependencies are presented. identifies four types of dependencies between YANG modules:

• Import: the "import" statement allows a module or submodule to reference definitions defined in other modules.
• Include: the "include" statement is used in a module to identify each submodule that belongs to it.
• Augmentation: the "augment" statement defines the location in the data model hierarchy where additional nodes are inserted.
• Deviation: the "deviation" statement defines a fragment of a module that the server does not implement.

The import and include are direct dependencies which can be obtained by parsing the YANG module source code, while the augmentation and deviation are reverse dependencies which are defined in another module. For the reverse dependencies, since they are defined externally, it is not possible to discover them by parsing the YANG module. The current way to discover the reverse dependencies is to query all YANG modules from the server and parse them. This is a lengthy process, which must be repeated for each client that requires these information. According to the definition of module ietf-yang-library defined in , in the schema content of a module in container yang-library, the deviation is provided to describe that a module is deviated by which other modules. If the YANG library could directly report all reverse dependencies, it would provide a much easier and light-weight solution to find module all dependencies, compared to obtaining and parsing all modules. The YANG library only provides the deviation list, without augmentations. With augmentations being more widely used and defined, and with use cases to automate network management, augmentations become essential information for clients to better understand the network management servers module relationships. Thus, the YANG library should be extended to also provide the augmentation information. From the perspective of implementation difficulty, it is easy to adapt the device implementation to include augmentation, since augmentation and deviation have similar way of working.

As the demand for YANG-based telemetry arises, there is a need for real-time knowledge of a specific YANG module's dependency list when a specific YANG-Push notification is received. The alternative for a YANG-Push receiver is to collect and store the entire module set for every single server who could be streaming data. This approach is not always practical due to the following reasons:

• For a YANG-Push receiver => we never know in advance which or from which YANG-Push publisher the subscribed YANG content will be received.
• Querying all the YANG modules is time consuming and overhead considering that only a subset of YANG nodes of management server are subscribed.

This section introduces two use cases that reflect the motivation for extending YANG library. One targets solving dependency problems in a data mesh architecture while the other aims at building a data catalog that makes YANG module information easily accessible.

A network analytics architecture that integrates YANG-Push and Kafka is proposed and is continuously growing and gaining influence, refer to the draft: An Architecture for YANG-Push to Apache Kafka Integration. This open-source project encompasses contributions such as Support of Versioning in YANG Notifications Subscription or Support of Network Observation Timestamping in YANG Notifications , among others. The purpose of this project is to provide adequate information to the YANG-Push subscription state change notifications so that when the module and its dependencies can be parsed and retrieved automatically from the vantage point. The architecture relies on the information of YANG dependencies to realize, to solve the problem of missing YANG semantics when notifications are transformed or indexed in Time Series Database. As a solution to provide the missing YANG semantics, a schema registry is introduced to store YANG modules and all their relationships (Direct and reverse dependencies). The schema is obtained by NETCONF <get-schema> of the subscribed YANG schema tree, which is obtained by parsing the <subscription-started> message of each YANG-Push subscription. When obtaining the dependency modules of a YANG module, an independent process containing multiple <get-schema> operations is launched after each new YANG-Push subscription module has been known. However, the complexity remains at:

• How dependencies of YANG modules are found (so that the YANG-Push subscription message has the complete set of module dependencies for its subscribed YANG schema tree)?
• How do we conduct <get-schema>?

Currently, the method used for obtaining modules and finding module dependencies is "get-all-schemas", where the YANG client retrieves all YANG modules from the network device to enable later the client can fully understand and utilize all modules and module dependencies of device. This process is very heavy because in a real situation, each device may implement hundreds of YANG modules, requiring up to several minutes to complete in the worst case. Besides, the need of parsing all YANG modules and finding all the dependencies adds a small extra delay. Applying this method to obtain YANG module will make the operation very costly, since after each subscribed module is learned, "get-all-schemas" needs to be re-performed. Therefore, considering the Network Observability real-time aspects, this extra delay in collecting (and processing) the dependencies through a get-all-schemas approach is not realistic. It's more efficient to get dependencies only for the required modules. By using the provided the augmentation information in ietf-yang-library, the YANG-Push receiver can directly obtain the YANG reverse dependencies by obtaining the contents of the YANG library, saving collection (and processing time) at the YANG-Push receiver and therefore helping with the near real-time aspects of Network Observability and enabling closed loop actions.

Finding the YANG modules implemented by a network management server is paramount for configuring and monitoring the status of a network. However, since the inception of YANG the network industry has experienced a tsunami of YANG modules developed by SDOs, open-source communities, and network vendors. This heterogeneity of YANG modules, that vary from one network device model to another, makes the management of a multi-vendor network a big challenge for operators. In this regard, a data catalog provides a registry of the datasets exposed by remote data sources for consumers to discover data of interest. Besides the location of the dataset (i.e., the data source), the data catalog registers additional metadata such as the data model (or schema) followed in the dataset or even related terms defined in a business glossary. Data catalog solutions typically implement collectors that ingest metadata from the data sources themselves and external metadata sources. For example, a Kafka Schema Registry is a metadata source that provides metadata about the data models followed by some data stored in a Kafka topic. In this sense, a YANG-enabled network device can be considered as another kind of data source, which the Data Catalog can pull metadata from. For instance, the data catalog can include a connector that fetches metadata about the YANG modules implemented by the network device. Combining these metadata with other such as the business concept "interface", would enable data consumers to discover which datasets related to the concept "interface" are exposed by the network device. Network devices that implement YANG library expose metadata about which YANG modules are implemented, and which are only imported. However, what a data consumer needs at the end are the YANG modules implemented by the device, hence, the combination of implemented YANG modules with other YANG modules that might deviate or augment the formers. Coming back to the example of datasets related to the "interface" concept, say we have a network device that implements the ietf-interfaces module and the ietf-ip module , where the latter augments the former. For a data catalog to collect these metadata, a connector would retrieve YANG library data from the target device. However, the current version of YANG library would not satisfy the use case as it would tell that the device implements both ietf-interfaces and ietf-ip modules, but will miss the augment dependency between them. The current workaround is in combination with the YANG library data to additionally obtain both YANG modules and process them to discover that there is an augment dependency. This adds extra burden on the connector, which is forced to combine multiple metadata collection mechanisms. This process could be softened by extending YANG library to also capture augment dependencies, in a similar fashion to deviation dependencies.

This YANG module augments the ietf-yang-library module by adding the augmented-by list in the "yang-library/module-set" and "yang-library/modules-state". The "yang-library/module-state" is augmented despite of its "deprecated" state to cope with the situation when container "modules-state" is used for compatibility reason with ietf-yang-library defined in . The name of list "augmented-by" indicates by which modules that the current module is being directly augmented. For the scope of "augmented-by", this draft only considers the direct augmentation relationship. The recursive result of augmentation or transitive dependency for module specified along the xpath, are out of the scope of this draft. Section 4.2 has given the implementation instructions

The following is the YANG tree diagram for model ietf-yang-library-augmentedby.

../../yanglib:module/name augment /yanglib:modules-state/yanglib:module: +--ro augmented-by* -> ../../yanglib:module/name ]]>

The following is the YANG tree diagram for module ietf-yang-library after adding augmentation from module ietf-yang-library-augmentedby. The RPCs and notifications are included as well.

../../module/name | | | +--ro yanglib-aug:augmented-by* -> ../../yanglib:module/name | | +--ro import-only-module* [name revision] | | +--ro name yang:yang-identifier | | +--ro revision union | | +--ro namespace inet:uri | | +--ro location* inet:uri | | +--ro submodule* [name] | | +--ro name yang:yang-identifier | | +--ro revision? revision-identifier | | +--ro location* inet:uri | +--ro schema* [name] | | +--ro name string | | +--ro module-set* -> ../../module-set/name | +--ro datastore* [name] | | +--ro name ds:datastore-ref | | +--ro schema -> ../../schema/name | +--ro content-id string x--ro modules-state x--ro module-set-id string x--ro module* [name revision] x--ro name yang:yang-identifier x--ro revision union +--ro schema? inet:uri x--ro namespace inet:uri x--ro feature* yang:yang-identifier x--ro deviation* [name revision] | x--ro name yang:yang-identifier | x--ro revision union x--ro conformance-type enumeration x--ro submodule* [name revision] | x--ro name yang:yang-identifier | x--ro revision union | +--ro schema? inet:uri +--ro yanglib-aug:augmented-by* -> ../../yanglib:module/name notifications: +---n yang-library-update | +--ro content-id -> /yang-library/content-id x---n yang-library-change x--ro module-set-id -> /modules-state/module-set-id ]]>

The YANG module source code of ietf-yang-library-augmentedby in which augmentation to the ietf-yang-library of is defined.

file "ietf-yang-library-augmentedby@2023-10-27.yang" module ietf-yang-library-augmentedby { yang-version 1.1; namespace "urn:ietf:params:xml:ns:yang:ietf-yang-library-augmentedby"; prefix yanglib-aug; import ietf-yang-library { prefix yanglib; reference "RFC 8525: YANG Library"; } organization "IETF NETCONF (Network Configuration) Working Group"; contact "WG Web: WG List: Author: Zhuoyao Lin Benoit Claise IGNACIO DOMINGUEZ MARTINEZ-CASANUEVA "; description "This module augments the ietf-yang-library defined in [RFC8525] to provide not only the deviation list, but also the augmented-by list, in order to give sufficient information about the YANG modules reverse dependency. It facilitates the process of obtaining the entire dependencies of YANG module. The key words 'MUST', 'MUST NOT', 'REQUIRED', 'SHALL', 'SHALL NOT', 'SHOULD', 'SHOULD NOT', 'RECOMMENDED', 'NOT RECOMMENDED', 'MAY', and 'OPTIONAL' in this document are to be interpreted as described in BCP 14 (RFC 2119) (RFC 8174) when, and only when, they appear in all capitals, as shown here. Copyright (c) 2022 IETF Trust and the persons identified as authors of the code. All rights reserved. Redistribution and use in source and binary forms, with or without modification, is permitted pursuant to, and subject to the license terms contained in, the Revised BSD License set forth in Section 4.c of the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info). This version of this YANG module is part of RFC XXXX; see the RFC itself for full legal notices. "; revision 2025-05-28 { description "Initial revision."; reference "RFC XXXX: Support of augmentedby in ietf-yang-library"; } grouping augmented-by { description "Augment the augmented-by list from module info with the module-augmented-by grouping" ; leaf-list augmented-by { type leafref { path "../../yanglib:module/yanglib:name"; } description "Leaf-list of the augmentation used by this server to modify the conformance of the module associated with this entry. Note that the same module can be used for augmented-by for multiple modules, so the same entry MAY appear within multiple 'module' entries. This reference MUST NOT (directly or indirectly) refer to the module being augmented. Robust clients may want to make sure that they handle a situation where a module augments itself (directly or indirectly) gracefully."; } } augment "/yanglib:yang-library/yanglib:module-set/yanglib:module" { description "Augment the augmented-by list from module info with the augmented-by grouping" ; uses augmented-by; } augment "/yanglib:modules-state/yanglib:module" { status deprecated; description "Augment the augmented-by list from module info with the augmented-by grouping" ; uses augmented-by; } } ]]>

This section defines the scope of augmented-by. The "augmented-by" list should only consider those YANG modules that directly augment the YANG module in question in the ietf-yang-library. The "directly augment" is identified by the relationship between the augment module and the target node's parent module that it augments to. Only the direct parent module of the target node is augmented, and the rest of parent modules defined in the schema tree are only indirect dependencies but not augmented modules. (Refer to "Target node" definition in Section 7.17 of ) In the case when a YANG application requires recursive dependency or specific schema tree dependency, the search logic should be implemented by the application itself. A YANG example with the expected augmented-by result is provided in Section 4.2.2.

There are module A, B, C, D and E, which have the following relationships:

• Module A is the base module with container "foo-a"
• Module B augments "/a:foo-a" with container "foo-b"
• Module C augments "/a:foo-a/b:foo-b" with leaf "leaf-c", and it defines a container "foo-c"
• Module D augments "/c:foo-c" with container "foo-d"
• Module E augments "/c:foo-c" with contaienr "foo-e"

The augmented-by result for module A, B and C is the following:

• Module A is augmented-by: Module B
• Module B is augmented-by: Module C
• Module C is augmented-by: Module D, E

Module D, E have no augmented-by result.

Note to the RFC-Editor: Please remove this section before publishing (This follows the template in RFC7942).

Zhuoyao Lin did the prototype implementation of the augmented-by list feature of this draft and demonstrated it based on Netopeer2 in IETF 119 Hackathon. Netopeer2 is a NETCONF server & client implementation developed by CESNET. Source code is here: . The actual feature is implemented by extending the libyang and sysrepo which are the base libraries for Netopeer2 to support populating the augmented-by list.

Zhuoyao Lin did a docker image of netopeer2 that integrates the augmented-by feauture in sysrepo and libyang. The result is presented at IETF 120 hackathon. The source code can be obtained here:

Zhuoyao Lin did an implementation of find-dependency based on the ietf-yang-library with augmented-by feature in the YANG-Push message parser library libyangpush. The result is presented in IETF 120 hackathon. The source code can be obtained here:

The list name has been updated from "augmentation" to "augmented-by", in order to represent the usage clearly. The leafref has been changed from absolute path "/yanglib:yang-libraray/yanglib:module-set/yanglib:module/yanglib:name" to relative path "../../yanglib:module/yanglib:name". The YANG validation in the appendix A shows that this path can work as expected. Section 5 Implementation and section 6 Changes has been added.

Updated the Use case content in Section 3.1. Add explanation: the scope of use case "Data Mesh Architecture" is limited to configured subscription. Updated Implementation status content.

Updated affiliations Update content of Section 3.1 Data Mesh use case. Explain the limitation of applying get-all-schemas solution under the background of using UDP-notif of configured subscription, and how the feature proposed in the draft can improve the solution. Full review of document. Nits and refinement of sections.

Rewrite Section 2 Motivation. Update Section 6 Changes's subsection title. Update the Section 7 security consideration and section 8 IANA Considerations. Added in the appendix the Impact Analysis of ietf-yang-library and proposal for the RFC8525bis draft.

Resubmitted the draft name from: draft-lincla-netconf-yang-library-augmentedby-02 to: draft-ietf-netconf-yang-library-augmentedby-00

Correct the yanglint validation invalid example. Updated the explaination to the yanglint validation example principle. Delete Section "ietf-yang-library Impact Analysis, as an evaluation for RFC8525bis". The idea of updating the RFC8525 is paused.

Update and rephrase the Introduction section. Add Section 4.2 Implementation Instructions. Address in Section 4.2.1 that the definition of "augmented-by" only consider the direct augment. A YANG example for explaining this purpose has been put into Section 4.2.2. Draft refinement. Reference update.

Merge review comment from Thomas.

Update module content ietf-yang-library-augmentedby: Organise the augmentation content to grouping; Add augmentation to modules-state container. Appendix B is deleted.

Update ietf-yang-library-augmentedby module revision.

The YANG module specified in this document defines a schema for data that is designed to be accessed via network management protocols such as NETCONF or RESTCONF . The lowest NETCONF layer is the secure transport layer, and the mandatory-to-implement secure transport is Secure Shell (SSH) . The lowest RESTCONF layer is HTTPS, and the mandatory-to-implement secure transport is TLS . The Network Configuration Access Control Model (NACM) provides the means to restrict access for particular NETCONF or RESTCONF users to a preconfigured subset of all available NETCONF or RESTCONF protocol operations and content. The readable node defined in this YANG module may be considered sensitive or vulnerable in some network environments. It is thus important to control read access(e.g., via get, get-config, or notification) to this data node. The following is the explanation to data node's sensitivity/vulnerability: The "augmented-by" node in this YANG module could reveal all modules that are augmenting one module. It could help attacker identify the relationship between modules and server implementations known bugs. Server vulnerabilities may include but not restricted to: 1. Too many augmented-by records causes buffer overflow. 2. The augmented-by node help identify through the inter-relation of modules how to cause the server to crash or significantly degrade device performance.

This document registers one URI in the "IETF XML Registry" . Following the formate in , the following registration has been made. URI: urn:ietf:params:xml:ns:yang:ietf-yang-library-augmentedby Registration Contact: The NETCONF WG of the IETF. XML: N/A, the requested URI is an XML namespace. This document registers one YANG module in the "YANG Module Names" registry name: ietf-yang-library-augmentedby namespace: urn:ietf:params:xml:ns:yang:ietf-yang-library-augmentedby prefix: yanglib-aug reference:

BSD-3-Clause license BSD-3-Clause license BSD-3-Clause license INSA-Lyon INSA-Lyon Swisscom Huawei Huawei Huawei Telefonica This document augments the ietf-yang-library to provide the augmented-by list. It facilitates the process of obtaining the entire dependencies between YANG modules, by directly querying the server's YANG module. Swisscom This document describes the motivation and architecture of a native YANG-Push notifications and YANG Schema integration into Apache Kafka Message Broker and YANG Schema Registry. Universidad Politecnica de Madrid, Telefonica I+D Universidad Politecnica de Madrid Universidad Politecnica de Madrid Universidad Politecnica de Madrid Telefonica I+D IEEE Communications Magazine

This section gives a few examples that the user can try themselves with yanglint. This is created to prove the syntax correctness. The examples shoud be used with YANG modules ietf-yang-library and ietf-yang-libarary-augmentedby as schemas. The examples provided are ietf-yang-library 'yang-library' data xml file containing the augmented-by field. The valid example should pass the validation while the invalid one will not. The difference is that in the invalid example, the module in one module-set has augmented module in another module-set, which is illegal according to the ietf-yang-library definition.

file "example_valid.xml" 1 ms1 module1 2024-02-29 urn:ietf:params:xml:ns:yang:module1 module2 module3 module2 2024-02-29 urn:ietf:params:xml:ns:yang:module2 module3 2024-02-29 urn:ietf:params:xml:ns:yang:module3 0 ]]>

file "example_invalid.xml" 1 ms1 module1 2024-02-29 urn:ietf:params:xml:ns:yang:module1 module3 module2 module3 2024-02-29 urn:ietf:params:xml:ns:yang:module3 ms2 module2 2024-02-29 urn:ietf:params:xml:ns:yang:module2 0 ]]>

The following people all contributed to creating this document:

The author would like to thank Jan Lindblad and Jean Quilbeuf for their help during the design of the YANG module, and Thomas Graf, Rob Wilton, Andy Bierman for their valuable review and comments.