]> Cisco

jlindbla@cisco.com

OPS NETMOD YANG telemetry collection aggregation time series database TSDB Timestamped telemetry data is collected en masse today. Mature tools are typically used, but the data is often collected in an ad hoc manner. While the dashboard graphs look great, the resulting data is often of questionable quality, not well defined, and hard to compare with seemingly similar data from other organizations. This document proposes a standard, extensible, cross domain framework for collecting and aggregating timestamped telemetry data in a way that combines YANG, metadata and Time Series Databases to produce more dependable and comparable results. About This Document The latest revision of this draft can be found at . Status information for this document may be found at . Source for this draft and an issue tracker can be found at .

Introduction

The Problem Many organizations today are collecting large amounts of telemetry data from their networks and data centers for a variety of purposes. Much (most?) of this data is funneled into a Time Series Database (TSDB) for display in a dashboard or further (AI-backed) processing and decision making. While this data collection is often handled using standard tools, there generally seems to be little commonality when it comes to what is meaured, how the data is aggregated, or definitions of the measured quantities (if any). Data science issues like adding overlapping quantities, adding quantities of different units of measurement, or quantities with different scopes, are likely common. Such errors are hard to detect given the ad hoc nature of the collection. This often leads to uncertainty regarding the quality of the conclusions drawn from the collected data.

The Solution The Philatelist framework proposes to standardize the collection, definitions of the quantities measured and meta data handling to provide a robust ground layer for telemetry collection. The architecture defines a few interfaces, but allows great freedom in the implementations with its plug-in architecture. This allows flexibility enough that any kind of quantitiy can be measured, any kind of collection protocol and mechanism employed, and the data flows aggregated using any kind of operation. To do this, YANG is used both to describe the quantities being measured, as well as act as the framework for the metadata management. Note that the usa of YANG here does not limit the architecture to traditional YANG-based transport protocols. YANG is used to describe the data, regardless of which format it arrives in. Initially developed in context of the Power and Energy Efficiency work (POWEFF), we realized both the potential and the need for this collection and aggregation architecture to become a general framework for collection of a variety of metrics. There is not much point in knowing the "cost side" of a running system (as in energy consumption or CO2-emissions) if that cannot be weighed against the "value side" delivered by the system (as in transported bytes, VPN connections, music streaming hours, or number of cat videos, etc.), which means traditional performance metrics will play an equally important role in the collection. In this initial version, we have done nothing to pull the proposed YANG modules out of its POWEFF roots and generalize it for general telemetry. We believe the ideas and merits of this framework architecture will be apparent nonetheless in this first version. For the next version, we certainly need to generalize the quantities measured and rename the YANG modules and node names.

The Philatelist Name This specification is about a framework for collection, aggregation and interpretation of timestamped telemetry data. The definition of "philatelist" seems close enough.

Source: https://www.synonym.com/synonyms/philatelist

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. This document uses the terminology defined in . In addition, this document defines the following terms:

TSDB

Time Series Database.

Sensor

An entity in a system that delivers a snapshot value of some quantity pertaining to the system. Sensors are identified by their Sensor Path.

Sensor Path

A textual representation of the sensor's address within the system.

Architecure Overview The deployment of a Philatelist framework consists of a collection of plug-in compomnents with well defined interfaces. Here is an example of a deployment. Each box is numbered in the lower right for easy reference.

Example Philatelist component deployment.

Each component in the above diagram, represents a logical function. Many boxes could be running within a single server, or they could be fully distrubuted, or anything in between. Provider components (61, 82, 91) are running on a telemetry source system that supports a YANG-based telemetry data server. The telemetry data flows from the telemetry source system to a Time Series Database (TSDB). Collector components (51, 72, 81) ensure the Providers are programmed properly to deliver the telemetry data to the TSDB designated by the collector. In some cases this flow may be direct from the source to the TSDB, in other cases, it may be going through the collector. In some cases the collector may be polling the source, in others it may have set up an automatic, periodic subscription. Many telemetry source systems will not have any on-board YANG-based telemetry server. Such servers will instead be managed by a collector specialized to handle a particular kind of source server (53, 54). These specialized collectors are still responsible to set up a telemetry data stream from them to the collector's TSDB. In this case, the collector will also supply a YANG description (52, 55) of the incoming data stream. Processor components (21, 41, 71) are transforming the data stream in some way, e.g. converting from one unit of measurement to another, or adjusting the data values recorded to also include some aspect that this particular sensor is not taking into account. Aggregator components (31, 42, 43, 44) combine the time series telemetry data flows using some operation, e.g. summing, averaging or computing the max or min over them. In this example there are aggregators for Network, Storage, Compute and the entire Data Center On top of the stack, we may often find a (graphical) user interface (11), for human consumption of the intelligence acquired by the system. Equally relevant is of course an (AI) application making decisions based on findings in the aggregated telemetry flow.

The Provider Component A Provider is a source of telemetry data that also offers a YANG-based management interface. Each provider typically has a large number of "sensors" that can be polled or in some cases subscribed to. One problem with the sensors is that they are spread around inside the source system, and may not be trivial to locate. Also, the metadata assciated with the sensor is often only missing or only available in human readable form (free form strings), rather than in a strict machine parsable format.

Example of scattered potential sensors in a device.

To solve these problems, the Provider YANG module contains a sensor-catalog list. Essentially a list of all interesting sensors available on the system, with their sensor paths and machine parsable units, definition and any other metadata. An admin user or application can then copy the sensor definition from the sensor catalog and insert into the configuration in the colletor.

YANG tree diagram of the Provider sensor-catalog.

Note: The "something" YANG-type is used in many places in this document. That is just a temporarty placeholder we use until we have figured out what the appropriate type should be. The sensor types are defined as YANG identities, making them maximally extensible. Examples of sensor types might be energy measured in kWh, or energy measured in J, or temperature measured in F, or in C, or in K.

The Collector Component Collector components collect data points from sources, typically by periodic polling or subscriptions, and ensure the collected data is stored in a Time Series Database (TSDB). The actual data stream may or may not be passing through the collector component; the collector is responsible for ensuring data flows from the source to the destination TSDB and that the data has a YANG description and is tagged with necessary metadata. How the collector agrees with a source to deliver data in a timely manner is beyond the scope of this document.

Example of Collector setting up three streams of telemetry data from two sources to one desitination. | | | - sensor 1 | | - sensor 1 | | | | - sensor 2 | | - sensor 4 | STREAM 2 | | | - sensor 3 | | - sensor 7 | =======> | | +------------+ +------------+ | | \\ STREAM 3 | | =============================> \___________/ ]]>

Each source holds a number of sensors that may be queried or subscribed to. The collector arranges the sensors into sensour groups that presumably are logically related, and that are collected using the same method. A number of collection methods (some YANG-based, some not) are modeled directly in the ietf-poweff-collector.yang module, but the set is designed to be easily extensible.

YANG tree diagram of the Collector sensor-groups and streams. ../../../sensor-groups/sensor-group/id +-- destination? -> ../../../destinations/destination/id ]]>

The sensor groups are then arranged into streams from a collection of sources (that support the same set of sensor groups) to a destination. This structure has been chosen with the assumption that there will be many source devices with the same set of sensor groups, and we want to minimize repetition.

The Processor and Aggregator Components Processor components take an incoming data flow and transforms it somehow, and possibly augments it with a flow of derived information. The purpose of the transformation could be to convert between different units of measurement, correct for known errors in in the input data, or fill in approximate values where there are holes in the input data. Aggregator components take multiple incoming data flows and combine them, typically by adding them together, taking possible differences in cadence in the input data flows into account. Processor and Aggregator components provide a YANG model of the output data, just like the Collector components, so that all data flowing in the system has a YANG description and is associated with metadata. Note: In the current version of the YANG modules, a Processor is simply an Aggregator with a single input and output. Unless we see a need to keep these two component types separate, we might remove the Processor component and keep it baked in with the Aggregator.

Example of an Aggregator setting up two flows of telemetry data from two sources to one desitination. | | | | | | | | FLOW 2 | | \___________/ ===##=> \___________/ || ___________ || / \ || ( SOURCE 2 ) // |\___________/| == | | | | | | \___________/ ]]>

In this diagram, the sources and destination look like separate TSDBs, which they might be. They may also be different buckets within the same TSDB. Each flow is associated with one or more inputs, one output and a series of processing operations. Each input flow and output flow may have an pre-processing or post-processing operation applied to it separately. Then all the input flows are combined using one or more aggregation operations. Some basic operations have been defined in the Aggregator YANG module, but the set of operations has been designed to be maximally extensible.

YANG tree diagram of the Aggregator flows and operations. ../../../../../sources/source/id | +--:(output) | | +-- output | | +-- destination? | | -> ../../../../../destinations/destination/id | +--:(middle) | +-- middle | +-- inputs* | | -> ../../../../flows/flow/id | +-- pre-process-inputs? | | -> ../../../../operations/operation/id | +-- aggregate? | | -> ../../../../operations/operation/id | +-- post-process-output? | -> ../../../../operations/operation/id +-- operations +-- operation* [id] +-- id? something +-- (op-type)? +--:(linear-sum) | +-- linear-sum +--:(linear-average) | +-- linear-average +--:(linear-max) | +-- linear-max +--:(linear-min) | +-- linear-min +--:(rolling-average) | +-- rolling-average | +-- timespan? something +--:(filter-age) | +-- filter-age | +-- min-age? something | +-- max-age? something +--:(function) +-- function +-- name? something ]]>

The operations listed above are basic aggregation operations. Linear-sum is just adding all the input sources together, with linear interpolation when their data points don't align perfectly in time. Rolling average is averaging the input flows over a given length of time. The filter-age drops all data points that are outside the min to max age. The function allows plugging in any other function the Aggregator may have defined, but more importantly, the operations choice is easily extended using YANG augment to include any other IETF or vendor specified extensions.

YANG-based Telemetry Outlook Much work has already gone into the area of telemetry, YANG, and even their intersection. E.g. and come to mind. Even though this work has a solid foundation and shares many or most of the goals with this work, we (the POWEFF team) have not found it easy to apply the above work directly in the practical work we do. So what we have tried to do is a very pragmatic approach to telemetry data collection the way we see it happening on the ground combined with the benefits of Model Driven Telemetry (MDT), in practice meaning YANG-based with additional YANG-modeled metadata. Many essential data sources in real world deployments do not support any YANG-based interfaces, and that situation is expected to remain for the forseable future, which is why we find it important to be able to ingest data from free form (often REST-based) interfaces, and then add the necessary rigor on the Collector level. Then output the datastreams in formats that existing, mature tools can consume directly. In particular, this draft depends on the mapping of YANG-based structures to the typical TSDB tag-based formats described in . For the evolution of the YANG-based telemetry area, we believe this approach, combining pragmatism in the data flow interfaces with rigor regarding the data content, is key. We would like to make this work fit in with the works of others in the field.

YANG Modules

Base types module for Philatelist WG List: Editor: Jan Lindblad Editor: Snezana Mitrovic Editor: Marisol Palmero "; description "This YANG module defines basic quantities, measurement units and sensor types for the POWEFF framework. Copyright (c) 2021 IETF Trust and the persons identified as authors of the code. All rights reserved. Redistribution and use in source and binary forms, with or without modification, is permitted pursuant to, and subject to the license terms contained in, the Simplified BSD License set forth in Section 4.c of the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info). This version of this YANG module is part of RFC XXXX (https://www.rfc-editor.org/info/rfcXXXX); see the RFC itself for full legal notices."; revision 2023-10-12 { description "Initial revision of POWEFF types"; reference "RFC XXXX: ..."; } typedef something { // FIXME: Used when we haven't decided the type yet type string; } typedef xpath { type string; // FIXME: Proper type needed } typedef sample-frequency { type string; // FIXME: Proper type needed } // ========== SENSOR-CLASS ============================== identity sensor-class { description "Sensor's relation to the asset it sits on."; } identity sc-input { base sensor-class; description "Sensor reports input quantity of the asset it sits on."; } identity sc-output { base sensor-class; description "Sensor reports output quantity of the asset it sits on."; } identity sc-allocated { base sensor-class; description "Sensor reports (maximum) allocated quantity of the asset it sits on."; } // ========== SENSOR-QUANTITY ============================== identity sensor-quantity { description "Sensor's quantity being measured."; } identity sq-voltage { base sensor-quantity; description "Sensor reports electric tension, voltage."; } identity sq-current { base sensor-quantity; description "Sensor reports electric current."; } identity sq-power { base sensor-quantity; description "Sensor reports power draw (energy per unit of time)."; } identity sq-power-apparent { base sq-power; description "Sensor reports apparent power, i.e. average electrical current times voltage (in VA)."; } identity sq-power-true { base sq-power; description "Sensor reports true power, i.e. integral over current and voltage at each instant in time."; } identity sq-energy { base sensor-quantity; description "Sensor reports actual energy drawn by asset."; } identity sq-co2-emission { base sensor-quantity; description "Sensor reports CO2 (carbon dioxide) emission by asset."; } identity sq-co2eq-emission { base sensor-quantity; description "Sensor reports CO2 (carbon dioxide) equivalent emission by asset."; } identity sq-temperature { base sensor-quantity; description "Sensor reports temperature of asset."; } // ========== SENSOR-UNIT ============================== identity sensor-unit { description "Sensor's unit of reporting."; } identity su-volt { base sensor-unit; base sq-voltage; description "Sensor unit volt, V."; } identity su-ampere { base sensor-unit; base sq-current; description "Sensor unit ampere, A."; } identity su-watt { base sensor-unit; base sq-power; description "Sensor unit watt, W."; } identity su-voltampere { base sensor-unit; base sq-power; description "Sensor unit Volt*Ampere, VA."; } identity su-kw { base sensor-unit; base sq-power; description "Sensor unit kilowatt, kW."; } identity su-joule { base sensor-unit; base sq-energy; description "Sensor unit joule, J."; } identity su-wh { base sensor-unit; base sq-energy; description "Sensor unit watthour, Wh."; } identity su-kwh { base sensor-unit; base sq-energy; description "Sensor unit kliowatthour, kWh."; } identity su-kelvin { base sensor-unit; base sq-temperature; description "Sensor unit kelvin, K."; } identity su-celsius { base sensor-unit; base sq-temperature; description "Sensor unit celsius, C."; } identity su-farenheit { base sensor-unit; base sq-temperature; description "Sensor unit farenheit, F."; } identity su-gram { base sensor-unit; base sq-co2-emission; description "Sensor unit gram, g."; } identity su-kg { base sensor-unit; base sq-co2-emission; description "Sensor unit kliogram, kg."; } identity su-ton { base sensor-unit; base sq-co2-emission; description "Sensor unit ton, t."; } // ========== SENSOR-TYPE ============================== identity sensor-type { description "Sensor's type, i.e. combination of class, quantity and unit."; } identity st-v-in { base sensor-type; base sc-input; base sq-voltage; base su-volt; description "Sensor reporting Voltage In to asset."; } identity st-v-out { base sensor-type; base sc-output; base sq-voltage; base su-volt; description "Sensor reporting Voltage Out of asset."; } identity st-i-in { base sensor-type; base sc-input; base sq-current; base su-ampere; description "Sensor reporting Current In to asset."; } identity st-i-out { base sensor-type; base sc-output; base sq-current; base su-ampere; description "Sensor reporting Current Out of asset."; } identity st-p-in-apparent-watt { base sensor-type; base sc-input; base sq-power-apparent; base su-voltampere; description "Sensor reporting Power In to asset as apparent (I*U) power."; } identity st-p-out-apparent-watt { base sensor-type; base sc-output; base sq-power-apparent; base su-voltampere; description "Sensor reporting Power Out of asset as apparent (I*U) power."; } identity st-p-in-true-watt { base sensor-type; base sc-input; base sq-power-true; base su-watt; description "Sensor reporting Power In to asset as true power."; } identity st-p-out-true-watt { base sensor-type; base sc-output; base sq-power-true; base su-watt; description "Sensor reporting Power Out of asset as true power."; } identity st-p-allocated-watt { base sensor-type; base sc-allocated; base sq-power; base su-watt; description "Sensor reporting Allocated Power for asset."; } identity st-w-j { base sensor-type; base sq-energy; base su-joule; description "Sensor reporting energy draw of asset in J."; } identity st-w-wh { base sensor-type; base sq-energy; base su-wh; description "Sensor reporting energy draw of asset in Wh."; } identity st-w-kwh { base sensor-type; base sq-energy; base su-kwh; description "Sensor reporting energy draw of asset in kWh."; } identity st-t-k { base sensor-type; base sq-temperature; base su-kelvin; description "Sensor reporting Temperature of asset in K."; } identity st-t-c { base sensor-type; base sq-temperature; base su-celsius; description "Sensor reporting Temperature of asset in °C."; } identity st-t-f { base sensor-type; base sq-temperature; base su-farenheit; description "Sensor reporting Temperature of asset in °F."; } // ========== COLLECTION-METHOD ============================== identity collection-method; identity cm-polled { base collection-method; } identity cm-gnmi { base collection-method; } identity cm-restconf { base collection-method; } identity cm-netconf { base collection-method; } identity cm-redfish { base collection-method; } identity get-static-url-once { base collection-method; } identity gnmi-polling { base cm-gnmi; base cm-polled; } identity restconf-get-polling { base cm-restconf; base cm-polled; } identity netconf-get-polling { base cm-netconf; base cm-polled; } identity restconf-yang-push-subscription { base cm-restconf; } identity netconf-yang-push-subscription { base cm-netconf; } identity redfish-polling { base cm-redfish; } } ]]>

Provider interface module for Philatelist WG List: Editor: Jan Lindblad Editor: Snezana Mitrovic Editor: Marisol Palmero "; description "This YANG module defines the POWEFF Provider. Copyright (c) 2021 IETF Trust and the persons identified as authors of the code. All rights reserved. Redistribution and use in source and binary forms, with or without modification, is permitted pursuant to, and subject to the license terms contained in, the Simplified BSD License set forth in Section 4.c of the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info). This version of this YANG module is part of RFC XXXX (https://www.rfc-editor.org/info/rfcXXXX); see the RFC itself for full legal notices."; revision 2023-10-12 { description "Initial revision of POWEFF Provider"; reference "RFC XXXX: ..."; } grouping provider-g { container sensor-catalog { config false; container sensors { list sensor { key path; leaf path { type ietf-poweff-types:xpath; } leaf sensor-type { type identityref { base ietf-poweff-types:sensor-type; }} leaf sensor-location { type ietf-poweff-types:something; description "Indicates the current location where the sensor is located in the chassis,typically refers to slot"; } leaf sensor-state { // FIXME: What does this mean? type ietf-poweff-types:something; description "Current state of the sensor"; } leaf sensor-current-reading { // FIXME: Do we want a copy of the value here? type ietf-poweff-types:something; description "Current reading of the sensor"; } leaf sensor-precision { type string; description "Maximum deviation to be considered. This attribute mainly will apply to drawn power, which corresponds to PSU PowerIn measured power or calculated power; assuming discrepancy between Real Power, power collected from a power meter, and power measured or calculated from the metrics provided by the sensors"; } container sensor-thresholds { // FIXME: Is this for generating alarms, or what? description "Threshold values for the particular sensor. Default values shall beprovided as part of static data but when configurable need to be pulledfrom the device. Ideally, the sensor should allow configuing thesethreshold values"; leaf minor-low { type string; description "minor-low"; } leaf minor-high { type string; description "minor-high"; } leaf major-low { type string; description "major-low"; } leaf major-high { type string; description "major-high"; } leaf critical-low { type string; description "critical-low"; } leaf critical-high { type string; description "critical-high"; } leaf shutdown { // FIXME: What does this mean for a sensor? type string; description "shutdown"; } } } } } } } ]]>

Collector interface module for Philatelist WG List: Editor: Jan Lindblad Editor: Snezana Mitrovic Editor: Marisol Palmero "; description "This YANG module defines the POWEFF Collector. Copyright (c) 2021 IETF Trust and the persons identified as authors of the code. All rights reserved. Redistribution and use in source and binary forms, with or without modification, is permitted pursuant to, and subject to the license terms contained in, the Simplified BSD License set forth in Section 4.c of the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info). This version of this YANG module is part of RFC XXXX (https://www.rfc-editor.org/info/rfcXXXX); see the RFC itself for full legal notices."; revision 2023-10-12 { description "Initial revision of POWEFF Collector"; reference "RFC XXXX: ..."; } /* A COLLECTOR programs one or more SOURCE(s) to generate a STREAM of telemetry data. The STREAM is sent to a specific DESTINATION. Each STREAM consists of timestamped sensor values from each sensor in a sensor group. +-------------+ | COLLECTOR | +-------------+ ___________ | / \ +------------------+ ( DESTINATION ) v v |\___________/| +------------+ +------------+ STREAM 1 | | | SOURCE | | SOURCE | =======> | | | - sensor 1 | | - sensor 1 | | | | - sensor 2 | | - sensor 4 | STREAM 2 | | | - sensor 3 | | - sensor 7 | =======> | | +------------+ +------------+ | | \\ STREAM 3 | | =============================> \___________/ */ grouping data-endpoint-g { leaf url { type ietf-poweff-types:something; } leaf organization { type ietf-poweff-types:something; } leaf bucket { type ietf-poweff-types:something; } container impl-specific { list binding { key key; leaf key { type string; } choice value-type { leaf value { type string; } leaf-list values { type string; ordered-by user; } leaf env-var { type string; } } } } } grouping sensor-group-g { leaf method { type identityref { base ietf-poweff-types:collection-method; } } container get-static-url-once { when "derived-from-or-self(../method, 'ietf-poweff-types:get-static-url-once')"; leaf url { type ietf-poweff-types:something; } leaf format { type ietf-poweff-types:something; } // JSON-IETF, XML, etc } container gnmi-polling { when "derived-from-or-self(../method, 'ietf-poweff-types:gnmi-polling')"; leaf encoding { type ietf-poweff-types:something; } // self-describing-gpb leaf protocol { type ietf-poweff-types:something; } // protocol grpc no-tls } container restconf-get-polling { when "derived-from-or-self(../method, 'ietf-poweff-types:restconf-get-polling')"; leaf xxx { type string; } } container netconf-get-polling { when "derived-from-or-self(../method, 'ietf-poweff-types:netconf-get-polling')"; leaf xxx { type string; } } container restconf-yang-push-subscription { when "derived-from-or-self(../method, 'ietf-poweff-types:restconf-yang-push-subscription')"; leaf xxx { type string; } } container netconf-yang-push-subscription { when "derived-from-or-self(../method, 'ietf-poweff-types:netconf-yang-push-subscription')"; leaf xxx { type string; } } container redfish-polling { when "derived-from-or-self(../method, 'ietf-poweff-types:redfish-polling')"; leaf xxx { type string; } } leaf frequency { when "derived-from(../method, 'ietf-poweff-types:cm-polled')"; type ietf-poweff-types:sample-frequency; } list path { key path; leaf path { type ietf-poweff-types:xpath; } leaf sensor-type { type identityref { base ietf-poweff-types:sensor-type; }} leaf attribution { type string; } } } grouping collector-g { container poweff-collector { container destinations { list destination { key id; leaf id { type ietf-poweff-types:something; } uses data-endpoint-g; } } container sensor-groups { list sensor-group { key id; leaf id { type ietf-poweff-types:something; } uses sensor-group-g; } } container streams { list stream { key id; leaf id { type ietf-poweff-types:something; } leaf-list source { type string; } // Implementation specific meaning, possibly wildcards list sensor-group { key name; leaf name { type leafref { path ../../../sensor-groups/sensor-group/id; }} } leaf destination { type leafref { path ../../../destinations/destination/id; }} } } } } } ]]>

Aggregator interface module for Philatelist WG List: Editor: Jan Lindblad Editor: Snezana Mitrovic Editor: Marisol Palmero "; description "This YANG module defines the POWEFF Aggregator. Copyright (c) 2021 IETF Trust and the persons identified as authors of the code. All rights reserved. Redistribution and use in source and binary forms, with or without modification, is permitted pursuant to, and subject to the license terms contained in, the Simplified BSD License set forth in Section 4.c of the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info). This version of this YANG module is part of RFC XXXX (https://www.rfc-editor.org/info/rfcXXXX); see the RFC itself for full legal notices."; revision 2023-10-12 { description "Initial revision of POWEFF Aggregator"; reference "RFC XXXX: ..."; } /* An AGGREGATOR ensures data from one or more SOURCE(s) are combined into a FLOW using a (sequence of) OPERATIONs (OPs) to generate a new data set in the DESTINATION (which could be a new collection in the same data storage system as the SOURCE). +-------------+ | AGGREGATOR | +-------------+ | +-----------+-----------+ v v ___________ ___________ / \ / \ ( SOURCE 1 ) ( DESTINATION ) |\___________/| FLOW 1 |\___________/| | | ======> | | | | | | | | FLOW 2 | | \___________/ ===##=> \___________/ || ___________ || / \ || ( SOURCE 2 ) // |\___________/| == | | | | | | \___________/ */ grouping aggregator-g { container poweff-aggregator { container sources { list source { key id; leaf id { type ietf-poweff-types:something; } uses ietf-poweff-collector:data-endpoint-g; } } container destinations { list destination { key id; leaf id { type ietf-poweff-types:something; } uses ietf-poweff-collector:data-endpoint-g; } } container flows { list flow { key id; leaf id { type string; } choice chain-position { container input { leaf source { type leafref { path ../../../../../sources/source/id; }} } container output { leaf destination { type leafref { path ../../../../../destinations/destination/id; }} } container middle { leaf-list inputs { type leafref { path ../../../../flows/flow/id; }} leaf pre-process-inputs { type leafref { path ../../../../operations/operation/id; }} leaf aggregate { type leafref { path ../../../../operations/operation/id; }} leaf post-process-output { type leafref { path ../../../../operations/operation/id; }} } } } } container operations { list operation { key id; leaf id { type ietf-poweff-types:something; } choice op-type { container linear-sum {} container linear-average {} container linear-max {} container linear-min {} container rolling-average { leaf timespan { type ietf-poweff-types:something; } } container filter-age { leaf min-age { type ietf-poweff-types:something; } leaf max-age { type ietf-poweff-types:something; } } container function { leaf name { type ietf-poweff-types:something; } } } } } } } } ]]>

Security Considerations TODO Security

IANA Considerations This document has no IANA actions.

References Normative References YANG is a data modeling language used to model configuration data, state data, Remote Procedure Calls, and notifications for network management protocols. This document describes the syntax and semantics of version 1.1 of the YANG language. YANG version 1.1 is a maintenance release of the YANG language, addressing ambiguities and defects in the original specification. There are a small number of backward incompatibilities from YANG version 1. This document also specifies the YANG mappings to the Network Configuration Protocol (NETCONF). Deutsche Telekom This document proposes a standardized approach for representing YANG- modeled configuration and state data, for storage in Time Series Databases (TSDBs) that identify time series using a label-set. It outlines procedures for translating YANG data representations to fit within the label-centric structures of TSDBs and vice versa. This mapping ensures clear and efficient storage and querying of YANG- modeled data in TSDBs. In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. Informative References Huawei Huawei Telefonica I+D Telefonica I+D Swisscom Network elements use Model-driven Telemetry, and in particular YANG- Push, to continuously stream information, including both counters and state information. This document documents the metadata that ensure that the collected data can be interpreted correctly. This document specifies the Data Manifest, composed of two YANG data models (the Platform Manifest and the Data Collection Manifest.) The Data Manifest must be streamed and stored along with the data, up to the collection and analytics system in order to keep the collected data fully exploitable by the data scientists. Huawei Cisco Systems, Inc. Cisco Systems, Inc. This document proposes a YANG module that provides per-node capabilities for optimum operational data collection. This YANG module augments the YANG Modules for describing System Capabilities and YANG-Push Notification capabilities. This module defines augmented nodes to publish the metadata information specific to YANG node-identifier as per ietf-system- capabilities datatree. Complementary RPCs, based on the same node capabilities, simplify the data collection operations.

Acknowledgments Kristian Larsson has provided invaluable insights, experience and validation of the design. Many thanks to the entire POWEFF team for their committment, flexibility and hard work behind this. Hat off to Benoît Claise, who inspires by the extensive work produced in IETF over the years, and in this area in particular.