]> Knowledge Graphs for Enhanced Cross-Operator Incident Management and Network Design Orange
lionel.tailhardat@orange.com
EURECOM
raphael.troncy@eurecom.fr
Orange
yoan.chabot@orange.com
Operations and Management Network Management Operations knowledge graphs incident management anomaly detection Operational efficiency in incident management on telecom and computer networks requires correlating and interpreting large volumes of heterogeneous technical information. Knowledge graphs can provide a unified view of complex systems through shared vocabularies. YANG data models enable describing network configurations and automating their deployment. However, both approaches face challenges in vocabulary alignment and adoption, hindering knowledge capitalization and sharing on network designs and best practices. To address this, the concept of a meta-knowledge graph is introduced to leverage existing network infrastructure descriptions in YANG format and enable abstract reasoning on network behaviors. An experiment is proposed to assess the potential of the meta-knowledge graph in improving network quality and designs. About This Document The latest revision of this draft can be found at . Status information for this document may be found at . Discussion of this document takes place on the Network Management Operations Working Group mailing list (), which is archived at . Subscribe at . Source for this draft and an issue tracker can be found at .
Introduction Incident management on telecom and computer networks, whether it is related to infrastructure or cybersecurity issues, requires the ability to simultaneously and quickly correlate and interpret a large number of heterogeneous technical information sources. Knowledge Graphs (KGs), by structuring heterogeneous data through shared vocabularies, enable providing a unified view of complex technical systems, their ecosystem, and the activities and operations related to them (see and ). Using such formal knowledge representation allows for a simplified interpretation of networks and their behavior, both for NetOps & SecOps teams and artificial intelligence (AI) algorithms (e.g. anomaly detection, root cause analysis, diagnostic aid, situation summarization), and paves the way, in line with the Network Digital Twin vision , for the development of tools for detecting and analyzing complex network incident situations through explainable, actionable, and shareable models (see , , and ). However, despite potential benefits of using knowledge graphs, these are not mainstream yet in commercial network deployment systems and decision support systems (see for more on the decision support systems perspective). YANG is a widely used standard among operators for describing network configurations and automating their deployment. Using YANG representations in the form of a KG, as suggested in , would minimize the effort required to adapt network management tools towards the unified vision and applications evoked above. The lack of alignment between various YANG models on key concepts (e.g. for describing network topology) is, however, hindering this evolution . Furthermore, although addresses the capitalization of incident management knowledge through a YANG model, it can be observed that the overall scope of YANG models does not naturally cover the description of the networks' ecosystem (e.g. physical equipment location, operator organization, supervision systems) or the description of network operations from an IT service management (ITSM) perspective (e.g. business processes and design rules used by the company, scheduled modification operations, remediation actions performed during incident handling). As a consequence, the continuous improvement of network quality & designs requires additional data cross-referencing operations to properly contextualize incidents and learn from remediation actions taken (e.g. analyzing intervention technicians' verbatim, comparing actions performed on similar incidents but occurring on different networks). As a result of these additional efforts of contextualization, the capitalization of knowledge typically remains confined at the level of each network operator. This, in turn, hinders the sharing of information within the community of researchers and system designers regarding failure modes and best practices to adopt, considering the concept of overall improvement of IT systems and the Internet. Realizing an ITSM knowledge graph for network deployment, anomaly detection and risk management applications has been studied for several years in the Semantic Web community (i.e. knowledge representation and automated reasoning leveraging Web technologies such as , , , and ). Among other examples: the DevOpsInfra ontology allows for describing sets of computing resources and how they are allocated for hosting services; the NORIA-O ontology allows for describing a network infrastructure & ecosystem, its events, diagnosis and repair actions performed during incident management. Assuming the continuous integration into a knowledge graph of data from ticketing systems, network monitoring solutions, and network configuration management databases, we remark that the resulting knowledge graph () implicitely holds the necessary information to (automatically) learn incident contexts (i.e. the network topology, its set of states and set of events prior to the incident) and remediation procedures (i.e. the set of actions and network configuration changes carried-out to resolve the incident).
Learning an incident signature seen as a classification model that is trained on the relationship of the incident context (i.e. a subgraph centered around a Resource entity concerned by a given TroubleTicket) to the problem class defined at the TroubleTicket entity level. Arrows are for object properties (owl:ObjectProperty), double line edges are for object class relationships (rdf:type). │ │ ▲ │ │ │ │ │ resourceType │ │ ┌────────┐ │ │ ┌─────────────┐ │ │Resource│ │ │ │TroubleTicket│ │ └──────┬┬┘ │ │ └─────┬┬──────┘ │ ││ │ │ ││ │ ──◄──troubleTicketRelatedResource── │ │ │ │ │ │ │ │ │ problemCategory │──────┼────── │ │ │ │ │ │ │ ▼ │ │ │ │ │ │ │ │ ││ │ │ │ ┌────┴┴──────┐ │ logOriginatingManagedObject │ │skos:Concept│ │ │ │ └────────────┘ │ ▼ │ │ ──────┐ │ │ ┌─────────┴┴┐ dcterms:type │ │ │EventRecord│ │ │ │ └───────────┘ ▼ │ │ │ │ ┌────┴┴──────┐ │ │ │skos:Concept│ │ │ └────────────┘ │ └───────────────────────────────────────────────┘ ]]>
By going a step further, we notice that a generic understanding of incident context can be extracted and shared among operators from knowledge graphs. Indeed, a knowledge graph, being an instantiation of shared vocabularies (e.g. RDFS/OWL ontologies and controlled vocabularies in SKOS syntax), sharing incident signatures can be done without revealing infrastructure details (e.g. hostname, IP address), but rather the abstract representation of the network (i.e. the class of the knowlegde graph entities and relationships, such as "server" or "router", and or "IPoWDM link"). The remainder of this document is organized as follows. Firstly, the concept of a meta-knowledge graph is introduced to leverage existing network infrastructure descriptions in YANG format and enable abstract reasoning on network behaviors. Secondly, an experiment is proposed to assess the potential of the meta-knowledge graph in improving network quality and designs. In addition to the main parts of the proposal, the document also covers data integration and data federation architectures in the Security Considerations section. This section specifically addresses the handling of event data streams and the provision of a unified view for different stakeholders.
Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.
A meta-knowledge graph to align operator-specificities and share behavioral models of technical architectures TODO Introduce and develop the following topics. Topics:
  • Principles for implicit learning of incident characteristics and resolution methods through a graph and activity tracing.
    • Aligning operator-specificities with a multi-faceted knowledge graph.
    • Learning and sharing behavioral models.
    • Relation to the Network Anomaly Lifecycle experiment .
    • Relation to the Digital Map concept .
  • Meta-KG construction
    • Integrating Service and Network topology data from YANG data models, such as Network Topologies and Service Assurance }.
    • Relation to the NORIA-O ontology .
    • Trends towards some Yang to OWL / Yang to RDF data transformation tool.
  • Experiments towards the meta-KG proposal.
Aligning operator-specificities with a multi-faceted knowledge graph TODO Meta-KG construction TODO Relate the Meta-KG construction part to the following REQ-DM-SCALES related considerations. The following figures illustrate different scenarios for constructing a meta-KG through an Extract-Transform-Load (ETL) data integration pipeline. From the perspective of the Digital Map Requirements (), the , and particularly address the REQ-DM-SCALES requirement.
KG-only data integration architecture for event data streams (object/RES/router1)│ └─┌──────────────────────────────────────────┐ │(event/LOG_login_03)=>(object/RES/router1)│ └─┌──────────────────────────────────────────┐ │(event/LOG_login_03)=>(object/RES/router1)│ └──────────────────────────────────────────┘ ]]>
Resulting knowledge representation for the KG-only data integration architecture for event data streams │ │ │ ┌────────┐ ─rdf:type─┤Resource│ │ └────────┘ │ logOriginatingManagedObject │ ┌───────────┐ ──rdf:type─┤EventRecord│ └───────────┘ ]]>
Mixed KG/non-KG data integration architecture for event data streams (object/RES/router1)│ │ └──────────────────────────────────────────┘ │ ┌────────┐ ┌──────┐ │ │ Stream │ │┌────┐│ └────────────────────────────►│ loader ├─►││TSDB││ │ │ │└────┘│ └────────┘ └──────┘ ]]>
Resulting knowledge representation for the mixed KG/non-KG data integration architecture for event data streams │ │ │ ┌────────┐ ─rdf:type─┤Resource│ │ └────────┘ logOriginatingManagedObject │ ┌───────────┐ ┌──────────────────►──rdf:type─┤EventRecord│ │ │ │ \ └───────────┘ │ duration │ \ │ │ │ dcterms:type │ "P0Y0M0DT0H3M30S"^^xsd:duration │ \ │ │ │ │ │ │ "2024-02-07T16:22:42Z"^^xsd:dateTime rdf:type │ ┌─────┴──────┐ │ │skos:Concept│ │ KG knowledge representation └────────────┘ │ ============================================================== │ Time series database (TSDB) data representation │ │ Timestamp Origin Event │ 2024-02-07T16:22:42Z Login Attempt │ 2024-02-07T16:23:13Z Login Attempt │ 2024-02-07T16:26:12Z Login Attempt │ ▲ └──shared─identifier──────────────┘ ]]>
Unified access to data distributed across various technological platforms
Learning and sharing behavioral models TODO NetOps perspective TODO SecOps perspective
Relation to the Digital Map Similar to the concept of meta-knowledge graph (meta-KG) discussed here, the concept of Digital Map discussed in emphasizes the need to structure heterogeneous data describing networks in order to simplify network management operations through unified access to this data. The meta-knowledge graph extends the Digital Map concept by adding information about the lifecycle of infrastructures and services, as well as the context of their usage. These additional pieces of information are considered essential for learning shareable activity models of systems. To clarify this positioning, the following lists reflect the compliance of the meta-KG concept with the Digital Map Requirements defined in . A symbol to the right of each requirement name indicates the nature of compliance: + for compatibility, / for partial satisfaction, - for non-compliance with the requirement. A comment is provided as necessary.
Core Requirements
+ REQ-BASIC-MODEL-SUPPORT:
nothing to report (n.t.r.)
+ REQ-LAYERED-MODEL:
n.t.r.
/ REQ-PROG-OPEN-MODEL:
Partially satifying the requirement as the concept of meta-KG mainly relate to the knowledge representation topic rather than to the platform running the Digital Map service on top of the meta-knowledge graph.
/ REQ-STD-API-BASED:
Same remark as for REQ-PROG-OPEN-MODEL.
+ REQ-COMMON-APP:
n.t.r.
+ REQ-SEMANTIC:
n.t.r.
+ REQ-LAYER-NAVIGATE:
n.t.r.
+ REQ-EXTENSIBLE:
Knowledge graphs implicitly satisfy this requirement, notably with OWL and SKOS constructs if considering RDF knowledge graphs for the meta-KG (e.g. owl:sameAs to relate a meta-KG entity to some other entity of another knowledge graph, owl:equivalentClass to link concepts and properties used to interpret the meta-KG to concepts and properties from other data models, skos:inScheme to group new items of a controled-vocabulary as part of a skos:ConceptScheme).
+ REQ-PLUGG:
Same remark as for REQ-EXTENSIBLE.
+ REQ-GRAPH-TRAVERSAL:
This capability is naturally enabled as the meta-KG concept involves using a graph data structure.
Design Requirements
- REQ-TOPO-ONLY:
Requirement not satisfied as the meta-KG involves to have more than topological data to interpret and contextualize the network behavior.
- REQ-PROPERTIES:
Same remark as for REQ-TOPO-ONLY.
- REQ-RELATIONSHIPS:
Same remark as for REQ-TOPO-ONLY.
+ REQ-CONDITIONAL:
Native, notably considering the expressiveness of SPARQL if using the Semantic Web protocol stack to run the meta-KG concept.
+ REQ-TEMPO-HISTO:
n.t.r.
Architectural Requirements
+ REQ-DM-SCALES:
This capability applies as we can use data aggregation at the graph level ( and compared to and ), aggregation without loss of information ( and ), and load balancing (horizontal scaling) by partitioning the meta-KG (). Further, ease of integration is enabled thanks to existing standard graph data access protocols (e.g. SPARQL Federated Queries , as illustrated in ).
/ REQ-DM-DISCOVERY:
Same remark as for REQ-PROG-OPEN-MODEL.
Experiments TODO Experiments
Security Considerations As this document covers the meta-knowledge graph concepts, and use cases, there is no specific security considerations. However, as the concept of a meta-knowledge graph involves the construction of a multi-faceted graph (i.e. including network topologies, operational data, and service and client data), it poses the risk of simplifying access to network operational data and functions that fall outside the knowledge graph users' responsibility or that could facilitate the intervention of malicious individuals. To support the discussion on mitigating this risk, we suggest referring to , which illustrates the concept of partial access to the meta-knowledge graph based on rights associated with each user group (UG) at the data domain level. We also recommend referring to for an example of implemention of access rights in a content management system that relies on Semantic Web models and technologies. This implementation uses the AMO ontology, which includes a set of classes and properties for annotating resources that require access control, as well as a base of inference rules that model the access management strategy to carry out.
IANA Considerations This document has no IANA actions.
References Normative References Experiment: Network Anomaly Lifecycle Huawei Huawei Swisscom Swisscom INSA-Lyon Network Anomaly Detection is the act of detecting problems in the network. Accurately detect problems is very challenging for network operators in production networks. Good results require a lot of expertise and knowledge around both the implied network technologies and the specific service provided to consumers, apart from a proper monitoring infrastructure. In order to facilitate network anomaly detection, novel techniques are being introduced, including programmatical, rule-based and AI-based, with the promise of improving scalability and the hope to keep a high detection accuracy. To guarantee acceptable results, the process needs to be properly designed, adopting well-defined stages to accurately collect evidence of anomalies, validate their relevancy and improve the detection systems over time, iteratively. This document describes the lifecycle process to iteratively improve network anomaly detection accurately. Three key stages are proposed, along with a YANG model specifying the required metadata for the network anomaly detection covering the exchange of information between different stages of the lifecycle. Key words for use in RFCs to Indicate Requirement Levels In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. Digital Map: Concept, Requirements, and Use Cases Huawei Huawei Telefonica Swisscom This document defines the concept of Digital Map, explains its connection to the Digital Twin, and identifies a set of Digital Map requirements and use cases. The document intends to be used as a reference for the assessment effort of the various topology modules to meet Digital Map requirements. Discussion Venues This note is to be removed before publishing as an RFC. Source for this draft and an issue tracker can be found at https://github.com/OlgaHuawei/draft-havel-nmop-digital-map-concept/. A YANG Data Model for Network Topologies This document defines an abstract (generic, or base) YANG data model for network/service topologies and inventories. The data model serves as a base model that is augmented with technology-specific details in other, more specific topology and inventory data models. A YANG Data Model for Service Assurance This document specifies YANG modules for representing assurance graphs. These graphs represent the assurance of a given service by decomposing it into atomic assurance elements called subservices. The companion document, "Service Assurance for Intent-Based Networking Architecture" (RFC 9417), presents an architecture for implementing the assurance of such services. The YANG data models in this document conform to the Network Management Datastore Architecture (NMDA) defined in RFC 8342. Informative References OWL 2 Web Ontology Language Document Overview (Second Edition) W3C Resource Description Framework (RDF): Concepts and Abstract Syntax W3C RDF Schema 1.1 W3C SPARQL 1.1 Query Language W3C SPARQL 1.1 Federated Query W3C SKOS Simple Knowledge Organization System Reference W3C NORIA-O: An Ontology for Anomaly Detection and Incident Management in ICT Systems Leveraging Knowledge Graphs For Classifying Incident Situations in ICT Systems Graphameleon: Relational Learning and Anomaly Detection on Web Navigation Traces Captured as Knowledge Graphs NORIA UI: Efficient Incident Management on Large-Scale ICT Systems Represented as Knowledge Graphs FLAGS: A Methodology for Adaptive Anomaly Detection and Root Cause Analysis on Sensor Data Streams by Fusing Expert Knowledge with Machine Learning Towards Adaptive Anomaly Detection and Root Cause Analysis by Automated Extraction of Knowledge from Risk Analyses A High-Level Ontology Network for ICT Infrastructures Ontology-Based Access Rights Management Knowledge Graphs for YANG-based Network Management Telefonica Telefonica The success of the YANG language and YANG-based protocols for managing the network has unlocked new opportunities in network analytics. However, the wide heterogeneity of YANG models hinders the consumption and analysis of network data. Besides, data encoding formats and transport protocols will differ depending on the network management protocol supported by the network device. These challenges call for new data management paradigms that facilitate the discovery, understanding, integration and access to silos of heterogenous YANG data, abstracting from the complexities of the network devices. This document introduces the knowledge graph paradigm as a solution to this data management problem, with focus on YANG-based network management. The document provides background on related topics such as ontologies and graph standards, and shares guidelines for implementing knowledge graphs from YANG data. Network Digital Twin: Concepts and Reference Architecture China Mobile China Mobile China Mobile Huawei Orange Orange Digital Twin technology has been seen as a rapid adoption technology in Industry 4.0. The application of Digital Twin technology in the networking field is meant to develop various rich network applications, realize efficient and cost-effective data-driven network management, and accelerate network innovation. This document presents an overview of the concepts of Digital Twin Network, provides the basic definitions and a reference architecture, lists a set of application scenarios, and discusses such technology's benefits and key challenges. RFC 3535, 20 Years Later: An Update of Operators Requirements on Network Management Protocols and Modelling Orange Telefonica Telefonica Swisscom Equinix The IAB has organized an important workshop to establish a dialog between network operators and protocol developers, and to guide the IETF focus on work regarding network management. The outcome of that workshop was documented in the "IAB Network Management Workshop" (RFC 3535) which was instrumental for developing NETCONF and YANG, in particular.
Acknowledgments We would like to thank Benoit Claise for spontaneously seeking to include the work of the NORIA research project in the vision of the NMOP working group through direct contact. We would also like to thank Fano Rampary for his initial analysis of the possibilities of defining a model conversion algebra for going from Yang data models to OWL ontologies.