MRC, Huawei Technologies

Munich Research Center Huawei Technologies Riesstr. 25 Muenchen 80992 Germany xun.xiao@huawei.com

Huawei Technologies

Q5, Huawei Campus No.156 Beiqing Road Hai-Dian District, Beijing 100095 P.R. China leo.liubing@huawei.com

MRC, Huawei Technologies

Munich Research Center Huawei Technologies Riesstr. 25 Muenchen 80992 Germany artur.hecker@huawei.com

Beijing University of Posts and Telecommunications

No. 10 Xitucheng Road Haidian District Beijing China shengjiang@bupt.edu.cn

Autonomic network infrastructure (ANI) is a generic platform for Autonomic Functions (AFs). In some scenarios, AFs may not only require communication capability supported from the infrastructure side, but also the capability the infrastructure can hold and re-distribute information on-demand. This document defines a set of solutions for information distribution in the ANI. Information distribution is categorized into two modes: 1) instantaneous distribution and 2) publishing for retrieval. In the former case, the information is sent, propagated and disposed of after reception. In the latter case, information needs to be stored in the network; additionally, conflict resolution is also needed when information stored in the network is updated with proposals from two different AFs. This document defines a series of extensions to GeneRic Autonomic Signaling Protocol (GRASP) in order to support such information distribution.the autonomic nodes and suggests an implementation based on GRASP. This document also describes typical use cases of them.

In an autonomic network, Autonomic Functions (AFs) running on autonomic nodes constantly exchange information, e.g. AF control/management signaling or AF data exchange. This document discusses the information distribution capability of such exchanges among AFs. In general, depending on the number of participants, the information can be distributed into the below categries: Point-to-point (P2P) Communication: information is exchanged between two AFs. One-to-Many Communication: information exchanges involve one source AF and multiple receiving AFs. Mechanisms of information distribution can be mainly categorized into two basic modes: An instantaneous mode (push): a source sends the actual content (e.g. control/management signaling, synchronization data and so on) to all interested receiver(s) immediately. Generally, some preconfigurations are required, where nodes interested in this information must be already known to all nodes because any source AF must be able to decide, to which AFs the data is to be sent. An asynchronous mode (delayed pull): here, a source AF publishes the content in some forms in the network, which may later be looked for, found and retrieved by some endpoints in the AN. Here, depending on the size of the content, either the whole content or only its metadata might be published into the AN. In the latter case the metadata (e.g. a content descriptor, e.g. a key, and a location in the ANI) may be used for the actual retrieval. Importantly, the source, i.e., here as a publisher, needs to be able to determine the location, where the information (or its metadata) can be stored. Note that in both cases, the total size of transferred information can be larger than the payload size of a single message of a used transport protocol (e.g., Synchronization and Flood messages in GRASP). In this situation, this document also considers a case of bulk data transfer. To avoid repetitive implementations by each AF developer, this document opts for a common support for information distribution implemented as a basic ANI capability. Therefore, it will be available to all AFs. In fact, GRASP already provides part of the capabilities. Regardless, an AF may still define and implement its own information distribution capability. Such a capability may then be advertised using the common information distribution capability defined in this document. Overall, ANI nodes and AFs may decide, which of the information distribution mechanisms they want to use for which type of information, according to their own preferences. This document first analyzes requirements for information distribution in autonomic networks () and then discuss the relevant node behaviors (). After that, the required GRASP extensions are formally introduced ().

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 and .

In this section, we present some important use cases where information distribution is required and ACP's support is commanly needed.

In addition to Internet, carrier networks (i.e. wireless mobile networks) is another world-wide networking system. The current architecture of 5G mobile networks from 3GPP has been defined to follow a service-based architecture (SBA) where any network function (NF) can dynamically interact with any other NF(s) when needed to compose a network service. Note that one NF can simultaneously associate with multiple other NFs, instead of being physically wired as in the previous generations of mobile networks. NFs communicate with each other over service-based interface (SBI), which is also standardized by 3GPP [3GPP.23.501]. To realize an SBA network system, detailed requirements are further defined to specify how NFs should interact with each other with information exchange over the SBI in corresponding 3GPP technical specifications. We now list three services that are closely related to information distribution here. NF Pub/Sub: Any NF should be able to expose its service status to the network and any NF should be able to subscribe the service status of an NF and get notified if the status is available. A concrete example is that a session management function (SMF) can subscribe to the REGISTER notification from an access management function (AMF) if there is a new user equipment trying to access the mobile network [3GPP.23.502]. Network Exposure Function (NEF): A particular network function that is required to manage the event exposure and distributions. Specifically, SBA requires such a functionality to register network events from the other NFs (e.g. AMF, SMF and so on), classify the events and properly handle event distributions accordingly in terms of different criteria (e.g. priorities) [3GPP.23.502]. Network Repository Function (NRF): A particular network function where all service status information is stored for the whole network. An SBA network system requires all NFs to be stateless so as to improve the resilience as well as agility of providing network services. Therefore, the information of the available NFs and the service status generated by those NFs will be globally stored in NRF as a repository of the system. This clearly implies storage capability that keeps the information in the network and provides those information when needed. A concrete example is that whenever a new NF comes up, it first of all registers itself at NRF with its profile. When a network service requires a certain NF, it first inquires NRF to retrieve the availability information and decides whether or not there is an available NF or a new NF must be instantiated [3GPP.23.502]. (Note: 3GPP adopted HTTP2.0/JSON as one option to implement the transmission protocol between defined NFs.) Notice that how the control plane such as connectivity and trust shall be bootstrapped and maintained among NFs are not specified. In fact, 3GPP only considers the necessary requirements and features of a 3GPP network shall present. Hence, ACP and GRASP could be utilized as a specific solution and even further promoted to 3GPP if a majority consensus is reached among 3GPP participants.

In-network computing recently gets a lot of attentions . INC improves the utilization of the computing resources in the network; INC also brings the processed results closer to the users, which may potentially improves the QoS of network services. Unlike existing network systems, INC deploys computing tasks directly in the network rather than pushing the tasks to endpoints outside the network. Therefore, a network device is not just a transport device, but a mixture of forwarding, routing and computing. The requires an INC-supported network device having storage by default. Furthermore, computing agents deployed on network nodes will have to communicate with each other by exchanging information. There are several typical applications, where information distribution capability is required, which are summarized below. Data Backup: There can be multiple computing agents that are created to serve the same purpose(s). In reality, the multiple agents can run for service resilience, load balancing and so on. This forms a service set. The instances in the service set can be deployed at different locations in the network while they need to keep synchronizing their local states for global consistency. In this case, the computing agents will have to constantly send and receive information across the network. Data Aggregation: Multiple computing agents may process different computing tasks but the derived results have to be aggregated or combined. Then a collective result can be derived. In this case, different computing agents collaborate with each other, where information data are exchanged during the processing. A popular example is distributed AI or federated learning applications, where data are stored at different places and model training with the local data is also done in a distributed way. After that, trained models by distributed agents will have to be aggregated. Information distribution will be utlized heavily, combining with local storage. Clearly, AFs running on network nodes in ANI are the abstraction of the INC use case. AFs can be deployed for both scenarios above.

The connected Autonomous Driving (AD) vehicles market is driving the evolution of the Internet of Vehicles (IoV) (or Vehicular IoT) and is growing at a five-year compound annual growth rate of 45%, which is 10 times as fast as the overall car market. V2X communication is an inevitable enabling technology that connects vehicles to networks, where value-added services can be provided and enhance the functionalities of a vehicle. In this section, we introduce some use cases that will be closely relevant to information distribution in an ANI. Real-time and High Definition Maps (HDM): In the era of autonomous driving, a digital map not only means for navigation, but real-time and detailed information is required when driving a vehicle. Real-time situational awareness is essential for autonomous vehicles especially at critical road segments in cases of changing road conditions (e.g. new traffic cone detected by another vehicle some time ago). In addition, the relevant high definition local maps have to be available with support from infrastructure side. In this regards, a digital map should not be considered static information stored on the vehicle, which is spontaneously updated in a periodical manner. Instead, it shall be considered a dynamic distribution based on information aggregated from the local area and such a distribution shall consider latency requirement. Clearly, the infrastructure side shall be able to hold the information in the network sufficiently close to the relevant area. In-car Infotaiment: This is another popular use case where in-car data demands will increase significantly in the near future. Today, users their mobile phone to access Internet for retrieving data for work or entertainment purposes. There is already a concensus among OTTs, carriers and car manufacturers that vehicle will become the center of information for passengers onboard. For entertainment, typical scenarios can be stereo HD video streaming and online gaming; for business purposes, examples can be mobile conference. This therefore requires the infrastructure side to be able to schedule and deliver requested information/data to the users with quality-of-service (QoS) considerations. Software Update: Software components of connected cars will be remotely maintained in future. Therefore, software update has to be supported by the infrastructure side. Although this can be done by centralized solution where all vehicles access to a central clouds, in terms of load balancing and efficiency, prepared update components can be stored in the network and delivered to endpoints in a distributed manner. Note that there could be different modes to support the potential use cases above. The first mode is that vehicles are not part of the ACP while simply accessing the edge nodes that are part of the ACP using information distribution to provide infomration required by the vechicles. The second mode is more radical where the vehciles also belong to the part of ACP while a dynamic ACP topology consisting of wireless link connectivity could exist. The latter scenario may further require all entities (both at the network side and the end point side) must be able to establlish a trust layer relying on the security mechanism with BRSKI.

According to the introduced use cases, the question of information distribution in an autonomic network can be discussed through particular use cases or more generally. Depending on the situation it can be quite simple or might require more complex provisions. Indeed, in the most general case, the information can be sent: at once (in one or multiple packets, in one flow), straightaway (send-and-forget), to all nodes. For the first scenario, presuming 1), 2) and 3) hold, information distribution in smaller or scarce topologies can be implemented using broadcast, i.e. unconstrained flooding. For reasons well-understood, this approach has its limits in larger and denser networks. In this case, a graph can be constructed such that it contains every node exactly once (e.g. a spanning tree), still allowing to distribute any information to all nodes straightaway. Multicast tree construction protocols could be used in this case. There are reasonable use cases for such scenarios, as presented in . Secondly, a more complex scenario arises, if only 1) and 2) hold, but the information only concerns a subset of nodes. Then, some kinds of selection become required, to which nodes the given information should be distributed. Here, a further distinction is necessary; notably, if the selection of the target nodes is with respect to the nature or position of the node, or whether it is with respect to the information content. If the first, some knowledge about the node types, its topological position, etc (e.g. the routing information within ANI) can be used to distinguish nodes accordingly. For instance, edge nodes and forwarding nodes can be distinguished in this way. If the distribution scope is primarily to be defined by the information elements, then a registration / join / subscription or label distribution mechanism is unavoidable. This would be the case, for instance, if the AFs can be dynamically deployed on nodes, and the information is majorily destined to the AFs. Then, depending on the current AF deployment, the distribution scope must be adjusted as well. Thirdly, if only 1) holds, but the information content might be required again and again, or might not yet be fully available, then more complex mechanisms might be required to store the information within the network for later, for further redistribution, and for notification of interested nodes. Examples for this include distribution of reconfiguration information for different AF instances, which might not require an immediate action, but only an eventual update of the parameters. Also, in some situations, there could be a significant delay between the occurrence of a new event and the full content availability (e.g. if the processing requires a lot of time). Finally, none of the three might hold. Then, along with the subscription and notification, the actual content might be different from its metadata, i.e. some descriptions of the content and, possibly, its location. The fetching can then be implemented in different, appropriate ways, if necessary as a complex transport session. In essence, as flooding is usually not an option, and the interest of nodes for particular information elements can change over time, ANI should support autonomics also for the information distribution. This calls for autonomic mechanisms in the ANI, allowing participating nodes to 1) advertise/publish, 2) look for/subscribe to 3) store, 4) fetch/retrieve and 5) instantaneously push data information. In the following cases, situations depicting complicated ways of information distribution are discussed. Long Communication Intervals. The actual sending of the information is not necessarily instantaneous with some events. Sophisticated AFs may involve into longer jobs/tasks (e.g. database lookup, validations, etc.) when processing requests, and might not be able to reply immediately. Instead of actively waiting for the reply, a better way for an interested AF might be to get notified, when the reply is finally available. Common Interest Distribution. AFs may share information that is a common interest. For example, the network intent needs to be distributed to network nodes enrolled, which is usually one-to-many scenario. Intent distribution can also be performed by an instant flooding (e.g. via GRASP) to every network node. However, because of network changes, not every node can be just ready at the moment when the network intent is broadcast. Also, a flooding often does not cover all network nodes as there is usually a limitation on the hop number. In fact, nodes may join in the network sequentially. In this situation, an asynchronous communication model could be a better choice where every (newly joining) node can subscribe the intent information and will get notified if it is ready (or updated). Distributed Coordination. With computing and storage resources on autonomic nodes, alive AFs not only consume but also generate data information. An example is AFs coordinating with each other as distributed schedulers, responding to service requests and distributing tasks. It is critical for those AFs to make correct decisions based on local information, which might be asymmetric as well. AFs may also need synthetic/aggregated data information (e.g. statistic info, like average values of several AFs, etc.) to make decisions. In these situations, AFs will need an efficient way to form a global view of the network (e.g. about resource consumption, bandwidth and statistics). Obviously, purely relying on instant communication model is inefficient, while a scalable, common, yet distributed data layer, on which AFs can store and share information in an asynchronous way, should be a better choice. Collision Update. Information data not only can be propagated and stored on network nodes in the network, they have to be conflict-free when information is updated especially when there is no central authority available. For example, when two AFs try to propose different updates for the same piece of information that already exist in the network, a decision has to be made for how the existing information shall be updated. Obviously, if this duty has to be handled by individual AFs, the implematation of an AF is too complicated. Therefore, information distribution should consider conflict resultion and provides a set of general solutions for AFs in order to keep information conflict free. Therefore, for ANI, in order to support various communication scenarios, an information distribution module is required, and both instantaneous and asynchronous communication models should be supported. Some real-world use cases are introduced in .

In this section, how a node should behave in order to support the two identified modes of information distribution is discussed. An ANI is a distributed system, so the information distribution module must be implemented in a distributed way as well.

In this case, an information sender directly specifies the information receiver(s). The instant information distribution sub-module will be the main element.

IID sub-module performs instant information transmission for ASAs running in an ANI. In specific, IID sub-module will have to retrieve the address of the information receiver specified by an ASA, then deliver the information to the receiver. Such a delivery can be done either in a connectionless or a connection-oriented way. Current GRASP provides the capability to support instant P2P synchronization for ASAs. A P2P synchronization is a use case of P2P information transmission. However, as mentioned in Section 3, there are some scenarios where one node needs to transmit some information to another node(s). This is different to synchronization because after transmitting the information, the local status of the information does not have to be the same as the information sent to the receiver. This is not directly support by existing GRASP.

IID sub-module finishes instant flooding for ASAs in an ANI. Instant flooding is for all ASAs in an ANI. An information sender has to specify a special destination address of the information and broadcast to all interfaces to its neighbors. When another IID sub- module receives such a broadcast, after checking its TTL, it further broadcast the message to the neighbors. In order to avoid flooding storms in an ANI, usually a TTL number is specified, so that after a pre-defined limit, the flooding message will not be further broadcast again. In order to avoid unnecessary flooding, a selective flooding can be done where an information sender wants to send information to multiple receivers at once. When doing this, sending information needs to contain criteria to judge on which interfaces the distributed information should and should not be sent. Specifically, the criteria contain: Matching Condition: a set of matching rules such as addresses of recipients, node features and so on. Action: what the node needs to do when the Matching Condition is fulfilled. For example, the action could be forwarding or discarding the distributed message. Sent information must be included in the message distributed from the sender. The receiving node reacts by first checking the carried Matching Condition in the message to decide who should consume the message, which could be either the node itself, some neighbors or both. If the node itself is a recipient, Action field is followed; if a neighbor is a recipient, the message is sent accordingly. An exemplary extension to support selective flooding on GRASP is described in Section 5.

In asynchronous information distribution, sender(s) and receiver(s) are not immediately specified while they may appear in an asynchronous way. Firstly, AID sub-module enables that the information can be stored in the network; secondly, AID sub-module provides an information publication and subscription (Pub/Sub) mechanism for ASAs. As sketched in the previous section, in general each node requires two modules: 1) Information Storage (IS) module and 2) Event Queue (EQ) module in the information distribution module. Details of the two modules are described in the following sections.

IS module handles how to save and retrieve information for ASAs across the network. The IS module uses a syntax to index information, generating the hash index value (e.g. a hash value) of the information and mapping the hash index to a certain node in ANI. Note that, this mechanism can use existing solutions. Specifically, storing information in an ANIMA network will be realized in the following steps. ASA-to-IS Negotiation. An ASA calls the API provided by information distribution module (directly supported by IS sub- module) to request to store the information somewhere in the network. The IS module performs various checks of the request (e.g. permitted information size). Storing Peer Mapping. The information block will be handled by the IS module in order to calculate/map to a peer node in the network. Since ANIMA network is a peer-to-peer network, a typical way is to use distributed hash table (DHT) to map information to a unique index identifier. For example, if the size of the information is reasonable, the information block itself can be hashed, otherwise, some meta-data of the information block can be used to generate the mapping. Storing Peer Negotiation Request. Negotiation request of storing the information will be sent from the IS module to the IS module on the destination node. The negotiation request contains parameters about the information block from the source IS module. According to the parameters as well as the local available resource, the requested storing peer will send feedback the source IS module. Storing Peer Negotiation Response. Negotiation response from the storing peer is sent back to the source IS module. If the source IS module gets confirmation that the information can be stored, source IS module will prepare to transfer the information block; otherwise, a new storing peer must be discovered (i.e. going to step 7). Information Block Transfer. Before sending the information block to the storing peer that already accepts the request, the IS module of the source node will check if the information block can be afforded by one GRASP message. If so, the information block will be directly sent by calling a GRASP API (). Otherwise, a bulk data transmission is needed. It can utilize one of existing protocols that is independent of the GRASP stack. A session connectivity can be established to the storing peer, and over the connection the bulky data can be transmitted part by part. In this case, the IS module should support basic TCP-based session protocols such as HTTP(s) or native TCP. Information Writing. Once the information block (or a smaller block) is received, the IS module of the storing peer will store the data block in the local storage is accessible. (Optional) New Storing Peer Discovery. If the previously selected storing peer is not available to store the information block, the source IS module will have to identify a new destination node to start a new negotiation. In this case, the discovery can be done by using discovery GRASP API to identify a new candidate, or more complex mechanisms can be introduced. Similarly, Getting information from an ANI will be realized in the following steps. ASA-to-IS Request. An ASA accesses the IS module via the APIs exposed by the information distribution module. The key/index of the interested information will be sent to the IS module. An assumption here is that the key/index should be known to an ASA before an ASA can ask for the information. This relates to the publishing/subscribing of the information, which are handled by other modules (e.g. Event Queue with Pub/Sub supported by GRASP). Storing Peer Mapping. IS module maps the key/index of the requested information to a peer that stores the information, and prepares the information request. The mapping here follows the same mechanism when the information is stored. Retrieval Negotiation Request. The source IS module sends a request to the storing peer and asks if such an information object is available. Retrieval Negotiation Response. The storing peer checks the key/index of the information in the request, and replies to the source IS module. If the information is found and the information block can be afforded within one GRASP message, the information will be sent together with the response to the source IS module. (Optional) New Destination Request. If the information is not found after the source IS module gets the response from the originally identified storing peer, the source IS module will have to discover the location of the requested information. IS module can reuse distributed databases and key value stores like NoSQL, Cassandra, DHT technologies. storage and retrieval of information are all event-driven responsible by the EQ module.

The Event Queue (EQ) module is to help ASAs to publish information to the network and subscribe to interested information in asynchronous scenarios. In an ANI, information generated on network nodes is an event labeled with an event ID, which is semantically related to the topic of the information. Key features of EQ module are summarized as follows. Event Group: An EQ module provides isolated queues for different event groups. If two groups of AFs could have completely different purposes, the EQ module allows to create multiple queues where only AFs interested in the same topic will be aware of the corresponding event queue. Event Prioritization: Events can have different priorities in ANI. This corresponds to how much important or urgent the event implies. Some of them are more urgent than regular ones. Prioritization allows AFs to differentiate events (i.e. information) they publish or subscribe to. Event Matching: an information consumer has to be identified from the queue in order to deliver the information from the provider. Event matching keeps looking for the subscriptions in the queue to see if there is an exact published event there. Whenever a match is found, it will notify the upper layer to inform the corresponding ASAs who are the information provider and subscriber(s) respectively. The EQ module on every network node operates as follows. Event ID Generation: If information of an ASA is ready, an event ID is generated according to the content of the information. This is also related to how the information is stored/saved by the IS module introduced before. Meanwhile, the type of the event is also specified where it can be of control purpose or user plane data. Priority Specification: According to the type of the event, the ASA may specify its priority to say how this event is to be processed. By considering both aspects, the priority of the event will be determined. Event Enqueue: Given the event ID, event group and its priority, a queue is identified locally if all criteria can be satisfied. If there is such a queue, the event will be simply added into the queue, otherwise a new queue will be created to accommodate such an event. Event Propagation: The published event will be propagated to the other network nodes in the ANIMA domain. A propagation algorithm can be employed to optimize the propagation efficiency of the updated event queue states. Event Match and Notification: While propagating updated event states, EQ module in parallel keeps matching published events and its interested consumers. Once a match is found, the provider and subscriber(s) will be notified for final information retrieval. The category of event priority is defined as the following. In general, there are two event types: Network Control Event: This type of events are defined by the ANI for operational purposes on network control. A pre-defined priority levels for required system messages is suggested. For highest level to lowest level, the priority value ranges from NC_PRIOR_HIGH to NC_PRIOR_LOW as integer values. The NC_PRIOR_* values will be defined later according to the total number system events required by the ANI. Custom ASA Event: This type of events are defined by the ASAs of users. This specifies the priority of the message within a group of ASAs, therefore it is only effective among ASAs that join the same message group. Within the message group, a group header/leader has to define a list of priority levels ranging from CUST_PRIOR_HIGH to CUST_PRIOR_LOW. Such a definition completely depends on the individual purposes of the message group. When a system message is delivered, its event type and event priority value have to be both specified. Event contains the address where the information is stored, after a subscriber is notified, it directly retrieves the information from the given location.

In both cases discussed previously, they are limited to distributing GRASP Objective Options contained in messages that cannot exceed the GRASP maximum message size of 2048 bytes. This places a limit on the size of data that can be transferred directly in a GRASP message such as a Synchronization or Flood operation for instantaneous information distribution. There are scenarios in autonomic networks where this restriction is a problem. One example is the distribution of network policy in lengthy formats such as YANG or JSON. Another case might be an Autonomic Service Agent (ASA) uploading a log file to the Network Operations Center (NOC). A third case might be a supervisory system downloading a software upgrade to an autonomic node. A related case might be installing the code of a new or updated ASA to a target node. Naturally, an existing solution such as a secure file transfer protocol or secure HTTP might be used for this. Other management protocols such as syslog [RFC5424] or NETCONF [RFC6241] might also be used for related purposes, or might be mapped directly over GRASP. The present document, however, applies to any scenario where it is preferable to re-use the autonomic networking infrastructure itself to transfer a significant amount of data, rather than install and configure an additional mechanism. The node behavior is to use the GRASP Negotiation process to transfer and acknowledge multiple blocks of data in successive negotiation steps, thereby overcoming the GRASP message size limitation. The emphasis is placed on simplicity rather than efficiency, high throughput, or advanced functionality. For example, if a transfer gets out of step or data packets are lost, the strategy is to abort the transfer and try again. In an enterprise network with low bit error rates, and with GRASP running over TCP, this is not considered a serious issue. Clearly, a more sophisticated approach could be designed but if the application requires that, existing protocols could be used, as indicated in the preceding paragraph. As for any GRASP operation, the two participants are considered to be Autonomic Service Agents (ASAs) and they communicate using a specific GRASP Objective Option, containing its own name, some flag bits, a loop count, and a value. In bulk transfer, we can model the ASA acting as the source of the transfer as a download server, and the destination as a download client. No changes or extensions are required to GRASP itself, but compared to a normal GRASP negotiation, the communication pattern is slightly asymmetric: The client first discovers the server by the GRASP discovery mechanism (M_DISCOVERY and M_RESPONSE messages). The client then sends a GRASP negotiation request (M_REQ_NEG message). The value of the objective expresses the requested item (e.g., a file name - see the next section for a detailed example). The server replies with a negotiation step (M_NEGOTIATE message). The value of the objective is the first section of the requested item (e.g., the first block of the requested file as a raw byte string). The client replies with a negotiation step (M_NEGOTIATE message). The value of the objective is a simple acknowledgement (e.g., the text string ’ACK’). The last two steps repeat until the transfer is complete. The server signals the end by transferring an empty byte string as the final value. In this case the client responds with a normal end to the negotiation (M_END message with an O_ACCEPT option). Errors of any kind are handled with the normal GRASP mechanisms, in particular by an M_END message with an O_DECLINE option in either direction. In this case the GRASP session terminates. It is then the client’s choice whether to retry the operation from the start, as a new GRASP session, or to abandon the transfer. The block size must be chosen such that each step does not exceed the GRASP message size limit of 2048 bits.

In summary, the general requirements for the information distribution module on each autonomic node are realized by two sub-modules handling instant communications and asynchronous communications, respectively. For instantaneous mode, node requirements are simple, calling for support for additional signaling. With minimum efforts, reusing the existing GRASP is possible. For asynchronous mode, information distribution module uses new primitives on the wire, and implements an event queue and an information storage mechanism. In both cases, a scenario of bulk information transfer is considered where the retrieved information cannot be fitted in one GRASP message. Based on GRASP Negotiation operation, multiple transmissions can be repeatedly done in order to transfer bulk informtion piece by piece.

This could be a new message in GRASP. In fragmentary CDDL, an Un- solicited Synchronization message follows the pattern: unsolicited_synch-message = [M_UNSOLIDSYNCH, session-id, objective] A node MAY actively send a unicast Un-solicited Synchronization message with the Synchronization data, to another node. This MAY be sent to port GRASP_LISTEN_PORT at the destination address, which might be obtained by GRASP Discovery or other possible ways. The synchronization data are in the form of GRASP Option(s) for specific synchronization objective(s).

Since normal flooding is already supported by GRASP, this section only defines the Selective-Flooding option. Since GRASP is based on CBOR (Concise Binary Object Representation) , the format of the Selective-Flooding option is described in the Concise Data Definition Language (CDDL) as follows: Selective-Flooding-option = [O_SELECTIVE_FLOOD, +O_MATCH-CONDITION, match-object, action] O_MATCH-CONDITION = [O_MATCH-CONDITION, Obj1, match-rule, Obj2] Obj1 = text match-rule = GREATER / LESS / WITHIN / CONTAIN Obj2 = text match-object = NEIGHBOR / SELF action = FORWARD / DROP The option field encapsulates a match-condition option which represents the conditions regarding to continue or discontinue flood the current message. For the match-condition option, the Obj1 and Obj2 are to objects that need to be compared. For example, the Obj1 could be the role of the device and Obj2 could be "RSG". The match rules between the two objects could be greater, less than, within, or contain. The match-object represents of which Obj1 belongs to, it could be the device itself or the neighbor(s) intended to be flooded. The action means, when the match rule applies, the current device just continues flood or discontinues.

In fragmentary CDDL, a Subscription Objective Option follows the pattern: objective = [Subscription, 2, 2, subobj] objective-name = Subscription objective-flags = 2 loop-count = 2 subobj = text This option MAY be included in GRASP M_Synchronization, when included, it means this message is for a subscription to a specific object.

In fragmentary CDDL, a Unsubscription Objective Option follows the pattern: objective = [Unsubscription, 2, 2, unsubobj] objective-name = Unsubscription objective-flags = 2 loop-count = 2 unsubobj = text This option MAY be included in GRASP M_Synchronization, when included, it means this message is for a un-subscription to a specific object.

In fragmentary CDDL, a Publishing Objective Option follows the pattern: objective = [Publishing, 2, 2, pubobj] objective-name = Publishing objective-flags = 2 loop-count = 2 pubobj = text This option MAY be included in GRASP M_Synchronization, when included, it means this message is for a publish of a specific object data.

The distribution source authentication could be done at multiple layers: Outer layer authentication: the GRASP communication is within ACP (). This is the default GRASP behavior. Inner layer authentication: the GRASP communication might not be within a protected channel, then there should be embedded protection in distribution information itself. Public key infrastructure might be involved in this case.

This document defines a new GRASP message named "M_UNSOLIDSYNCH" and a new option named "O_SELECTIVE_FLOOD" which need to be added to the "GRASP Messages and Options" registry defined by . And this document defines three new GRASP Objectives, "Subscription", "Unsubscribe" and "Publishing" which need to be added to the "GRASP Objective Names" .

Valuable comments were received from Zoran Despotovic, Brian Carpenter, Michael Richardson, Roland Bless, Mohamed Boucadair, Diego Lopez, Toerless Eckert and other participants in the ANIMA working group. This document was produced using the xml2rfc tool .

School of Computer Science University of Auckland PB 92019 Auckland 1142 New Zealand brian.e.carpenter@gmail.com

In-network computing accelerates applications natively running on the host by executing them within network devices. While in-network computing offers significant performance improvements, its limitations and design trade-offs have not been explored. To usefully and efficiently run applications within the network, we first need to understand the implications of their design. In this work we introduce LaKe, a Layered Key-Value Store design, running as an in-network application. LaKe is a scalable design, enabling the exploration of design decisions and their effect on throughput, latency and power efficiency. LaKe achieves full line rate throughput, while maintaining a latency of 1.1ms and better power efficiency than existing hardware based memcached designs.

Better explanation of the required context of the Connected-Car case: Not applicable unless the ACP will be extended to the car, which may not be desirable with the current ACP design, but maybe refocussing on an "autonomous fleet" use-case (e.g.: all cars operated by some taxi like service). Consider use-case/example of firmware update. By abstracting the location of the firmware from the name of the firmware, while providing a way to notify about it, this significantly supports distribution of firmware updates. References to SUIT would appropriate. Issues discussed in https://mailarchive.ietf.org/arch/msg/anima/_0fYQPBcLPt8xzQee7P4dILsn3A Rethink/refine terminology, e.g.: "module" seems to be too prescriptive. Refine proposed extensions. Provide more protocol behavior description instead of only implementation / software module architecture description. Reduce the latter or provide better justification for their presence due to explained interoperability requirements. Consider moving examples from appendices into core-text . Ideally craft a single use-case showing/applying all extensions (most simple use case that uses them all).

Actions triggered to the information distribution module will eventually invoke underlying GRASP APIs. Moreover, EQ and IS modules are usually correlated. When an AF(ASA) publishes information, not only such an event is translated and sent to EQ module, but also the information is indexed and stored simultaneously. Similarly, when an AF(ASA) subscribes information, not only subscribing event is triggered and sent to EQ module, but also the information will be retrieved by IS module at the same time. Storing and publishing information: This action involves both IS and EQ modules where a node that can store the information will be discovered first and related event will e published to the network. For this, GRASP APIs discover(), synchronize() and flood() are combined to compose such a procedure. In specific, discover() call will specific its objective being to "store_data" and the return parameters could be either an ASA_locator who will accept to store the data, or an error code indicating that no one could afford such data; after that, synchronize() call will send the data to the specified ASA_locator and the data will be stored at that node, with return of processing results like store_data_ack; meanwhile, such a successful event (i.e. data is stored successfully) will be flooded via a flood() call to interesting parties (such a multicast group existed). Subscribing and getting information: This action involves both IS and EQ modules as well where a node that is interested in a topic will subscribe the topic by triggering EQ module and if the topic is ready IS module will retrieve the content of the topic (i.e. the data). GRASP APIs such as register_objective(), flood(), synchronize() are combined to compose the procedure. In specific, any subscription action received by EQ module will be translated to register_objective() call where the interested topic will be the parameter inside of the call; the registration will be (selectively) flooded to the network by an API call of flood() with the option we extended in this draft; once a matched topic is found (because of the previous procedure), the node finding such a match will call API synchronize() to send the stored data to the subscriber.