The Requirements of a Unified Transport Protocol for In-Network Computing in Support of RPC-based Applications

Motivation In a broader sense, COmputing-In-Network (COIN) covers many distinct types of applications which rely on networks to do more than packet forwarding (e.g., active networking, edge computing, and service function chaining). However, the emerging term In-Network Computing (INC) in particular refers to a narrower scope which applies on-path programmable networking devices (e.g., switches and routers between clients and servers) as an accelerator or function offloader to boost throughput, reduce server load, or improve latency, typically in a well-controlled data center network environment. INC is a natural outgrowth of the programmable data plane progress and the trend of network programmability at large. In recent year, it has been shown to support many promising applications (e.g., caching, aggregation, and agreement). An unfortunate consequence of INC is that it breaks the end- to-end principle and the commonly accepted network protocol layering model as used in packet networks for decades. Conventionally, the network devices are only supposed to process the packets up to the network layer and leave the upper layers (i.e., transport layer and application layer) intact for the end hosts to process; however, INC requires the network devices to participate in the application logic so inevitably they need to process the related packets up to the application layer, as shown in .

Network Protocol Stack in INC | network |<----+--->| network | +-----------+ \ +-----------+ / +-----------+ client \---------------------/ server network ]]> Although such an architectural deviation does introduce some complexity to the network system, given the significant benefits presented by the applications, it is worthwhile to make the effort, as long as we can limit the use to just the beneficial applications and confine the scope in a confined network domain (e.g., a data center network). The computing functions need to be done in data plane fast path. If a network device needs to direct the application packets to the slow path (e.g., a local CPU or a remote server) for processing, that is no longer the INC in the scope of this draft (and its rationale becomes suspicious in this case). Programmable data plane devices use different programming languages (e.g., P4 and HDL) and have different chip architectures (e.g., RMT pipeline, RTC, and FPGA). These devices are optimized for simple packet processing and forwarding with limited hardware resources. Specifically, the devices are difficult to support complex stateful operations and mathematical calculations beyond integer addition and shift. No surprise the in-network computing functions for the supported applications are all relatively simple (e.g., resorting to lookup tables or counters). However, the programmable switch chip technology is also progressing fast with better stateful operation support and computing capabilities. It is conceivable that future programmable switches could undertake more computing tasks, albeit still in a facilitating role. To correctly handle the computing tasks, however, a reliable transport layer must be present. The transport layer provides the common services such as connection maintenance, reliability, flow control, and multiplexing. The existing INC applications either make oversimplified assumption to eschew this problem (e.g., assume the use of UDP as the transport layer protocol or ignore it) or provided ad hoc solution dedicated to a particular application which entangles the transport and application functions (e.g., ATP). A general protocol for the transport layer is needed for INC to take care the common transport issues. It can free the application developers from worrying about the transport issues and help them focus on the application logic itself. This draft provides the background of a suite of RPC-based applications which can take advantage of INC support, surveys the existing transport protocols to show they are insufficient or improper to be used in this context, and lays out the requirements to develop a general transport protocol tailored for such applications. The purpose of this draft is to help understand the problem domain and inspire the design and development a unified INC transport protocol.

INC Application Classification The INC applications concerned in this draft all follow the communication paradigm of Remote Procedure Call (RPC): A client sends a message with arguments to a server and get a response back which reflects the computation result based on the arguments. On the one hand, it is unlike TCP which is mainly used for transferring byte streams; on the other hand, it requires a reliable datagram service more than what UDP can support. We can classify these INC applications into three service models: from a set of clients, each sends a piece of data to a server roughly at the same time. The result can be computed and sent back to the clients when all the data pieces are received. A notable example is AllReduce. from a set of clients, each sends multiple data items to a server. The result can be computed when all the data items are received. An example of such applications is MapReduce a client sends individual requests to a server and get a response for each request. An example of such application is NetCache. From a different perspective, we can observe that there are three basic communication modes depending on the applications, as shown in : the INC network devices alone can completely finish a computing task. Therefore a client can choose to send a task to the INC network devices instead of a server and the final result is directly returned to the client from the INC network devices. the INC network devices can only partially finish a computing task and the intermediate result still needs to be sent to a server to finalize. The final result must be returned to the client from a server. the INC network devices may or may not finish a computing task, therefore the final result may be returned by the INC network devices or by a server. Each mode has its dominant benefits: Using DO mainly aims to reduce the latency and using DS mainly aims to reduce the traffic bandwidth and server load. Using HM may achieve both benefits, albeit with more implementation complexity.

In Network Computing Working Modes |devices| | |server| | | | |-+ | | +--^---+ +-------+ +---^--+ | | +------------------------------------+ Device Only Mode (DO) +-------+ +------+ +-------+ | +------+ | | |network| | | | |client+-------->|devices+-+------->|server| | | | |-+ | | +--^---+ +-------+ +--+---+ | | +-----------------------------------+ Device+Server Mode (DS) +-------+ +------+ +-------+ | +------+ | | |network| | | | |client+-------->|devices+.........>|server| | |<--------| |-+ | | +--^---+ +-------+ +--.---+ : : ..................................... Hybrid Mode (HM) ]]> provides the dominant combinations of the service model and communication model. Since AC may require too much resources which exceed network device's capability, so it is less used with the DO mode; IR usually aims to optimize the response latency, so the DS mode is less helpful, yet HM may provide a fallback mechanism for unsatisfied requests.

Service Model and Communication Model

Existing Transport Protocols We argue that the existing transport protocols are not suitable for INC. As the most widely used transport protocol, TCP (as well as its variants such as DCTCP and MPTCP) is ruled out because of its end-to-end streaming semantics. Any mutation to the TCP packet payloads is consider a break to the stream, but the INC applications which require network device collaboration do need to modify the packet payload. Also, any dropped packet in a TCP stream sensed by the receiver must be re-transmitted; this prohibits the INC applications which can terminate a packet and return the computing result directly. While theoretically it is possible to make the network device maintain two separate TCP connections with the two communicating end hosts, the cost of implementation is prohibitively large. More issues about TCP in data center can be found in . As another common transport protocol, UDP is unreliable and lack of mechanisms for flow control. Some previous INC application assumes the use of UDP as the transport layer for simplicity, but the provisional measure cannot meet the production level requirement and provide enough transport layer support for all the concerned INC applications. QUIC works for the RPC kind of communication. However, it is designed for wide area network, and a part of the packet header and the payload are encrypted which prohibits the application layer packet processing in network devices. Even without the encryption, the QUIC header information is not enough to support the INC applications. MTP is the first transport protocol dedicated for INC. It grasps some core requirements for INC and is open to different congestion control algorithms. But it is inspired by the pathlet routing and mainly focus on pathlet-based congestion control support. It is lack of efficient support to all the application types aforementioned. RDMA allows two end hosts to exchange data quickly. With either native support (i.e., Infiniband) or piggybacked by UDP or TCP, it requires in-order and immutable transport which has similar challenges as TCP for INC applications. HOMA is proposed to be a transport protocol in data center to replace TCP. However, HOMA is not designed with INC in mind either. Several INC applications (e.g., ATP and ASK) provide a customized transport layer. However, these protocols only work for a particular application. Moreover, there is a lack of a clear separation between the transport layer and the application layer. Some application layer function leaks into the transport layer, further limiting their generality.

Requirements The premise of the E2E principle is that it is more costly to guarantee the level of reliability by relying on the network than relying on the end hosts. INC introduces multiple end points in the communication with one of them resides in the network, effectively changing the communication paradigm from E2E to E2I2E (I means intermediate nodes which conduct the transport layer functionalities). Therefore, we need to revisit the E2E principle to see if we can break it or adapt to it in the new context. We can observe several properties for the covered INC applications. These applications are usually applied in a limited network scale with topology regularity (e.g., a data center network or an access network). The network is under the control of a single administrative entity. The benefit-cost ratio is significant enough to warrant the INC deployment, which usually requires a software- hardware and host-device co-design. Multiple applications with the same or different service models, or multiple jobs for the same applications can be active at the same time. Based on these observation, a new transport layer protocol, for INC in support of RPC-based applications can be designed. The protocol only works in a limited domain and it virtualizes the network as a single logical middle point. That is, if multiple network devices collaborate on a computing task, they are considered as one device. Packet forwarding among these devices needs to be handled by the network layer using techniques such as Segment Routing (SR) and Service Function Chaining (SFC). From the previous discussion, we lay out the design requirements of a transport protocol dedicated for INC : Due to the limited resource and capability of the programmable network devices, the transport layer functions in them cannot be complex. For example, the per-flow state machine and congestion control algorithms are difficult to be implemented in the programmable network devices. The protocol should aim to leave the complexity to the end hosts and require only simple processing in the programmable network devices. The different service models and communication models should be all supported. The protocol should also be independent of the underlying network layer protocol. Since the performance requirements of the applications may vary, the flow control and reliability mechanism of the protocol should be open to different algorithms. The protocol should be able to coexist with the other transport protocols.

IANA Considerations This document includes no request to IANA.

Security Considerations tbd