High speed data express in IP: Concept, Reference Architecture and Technologies

With the rapid development of AI technology, intelligent computing, and super-computing services, the demand for wide-area transmission of massive data is increasing. How to transfer a large amount of data quickly, efficiently and conveniently has become an important problem that needs to be solved in the development of communication networks. With the network bandwidth from 10G to 25G, 100G, 200G and even 400G, bandwidth is not the bottleneck of data transmission, the key to data transmission is how to make full use of large bandwidth to achieve efficient throughput. In most cases, the effective throughput is less than the throughput, which in turn is less than the bandwidth. Bandwidth refers to the number of bits that can be transmitted per second on the link, which depends on the link clock rate and channel coding, also known as line speed (network device transmission capacity). Throughput refers to the actual amount of data transmitted on the bandwidth (terminal device transmission capacity). Goodput refers to the throughput that truly meets service requirements, excluding invalid packets such as retransmission and discarding (service data transmission capability).. In-depth analysis shows that the data transmission throughput performance is mainly affected by five factors: transmission distance, network packet loss rate, effective data length , sending window and the impact factor. As the transmission distance increases, RTT (round trip delay) increases. In reliable transmission mode, the reply time of ACK acknowledgement messages becomes longer, and the sending sliding window cannot be adjusted in time. Network packet loss rate will affect the rate adjustment of congestion control algorithm, and increase the tail delay of data transmission. Effective data length refers to the effective data length of the packet header removed, which can be regarded as MSS. Increasing MSS will help improve throughput. The sending window affects the duration of continuous data sending. If the window is too small, data will be sent before the ACK reply is completed, and sending will be paused. The impact factor F is a comprehensive parameter, such as the CPU frequency, PCIE rate, disk I/O rate, and bandwidth that affect the packet processing rate, 0<F<1, if these have no impact, the value of F is 1. In summary, we can draw the following conclusion: in addition to improving the network environment (improving the physical bandwidth, etc.), the end-to-end data sending and receiving mechanism and transmission protocol are also a key part of the effective throughput. In order to improve the data transmission efficiency and reduce the data transmission cost as much as possible, we propose to build a highly elastic and high-throughput data transmission network based on IP network, and use elastic bandwidth, load balancing, security encryption and other technologies to realize new data express services.

The data express solution enables massive data transmission from the data source to the destination. Data sources are devices or systems that generate or store large amounts of data, such as enterprise servers, databases, and storage devices. Data sources can be multiple nodes distributed across different geographies and network environments, or they can be a single node concentrated in one location. Data destination refers to the device or system that receives or processes a large amount of data, such as cloud computing platform, intelligent computing center, and supercomputing center. The data destination can be a public service provided by a third party or a cloud data center built by a data express operator. Based on the massive transmission requirements from the data source to the destination end, the overall architecture of the data express solution is designed according to the hierarchical idea, which mainly includes three parts: cloud private network infrastructure, network controller and data express operation platform, as shown in

+--------------------+ | | | Data Express | Task -------> | Operation Platform | Delivery | | +---------+----------+ | +----------+-----------+ | | +---------------+ Network Controller +---------+ | | | | | +-----------+----------+ | | | | | | | +-------+---------------------------+--------------------+----------+ | | | +-----------+ +-----------+ | | |Data Source| | Super | | | | Memory | +------+ Computing | | | | card/disk | Wide Area | | Center | | | +-----+-----+ +------+Interconnection +--+--+ +-----------+ | | | |Cloud +----------------+Cloud| | | | | PE | | PE | | | | +---+--+ +--+--+ +-----------+ | | +-----v------+ | | | General | | | |Data Express| | +------+ Computing | | | | Station +--------+ | Center | | | +------------+ +-----------+ | | | | Infrastructure | +-------------------------------------------------------------------+ Infrastructure Data Express can use the existing network infrastructure to provide data transmission services for customers, and can access different data source and data consumption end through SPN, PON, optical fiber direct connection and other means according to the situation of live network resources, so as to achieve dynamic scheduling and optimization of network resources and ensure the quality and efficiency of data transmission. SPN is the recommended access mode for data express because of its large access bandwidth, elastic bandwidth configuration, and wide coverage. Network Controller Network controllers include SPN and other network controllers in different fields, which complete the network configuration according to the requirements of the service platform, and realize the whole process automation and intelligent opening of the data express service. The main functions include: Network configuration: Automatically or manually configure parameters such as network topology, routes, and security policies based on network requirements and scenarios to achieve flexibility and customization. Information collection: Collects and analyzes network performance, quality, and security indicators in real time to optimize and improve data express services. Network monitoring: Monitors the network in real time to detect and handle anomalies such as network faults, attacks, and leaks, ensuring data transmission security and stability. Operation Platform Operation platform refers to the online platform that provides users with one-stop data express service management and monitoring functions, and provides users with business ordering, opening, monitoring, billing and other functions. The operation platform can be accessed through the web or mobile terminal to realize the visualization and intelligence of the whole process of data express business. The main functions include: Service order: Users can order data express services online on the operation platform, and select parameters such as data source, destination, transmission mode, and transmission time to generate service orders. Enabling the service: Users can enable the online data express service on the operation platform. Based on the order information, users can automatically or manually configure data sources, destinations, and network resources to start data transfer tasks. Service monitoring: Users can monitor data express services online on the operation platform and view data transfer progress, status, rate, and quality in real time to detect and handle anomalies in a timely manner. Service charging: Users can charge online data express services on the operating platform, and generate and pay bills based on preset charging rules based on data transfer volume, duration, and bandwidth.

TBD

TBD.

This document has no requests to IANA.