]> Hammerspace

loghyr@gmail.com

Hammerspace

pierre.evenou@hammerspace.com

General Network File System Version 4 Internet-Draft Parallel NFS (pNFS) allows a separation between the metadata (onto a metadata server) and data (onto a storage device) for a file. The flex file layout type version 2 further allows for erasure encoding types to provide data integrity. In this document, a new erasure encoding type for the Mojette Transformation is introduced. Note to Readers Discussion of this draft takes place on the NFSv4 working group mailing list (nfsv4@ietf.org), which is archived at . Source code and issues list for this draft can be found at . Working Group information can be found at .

Introduction In Parallel NFS (pNFS) (), the metadata server returns layout type structures that describe where file data is located. There are different layout types for different storage systems and methods of arranging data on storage devices. defined the Flexible File Version 2 Layout Type used with file-based data servers that are accessed using the NFS protocols: NFSv3 , NFSv4.0 , NFSv4.1 , and NFSv4.2 . This document introduces a new Erasure Encoding Type called Mojette Transformation. Using the process detailed in , the revisions in this document become an extension of NFSv4.2 . They are built on top of the external data representation (XDR) generated from .

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Mojette Transform

Introduction The Mojette Transform is an erasure coding technique that provides fault tolerance for data storage systems by enabling the recovery of lost data blocks. This section describes the integration of the systematic Mojette Transform into the NFS protocol, focusing on encoding and decoding file system blocks, typically sized at 4KB or 8KB.

Encoding The Mojette Transform involves the following steps to encode a data block:

Initialization

Each data block is treated as a 2D grid of data elements (pixels). Typically, a block is structured as a matrix of size , where and are the dimensions of the grid.

Projections Calculation

Projections are computed along specific directions defined by pairs of coprime integers . Each projection sums the values of the data elements (pixels) along a line defined by these directions. The size of a projection is given as in . For a given projection direction , and if we let be 1 if the argument is zero and 0 otherwise, the projection values are calculated as in

Projection(b, p_i, q_i) =
∑{k=0}^{Q−1} ∑{l=0}^{P−1} Data(k, l) ⋅ Δ(b − l ⋅ p_i + k ⋅ q_i)

Size of Projection Calculation

Decoding Decoding the Mojette Transform is the inverse of encoding, involving the reconstruction of data from projection data to fill an empty grid. This involves solving a system of linear equations defined by the projection differences and the projection directions . The algorithm iterates to refine the values of the missing data elements until the original data block is reconstructed. Data reconstruction is possible if Katz's criterion holds, which was extended to any convex shape. It specifies that reconstruction is valid if for a given set of n projections along directions either or . Adjusting the number of lines Q and the projections set allows setting a desired fault-tolerance threshold. For example a 64x4 grid can be decoded by the projection set as .

Systematic and Non Systematic Implementations A systematic code is an error-correcting code where the input data is embedded directly in the encoded output. In contrast, a non-systematic code produces an output that does not contain the original input symbols. The Mojette Transform can be implemented in both ways, allowing it to adapt to various use cases.

Data Block Representation In the context of NFS, a data block corresponds to a file system block, which is a contiguous segment of data, typically 4KB or 8KB in size. The Mojette Transform encodes these blocks to ensure data integrity and availability in distributed storage environments.

Non-Systematic Mojette Transform

Block Encoding In the non-systematic version of the Mojette Transform, the original data block is not directly included in the encoded output. Instead, the entire encoded output consists of projections computed from the original data. The number of computed projections n is larger than the number of projections m required to rebuild the initial data.

Block Decoding To decode a file system block that has undergone the non-systematic Mojette Transform, the following steps are followed:

Identify Available Projections

Determine which projections are available. A least m projections (what ever they are) out of n must be available.

Recompute Data Block

Apply the inverse Mojette Transform to rebuild the original Data.

Example Assume a file system block of 4KB is divided into a 128x4 matrix of 128-bit elements. Using the non-systematic Mojette Transform, we compute projections along selected directions, such as (-2,1), (-1, -1), (0,1), (1,1), (2,1) and (3,1). The original data is not stored directly; instead, the projections are stored. If a data loss occurs, for instance, if two projections are lost, the missing elements can be recovered by using the remaining projections and solving the inverse problem. Any set of 4 projections among the 6 generated can rebuild exactly the original data block

Systematic Mojette Transform

Block Encoding In the systematic version, the original data block (file system block) is part of the encoded output. Additional projections are calculated to provide redundancy. If is the number of original data blocks and is the total number of encoded blocks (including projections), the systematic code will have the first blocks as the original data and the remaining blocks as projections.

Block Decoding To decode a file system block that has undergone the Systematic Mojette Transform, the following steps are followed:

Identify Missing Data

Determine which data lines are missing in the block. Let e be the number of missing lines.

Recompute Projections

Compute the projections of the available (partial) data blocks. Calculate the differences between the projections of the full data and the partial data.

Combine Existing Data and Recomputed projection

Recreate the full block of data by combining the existing data with the reconstructed data according to their positions in the block.

Example Assume a file system block of 4KB is divided into a $64\times4$ matrix of 128-bit elements. Using the systematic Mojette Transform, we first compute projections along selected directions, such as (0,1), and (1,1). The original 4 blocks of 64 128-bit elements remains part of the encoded data, and the 2 additional projections are stored for redundancy. If a data loss occurs, the missing elements can be recovered by using the projections and solving the inverse problem.

Conclusion The Mojette Transform provides two implementations for an efficient and effective way to enhance data reliability by encoding file system blocks with additional projections. These methods ensure that data can be reconstructed even in the presence of failures, thereby enhancing the fault tolerance of the file system. In summary, the Mojette Transform offers robust solutions for data reliability in file systems, balancing redundancy, efficiency, and performance to ensure data integrity and quick recovery from failures.

Benefits of Non-Systematic Mojette Transform

Fast Failure Detection

By storing only encoded data and having the ability to rebuild the original block from any sufficient subset of projections, the non-systematic encoding implementation offers great flexibility and allows fast failure detection and recovery.

Constant Performance

The non-systematic decoding algorithm is the most performant of all implementations. Although decoding always occurs, the overhead is low, and unlike systematic encoding, performance remains constant regardless of the number of failures.

Benefits of Systematic Mojette Transform

Redundancy Reduction

Systematic Mojette coding reduces redundancy by integrating the original data blocks into the encoded data, unlike non-systematic codes that generate entirely new data from the original.

Efficiency

Fewer projections need to be calculated and stored, reducing both computational and storage overhead.

Performance

Decoding is faster and simpler, especially when some original data blocks are available, enabling quicker data access. However, performance is slightly degraded in the case of failures and depends on the number of failures.

Extension of Flexible File Layout Type Version 2 Encoding Type

ffv2_encoding_type4

enum ffv2_encoding_type4

fv2_encoding_type4 is extended in to introduce two different erasure encoding types: FFV2_ENCODING_MOJETTE_SYSTEMATIC and FFV2_ENCODING_MOJETTE_NON_SYSTEMATIC. They are intoduced at this level instead of at the ffv2_encoding_type_data4 () in order for the client to negotiate the support of one over the other with the NFS4ERR_ERASURE_ENCODING_NOT_SUPPORTED error ().

ffv2_mojette_faulty_devices4

enum ffv2_mojette_faulty_devices4

The ffv2_mojette_faulty_devices4 (see Figure 4) can be used in both the layout_hint ( and ) and the ffl_encoding_type_data () to convey the distribution of FFV2_DS_FLAGS_ACTIVE and FFV2_DS_FLAGS_SPARE projection blocks () in the layouts for the Flexible File Version 2 Layout Type. The name of each of the enum targets ends with 'X_Y', which states that there is a need for files to compose the projection. X is the number of FFV2_DS_FLAGS_ACTIVE blocks and Y is the number of FFV2_DS_FLAGS_SPARE blocks.

ffv2_encoding_type_data4

union ffv2_encoding_type_data4

This addition of FFV2_ENCODING_MOJETTE_SYSTEMATIC and FFV2_ENCODING_MOJETTE_NON_SYSTEMATIC to the ffv2_encoding_type_data4 () allows for the metadata server to inform the client as to the expected distribution of FFV2_DS_FLAGS_ACTIVE and FFV2_DS_FLAGS_SPARE projection blocks inside the ffm_data_servers arm for the Mojette erasure encoding types.

Examples Tying It All Together

Block_Sizes Consider a FFV2_ENCODING_MOJETTE_NON_SYSTEMATIC encoding type which needs 6 active blocks and 2 spare blocks in the payload (Not a valid ffv2_mojette_faulty_devices4, but needed to show varied block sizes). As can be seen in , 4kB data blocks need blocks about 1kB in size. But not all blocks are the same size. Example sizes of a Mojette Projection

Projection ID	0	1	2	3	4	5	6	7
p value	-3	-2	-1	0	1	2	0	0
q value	1	1	1	1	1	1	0	0
size (bytes)	1048	1080	1080	1048	1064	1048	0	0

Extraction of XDR This document contains the external data representation (XDR) description of the uncacheable attribute. The XDR description is presented in a manner that facilitates easy extraction into a ready-to-compile format. To extract the machine-readable XDR description, use the following shell script: For example, if the script is named 'extract.sh' and this document is named 'spec.txt', execute the following command: uncacheable_prot.x ]]> This script removes leading blank spaces and the sentinel sequence '///' from each line. XDR descriptions with the sentinel sequence are embedded throughout the document. Note that the XDR code contained in this document depends on types from the NFSv4.2 nfs4_prot.x file (generated from ). This includes both nfs types that end with a 4, such as offset4, length4, etc., as well as more generic types such as uint32_t and uint64_t. While the XDR can be appended to that from , the code snippets should be placed in their appropriate sections within the existing XDR.

Security Considerations This document has the same security considerations as both Flex Files Layout Type version 1 () and NFSv4.2 ().

IANA Considerations This document introduces changes in the 'Flex Files V2 Erasure Encoding Type Registry'. This document defines both the FFV2_ENCODING_MOJETTE_SYSTEMATIC and FFV2_ENCODING_MOJETTE_NON_SYSTEMATIC types for Client-Side Mojette Transformations. Flex Files V2 Erasure Encoding Type Assignments

Erasure Encoding Type Name	Value	RFC	How	Minor Versions
FFV2_ENCODING_MOJETTE_SYSTEMATIC	2	RFCTBD10	L	2
FFV2_ENCODING_MOJETTE_NON_SYSTEMATIC	3	RFCTBD10	L	2

References Normative References In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. This document describes the External Data Representation Standard (XDR) protocol as it is currently deployed and accepted. This document obsoletes RFC 1832. [STANDARDS-TRACK] The Network File System (NFS) version 4 protocol is a distributed file system protocol that builds on the heritage of NFS protocol version 2 (RFC 1094) and version 3 (RFC 1813). Unlike earlier versions, the NFS version 4 protocol supports traditional file access while integrating support for file locking and the MOUNT protocol. In addition, support for strong security (and its negotiation), COMPOUND operations, client caching, and internationalization has been added. Of course, attention has been applied to making NFS version 4 operate well in an Internet environment. This document, together with the companion External Data Representation (XDR) description document, RFC 7531, obsoletes RFC 3530 as the definition of the NFS version 4 protocol. This document describes NFS version 4 minor version 2; it describes the protocol extensions made from NFS version 4 minor version 1. Major extensions introduced in NFS version 4 minor version 2 include the following: Server-Side Copy, Application Input/Output (I/O) Advise, Space Reservations, Sparse Files, Application Data Blocks, and Labeled NFS. This document provides the External Data Representation (XDR) description for NFS version 4 minor version 2. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This document describes the rules relating to the extension of the NFSv4 family of protocols. It covers the creation of minor versions, the addition of optional features to existing minor versions, and the correction of flaws in features already published as Proposed Standards. The rules relating to the construction of minor versions and the interaction of minor version implementations that appear in this document supersede the minor versioning rules in RFC 5661 and other RFCs defining minor versions. Parallel NFS (pNFS) allows a separation between the metadata (onto a metadata server) and data (onto a storage device) for a file. The flexible file layout type is defined in this document as an extension to pNFS that allows the use of storage devices that require only a limited degree of interaction with the metadata server and use already-existing protocols. Client-side mirroring is also added to provide replication of files. This document describes the Network File System (NFS) version 4 minor version 1, including features retained from the base protocol (NFS version 4 minor version 0, which is specified in RFC 7530) and protocol extensions made subsequently. The later minor version has no dependencies on NFS version 4 minor version 0, and is considered a separate protocol. This document obsoletes RFC 5661. It substantially revises the treatment of features relating to multi-server namespace, superseding the description of those features appearing in RFC 5661. Hammerspace Parallel NFS (pNFS) allows a separation between the metadata (onto a metadata server) and data (onto a storage device) for a file. The Flexible File Version 2 Layout Type is defined in this document as an extension to pNFS that allows the use of storage devices that require only a limited degree of interaction with the metadata server and use already-existing protocols. Data replication is also added to provide integrity. Informative References This paper describes the NFS version 3 protocol. This paper is provided so that people can write compatible implementations. This memo provides information for the Internet community. This memo does not specify an Internet standard of any kind.

Acknowledgments The following from Hammerspace were instrumental in driving the incorporation of the Mojette Transformation into an encoding type for Flexible Files Version 2 Layout Type: David Flynn, Trond Myklebust, Tom Haynes, Didier Feron, Jean-Pierre Monchanin, Pierre Evenou, and Brian Pawlowski.