Hammerspace

loghyr@hammerspace.com

Transport Network File System Version 4 NFSv4 The Parallel Network File System (pNFS) allows for the metadata server to use CB_LAYOUTRECALL to recall a layout from a client by file id or file system id or all. It also allows the server to use CB_NOTIFY_DEVICEID to delete a devicid. It does not provide a mechanism for the metadata server to recall all layouts that have a data file on a specific deviceid. This document presents an extension to RFC8881 to allow the server recall layouts from clients based on deviceid. Discussion of this draft takes place on the NFSv4 working group mailing list (nfsv4@ietf.org), which is archived at . Working Group information can be found at .

Introduction In the Network File System version4 (NFSv4) with a Parallel NFS (pNFS) metadata server (), there is no mechanism for the metadata server to recall layouts from the client for when a particular deviceid (see Section 3.3.14 of ) either temporarily or permanently is no longer available. One use case is when the deviceids in a layout are separated by power fault domains. Each layout might describe 3 different devices, each contained in a different power fault domain. In such a scenario, a single fault domain can have the power removed and not cause the loss of access to the data. However, client I/O will be impacted as the client still has to perform WRITEs (see Section 18.32 of ) to the unavailable device, send LAYOUTERRORs (see Section 15.6 of ) to inform the metadata server of NFS4ERR_NXIO (see Section 15.1.16.3 of ). If the metadata sever had the means to recall layouts by deviceid, a lot of this unnecessary traffic could be eliminated. Finally, while the metadata server could recall layouts one by one, this is again unnecessary traffic and can be offloaded to the client. Besides the use case above, consider if the metadata server wants to set the NOTIFY4_DEVICEID_DELETE in the CB_NOTIFY_DEVICEID callback (see Section 20.12 of ). This flag cannot be set if a layout is outstanding for a deviceid. While the metadata server can revoke all such layouts, there is no way to know that the client has acknowledged that revocation and hence is still not doing I/O to other data files in the layout. The metadata server could fence those layouts as well (see Section 12.5.5 of ), but that can be an expensive operation. Using the process detailed in , the revisions in this document become an extension of NFSv4.2 . They are built on top of the external data representation (XDR) generated from .

Do we need ? The authors have tried to introduce this new functionality outside of a particular pNFS Layout Type. Does that work?

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Extension to Operation 5: CB_LAYOUTRECALL - Recall Layout from Client The original union layoutrecall4 (see Section 20.3.1 of ) is: The proposed extension is: With this minimal change, all of the semantics of CB_LAYOUTRECALL in (see Section 20.3 of ) remain the same, i.e., the client and server are aware of how CB_LAYOUTRECALL interacts with each other. The one issue to investigated is what happens if a NFSv4.2 client sees a LAYOUTRECALL4_DEVICEID in a CB_LAYOUTRECALL. They SHOULD return NFS4ERR_UNION_NOTSUPP, but the implementations might not be compliant with . As such, a survey should be conducted of the major implementations. Finally, when the client does handle a LAYOUTRECALL4_DEVICEID in a CB_LAYOUTRECALL, it MUST return all layouts which have a given deviceid. The server can determine that the client no longer has any layouts with the given devicedid once the client replies with NFS4ERR_NOMATCHING_LAYOUT.

Extraction of XDR This document contains the external data representation (XDR) description of the new open flags for delegating the file to the client. The XDR description is embedded in this document in a way that makes it simple for the reader to extract into a ready-to-compile form. The reader can feed this document into the following shell script to produce the machine readable XDR description of the new flags: That is, if the above script is stored in a file called "extract.sh", and this document is in a file called "spec.txt", then the reader can do: layout_wcc.x ]]> The effect of the script is to remove leading white space from each line, plus a sentinel sequence of "///". XDR descriptions with the sentinel sequence are embedded throughout the document. Note that the XDR code contained in this document depends on types from the NFSv4.2 nfs4_prot.x file (generated from ). This includes both nfs types that end with a 4, such as offset4, length4, etc., as well as more generic types such as uint32_t and uint64_t. While the XDR can be appended to that from , the various code snippets belong in their respective areas of the that XDR.

Code Components Licensing Notice Both the XDR description and the scripts used for extracting the XDR description are Code Components as described in Section 4 of "Legal Provisions Relating to IETF Documents". These Code Components are licensed according to the terms of that document.

Security Considerations There are no new security considerations beyond those in .

IANA Considerations IANA should use the current document (RFC-TBD) as the reference for the new entries.

References Normative References Informative References IETF Trust

Acknowledgments Trond Myklebust and Paul Saab have were invloved in the initial requirements for this functionality.