]> Hammerspace

loghyr@gmail.com

General Network File System Version 4 Internet-Draft The Network File System version 4.2 (NFSv4.2) allows a client to cache both metadata and data for file objects, as well as metadata for directory objects. While caching directory entries (dirents) can improve performance, it can also prevent the server from enforcing access control on individual dirents. Similarly, caching file data can lead to performance issues if the cache hit rate is low. This document introduces a new uncacheable attribute for NFSv4. Files and dirents marked as uncacheable MUST NOT be stored in client-side caches. This ensures data consistency and integrity by requiring clients to always retrieve the most recent data directly from the server. This document extends NFSv4.2 (see RFC7862). Note to Readers Discussion of this draft takes place on the NFSv4 working group mailing list (nfsv4@ietf.org), which is archived at . Source code and issues list for this draft can be found at . Working Group information can be found at .

Introduction With a remote filesystem, the client typically caches both directory entries (dirents) and file contents in order to improve performance. Several assumptions are made about the rate of change in the dirents and the number of clients trying to concurrently access a file. With NFSv4.2, this could theoretically be mitigated by directory delegations for the dirents and practically by file delegations for the file contents. There are prior efforts to bypass both the dirent and file caching. Access Based Enumeration (ABE) is used in Server Message Block (SMB) protocol to effectively limit the namespace visibility per user. In Highly Parallel Computing (HPC) workloads, file caching is bypassed in order to achieve consistent work flows. This document introduces the uncacheable attribute to NFSv4.2 to implement ABE and to bypass file caching on the client. As such, it is an OPTIONAL to implement attribute for NFSv4.2. However, if both the client and the server support this attribute, then the client MUST follow the semantics of uncacheable.What about mixed modes? A client can easily determine whether or not a server supports the uncacheable attribute with a simple GETATTR on any dirent. If the server does not support the uncacheable attribute, it will return an error of NFS4ERR_ATTRNOTSUPP. The only way that the server can determine that the client supports the attribute is if the client sends either a GETATTR or a SETATTR with the uncacheable attribute. While some argument could be made to introduce two new attributes, the functionality of the uncacheable attribute dictates which cache is to be bypassed. As ABE is concerned with walking the namespace, it is only applicable to be acted on dirents which are of type attribute value of NF4DIR. And as bypassing file caching is file based, it is only applicable for dirents which are of type attribute value of NF4REG.What about the other file types? Using the process detailed in , the revisions in this document become an extension of NFSv4.2 . They are built on top of the external data representation (XDR) generated from .

Definitions

Access Based Enumeration (ABE)

When servicing a READDIR or GETATTR operation, the server provides results based on the access permissions of the user making the request.

dirent

A directory entry, representing either a file or a subdirectory. In the context of NFSv4, a dirent marked as uncacheable MUST NOT be cached by clients.

dirent caching

A client cache that is used to avoid looking up attributes.

file caching

A client cache, normally called the page cache, which caches the contents of a regular file. Typical usage would be to accumulate changes to be bunched together for writing to the server.

Further, the definitions of the following terms are referenced as follows:

• directory delegations ()
• file delegations ()
• GETATTR ()
• hidden ()
• Mandatory Access Control (MAC) ()
• NF4DIR ()
• NF4REG ()
• NFS4ERR_ATTRNOTSUPP (
• mode ()
• offline ()
• owner ()
• owner_group ()
• READDIR ()
• SETATTR ()
• system ()
• type ()

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Caching of Dirents With a remote filesystem, the client typically caches directory entries (dirents) locally to improve performance. This cooperation succeeds because both the server and client operate under POSIX semantics () and agree to interpretation of mode bits with respect to the uid and gid in NFSv3 . For NFSv4.2, these would respectively be the mode, owner, and owner_group attributes defined in . Note that this cooperation does not apply to Access Control List (ACLs) entries as NFSv4.2 does not implement a strict POSIX style ACL. NFSv4.2 does implement NFSv4.1 ACLs, which are enforced on the server and not the client. As such, ACL enforcement requires the client to bypass the dirent cache to have checks done when a new user attempts to access the dirent. Another consideration is that not all server implementations natively support the SMB . Instead, they layer Samba on top of the NFSv4.2 service. The attributes of hidden, system, and offline have already been introduced in the NFSv4.2 protocol to support Samba. The Samba implementation can utilize these attributes to provide SMB semantics. While private protocols can supply these features, it is better to drive them into open standards. Another concept that can be adapted from SMB is that of Access Based Enumeration (ABE). If a share or a folder has ABE enabled, then the user can only see the files and sub-folders for which they have permissions. Under the POSIX model, this can be done on the client and not the server. However, that only works with uid, gid, and mode bits. If we consider identity mappings, ACLS, and server local policies, then the determination of ABE MUST be done on the server. Since cached dirents are shared by all users on a client, and the client cannot determine access permissions for individual dirents, all users are presented with the same set of attributes. To address this, this document introduces the new uncacheable attribute. This attribute instructs the client not to cache the dirent for a file or directory object. Consequently, each time a client queries for these attributes, the server's response can be tailored to the specific user making the request.

Uncacheable Dirents If a file object or directory has the uncacheable attribute set, then the client MUST NOT cache its dirent attributes. This means that even if the client has previously retrieved the attributes for a user, it MUST query the server again for those attributes on subsequent requests. Additionally, the client MUST NOT share attributes between different users.

Caching of File Data In addition to caching metadata, clients can also cache file data. The uncacheable attribute also instructs the client to bypass its page cache for the file. This behavior is similar to using the O_DIRECT flag with the open call (). This can be beneficial for files that are not shared or for files that do not exhibit access patterns suitable for caching. However, the real need for bypassing write caching is evident in HPC workloads. In general, these involve massive data transfers and require extremely low latency. Write caching can introduce unpredictable latency, as data is buffered and flushed later.

Uncacheable Files If a file object is marked as uncacheable, all modifications to the file MUST be immediately sent from the client to the server. In other words, the file data is also not cacheable.

XDR for Offline Attribute

Extraction of XDR This document contains the external data representation (XDR) description of the uncacheable attribute. The XDR description is presented in a manner that facilitates easy extraction into a ready-to-compile format. To extract the machine-readable XDR description, use the following shell script: For example, if the script is named 'extract.sh' and this document is named 'spec.txt', execute the following command: uncacheable_prot.x ]]> This script removes leading blank spaces and the sentinel sequence '///' from each line. XDR descriptions with the sentinel sequence are embedded throughout the document. Note that the XDR code contained in this document depends on types from the NFSv4.2 nfs4_prot.x file (generated from ). This includes both nfs types that end with a 4, such as offset4, length4, etc., as well as more generic types such as uint32_t and uint64_t. While the XDR can be appended to that from , the code snippets should be placed in their appropriate sections within the existing XDR.

Security Considerations For a given user A, a client MUST NOT make access decisions for uncacheable dirents retrieved for another user B. These decisions MUST be made by the server. If the client is Labeled NFS aware (), then the client MUST locally enforce the MAC security policies. The uncacheable attribute allows dirents to be annotated such that attributes are presented to the user based on the server's access control decisions.

IANA Considerations This document has no IANA actions.

References Normative References In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. This document describes the External Data Representation Standard (XDR) protocol as it is currently deployed and accepted. This document obsoletes RFC 1832. [STANDARDS-TRACK] This Glossary provides definitions, abbreviations, and explanations of terminology for information system security. The 334 pages of entries offer recommendations to improve the comprehensibility of written material that is generated in the Internet Standards Process (RFC 2026). The recommendations follow the principles that such writing should (a) use the same term or definition whenever the same concept is mentioned; (b) use terms in their plainest, dictionary sense; (c) use terms that are already well-established in open publications; and (d) avoid terms that either favor a particular vendor or favor a particular technology or mechanism over other, competing techniques that already exist or could be developed. This memo provides information for the Internet community. This memo outlines high-level requirements for the integration of flexible Mandatory Access Control (MAC) functionality into the Network File System (NFS) version 4.2 (NFSv4.2). It describes the level of protections that should be provided over protocol components and the basic structure of the proposed system. The intent here is not to present the protocol changes but to describe the environment in which they reside. This document describes NFS version 4 minor version 2; it describes the protocol extensions made from NFS version 4 minor version 1. Major extensions introduced in NFS version 4 minor version 2 include the following: Server-Side Copy, Application Input/Output (I/O) Advise, Space Reservations, Sparse Files, Application Data Blocks, and Labeled NFS. This document provides the External Data Representation (XDR) description for NFS version 4 minor version 2. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This document describes the rules relating to the extension of the NFSv4 family of protocols. It covers the creation of minor versions, the addition of optional features to existing minor versions, and the correction of flaws in features already published as Proposed Standards. The rules relating to the construction of minor versions and the interaction of minor version implementations that appear in this document supersede the minor versioning rules in RFC 5661 and other RFCs defining minor versions. This document describes the Network File System (NFS) version 4 minor version 1, including features retained from the base protocol (NFS version 4 minor version 0, which is specified in RFC 7530) and protocol extensions made subsequently. The later minor version has no dependencies on NFS version 4 minor version 0, and is considered a separate protocol. This document obsoletes RFC 5661. It substantially revises the treatment of features relating to multi-server namespace, superseding the description of those features appearing in RFC 5661. Hammerspace Hammerspace The Network File System v4 (NFSv4) allows a client to both open a file and be granted a delegation of that file. This delegation provides the client the right to authoritatively cache metadata on the file locally. This document presents several extensions for both the opening and delegating of the file to the client. This document extends NFSv4.2 (see RFC7863). Informative References n.d. IEEE This paper describes the NFS version 3 protocol. This paper is provided so that people can write compatible implementations. This memo provides information for the Internet community. This memo does not specify an Internet standard of any kind. n.d. Microsoft Learn n.d.

Acknowledgments Trond Myklebust and Thomas Haynes all worked on the prototype at Hammerspace.