MarketReFS
Company Profile

ReFS

Resilient File System (ReFS), codenamed "Protogon", is a Microsoft proprietary file system introduced with Windows Server 2012 with the intent of becoming the "next generation" file system after NTFS.

Comparison with NTFS
Major new features Improved reliability for on-disk structures ReFS uses B+ trees for all on-disk structures, including all metadata and file data. Metadata and file data are organized into tables similar to a relational database. The file size, number of files in a folder, total volume size, and number of folders in a volume are limited by 64-bit numbers; as a result, in practice ReFS supports a maximum file size of 35 petabytes, and a maximum volume size of 35 petabytes. The theoretical maximum is way beyond that however not even Windows supports ReFS volumes with the full theoretical limits. The design allows for a theoretical maximum of a single volume of 2^78 bytes with 16KB cluster size (2^64 * 16 * 2^10) with a maximum of 2^64-1 bytes for a single file. That maximum volume size is currently soft limited by Windows’ stack addressing being limited to 2^64 bytes; a maximum path length of 32K (soft limited to 255 unicode characters on Windows to align with NTFS and because of restrictions in some older APIs for handling file paths); a maximum of 2^64 directories on a single volume; 2^64 files per directory; a single storage pool is limited to a maximum of 4 PB. Built-in resilience ReFS employs an allocation-on-write update strategy for metadata, in which case the file update strategy also implements allocation-on-write for file data; this is controlled by a new "integrity" attribute applicable to both files and directories. If file data or metadata become corrupt, the file can be deleted without taking the whole volume offline for maintenance, and then be restored from the backup. As a result of built-in resiliency, administrators do not need to periodically run error-checking tools such as CHKDSK when using ReFS. In contrast, NTFS only calculates a checksum for metadata, the check for sectors is done by storage hardware (such as sector CRC-32 command of SATA and NVMe). Compatibility with existing APIs and technologies ReFS supports only a subset of NTFS features, and only supports Win32 APIs that are "widely adopted". It does not require new system APIs, and most file system filters continue to work with ReFS volumes. symbolic links, junction points, mount points, reparse points, volume snapshots, file IDs, and oplock. ReFS seamlessly integrates with Storage Spaces, ReFS resiliency features enhance the mirroring feature provided by Storage Spaces and can detect whether any mirrored copies of files become corrupt using a data scrubbing process, NTFS features missing from ReFS Some NTFS features are not implemented in ReFS. These include object IDs, 8.3 filename, NTFS compression, Encrypting File System (EFS), transactional NTFS, extended attributes, and disk quotas. Dynamic disks with mirrored or striped volumes are replaced with mirrored or striped storage pools provided by Storage Spaces; however, automated error-correction is only supported on mirrored spaces. Data deduplication was missing in early versions of ReFS. Support for alternate data streams and hard links was initially not implemented in ReFS. In Windows 8.1 64-bit and Server 2012 R2, the file system reacquired support for alternate data streams, with lengths of up to 128K, and automatic correction of corruption when integrity streams are used on parity spaces. ReFS had initially been unsuitable for Microsoft SQL Server instance allocation due to the absence of alternate data streams. Hard links were introduced in v3.5, debuting in a preview version of Windows 10 Enterprise (19536) , but this feature was only available on newly formatted volumes would, not upgraded ones. == Implementations ==
Implementations
ReFS was initially added to Windows Server 2012 only, with the aim of gradual migration to consumer systems in future versions; this was achieved as of Windows 8.1. but far slower when full integrity checking was enabled, a result attributed to the relative newness of ReFS. The ability to create ReFS volumes was removed in Windows 10's 2017 Fall Creators Update for all editions except Enterprise and Pro for Workstations. Starting with Windows 11 build 22621.2338, ReFS is re-introduced via a Dev Drive feature; allowing fixed storage drives and VHDs to be formatted as ReFS, with special file and Microsoft Defender policies added during use. The cluster size of a ReFS volume is either 4 KB or 64 KB. == History ==
History
Server 2016 At the Storage Developer Conference 2015, a Microsoft developer presented enhancements of ReFS expected to be released with Windows Server 2016 and included in Technical Preview 4, titled "ReFS v2". It highlighted that ReFS now included capabilities for very high speed moving, reordering, and cloning of blocks between files (which can be done for all blocks of a file). This is particularly needed for virtualization, and is stated to allow fast provisioning, diff merging, and tiering. Other enhancements cover the redo log (for synchronous disk writes), parallelization, efficient tracking of uninitialized sparse data and files, and efficient 4k I/O. Server 2025 Windows Server 2025 (using ReFS version 3.14) supports Encrypted File System. Server vNext Windows Server vNext Insider Preview build 29531 enables ReFS booting; this requires a Windows Recovery Environment (WinRE) partition that is at least 2 GB. Versions ReFS has some different versions, with various degrees of compatibility between operating system versions. Aside from development versions of the filesystem, usually, later operating system versions can mount filesystems created with earlier OS versions (backwards compatibility). Some features may not be compatible with the feature set of the OS. The version, cluster size and other features of the filesystem can be queried with the command fsutil fsinfo refsinfo volumename. • 1.1: The original version, formatted by Windows Server 2012. • 1.2: Default version if formatted by Windows 8.1, Windows 10 RTM to v1607, Windows Server 2012 R2, and when specified ReFSv1 on Windows Server 2016. Can use alternate data streams under Windows Server 2012 R2. • 2.0: Default version formatted by Windows Server 2016 TP2 and TP3. Could not be mounted in Windows 10 Build 10130 and later, or Windows Server 2016 TP4 and later. • 3.0: Default version formatted by Windows Server 2016 TP4 and TP5. • 3.1: Default version formatted by Windows Server 2016 RTM. • 3.2: Default version formatted by Windows 10 v1703 and Windows Server Insider Preview build 16237. Can be formatted with Windows 10 Insider Preview 15002 or later (though only became the default somewhere between 15002 and 15019). Supports deduplication in the server version. • 3.3: Default version formatted by Windows 10 Enterprise v1709 (ReFS volume creation ability removed from all editions except Enterprise and Pro for Workstations starting with build 16226; read/write ability remains == Known problems ==
Known problems
Issues identified or suggested for ReFS, when running on Storage Spaces, include: • Adding thin-provisioned ReFS on top of Storage Spaces (according to a 2012 pre-release article) can fail in a non-graceful manner, in which the volume without warning becomes inaccessible or unmanageable. This can happen, for example, if the physical disks underlying a storage space became too full. Smallnetbuilder comments that, in such cases, recovery could be "prohibitive" as a "breakthrough in theory" is needed to identify storage space layouts and recover them, which is required before any ReFS recovery of file system contents can be started; therefore it recommends using backups as well. • Poor backward compatibility handling. Windows automatically upgrades any ReFS volumes it sees to the current version of ReFS, without requiring any user interaction or confirmation. This can cause volumes to be unreadable with older versions of Windows. == Comparison with other file systems ==
Comparison with other file systems
Like ReFS, ZFS, Bcachefs, and Btrfs are designed to integrate data protection, snapshots, and background error correction. In 2012, Phoronix wrote an analysis of ReFS vs Btrfs. At the time, their features were similar, with both supporting checksums, RAID-like use of multiple disks, and error correction. However, ReFS lacked copy-on-write snapshots and compression, both found in Btrfs and ZFS. In 2014, BetaNews wrote a review of ReFS and assessed its readiness for production use. The review concluded that ReFS had at least some advantages over two file systems then available for file servers running Unix-like operating systems, ZFS and ReiserFS. ZFS (used in Solaris, illumos, FreeBSD and others) was widely criticized for its comparatively extreme memory requirements of many gigabytes of RAM for online deduplication. However, online deduplication was not enabled by default in ZFS and was not supported at the time by ReFS (it has since been added), so not enabling ZFS online deduplication yielded a more even comparison between the two file systems as ZFS then has a memory requirement of only a few hundred megabytes. == Reverse engineering and internals ==
Reverse engineering and internals
, Microsoft has not published any specifications for ReFS. An open source read-only FUSE / Linux kernel driver is under active development and a third-party open-source project to document ReFS is on GitHub. Paragon Software Group provides a closed-source driver for Windows and Linux. Some other Open Specifications mention ReFS specifics and allow for some information about the platform specific behaviour to be drawn from: • [MS-FSA Appendix A: Product Behavior] - Covers Windows specific platform limitations and implementation details for ReFS, NTFS, FAT, EXFAT, UDFS, CDFS • [MS-FSCC Appendix B: Product Behavior] - Describes various ReFS internals, e.g. how its 64-bit and 128-bit file ids are generated • Windows Internals, Part 2, 7th Edition - Microsoft has also released a book containing more internals (which is also sometimes referenced within some of the specifications using "[WININTERNALS]", there even though it was released in two parts it is referred to as a single thing with the chapter number continuing from the first part. The filesystem specific part containing information about ReFS is within part 2. ISBN Part 1: 978-0-7356-8418-8 and ISBN Part 2: 978-0-13-546240-9 == See also ==
tickerdossier.comtickerdossier.substack.com