-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 ============================================================================= FreeBSD-EN-23:16.openzfs Errata Notice The FreeBSD Project Topic: OpenZFS data corruption Category: contrib Module: OpenZFS Announced: 2023-12-01 Affects: All supported versions of FreeBSD. Corrected: 2023-11-28 21:00:48 UTC (stable/14, 14.0-STABLE) 2023-12-01 00:38:38 UTC (releng/14.0, 14.0-RELEASE-p1) 2023-11-28 21:07:30 UTC (stable/13, 13.2-STABLE) 2023-12-01 00:38:47 UTC (releng/13.2, 13.2-RELEASE-p6) 2023-11-30 05:28:33 UTC (stable/12, 12.4-STABLE) 2023-12-01 00:40:23 UTC (releng/12.4, 12.4-RELEASE-p8) For general information regarding FreeBSD Errata Notices and Security Advisories, including descriptions of the fields above, security branches, and the following sections, please visit . I. Background FreeBSD has included a version of the powerful and feature-rich ZFS file system beginning with FreeBSD 7.0 released in 2008. The ZFS implementation in FreeBSD 12 and earlier is based on the Illumos ZFS codebase. In FreeBSD 13 and later OpenZFS is used as the ZFS implementation. Sparse files in a file system refer to a technique that optimizes storage space by allowing the creation of files with unallocated or unwritten gaps, known as holes. When reading a file, holes appear as zero or NUL bytes. Certain system calls can access hole location metadata, including lseek(2) with SEEK_HOLE and copy_file_range(2). In OpenZFS a dnode is a data structure used to represent and manage metadata about files and directories. In file systems, "dirty" refers to data or metadata that has been modified in memory but not yet written to the storage device. Thus, a dirty dnode is one which has uncommitted data or metadata. In FreeBSD 13.2 and FreeBSD 14.0 cp(1) uses copy_file_range(2) to perform the data copying in the kernel. copy_file_range attempts to find file holes in the source file and preserve them in the copy. In FreeBSD 12.4 cp does not use copy_file_range. II. Problem Description A check did not test both the dnode itself and its data for dirtiness. This provides a very small window of time while a file is being modified where the dirtiness check can falsely report that the dnode is clean. If this happens a hole may incorrectly be reported where data was written. III. Impact If an access occurs while a file is being modified and a hole is incorrectly reported, the data may instead be interpreted as zero bytes. Any application which checks for holes may be affected by this issue; if this occurs during a file copy it will result in a corrupt copy that retains the incorrect data. Note that the source file remains intact (a subsequent read will return the correct data). IV. Workaround Setting the vfs.zfs.dmu_offset_next_sync sysctl to 0 disables forcing TXG sync to find holes. This is an effective workaround that greatly reduces the likelihood of encountering data corruption, although it does not completely eliminate it. Note that with the workaround holes will not be reported in recently dirtied files. See the zfs(4) man page for more information of the impact of this sysctl setting. The workaround should be removed once the system is updated to include the fix described in this notice. V. Solution Upgrade your system to a supported FreeBSD stable or release / security branch (releng) dated after the correction date, and reboot. Perform one of the following: 1) To update your system via a binary patch: Systems running a RELEASE version of FreeBSD on the amd64 or arm64 platforms, or the i386 platfrom on FreeBSD 13 and earlier, can be updated via the freebsd-update(8) utility: # freebsd-update fetch # freebsd-update install # shutdown -r +10min "Rebooting to apply OpenZFS erratum update" 2) To update your system via a source code patch: The following patches have been verified to apply to the applicable FreeBSD release branches. a) Download the relevant patch from the location below, and verify the detached PGP signature using your PGP utility. NOTE: The FreeBSD 14.0 patch includes additional bug fixes which were found during the investigation of this issue. These bug fixes do not apply to FreeBSD 13.2 or FreeBSD 12.4. [FreeBSD 14.0] # fetch https://security.FreeBSD.org/patches/EN-23:16/openzfs.14.patch # fetch https://security.FreeBSD.org/patches/EN-23:16/openzfs.14.patch.asc # gpg --verify openzfs.14.patch.asc [FreeBSD 13.2] # fetch https://security.FreeBSD.org/patches/EN-23:16/openzfs.13.patch # fetch https://security.FreeBSD.org/patches/EN-23:16/openzfs.13.patch.asc # gpg --verify openzfs.13.patch.asc [FreeBSD 12.4] # fetch https://security.FreeBSD.org/patches/EN-23:16/openzfs.12.patch # fetch https://security.FreeBSD.org/patches/EN-23:16/openzfs.12.patch.asc # gpg --verify openzfs.12.patch.asc b) Apply the patch. Execute the following commands as root: # cd /usr/src # patch < /path/to/patch c) Recompile your kernel as described in and reboot the system. VI. Correction details This issue is corrected as of the corresponding Git commit hash or Subversion revision number in the following stable and release branches: Branch/path Hash Revision - ------------------------------------------------------------------------- stable/14/ 99385ec7c296 stable/14-n265836 releng/14.0/ 154870526943 releng/14.0-n265384 stable/13/ 5858f93a8b66 stable/13-n256744 releng/13.2/ 0bb76997ce58 releng/13.2-n254644 stable/12/ r373278 releng/12.4/ r373279 - ------------------------------------------------------------------------- For FreeBSD 13 and later: Run the following command to see which files were modified by a particular commit: # git show --stat Or visit the following URL, replacing NNNNNN with the hash: To determine the commit count in a working tree (for comparison against nNNNNNN in the table above), run: # git rev-list --count --first-parent HEAD For FreeBSD 12 and earlier: Run the following command to see which files were modified by a particular revision, replacing NNNNNN with the revision number: # svn diff -cNNNNNN --summarize svn://svn.freebsd.org/base Or visit the following URL, replacing NNNNNN with the revision number: VII. References The latest revision of this advisory is available at -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEthUnfoEIffdcgYM7bljekB8AGu8FAmVpPo4ACgkQbljekB8A Gu/7rg/8DV0CgrVWVW8lvywaBry/oFOAcB1s+b49fcW1wt4g4GOnFtU0VGuRYXJh 2pT2xnCVKgWKWciaFAoFN/N29GOxCuMkcPNoYPf8laiBNAmYTGGBMK6FI4YukI2V 6GKSU8hYPgxwRSRW7ZSXfzWl2MuLI2NdrRZwY+L/2cgr/uJVq/u7b1s7y7A9CdbQ 0euotytR77yrSHecA7Ye5PVRFp1behuiK9kbIVUTdFJRB0eQkpap5e3Af9b7GeLe t3kFI5cHKim7PnquLpljxjRxwcWKeJBMf0a8X6nhXYJ7FHxh6YfRL1t4tPQIRHLq 5A4x9oDoZP5kPRQgdxYT4J/VuoCEsq9/D83DwLK6fMY9qcY/TYrp1rOnYKwBQDUj FMIbaipxss/j8KWEyAwc3dIwJBFCW40yRFR2cg7SCeZ0UJzZEkuDOaIvzkWIGtc3 AqW0R+lvAQ2f+ObbP7iQCGj4HrCgIlPUCDX2SckNuAwaXQIdu5GO+HDjuKb49sw3 8zimt4dAT+OuvZxXDacIhIz53LCJHD/cAyF2CqTdNYpwne892drfiK4FQZ1Jq75Q 4nRedE8YLD2ZwuUALqR1PqHJQKra5hlAhAoITHuTpBG1fggSx6dyj6kSkMR8p6Mb tADR8onFzUHZgOlkEOjddKaVqAP3z4jW+lfrlk7J/9j5jgRrtLM= =pM+u -----END PGP SIGNATURE-----