Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Tags: Architectures Platforms default
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

Architectures Platforms default
Gregory Bartholomew
contrib: dracut: zfs-snapshot-bootfs: exit status fix

Signed-off-by: Gregory Bartholomew <gregory.lee.bartholomew@gmail.com>

Pull-request: #13769 part 1/1
Brian Behlendorf
Introduce a tunable to exclude special class buffers from L2ARC

Special allocation class or dedup vdevs may have roughly the same
performance as L2ARC vdevs. Introduce a new tunable to exclude those
buffers from being cacheable on L2ARC.

Signed-off-by: George Amanakis <gamanakis@gmail.com>

Pull-request: #13768 part 1/1
Roberto Ricci
arcstat: fix -p option

When the -p option is used, a list of floats is passed to sep.join(),
which expects strings. Fix this by converting each value to a string.

Closes #12916

Signed-off-by: Roberto Ricci <ricci@disroot.org>

Pull-request: #13767 part 1/1
András Korn
initramfs: allow user to supply their own unlock mechanism

Closes #13757.

The user can set `org.openzfs:zfs-initramfs:user-decrypt-function` to `true`
and supply a `decrypt_fs_user` function in `/etc/zfs/zfs-user-functions`,
which will then be called and is expected to load the decryption key. If it
succeeds, further prompts for the key are skipped.

Signed-off-by: András Korn <korn-github.com@elan.rulez.org>

Pull-request: #13762 part 1/1
Allan Jude
Allow mounting snapshots in .zfs/snapshot as a regular user

Rather than doing a terrible credential swapping hack, we just
check that the thing being mounted is a snapshot, and the mountpoint
is the zfsctl directory, then we allow it.

Signed-off-by: Allan Jude <allan@klarasystems.com>
Sponsored-by: Modirum MDPay
Sponsored-by: Klara Inc.

Pull-request: #13758 part 1/1
Allan Jude
Allow mounting snapshots in .zfs/snapshot as a regular user

Rather than doing a terrible credential swapping hack, we just
check that the thing being mounted is a snapshot, and the mountpoint
is the zfsctl directory, then we allow it.

Signed-off-by: Allan Jude <allan@klarasystems.com>
Sponsored-by: Modirum MDPay
Sponsored-by: Klara Inc.

Pull-request: #13758 part 1/1
Allan Jude
Actually call VFS_MOUNT() still

Signed-off-by: Allan Jude <allan@klarasystems.com>

Pull-request: #13758 part 2/2
Allan Jude
Allow mounting snapshots in .zfs/snapshot as a regular user

Rather than doing a terrible credential swapping hack, we just
check that the thing being mounted is a snapshot, and the mountpoint
is the zfsctl directory, then we allow it.

Signed-off-by: Allan Jude <allan@klarasystems.com>
Sponsored-by: Modirum MDPay
Sponsored-by: Klara Inc.

Pull-request: #13758 part 1/1
Ryan Moeller
FreeBSD: Organize sysctls

FreeBSD had a few platform-specific ARC tunables in the wrong place:

- Move FreeBSD-specifc ARC tunables into the same vfs.zfs.arc node as
  the rest of the ARC tunables.
- Move the handlers from arc_os.c to sysctl_os.c and add compat sysctls
  for the legacy names.

While here, some additional clean up:

- Most handlers are specific to a particular variable and don't need a
  pointer passed through the args.
- Group blocks of related variables, handlers, and sysctl declarations
  into logical sections.
- Match variable types for temporaries in handlers with the type of the
  global variable.
- Remove leftover comments.

Signed-off-by: Ryan Moeller <ryan@iXsystems.com>

Pull-request: #13756 part 2/2
Ryan Moeller
FreeBSD: Mark ZFS_MODULE_PARAM_CALL as MPSAFE

ZFS_MODULE_PARAM_CALL handlers implement their own locking if needed
and do not require Giant.

Signed-off-by: Ryan Moeller <ryan@iXsystems.com>

Pull-request: #13756 part 1/2
Paul Dagnelie
man page fix

Signed-off-by: Paul Dagnelie <pcd@delphix.com>

Pull-request: #13753 part 2/2
Paul Dagnelie
Prevent zevent list from consuming all of kernel memory

Signed-off-by: Paul Dagnelie <pcd@delphix.com>

Pull-request: #13753 part 1/2
Tino Reichardt
Add new SHA2 files

The C files are generic public domain files which are mostly faster the
the BSD ones - this depends on compiler and it's flags of cause ;)

The assembly files are taken from current openssl master, and are
licensed under the Apache License Version 2.0:
- sha256-x86_64.S: SSSE3, AVX, AVX2, SHA-NI
- sha512-x86_64.S: AVX, AVX2
- sha256-armv8.S:  NEON, ARMv8 Cryptography Extension
- sha512-armv8.S:  ARMv8 Cryptography Extension
- sha256-ppc.S:    generic PPC64 LE/BE
- sha512-ppc.S:    generic PPC64 LE/BE
- sha256-p8.S:    Power ISA Version 2.07 LE/BE
- sha512-p8.S:    Power ISA Version 2.07 LE/BE

Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de>

Pull-request: #13741 part 3/3
Tino Reichardt
Add SHA_NI instruction check to the SIMD headers

This commit adds the function zfs_shani_available() to the libspl,
os/freebsd and os/linux headers.

Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de>

Pull-request: #13741 part 2/3
Tino Reichardt
Fully remove old SHA2 files.

We had three sha2.h headers in different places. The BSD version
within the SPL layer, the generic solaris version within the ICP
module and some old openssl implementation for generic speedups on
the x86_64 architecture.

Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de>

Pull-request: #13741 part 1/3
Tino Reichardt
Add new SHA2 files

The C files are generic public domain files which are mostly faster the
the BSD ones - this depends on compiler and it's flags of cause ;)

The assembly files are taken from current openssl master, and are
licensed under the Apache License Version 2.0:
- sha256-x86_64.S: SSSE3, AVX, AVX2, SHA-NI
- sha512-x86_64.S: AVX, AVX2
- sha256-armv8.S:  NEON, ARMv8 Cryptography Extension
- sha512-armv8.S:  ARMv8 Cryptography Extension
- sha256-ppc.S:    generic PPC64 LE/BE
- sha512-ppc.S:    generic PPC64 LE/BE
- sha256-p8.S:    Power ISA Version 2.07 LE/BE
- sha512-p8.S:    Power ISA Version 2.07 LE/BE

Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de>

Pull-request: #13741 part 3/3
Tino Reichardt
Add SHA_NI instruction check to the SIMD headers

This commit adds the function zfs_shani_available() to the libspl,
os/freebsd and os/linux headers.

Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de>

Pull-request: #13741 part 2/3
Tino Reichardt
Fully remove old SHA2 files.

We had three sha2.h headers in different places. The BSD version
within the SPL layer, the generic solaris version within the ICP
module and some old openssl implementation for generic speedups on
the x86_64 architecture.

Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de>

Pull-request: #13741 part 1/3
Tony Hutter
zed: Fix config_sync autoexpand flood

Users were seeing floods of `config_sync` events when autoexpand was
enabled.  This happened because all "disk status change" udev events
invoke the autoexpand codepath, which calls zpool_relabel_disk(),
which in turn cause another "disk status change" event to happen,
in a feedback loop.  Note that "disk status change" happens every time
a user calls close() on a block device.

This commit breaks the feedback loop by only allowing an autoexpand
to happen if the disk actually changed size.

Fixes: #7132
Fixes: #7366

Signed-off-by: Tony Hutter <hutter2@llnl.gov>

Pull-request: #13729 part 1/1
George Amanakis
Fix zpool status in case of unloaded keys

When scrubbing an encrypted filesystem with unloaded key still report an
error in zpool status.

Signed-off-by: George Amanakis <gamanakis@gmail.com>

Pull-request: #13717 part 1/1
George Amanakis
Fix zpool status in case of unloaded keys

When scrubbing an encrypted filesystem with unloaded key still report an
error in zpool status.

Signed-off-by: George Amanakis <gamanakis@gmail.com>

Pull-request: #13717 part 1/1
George Amanakis
Fix zpool status in case of unloaded keys

When scrubbing an encrypted filesystem with unloaded key still report an
error in zpool status.

Signed-off-by: George Amanakis <gamanakis@gmail.com>

Pull-request: #13717 part 1/1
George Amanakis
Fix zpool status in case of unloaded keys

When scrubbing an encrypted filesystem with unloaded key still report an
error in zpool status.

Signed-off-by: George Amanakis <gamanakis@gmail.com>

Pull-request: #13717 part 1/1
George Amanakis
Fix zpool status in case of unloaded keys

When scrubbing an encrypted filesystem with unloaded key still report an
error in zpool status.

Signed-off-by: George Amanakis <gamanakis@gmail.com>

Pull-request: #13717 part 1/1
Ameer Hamza
Add zilstat script to report zil kstats in a user friendly manner

Added a python script to process both global and per dataset
zil kstats and report them in a user friendly manner similar
to arcstat and dbufstat.

Signed-off-by: Ameer Hamza <ahamza@ixsystems.com>

Pull-request: #13704 part 1/1
Ameer Hamza
Add zilstat script to report zil kstats in a user friendly manner

Added a python script to process both global and per dataset
zil kstats and report them in a user friendly manner similar
to arcstat and dbufstat.

Signed-off-by: Ameer Hamza <ahamza@ixsystems.com>

Pull-request: #13704 part 1/1
Finix
I could not know if HAVE_FILE_FADVISE is defined
or not in file_fadvise.c. Change test program to check
former data_size less or eaqul to latter one.

Signed-off-by: Finix Yan yancw@info2soft.com

Pull-request: #13694 part 10/10
GitHub
Merge branch 'openzfs:master' into fadvise

Pull-request: #13694 part 9/9
Finix
use (1) instead of 1 in return statement

Signed-off-by: Finix Yan yancw@info2soft.com

Pull-request: #13694 part 8/8
Finix
1. modify zpl_file.c to make style checker happy
2. file_fadvise only issue posix_fadvise if HAVE_FILE_FADVISE exist
3. fadvise test program only take effect if file_fadvise return 0

Signed-off-by: Finix Yan yancw@info2soft.com

Pull-request: #13694 part 7/7
Finix
1. Check whether generic_fadvise is exported or not
2. Call generic_fadvise in zpl_fadvise if it is available
3. Use dd instead of cat in filetest_002_pos.ksh to avoid prefetch
4. Use dd instead of cp in auto_spare_002_pos.ksh to avoid prefetch
5. Add fadvise test program

Signed-off-by: Finix Yan yancw@info2soft.com

Pull-request: #13694 part 6/6
George Melikov
cache_012_pos: make it less flaky

This test relies on atime writes to force the L2 ARC discards.
In addition, speed it up nearly twice with larger l2arc_write_max.

Signed-off-by: George Melikov <mail@gmelikov.ru>

Pull-request: #13614 part 2/2
George Melikov
Relatime: set on by default

Linux set's relatime on mount by default
for any file system,
but relatime=off in ZFS disables it explicitly.

Let's be on par with other file systems on Linux
(and it's our largest target OS too).

Signed-off-by: George Melikov <mail@gmelikov.ru>

Pull-request: #13614 part 1/2
Brian Atkinson
Updating based on PR feedback

Added VDEV_DIRECT_WR_VERIFY_CNT to tunables.cfg and set the module
parameter in the ZTS dio_writy_verify test using set_tunable32. I also
updated the name for the parameter name for FreeBSD to match the Linux
name.

Fixed ZTS dio_mmap test comments in the STRATEGY section to be correct.

Use DIO instead of DIO VERIFY as the column header to report checksum
validation errors, for Direct I/O writes, with zpool status -d. This
keeps the header to a single word for consistency with other reported
errors from zpool status. This also makes it much easier to parse
output and extract a whole DIO column.

Removed requirement for module parameter
zfs_vdev_direct_write_verify_cnt to only be checked for Linux. While
there should never be a case that FreeBSD has a checksum validation
error, because it can place user pages under write protection, it still
could be used as a debug option for FreeBSD. By default
zfs_vdev_direct_write_verify_cnt is set to 0 for FreeBSD and 100 for
Linux.

Minor cleanup of man pages, stray new lines, and code comments blocks.

Signed-off-by: Brian Atkinson <batkinson@lanl.gov>

Pull-request: #10018 part 3/3
Brian Atkinson
Applying OpenZFS Leadership Meeting feedback

Adding in suggestions from the OpenZFS Leadership meeting from June
2022.

The first suggestion was that instead of using a boolean module
parameter for determining when to perform a checksum verify with Direct
I/O writes, we instead switch over to a count based module parameter.
This at least allows for checksum verifies to take place every so often
without incuring the penalty for every write. I have updated the module
parameter to handle this to be zfs_vdev_direct_wite_verify_cnt. By
default, this is set to 100. This also only applies to Linux and at the
top-level VDEV. This is necessary because in Linux we can not place user
pages into write protection, because those pages can come in as
anonymous pages (AKA they are not backed by a file). For FreeBSD, this
is not necessary because user pages can be placed in write protection.
This module parameter determines the number of Direct I/O writes to a
top-level VDEV before a checksum verify should occur and before
commiting the write to the BP. If there is a checksum verify error then
EINVAL is returned. This can somewhat help correlate checksum failures
from the VDEVs back to Direct I/O write checksum verify errors.
These errors can be seen by doing zpool status -d.

As part of this, we also need to make sure that when an ABD is flagged
with ABD_FLAG_FROM_PAGES we actualy copy the contents of the pages when
calling abd_borrow_buf_copy(). This also requires abd_borrow_buf() to
always allocate a buffer to copy the pages into. Finally we need to also
take this into account for abd_return_buf_copy() and abd_return_buf().
This is only relevant to Linux as we can not guarantee the user has not
manipulated the contents of the pages. In order to handle this,
abd_buffor_buf_copy(), abd_borrow_buf(), abd_return_buf_copy(), and
abd_return_buf() were moved out of the general ABD code into OS specfic
ABD code. Normaly ABD's flagged with ABD_FLAG_FROM_PAGES always copied
their data because they are scatter ABDs, but we do allow for
LINEAR_PAGE ABDs when the size of ABD is less than a single page.

Signed-off-by: Brian Atkinson <batkinson@lanl.gov>

Pull-request: #10018 part 2/3
Brian Atkinson
WIP Direct IO

Adding O_DIRECT support to ZFS to bypass the ARC for writes/reads.

O_DIRECT support in ZFS will always ensure there is coherency between
buffered and O_DIRECT IO requests. This ensures that all IO requests,
whether buffered or direct, will see the same file contents at all
times. Just as in other FS's , O_DIRECT does not imply O_SYNC. While
data is written directly to VDEV disks, metadata will not be synced
until the associated  TXG is synced.
For both O_DIRECT read and write request the offset and requeset sizes,
at a minimum, must be PAGE_SIZE aligned. In the event they are not,
then EINVAL is returned unless the direct property is set to always (see
below).

For O_DIRECT writes:
The request also must be block aligned (recordsize) or the write
request will take the normal (buffered) write path. In the event that
request is block aligned and a cached copy of the buffer in the ARC,
then it will be discarded from the ARC forcing all further reads to
retrieve the data from disk.

For O_DIRECT reads:
The only alignment restrictions are PAGE_SIZE alignment. In the event
that the requested data is in buffered (in the ARC) it will just be
copied from the ARC into the user buffer.

To ensure data integrity for all data written using O_DIRECT, all user
pages are made stable in the event one of the following is required:
Checksum
Compression
Dedup
Encryption
Parity
By making the user pages stable, we make sure the contents of the user
provided buffer can not be changed after any of the above operations
have taken place.

A new dataset property `direct` has been added with the following 3
allowable values:
disabled - Accepts O_DIRECT flag, but silently ignores it and treats
  the request as a buffered IO request.
standard - Follows the alignment restrictions  outlined above for
  write/read IO requests when the O_DIRECT flag is used.
always  - Treats every write/read IO request as though it passed
          O_DIRECT and will do O_DIRECT if the alignment restrictions
  are met otherwise will redirect through the ARC. This
  property will not allow a request to fail.

Signed-off-by: Brian Atkinson <batkinson@lanl.gov>
Co-authored-by: Mark Maybee <mark.maybee@delphix.com>
Co-authored-by: Matt Macy <mmacy@FreeBSD.org>
Co-authored-by: Brian Behlendorf <behlendorf@llnl.gov>

Pull-request: #10018 part 1/3
Brian Atkinson
Updating based on PR feedback

Added VDEV_DIRECT_WR_VERIFY_CNT to tunables.cfg and set the module
parameter in the ZTS dio_writy_verify test using set_tunable32. I also
updated the name for the parameter name for FreeBSD to match the Linux
name.

Fixed ZTS dio_mmap test comments in the STRATEGY section to be correct.

Signed-off-by: Brian Atkinson <batkinson@lanl.gov>

Pull-request: #10018 part 3/3
  • Kernel.org Built-in x86_64 (BUILD): cloning linux -  stdio
Brian Atkinson
Applying OpenZFS Leadership Meeting feedback

Adding in suggestions from the OpenZFS Leadership meeting from June
2022.

The first suggestion was that instead of using a boolean module
parameter for determining when to perform a checksum verify with Direct
I/O writes, we instead switch over to a count based module parameter.
This at least allows for checksum verifies to take place every so often
without incuring the penalty for every write. I have updated the module
parameter to handle this to be zfs_vdev_direct_wite_verify_cnt. By
default, this is set to 100. This also only applies to Linux and at the
top-level VDEV. This is necessary because in Linux we can not place user
pages into write protection, because those pages can come in as
anonymous pages (AKA they are not backed by a file). For FreeBSD, this
is not necessary because user pages can be placed in write protection.
This module parameter determines the number of Direct I/O writes to a
top-level VDEV before a checksum verify should occur and before
commiting the write to the BP. If there is a checksum verify error then
EINVAL is returned. This can somewhat help correlate checksum failures
from the VDEVs back to Direct I/O write checksum verify errors.
These errors can be seen by doing zpool status -d.

As part of this, we also need to make sure that when an ABD is flagged
with ABD_FLAG_FROM_PAGES we actualy copy the contents of the pages when
calling abd_borrow_buf_copy(). This also requires abd_borrow_buf() to
always allocate a buffer to copy the pages into. Finally we need to also
take this into account for abd_return_buf_copy() and abd_return_buf().
This is only relevant to Linux as we can not guarantee the user has not
manipulated the contents of the pages. In order to handle this,
abd_buffor_buf_copy(), abd_borrow_buf(), abd_return_buf_copy(), and
abd_return_buf() were moved out of the general ABD code into OS specfic
ABD code. Normaly ABD's flagged with ABD_FLAG_FROM_PAGES always copied
their data because they are scatter ABDs, but we do allow for
LINEAR_PAGE ABDs when the size of ABD is less than a single page.

Signed-off-by: Brian Atkinson <batkinson@lanl.gov>

Pull-request: #10018 part 2/3
Brian Atkinson
WIP Direct IO

Adding O_DIRECT support to ZFS to bypass the ARC for writes/reads.

O_DIRECT support in ZFS will always ensure there is coherency between
buffered and O_DIRECT IO requests. This ensures that all IO requests,
whether buffered or direct, will see the same file contents at all
times. Just as in other FS's , O_DIRECT does not imply O_SYNC. While
data is written directly to VDEV disks, metadata will not be synced
until the associated  TXG is synced.
For both O_DIRECT read and write request the offset and requeset sizes,
at a minimum, must be PAGE_SIZE aligned. In the event they are not,
then EINVAL is returned unless the direct property is set to always (see
below).

For O_DIRECT writes:
The request also must be block aligned (recordsize) or the write
request will take the normal (buffered) write path. In the event that
request is block aligned and a cached copy of the buffer in the ARC,
then it will be discarded from the ARC forcing all further reads to
retrieve the data from disk.

For O_DIRECT reads:
The only alignment restrictions are PAGE_SIZE alignment. In the event
that the requested data is in buffered (in the ARC) it will just be
copied from the ARC into the user buffer.

To ensure data integrity for all data written using O_DIRECT, all user
pages are made stable in the event one of the following is required:
Checksum
Compression
Dedup
Encryption
Parity
By making the user pages stable, we make sure the contents of the user
provided buffer can not be changed after any of the above operations
have taken place.

A new dataset property `direct` has been added with the following 3
allowable values:
disabled - Accepts O_DIRECT flag, but silently ignores it and treats
  the request as a buffered IO request.
standard - Follows the alignment restrictions  outlined above for
  write/read IO requests when the O_DIRECT flag is used.
always  - Treats every write/read IO request as though it passed
          O_DIRECT and will do O_DIRECT if the alignment restrictions
  are met otherwise will redirect through the ARC. This
  property will not allow a request to fail.

Signed-off-by: Brian Atkinson <batkinson@lanl.gov>
Co-authored-by: Mark Maybee <mark.maybee@delphix.com>
Co-authored-by: Matt Macy <mmacy@FreeBSD.org>
Co-authored-by: Brian Behlendorf <behlendorf@llnl.gov>

Pull-request: #10018 part 1/3