Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Tags: Architectures Platforms default
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

Architectures Platforms default
Daniel Reichelt
initramfs: use correct dataset for rootfs on rollback=1

fixes #12585

Pull-request: #12586 part 1/1
Kevin Bowling
Detect HAVE_LARGE_STACKS at compile time

Move HAVE_LARGE_STACKS definitions to header and set when appropriate.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Allan Jude <allan@klarasystems.com>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Kevin Bowling <kbowling@FreeBSD.org>
Closes #12350

Pull-request: #12584 part 1/1
Ryan Moeller
ZTS: Eliminate udev_wait in zvol_misc tests

The zvol_misc tests, in particular zvol_misc_volmode, make use of a
common udev_wait function to wait for zvol devices in /dev to quiesce
on Linux.  On other platforms this function currently only sleeps for
one second before returning.  This is insufficient, and
zvol_misc_volmode has been flaky on FreeBSD as a result.

Replace udev_wait with block_device_wait, passing through the optional
device parameter where possible.  Rearrange a few checks to strengthen
the verifications we are making and avoid unnecessarily sleeping.
Remove zvol_misc_volmode from the maybe failing tests on FreeBSD in
zts-report.py.

Signed-off-by: Ryan Moeller <ryan@iXsystems.com>

Pull-request: #12583 part 1/1
Ryan Moeller
ZTS: Improve udev_wait in zvol_misc tests

The zvol_misc tests, in particular zvol_misc_volmode, make use of a
common udev_wait function to wait for zvol devices in /dev to quiesce on
Linux.  On other platforms this function currently only sleeps for one
second before returning.  This is insufficient, and zvol_misc_volmode
has been flaky on FreeBSD as a result.

Make udev_wait invoke block_device_wait on platforms other than Linux
rather than simply sleeping.  Pass through an optional device parameter.
Rearrange a few checks to strengthen the verifications we are making and
avoid unnecessarily sleeping.  Remove zvol_misc_volmode from the maybe
failing tests on FreeBSD in zts-report.py.

Signed-off-by: Ryan Moeller <ryan@iXsystems.com>

Pull-request: #12583 part 1/1
√Črico Nogueira
libspl: fix warning about missing spl_pagesize declaration

Though it's unlikely anyone will alter its signature, avoids any
possible type mismatch.

Signed-off-by: √Črico Nogueira <erico.erc@gmail.com>

Pull-request: #12567 part 1/1
Rich Ercolani
Add the ability to uninitialize

zpool initialize functions well for touching every free byte...once.
But if we want to do it again, we're currently out of luck.

So let's add zpool initialize -u to clear it.

Signed-off-by: Rich Ercolani <rincebrain@gmail.com>

Pull-request: #12451 part 1/1
Rich Ercolani
ZTS: Enable punch-hole tests on FreeBSD

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Ka Ho Ng <khng@FreeBSD.org>
Sponsored-by: The FreeBSD Foundation
Closes #12458

Pull-request: #12451 part 30/1
Rich Ercolani
FreeBSD: Implement hole-punching support

This adds supports for hole-punching facilities in the FreeBSD kernel
starting from __FreeBSD_version 1400032.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Ka Ho Ng <khng@FreeBSD.org>
Sponsored-by: The FreeBSD Foundation
Closes #12458

Pull-request: #12451 part 29/1
Rich Ercolani
ZTS: Waiting for zvols to be available

The ZTS block_device_wait helper function should use -e when waiting
for a file to appear since it will be either a block special device
or a symlink.  This didn't cause any failures but when a device path
was specified the function would wait longer than needed.

Additionally update the most flakey test cases to pass the file path
to block_device_wait to try and improve the test reliability.  The
udev behavior on Fedora in particular can result in frequent false
positives.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #12515

Pull-request: #12451 part 28/1
Rich Ercolani
Correct checking bdev_check_media_change message

We're not looking for bdev_disk_changed.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Closes #12492

Pull-request: #12451 part 27/1
Rich Ercolani
Make 'zpool labelclear -f' work on offlined disks

This patch allows you to clear the label on offlined disks in an active
pool with `-f`.  Previously, labelclear wouldn't let you do that.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Signed-off-by: Tony Hutter <hutter2@llnl.gov>
Closes #12511

Pull-request: #12451 part 26/1
Rich Ercolani
Extend zpool-iostat to account for ZIO_PRIORITY_REBUILD (#12319)

Previously, zpool-iostat did not display any data regarding rebuild I/Os
in either the latency/size histograms (-w/-l/-r) or the queue data (-q).
This fix essentially utilizes the existing infrastructure for tracking
rebuild queue data and displays this data in the proper places within
zpool-iostat's output.

Signed-off-by: Trevor Bautista <tbautista@newmexicoconsortium.org>
Signed-off-by: Trevor Bautista <tbautista@lanl.gov>
Co-authored-by: Trevor Bautista <tbautista@newmexicoconsortium.org>
Reviewed-by: Richard Elling <Richard.Elling@RichardElling.com>
Reviewed-by: Tony Hutter <hutter2@llnl.gov>

Pull-request: #12451 part 25/1
Rich Ercolani
vdev_id: Return an error if config file is not found (#12508)

Signed-off-by: Anton Gubarkov <anton.gubarkov@gmail.com>
Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>

Pull-request: #12451 part 24/1
Rich Ercolani
zpool-remove.8: describe top-level vdev sector size limitation

Document that top-level vdevs cannot be removed unless all top-level
vdevs have the same sector size.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Sam Hathaway <sam@sam-hathaway.com>
Closes #11339
Closes #12472

Pull-request: #12451 part 23/1
Rich Ercolani
Initialize parity blocks before RAID-Z reconstruction benchmarking

benchmark_raidz() allocates a row to benchmark parity calculation and
reconstruction.  In the latter case, the parity blocks are left
uninitialized, leading to reports from KMSAN.

Initialize parity blocks to 0xAA as we do for the data earlier in the
function.  This does not affect the selected RAID-Z implementation on
any of several systems tested.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Mark Johnston <markj@FreeBSD.org>
Closes #12473

Pull-request: #12451 part 22/1
Rich Ercolani
ZTS: Add tests for creation time

Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Reviewed-by: Allan Jude <allan@klarasystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Closes #12432

Pull-request: #12451 part 21/1
Rich Ercolani
Linux 4.11 compat: statx support

Linux 4.11 added a new statx system call that allows us to expose crtime
as btime. We do this by caching crtime in the znode to match how atime,
ctime and mtime are cached in the inode.

statx also introduced a new way of reporting whether the immutable,
append and nodump bits have been set. It adds support for reporting
compression and encryption, but the semantics on other filesystems is
not just to report compression/encryption, but to allow it to be turned
on/off at the file level. We do not support that.

We could implement semantics where we refuse to allow user modification
of the bit, but we would need to do a dnode_hold() in zfs_znode_alloc()
to find out encryption/compression information. That would introduce
locking that will have a minor (although unmeasured) performance cost.
It also would be inferior to zdb, which reports far more detailed
information. We therefore omit reporting of encryption/compression
through statx in favor of recommending that users interested in such
information use zdb.

Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Reviewed-by: Allan Jude <allan@klarasystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Richard Yao <ryao@gentoo.org>
Closes #8507

Pull-request: #12451 part 20/1
Rich Ercolani
zfs.4: Fix typo s/compatiblity/compatibility/

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Signed-off-by: Gordon Bergling <gbergling@googlemail.com>
Closes #12464

Pull-request: #12451 part 19/1
Rich Ercolani
Remove b_pabd/b_rabd allocation from arc_hdr_alloc()

When a header is allocated for full overwrite it is a waste of time
to allocate b_pabd/b_rabd for it, since arc_write() will free them
without ever being touched.  If it is a read or a partial overwrite
then arc_read() and arc_hdr_decrypt() allocate them explicitly.

Reduced memory allocation in user threads also reduces ARC eviction
throttling there, proportionally increasing it in ZIO threads, that
is not good.  To minimize or even avoid it introduce ARC allocation
reserve, allowing certain arc_get_data_abd() callers to allocate a
bit longer in situations where user threads will already throttle.

Reviewed-by: George Wilson <gwilson@delphix.com>
Reviewed-by: Mark Maybee <mark.maybee@delphix.com>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Sponsored-By: iXsystems, Inc.
Closes #12398

Pull-request: #12451 part 18/1
Rich Ercolani
Increase default volblocksize from 8KB to 16KB

Many things has changed since previous default was set many years ago.
Nowadays 8KB does not allow adequate compression or even decent space
efficiency on many of pools due to 4KB disk physical block rounding,
especially on RAIDZ and DRAID.  It effectively limits write throughput
to only 2-3GB/s (250-350K blocks/s) due to sync thread, allocation,
vdev queue and other block rate bottlenecks.  It keeps L2ARC expensive
despite many optimizations and dedup just unrealistic.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Closes #12406

Pull-request: #12451 part 17/1
Rich Ercolani
Optimize arc_l2c_only lists assertions

It is very expensive and not informative to call multilist_is_empty()
for each arc_change_state() on debug builds to check for impossible.
Instead implement special index function for arc_l2c_only->arcs_list,
multilists, panicking on any attempt to use it.

Reviewed-by: Mark Maybee <mark.maybee@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Sponsored-By: iXsystems, Inc.
Closes #12421

Pull-request: #12451 part 16/1
Rich Ercolani
Fix/improve dbuf hits accounting

Instead of clearing stats inside arc_buf_alloc_impl() do it inside
arc_hdr_alloc() and arc_release().  It fixes statistics being wiped
every time a new dbuf is filled from the ARC.

Remove b_l1hdr.b_l2_hits. L2ARC hits are accounted at b_l2hdr.b_hits.
Since the hits are accounted under hash lock, replace atomics with
simple increments.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Wilson <george.wilson@delphix.com>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Sponsored-By: iXsystems, Inc.
Closes #12422

Pull-request: #12451 part 15/1
Rich Ercolani
Avoid vq_lock drop in vdev_queue_aggregate()

vq_lock is already too congested for two more operations per I/O.
Instead of dropping and reacquiring it inside vdev_queue_aggregate()
delegate the zio_vdev_io_bypass() and zio_execute() calls for parent
I/Os to callers, that drop the lock any way to execute the new I/O.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Mark Maybee <mark.maybee@delphix.com>
Reviewed-by: Brian Atkinson <batkinson@lanl.gov>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Sponsored-By: iXsystems, Inc.
Closes #12297

Pull-request: #12451 part 14/1
Rich Ercolani
Use more atomics in refcounts

Use atomic_load_64() for zfs_refcount_count() to prevent torn reads
on 32-bit platforms.  On 64-bit ones it should not change anything.

When built with ZFS_DEBUG but running without tracking enabled use
atomics instead of mutexes same as for builds without ZFS_DEBUG.
Since rc_tracked can't change live we can check it without lock.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Sponsored-By: iXsystems, Inc.
Closes #12420

Pull-request: #12451 part 13/1
Rich Ercolani
ZTS: Avoid unset $tmpdir in redacted_panic

The redacted_send tests make use of a $tmpdir variable, except in
redacted_send/redacted_panic the variable is never defined.

Use $TEST_BASE_DIR instead.

Clean up the stream file after the test.

Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Closes #12455

Pull-request: #12451 part 12/1
Rich Ercolani
Restore FreeBSD sysctl processing for arc.min and arc.max

Before OpenZFS 2.0, trying to set the FreeBSD sysctl vfs.zfs.arc_max
to a disallowed value would return an error.
Since the switch, it instead only generates WARN_IF_TUNING_IGNORED

Keep the ability to set the sysctl's specifically to 0, even though
that is less than the minimum, because some tests depend on this.

Also lost, was the ability to set vfs.zfs.arc_max to a value less
than the default vfs.zfs.arc_min at boot time. Restore this as well.

Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Signed-off-by: Allan Jude <allan@klarasystems.com>
Closes #12161

Pull-request: #12451 part 11/1
Rich Ercolani
zfs: add missed dependency of zfs module on zlib

Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Martin Matuska <mm@FreeBSD.org>
Co-authored-by: Konstantin Belousov <kib@FreeBSD.org>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
External-issue: https://reviews.freebsd.org/D31207
Closes #12442

Pull-request: #12451 part 10/1
Rich Ercolani
Add zfs.sh -r flag to reload modules

zfs.sh already can load and unload, so why not both?

This is convenient when developing changes to the module and you want
to rapidly make some changes, rebuild the module, reload the module,
and test the changes.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Closes #12450

Pull-request: #12451 part 9/1
Rich Ercolani
Fix usage of find in tests/Makefile.am

The path is not optional on FreeBSD.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Closes #12453

Pull-request: #12451 part 8/1
Rich Ercolani
Run arc_evict thread at higher priority

Run arc_evict thread at higher priority, nice=0, to give it more CPU
time which can improve performance for workload with high ARC evict
activities.

On mixed read/write and sequential read workloads, I've seen between
10-40% better performance.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Signed-off-by: Tony Nguyen <tony.nguyen@delphix.com>
Closes #12397

Pull-request: #12451 part 7/1
Rich Ercolani
Make get_key_material_file fail more verbosely

It turns out, there are a lot of possible reasons for fopen to fail.
Let's share which reason we failed for today.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes #12410

Pull-request: #12451 part 6/1
Rich Ercolani
Enable /proc/diskstats for zvols

The /proc/diskstats accounting needs to be explicitly enabled
for block devices which do not use multi-queue.

Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #12440
Closes #12066

Pull-request: #12451 part 5/1
Rich Ercolani
Man zpool-scrub.8: describe sequential scrub

Describe sequential scrub and add examples of scrub status.

Reviewed-by: Richard Laager <rlaager@wiktel.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Signed-off-by: George Melikov <mail@gmelikov.ru>
Closes #12429

Pull-request: #12451 part 4/1
  • Debian 8 ppc64 (BUILD): cloning zfs -  stdio
Aleksa Sarai
tests: add tests for renameat2(2) flags

Since mv(1) doesn't expose the RENAME_* flags we need to have our own
variation in the tests/ tree. The tests are fairly obvious functional
tests, though in the future (once we solve the d_revalidate issue) we
might also add a full-stack overlayfs integration test.

Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>

Pull-request: #12209 part 5/5
Aleksa Sarai
tests: functional: switch to rsync for directory diffs

While "diff -r" is the most straightforward way of comparing directory
trees for differences, it has two major issues:

* File metadata is not compared, which means that subtle bugs may be
  missed even if a test is written that exercises the buggy behaviour.
* diff(1) doesn't know how to compare special files -- it assumes they
  are always different, which means that a test using diff(1) on
  special files will always fail (resulting in such tests not being
  added).

rsync can be used in a very similar manner to diff (with the -ni flags),
but has the additional benefit of being able to detect and resolve many
more differences between directory trees. In addition, rsync has a
standard set of features and flags while diffs feature set depends on
whether you're using GNU or BSD binutils.

Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>

Pull-request: #12209 part 4/5
Aleksa Sarai
renameat2: add EL7 compatibility handling

EL7 has backported renameat2(2) support, but this was done by creating a
wrapper struct around the inode_operations struct and adding the rename2
callback to the wrapping struct. The semantics are the same as the
upstream callback, so just detect and use the wrapper.

Since we only need this wrapper for rename2 (and renaming is a directory
operation) we only use the wrapper for the directory iops struct.

Signed-off-by: Pavel Snajdr <snajpa@snajpa.net>
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>

Pull-request: #12209 part 3/5
Aleksa Sarai
zfs_rename: support RENAME_* flags

Implement support for Linux's RENAME_* flags (for renameat2). Aside from
being quite useful for userspace (providing race-free ways to exchange
paths and implement mv --no-clobber), they are used by overlayfs and are
thus required in order to use overlayfs-on-ZFS.

In order for us to represent the new renameat2(2) flags in the ZIL, we
emulate the renameat2(2) operations using several transaction entries.
This allows us to avoid adding new ZIL types for something which is
Linux-specific -- and the operations are safe because of the context in
which a ZIL replay will be executed. RENAME_EXCHANGE requires a bit more
work because we need to create a temporary file with a random name. See
the comment in module/zfs/zfs_log.c for more details.

This feature was worked on by both Aleksa and Pavel. Aleksa did the
initial implementation, Pavel reworked the ZIL handling to not require
new ZIL types and various other improvements, and then Aleksa did the
final rebase and carry.

Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
Signed-off-by: Pavel Snajdr <snajpa@snajpa.net>
Co-authored-by: Pavel Snajdr <snajpa@snajpa.net>
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>

Pull-request: #12209 part 2/5
Aleksa Sarai
zfs_rename: restructure to have cleaner fallbacks

This is in preparation for RENAME_EXCHANGE and RENAME_WHITEOUT support
for ZoL, but the changes here allow for far nicer fallbacks than the
previous implementation (the source and target are re-linked in case of
the final link failing).

In addition, a small cleanup was done for the "target exists but is a
different type" codepath so that it's more understandable.

Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>

Pull-request: #12209 part 1/5
Jitendra Patidar
ZFS fix to make xattr=sa logging to ZIL on xattr create/remove/update.

As such, there are no specific synchronous semantics defined for
the xattrs. But for xattr=on, it does log to ZIL and zil_commit() is
done, if sync=always is set on dataset. This provides sync semantics
for xattr=on with sync=always set on dataset.
For the xattr=sa implementation, it doesn't log to ZIL, so, even with
sync=always, xattrs are not guaranteed to be synced before xattr call
returns to caller. so xattr can be lost if system crash happens, before
txg carrying xattr transaction is synced.

This change makes xattr=sa logging to ZIL on xattr create/remove/update
and xattrs are synced to ZIL (zil_commit() done) for sync=always.
This make xattr=sa behavior similar to xattr=on.

This could also provide basic framework to support implementing sync
semantics at file level for xattr=sa.

Signed-off-by: Jitendra Patidar <jitendra.patidar@nutanix.com>
Closes #8768

Pull-request: #9078 part 1/1
Jitendra Patidar
ZFS fix to make xattr=sa logging to ZIL on xattr create/remove/update.

As such, there are no specific synchronous semantics defined for
the xattrs. But for xattr=on, it does log to ZIL and zil_commit() is
done, if sync=always is set on dataset. This provides sync semantics
for xattr=on with sync=always set on dataset.
For the xattr=sa implementation, it doesn't log to ZIL, so, even with
sync=always, xattrs are not guaranteed to be synced before xattr call
returns to caller. so xattr can be lost if system crash happens, before
txg carrying xattr transaction is synced.

This change makes xattr=sa logging to ZIL on xattr create/remove/update
and xattrs are synced to ZIL (zil_commit() done) for sync=always.
This make xattr=sa behavior similar to xattr=on.

This could also provide basic framework to support implementing sync
semantics at file level for xattr=sa.

Signed-off-by: Jitendra Patidar <jitendra.patidar@nutanix.com>
Closes #8768

Pull-request: #9078 part 1/1