Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Tags: Architectures Distributions Performance Style Tests
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

Architectures Distributions Performance Style Tests
Brian Behlendorf
ZTS: Update project quota tests

e2fsprogs v1.44.1, which provides lsattr, added a new attibute
for ext3 called "verity".  It is reported after the project quota
flag as a 'V' character in the `lsattr` output.

Update projectid_001_pos.ksh and projecttree_001_pos.ksh to use
a pattern which will match the expected output in both cases.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>

Pull-request: #8043 part 1/1
Brian Behlendorf
OpenZFS 9688 - aggsum_fini leaks memory

Porting Notes:
- Most of these fixes were applied in the original 37fb3e43
  commit when this change was ported for Linux.

Authored by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Reviewed by: Jorgen Lundman <lundman@lundman.net>
Reviewed by: Igor Kozhukhov <igor@dilos.org>
Approved by: Robert Mustacchi <rm@joyent.com>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>

OpenZFS-issue: https://www.illumos.org/issues/9688
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/29bf2d68be

Pull-request: #8042 part 1/1
Brian Behlendorf
OpenZFS 9681 - ztest failure in spa_history_log_internal due to spa_rename()

Authored by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>

OpenZFS-issue: https://www.illumos.org/issues/9681
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/6aee0ad7

Pull-request: #8041 part 1/1
Brian Behlendorf
OpenZFS 9690 - metaslab of vdev with no space maps was flushed during removal

Authored by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>

OpenZFS-issue: https://www.illumos.org/issues/9690
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/4e75ba6826

Pull-request: #8039 part 1/1
Allan Jude
OpenZFS 9862 - fix typo in comment in vdev_impl.h

Authored by: Allan Jude <allanjude@freebsd.org>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Tony Hutter <hutter2@llnl.gov>
Approved by: Robert Mustacchi <rm@joyent.com>
Ported-by: George Melikov <mail@gmelikov.ru>

OpenZFS-issue: https://www.illumos.org/issues/9862
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/84927f52
Closes #8036
George Melikov
OpenZFS 9682 - page fault in dsl_async_clone_destroy() while opening pool

Authored by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Sara Hartse <sara.hartse@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Ported-by: George Melikov <mail@gmelikov.ru>

OpenZFS-issue: https://www.illumos.org/issues/9682
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/ade2c82828
Signed-off-by: George Melikov <mail@gmelikov.ru>

Pull-request: #8037 part 1/1
George Melikov
OpenZFS 9862 - fix typo in comment in vdev_impl.h

Authored by: Allan Jude <allanjude@freebsd.org>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Approved by: Robert Mustacchi <rm@joyent.com>
Ported-by: George Melikov <mail@gmelikov.ru>

OpenZFS-issue: https://www.illumos.org/issues/9862
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/84927f52bd837f6e4882a19e43fd026f1828d910

Signed-off-by: George Melikov <mail@gmelikov.ru>

Pull-request: #8036 part 1/1
Sebastian Gottschall
return default compression level to avoid ztest fuzzing

Signed-off-by: Sebastian Gottschall <s.gottschall@dd-wrt.com>

Pull-request: #8028 part 12/12
Sebastian Gottschall
mask out zstd levels from compression

otherwise ztest fuzzing will fail

Signed-off-by: Sebastian Gottschall <s.gottschall@dd-wrt.com>

Pull-request: #8028 part 11/12
Sebastian Gottschall
Merge branch 'master' of https://github.com/zfsonlinux/zfs

Signed-off-by: Sebastian Gottschall <s.gottschall@dd-wrt.com>

Pull-request: #8028 part 10/12
Sebastian Gottschall
Support zstd compression

Signed-off-by: Sebastian Gottschall <s.gottschall@dd-wrt.com>

Pull-request: #8028 part 9/12
Sebastian Gottschall
Allow copy-builtin to work with modified sources

`scripts/make_gitrev.sh` had 'set -e' so if any command failed it would
fail and cause copy-builtin to fail (copy-builtin also has `set -e`.
This commit also simplifies scripts/make_gitrev.sh to always write a
file by using a cleanup function.  It also simplifies other areas of
the script as well (making it much shorter).

Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matthew Thode <mthode@mthode.org>
Closes #8022
Closes #8025

Pull-request: #8028 part 8/12
Sebastian Gottschall
zpool: allow sharing of spare device among pools

ZFS allows, by default, sharing of spare devices among different pools;
this commit simply restores this functionality for disk devices and
adds an additional tests case to the ZFS Test Suite to prevent future
regression.

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu@gmail.com>
Closes #7999

Pull-request: #8028 part 7/12
Sebastian Gottschall
Linux does not HAVE_SMB_SHARE

Since Linux does not have an in-kernel SMB server, we don't need the
code to manage it.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Richard Elling <Richard.Elling@RichardElling.com>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes #8032

Pull-request: #8028 part 6/12
Sebastian Gottschall
Linux does not HAVE_DNLC

Since Linux does not have the Directory Name Lookup Cache, we don't need
the code to manage it.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Tim Chase <tim@chase2k.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Richard Elling <Richard.Elling@RichardElling.com>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes #8031

Pull-request: #8028 part 5/12
Sebastian Gottschall
Advise users to retain issue/PR templates

Occasionally we get issues and PRs from users who delete the
templates.  Advise users that their issues and PRs may be closed if
they do not fill out the templates as we really need this information.

Also updating PR template to drop unneeded approval toggle as we are
now using issue labels for status tracking.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: bunder2015 <omfgbunder@gmail.com>
Closes #8029

Pull-request: #8028 part 4/12
Sebastian Gottschall
Add types to featureflags in zfs

The boolean featureflags in use thus far in ZFS are extremely useful,
but because they take advantage of the zap layer, more interesting data
than just a true/false value can be stored in a featureflag. In redacted
send/receive, this is used to store the list of redaction snapshots for
a redacted dataset.

This change adds the ability for ZFS to store types other than a boolean
in a featureflag. The only other implemented type is a uint64_t array.
It also modifies the interfaces around dataset features to accomodate
the new capabilities, and adds a few new functions to increase
encapsulation.

This functionality will be used by the Redacted Send/Receive feature.

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes #7981

Pull-request: #8028 part 3/12
Sebastian Gottschall
Add types to featureflags in zfs

The boolean featureflags in use thus far in ZFS are extremely useful,
but because they take advantage of the zap layer, more interesting data
than just a true/false value can be stored in a featureflag. In redacted
send/receive, this is used to store the list of redaction snapshots for
a redacted dataset.

This change adds the ability for ZFS to store types other than a boolean
in a featureflag. The only other implemented type is a uint64_t array.
It also modifies the interfaces around dataset features to accomodate
the new capabilities, and adds a few new functions to increase
encapsulation.

This functionality will be used by the Redacted Send/Receive feature.

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes #7981

Pull-request: #8028 part 2/12
Sebastian Gottschall
deadlock between mm_sem and tx assign in zfs_write() and page fault

The bug time sequence:
1. thread #1, `zfs_write` assign a txg "n".
2. In a same process, thread #2, mmap page fault (which means the
  `mm_sem` is hold) occurred, `zfs_dirty_inode` open a txg failed,
  and wait previous txg "n" completed.
3. thread #1 call `uiomove` to write, however page fault is occurred
  in `uiomove`, which means it need `mm_sem`, but `mm_sem` is hold by
  thread #2, so it stuck and can't complete,  then txg "n" will
  not complete.

So thread #1 and thread #2 are deadlocked.

Reviewed-by: Chunwei Chen <tuxoko@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Grady Wong <grady.w@xtaotech.com>
Closes #7939

Pull-request: #8028 part 1/12
Sebastian Gottschall
mask out zstd levels from compression

otherwise ztest fuzzing will fail

Signed-off-by: Sebastian Gottschall <s.gottschall@dd-wrt.com>

Pull-request: #8028 part 11/11
Sebastian Gottschall
Merge branch 'master' of https://github.com/zfsonlinux/zfs

Signed-off-by: Sebastian Gottschall <s.gottschall@dd-wrt.com>

Pull-request: #8028 part 10/11
Sebastian Gottschall
Support zstd compression

Signed-off-by: Sebastian Gottschall <s.gottschall@dd-wrt.com>

Pull-request: #8028 part 9/11
Sebastian Gottschall
Allow copy-builtin to work with modified sources

`scripts/make_gitrev.sh` had 'set -e' so if any command failed it would
fail and cause copy-builtin to fail (copy-builtin also has `set -e`.
This commit also simplifies scripts/make_gitrev.sh to always write a
file by using a cleanup function.  It also simplifies other areas of
the script as well (making it much shorter).

Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matthew Thode <mthode@mthode.org>
Closes #8022
Closes #8025

Pull-request: #8028 part 8/11
Sebastian Gottschall
zpool: allow sharing of spare device among pools

ZFS allows, by default, sharing of spare devices among different pools;
this commit simply restores this functionality for disk devices and
adds an additional tests case to the ZFS Test Suite to prevent future
regression.

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu@gmail.com>
Closes #7999

Pull-request: #8028 part 7/11
Tom Caputi
ztest: convert active removal flag to lock

The ztest_device_removal_active flag did not compeltely
prevent damage injection from running concurrently with
device removal.  Convert the flag to a mutex.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>

TEST_ZTEST_TIMEOUT=7200

Pull-request: #8010 part 8/8
Tom Caputi
Fix ztest deadman panic with indirect vdev damage

This patch fixes an issue where ztest's deadman thread would
trigger a panic because reconstructing artifically damaged
blocks would take too long to reconstruct. This patch simply
limits how often ztest inflicts split-block damage and how
many segments it can damage when it does.

Signed-off-by: Tom Caputi <tcaputi@datto.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>

Pull-request: #8010 part 7/8
Tom Caputi
Fix issue with scanning dedup blocks as scan ends

This patch fixes an issue discovered by ztest where
dsl_scan_ddt_entry() could add I/Os to the dsl scan queues
between when the scan had finished all required work and
when the scan was marked as complete. This caused the scan
to spin indefinitely without ending.

Signed-off-by: Tom Caputi <tcaputi@datto.com>

Pull-request: #8010 part 6/8
Tom Caputi
Fix lock inversion in txg_sync_thread()

This patch fixes a lock inversion issue in txg_sync_thread() where
the code would attempt hold the spa config lock as a writer while
holding tx->tx_sync_lock. This races with spa_vdev_remove() which
attempts to hold the tx->tx_sync_lock to assign a new tx while
holding the spa config lock as a reader.

TEST_ZTEST_TIMEOUT=3600

Co-authored-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Tom Caputi <tcaputi@datto.com>

Pull-request: #8010 part 5/8
Tom Caputi
Fix dbgmsg printing in ztest and zdb

This patch resolves a problem where the -G option in both zdb and
ztest would cause the code to call __dprintf() to print zfs_dbgmsg
output. This function was not properly wired to add messages to the
dbgmsg log as it is in userspace and so the messages were simply
dropped. This patch also tries to add some degree of distinction to
dprintf() (which now prints directly to stdout) and zfs_dbgmsg()
(which adds messages to an internal list that can be dumped with
zfs_dbgmsg_print()).

In addition, this patch corrects an issue where ztest used a global
variable to decide whether to dump the dbgmsg buffer on a crash.
This did not work because ztest spins up more instances of itself
using execv(), which did not copy the global variable to the new
process. The option has been moved to the ztest_shared_opts_t
which already exists for interprocess communication.

This patch also changes zfs_dbgmsg_print() to use write() calls
instead of printf() so that it will not fail when used in a signal
handler.

TEST_ZTEST_TIMEOUT=3600

Signed-off-by: Tom Caputi <tcaputi@datto.com>

Pull-request: #8010 part 4/8
Tom Caputi
Fix ASSERT in zil_create() during ztest

This patch corrects an ASSERT in zil_create() that will only be
true if the call to zio_alloc_zil() does not fail.

TEST_ZTEST_TIMEOUT=3600

Signed-off-by: Tom Caputi <tcaputi@datto.com>

Pull-request: #8010 part 3/8
Tom Caputi
fixup

Pull-request: #8010 part 2/8
Tom Caputi
Fix random ztest_deadman_thread failures

The zloop test has been failing in buildbot for the last few weeks
with various failures in ztest_deadman_thread(). This is due to the
fact that this thread is not stopped when performing pool import /
export tests as it should be. This patch simply corrects this.

TEST_ZTEST_TIMEOUT=3600

Signed-off-by: Tom Caputi <tcaputi@datto.com>

Pull-request: #8010 part 1/8
Tom Caputi
Fix ztest deadman panic with indirect vdev damage

This patch fixes an issue where ztest's deadman thread would
trigger a panic because reconstructing artifically damaged
blocks would take too long to reconstruct. This patch simply
limits how often ztest inflicts split-block damage and how
many segments it can damage when it does.

Signed-off-by: Tom Caputi <tcaputi@datto.com>

Pull-request: #8010 part 7/7
Tom Caputi
Fix issue with scanning dedup blocks as scan ends

This patch fixes an issue discovered by ztest where
dsl_scan_ddt_entry() could add I/Os to the dsl scan queues
between when the scan had finished all required work and
when the scan was marked as complete. This caused the scan
to spin indefinitely without ending.

Signed-off-by: Tom Caputi <tcaputi@datto.com>

Pull-request: #8010 part 6/7
Tom Caputi
Fix lock inversion in txg_sync_thread()

This patch fixes a lock inversion issue in txg_sync_thread() where
the code would attempt hold the spa config lock as a writer while
holding tx->tx_sync_lock. This races with spa_vdev_remove() which
attempts to hold the tx->tx_sync_lock to assign a new tx while
holding the spa config lock as a reader.

TEST_ZTEST_TIMEOUT=3600

Co-authored-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Tom Caputi <tcaputi@datto.com>

Pull-request: #8010 part 5/7
Tom Caputi
Fix dbgmsg printing in ztest and zdb

This patch resolves a problem where the -G option in both zdb and
ztest would cause the code to call __dprintf() to print zfs_dbgmsg
output. This function was not properly wired to add messages to the
dbgmsg log as it is in userspace and so the messages were simply
dropped. This patch also tries to add some degree of distinction to
dprintf() (which now prints directly to stdout) and zfs_dbgmsg()
(which adds messages to an internal list that can be dumped with
zfs_dbgmsg_print()).

In addition, this patch corrects an issue where ztest used a global
variable to decide whether to dump the dbgmsg buffer on a crash.
This did not work because ztest spins up more instances of itself
using execv(), which did not copy the global variable to the new
process. The option has been moved to the ztest_shared_opts_t
which already exists for interprocess communication.

This patch also changes zfs_dbgmsg_print() to use write() calls
instead of printf() so that it will not fail when used in a signal
handler.

TEST_ZTEST_TIMEOUT=3600

Signed-off-by: Tom Caputi <tcaputi@datto.com>

Pull-request: #8010 part 4/7
Tom Caputi
Fix ASSERT in zil_create() during ztest

This patch corrects an ASSERT in zil_create() that will only be
true if the call to zio_alloc_zil() does not fail.

TEST_ZTEST_TIMEOUT=3600

Signed-off-by: Tom Caputi <tcaputi@datto.com>

Pull-request: #8010 part 3/7
Tom Caputi
fixup

Pull-request: #8010 part 2/7
Tom Caputi
Fix random ztest_deadman_thread failures

The zloop test has been failing in buildbot for the last few weeks
with various failures in ztest_deadman_thread(). This is due to the
fact that this thread is not stopped when performing pool import /
export tests as it should be. This patch simply corrects this.

TEST_ZTEST_TIMEOUT=3600

Signed-off-by: Tom Caputi <tcaputi@datto.com>

Pull-request: #8010 part 1/7
Olaf Faaland
MMP interval and fail_intervals in uberblock

When Multihost is enabled, and a pool is imported, uberblock writes
include ub_mmp_delay to allow an importing node to calculate the
duration of an activity test.  This value, however, is not enough
information.

If zfs_multihost_fail_intervals > 0 on the node with the pool imported,
the safe minimum duration of the activity test is well defined, but does
not depend on ub_mmp_delay:

  zfs_multihost_fail_intervals * zfs_multihost_interval

and if zfs_multihost_fail_intervals == 0, there is no such well defined
safe duration, but the importing host cannot tell whether mmp_delay is
high due to of I/O delays, or due to a very large zfs_multihost_interval
setting on the host which last imported the pool.  As a result, it may
use a far longer period for the activity test than is necessary.

This patch renames ub_mmp_sequence to ub_mmp_config and uses it to
record the zfs_multihost_interval and zfs_multihost_fail_intervals
values, as well as the mmp sequence.  This allows the optimal activity
test duration to be calculated by the importing host.

ZTS tests are added to verify the new functionality.

In addition, it makes a few other improvements:
* It updates the "sequence" part of ub_mmp_config when MMP writes
  in between syncs occur.  This allows an importing host to detect MMP
  on the remote host sooner, when the pool is idle, as it is not limited
  to the granularity of ub_timestamp (1 second).
* It fixes a bug where setting zfs_multihost_fail_intervals = 1 results
  in immediate pool suspension.
* It reports nanoseconds remaining in the activity test via
  /proc/spl/kstat/zfs/<pool>/activity_test (during a tryimport,
  when the test is normally performed, the pool name is $import)
* It fixes a cleanup issue with test mmp_active_import, where ztest is
  not killed for some failure modes.
* In ZTS, when checking whether the activity test occurred, check
  against a duration specified via an argument, so it is clear reading
  the test what is expected.

Signed-off-by: Olaf Faaland <faaland1@llnl.gov>

Pull-request: #7842 part 1/1