Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Tags: Architectures Platforms default
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

Architectures Platforms default
George Wilson
'zfs share -a' should handle 'canmount=noauto'

The 'zfs share -a' currently skips any filesystems which
have 'canmount=noauto' set. This behavior is unexpected since the
one would expect 'zfs share -a' to share any mounted filesystem
that has the 'sharenfs' property already set.

This changes the behavior of 'zfs share -a' to allow the sharing
of 'canmount=noauto' datasets if they are mounted.

Signed-off-by: George Wilson <gwilson@delphix.com>
External-issue: DLPX-71313

Pull-request: #10688 part 1/1
Matthew Macy
FreeBSD: update vaccess signature on most recent HEAD

Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Matt Macy <mmacy@FreeBSD.org>
Closes #10682
Paul Dagnelie
Clarify error message when a range-tree double-add occurs

In various other pieces of logic have resulted in situations where
we double-free space in ZFS. This in turn results in a double-add
to the range trees. These issues have been much more difficult to
diagnose than they should have been, because the error handling
around this case is much weaker than around the double remove case.

Reviewed-by: Matt Ahrens <matt@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Wilson <gwilson@delphix.com>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes #10654
Ryan Moeller
ZTS: Remove bashisms from zfs-tests.sh

Bring zfs-tests.sh in to compliance with the other scripts
by converting it /bin/sh for to avoid a dependency on bash.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Closes #10640
Matthew Ahrens
KMC_KVMEM should use kvmalloc()

`KMC_KVMEM` was designed to use `kvmalloc()` (see #9034).  However, this
was changed by #9813 to use `vmalloc()`, because `kvmalloc()` doesn't
always return page-aligned addresses.  However, the SPL kmem-cache
implementation doesn't need page-aligned addresses, because no SPL
caches request any particular alignment.

This commit changes `KMC_KVMEM` to use `kvmalloc()`, removes the
assertion that it returns page-aligned addresses, and asserts that SPL
caches to not request any particular alignment.

Signed-off-by: Matthew Ahrens <mahrens@delphix.com>

Pull-request: #10686 part 2/2
  • CentOS 7 x86_64 (TEST): zfstests failed -  stdio
  • Ubuntu 20.04 x86_64 (TEST): zfstests failed -  stdiotests
Matthew Ahrens
remove KMC_KMEM and KMC_VMEM

`KMC_KMEM` and `KMC_VMEM` are now unused since all SPL-implemented
caches are `KMC_KVMEM`.

KMC_KMEM: Given the default value of `spl_kmem_cache_kmem_limit`, we
don't use kmalloc to back the SPL caches, instead we use kvmalloc
(KMC_KVMEM).  The flag, module parameter, /proc entries, and associated
code are removed.

KMC_VMEM: This flag is not used, and kvmalloc() is always preferable to
vmalloc().  The flag, /proc entries, and associated code are removed.

Signed-off-by: Matthew Ahrens <mahrens@delphix.com>

Pull-request: #10686 part 1/2
Ryan Moeller
Move ZVOL_DIR back to zfs.h

Issue #10361

This was previously moved because nothing else in-tree uses it, but
evidently DilOS uses it out of tree.

Signed-off-by: Ryan Moeller <freqlabs@freebsd.org>

Pull-request: #10685 part 1/1
Matthew Ahrens
use zfs_dbgmsg to log metaslab_load/unload

Metaslabs are now (usually) loaded and unloaded infrequently, but when
that is not the case, it is useful to have a log of when and why these
events happened.

This commit enables the zfs_dbgmsg() in metaslab_load(), and adds a
zfs_dbgmsg() in metaslab_unload().

Signed-off-by: Matthew Ahrens <mahrens@delphix.com>

Pull-request: #10683 part 1/1
Matt Macy
FreeBSD: update vaccess signature on most recent HEAD

Signed-off-by: Matt Macy <mmacy@FreeBSD.org>

Pull-request: #10682 part 1/1
Matthew Ahrens
Remove commented-out code

Remove dead code to make the implementation easier to understand.

Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matt Ahrens <matt@delphix.com>
Closes #10650
Matthew Ahrens
Remove KM_NODEBUG

Remove dead code to make the implementation easier to understand.

Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matt Ahrens <matt@delphix.com>
Closes #10650
Matthew Ahrens
Remove KMC_NOMAGAZINE

Remove dead code to make the implementation easier to understand.

Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matt Ahrens <matt@delphix.com>
Closes #10650
Matthew Ahrens
Remove KMC_QCACHE

Remove dead code to make the implementation easier to understand.

Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matt Ahrens <matt@delphix.com>
Closes #10650
Matthew Ahrens
Remove KMC_NOHASH

Remove dead code to make the implementation easier to understand.

Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matt Ahrens <matt@delphix.com>
Closes #10650
Matthew Ahrens
Remove KMC_NOTOUCH

Remove dead code to make the implementation easier to understand.

Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matt Ahrens <matt@delphix.com>
Closes #10650
Matthew Ahrens
Remove KMC_OFFSLAB

Remove dead code to make the implementation easier to understand.

Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matt Ahrens <matt@delphix.com>
Closes #10650
Matthew Ahrens
Fix i/o error handling of livelists and zap iteration

Pool-wide metadata is stored in the MOS (Meta Object Set).  This
metadata is stored in triplicate, in addition to any pool-level
reduncancy (e.g. RAIDZ).  However, if all 3+ copies of this metadata are
not available, we can still get EIO/ECKSUM when reading from the MOS.
If we encounter such an error in syncing context, we have typically
already committed to making a change that we now can't do because of the
corrupt/missing metadata.  We typically "handle" this with a `VERIFY()`
or `zfs_panic_recover()`.  This prevents the system from continuing on
in an undefined state, while minimizing the amount of error-handling
code.

However, there are some code paths that ignore these i/o errors, or
`ASSERT()` that they don't happen.  Since assertions are disabled on
non-debug builds, they effectively ignore them as well.  This can lead
to ZFS continuing on in an incorrect state, potentially leading to
on-disk inconsistencies.

This commit adds handling for these i/o errors on MOS metadata,
typically with a `VERIFY()`:

* Handle error return from `zap_cursor_retrieve()` in 4 places in
`dsl_deadlist.c`.

* Handle error return from `zap_contains()` in `dsl_dir_hold_obj()`.
Turns out this call isn't necessary because we can always call
`zap_lookup()`.

* Handle error return from `zap_lookup()` in `dsl_fs_ss_limit_check()`.

* Handle error return from `zap_remove()` in `dsl_dir_rename_sync()`.

* Handle error return from `zap_lookup()` in
`dsl_dir_remove_livelist()`.

* Handle error return from `dsl_process_sub_livelist()` in
`spa_livelist_delete_cb()`.

Additionally:

* Augment the internal history log message for `zfs destroy` to note
which method is used (e.g. bptree, livelist, or, synchronous) and the
mintxg.

* Correct a comment in `dbuf_init()`.

* Correct indentation in `dsl_dir_remove_livelist()`.

Reviewed by: Sara Hartse <sara.hartse@delphix.com>
Reviewed-by: George Wilson <george.wilson@delphix.com>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes #10643
Matthew Macy
FreeBSD: Add support for lockless lookup

Authored-by: mjg <mjg@FreeBSD.org>
Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Signed-off-by: Matt Macy <mmacy@FreeBSD.org>
Closes #10657
Matthew Macy
Add missed thread_exit() to vdev_{autotrim,rebuild}_thread

Reviewed-by: Jorgen Lundman <lundman@lundman.net>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Co-authored-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Matt Macy <mmacy@FreeBSD.org>
Closes #10668
Aaron Wood
ZDB: Fix -A and -AA handling

Fixes an issue that was causing -A and -AA to not be handled correctly.

Signed-off-by: Aaron Wood <aaronjwood@gmail.com>

Pull-request: #10675 part 1/1
Matthew Ahrens
remove KMC_KMEM and KMC_VMEM

`KMC_KMEM` and `KMC_VMEM` are now unused since all SPL-implemented
caches are `KMC_KVMEM`.

KMC_KMEM: Given the default value of `spl_kmem_cache_kmem_limit`, we
don't use kmalloc to back the SPL caches, instead we use kvmalloc
(KMC_KVMEM).  The flag, module parameter, /proc entries, and associated
code are removed.

KMC_VMEM: This flag is not used, and kvmalloc() is always preferable to
vmalloc().  The flag, /proc entries, and associated code are removed.

Signed-off-by: Matthew Ahrens <mahrens@delphix.com>

Pull-request: #10673 part 1/1
Matthew Ahrens
revert removal of commented-out code

Pull-request: #10673 part 4/4
Matthew Ahrens
manpage

Pull-request: #10673 part 3/4
Matthew Ahrens
remove KMC_VMEM

Pull-request: #10673 part 2/4
Matthew Ahrens
remove unused flag KMC_KMEM

Pull-request: #10673 part 1/4
Pavel Snajdr
Fix arc__wait__for__eviction tracepoint

3442c2a02d added new `arc_wait_for_eviction` tracepoint, which fails to
compile, when tracepoints are enabled.

The tracepoint definition begins with `DEFINE_ARC_WAIT_FOR_EVICTION_EVENT`
and is a multi-line definition, so this fixes the backslash
and parenthesis accordingly.

Reviewed-by: Matt Ahrens <matt@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Pavel Snajdr <snajpa@snajpa.net>
Closes #10669
Pavel Snajdr
Fix arc__wait__for__eviction tracepoint

3442c2a02d added new `arc_wait_for_eviction` tracepoint, which fails to
compile, when tracepoints are enabled.

The tracepoint definition begins with `DEFINE_ARC_WAIT_FOR_EVICTION_EVENT`
and is a multi-line definition, so this fixes the backslash
and parenthesis accordingly.

Signed-off-by: Pavel Snajdr <snajpa@snajpa.net>

Pull-request: #10669 part 1/1
Ryan Moeller
Add missed thread_exit() to vdev_rebuild_thread

Signed-off-by: Ryan Moeller <ryan@iXsystems.com>

Pull-request: #10668 part 2/2
Matt Macy
Add missed thread_exit() to vdev_autotrim_thread

Signed-off-by: Matt Macy <mmacy@FreeBSD.org>

Pull-request: #10668 part 1/1
Roland Fehrenbacher
Prevent double insert into sublist for userquota_updates_task

This is to protect multilists with a lock against access from
multiple threads.

Signed-off-by: Roland Fehrenbacher <rf@q-leap.de>

Pull-request: #10665 part 1/1
  • Amazon 2 x86_64 (BUILD): cloning spl -  stdio
  • Debian 8 arm (BUILD): cloning spl -  stdio
  • Debian 8 ppc64 (BUILD): cloning spl -  stdio
  • Ubuntu 16.04 aarch64 (BUILD): cloning spl -  stdio
  • Ubuntu 16.04 i386 (BUILD): cloning spl -  stdio
  • Kernel.org Built-in x86_64 (BUILD): cloning spl -  stdio
Paul Dagnelie
Clarify error message when a range-tree double-add occurs

Signed-off-by: Paul Dagnelie <pcd@delphix.com>

Pull-request: #10654 part 1/1
Matthew Ahrens
remove commented-out code

Pull-request: #10650 part 7/7
Michael Niewöhner
(hopefully) fix freebsd

Signed-off-by: Michael Niewöhner <foss@mniewoehner.de>

Pull-request: #10278 part 39/39
Michael Niewöhner
next round...

Signed-off-by: Michael Niewöhner <foss@mniewoehner.de>

Pull-request: #10278 part 38/38
Michael Niewöhner
readme

Signed-off-by: Michael Niewöhner <foss@mniewoehner.de>

Pull-request: #10278 part 34/34
Michael Niewöhner
another rename

Signed-off-by: Michael Niewöhner <foss@mniewoehner.de>

Pull-request: #10278 part 33/33
GitHub
Merge pull request #16 from allanjude/openzfs_zstd_allan

ZFS replication compatibility and per-dataset feature activation

Pull-request: #10278 part 32/32
Michael Niewöhner
Revert "well, I didn't say 'finally', did I?"

This reverts commit 3448c69fca86c25097bd1bcaf78ca5dab0917979.

Revert "finally fix it...."

This reverts commit ab27e7eb6e8d4bcf5deda6d5d175c738ae9dcf1c.

Revert "another one"

This reverts commit 9adb5fee47b45242c825120faea4c3dfb3a295d7.

Revert "meh. I knew there was something missing..."

This reverts commit 8a2a2c65f089e06931e6ff756f0e59df45bf3850.

Revert "[RFC] Restructure a little bit"

This reverts commit ce812db4b4e11a2bec0ff29ea777b92dedd5de18.

Pull-request: #10278 part 27/27
Brian Behlendorf
[WIP] Distributed Parity (dRAID) Feature

WARNING: This is still work in progress.  The user interface
and on-disk format have changed from previous versions of this
PR.  It is not compatible with previous versions.  The on-disk
format is not finalized and may continue to change in future
versions.

This patch adds a new top-level vdev type called dRAID, which
stands for Distributed parity RAID.  This pool configuration
allows all dRAID vdevs to participate when rebuilding to a hot
spare device.  This can substantially reduce the total time
required to restore full parity to pool with a failed device.

A dRAID pool can be created using the new top-level `draid` type.
Like `raidz`, the desired redundancy is specified after the type:
`draid[1,2,3]`.  No additional information is required to create
the pool and reasonable default values will be chosen based on
the number of child vdevs in the dRAID vdev.

    zpool create <pool> draid[1,2,3] <vdevs...>

Unlike raidz, additional optional dRAID configuration values can
be provided as part of the draid type as colon separated values.
This allows administrators to fully specify a layout for either
performance or capacity reasons.  The supported options include:

  - draid[:<groups>g] - Redundancy groups
  - draid[:<spares>s] - Distributed hot spares (default 1)
  - draid[:<data>d]  - Data devices per group
  - draid[:<iter>i]  - Iterations perform when generating
                        a valid dRAID mapping (default 3)

As part of adding test coverage for the new dRAID vdev type
the following options were added to the ztest command.  These
options are leverages by the zloop.sh test script to test a
wide range of dRAID configurations.

  -K draid|raidz|random -- kind of RAID to test
  -D <value> -- dRAID data drives per redundancy group
  -G <value> -- dRAID redundancy group count
  -S <value> -- dRAID distributed spare drives
  -R <value> -- RAID parity (raidz or dRAID)
  -L        -- (Existing -G (dump log option) was renamed -L)

The zpool_create, zpool_import, redundancy, replacement and
fault test groups have all been updated provide test coverage
for the dRAID feature.

TODO:
- [x] - Rebased on master, will be frequently rebased from now on.
- [x] - Add dRAID config validation functionality.
- [x] - Enforced reasonable defaults to prevent harmful configs.
- [x] - Move common dRAID functions to zcommon.
- [x] - Replaced `draidcfg` command with `zpool create ...`.
- [x] - Add functionalty to load/save known dRAID layouts.
- [x] - Convert custom dRAID debugging to normal ZFS debugging.
- [x] - Cleaned up 'zpool status' output.
- [x] - Permutations for 255 device pool reduce to fit in label.
- [x] - Rebuild works with virtual hot spare and physical device.
- [x] - Allow adding new top-level dRAID vdevs (no removal).
- [x] - Logical spares are now mandatory for dRAID.
- [x] - Update `ztest` to add dRAID pools to its pool layouts.
- [x] - User commands updated to detect dRAID kmod support.
- [x] - Resolve checksum errors for non-uniform groups in pools
- [x] - Investigate reducing the permutations size in the label.
- [x] - Review and update the sequential rebuild code.
- [x] - Debug (or remove) dRAID mirror code (currently disabled).
- [x] - Add `zpool replace` option to request rebuild or resilver.
- [x] - Add new and extend existing ZTS test cases.
- [x] - Investigate stale labels on disk preventing pool import.
- [x] - Verify checksum errors are reported correctly (zinject).
- [x] - Support 'zpool detach/attach' for rebuild during sparing.
- [x] - Add support for ZED to kick in logical spares.
- [x] - Developer documention for vdev_rebuild.c
- [x] - Verify gang block handling works correctly.
- [x] - Verify rebuild/resilver `zpool status` reporting.
- [x] - Update packaging as needed.
- [x] - Add new and extend existing ZTS test cases.
- [x] - Documentation updates (man pages, comments, wiki, etc).
- [x] - Verify corruption repair works correctly.
- [x] - Implement vdev_xlate() to support initialize and trim.
- [x] - Developer documention for vdev_draid.c
- [x] - Performance optimizaiton / analysis.

Co-authored-by: Isaac Huang <he.huang@intel.com>
Co-authored-by: Mark Maybee <mmaybee@cray.com>
Co-authored-by: Don Brady <don.brady@delphix.com>
Co-authored-by: Srikanth N S <nsrikanth@cray.com>
Co-authored-by: Stuart Maybee <smaybee@cray.com>
Co-authored-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>

External-issue: ZFS-12 ZFS-35 ZFS-36 ZFS-17 ZFS-56 ZFS 95 ZFS-96
External-issue: ZFS-100 ZFS-103 ZFS-106 ZFS-110 ZFS-111 ZFS-117
External-issue: ZFS-137 ZFS-139 ZFS-202
Issue #9558

Pull-request: #10102 part 1/1