Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View


Tags: Architectures Platforms default
Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

Architectures Platforms default
Jorgen Lundman
Move EMPTY_TASKQ into taskq

As it accesses internal struct members outside of taskq, it
should really belong inside SPL.

This assumes FreeBSD still does not use icp, but I have a feeling
they recently changed this?

Signed-off-by: Jorgen Lundman <lundman@lundman.net>

Pull-request: #12802 part 1/1
Jorgen Lundman
Add spa _os() hooks

Add hooks for when spa is created, exported, activated and
deactivated. Used by macOS to attach iokit, and lock
kext as busy (to stop unloads).

Linux and FreeBSD have empty stubs.

Signed-off-by: Jorgen Lundman <lundman@lundman.net>

Pull-request: #12801 part 1/1
Coleman Kane
Linux 5.16 compat: asm/fpu/xcr.h is new location for xgetbv/xsetbv

Linux 5.16 moved these functions into this new header in commit
1b4fb8545f2b00f2844c4b7619d64d98440a477c. This change adds code to look
for the presence of this header, and include it so that the code using
xgetbv & xsetbv will compile again.

Signed-off-by: Coleman Kane <ckane@colemankane.org>

Pull-request: #12800 part 2/2
Coleman Kane
Linux 5.16: wait_on_page_bit() no longer available to modules

Instead, linux/pagemap.h offers a number of folio-specific functions to
be called instead. In this case, module/os/linux/zfs/zfs_vnops_os.c
wants to call wait_on_page_bit(pp, PG_writeback). This gets replaced
with folio_wait_bit(folio_page(pp), PG_writeback). This change modifies
the code to conditionally compile that if configure identifies th
presence of the folio_wait_bit() function.

Signed-off-by: Coleman Kane <ckane@colemankane.org>

Pull-request: #12800 part 1/2
Coleman Kane
Linux 5.16 compat: asm/fpu/xcr.h is new location for xgetbv/xsetbv

Linux 5.16 moved these functions into this new header in commit
1b4fb8545f2b00f2844c4b7619d64d98440a477c. This change adds code to look
for the presence of this header, and include it so that the code using
xgetbv & xsetbv will compile again.

Signed-off-by: Coleman Kane <ckane@colemankane.org>

Pull-request: #12800 part 2/2
Coleman Kane
Linux 5.16: wait_on_page_bit() no longer available to modules

Instead, linux/pagemap.h offers a number of folio-specific functions to
be called instead. In this case, module/os/linux/zfs/zfs_vnops_os.c
wants to call wait_on_page_bit(pp, PG_writeback). This gets replaced
with folio_wait_bit(folio_page(pp), PG_writeback). This change modifies
the code to conditionally compile that if configure identifies th
presence of the folio_wait_bit() function.

Signed-off-by: Coleman Kane <ckane@colemankane.org>

Pull-request: #12800 part 1/2
  • Debian 8 arm (BUILD): cloning zfs -  stdio
Coleman Kane
Linux 5.16 compat: asm/fpu/xcr.h is new location for xgetbv/xsetbv

Linux 5.16 moved these functions into this new header in commit
1b4fb8545f2b00f2844c4b7619d64d98440a477c. This change adds code to look
for the presence of this header, and include it so that the code using
xgetbv & xsetbv will compile again.

Signed-off-by: Coleman Kane <ckane@colemankane.org>

Pull-request: #12800 part 2/2
Coleman Kane
Linux 5.16: wait_on_page_bit() no longer available to modules

Instead, linux/pagemap.h offers a number of bit-specific functions to
be called instead. In this case, module/os/linux/zfs/zfs_vnops_os.c
wants to call wait_on_page_bit(pp, PG_writeback). This gets replaced
with wait_on_page_writeback(pp). This change modifies the code to
conditionally compile that if configure identifies the presence of the
wait_on_page_writeback(page*) function, which was introduced prior to
deprecating the wait_on_page_bit(...) interface.

Signed-off-by: Coleman Kane <ckane@colemankane.org>

Pull-request: #12800 part 1/2
Jorgen Lundman
Add macOS to zfs_xattr_owner_unlinked()

Signed-off-by: Jorgen Lundman <lundman@lundman.net>

Pull-request: #12794 part 1/1
  • Debian 10 arm64 (BUILD): cloning zfs -  stdio
Jorgen Lundman
ABI changes

Signed-off-by: Jorgen Lundman <lundman@lundman.net>

Pull-request: #12792 part 2/2
Jorgen Lundman
Add macOS to headers

Attempt to group most of the sweeping changes to headers in there,
unless they fit better with an individual commit.

In particular;

zfs_ioctl_register_pool(): promoted away from static, so macOS can
register an ioctl of this style.

fletcher_4_get(); added to header, even though Linux defines their
own version, and macOS its own. Can #ifdef around, if it upsets
Linux compiles.

nvpair_alloc_spl.c: including zfs_context_os.h - lots of
porting compile magic in here which helps. But could probably
be rejigged in headers if it is a bit questionable.

Some APPLE comments should perhaps live in macOS sources rather
than shared sources?

Signed-off-by: Jorgen Lundman <lundman@lundman.net>

Pull-request: #12792 part 1/1
Jorgen Lundman
Add macOS to headers

Attempt to group most of the sweeping changes to headers in there,
unless they fit better with an individual commit.

In particular;

zfs_ioctl_register_pool(): promoted away from static, so macOS can
register an ioctl of this style.

fletcher_4_get(); added to header, even though Linux defines their
own version, and macOS its own. Can #ifdef around, if it upsets
Linux compiles.

nvpair_alloc_spl.c: including zfs_context_os.h - lots of
porting compile magic in here which helps. But could probably
be rejigged in headers if it is a bit questionable.

Some APPLE comments should perhaps live in macOS sources rather
than shared sources?

Signed-off-by: Jorgen Lundman <lundman@lundman.net>

Pull-request: #12792 part 1/1
Ubuntu
Add failing test

Signed-off-by: Shaan Nobee <sniper111@gmail.com>

Pull-request: #12790 part 2/2
shaan1337
If we're doing a writeback with WB_SYNC_ALL and there's an active writeback, do a commit to speed up its completion
If we're doing a writeback with WB_SYNC_NONE, add a small delay to make sure we catch any WB_SYNC_ALL waiters due to a race condition in filemap_write_and_wait_range()
Introduce a parameter named zfs_page_writeback_no_sync_delay_us to control the small delay.

Signed-off-by: Shaan Nobee <sniper111@gmail.com>
Closes #12662

Pull-request: #12790 part 1/2
Ubuntu
Add failing test

Signed-off-by: Shaan Nobee <sniper111@gmail.com>

Pull-request: #12790 part 2/2
shaan1337
If we're doing a writeback with WB_SYNC_ALL and there's an active writeback, do a commit to speed up its completion
If we're doing a writeback with WB_SYNC_NONE, add a small delay to make sure we catch any WB_SYNC_ALL waiters due to a race condition in filemap_write_and_wait_range()
Introduce a parameter named zfs_page_writeback_no_sync_delay_us to control the small delay.

Signed-off-by: Shaan Nobee <sniper111@gmail.com>
Closes #12662

Pull-request: #12790 part 1/2
Ubuntu
Add failing test

Signed-off-by: Shaan Nobee <sniper111@gmail.com>

Pull-request: #12790 part 2/2
shaan1337
If we're doing a writeback with WB_SYNC_ALL and there's an active writeback, do a commit to speed up its completion
If we're doing a writeback with WB_SYNC_NONE, add a small delay to make sure we catch any WB_SYNC_ALL waiters due to a race condition in filemap_write_and_wait_range()
Introduce a parameter named zfs_page_writeback_no_sync_delay_us to control the small delay.

Signed-off-by: Shaan Nobee <sniper111@gmail.com>
Closes #12662

Pull-request: #12790 part 1/2
shaan1337
If we're doing a writeback with WB_SYNC_ALL and there's an active writeback, do a commit to speed up its completion
If we're doing a writeback with WB_SYNC_NONE, add a small delay to make sure we catch any WB_SYNC_ALL waiters due to a race condition in filemap_write_and_wait_range()
Introduce a parameter named zfs_page_writeback_no_sync_delay_us to control the small delay. The default value is 0 since even without the artificial delay, the issue occurs much more rarely now.

Signed-off-by: Shaan Nobee <sniper111@gmail.com>
Closes #12662

Pull-request: #12790 part 1/1
shaan1337
If we're doing a writeback with WB_SYNC_ALL and there's an active writeback, do a commit to speed up its completion
If we're doing a writeback with WB_SYNC_NONE, add a small delay to make sure we catch any WB_SYNC_ALL waiters due to a race condition in filemap_write_and_wait_range()

Signed-off-by: Shaan Nobee <sniper111@gmail.com>
Closes #12662

Pull-request: #12790 part 1/1
Alexander Motin
Improve log spacemap load time after unclean export.

Previous flushing algorithm limited only total number of log blocks to
the minimum of 256K and 4x number of metaslabs in the pool.  As result,
system with 1500 disks with 200 metaslabs each, touching several new
metaslabs each TXG could grow spacemap log to huge size without much
benefits.  We've observed one of such systems importing pool after
unclean export for about 45 minutes.

This patch improves the situation from four sides:
- By limiting maximum period for each metaslab to be flushed to 1000
TXGs, that effectively limits maximum number of per-TXG spacemap logs
to load after unclean export to the same number.
- By making flushing more smooth via accounting number of metaslabs
that were touched after the last flush and actually need another flush,
not just ms_unflushed_txg bump.
- By applying zfs_unflushed_log_block_pct to the number of metaslabs
that were touched after the last flush, not all metaslabs in the pool.
- By aggressively prefetching per-TXG spacemap logs up to 16 TXGs in
advance, making log spacemap load process for wide HDD pool CPU-bound,
accelerating it by many times.

As further optimization we could skip bumping ms_unflushed_txg for
metaslabs not touched since the last flush, but that would be an
incompatible change, requiring new pool feature.

Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Sponsored-By: iXsystems, Inc.

Pull-request: #12789 part 1/1
jokersus
Remove REMAKE_INITRD

The option has been deprecated in dkms and will break packaging in
future versions. See https://github.com/dell/dkms/commit/7114c62

Signed-off-by: jokersus <jokersus.cava@gmail.com>

Pull-request: #12781 part 1/1
Allan Jude
Add Linux namespace delegation support

This allows ZFS datasets to be delegated to a user/mount namespace
Within that namespace, only the delegated datasets are visible
Works very similarly to Zones/Jailes on other ZFS OSes

As a user:
```
$ unshare -Um
$ zfs list
no datasets available
$ readlink /proc/self/ns/user
user:[4026532291]
```

As root:
```
# zfs list
NAME                            ZONED  MOUNTPOINT
containers                      off    /containers
containers/host                off    /containers/host
containers/host/child          off    /containers/host/child
containers/host/child/gchild    off    /containers/host/child/gchild
containers/unpriv              on    /unpriv
containers/unpriv/child        on    /unpriv/child
containers/unpriv/child/gchild  on    /unpriv/child/gchild

# zfs zone 4026532291 containers/unpriv
```

Back to the user namespace:
```
$ zfs list
NAME                            USED  AVAIL    REFER  MOUNTPOINT
containers                      129M  47.8G      24K  /containers
containers/unpriv                128M  47.8G      24K  /unpriv
containers/unpriv/child          128M  47.8G      128M  /unpriv/child
```

Signed-off-by: Will Andrews <will.andrews@klarasystems.com>
Signed-off-by: Allan Jude <allan@klarasystems.com>
Sponsored-by: Buddy <https://buddy.works>

Pull-request: #12263 part 2/2
  • Debian 10 x86_64 (TEST): removed zfs failed -  stdio
  • FreeBSD stable/13 amd64 (TEST): removed zfs failed -  stdio
Allan Jude
Revert parts of 938cfeb0f27303721081223816d4f251ffeb1767

When read and writing the UID/GID, we always want the value
relative to the root user namespace, the kernel will take care
of remapping this to the user namespace for us.

Calling from_kuid(user_ns, uid) with a unmapped uid will return -1
as that uid is outside of the scope of that namespace, and will result
in the files inside the namespace all being owned by 'nobody' and not
being allowed to call chmod or chown on them.

Signed-off-by: Allan Jude <allan@klarasystems.com>

Pull-request: #12263 part 1/2
Allan Jude
Add Linux namespace delegation support

This allows ZFS datasets to be delegated to a user/mount namespace
Within that namespace, only the delegated datasets are visible
Works very similarly to Zones/Jailes on other ZFS OSes

As a user:
```
$ unshare -Um
$ zfs list
no datasets available
$ readlink /proc/self/ns/user
user:[4026532291]
```

As root:
```
# zfs list
NAME                            ZONED  MOUNTPOINT
containers                      off    /containers
containers/host                off    /containers/host
containers/host/child          off    /containers/host/child
containers/host/child/gchild    off    /containers/host/child/gchild
containers/unpriv              on    /unpriv
containers/unpriv/child        on    /unpriv/child
containers/unpriv/child/gchild  on    /unpriv/child/gchild

# zfs zone 4026532291 containers/unpriv
```

Back to the user namespace:
```
$ zfs list
NAME                            USED  AVAIL    REFER  MOUNTPOINT
containers                      129M  47.8G      24K  /containers
containers/unpriv                128M  47.8G      24K  /unpriv
containers/unpriv/child          128M  47.8G      128M  /unpriv/child
```

Signed-off-by: Will Andrews <will.andrews@klarasystems.com>
Signed-off-by: Allan Jude <allan@klarasystems.com>
Sponsored-by: Buddy <https://buddy.works>

Pull-request: #12263 part 2/2
Allan Jude
Revert parts of 938cfeb0f27303721081223816d4f251ffeb1767

When read and writing the UID/GID, we always want the value
relative to the root user namespace, the kernel will take care
of remapping this to the user namespace for us.

Calling from_kuid(user_ns, uid) with a unmapped uid will return -1
as that uid is outside of the scope of that namespace, and will result
in the files inside the namespace all being owned by 'nobody' and not
being allowed to call chmod or chown on them.

Signed-off-by: Allan Jude <allan@klarasystems.com>

Pull-request: #12263 part 1/2
Rich Ercolani
Corrected a case where we could read uninited ABD memory

For my sins, I started running valgrind over ztest to try and fix
that pesky intermittent "zloop dies with malloc errors" problem.

This one seemed exciting enough to merit cutting a PR for before
the rest get polished.

Suggested-by: @pcd1193182

Signed-off-by: Rich Ercolani <rincebrain@gmail.com>

Pull-request: #12214 part 1/1
Jorgen Lundman
macOS: compile fixes after rebase

Signed-off-by: Jorgen Lundman <lundman@lundman.net>

Pull-request: #12110 part 36/36
Allan Jude
Catch up ABI files

Signed-off-by: Allan Jude <allan@klarasystems.com>

Pull-request: #11711 part 4/4
Allan Jude
Add a new vdevprops.7 man page, and augment zpool-get/set man page

Signed-off-by: Allan Jude <allan@klarasystems.com>
Signed-off-by: jebsolutions

Pull-request: #11711 part 3/4
Allan Jude
Fixups

- Change the way we set the vdev_path to reuse existing code
- ZPOOL_VDEV_PROPS_GET_PROPS being null is ok (means all props)
- Add some missing SET_ERROR()s

Signed-off-by: Allan Jude <allan@klarasystems.com>

Pull-request: #11711 part 2/4
Allan Jude
vdev properties

Add properties, similar to pool properties, to each vdev.
This makes use of the existing per-vdev ZAP that was added as
part of device evacuation/removal.

A large number of read-only properties are exposed as well,
many of the members of struct vdev_t that provide useful
statistics.

Add support for properties.vdev in SYSFS

Add support for read-only "removing" vdev property
Add the "allocation" property that can be set to "off"
Refactor space management for non-allocating devices
Add ability to remove user vdev property by setting value to ""

Signed-off-by: Allan Jude <allan@klarasystems.com>
Signed-off-by: Mark Maybee <mark.maybee@delphix.com>

Pull-request: #11711 part 1/4
Allan Jude
Fixups

- Change the way we set the vdev_path to reuse existing code
- ZPOOL_VDEV_PROPS_GET_PROPS being null is ok (means all props)
- Add some missing SET_ERROR()s

Signed-off-by: Allan Jude <allan@klarasystems.com>

Pull-request: #11711 part 2/2
Allan Jude
vdev properties

Add properties, similar to pool properties, to each vdev.
This makes use of the existing per-vdev ZAP that was added as
part of device evacuation/removal.

A large number of read-only properties are exposed as well,
many of the members of struct vdev_t that provide useful
statistics.

Add support for properties.vdev in SYSFS

Add support for read-only "removing" vdev property
Add the "allocation" property that can be set to "off"
Refactor space management for non-allocating devices
Add ability to remove user vdev property by setting value to ""

Signed-off-by: Allan Jude <allan@klarasystems.com>
Signed-off-by: Mark Maybee <mark.maybee@delphix.com>

Pull-request: #11711 part 1/2
Allan Jude
Fixups

- Change the way we set the vdev_path to reuse existing code
- ZPOOL_VDEV_PROPS_GET_PROPS being null is ok (means all props)
- Add some missing SET_ERROR()s

Signed-off-by: Allan Jude <allan@klarasystems.com>

Pull-request: #11711 part 7/7
Allan Jude
zfs list: Allow more fields in ZFS_ITER_SIMPLE mode

If the fields to be listed and sorted by is constrained
to those populated by dsl_dataset_fast_stat(), then
zfs list is much faster, as it does not need to open each
objset and reads its properties.

A previous optimization by Pawel Dawidek
(0cee24064a79f9c01fc4521543c37acea538405f) took advantage
of this to make listing snapshot names sorted only by name
much faster.

However, it was limited to `-o name -s name`, this work
extends this optimization to work with:
  - name
  - guid
  - createtxg
  - numclones
  - inconsistent
  - redacted
  - origin
and could be further extended to any other properties
supported by dsl_dataset_fast_stat() or similar, that do
not require extra locking or reading from disk.

Signed-off-by: Allan Jude <allan@klarasystems.com>

Pull-request: #11080 part 1/1
Allan Jude
zfs list: Allow more fields in ZFS_ITER_SIMPLE mode

If the fields to be listed and sorted by is constrained
to those populated by dsl_dataset_fast_stat(), then
zfs list is much faster, as it does not need to open each
objset and reads its properties.

A previous optimization by Pawel Dawidek
(0cee24064a79f9c01fc4521543c37acea538405f) took advantage
of this to make listing snapshot names sorted only by name
much faster.

However, it was limited to `-o name -s name`, this work
extends this optimization to work with:
  - name
  - guid
  - createtxg
  - numclones
  - inconsistent
  - redacted
  - origin
and could be further extended to any other properties
supported by dsl_dataset_fast_stat() or similar, that do
not require extra locking or reading from disk.

Signed-off-by: Allan Jude <allan@klarasystems.com>

Pull-request: #11080 part 1/1
Alek Pinchuk
Implement a new type of zfs receive: corrective receive (-c)

This type of recv is used to heal corrupted data when a replica
of the data already exists (in the form of a send file for example).
With the provided send stream, corrective receive will read from
disk blocks described by the WRITE records. When any of the reads
come back with ECKSUM we use the data from the corresponding WRITE
record to rewrite the corrupted block.

Signed-off-by: Alek Pinchuk <apinchuk@axcient.com>

Pull-request: #9372 part 1/1