Console View
|
Tags: Architectures Platforms default |
|
| Architectures | Platforms | default | ||||||||||||||||||||
|
|
|
|
||||||||||||||||||||
|
Coleman Kane
ckane@colemankane.org |
|
|
|
|||||||||||||||||||
|
5.14 compat: explicity assign set_page_dirty Kernel 5.14 introduced a change where set_page_dirty of struct address_space_operations is no longer implicitly set to __set_page_dirty_buffers(), which ended up resulting in a NULL pointer deref in the kernel when it is attempted to be called. Both of these interfaces have been around for a long time, so this change tests if the set_page_dirty member can be assigned to __set_page_dirty_buffers - which means this change forces this behavior on older kernels than for which the breaking change was introduced. Signed-off-by: Coleman Kane <ckane@colemankane.org> Pull-request: #12427 part 1/1 |
||||||||||||||||||||||
|
Alexander Motin
mav@FreeBSD.org |
|
|
|
|||||||||||||||||||
|
Avoid small buffer copying on write. It is wrong for arc_write_ready() to use zfs_abd_scatter_enabled to decide whether to reallocate/copy the buffer, because the answer is OS-specific and depends on the buffer size. Instead of that use abd_size_alloc_linear(), moved into public header. Signed-off-by: Alexander Motin <mav@FreeBSD.org> Sponsored-By: iXsystems, Inc. Pull-request: #12425 part 1/1 |
||||||||||||||||||||||
|
George Amanakis
gamanakis@gmail.com |
|
|
|
|||||||||||||||||||
|
When raw sending in chunks, use the physical payload size When raw sending and splitting data in chunks dmu_dump_write() will panic as it tries to dereference a NULL block pointer. In the same case use the physical payload size instead of the logical one to determine the number of chunks needed. Signed-off-by: George Amanakis <gamanakis@gmail.com> Pull-request: #12424 part 1/1 |
||||||||||||||||||||||
|
Alexander Motin
mav@FreeBSD.org |
|
|
|
|||||||||||||||||||
|
Fix/improve dbuf hits accounting. Instead of clearing stats inside arc_buf_alloc_impl() do it inside arc_hdr_alloc() and arc_release(). It fixes statistics being wiped every time new dbuf is filled from ARC. Remove b_l1hdr.b_l2_hits. L2ARC hits are accounted at b_l2hdr.b_hits. With removal of the field compact struct l1arc_buf_hdr by 8 bytes. Since the hits are accounted under hash lock, replace atomics with simple increments. Signed-off-by: Alexander Motin <mav@FreeBSD.org> Sponsored-By: iXsystems, Inc. Pull-request: #12422 part 1/1 |
||||||||||||||||||||||
|
Alexander Motin
mav@FreeBSD.org |
|
|
|
|||||||||||||||||||
|
Optimize arc_l2c_only lists assertions. It is very expensive and not informative to call multilist_is_empty() for each arc_change_state() on debug builds to check for impossible. Instead implement special index function for arc_l2c_only->arcs_list, multilists, panicking on any attempt to use it. Signed-off-by: Alexander Motin <mav@FreeBSD.org> Sponsored-By: iXsystems, Inc. Pull-request: #12421 part 1/1 |
||||||||||||||||||||||
|
Alexander Motin
mav@FreeBSD.org |
|
|
|
|||||||||||||||||||
|
Use more atomics in refcounts. Use atomic_load_64() for zfs_refcount_count() to prevent torn reads on 32-bit platforms. On 64-bit ones it should not change anything. When built with ZFS_DEBUG but running without tracking enabled use atomics instead of mutexes same as for builds without ZFS_DEBUG. Since rc_tracked can't change live we can check it without lock. Signed-off-by: Alexander Motin <mav@FreeBSD.org> Sponsored-By: iXsystems, Inc. Pull-request: #12420 part 1/1 |
||||||||||||||||||||||
|
Alexander Motin
mav@FreeBSD.org |
|
|
|
|||||||||||||||||||
|
Add comment on metaslab_class_throttle_reserve() locking. Signed-off-by: Alexander Motin <mav@FreeBSD.org> Pull-request: #12419 part 1/1 |
||||||||||||||||||||||
|
Jorgen Lundman
lundman@lundman.net |
|
|
|
|||||||||||||||||||
|
Upstream: fixes to pass -Werror Signed-off-by: Jorgen Lundman <lundman@lundman.net> Pull-request: #12416 part 3/3 |
||||||||||||||||||||||
|
Jorgen Lundman
lundman@lundman.net |
|
|
|
|||||||||||||||||||
|
Upstream: handle pthread_setname_np() differently for macOS Pull-request: #12416 part 2/3 |
||||||||||||||||||||||
|
Jorgen Lundman
lundman@lundman.net |
|
|
|
|||||||||||||||||||
|
Upstream: Add snapshot and zvol events For kernel to send snapshot mount/unmount events to zed. For kernel to send symlink creates/removes on zvol plumbing. (/dev/run/dsk/zvol/$pool/$zvol -> /dev/diskX) If zed misses the ENODEV, all errors after are EINVAL. Treat any error as kernel module failure. Signed-off-by: Jorgen Lundman <lundman@lundman.net> Pull-request: #12416 part 1/3 |
||||||||||||||||||||||
|
||||||||||||||||||||||
|
George Amanakis
gamanakis@gmail.com |
|
|
|
|||||||||||||||||||
|
Avoid panic on ARC hdr missing from hash table If an ARC header is missing from the hash table, instead of panicking increment an arcstat and report it as a warning. Signed-off-by: George Amanakis <gamanakis@gmail.com> Pull-request: #12412 part 1/1 |
||||||||||||||||||||||
|
Ryan Moeller
ryan@iXsystems.com |
|
|
|
|||||||||||||||||||
|
zloop: Add a max iterations option, use default run/pass times It is useful to have control over the number of iterations of zloop so we can easily produce "x core dumps found *in y iterations*" metrics. Using random values for run/pass times doesn't improve coverage in a meaningful way. Randomizing run time could be seen as a compromise between running a greater variety of shorter tests versus a smaller variety of longer tests within a fixed time span. However, it is not desirable when running a fixed number of iterations. Pass time already incorporates randomness within ztest. Either parameter can be passed to ztest explicitly if the defaults are not satisfactory. Signed-off-by: Ryan Moeller <ryan@iXsystems.com> Pull-request: #12411 part 1/1 |
||||||||||||||||||||||
|
Ryan Moeller
ryan@iXsystems.com |
|
|
|
|||||||||||||||||||
|
zloop: Use default run/pass times Using random values for run/pass times doesn't improve coverage in a meaningful way. Randomizing run time could be seen as a compromise between running a greater variety of shorter tests versus a smaller variety of longer tests within a fixed time span. However, it is not desirable when running a fixed number of iterations. Pass time already incorporates randomness within ztest. Either parameter can be passed to ztest explicitly if the defaults are not satisfactory. Signed-off-by: Ryan Moeller <ryan@iXsystems.com> Pull-request: #12411 part 2/2 |
||||||||||||||||||||||
|
Ryan Moeller
ryan@iXsystems.com |
|
|
|
|||||||||||||||||||
|
zloop: Add a max iterations option It is useful to have control over the number of iterations of zloop so we can easily produce "x core dumps found *in y iterations*" metrics. Signed-off-by: Ryan Moeller <ryan@iXsystems.com> Pull-request: #12411 part 1/2 |
||||||||||||||||||||||
|
Rich Ercolani
rincebrain@gmail.com |
|
|
|
|||||||||||||||||||
|
Make get_key_material_file fail more verbosely It turns out, there are a lot of possible reasons for fopen to fail. Let's share which reason we failed for today. Signed-off-by: Rich Ercolani <rincebrain@gmail.com> Pull-request: #12410 part 1/1 |
||||||||||||||||||||||
|
Brian Behlendorf
behlendorf1@llnl.gov |
|
|
|
|||||||||||||||||||
|
Linux 5.14 compat: blk_alloc_disk() In Linux 5.14, blk_alloc_queue is no longer exported, and its usage has been superseded by blk_alloc_disk, which returns a gendisk struct from which we can still retrieve the struct request_queue* that is needed in the one place where it is used. This also replaces the call to alloc_disk(minors), and minors is now set via struct member assignment. Reviewed-by: Coleman Kane <ckane@colemankane.org> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Pull-request: #12409 part 1/1 |
||||||||||||||||||||||
|
John Wren Kennedy
john.kennedy@delphix.com |
|
|
|
|||||||||||||||||||
|
Use zinject to clear the ARC over export/import Export and import was used at a time when zinject was only available on systems built with DEBUG enabled. Signed-off-by: John Kennedy <john.kennedy@delphix.com> Pull-request: #12408 part 4/4 |
||||||||||||||||||||||
|
John Wren Kennedy
john.kennedy@delphix.com |
|
|
|
|||||||||||||||||||
|
Refactor variables for perf tests - Remove weekly vs. nightly settings - Move variables with common values to perf.shlib Signed-off-by: John Kennedy <john.kennedy@delphix.com> Pull-request: #12408 part 3/4 |
||||||||||||||||||||||
|
John Wren Kennedy
john.kennedy@delphix.com |
|
|
|
|||||||||||||||||||
|
Fix dbuf cache size calculation When the meaning of `dbuf_cache_max_bytes` changed, the performance test that covers the dbuf cache started to fail. The test would try to write files for the test using the max possible size of the cache, inevitably filling the pool and failing. This change uses `dbuf_cache_shift` to correctly calculate the dbuf cache size. Signed-off-by: John Kennedy <john.kennedy@delphix.com> Pull-request: #12408 part 2/4 |
||||||||||||||||||||||
|
John Wren Kennedy
john.kennedy@delphix.com |
|
|
|
|||||||||||||||||||
|
Modify disk rules when running perf tests - Bail out early if we're running the perf tests and forget to specify disks. - Allow perf tests to run with any number of disks. Signed-off-by: John Kennedy <john.kennedy@delphix.com> Pull-request: #12408 part 1/4 |
||||||||||||||||||||||
|
Alexander Motin
mav@FreeBSD.org |
|
|
|
|||||||||||||||||||
|
Increase default volblocksize from 8KB to 16KB. Many things has changed since previous default was set many years ago. Nowadays 8KB does not allow adequate compression or even decent space efficiency on many of pools due to 4KB disk physical block rounding, especially on RAIDZ and DRAID. It effectively limits write throughput to only 2-3GB/s (250-350K blocks/s) due to sync thread, allocation, vdev queue and other block rate bottlenecks. It keeps L2ARC expensive despite many optimizations and dedup just unrealistic. In FreeNAS/TrueNAS we for years default to at least 16KB volblocksize for mirror pools and even bigger (32-64KB) for RAIDZ, and so far we can find very few scenarios (not synthetic benchmarks) when smaller blocks would show sufficient benefits. Signed-off-by: Alexander Motin <mav@FreeBSD.org> Pull-request: #12406 part 1/1 |
||||||||||||||||||||||
|
Alexander Motin
mav@FreeBSD.org |
|
|
|
|||||||||||||||||||
|
Increase default volblocksize from 8KB to 16KB. Many things has changed since previous default was set many years ago. Nowadays 8KB does not allow adequate compression or even decent space efficiency on many of pools due to 4KB disk physical block rounding, especially on RAIDZ and DRAID. It effectively limits write throughput to only 2-3GB/s (250-350K blocks/s) due to sync thread, allocation, vdev queue and other block rate bottlenecks. It keeps L2ARC expensive despite many optimizations and dedup just unrealistic. In FreeNAS/TrueNAS we for years default to at least 16KB volblocksize for mirror pools and even bigger (32-64KB) for RAIDZ, and so far we can find very few scenarios (not synthetic benchmarks) when smaller blocks would show sufficient benefits. Signed-off-by: Alexander Motin <mav@FreeBSD.org> Pull-request: #12406 part 1/1 |
||||||||||||||||||||||
|
George Amanakis
gamanakis@gmail.com |
|
|
|
|||||||||||||||||||
|
Teach zpool scrub to scrub only blocks in error log Added a flag -e in zpool scrub to scrub only blocks in error log. A user can pause, resume and cancel the error scrub by passing additional command line arguments -p -s just like a regular scrub. This involves adding a new flag, creating new libzfs interfaces, a new ioctl, and the actual iteration and read-issuing logic. Error scrubbing is executed in multiple txg to make sure pool performance is not affected. Signed-off-by: George Amanakis <gamanakis@gmail.com> Pull-request: #12355 part 1/1 |
||||||||||||||||||||||
|
George Amanakis
gamanakis@gmail.com |
|
|
|
|||||||||||||||||||
|
Teach zpool scrub to scrub only blocks in error log Added a flag -e in zpool scrub to scrub only blocks in error log. A user can pause, resume and cancel the error scrub by passing additional command line arguments -p -s just like a regular scrub. This involves adding a new flag, creating new libzfs interfaces, a new ioctl, and the actual iteration and read-issuing logic. Error scrubbing is executed in multiple txg to make sure pool performance is not affected. Signed-off-by: George Amanakis <gamanakis@gmail.com> Pull-request: #12355 part 1/1 |
||||||||||||||||||||||
|
GitHub
noreply@github.com |
|
|
|
|||||||||||||||||||
|
Remove unnecessary whitespace Remove unnecessary whitespace on the line Pull-request: #12343 part 3/3 |
||||||||||||||||||||||
|
GitHub
noreply@github.com |
|
|
|
|||||||||||||||||||
|
Fix code style issues Modify the code style to conform to the specification Pull-request: #12343 part 2/3 |
||||||||||||||||||||||
|
GitHub
noreply@github.com |
|
|
|
|||||||||||||||||||
|
Modify checksum obtain method of QAT CpaDcGeneratefooter function that obtain the checksum code does not support the CPA_DC_STATELESS mode. So we get the adler32 chencksum of the end of the zlib from dc_results Pull-request: #12343 part 1/3 |
||||||||||||||||||||||
|
Allan Jude
allan@klarasystems.com |
|
|
|
|||||||||||||||||||
|
Regenerate ABI files Signed-off-by: Allan Jude <allan@klarasystems.com> Pull-request: #12263 part 3/3 |
||||||||||||||||||||||
|
||||||||||||||||||||||
|
Allan Jude
allan@klarasystems.com |
|
|
|
|||||||||||||||||||
|
Add Linux namespace delegation support This allows ZFS datasets to be delegated to a user/mount namespace Within that namespace, only the delegated datasets are visible Works very similarly to Zones/Jailes on other ZFS OSes As a user: ``` $ unshare -Um $ zfs list no datasets available $ readlink /proc/$$/ns/user user:[4026532291] ``` As root: ``` # zfs list NAME ZONED MOUNTPOINT containers off /containers containers/host off /containers/host containers/host/child off /containers/host/child containers/host/child/gchild off /containers/host/child/gchild containers/unpriv on /unpriv containers/unpriv/child on /unpriv/child containers/unpriv/child/gchild on /unpriv/child/gchild # zfs userns add 4026532291 containers/unpriv ``` Back to the user: ``` $ zfs list NAME USED AVAIL REFER MOUNTPOINT containers 129M 47.8G 24K /containers containers/unpriv 128M 47.8G 24K /unpriv containers/unpriv/child 128M 47.8G 128M /unpriv/child ``` Signed-off-by: Will Andrews <will.andrews@klarasystems.com> Signed-off-by: Allan Jude <allan@klarasystems.com> Sponsored-by: Buddy <https://buddy.works> Pull-request: #12263 part 2/3 |
||||||||||||||||||||||
|
Allan Jude
allan@klarasystems.com |
|
|
|
|||||||||||||||||||
|
Revert parts of 938cfeb0f27303721081223816d4f251ffeb1767 When read and writing the UID/GID, we always want the value relative to the root user namespace, the kernel will take care of remapping this to the user namespace for us. Calling from_kuid(user_ns, uid) with a unmapped uid will return -1 as that uid is outside of the scope of that namespace, and will result in the files inside the namespace all being owned by 'nobody' and not being allowed to call chmod or chown on them. Signed-off-by: Allan Jude <allan@klarasystems.com> Pull-request: #12263 part 1/3 |
||||||||||||||||||||||
|
Allan Jude
allan@klarasystems.com |
|
|
|
|||||||||||||||||||
|
Regenerate ABI files Signed-off-by: Allan Jude <allan@klarasystems.com> Pull-request: #12263 part 3/3 |
||||||||||||||||||||||
|
||||||||||||||||||||||
|
Allan Jude
allan@klarasystems.com |
|
|
|
|||||||||||||||||||
|
Add Linux namespace delegation support This allows ZFS datasets to be delegated to a user/mount namespace Within that namespace, only the delegated datasets are visible Works very similarly to Zones/Jailes on other ZFS OSes As a user: ``` $ unshare -Um $ zfs list no datasets available $ readlink /proc/$$/ns/user user:[4026532291] ``` As root: ``` # zfs list NAME ZONED MOUNTPOINT containers off /containers containers/host off /containers/host containers/host/child off /containers/host/child containers/host/child/gchild off /containers/host/child/gchild containers/unpriv on /unpriv containers/unpriv/child on /unpriv/child containers/unpriv/child/gchild on /unpriv/child/gchild # zfs userns add 4026532291 containers/unpriv ``` Back to the user: ``` $ zfs list NAME USED AVAIL REFER MOUNTPOINT containers 129M 47.8G 24K /containers containers/unpriv 128M 47.8G 24K /unpriv containers/unpriv/child 128M 47.8G 128M /unpriv/child ``` Signed-off-by: Will Andrews <will.andrews@klarasystems.com> Signed-off-by: Allan Jude <allan@klarasystems.com> Sponsored-by: Buddy <https://buddy.works> Pull-request: #12263 part 2/3 |
||||||||||||||||||||||
|
Allan Jude
allan@klarasystems.com |
|
|
|
|||||||||||||||||||
|
Revert parts of 938cfeb0f27303721081223816d4f251ffeb1767 When read and writing the UID/GID, we always want the value relative to the root user namespace, the kernel will take care of remapping this to the user namespace for us. Calling from_kuid(user_ns, uid) with a unmapped uid will return -1 as that uid is outside of the scope of that namespace, and will result in the files inside the namespace all being owned by 'nobody' and not being allowed to call chmod or chown on them. Signed-off-by: Allan Jude <allan@klarasystems.com> Pull-request: #12263 part 1/3 |
||||||||||||||||||||||
|
Ryan Moeller
ryan@iXsystems.com |
|
|
|
|||||||||||||||||||
|
Update ABI after xattr_compat Signed-off-by: Ryan Moeller <ryan@iXsystems.com> Pull-request: #11919 part 2/2 |
||||||||||||||||||||||
|
Ryan Moeller
ryan@iXsystems.com |
|
|
|
|||||||||||||||||||
|
Cross-platform xattr user namespace compatibility ZFS on Linux originally implemented xattr namespaces in a way that is incompatible with other operating systems. On illumos, xattrs do not have namespaces. Every xattr name is visible. FreeBSD has two universally defined namespaces: EXTATTR_NAMESPACE_USER and EXTATTR_NAMESPACE_SYSTEM. The system namespace is used for protected FreeBSD-specific attributes such as MAC labels and pnfs state. These attributes have the namespace string "freebsd:system:" prefixed to the name in the encoding scheme used by ZFS. The user namespace is used for general purpose user attributes and obeys normal access control mechanisms. These attributes have no namespace string prefixed, so xattrs written on illumos are accessible in the user namespace on FreeBSD, and xattrs written to the user namespace on FreeBSD are accessible by the same name on illumos. Linux has several xattr namespaces. On Linux, ZFS encodes the namespace in the xattr name for every namespace, including the user namespace. As a consequence, an xattr in the user namespace with the name "foo" is stored by ZFS with the name "user.foo" and therefore appears on FreeBSD and illumos to have the name "user.foo" rather than "foo". Conversely, none of the xattrs written on FreeBSD or illumos are accessible on Linux unless the name happens to be prefixed with one of the Linux xattr namespaces, in which case the namespace is stripped from the name. This makes xattrs entirely incompatible between Linux and other platforms. We want to make the encoding of user namespace xattrs compatible across platforms. A critical requirement of this compatibility is for xattrs from existing pools from FreeBSD and illumos to be accessible by the same names in the user namespace on Linux. It is also necessary that existing pools with xattrs written by Linux retain access to those xattrs by the same names on Linux. Making user namespace xattrs from Linux accessible by the correct names on other platforms is important. The handling of other namespaces is not required to be consistent. Add a fallback mechanism for listing and getting xattrs to treat xattrs as being in the user namespace if they do not match a known prefix. When setting user namespace xattrs, do not prefix the namespace to the name. If the xattr is already present with the namespace prefix, remove it so only the non-prefixed version persists. This ensures other platforms will be able to read the xattr with the correct name. Do not allow setting or getting xattrs with a name that is prefixed with one of the namespace names used by ZFS on supported platforms. Make xattr namespace compatibility dependent on a new feature. New pools will use the compatible namespace encoding by default, and existing pools will continue using the old Linux-specific encoding until the feature is enabled. Allow choosing between cross-platform compatability and legacy Linux compatibility with a per-dataset property. This facilitates replication between hosts with different compatibility needs. Add xattr_fallback property to skip fallbacks. This property currently defaults to "on" so we do not miss xattrs in datasets from porrls for which the encoding cannot be known or may even be a mix of both. It can be turned "off" for a performance boost when it is known only the configured xattr_compat format is relevant. Future work will investigate how we can change the default to "off" and automatically force it to "on" when we cannot know whether skipping the fallbacks is safe. TODO: * New tests should be added. * Performance optimizations should be investigated. Signed-off-by: Ryan Moeller <ryan@iXsystems.com> Pull-request: #11919 part 1/2 |
||||||||||||||||||||||
|
Ryan Moeller
ryan@iXsystems.com |
|
|
|
|||||||||||||||||||
|
Cross-platform xattr user namespace compatibility ZFS on Linux originally implemented xattr namespaces in a way that is incompatible with other operating systems. On illumos, xattrs do not have namespaces. Every xattr name is visible. FreeBSD has two universally defined namespaces: EXTATTR_NAMESPACE_USER and EXTATTR_NAMESPACE_SYSTEM. The system namespace is used for protected FreeBSD-specific attributes such as MAC labels and pnfs state. These attributes have the namespace string "freebsd:system:" prefixed to the name in the encoding scheme used by ZFS. The user namespace is used for general purpose user attributes and obeys normal access control mechanisms. These attributes have no namespace string prefixed, so xattrs written on illumos are accessible in the user namespace on FreeBSD, and xattrs written to the user namespace on FreeBSD are accessible by the same name on illumos. Linux has several xattr namespaces. On Linux, ZFS encodes the namespace in the xattr name for every namespace, including the user namespace. As a consequence, an xattr in the user namespace with the name "foo" is stored by ZFS with the name "user.foo" and therefore appears on FreeBSD and illumos to have the name "user.foo" rather than "foo". Conversely, none of the xattrs written on FreeBSD or illumos are accessible on Linux unless the name happens to be prefixed with one of the Linux xattr namespaces, in which case the namespace is stripped from the name. This makes xattrs entirely incompatible between Linux and other platforms. We want to make the encoding of user namespace xattrs compatible across platforms. A critical requirement of this compatibility is for xattrs from existing pools from FreeBSD and illumos to be accessible by the same names in the user namespace on Linux. It is also necessary that existing pools with xattrs written by Linux retain access to those xattrs by the same names on Linux. Making user namespace xattrs from Linux accessible by the correct names on other platforms is important. The handling of other namespaces is not required to be consistent. Add a fallback mechanism for listing and getting xattrs to treat xattrs as being in the user namespace if they do not match a known prefix. When setting user namespace xattrs, do not prefix the namespace to the name. If the xattr is already present with the namespace prefix, remove it so only the non-prefixed version persists. This ensures other platforms will be able to read the xattr with the correct name. Do not allow setting or getting xattrs with a name that is prefixed with one of the namespace names used by ZFS on supported platforms. Make xattr namespace compatibility dependent on a new feature. New pools will use the compatible namespace encoding by default, and existing pools will continue using the old Linux-specific encoding until the feature is enabled. Allow choosing between cross-platform compatability and legacy Linux compatibility with a per-dataset property. This facilitates replication between hosts with different compatibility needs. Add xattr_fallback property to skip fallbacks. This property currently defaults to "on" so we do not miss xattrs in datasets from porrls for which the encoding cannot be known or may even be a mix of both. It can be turned "off" for a performance boost when it is known only the configured xattr_compat format is relevant. Future work will investigate how we can change the default to "off" and automatically force it to "on" when we cannot know whether skipping the fallbacks is safe. TODO: * New tests should be added. * Performance optimizations should be investigated. Signed-off-by: Ryan Moeller <ryan@iXsystems.com> Pull-request: #11919 part 1/1 |
||||||||||||||||||||||
|
||||||||||||||||||||||
|
Allan Jude
allan@klarasystems.com |
|
|
|
|||||||||||||||||||
|
zfs: support force exporting pools This is primarily of use when a pool has lost its disk, while the user doesn't care about any pending (or otherwise) transactions. Implement various control methods to make this feasible: - txg_wait can now take a NOSUSPEND flag, in which case the caller will be alerted if their txg can't be committed. This is primarily of interest for callers that would normally pass TXG_WAIT, but don't want to wait if the pool becomes suspended, which allows unwinding in some cases, specifically when one is attempting a non-forced export. Without this, the non-forced export would preclude a forced export by virtue of holding the namespace lock indefinitely. - txg_wait also returns failure for TXG_WAIT users if a pool is actually being force exported. Adjust most callers to tolerate this. - spa_config_enter_flags now takes a NOSUSPEND flag to the same effect. - DMU objset initiator which may be set on an objset being forcibly exported / unmounted. - SPA export initiator may be set on a pool being forcibly exported. - DMU send/recv now use an interruption mechanism which relies on the SPA export initiator being able to enumerate datasets and closing any send/recv streams, causing their EINTR paths to be invoked. - ZIO now has a cancel entry point, which tells all suspended zios to fail, and which suppresses the failures for non-CANFAIL users. - metaslab, etc. cleanup, which consists of simply throwing away any changes that were not able to be synced out. - Linux specific: introduce a new tunable, zfs_forced_export_unmount_enabled, which allows the filesystem to remain in a modified 'unmounted' state upon exiting zpl_umount_begin, to achieve parity with FreeBSD and illumos, which have VFS-level support for yanking filesystems out from under users. However, this only helps when the user is actively performing I/O, while not sitting on the filesystem. In particular, this allows test #3 below to pass on Linux. - Add basic logic to zpool to indicate a force-exporting pool, instead of crashing due to lack of config, etc. Add tests which cover the basic use cases: - Force export while a send is in progress - Force export while a recv is in progress - Force export while POSIX I/O is in progress This change modifies the libzfs ABI: - New ZPOOL_STATUS_FORCE_EXPORTING zpool_status_t enum value. - New field libzfs_force_export for libzfs_handle. Signed-off-by: Will Andrews <will@firepipe.net> Signed-off-by: Allan Jude <allan@klarasystems.com> Sponsored-by: Klara, Inc. Sponsored-by: Catalogics, Inc. Closes #3461 Signed-off-by: Allan Jude <allan@klarasystems.com> Pull-request: #11082 part 1/1 |
||||||||||||||||||||||
|
Olaf Faaland
faaland1@llnl.gov |
|
|
|
|||||||||||||||||||
|
zpool import progress reports for long imports When an import requires a long MMP activity check, or when the user requests pool recovery, the import make take a long time. The user may not know why, or be able to tell whether the import is progressing or is hung. Add a new option to "zpool import", "-v" (for verbose). When the -v option is used, create an additional thread which periodically checks the contents of kstat/zfs/import_progress. If the import does not finish quickly, it reports the following on stdout: * Name and guid of pool being imported * User-friendly version of spa_load_state * Expected time import will complete * the max txg if recovering the pool If one or more of these values change, the thread prints a new record. A new record will be printed to stdout with a maximum frequency of one per second so as not to spam the user. As a result the printed output may reflect only some of the import states transitioned through. Use import kstat to check for Multihost activity check in relevant tests, instead of using import duration, which works poorly when testing on a slow machine. Sample output: Pool tank1 (guid 4591991398949870326): Checking for a remote import. Check will complete by Tue Apr 16 08:43:58 2019 Pool tank1 (guid 4591991398949870326): Checking for a remote import. Check will complete by Tue Apr 16 08:44:43 2019 Recovering Pool max txg 745 Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Pull-request: #8646 part 1/1 |
||||||||||||||||||||||
|
Olaf Faaland
faaland1@llnl.gov |
|
|
|
|||||||||||||||||||
|
zpool import progress reports for long imports When an import requires a long MMP activity check, or when the user requests pool recovery, the import make take a long time. The user may not know why, or be able to tell whether the import is progressing or is hung. Add a new option to "zpool import", "-v" (for verbose). When the -v option is used, create an additional thread which periodically checks the contents of kstat/zfs/import_progress. If the import does not finish quickly, it reports the following on stdout: * Name and guid of pool being imported * User-friendly version of spa_load_state * Expected time import will complete * the max txg if recovering the pool If one or more of these values change, the thread prints a new record. A new record will be printed to stdout with a maximum frequency of one per second so as not to spam the user. As a result the printed output may reflect only some of the import states transitioned through. Use import kstat to check for Multihost activity check in relevant tests, instead of using import duration, which works poorly when testing on a slow machine. Sample output: Pool tank1 (guid 4591991398949870326): Checking for a remote import. Check will complete by Tue Apr 16 08:43:58 2019 Pool tank1 (guid 4591991398949870326): Checking for a remote import. Check will complete by Tue Apr 16 08:44:43 2019 Recovering Pool max txg 745 Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Pull-request: #8646 part 1/1 |
||||||||||||||||||||||