Console View
|
Tags: Architectures Distributions Performance Style Tests |
|
| Architectures | Distributions | Performance | Style | Tests | ||||||||||||||||||||||||||
|
|
|
|
|
|
||||||||||||||||||||||||||
|
John Gallagher
john.gallagher@delphix.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Fixes for procfs files backed by linked lists There are some issues with the way the seq_file interface is implemented for kstats backed by linked lists (zfs_dbgmsgs and certain per-pool debugging info): * We don't account for the fact that seq_file sometimes visits a node multiple times, which results in missing messages when read through procfs. * We don't keep separate state for each reader of a file, so concurrent readers will receive incorrect results. * We don't account for the fact that entries may have been removed from the list between read syscalls, so reading from these files in procfs can cause the system to crash. This change fixes these issues and adds procfs_list, a wrapper around a linked list which abstracts away the details of implementing the seq_file interface for a list and exposing the contents of the list through procfs. Signed-off-by: John Gallagher <john.gallagher@delphix.com> External-issue: LX-1211 Pull-request: #7819 part 1/1 |
||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||
|
Tim Chase
tim@chase2k.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
s/VERIFY/VERIFY3S in vdev_checkpoint_sm_object Using VERIFY3S allows to view the unexpected error value in the system log. Signed-off-by: Tim Chase <tim@chase2k.com> Pull-request: #7818 part 1/1 |
||||||||||||||||||||||||||||||
|
Tony Hutter
hutter2@llnl.gov |
|
|
|
|
|
|||||||||||||||||||||||||
|
Make zpool status counters match err events count The number of IO and checksum events should match the number of errors seen in zpool status. Previously there was a mismatch between the two counts because zpool status would only count unrecovered errors, while zpool events would get an event for *all* errors (recovered or not). This lead to situations where disks could be faulted for "too many errors", while at the same time showing zero errors in zpool status. This fixes the zpool status error counters to increment at the same times we post the error events. Signed-off-by: Tony Hutter <hutter2@llnl.gov> Closes: #4851 Pull-request: #7817 part 1/1 |
||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||
|
Tom Caputi
tcaputi@datto.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Fix issues with raw receive_write_byref() This patch fixes 2 issues with raw, deduplicated send streams. The first is that datasets who had been completely received earlier in the stream were not still marked as raw receives. This caused problems when newly received datasets attempted to fetch raw data from these datasets without this flag set. The second problem was that the arc freeze checksum code was not consistent about which locks needed to be held while performing its asserts. The proper locking needed to run these asserts is actually fairly nuanced, since the asserts touch the linked list of buffers (requiring the header lock), the arc_state (requiring the b_evict_lock), and the b_freeze_cksum (requiring the b_freeze_lock). This seems like a large performance sacrifice and a lot of unneeded complexity to verify that this relatively small debug feature is working as intended, so this patch simply removes these asserts instead. Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tom Caputi <tcaputi@datto.com> Closes #7701 |
||||||||||||||||||||||||||||||
|
LOLi
loli10K@users.noreply.github.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
pyzfs: add missing libzfs_core functions This change adds the following libzfs_core functions to pyzfs: lzc_remap, lzc_pool_checkpoint, lzc_pool_checkpoint_discard Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: loli10K <ezomori.nozomu@gmail.com> Closes #7793 Closes #7800 |
||||||||||||||||||||||||||||||
|
Olaf Faaland
faaland1@llnl.gov |
|
|
|
|
|
|||||||||||||||||||||||||
|
Skip import activity test in more zdb code paths Since zdb opens the pools read-only, it cannot damage the pool in the event the pool is already imported either on the same host or on another one. If the pool vdev structure is changing while zdb is importing the pool, it may cause zdb to crash. However this is unlikely, and in any case it's a user space process and can simply be run again. For this reason, zdb should disable the multihost activity test on import that is normally run. This commit fixes a few zdb code paths where that had been overlooked. It also adds tests to ensure that several common use cases handle this properly in the future. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Gu Zheng <guzheng2331314@163.com> Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Closes #7797 Closes #7801 |
||||||||||||||||||||||||||||||
|
DeHackEd
DeHackEd@users.noreply.github.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Don't modify argv[] in user tools argv[] gets modified during string parsing for input arguments. This is reflected in the live process listing. Don't do that. Reviewed-by: Serapheim Dimitropoulos <serapheim@delphix.com> Reviewed-by: loli10K <ezomori.nozomu@gmail.com> Reviewed-by: Giuseppe Di Natale <guss80@gmail.com> Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: DHE <git@dehacked.net> Closes #7760 |
||||||||||||||||||||||||||||||
|
Serapheim Dimitropoulos
serapheimd@gmail.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Introduce read/write kstats per dataset The following patch introduces a few statistics on reads and writes grouped by dataset. These statistics are implemented as kstats (backed by aggregate sums for performance) and can be retrieved by using the dataset objset ID number. The motivation for this change is to provide some preliminary analytics on dataset usage/performance. Reviewed-by: Richard Elling <Richard.Elling@RichardElling.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Signed-off-by: Serapheim Dimitropoulos <serapheim@delphix.com> Closes #7705 |
||||||||||||||||||||||||||||||
|
Rich Ercolani
rincebrain@gmail.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Reordered the variable assignments per review feedback Signed-off-by: Rich Ercolani <rincebrain@gmail.com> Pull-request: #7815 part 2/2 |
||||||||||||||||||||||||||||||
|
Rich Ercolani
rincebrain@gmail.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Added metadata/dnode cache info to arc_summary Signed-off-by: Rich Ercolani <rincebrain@gmail.com> Pull-request: #7815 part 1/1 |
||||||||||||||||||||||||||||||
|
DHE
git@dehacked.net |
|
|
|
|
|
|||||||||||||||||||||||||
|
Install suitable arc_summary.py script Use the "standard" version of python (as determined by /usr/bin/python) to decide which of the two versions to install, and install only that one. Previously we installed both regardless of the system version which could lead to dependency problems in package managers. Fixes #7651 Signed-off-by: DHE <git@dehacked.net> Pull-request: #7813 part 1/1 |
||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||
|
loli10K
ezomori.nozomu@gmail.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Stack overflow when destroying deeply nested clones Destroy operations on deeply nested chains of clones can overflow the stack: Depth Size Location (221 entries) ----- ---- -------- 0) 15664 48 mutex_lock+0x5/0x30 1) 15616 8 mutex_lock+0x5/0x30 ... 26) 13576 72 dsl_dataset_remove_clones_key.isra.4+0x124/0x1e0 [zfs] 27) 13504 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 28) 13432 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] ... 185) 2128 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 186) 2056 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 187) 1984 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 188) 1912 136 dsl_destroy_snapshot_sync_impl+0x4e0/0x1090 [zfs] 189) 1776 16 dsl_destroy_snapshot_check+0x0/0x90 [zfs] ... 218) 304 128 kthread+0xdf/0x100 219) 176 48 ret_from_fork+0x22/0x40 220) 128 128 kthread+0x0/0x100 Fix this issue by converting dsl_dataset_remove_clones_key() from recursive to iterative. Signed-off-by: loli10K <ezomori.nozomu@gmail.com> Pull-request: #7810 part 1/1 |
||||||||||||||||||||||||||||||
|
loli10K
ezomori.nozomu@gmail.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Stack overflow when destroying deeply nested clones Destroy operations on deeply nested chains of clones can overflow the stack: Depth Size Location (221 entries) ----- ---- -------- 0) 15664 48 mutex_lock+0x5/0x30 1) 15616 8 mutex_lock+0x5/0x30 ... 26) 13576 72 dsl_dataset_remove_clones_key.isra.4+0x124/0x1e0 [zfs] 27) 13504 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 28) 13432 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] ... 185) 2128 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 186) 2056 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 187) 1984 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 188) 1912 136 dsl_destroy_snapshot_sync_impl+0x4e0/0x1090 [zfs] 189) 1776 16 dsl_destroy_snapshot_check+0x0/0x90 [zfs] ... 218) 304 128 kthread+0xdf/0x100 219) 176 48 ret_from_fork+0x22/0x40 220) 128 128 kthread+0x0/0x100 Fix this issue by converting dsl_dataset_remove_clones_key() from recursive to iterative. Signed-off-by: loli10K <ezomori.nozomu@gmail.com> Pull-request: #7810 part 1/1 |
||||||||||||||||||||||||||||||
|
loli10K
ezomori.nozomu@gmail.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Stack overflow when destroying deeply nested clones Destroy operations on deeply nested chains of clones can overflow the stack: Depth Size Location (221 entries) ----- ---- -------- 0) 15664 48 mutex_lock+0x5/0x30 1) 15616 8 mutex_lock+0x5/0x30 ... 26) 13576 72 dsl_dataset_remove_clones_key.isra.4+0x124/0x1e0 [zfs] 27) 13504 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 28) 13432 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] ... 185) 2128 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 186) 2056 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 187) 1984 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 188) 1912 136 dsl_destroy_snapshot_sync_impl+0x4e0/0x1090 [zfs] 189) 1776 16 dsl_destroy_snapshot_check+0x0/0x90 [zfs] ... 218) 304 128 kthread+0xdf/0x100 219) 176 48 ret_from_fork+0x22/0x40 220) 128 128 kthread+0x0/0x100 Fix this issue by converting dsl_dataset_remove_clones_key() from recursive to iterative. Signed-off-by: loli10K <ezomori.nozomu@gmail.com> Pull-request: #7810 part 1/1 |
||||||||||||||||||||||||||||||
|
loli10K
ezomori.nozomu@gmail.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Stack overflow when destroying deeply nested clones Destroy operations on deeply nested chains of clones can overflow the stack: Depth Size Location (221 entries) ----- ---- -------- 0) 15664 48 mutex_lock+0x5/0x30 1) 15616 8 mutex_lock+0x5/0x30 ... 26) 13576 72 dsl_dataset_remove_clones_key.isra.4+0x124/0x1e0 [zfs] 27) 13504 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 28) 13432 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] ... 185) 2128 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 186) 2056 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 187) 1984 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 188) 1912 136 dsl_destroy_snapshot_sync_impl+0x4e0/0x1090 [zfs] 189) 1776 16 dsl_destroy_snapshot_check+0x0/0x90 [zfs] ... 218) 304 128 kthread+0xdf/0x100 219) 176 48 ret_from_fork+0x22/0x40 220) 128 128 kthread+0x0/0x100 Fix this issue by converting dsl_dataset_remove_clones_key() from recursive to iterative. Signed-off-by: loli10K <ezomori.nozomu@gmail.com> Pull-request: #7810 part 1/1 |
||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||
|
loli10K
ezomori.nozomu@gmail.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Stack overflow when destroying deeply nested clones Destroy operations on deeply nested chains of clones can overflow the stack: Depth Size Location (221 entries) ----- ---- -------- 0) 15664 48 mutex_lock+0x5/0x30 1) 15616 8 mutex_lock+0x5/0x30 ... 26) 13576 72 dsl_dataset_remove_clones_key.isra.4+0x124/0x1e0 [zfs] 27) 13504 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 28) 13432 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] ... 185) 2128 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 186) 2056 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 187) 1984 72 dsl_dataset_remove_clones_key.isra.4+0x18a/0x1e0 [zfs] 188) 1912 136 dsl_destroy_snapshot_sync_impl+0x4e0/0x1090 [zfs] 189) 1776 16 dsl_destroy_snapshot_check+0x0/0x90 [zfs] ... 218) 304 128 kthread+0xdf/0x100 219) 176 48 ret_from_fork+0x22/0x40 220) 128 128 kthread+0x0/0x100 Fix this issue by converting dsl_dataset_remove_clones_key() from recursive to iterative. Signed-off-by: loli10K <ezomori.nozomu@gmail.com> Pull-request: #7810 part 1/1 |
||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||
|
bunder2015
omfgbunder@gmail.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
ZTS: events path cleanup Removing hardcoded paths in events.cfg Reviewed-by: Giuseppe Di Natale <guss80@gmail.com> Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: bunder2015 <omfgbunder@gmail.com> Closes #7805 |
||||||||||||||||||||||||||||||
|
bunder2015
omfgbunder@gmail.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
ZTS: largest_pool_001 path cleanup Removing hardcoded paths in largest_pool_001 Reviewed-by: Giuseppe Di Natale <guss80@gmail.com> Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: bunder2015 <omfgbunder@gmail.com> Closes #7804 |
||||||||||||||||||||||||||||||
|
bunder2015
omfgbunder@gmail.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
ZTS: privilege group path cleanup Removing hardcoded paths in privilege group tests Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: bunder2015 <omfgbunder@gmail.com> Closes #7803 |
||||||||||||||||||||||||||||||
|
Brian Behlendorf
behlendorf1@llnl.gov |
|
|
|
|
|
|||||||||||||||||||||||||
|
ZTS: Fix import_cache_device_replaced Allow the 'zpool replace' to run slowly without overwhelming the vdev queues by setting zfs_scan_vdev_limit=128k. This limits the number of concurrent slow IOs which need to be handled. The net effect is the test case runs approximately 3x faster putting it well under the 10 minute per-test time limit. Rename import_cache* test cases to imprt_cachefile*. Originally these were renamed due to a maximum tar name limit, this limit was removed by commit 1dfde3d9b. Replaced instances of /var/tmp in zpool_import.cfg with $TEST_BASE_DIR. Reviewed-by: bunder2015 <omfgbunder@gmail.com> Reviewed-by: George Melikov <mail@gmelikov.ru> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #7765 Closes #7802 |
||||||||||||||||||||||||||||||
|
LOLi
loli10K@users.noreply.github.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
'zfs holds' scripted mode is not documented This change simply documents the existing "scripted mode" option in both command help and man page. Reviewed-by: Giuseppe Di Natale <guss80@gmail.com> Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: loli10K <ezomori.nozomu@gmail.com> Closes #7798 |
||||||||||||||||||||||||||||||
|
LOLi
loli10K@users.noreply.github.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Fix arcstat.py handling of unsupported options This change allows the arcstat.py script to handle unsupported options gracefully and print both error and usage messages when one such option is provided. Reviewed-by: Giuseppe Di Natale <guss80@gmail.com> Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: loli10K <ezomori.nozomu@gmail.com> Closes #7799 |
||||||||||||||||||||||||||||||
|
Olaf Faaland
faaland1@llnl.gov |
|
|
|
|
|
|||||||||||||||||||||||||
|
Skip import activity test in more zdb code paths Since zdb opens the pools read-only, it cannot damage the pool in the event the pool is already imported either on the same host or on another one. If the pool vdev structure is changing while zdb is importing the pool, it may cause zdb to crash. However this is unlikely, and in any case it's a user space process and can simply be run again. For this reason, zdb should disable the multihost activity test on import that is normally run. This commit fixes a few zdb code paths where that had been overlooked. It also adds tests to ensure that several common use cases handle this properly in the future. Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Pull-request: #7801 part 1/1 |
||||||||||||||||||||||||||||||
|
Tom Caputi
tcaputi@datto.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Always wait for txg sync when umounting dataset Currently, when unmounting a filesystem, ZFS will only wait for a txg sync if the dataset is dirty and not readonly. However, this can be problematic in cases where a dataset is remounted readonly immediately before being unmounted, which often happens when the system is being shut down. Since encrypted datasets require that all I/O is completed before the dataset is disowned, this issue causes problems when write I/Os leak into the txgs after the dataset is disowned, which can happen when sync=disabled. While looking into fixes for this issue, it was discovered that dsl_dataset_is_dirty() does not return B_TRUE when the dataset has been removed from the txg dirty datasets list, but has not actually been processed yet. Furthermore, the implementation is comletely different from dmu_objset_is_dirty(), adding to the confusion. Rather than relying on this function, this patch forces the umount code path (and the remount readonly code path) to always perform a txg sync on read-write datasets and removes the function altogether. Fixes: #7753 Signed-off-by: Tom Caputi <tcaputi@datto.com> Pull-request: #7795 part 2/2 |
||||||||||||||||||||||||||||||
|
Tom Caputi
tcaputi@datto.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Small rework of txg_list code This patch simply adds some missing locking to the txg_list functions and refactors txg_verify() so that it is only compiled in for debug builds. Signed-off-by: Tom Caputi <tcaputi@datto.com> Pull-request: #7795 part 1/2 |
||||||||||||||||||||||||||||||
|
Tom Caputi
tcaputi@datto.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Always wait for txg sync when umounting dataset Currently, when unmounting a filesystem, ZFS will only wait for a txg sync if the dataset is dirty and not readonly. However, this can be problematic in cases where a dataset is remounted readonly immediately before being unmounted, which often happens when the system is being shut down. Since encrypted datasets require that all I/O is completed before the dataset is disowned, this issue causes problems when write I/Os leak into the txgs after the dataset is disowned, which can happen when sync=disabled. While looking into fixes for this issue, it was discovered that dsl_dataset_is_dirty() does not return B_TRUE when the dataset has been removed from the txg dirty datasets list, but has not actually been processed yet. Furthermore, the implementation is comletely different from dmu_objset_is_dirty(), adding to the confusion. Rather than relying on this function, this patch forces the umount code path (and the remount readonly code path) to always perform a txg sync on read-write datasets and removes the function altogether. Fixes: #7753 Signed-off-by: Tom Caputi <tcaputi@datto.com> Pull-request: #7795 part 2/2 |
||||||||||||||||||||||||||||||
|
Tom Caputi
tcaputi@datto.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Small rework of txg_list code This patch simply adds some missing locking to the txg_list functions and refactors txg_verify() so that it is only compiled in for debug builds. Signed-off-by: Tom Caputi <tcaputi@datto.com> Pull-request: #7795 part 1/2 |
||||||||||||||||||||||||||||||
|
Tom Caputi
tcaputi@datto.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
added zpool resilver command Pull-request: #7732 part 4/4 |
||||||||||||||||||||||||||||||
|
Tom Caputi
tcaputi@datto.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
test fixups and code cleanups Pull-request: #7732 part 3/4 |
||||||||||||||||||||||||||||||
|
Tom Caputi
tcaputi@datto.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Address comments after review. Still working on mechanism for preventing deferrals. Signed-off-by: Tom Caputi <tcaputi@datto.com> Pull-request: #7732 part 2/4 |
||||||||||||||||||||||||||||||
|
Tom Caputi
tcaputi@datto.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Defer new resilvers until the current one ends Currently, if a resilver is triggered for any reason while an existing one is running, zfs will immediately restart the existing resilver from the beginning to include the new drive. This causes problems for system administrators when a drive fails while another is already resilvering. In this case, the optimal thing to do to reduce risk of data loss is to wait for the current resilver to end before immediately replacing the second failed drive, which allows the system to operate with two incomplete drives for the minimum amount of time. This patch introduces the resilver_defer feature that essentially does this for the admin without forcing them to wait and monitor the resilver manually. The change requires an on-disk feature since we must mark drives that are part of a defered resilver in the vdev config to ensure that we do not assume they are done resilvering when an existing resilver completes. Signed-off-by: Tom Caputi <tcaputi@datto.com> Pull-request: #7732 part 1/4 |
||||||||||||||||||||||||||||||
|
Tom Caputi
tcaputi@datto.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Test commit Signed-off-by: Tom Caputi <tcaputi@datto.com> Pull-request: #7732 part 5/5 |
||||||||||||||||||||||||||||||
|
Tom Caputi
tcaputi@datto.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
added zpool resilver command Pull-request: #7732 part 4/5 |
||||||||||||||||||||||||||||||
|
Tom Caputi
tcaputi@datto.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
test fixups and code cleanups Pull-request: #7732 part 3/5 |
||||||||||||||||||||||||||||||
|
Tom Caputi
tcaputi@datto.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Address comments after review. Still working on mechanism for preventing deferrals. Signed-off-by: Tom Caputi <tcaputi@datto.com> Pull-request: #7732 part 2/5 |
||||||||||||||||||||||||||||||
|
Tom Caputi
tcaputi@datto.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Defer new resilvers until the current one ends Currently, if a resilver is triggered for any reason while an existing one is running, zfs will immediately restart the existing resilver from the beginning to include the new drive. This causes problems for system administrators when a drive fails while another is already resilvering. In this case, the optimal thing to do to reduce risk of data loss is to wait for the current resilver to end before immediately replacing the second failed drive, which allows the system to operate with two incomplete drives for the minimum amount of time. This patch introduces the resilver_defer feature that essentially does this for the admin without forcing them to wait and monitor the resilver manually. The change requires an on-disk feature since we must mark drives that are part of a defered resilver in the vdev config to ensure that we do not assume they are done resilvering when an existing resilver completes. Signed-off-by: Tom Caputi <tcaputi@datto.com> Pull-request: #7732 part 1/5 |
||||||||||||||||||||||||||||||
|
Tim Chase
tim@chase2k.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
OpenZFS 9102 - zfs should be able to initialize... storage devices Authored by: George Wilson <george.wilson@delphix.com> Reviewed by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com> Reviewed by: Prakash Surya <prakash.surya@delphix.com> Approved by: Richard Lowe <richlowe@richlowe.net> Signed-off-by: Tim Chase <tim@chase2k.com> Ported-by: Tim Chase <tim@chase2k.com> OpenZFS-commit: https://github.com/openzfs/openzfs/commit/c3963210eb877c0bfb01c9ce117d0e7c1ac272e4 OpenZFS-issue: https://www.illumos.org/issues/9102 PROBLEM ======== The first access to a block incurs a performance penalty on some platforms (e.g. AWS's EBS, VMware VMDKs). Therefore we recommend that volumes are "thick provisioned", where supported by the platform (VMware). This can create a large delay in getting a new virtual machines up and running (or adding storage to an existing Engine). If the thick provision step is omitted, write performance will be suboptimal until all blocks on the LUN have been written. SOLUTION ========= This feature introduces a way to 'initialize' the disks at install or in the background to make sure we don't incur this first read penalty. When an entire LUN is added to ZFS, we make all space available immediately, and allow ZFS to find unallocated space and zero it out. This works with concurrent writes to arbitrary offsets, ensuring that we don't zero out something that has been (or is in the middle of being) written. This scheme can also be applied to existing pools (affecting only free regions on the vdev). Detailed design: - new subcommand:zpool initialize [-cs] <pool> [<vdev> ...] - start, suspend, or cancel initialization - Creates new open-context thread for each vdev - Thread iterates through all metaslabs in this vdev - Each metaslab: - select a metaslab - load the metaslab - mark the metaslab as being zeroed - walk all free ranges within that metaslab and translate them to ranges on the leaf vdev - issue a "zeroing" I/O on the leaf vdev that corresponds to a free range on the metaslab we're working on - continue until all free ranges for this metaslab have been "zeroed" - reset/unmark the metaslab being zeroed - if more metaslabs exist, then repeat above tasks. - if no more metaslabs, then we're done. - progress for the initialization is stored on-disk in the vdev’s leaf zap object. The following information is stored: - the last offset that has been initialized - the state of the initialization process (i.e. active, suspended, or canceled) - the start time for the initialization - progress is reported via the zpool status command and shows information for each of the videos that are initializing Porting notes: Added zfs_initialize_value module parameter to set the pattern written by "zpool initialize". Pull-request: #7708 part 1/1 |
||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||
|
Paul Zuchowski
pzuchowski@datto.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
bpobj_iterate overflows the stack The function bpobj_iterate_impl overflows the stack when bpobjs are deeply nested. Rewrite the function to eliminate the recursion. Signed-off-by: Paul Zuchowski <pzuchowski@datto.com> Fixes #7674 Pull-request: #7675 part 1/1 |
||||||||||||||||||||||||||||||
|
Don Brady
don.brady@delphix.com |
|
|
|
|
|
|||||||||||||||||||||||||
|
Pool allocation classes Allocation Classes adds the ability to have allocation classes in a pool that are dedicated to serving specific block categories, such as DDT data, metadata, and small file blocks. A pool can opt-in to this feature by adding a 'special' or 'dedup' top-level VDEV. Signed-off-by: Don Brady <don.brady@delphix.com> Pull-request: #5182 part 1/1 |
||||||||||||||||||||||||||||||