diff options
| author | Qu Wenruo <wqu@suse.com> | 2023-06-08 19:48:36 +0800 |
|---|---|---|
| committer | Zorro Lang <zlang@kernel.org> | 2023-06-18 21:08:32 +0800 |
| commit | ab41f0bddb738000c9418c8d764f56fb542e7380 (patch) | |
| tree | 74793cf28e310bc013352c190da8c77888050d75 | |
| parent | 17c0dedf05fc4a8cb341c1779f1753c1ff684f16 (diff) | |
| download | xfstests-dev-ab41f0bddb738000c9418c8d764f56fb542e7380.tar.gz | |
common/btrfs: use _scratch_cycle_mount to ensure all page caches are dropped
[BUG]
There is a chance that btrfs/266 would fail on aarch64 with 64K page
size. (No matter if it's 4K sector size or 64K sector size)
The failure indicates that one or more mirrors are not properly fixed.
[CAUSE]
I have added some trace events into the btrfs IO paths, including
__btrfs_submit_bio() and __end_bio_extent_readpage().
When the test failed, the trace looks like this:
112819.764977: __btrfs_submit_bio: r/i=5/257 fileoff=0 mirror=1 len=196608 pid=33663
^^^ Initial read on the full 192K file
112819.766604: __btrfs_submit_bio: r/i=5/257 fileoff=0 mirror=2 len=65536 pid=21806
^^^ Repair on the first 64K block
Which would success
112819.766760: __btrfs_submit_bio: r/i=5/257 fileoff=65536 mirror=2 len=65536 pid=21806
^^^ Repair on the second 64K block
Which would fail
112819.767625: __btrfs_submit_bio: r/i=5/257 fileoff=65536 mirror=3 len=65536 pid=21806
^^^ Repair on the third 64K block
Which would success
112819.770999: end_bio_extent_readpage: read finished, r/i=5/257 fileoff=0 len=196608 mirror=1 status=0
^^^ The repair succeeded, the
full 192K read finished.
112819.864851: __btrfs_submit_bio: r/i=5/257 fileoff=0 mirror=3 len=196608 pid=33665
112819.874854: __btrfs_submit_bio: r/i=5/257 fileoff=0 mirror=1 len=65536 pid=31369
112819.875029: __btrfs_submit_bio: r/i=5/257 fileoff=131072 mirror=1 len=65536 pid=31369
112819.876926: end_bio_extent_readpage: read finished, r/i=5/257 fileoff=0 len=196608 mirror=3 status=0
But above read only happen for mirror 1 and mirror 3, mirror 2 is not
involved.
This means by somehow, the read on mirror 2 didn't happen, mostly
due to something wrong during the drop_caches call.
It may be an async operation or some hardware specific behavior.
On the other hand, for test cases doing the same operation but utilizing
direct IO, aka btrfs/267, it never fails as we never populate the page
cache thus would always read from the disk.
[WORKAROUND]
The root cause is in the "echo 3 > drop_caches", which I'm still
debugging.
But at the same time, we can workaround the problem by forcing a
cycle mount of scratch device, inside _btrfs_buffered_read_on_mirror().
By this we can ensure all page caches are dropped no matter if
drop_caches is working correctly.
With this patch, I no longer hit the failure on aarch64 with 64K page
size anymore, while before this the test case would always fail during a
-I 10 run.
[zlang: remove the duplicated drop_caches line]
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Christoph Hellwig <hch@infradead.org>
Reviewed-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Zorro Lang <zlang@kernel.org>
| -rw-r--r-- | common/btrfs | 6 |
1 files changed, 5 insertions, 1 deletions
diff --git a/common/btrfs b/common/btrfs index bd4dc31fa5..175b33aee0 100644 --- a/common/btrfs +++ b/common/btrfs @@ -598,7 +598,11 @@ _btrfs_buffered_read_on_mirror() local offset=$4 local size=$5 - echo 3 > /proc/sys/vm/drop_caches + # The drop_caches doesn't seem to drop every pages on aarch64 with + # 64K page size. + # So here as another workaround, cycle mount the SCRATCH_MNT to ensure + # the cache are dropped. + _scratch_cycle_mount while [[ -z $( (( BASHPID % nr_mirrors == mirror )) && exec $XFS_IO_PROG \ -c "pread -b $size $offset $size" $file) ]]; do |