| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
The test case assumed that the test target devices have 128 or more
sequential write required zones and uses 128 as the minimum number of
zones to write. This caused failure when the devices had a smaller
number of sequential write required zones. To avoid the failure, count
the actual number of sequential write required zones and use it if it is
smaller than 128.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/20240430103022.4136039-4-shinichiro.kawasaki@wdc.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
The previous commit fixed the verify failure due to the zone reset with
the verify_backlog option. Add a test to confirm the fix.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/20240430103022.4136039-3-shinichiro.kawasaki@wdc.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
The commit c5c8b92be5a2 ("zbd: fix zone reset condition for verify")
improved zbd_file_reset() to not reset zones when data to verify is
left. To check the left verify data, it tried to do the same as
check_get_verify() including the check for the modulo operation
"td->io_hist_len % td->o.verify_backlog". This check is required in
check_get_verify() to know when to do the verify backlog operation.
However, this check is not required in zbd_file_reset() since zone reset
is not related to the verify backlog timing. The unnecessary check for
"td->io_hist_len % td->o.verify_backlog" allows to reset zones even when
td->io_hist_len is non-zero and the data to verify is left. It erases
the data to verify and causes verify errors. Fix this by removing the
unnecessary check.
Fixes: c5c8b92be5a2 ("zbd: fix zone reset condition for verify")
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/20240430103022.4136039-2-shinichiro.kawasaki@wdc.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
The final sequence of tests uses a block size of 4096 bytes. This can be
slow enough on some platforms to trigger a 10-minute timeout. Increase
the block size to 256K to reduce the run time.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Update the HOWTO and man page for the unified data placement options
that cover both FDP and Streams.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
This test script uses the io_uring pass-through ioengine to test NVMe
streams support.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
This is useful for testing.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Placement IDs are a 16-bit value. So we should notify users if the
provided placement IDs are too large.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Make small adjustments to the code supporting FDP to accommodate NVMe
streams.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
This test script uses the io_uring pass-through ioengine to test fio's
FDP support. This uses both the orignal and the new fdp-related options.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Since FDP and streams are similar, we should have an over-arching data
placement option that encompasses both of these frameworks instead of
having separate sets of similar options for FDP and streams.
With a common set of options, users will be able to select the data
placement strategy (fdp or streams), the placement identifiers to use,
and the algorithm for selecting from the list of placement identifiers.
The original set of FDP options is retained for backward compatibility.
No functional change.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
We can use code in the files to support NVMe streams. Streams also
falls under the umbrella of data placement, so it seems reasonable to
put streams and FDP code in the same source files.
Also change the prefix of some functions from fdp_ to dp_ to indicate
that they are not specific to FDP but apply more generally to the two
data placement features.
No functional change.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
* 'master' of https://github.com/celestinechen/fio:
FIO with fsync option issues more DDIR_SYNC commands than expected
|
|
Issue and root cause:
When fsync option is used, the number of flush (or DDIR_SYNC) commands
issued is more than the expected number of flush commands.
To elaborate:
- In the fio config file, consider fsync=1
1. FIO issues 1 write command
2. After write completes, FIO sets last_was_sync variable to false
3. FIO issues 1 flush command
4. FIO keeps issuing flush commands since last_was_sync is still false
and this causes more flush commands to be issued than expected
5. last_was_sync is set to true after the flush command completes
- The above steps repeats until the workload is completed.
Fix:
Instead of setting last_was_sync to true after flush command is completed
and setting last_was_sync to false after write command is completed,
set last_was_sync to true after flush command is issued and set
last_was_sync to false after write command is issued.
Signed-off-by: Celestine Chen celestinechen@google.com
|
|
Convert how we define ioengine flags to use the same construction used
for thread flags. This allows us to count the number of ioengine flags
and write an assertion to ensure that thread flags and ioengine flags
fit inside td->flags.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
TD_NR is the number of fio run states. It's not related to the number of
bits we need to shift ioengine flags in order to store them in
td->flags. Testing this assertion doesn't make sense.
Fixes: 9b87f09b ("fio: inherit IO engine flags to 'td'")
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
We store the thread and ioengine flags in the same unsigned long long
with the ioengine flags bit shifted by the number of thread flags. In
the macros related to the bit shift, just use the count of the number of
thread flags. This way any time we change the number of thread flags the
required shift will be automatically updated.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
test the previous commit: Don’t break too early in readwrite mode
In rw mode if io_size > size An early reset of the addresses occurred.
Make sure all addresses are done correctly.
Signed-off-by: Chana-Zaks-wdc <chana.zaks@wdc.com>
Link: https://lore.kernel.org/r/20240418092222.20543-3-chana.zaks@wdc.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
In readwrite mode, when io-size > size, we break too early and start
looping the addresses too soon. Instead of reset the address when
reaching size, we start looping after size/2, e.g. in a 50/50 rw. This
wrong behavior is demonstrated in the example below.
Before the fix:
size is 0x140000, io-size is 0x280000, max-address = 0xa0000 = size / 2.
./fio --debug=io --name=global --filename=/dev/mmcblk0p3 --size=1280K
--io_size=2560k --bs=128K --offset=0 --name=test1 --rw=rw| grep "complete: io_u"
io 5646 complete: io_u 0x560337a20840: off=0x60000, len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 5646 complete: io_u 0x560337a20840: off=0x80000, len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 5646 complete: io_u 0x560337a20840: off=0x20000, len=0x20000, ddir=1, file=/dev/mmcblk0p3
io 5646 complete: io_u 0x560337a20840: off=0x40000, len=0x20000, ddir=1, file=/dev/mmcblk0p3
io 5646 complete: io_u 0x560337a20840: off=0x60000, len=0x20000, ddir=1, file=/dev/mmcblk0p3
io 5646 complete: io_u 0x560337a20840: off=0xa0000, len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 5646 complete: io_u 0x560337a20840: off=0x0, len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 5646 complete: io_u 0x560337a20840: off=0x20000, len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 5646 complete: io_u 0x560337a20840: off=0x40000, len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 5646 complete: io_u 0x560337a20840: off=0x0, len=0x20000, ddir=1, file=/dev/mmcblk0p3
Io 5646 complete: io_u 0x560337a20840: off=0x60000, len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 5646 complete: io_u 0x560337a20840: off=0x80000, len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 5646 complete: io_u 0x560337a20840: off=0x20000, len=0x20000, ddir=1, file=/dev/mmcblk0p3
io 5646 complete: io_u 0x560337a20840: off=0x40000, len=0x20000, ddir=1, file=/dev/mmcblk0p3
io 5646 complete: io_u 0x560337a20840: off=0x60000, len=0x20000, ddir=1, file=/dev/mmcblk0p3
io 5646 complete: io_u 0x560337a20840: off=0xa0000, len=0x20000, ddir=0, file=/dev/mmcblk0p3
After the fix max-address = 0x120000 = size – len:
io 32594 complete: io_u 0x70fc000980: off=0x0, len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0x20000, len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0x40000, len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0x0, len=0x20000, ddir=1, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0x60000, len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0x80000, len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0x20000, len=0x20000, ddir=1, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0x40000, len=0x20000, ddir=1, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0x60000, len=0x20000, ddir=1, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0xa0000, len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0x80000, len=0x20000, ddir=1, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0xc0000, len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0xa0000, len=0x20000, ddir=1, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0xe0000, len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0x100000,len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0x120000,len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0xc0000, len=0x20000, ddir=1, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0xe0000, len=0x20000, ddir=1, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0x0, len=0x20000, ddir=0, file=/dev/mmcblk0p3
io 32594 complete: io_u 0x70fc000980: off=0x20000, len=0x20000, ddir=0, file=/dev/mmcblk0p3
Fixes: ac002339c382 (Fix bug with rw sequence offset and io_limit)
Signed-off-by: Chana-Zaks-wdc <chana.zaks@wdc.com>
Link: https://lore.kernel.org/r/20240418092222.20543-2-chana.zaks@wdc.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
If NOPs are used, then no files exist, and hence the app will throw
a floating point error when trying to divide by zero.
Fixes: 6067863c7016 ("t/io_uring: pre-calculate per-file depth")
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Remind contributors to bump the server version and ioengine ops version
when appropriate.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
* 'ioops_ver' of https://github.com/ankit-sam/fio:
ioengines: bump up FIO_IOOPS_VERSION
|
|
This was left out when multi range trim support was added.
Fixes: commit b3251e31 (trim: add support for multiple ranges)
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
|
|
* 'directory-operation' of https://github.com/friendy-su/fio:
engines/fileoperations: add more description for file/directory operation engines
|
|
We should not be making these error prints, they are purely
information, and in fact they should probably get deleted all
together.
For now, as this is actually breaking test frameworks, just revert
the change.
This reverts commit 3c826d1cafe6c703237378314bdcd123770971e3.
https://lore.kernel.org/fstests/20240415182231.GA2649469@mit.edu/
Reported-by: "Theodore Ts'o" <tytso@mit.edu>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
engines
file/directory operations are quite different from I/O operation.
Add more description for how to configure the measurement
and how to utilize the data.
Signed-off-by: friendy-su <friendy.su@sony.com>
|
|
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
* 'delete-instead-of-unlink' of https://github.com/edigaryev/fio:
docs: use "delete" term instead of "unlink", which is less common
|
|
Signed-off-by: Nikolay Edigaryev <edigaryev@gmail.com>
|
|
* 'issue-1735' of https://github.com/yygcode/fio:
iolog: fix disk stats issue
|
|
In the iolog replay scenario, the disk util in the td structure is
not initialized, resulting in the disk stats not being correctly
updated.
Fixes: https://github.com/axboe/fio/issues/1735
Signed-off-by: Yong Gang <yygcode@gmail.com>
|
|
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Improve code readability by using a local variable for ioengine data.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Similar to file operation, directory operation performance is an
important benchmark to file system in practice.
* dircreate engine measures directories create performance
* dirstat engine measures directories lookup performance
* dirdelete engine measures directories delete performance
Signed-off-by: friendy-su <friendy.su@sony.com>
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
This test uses libaio. So run it only on Linux when libaio is available.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Add t/jobs/t0031-pre.fio and t/jobs/t0031.fio to test that the log
space is regrown for the asynchronous I/O replay jobs. This test case
confirms the fix by the previous commit titled "iolog: regrow logs in
iolog_delay()".
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/20240321031011.4140040-3-shinichiro.kawasaki@wdc.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
The commit b85c01f7e9df ("iolog.c: fix inaccurate clat when replay
trace") triggered the assertion failure below for the workload which
does I/O replay as asynchronous I/O together with log recording options
such as write_lat_log.
fio: stat.c:3030: get_cur_log: Assertion `iolog->pending->nr_samples < iolog->pending->max_samples' failed.
fio: pid=40120, got signal=6
The assertion means that too many logs are recorded in the pending log
space which keeps the logs until next log space regrow by reglow_logs()
call. However, reglow_logs() is not called, and the pending log space
runs out.
The trigger commit modified iolog_delay() to call io_u_queued_complete()
so that the asynchronous I/Os can be completed during delays between
replayed I/Os. Before this commit, replayed I/Os were not completed
until all I/O units are consumed. So the free I/O unit list gets empty
periodically, then wait_for_completion() and regrow_logs() were called
periodically. After this commit, all I/O units are not consumed, so
wait_for_completion() and regrow_logs() are no longer called for long
duration. Hence the assertion failure.
To avoid the assertion, add the check for log regrow and reglow_logs()
call in iolog_delay().
Fixes: b85c01f7e9df ("iolog.c: fix inaccurate clat when replay trace")
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/20240321031011.4140040-2-shinichiro.kawasaki@wdc.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Add a test to detect changes that break the --bandwidth-log option. This
option uses the logging data structures in a way that differs from how
the other logs use them. So it's easy to forget about this special case.
Recent patches resolving related breakage are:
d72244761b2230fbb2d6eaec59cdedd3ea651d4f ("stat: fix segfault with fio
option --bandwidth-log")
acc481b6d34aab3ee6e19f22b64f8bf0dd30480c ("iolog: fix reported defect
from coverity scan")
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Add a means to specify arbitrary command-line options when we are
running a test that consists of a fio job file.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
rate_cycle affects only rate_min and rate_iops_min.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Add a command-line option to run this script using the xNVMe ioengine
with the async io_uring_cmd backend. The default remains to use the
io_uring_cmd ioengine.
Example:
python3 t/nvmept_pi.py --dut /dev/ng1n1 --lbaf 6 --ioengine xnvme --fio ./fio
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Avoid errors decoding JSON data when testing invalid configurations.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Which is also a supported form of delimiter.
Signed-off-by: Avri Altman <avri.altman@wdc.com>
Link: https://lore.kernel.org/r/20240305090008.1216-6-avri.altman@wdc.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
s/bsrange/bssplit
Fixes: commit c37183f8a161 (test: test job for randtrimwrite)
Signed-off-by: Avri Altman <avri.altman@wdc.com>
Link: https://lore.kernel.org/r/20240305090008.1216-5-avri.altman@wdc.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Make it designate the correct fixing commit.
Signed-off-by: Avri Altman <avri.altman@wdc.com>
Link: https://lore.kernel.org/r/20240305090008.1216-4-avri.altman@wdc.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Add some more details explaining why the the successful result should be
87,040KB data.
Signed-off-by: Avri Altman <avri.altman@wdc.com>
Link: https://lore.kernel.org/r/20240305090008.1216-3-avri.altman@wdc.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
limit the scope of variables when possible, fix style isses etc.
Signed-off-by: Avri Altman <avri.altman@wdc.com>
Link: https://lore.kernel.org/r/20240305090008.1216-2-avri.altman@wdc.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
* 'patch-ioengines' of https://github.com/kcoms555/fio:
ioengines: Make td_io_queue print log_err when got error
|
|
* ioengines prints error log when got I/O error
Signed-off-by: Cho Jaeho <kcoms555@naver.com>
|
|
Starting with gcc 11 __sync_synchronize() compiles to
lock or QWORD PTR [rsp], 0
on x86_64 platforms. Previously it compiled to an mfence instruction.
See line 47 of https://godbolt.org/z/xfE18K7b4 for an example.
On Intel platforms this change does not affect the result of fio's CPU
clock test. But on AMD platforms, this change causes fio's CPU clock
test to fail and fio to fall back to clock_gettime() instead of using
the CPU clock for timing.
This patch has fio explicitly use an mfence instruction instead of
__sync_synchornize() in the CPU clock test code on x86_64 platforms in
order to allow the CPU clock test to pass on AMD platforms.
Reviewed-by: Jens Axboe <axboe@kernel.dk>
Link: https://lore.kernel.org/r/20240227155856.5012-1-vincent.fu@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
We have had a lot of failures installing sphinx on macOS. The latest
failure suggested using pip instead of homebrew to install sphinx. So
let's try that.
https://github.com/axboe/fio/actions/runs/8004639029/job/21863677360?pr=1727
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
* 'fix-tests-cfi' of https://github.com/mikoxyz/fio:
options: declare *__val as long long
t/io_uring: use char * for name arg in detect_node
t/io_uring: include libgen.h
|
|
Fixes CFI as the function signatures will now match with parse.h
Signed-off-by: Miko Larsson <mikoxyzzz@gmail.com>
|
|
Fixes the following compiler warning:
warning: passing 'const char *' to parameter of type 'char *' discards
qualifiers [-Wincompatible-pointer-types-discards-qualifiers]
Signed-off-by: Miko Larsson <mikoxyzzz@gmail.com>
|
|
This fixes the build with musl + clang >=15; musl doesn't declare
basename() anywhere else, and clang >=15 doesn't allow implicit
declarations.
Fixes: 4b9e13dc27fb (t/io_uring: support NUMA placement)
Signed-off-by: Miko Larsson <mikoxyzzz@gmail.com>
|
|
nofiles and depth are 32-bit integers. So we shouldn't use 64-bit
conversion functions and casts. The current code actually works fine on
little-endian platforms since the conversion is a noop but this is
broken on big-endian platforms.
Fixes: 94a6e1bb ("Fix verify state for multiple files")
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Add the pass-through multi-range trim tests to the automated test suite.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
This test script contains some regression tests for existing
functionality and also some tests for the new feature.
The multi-range trim tests basically count the number of requests
submitted and make sure that they are consistent with the block sizes
and number of ranges per request.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Move it under clat_prio_index, which already has a 6 byte gap. This
saves 8 bytes total in struct io_u, which is always nice.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Update the io_uring_cmd ioengine to support multiple ranges for trim.
This includes allocating buffer for multiple ranges, and changes to
the nvme trim helper functions.
Add an example on how to use multi range trim.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Link: https://lore.kernel.org/r/20240215151812.138370-4-ankit.kumar@samsung.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Remove io_u dependency while calculating start and number of lba.
This is a prep patch for multi-range trim support where an io_u entry
will contain multiple start offset and buffer len.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Link: https://lore.kernel.org/r/20240215151812.138370-3-ankit.kumar@samsung.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
NVMe specification allow multiple ranges for the dataset management
commands. Currently the block ioctl only allows a single range for
trim, however multiple ranges can be specified using nvme character
device.
Add an option num_range to send multiple range per trim request, which
only works if the data direction is solely trim i.e. trim or randtrim.
Add FIO_MULTI_RANGE_TRIM as the ioengine flag, to restrict the usage of
this new option.
For multi range trim request this modifies the way IO buffers are used.
The buffer length will depend on number of trim ranges and the actual
buffer will contains start and length of each range entry.
This increases fio server version (FIO_SERVER_VER) to 103.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Link: https://lore.kernel.org/r/20240215151812.138370-2-ankit.kumar@samsung.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Rather than do this math every time, just calculate it at setup time
and use that.
Note that there's something wonky with how the depths are distributed,
it's quite common to see the first file eat the majority of the queue
depth, causing multiple file runs to be both slower than they should be,
and with very unevenly distributed depths. To be investigated.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
It's still useful to test on devices that may trigger a single or
few IO errors, just log them and ignore them in the IOPS calculations.
And for stats calculations.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Add t/jobs/t0029.fio to test that the loops option works together with
read-verify workloads.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/20240214122008.4123286-3-shinichiro.kawasaki@wdc.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
The commit 191d6634e8a6 ("verify: fix bytes_done accounting of
experimental verify") introduced td->bytes_verified to separate the
verified bytes from the read bytes in td->bytes_done[]. This fixed the
issue of experimental verify feature. However, it caused another issue.
When the verify workload does only read and does not do write, the read
bytes in td->bytes_done[] is no longer updated and always zero. This
zero value is returned from do_io() to thread_main() in the bytes_done
array. If the read bytes is zero, thread_main() marks the job to
terminate and it makes the loops option ignored. For example, the job
below should do 8k read, but it does only 4k read.
[global]
filename=/tmp/fio.test
size=4k
verify=md5
[write]
rw=write
do_verify=0
[read]
stonewall=1
rw=read
loops=2
do_verify=1
To make the loops option work together with the read-verify workloads,
modify io_u_update_bytes_done(). After updating td->bytes_verified,
check if the workload does not write. If so, do not return from
io_u_update_bytes_done() and update td->bytes_done[] for DDIR_READ in
the following code.
Fixes: 191d6634e8a6 ("verify: fix bytes_done accounting of experimental verify")
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/20240214122008.4123286-2-shinichiro.kawasaki@wdc.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Reviewed-by: Jens Axboe <axboe@kernel.dk>
Link: https://lore.kernel.org/r/20240213153315.134202-6-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Add checks to ensure that the correct block size and metadata size is
passed by the user.
Disable normal verify checks when end to end data protection checks
are enabled because of CRC conflict.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Reviewed-by: Jens Axboe <axboe@kernel.dk>
Link: https://lore.kernel.org/r/20240213153315.134202-5-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
This patch enables support for protection information to xnvme
ioengine.
This adds 4 new ioengine specific options
* pi_act - Protection information action. Default: 1
* pi_chk - Can be set to GUARD, APPTAG or REFTAG
* apptag - Sets apptag field of command dword 15
* apptag_mask - Sets apptag_mask field of command dword 15
For the sake of consistency these options are the same as the ones used
by io_uring_cmd ioengine and SPDK's external ioengine.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Reviewed-by: Jens Axboe <axboe@kernel.dk>
Link: https://lore.kernel.org/r/20240213153315.134202-4-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
This enables support for separate metadata buffers with xnvme ioengine.
This is done by providing xnvme specific option md_per_io_size, which
for the sake of consistency is the same option used by io_uring_cmd
engine and SPDK's external ioengine.
Bump up the required xnvme support to v0.7.4
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Reviewed-by: Jens Axboe <axboe@kernel.dk>
Link: https://lore.kernel.org/r/20240213153315.134202-3-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Reviewed-by: Jens Axboe <axboe@kernel.dk>
Link: https://lore.kernel.org/r/20240213153315.134202-2-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
When a fio job ends, it cleans up by flushing any accumulated latency log
data for jobs with log_avg_msec enabled. This means that the final logging
interval may be different from what was specified by log_avg_msec. In
some cases there may even be duplicate timestamps. Add an explanation
for this phenomenon to the documentation.
During job cleanup it's possible to simply suppress the final log entry
if log_avg_msec has not passed since the previous log entry was recorded
but this throws away data that some users may depend on. For instance, a
55s job with log_avg_msec=10000 would have no long entry for the final
5s if we suppressed the final log entry.
Users concerned about final log entries with duplicate timestamps should
just ignore the second entry since it is likely based on only a handful
of I/Os.
Duplicate log entry example:
$ sudo ./fio --name=test --iodepth=2 --ioengine=libaio --time_based --runtime=5s --log_avg_msec=1000 --write_lat_log=test --filename=/dev/vda --direct=1
test: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=2
fio-3.36-61-g9cfa-dirty
Starting 1 process
Jobs: 1 (f=1): [R(1)][100.0%][r=250MiB/s][r=64.0k IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=1490: Mon Feb 12 12:19:13 2024
read: IOPS=63.6k, BW=248MiB/s (260MB/s)(1242MiB/5001msec)
slat (nsec): min=691, max=37070, avg=1272.54, stdev=674.26
clat (usec): min=4, max=1731, avg=29.83, stdev= 7.03
lat (usec): min=15, max=1734, avg=31.10, stdev= 7.19
clat percentiles (usec):
| 1.00th=[ 23], 5.00th=[ 25], 10.00th=[ 26], 20.00th=[ 27],
| 30.00th=[ 28], 40.00th=[ 29], 50.00th=[ 30], 60.00th=[ 31],
| 70.00th=[ 32], 80.00th=[ 33], 90.00th=[ 35], 95.00th=[ 37],
| 99.00th=[ 41], 99.50th=[ 43], 99.90th=[ 58], 99.95th=[ 74],
| 99.99th=[ 104]
bw ( KiB/s): min=244464, max=258112, per=100.00%, avg=254410.67, stdev=4788.90, samples=9
iops : min=61116, max=64528, avg=63602.67, stdev=1197.23, samples=9
lat (usec) : 10=0.01%, 20=0.06%, 50=99.76%, 100=0.16%, 250=0.01%
lat (usec) : 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%
cpu : usr=11.46%, sys=18.20%, ctx=159414, majf=0, minf=49
IO depths : 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=317842,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=2
Run status group 0 (all jobs):
READ: bw=248MiB/s (260MB/s), 248MiB/s-248MiB/s (260MB/s-260MB/s), io=1242MiB (1302MB), run=5001-5001msec
Disk stats (read/write):
vda: ios=311248/0, sectors=2489984/0, merge=0/0, ticks=7615/0, in_queue=7615, util=98.10%
$ cat test_lat.1.log
1000, 31907, 0, 0, 0
2000, 30705, 0, 0, 0
3000, 30738, 0, 0, 0
4000, 31196, 0, 0, 0
5000, 30997, 0, 0, 0
5000, 31559, 0, 0, 0
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Currently recording log entries for IOPS and BW logging with
log_avg_msec enabled only happens when a thread has completed its ramp
time and is in the TD_RUNNING or TD_VERIFYING run states.
It may happen that a final bandwidth or IOPS log entry is missed
when a job transitions from TD_RUNNING/TD_VERIFYING to TD_FINISHING
before the helper thread has a chance to calculate a final log entry.
This patch expands the run states where logging is permitted, allowing
log entries to be recorded for jobs in the TD_FINISHING or TD_EXITED
states. Each job cleans itself up and typically transitions quickly from
TD_FINISHING to TD_EXITED. The main fio backend thread carries out the
transition from TD_EXITED to TD_REAPED.
The window during which a job is in the TD_FINISHING and TD_EXITED
states is short, so measurements should still be reasonablly accurate.
I tested these patches with the following job:
fio --name=test --ioengine=null --time_based --runtime=3s --filesize=1T \
--write_iops_log=test --write_bw_log=test --log_avg_msec=1000 \
&& cat test_iops.1.log && cat test_bw.1.log
Before this patch series 10/10 trials had missing log entries. With only
the helper_thread change in the preceding patch 3/10 trials had missing
log entries. With this entire patch series, only 1/10 trials had missing
log entries.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
When helper_thread_exit() is called, it uses two means to tell
helper_thread_main() to stop. It sends an A_EXIT message via the pipes
used for inter-process communication and it also sets the exit flag in
struct helper_data.
When helper_thread_main() receives the A_EXIT message, it immediately
breaks out of its main loop without carrying out any further activity
such as recording final log entries. This leads to missing final log
entries when log_avg_msec is enabled.
Removing the A_EXIT message helps resolve this issue in some cases as
with its removal the actions in the body of the loop have an opportunity
to be taken before checking the flag in struct helper_data.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Instead of recording a timestamp once before iterating through all the
threads to check if we should add a new bw or iops log measurement,
record a new timestamp before checking each thread.
We were already querying the time anyway for the mtime_since_now() call.
We might as well have it available locally for more accurate logging.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
* 'vsock' of https://github.com/MPinna/fio:
Add support for VSOCK to engine/net.c
|
|
* configure: add option to enable/disable vsock support
* engines/net.c: add vsock support
The VSOCK address family facilitates communication between virtual
machines and the host they are running on.
The addressing is formed by 2 integers: <CID, port>
- CID: Context ID, it is the ID assigned to the VM
0, 1, 2 CIDs are reserved:
0 - hypervisor CID (rarely used)
1 - local communication (loopback)
2 - host CID (the guest can always reach the host using CID=2)
- port: port number on 32bit to reach a specific process
* examples: add 3 simple job files for vsock (one sender, one receiver
and one that uses vsock loopback interface similar to
examples/netio.fio)
* fio.1: add vsock to supported protocols together with the required
parameters
* HOWTO.rst: add vsock to supported protocols together with the required
parameters
Signed-off-by: Marco Pinna <marco.pinn95@gmail.com>
|
|
vc->hdr_num and len are both 32 bits wide and their product will be a
32-bit result. So any overflow will be lost. Cast hdr_num to unsigned
long long so that nothing is lost if the product overflows a 32-bit
integer.
This fixes the following issue reported by Coverity.
** CID 486274: Integer handling issues (OVERFLOW_BEFORE_WIDEN)
/verify.c: 347 in log_verify_failure()
________________________________________________________________________________________________________
*** CID 486274: Integer handling issues (OVERFLOW_BEFORE_WIDEN)
/verify.c: 347 in log_verify_failure()
341 uint32_t len;
342 struct thread_data *td = vc->td;
343
344 offset = vc->io_u->verify_offset;
345 if (td->o.verify != VERIFY_PATTERN_NO_HDR) {
346 len = hdr->len;
>>> CID 486274: Integer handling issues (OVERFLOW_BEFORE_WIDEN)
>>> Potentially overflowing expression "vc->hdr_num * len" with type "unsigned int" (32 bits, unsigned) is evaluated using 32-bit arithmetic, and then used in a context that expects an expression of type "unsigned long long" (64 bits, unsigned).
347 offset += vc->hdr_num * len;
348 } else {
349 len = vc->io_u->buflen;
350 }
351
352 log_err("%.8s: verify failed at file %s offset %llu, length %u"
Fixes: 9c8b90ae ("fix wrong offset for VERIFY_PATTERN_NO_HDR")
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
* 'fix-offset' of https://github.com/onkrasnov/fio:
fix wrong offset for VERIFY_PATTERN_NO_HDR
https://github.com/axboe/fio/pull/1709
|
|
The total number of ZBD tests in test-zbd-support script has grown
considerably over the years and zoned drive capacity has significantly
increased as well. Today, the test run duration may reach one hour for
large drives. If a terminal session failure happens during a run, it
is more efficient to restart the tests from the point where the last
run stopped rather than from the beginning.
Add -s option to the script command line to specify the starting
test number.
Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Reviewed-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/20240206105755.214891-6-dmitry.fomichev@wdc.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
ZBD unit tests in t/zbd/test-zbd-support currently assume that the
drive that is being tested supports unrestricted reads, i.e. reads
that (partially or entirely) occur above the write pointer. This is
always the case with ZBD core code because Linux kernel rejects zoned
devices with restricted reads. However, libzbc ioengine does support
such devices.
The restricted/unrestricted reads feature is controlled by URSWRZ
device bit ("Unrestricted Reads of Sequential Write Required Zones")
which, depending on the device design, can be hard-coded to be reported
as 1 or 0 or it can be made configurable via MODE SET or SET FEATURES
commands. The unit tests need to behave correctly with any URSWRZ bit
value reported by the device if libzbc ioengine is used for testing.
Test #4 in the test script currently expects the device to have
unrestricted SWR zone reads. This test is guaranteed to fail if
the script is run against a drive that reports URSWRZ=0 with libzbc
ioengine.
Check if the drive has unrestricted read support disabled and process
the outcome of test #4 accordingly.
Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Reviewed-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/20240206105755.214891-5-dmitry.fomichev@wdc.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
zone_lock() function contains the debug code to verify that the zone
being locked belongs to the job's working zone range. Clean up this
code by using a previously defined helper for calculating the zone
index to check.
Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Reviewed-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/20240206105755.214891-4-dmitry.fomichev@wdc.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
BLKREPORTZONE may fail because of a variety of reasons.
Log both the return code and errno when this ioctl fails.
Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Reviewed-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/20240206105755.214891-3-dmitry.fomichev@wdc.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
The following assert has been observed to be triggered
if the I/O offset + I/O size exceeds the device capacity in a
sequential read workload -
“zbd.c:110: zone_lock: Assertion `f->min_zone <= nz && nz < f->max_zone' failed.”
The code in zbd_zone_align_file_sizes() rounds down the I/O size to
avoid these situations, but it is bypassed if
td->o.td_ddir == TD_DDIR_READ.
Avoid this issue by modifying zbd_zone_align_file_sizes() to round down
the I/O size for read I/Os and leave the I/O offset untouched.
Fixes: bfbdd35b3e8f ("Add support for zoned block devices")
Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Reviewed-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/20240206105755.214891-2-dmitry.fomichev@wdc.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
|
|
Add an example of how to use cmdprio_bssplit with Command Duration Limits.
Signed-off-by: Niklas Cassel <cassel@kernel.org>
Link: https://lore.kernel.org/r/20240126181353.4151771-2-cassel@kernel.org
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Replace IO with I/O.
Signed-off-by: Niklas Cassel <cassel@kernel.org>
Link: https://lore.kernel.org/r/20240126181353.4151771-1-cassel@kernel.org
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
* 'coverity-fix' of https://github.com/ankit-sam/fio:
iolog: fix reported defect from coverity scan
|
|
Fix the two Null pointer dereferences issue reported by Coverity scan
Null pointer dereferences (FORWARD_NULL)
Dereferencing null pointer "l->td"
Null pointer dereferences (REVERSE_INULL)
Null-checking "p->td" suggests that it may be null, but it has already
been dereferenced on all paths leading to the check.
For aggregate read, write and trim bandwidth log, the setup_log function
gets called with NULL pointer reference for thread data. Thus before
dereferencing further we should check "l->td".
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
|
|
This didn't make it into the kernel just yet, and if/when it does, it'll
likely look a bit different than the prototype. Since it's dead code for
now, just prune it and we can always reintroduce it when kernel support
is there.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Link: https://lore.kernel.org/r/20240125110124.55137-3-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Add option log_window_value alias of log_max_value which reports
average, max or both the values. Retain backward compatibility by
allowing =0 and =1 values to specify avg and max values respectively.
There is no change to existing log formats while reporting only average
or max values.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Link: https://lore.kernel.org/r/20240125110124.55137-2-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Switch from actions/checkout@v3 to v4 and from
actions/upload-artifacts@v3 to v4.
This resolves the below warnings from GitHub Actions:
Node.js 16 actions are deprecated. Please update the following actions
to use Node.js 20: actions/checkout@v3. For more information see:
https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Node.js 16 actions are deprecated. Please update the following actions
to use Node.js 20: actions/upload-artifact@v3. For more information see:
https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
pkg-config doesn't need to have a cross prefixed version. It can be
pointed at alternate sysroots with environment variables like
PKG_CONFIG_SYSROOT_DIR. The configure script was already inconsistent
with using `pkg-config` in some places and `${cross_prefix}pkg-config`
in others. Make check_min_lib_version() and the gtk checks consistent
with the rest by dropping the `${cross_prefix}` usage with pkg-config.
Signed-off-by: Chris Packham <chris.packham@alliedtelesis.co.nz>
Link: https://lore.kernel.org/r/20240123202636.179467-1-chris.packham@alliedtelesis.co.nz
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
If verify_only is set we don't need to open the file with the O_RDWR
flagi for write workloads. So we should clear this flag. This will help
when the file is on a read-only file system.
Fixes: https://github.com/axboe/fio/issues/1681
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
GitHub increased the number of CPUs in its GitHub-hosted runners from
two to four for Linux and Windows. macOS remains at two CPUs. Stop
hard-coding the number of CPUs when we build fio and detect the number
at runtime.
https://github.blog/2024-01-17-github-hosted-runners-double-the-power-for-open-source/
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Link: https://github.com/axboe/fio/issues/1701
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
* 'group_reporting_indentation' of https://github.com/0mp/fio:
doc: group_reporting: Fix indentation and syntax
|
|
Use tabs consistently for the paragraphs describing the group_reporting
option. Also, make sure that there is no space between ":option:" and
"`group_reporting`".
Signed-off-by: Mateusz Piotrowski <0mp@FreeBSD.org>
|
|
check_result() erroneously returned True or False. Drop the return value
to be consistent with the parent and related classes.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Make sure we call the parent class' check_result() method to check the
return code, stderr output, etc.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Make sure we call the parent class' check_result() method to check the
return code, stderr output, etc.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Make sure we call the parent class' check_result() method to check the
return code, stderr output, etc.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
The previous commit fixed the issue of the unexpected write with options
zonemode=zbd, rw=randrw and rwmixwrite=0. Add a test to confirm the fix.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
Link: https://lore.kernel.org/r/20231220005846.1371456-3-shinichiro.kawasaki@wdc.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Since the commit fb0259fb276a ("zbd: Ensure first I/O is write for
random read/write to sequential zones"), fio issues write as the first
I/O when zonemode=zbd and rw=randrw options are specified. However, fio
issues the first write even when rwmixwrite=0 option is specified. Users
do not expect such write and it confuses the users. To avoid the
confusion, suppress the write by referring td->o.rwmix[DDIR_WRITE].
Fixes: fb0259fb276a ("zbd: Ensure first I/O is write for random read/write to sequential zones")
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
Link: https://lore.kernel.org/r/20231220005846.1371456-2-shinichiro.kawasaki@wdc.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
The engine sets -1 for some odd reason, where ->error fields are
supposed to be a positive value. Set it to EIO.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
* 'master' of https://github.com/preichl/fio:
engines/http: Drop unused varible
client/server: remove dead code
engines/rdma: remove dead code
|
|
Signed-off-by: Pavel Reichl <preichl@redhat.com>
|
|
The issue was found by Red Hat's OpenScanHub:
fio-3.35/server.c:1884:3: warning[deadcode.DeadStores]:
Value stored to 'extended_buf_wp' is never read
Signed-off-by: Pavel Reichl <preichl@redhat.com>
|
|
The issues was found by Red Hat's OpenScanHub:
fio-3.35/engines/rdma.c:279:3: warning[deadcode.DeadStores]:
Value stored to 'ret' is never read
Signed-off-by: Pavel Reichl <preichl@redhat.com>
|
|
* 'master' of https://github.com/preichl/fio:
engines/http: Fix memory leak
|
|
Found by Red Hat's OpenScanHub:
fio-3.35/engines/http.c:253: leaked_storage: Variable r going out of scope leaks the storage it points to.
Signed-off-by: Pavel Reichl <preichl@redhat.com>
|
|
* 'patch-3' of https://github.com/0mp/fio:
doc: Reference geom(4) for FreeBSD users
|
|
On FreeBSD, write access to Rank 1 geom providers is disabled
by default. However, it can be enabled with the `kern.geom.debugflags`
sysctl as documented in geom(4). Point users to that manual page
for smoother experience.
Signed-off-by: Mateusz Piotrowski 0mp@FreeBSD.org
|
|
If any error is observed for read requests, skip all end to end data
protection checks.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Link: https://lore.kernel.org/r/20231212144718.568406-1-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
* 'master' of https://github.com/bvanassche/fio:
Fall back to F_SET_RW_HINT if F_SET_FILE_RW_HINT is not supported
|
|
It can tricky to specify block sizes for devices formatted in extended
LBA mode because the block sizes are no longer powers of two. Add a
suggested block size in the error message when we abort a job because the
specified block size is not a multiple of the extended LBA size.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Linux kernel commit 7b12e49669c9 ("fs: remove fs.f_write_hint") removed
F_SET_FILE_RW_HINT support from the Linux kernel. Hence fall back to
F_SET_RW_HINT if F_SET_FILE_RW_HINT is not supported.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
|
|
* 'xnvme_includes' of https://github.com/safl/fio:
engines/xnvme: only include entry-header ('libxnvme.h')
|
|
This changes how the xNVMe fio io-engine consumes the xNVMe library by
only including the library-entry header "libxnvme.h".
From version 0.7.0 the xNVMe API headers are refactored to drop header
guards on the individual headers and abide by the idiom of "headers must
not include other headers".
The exception is the library-entry header "libxnvme.h". The
library-entry-header includes all headers provided with xNVMe, which is
a convenient approach to consuming the library. One where, in case the
API namespace grows or shrinks, then the xNVMe fio io-engine need not
change how it includes xNVMe.
However, since fio has consumed the main-entry header and individual
headers, xNVMe has held back on removing the guards on _nvm, _zns, and
_spec to avoid breaking the xNVMe fio engine. They will eventually be
deprecated. Thus, this change to consume xNVMe in the manner intended
from version v0.7.0 and onwards.
Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
|
|
From time to time we see failures (especially on Windows) with this test
because one of its jobs does not run long enough. Make the jobs run for
this test time_based so that they run for a minimum of 12s.
Example error output:
DEBUG:root:sample 0: job1 iops=343858.0 job2 iops=1719327.0 job3 iops=3438690.0 job3/job2=2.000 job3/job1=10.000
DEBUG:root:sample 1: job1 iops=682225.0 job2 iops=3411146.0 job3 iops=6822273.0 job3/job2=2.000 job3/job1=10.000
DEBUG:root:sample 2: job1 iops=1019335.0 job2 iops=5096697.0 job3 iops=10193375.0 job3/job2=2.000 job3/job1=10.000
DEBUG:root:sample 3: job1 iops=1367550.0 job2 iops=6837771.0 job3 iops=13675524.0 job3/job2=2.000 job3/job1=10.000
DEBUG:root:sample 4: job1 iops=1704137.0 job2 iops=8520706.0 job3 iops=17041393.0 job3/job2=2.000 job3/job1=10.000
DEBUG:root:sample 5: job1 iops=2056314.0 job2 iops=10281595.0 job3 iops=20563173.0 job3/job2=2.000 job3/job1=10.000
DEBUG:root:sample 6: job1 iops=2399343.0 job2 iops=11996744.0 job3 iops=23993468.0 job3/job2=2.000 job3/job1=10.000
DEBUG:root:Test 12 exception:
Traceback (most recent call last):
File "D:\a\fio\fio\t\fiotestlib.py", line 465, in run_fio_tests
test.check_result()
File "D:\a\fio\fio\t\run-fio-tests.py", line 180, in check_result
iops3 = iops3 + float(iops_files[2][i].split(',')[1])
Test 12 FAILED: list index out of range t0012.fio
IndexError: list index out of range
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Add a warning about the file open mode used when per_job_logs is set to
false. Log files are opened in append mode, so if the files already
exist, the previous contents will not be overwritten.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
On the client side log files were being overwritten when per_job_logs
was set to false because of the flags used when log files were opened.
Add per_job_logs to the on-wire protocol so that the client can adjust
the flags and open files in append mode when per_job_logs is set to
false.
Fixes: https://github.com/axboe/fio/issues/1032
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
* 'thinkcycles-parameter' of https://github.com/cloehle/fio:
fio: Introduce new constant thinkcycles option
|
|
The thinkcycles parameter allows to set a number of cycles to spin between
requests to model real-world applications more realistically
The thinktime parameter family can be used to model an application processing
the data to be able to model real-world applications more closely.
Unfortunately this is currently set per constant time and therefore is affected
by CPU frequency settings or task migration to a CPU with different capacity.
The new thinkcycles parameter closes that gap and allows specifying a constant
number of cycles instead, such that CPU capacity is taken into account.
Signed-off-by: Christian Loehle <christian.loehle@arm.com>
|
|
The xNVMe backend supports FDP commands for userspace drivers
such as SPDK. Enable support in the xnvme ioengine.
Update the xnvme fdp example file accordingly.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Link: https://lore.kernel.org/r/20231102135928.195372-1-ankit.kumar@samsung.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
* 'pi-perf' of https://github.com/ankit-sam/fio:
crct10: use isa-l for crc if available
|
|
isa-l provides fast implementation for various polynomials.
This will be only used for end to end data protection, and has
a significant impact on performance.
See: https://github.com/intel/isa-l
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
|
|
* 'englist' of https://github.com/vt-alt/fio:
nfs: Fix incorrect engine registering for '--enghelp' list
|
|
We round up the iodepth to the next highest power of 2. So io_u->index
can be greater than the iodepth specified by the user. Make sure we
allocate enough of the buffers used to store the ranges for async trim
commands when the iodepth specified by the user is not a power of 2.
Fixes: 4885a6eba420ce216e4102df3e42229e167d1b7b ("engines/io_uring_cmd:
make trims async")
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
`ioengine` from `nfs` (internal) engine is incorrectly exported thus
overriding its value in constructor callbacks of other external engines,
that are used for registering engine for listing with `--enghelp`.
Because flist is unsafe to double adding it also making `engine_list` to
become corrupt and causing infinite loop or abnormal list termination
when printing engine list.
Issue: https://github.com/axboe/fio/issues/1655
Fixes: 9326926b ("NFS engine")
Signed-off-by: Vitaly Chikunov <vt@altlinux.org>
|
|
* 'spellingfixes-2023-10-23' of https://github.com/proact-de/fio:
Various spelling fixes.
|
|
Most of them have been reported by Debian's Lintian tool.
Signed-off-by: Martin Steigerwald <martin.steigerwald@proact.de>
|
|
* 'fix-riscv64-cpu-clock' of https://github.com/gilbsgilbs/fio:
riscv64: get clock from `rdtime` instead of `rdcycle`
|
|
`rdcycle` pseudo-instruction accesses the "cycle CSR" which holds the
real count of CPU core clock cycles [1]. As this leaves room for
side-channel attacks, access to this register from userland might be
forbidden by the kernel, which results in a SIGILL [2].
Anyhow, it seems that the actual usage of the `get_cpu_clock` function
in fio is about getting a wall-clock rather than the actual CPU core
clock (for instance, x86 uses `rdtsc`), so this is technically a bug.
The "time CSR" is the proper register to track time on riscv64. Also,
the "time CSR" is more likely to be available from userspace and not
cause a crash.
[1] RISC-V ISA Section 10.1: https://github.com/riscv/riscv-isa-manual/releases/download/Ratified-IMAFDQC/riscv-spec-20191213.pdf
[2] https://lore.kernel.org/all/YxIzgYP3MujXdqwj@aurel32.net/T/
Signed-off-by: N. Le Roux <gilbsgilbs@gmail.com>
Signed-off-by: Gilbert Gilb's <gilbsgilbert@gmail.com>
|
|
* 'master' of https://github.com/michalbiesek/fio:
riscv64: add syscall helpers
|
|
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Use syscall helpers to optimized io_uring_enter(2) calls, eliminating
the need for libc functions. These wrappers are adapted from liburing.
Signed-off-by: michalbiesek <michalbiesek@gmail.com>
|
|
* 'master' of https://github.com/shailevi23/fio:
helper_thread: fix pthread_sigmask typo.
configure: improve pthread_sigmask detection.
|
|
* 'fix_issue_1642' of https://github.com/zqs-Oppenauer/fio:
fix assert failed when timeout during call rate_ddir.
|
|
Adding DDIR_TIMEOUT in enum fio_ddir, and rate_ddir returns it when fio timeouts.
set_io_u_file will directly break out of the loop, and fill_io_u won't be called,
which causes assert to fail in rate_ddir, because td->rwmix_ddir is DDIR_INVAL.
Signed-off-by: QingSong Zhu zhuqingsong.0909@bytedance.com
|
|
Signed-off-by: Shai Levy <shailevy23@gmail.com>.
|
|
On Windows system, pthread_sigmask is defined as a noop which will
trigger unused variable warning for sigmask.
By triggering the same warning in the configure script, we make
CONFIG_PTHREAD_SIGMASK undefined in the Windows msys2 build.
Signed-off-by: Shai Levy <shailevy23@gmail.com>.
|
|
The documentation build on macOS started failing because of errors with
the pygments and certifi modules. Homebrew is not automatically
installing pygments and python-certifi which are listed as packages that
sphinx-doc depends on because they are already present in the runner
image at the required versions (2.16.1 and 2023.7.22, respectively).
Explicitly installing the two packages bumps the versions to slightly
newer ones (2.16.1_1 and 2023.7.22_1, respectively). This appears to
resolve the documentation build problem.
https://formulae.brew.sh/formula/sphinx-doc
https://github.com/axboe/fio/actions/runs/6533001329/job/17739452911#step:13:155
https://github.com/vincentkfu/fio/actions/runs/6535039949/job/17743571376#step:13:148
https://github.com/vincentkfu/fio/actions/runs/6535229986/job/17744177918#step:6:10
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
When zonemode=zbd option is not specified, random writes to zoned block
devices fail because writes to sequential write required zones shall
happen only at write pointers. Randomly chosen write addresses do not
match with the write pointers, then fail. On such failures, fio prints
out the message below and tell users how to avoid the failures:
"fio: first I/O failed. If .* is a zoned block device, consider --zonemode=zbd".
The test case 45 in t/zbd/test-zbd-support confirms the message is
printed when the first random write command to a sequential write
required zone fails. However, the random write can succeed very rarely
since the randomly chosen write address can be same as the write pointer
address. For example, a zoned block device with 1MB zone size with 4KB
block size device can have the first random write at write pointer with
ratio of 4KB/1MB = 1/256. This causes sporadic test case failures.
Avoid the failures by two changes. Firstly, change the random write
range from a zone to whole sequential write required zones to reduce the
failure ratio. Secondly, repeat the test if the message is not printed
by the accidental write success. As the test repeated, failure ratio is
multiplied and the failure ratio becomes as small as it can be ignored.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/20231006071320.425270-1-shinichiro.kawasaki@wdc.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
* 'fix-stat-overflow' of https://github.com/stilor/fio:
Handle 32-bit overflows in disk utilization stats
Change memcpy() calls to assignments
|
|
Linux prints [1] some of the values reported in block device's `stat`
sysfs file as 32-bit unsigned integers. fio interprets them as 64-bit
integers when reading that sysfs file and performs further arithmetics
on them in 64-bits. If the reported value overflows during fio run,
a huge bogus value is reported in the "disk utilization" block instead.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/block/genhd.c#n962
Signed-off-by: Alexey Neyman <aneyman@google.com>
|
|
This is to avoid triggering a spurious warning caused by [1], which
is triggered by the next commit in chain (unrelated change in
update_io_tick_disk()).
[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111696
Signed-off-by: Alexey Neyman <aneyman@google.com>
|
|
We store iolog timestamps as 64-bit unsigned integers but when we print
timestamps in the logs we type cast them to unsigned longs. This is fine
on platforms with 64-bit longs but on platforms like Windows with 32-bit
longs this truncates the timestamps. Truncation causes problems
especially when options such as --log_unix_epoch are enabled because the
number of milliseconds since 01-01-1970 does not fit in a 32-bit
unsigned long.
Fix this by getting rid of the type cast and using PRIu64 as the format
specifier.
Fixes: https://github.com/axboe/fio/issues/1638
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
macOS 13 was released in Oct 2022 and the GitHub Actions image debuted
Apr 2023. Let's switch to this new platform.
It seems to work fine based on two runs testing the nice fix in the
previous patch.
https://github.com/vincentkfu/fio/actions/runs/6352923622/job/17256648185
https://github.com/vincentkfu/fio/actions/runs/6353159388/job/17257286332
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
nice returns the program's schedule priority. This can be a negative
value when there is no error condition. To check if nice is triggering
an error we have to check errno.
The most recent three macOS test failures appear to be due to this
problem.
https://github.com/axboe/fio/actions/runs/6347762639/job/17243247222
https://github.com/axboe/fio/actions/runs/6346019205/job/17239370410
https://github.com/axboe/fio/actions/runs/6241934089/job/16944981876
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
* 'fix_verify_block_offset' of https://github.com/ipylypiv/fio:
verify: Fix the bad pattern block offset value
|
|
We offset buf by header_size for pattern verification. Add header_size
to the mismatched buf offset to get the correct block offset value.
Signed-off-by: Igor Pylypiv <ipylypiv@google.com>
|
|
When write workloads run on zoned block devices, mq-deadline scheduler is
required to ensure write operations are sequential. To fulfill this
requirement, the test script t/zbd/test-zbd-support sets mq-deadline to
the sysfs attribute "queue/scheduler". However, this preparation does
not work when the write target device is a bio based device-mapper
device. The device is bio based then I/O scheduler does not work.
Setting mq-deadline to the sysfs attribute has no effect. On top of
that, the sysfs attribute "queue/scheduler" is no longer available for
bio based device-mapper devices since Linux kernel version v6.5.
To ensure mq-deadline scheduler for bio based device-mapper devices,
improve the helper function set_io_scheduler. If the sysfs attribute
"queue/scheduler" is available, use it. Otherwise, check if the test
device is a zoned device-mapper (linear, flakey or crypt). If so, set
mq-deadline scheduler to destination devices of the device-mapper
device. To implement these, add some helper functions.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Link: https://lore.kernel.org/r/20230913015249.2226799-1-shinichiro.kawasaki@wdc.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
* 'fio_client_server_doc_fix' of https://github.com/pcpartpicker/fio:
Update docs to clarify how to pass job options in client mode
|
|
When run in client mode, fio does not pass any job options specified on
the command line to the fio server. When run in client mode, all job
options must be specified via local or remote job files. Update the docs
to indicate this to avoid end-user confusion.
Fixes #1629
Signed-off-by: Nick Neumann nick@pcpartpicker.com
|
|
Make sure we open the verify state file in binary mode to avoid any
possible conversion of NL to CR+NL. This is the same fix we did for
1fb215e991d260a128e35d761f6850e8d9e4c333.
Fixes: https://github.com/axboe/fio/issues/1631
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Fill the application and reference tag field for read and write
command only when pi_chk has the relevant bit set.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
For extended logical block sizes we cannot use verify when end to end
data protection checks are enabled. The CRC field in PI section of
data buffer creates conflict during verify phase.
The verify check is also redundant as end to end data protection already
ensures data integrity. So disallow use of verify for this case.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
* 'pcpp_epoch_fixing_2' of https://github.com/PCPartPicker/fio:
Make log_unix_epoch an official alias of log_alternate_epoch
Record job start time to fix time pain points
|
|
log_alternate_epoch was introduced along with
log_alternate_epoch_clock_id, and generalized the idea of
log_unix_epoch. Both options had the same effect. So we make
log_unix_epoch an official alias of log_alternate_epoch, instead of
maintaining both redundant options.
Signed-off-by: Nick Neumann nick@pcpartpicker.com
|
|
Add a new key in the json per-job output, job_start, that records the
job start time obtained via a call to clock_gettime using the clock_id
specified by the new job_start_clock_id option. This allows times of fio
jobs and log entries to be compared/ordered against each other and
against other system events recorded against the same clock_id.
Add a note to the documentation for group_reporting about how there are
several per-job values for which only the first job's value is recorded
in the json output format when group_reporting is enabled.
Fixes #1544
Signed-off-by: Nick Neumann nick@pcpartpicker.com
|
|
* 'pcpp_parse_nr_fix' of https://github.com/PCPartPicker/fio:
Add basic error checking to parsing nr from rw=randrw:<nr>, etc
|
|
Previously this was parsed by just doing atoi(). This returns 0 or has
undefined behavior in error cases.
Silently getting a 0 for nr is not great. In fact, 0 (or less) should
likely not be allowed for nr; while the code handles it, the effective
result is that the randomness is gone - all I/O becomes sequential. It
makes sense to prohibit 0 as an nr value in the random case.
We leverage str_to_decimal to do our parsing instead of atoi. It isn't
perfect, but it is a lot more resilient than atoi, and used in other
similar places. We can then return an error when parsing fails, and also
return an error when the parsed numeric value is outside of the ranges
that can be stored in the unsigned int used for nr, along with when nr
is 0.
Fixes #1622
Signed-off-by: Nick Neumann nick@pcpartpicker.com
|
|
* 'master' of https://github.com/michalbiesek/fio:
Add RISC-V 64 support
|
|
Signed-off-by: Michal Biesek <michalbiesek@gmail.com>
|
|
Add missing io_uring_cmd ioengine options to fiograph config.
Add two example job files for the protection information options.
These include one for DIF i.e. extended LBA data size, and the other
for DIX i.e. separate metadata buffer case.
Add the corresponding fiograph diagram for these.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Link: https://lore.kernel.org/r/20230816094616.132240-1-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Not really important as we're exiting anyway, but this silences some
of the static checkers that like to complain about this sort of
thing.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Carry out tests of the code supporting end-to-end data protection via
the io_uring_cmd ioengine's nvme command type.
The test script detects the available protection information formats
supported by the target device. Then for each of these configurations,
the script formats the device and runs a series of tests.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Check a test config variable to skip a test at runtime. This will be
used to skip a test when the test runner determines that it should not
be run.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Generate and verify protection information for 64 bit guard format, for
the nvme backend of io_uring_cmd ioengine. The support is there for
both the cases where metadata is transferred in separate buffer, or
transferred at the end of logical block creating an extended logical
block.
This support also takes into consideration when protection information
resides in last or first 16 bytes of metadata.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Link: https://lore.kernel.org/r/20230814145747.114725-11-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Pull the 48 bit helpers, required for supporting 48 bit reference tags.
Add GPL 2.0 license to nvme.c and nvme.h files.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Link: https://lore.kernel.org/r/20230814145747.114725-10-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Pull the required nvme crc64 apis and table from the linux kernel. This
is required to generate and verify 64 bit guard tag for nvme backend
of io_uring_cmd ioengine.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Link: https://lore.kernel.org/r/20230814145747.114725-9-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Generate and verify protection information for 16 bit guard format, for
the nvme backend of io_uring_cmd ioengine. The support is there for
both the cases where metadata is transferred in separate buffer, or
transferred at the end of logical block creating an extended logical
block.
This support also takes into consideration when protection information
resides in last or first 8 bytes of metadata.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Link: https://lore.kernel.org/r/20230814145747.114725-8-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Pull the required crc16 t10 files from the linux kernel. This is
required to generate and verify guard tag for nvme backend of
io_uring_cmd ioengine.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Link: https://lore.kernel.org/r/20230814145747.114725-7-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
io_uring_cmd ioengine requires engine data to store nvme protection
information data.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Link: https://lore.kernel.org/r/20230814145747.114725-6-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
This patch enables support for protection information to nvme command
backend of io_uring_cmd ioengine. The patch only supports protection
information action bit set to 1, for read and write operation.
This adds 4 new ioengine specific options
* pi_act - Protection information action. Default: 1
* pi_chk - Can be set to GUARD, APPTAG or REFTAG
* apptag - Sets apptag field of command dword 15
* apptag_mask - Sets apptag_mask field of command dword 15
For the sake of consistency these options are the same as the ones used
by SPDK's external ioengine.
For pi_act=1, if namespace is formatted with metadata size equal to
protection information size, the nvme controller inserts and removes
protection information for write and read command respectively.
Added a check so that fio doesn't send metadata for such cases.
Storage tag support is not present, so return an error for that.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Link: https://lore.kernel.org/r/20230814145747.114725-5-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
This patch enables support for separate metadata buffer with
io_uring_cmd ioengine. As we are unaware of metadata size during buffer
allocation, we provide an option md_per_io_size. This option must be
used to specify metadata buffer size for single IO, if namespace is
formatted with a separate metadata buffer.
For the sake of consistency this is the same option as used by SPDK's
external ioengine.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Link: https://lore.kernel.org/r/20230814145747.114725-4-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
This is a prep patch to keep number of arguments for fio_nvme_get_info
in check. The follow up patches will enable metadata, protection info.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Link: https://lore.kernel.org/r/20230814145747.114725-3-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
This change ensures the error is propogated to upper layers to make fio
exit with a non-zero return code.
Add filename for errors when block size is not a multiple of logical
blocks.
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Link: https://lore.kernel.org/r/20230814145747.114725-2-ankit.kumar@samsung.com
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
The --bandwidth-log command-line option instructs fio to generate
aggregate bandwidth log files. These measurements are recorded by the
code generating the eta status line. When eta is disabled the aggregate
bandwidth log measurements are not calculated. Change the eta code to
record the measurements even when eta is not needed.
eta is disabled under these conditions
- explicitly with --eta=never
- STDOUT is not a TTY (shell redirection, nohup, etc)
- output format excludes normal output
Fixes: https://github.com/axboe/fio/issues/1599
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Instead of recording fio test commands as a single very long line, put
each option on its own line to make the command easier for humans to
digest.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
Make the filenames for the nvmept artifacts start with nvmept instead
of readonly.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
|
|
* 'master' of https://github.com/min22/fio:
iolog.c: fix inaccurate clat when replay trace
|
|
* 'improment/constness' of https://github.com/dpronin/fio:
use 'const' where it is required
|
|
protect variables and parameters from programmer's point of view with
'constness'
Signed-off-by: Denis Pronin <dannftk@yandex.ru>
|
|
This reverts commit 913028e97ceedcf2cf1ec6ec32228b3c50e7337c.
This commit is causing the static analyzers to freak out, and also
crashes on Windows. Revert it for now.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
* 'td-eo-double-free-fix' of https://github.com/dpronin/fio:
correctly free thread_data options at the topmost parent process
|
|
* 'master' of https://github.com/dpronin/fio:
fix missing headers in multiple files
|
|
* 'io_uring' of https://github.com/dpronin/fio:
io_uring engine: 'atomic_load_relaxed' instead of 'atomic_load_acquire'
|
|
motivation here is that we do not have here any explicit READ dependency
on atomic load because actually we just need in these places only
operation to perform atomically without any explicit barriers given by
memory model
Signed-off-by: Denis Pronin <dannftk@yandex.ru>
|
