[LTP] [PATCH v2 00/30] Introduce runtime and conver tests
Cyril Hrubis
chrubis@suse.cz
Tue May 3 19:46:48 CEST 2022
This patchset introduces a concept of runtime, which is described well
enough in the first patch of the series and converts all tests that were
previously using timeout API to runtime API.
The main change to the v1 of the patchset is that the runtime is now per
iteration, e.g. if we run the test for all_filesystem the runtime
applies to a single filesystem only. And at the same time the overall
test timeout is a sum of static timeout and runtime. The static part of
the timeout is meant to cover test teardown and any inaccuracy in the
test runtime accounting.
The patches that convert tests to runtime API fall into these classes:
- fzsync tests
- faster timeout tests
- I/O tests
- timer precision tests
- oom tests
- misc tests that run for some time (controlled by command line
parameter)
fzsync tests
============
These tests previously defined their runtime based on scale factor that
used slice of the default timeout (300s) for the test runtime.
Now the runtime is explicitly encoded in the test, which I find much
more user friendly because it's clear how long will the test run.
faster timeout tests
====================
There were several tests that set low timeout becuase the test may lock
up. In these cases the timeout was simply removed because the default
value (30s) should be small enough.
I/O tests
=========
The aiodio tests were modified so that their runtime is capped to
30 minutes by default. That means that even on very slow machine they
finish in a reasonable time.
timer precision tests
=====================
These test simply set the test timeout to be 110% of the sum of the
sleep_time * samples. That is to accomodate for the context switches and
upkeeping between the calls in the timer library.
oom tests
=========
Here we just set the .max_iteration_runtime to TST_RUNTIME_UNLIMITED,
which has the very same effect as .timeout = -1 previously had.
misc test
=========
In this case the test code was simplified quite a lot since the runtime
is now passed in a unified way and parsed in the test library.
TODO:
This patchset is reasonably complete in a sense that it removes the
timeout API at the end. Still there are a few things to consider:
- is reusing the -I parameter a good idea? Wouldn't adding new parameter
(-r) be better?
- there are quite likely tests that run for more than a second or a two
and should be made runtime aware
- anything else?
Cyril Hrubis (30):
Introduce a concept of max runtime
mtest06/mmap1: Convert to runtime
mtest06/mmap3: Convert to runtime
mtest01/mtest01: Convert to runtime
cve/cve-2015-3290: Convert to runtime
crypto/af_alg02: Convert to runtime.
crypto/pcrypt_aead01: Convert to runtime
syscalls/clock_gettime01: Remove useless timeout
syscalls/fanotify22: Remove useless timeout
syscalls/gettimeofday02: Convert to runtime.
syscalls/inotify06: Convert to runtime.
syscalls/inotify01: Remove now useless timeout
syscalls/perf_event_open03: Convert to runtime
syscalls/readv01: Remove now useless timeout
syscalls/tgkill03: Remove now unused timeout
syscalls/setsockopt09: Remove now useless timeout
syscalls/userfaultfd01: Remove now useless timeout
syscalls/move_pages12: Convert to runtime
syscalls/rt_sigqueueinfo01: Remove now useless timeout
mem/mallocstress: Convert to runtime
mem/{oom,min_free_kbytes}: Convert to runtime
crypto/af_alg02: Convert to runtime
fuzzy_sync: Convert to runtime
ltp-aiodio/dio_sparse,aiodio_sparse: Convert to runtime.
ltp-aiodio/read_checkzero: Remove
ltp-aiodio/dio_{truncate,append}: Convert to runtime
ltp-aiodio/dio_read: Convert to runtime
ltp-aiodio/aiodio_append: Convert to runtime
timer_test: Convert to runtime
tst_test: Remove timeout stubs
doc/user-guide.txt | 4 +
include/tst_fuzzy_sync.h | 19 +--
include/tst_test.h | 31 +++-
lib/newlib_tests/.gitignore | 5 +-
lib/newlib_tests/test10.c | 22 ---
lib/newlib_tests/test12.c | 21 ---
lib/newlib_tests/test13.c | 1 -
lib/newlib_tests/test_children_cleanup.c | 1 -
.../{test18.c => test_runtime01.c} | 10 +-
lib/newlib_tests/test_runtime02.c | 31 ++++
lib/tst_test.c | 144 +++++++++++++-----
lib/tst_timer_test.c | 12 +-
runtest/ltp-aiodio.part4 | 2 -
testcases/cve/cve-2015-3290.c | 8 +-
testcases/kernel/crypto/af_alg02.c | 4 +-
testcases/kernel/crypto/af_alg07.c | 1 +
testcases/kernel/crypto/pcrypt_aead01.c | 3 +-
testcases/kernel/io/ltp-aiodio/.gitignore | 1 -
.../kernel/io/ltp-aiodio/aiodio_append.c | 25 ++-
.../kernel/io/ltp-aiodio/aiodio_sparse.c | 26 +++-
testcases/kernel/io/ltp-aiodio/common.h | 12 +-
testcases/kernel/io/ltp-aiodio/dio_append.c | 4 +
testcases/kernel/io/ltp-aiodio/dio_read.c | 9 ++
testcases/kernel/io/ltp-aiodio/dio_sparse.c | 6 +-
testcases/kernel/io/ltp-aiodio/dio_truncate.c | 6 +
.../kernel/io/ltp-aiodio/read_checkzero.c | 84 ----------
testcases/kernel/mem/mtest01/mtest01.c | 14 +-
testcases/kernel/mem/mtest06/mmap1.c | 24 +--
testcases/kernel/mem/mtest06/mmap3.c | 12 +-
testcases/kernel/mem/mtest07/mallocstress.c | 11 +-
testcases/kernel/mem/oom/oom01.c | 2 +-
testcases/kernel/mem/oom/oom02.c | 2 +-
testcases/kernel/mem/oom/oom03.c | 2 +-
testcases/kernel/mem/oom/oom04.c | 2 +-
testcases/kernel/mem/oom/oom05.c | 2 +-
testcases/kernel/mem/thp/thp04.c | 1 +
.../kernel/mem/tunable/min_free_kbytes.c | 2 +-
testcases/kernel/pty/pty03.c | 2 +-
testcases/kernel/pty/pty05.c | 1 +
testcases/kernel/pty/pty06.c | 1 +
testcases/kernel/pty/pty07.c | 1 +
testcases/kernel/sound/snd_seq01.c | 2 +-
testcases/kernel/sound/snd_timer01.c | 1 +
testcases/kernel/syscalls/bind/bind06.c | 2 +-
.../syscalls/clock_gettime/clock_gettime01.c | 1 -
.../kernel/syscalls/fanotify/fanotify22.c | 1 -
.../syscalls/gettimeofday/gettimeofday02.c | 15 +-
testcases/kernel/syscalls/inotify/inotify06.c | 7 +-
testcases/kernel/syscalls/inotify/inotify09.c | 1 +
testcases/kernel/syscalls/inotify/inotify11.c | 1 -
.../kernel/syscalls/ipc/shmctl/shmctl05.c | 2 +-
.../kernel/syscalls/move_pages/move_pages12.c | 4 +-
.../perf_event_open/perf_event_open03.c | 15 +-
testcases/kernel/syscalls/readv/readv01.c | 1 -
.../rt_sigqueueinfo/rt_sigqueueinfo01.c | 1 -
testcases/kernel/syscalls/sendmsg/sendmsg03.c | 1 +
.../kernel/syscalls/setsockopt/setsockopt06.c | 2 +-
.../kernel/syscalls/setsockopt/setsockopt07.c | 1 +
.../kernel/syscalls/setsockopt/setsockopt09.c | 1 -
testcases/kernel/syscalls/tgkill/tgkill03.c | 1 -
.../syscalls/timerfd/timerfd_settime02.c | 1 +
.../syscalls/userfaultfd/userfaultfd01.c | 1 -
testcases/kernel/syscalls/writev/writev03.c | 2 +-
63 files changed, 339 insertions(+), 296 deletions(-)
delete mode 100644 lib/newlib_tests/test10.c
delete mode 100644 lib/newlib_tests/test12.c
rename lib/newlib_tests/{test18.c => test_runtime01.c} (58%)
create mode 100644 lib/newlib_tests/test_runtime02.c
delete mode 100644 testcases/kernel/io/ltp-aiodio/read_checkzero.c
--
2.35.1
More information about the ltp
mailing list