[LTP] [PATCH v3 00/29] Introduce runtime and conver tests
Cyril Hrubis
chrubis@suse.cz
Thu May 12 14:37:47 CEST 2022
This patchset introduces a concept of runtime, which is described well
enough in the first patch of the series and converts all tests that were
previously using timeout API to runtime API.
The main change to the v1 of the patchset is that the runtime is now per
iteration, e.g. if we run the test for all_filesystem the runtime
applies to a single filesystem only. And at the same time the overall
test timeout is a sum of static timeout and runtime. The static part of
the timeout is meant to cover test teardown and any inaccuracy in the
test runtime accounting.
The patches that convert tests to runtime API fall into these classes:
- fzsync tests
- faster timeout tests
- I/O tests
- timer precision tests
- oom tests
- misc tests that run for some time (controlled by command line
parameter)
fzsync tests
============
These tests previously defined their runtime based on scale factor that
used slice of the default timeout (300s) for the test runtime.
Now the runtime is explicitly encoded in the test, which I find much
more user friendly because it's clear how long will the test run.
faster timeout tests
====================
There were several tests that set low timeout becuase the test may lock
up. In these cases the timeout was simply removed because the default
value (30s) should be small enough.
I/O tests
=========
The aiodio tests were modified so that their runtime is capped to
30 minutes by default. That means that even on very slow machine they
finish in a reasonable time.
timer precision tests
=====================
These test simply set the test timeout to be 110% of the sum of the
sleep_time * samples. That is to accomodate for the context switches and
upkeeping between the calls in the timer library.
oom tests
=========
Here we just set the .max_iteration_runtime to TST_RUNTIME_UNLIMITED,
which has the very same effect as .timeout = -1 previously had.
misc test
=========
In this case the test code was simplified quite a lot since the runtime
is now passed in a unified way and parsed in the test library.
v3:
- renamed max_iteration_runtime to just max_runtime
and tst_set_runtime() to tst_set_max_runtime()
- fixed newlib_tests and CI runtest.sh
- merged the two patches for af_alg02
- added and fixed documentation
- fixed a few bugs in the tst_test.c
(LTP_RUNTIME_MUL etc.)
Cyril Hrubis (29):
Introduce a concept of max runtime
mtest06/mmap1: Convert to runtime
mtest06/mmap3: Convert to runtime
mtest01/mtest01: Convert to runtime
cve/cve-2015-3290: Convert to runtime
crypto/af_alg02: Convert to runtime.
crypto/pcrypt_aead01: Convert to runtime
syscalls/clock_gettime01: Remove useless timeout
syscalls/fanotify22: Remove useless timeout
syscalls/gettimeofday02: Convert to runtime.
syscalls/inotify06: Convert to runtime.
syscalls/inotify01: Remove now useless timeout
syscalls/perf_event_open03: Convert to runtime
syscalls/readv01: Remove now useless timeout
syscalls/tgkill03: Remove now unused timeout
syscalls/setsockopt09: Remove now useless timeout
syscalls/userfaultfd01: Remove now useless timeout
syscalls/move_pages12: Convert to runtime
syscalls/rt_sigqueueinfo01: Remove now useless timeout
mem/mallocstress: Convert to runtime
mem/{oom,min_free_kbytes}: Convert to runtime
fuzzy_sync: Convert to runtime
ltp-aiodio/dio_sparse,aiodio_sparse: Convert to runtime.
ltp-aiodio/read_checkzero: Remove
ltp-aiodio/dio_{truncate,append}: Convert to runtime
ltp-aiodio/dio_read: Convert to runtime
timer_test: Convert to runtime
ltp-aiodio/aiodio_append: Convert to runtime
tst_test: Remove timeout stubs
doc/c-test-api.txt | 42 +++--
doc/user-guide.txt | 30 +++-
include/tst_fuzzy_sync.h | 19 +--
include/tst_test.h | 31 +++-
lib/newlib_tests/.gitignore | 5 +-
lib/newlib_tests/runtest.sh | 2 +-
lib/newlib_tests/test10.c | 22 ---
lib/newlib_tests/test12.c | 21 ---
lib/newlib_tests/test13.c | 1 -
lib/newlib_tests/test18.c | 22 ---
lib/newlib_tests/test_children_cleanup.c | 1 -
lib/newlib_tests/test_runtime01.c | 30 ++++
lib/newlib_tests/test_runtime02.c | 28 ++++
lib/newlib_tests/tst_fuzzy_sync01.c | 1 +
lib/newlib_tests/tst_fuzzy_sync02.c | 1 +
lib/newlib_tests/tst_fuzzy_sync03.c | 1 +
lib/tst_test.c | 143 +++++++++++++-----
lib/tst_timer_test.c | 12 +-
runtest/ltp-aiodio.part4 | 2 -
testcases/cve/cve-2015-3290.c | 8 +-
testcases/kernel/crypto/af_alg02.c | 4 +-
testcases/kernel/crypto/af_alg07.c | 1 +
testcases/kernel/crypto/pcrypt_aead01.c | 3 +-
testcases/kernel/io/ltp-aiodio/.gitignore | 1 -
.../kernel/io/ltp-aiodio/aiodio_append.c | 25 ++-
.../kernel/io/ltp-aiodio/aiodio_sparse.c | 26 +++-
testcases/kernel/io/ltp-aiodio/common.h | 12 +-
testcases/kernel/io/ltp-aiodio/dio_append.c | 4 +
testcases/kernel/io/ltp-aiodio/dio_read.c | 9 ++
testcases/kernel/io/ltp-aiodio/dio_sparse.c | 6 +-
testcases/kernel/io/ltp-aiodio/dio_truncate.c | 6 +
.../kernel/io/ltp-aiodio/read_checkzero.c | 84 ----------
testcases/kernel/mem/mtest01/mtest01.c | 14 +-
testcases/kernel/mem/mtest06/mmap1.c | 24 +--
testcases/kernel/mem/mtest06/mmap3.c | 12 +-
testcases/kernel/mem/mtest07/mallocstress.c | 11 +-
testcases/kernel/mem/oom/oom01.c | 2 +-
testcases/kernel/mem/oom/oom02.c | 2 +-
testcases/kernel/mem/oom/oom03.c | 2 +-
testcases/kernel/mem/oom/oom04.c | 2 +-
testcases/kernel/mem/oom/oom05.c | 2 +-
testcases/kernel/mem/thp/thp04.c | 1 +
.../kernel/mem/tunable/min_free_kbytes.c | 2 +-
testcases/kernel/pty/pty03.c | 2 +-
testcases/kernel/pty/pty05.c | 1 +
testcases/kernel/pty/pty06.c | 1 +
testcases/kernel/pty/pty07.c | 1 +
testcases/kernel/sound/snd_seq01.c | 2 +-
testcases/kernel/sound/snd_timer01.c | 1 +
testcases/kernel/syscalls/bind/bind06.c | 2 +-
.../syscalls/clock_gettime/clock_gettime01.c | 1 -
.../kernel/syscalls/fanotify/fanotify22.c | 1 -
.../syscalls/gettimeofday/gettimeofday02.c | 15 +-
testcases/kernel/syscalls/inotify/inotify06.c | 7 +-
testcases/kernel/syscalls/inotify/inotify09.c | 1 +
testcases/kernel/syscalls/inotify/inotify11.c | 1 -
.../kernel/syscalls/ipc/shmctl/shmctl05.c | 2 +-
.../kernel/syscalls/move_pages/move_pages12.c | 4 +-
.../perf_event_open/perf_event_open03.c | 15 +-
testcases/kernel/syscalls/readv/readv01.c | 1 -
.../rt_sigqueueinfo/rt_sigqueueinfo01.c | 1 -
testcases/kernel/syscalls/sendmsg/sendmsg03.c | 1 +
.../kernel/syscalls/setsockopt/setsockopt06.c | 2 +-
.../kernel/syscalls/setsockopt/setsockopt07.c | 1 +
.../kernel/syscalls/setsockopt/setsockopt09.c | 1 -
testcases/kernel/syscalls/tgkill/tgkill03.c | 1 -
.../syscalls/timerfd/timerfd_settime02.c | 1 +
.../syscalls/userfaultfd/userfaultfd01.c | 1 -
testcases/kernel/syscalls/writev/writev03.c | 2 +-
69 files changed, 419 insertions(+), 327 deletions(-)
delete mode 100644 lib/newlib_tests/test10.c
delete mode 100644 lib/newlib_tests/test12.c
delete mode 100644 lib/newlib_tests/test18.c
create mode 100644 lib/newlib_tests/test_runtime01.c
create mode 100644 lib/newlib_tests/test_runtime02.c
delete mode 100644 testcases/kernel/io/ltp-aiodio/read_checkzero.c
--
2.35.1
More information about the ltp
mailing list