[LTP] [PATCH v2 00/30] Introduce runtime and conver tests

Cyril Hrubis chrubis@suse.cz
Tue May 3 19:46:48 CEST 2022


This patchset introduces a concept of runtime, which is described well
enough in the first patch of the series and converts all tests that were
previously using timeout API to runtime API.

The main change to the v1 of the patchset is that the runtime is now per
iteration, e.g. if we run the test for all_filesystem the runtime
applies to a single filesystem only. And at the same time the overall
test timeout is a sum of static timeout and runtime. The static part of
the timeout is meant to cover test teardown and any inaccuracy in the
test runtime accounting.

The patches that convert tests to runtime API fall into these classes:

- fzsync tests
- faster timeout tests
- I/O tests
- timer precision tests
- oom tests
- misc tests that run for some time (controlled by command line
  parameter)

fzsync tests
============

These tests previously defined their runtime based on scale factor that
used slice of the default timeout (300s) for the test runtime.

Now the runtime is explicitly encoded in the test, which I find much
more user friendly because it's clear how long will the test run.

faster timeout tests
====================

There were several tests that set low timeout becuase the test may lock
up. In these cases the timeout was simply removed because the default
value (30s) should be small enough.

I/O tests
=========

The aiodio tests were modified so that their runtime is capped to
30 minutes by default. That means that even on very slow machine they
finish in a reasonable time.

timer precision tests
=====================

These test simply set the test timeout to be 110% of the sum of the
sleep_time * samples. That is to accomodate for the context switches and
upkeeping between the calls in the timer library.


oom tests
=========

Here we just set the .max_iteration_runtime to TST_RUNTIME_UNLIMITED,
which has the very same effect as .timeout = -1 previously had.

misc test
=========

In this case the test code was simplified quite a lot since the runtime
is now passed in a unified way and parsed in the test library.

TODO:

This patchset is reasonably complete in a sense that it removes the
timeout API at the end. Still there are a few things to consider:

- is reusing the -I parameter a good idea? Wouldn't adding new parameter
  (-r) be better?

- there are quite likely tests that run for more than a second or a two
  and should be made runtime aware

- anything else?

Cyril Hrubis (30):
  Introduce a concept of max runtime
  mtest06/mmap1: Convert to runtime
  mtest06/mmap3: Convert to runtime
  mtest01/mtest01: Convert to runtime
  cve/cve-2015-3290: Convert to runtime
  crypto/af_alg02: Convert to runtime.
  crypto/pcrypt_aead01: Convert to runtime
  syscalls/clock_gettime01: Remove useless timeout
  syscalls/fanotify22: Remove useless timeout
  syscalls/gettimeofday02: Convert to runtime.
  syscalls/inotify06: Convert to runtime.
  syscalls/inotify01: Remove now useless timeout
  syscalls/perf_event_open03: Convert to runtime
  syscalls/readv01: Remove now useless timeout
  syscalls/tgkill03: Remove now unused timeout
  syscalls/setsockopt09: Remove now useless timeout
  syscalls/userfaultfd01: Remove now useless timeout
  syscalls/move_pages12: Convert to runtime
  syscalls/rt_sigqueueinfo01: Remove now useless timeout
  mem/mallocstress: Convert to runtime
  mem/{oom,min_free_kbytes}: Convert to runtime
  crypto/af_alg02: Convert to runtime
  fuzzy_sync: Convert to runtime
  ltp-aiodio/dio_sparse,aiodio_sparse: Convert to runtime.
  ltp-aiodio/read_checkzero: Remove
  ltp-aiodio/dio_{truncate,append}: Convert to runtime
  ltp-aiodio/dio_read: Convert to runtime
  ltp-aiodio/aiodio_append: Convert to runtime
  timer_test: Convert to runtime
  tst_test: Remove timeout stubs

 doc/user-guide.txt                            |   4 +
 include/tst_fuzzy_sync.h                      |  19 +--
 include/tst_test.h                            |  31 +++-
 lib/newlib_tests/.gitignore                   |   5 +-
 lib/newlib_tests/test10.c                     |  22 ---
 lib/newlib_tests/test12.c                     |  21 ---
 lib/newlib_tests/test13.c                     |   1 -
 lib/newlib_tests/test_children_cleanup.c      |   1 -
 .../{test18.c => test_runtime01.c}            |  10 +-
 lib/newlib_tests/test_runtime02.c             |  31 ++++
 lib/tst_test.c                                | 144 +++++++++++++-----
 lib/tst_timer_test.c                          |  12 +-
 runtest/ltp-aiodio.part4                      |   2 -
 testcases/cve/cve-2015-3290.c                 |   8 +-
 testcases/kernel/crypto/af_alg02.c            |   4 +-
 testcases/kernel/crypto/af_alg07.c            |   1 +
 testcases/kernel/crypto/pcrypt_aead01.c       |   3 +-
 testcases/kernel/io/ltp-aiodio/.gitignore     |   1 -
 .../kernel/io/ltp-aiodio/aiodio_append.c      |  25 ++-
 .../kernel/io/ltp-aiodio/aiodio_sparse.c      |  26 +++-
 testcases/kernel/io/ltp-aiodio/common.h       |  12 +-
 testcases/kernel/io/ltp-aiodio/dio_append.c   |   4 +
 testcases/kernel/io/ltp-aiodio/dio_read.c     |   9 ++
 testcases/kernel/io/ltp-aiodio/dio_sparse.c   |   6 +-
 testcases/kernel/io/ltp-aiodio/dio_truncate.c |   6 +
 .../kernel/io/ltp-aiodio/read_checkzero.c     |  84 ----------
 testcases/kernel/mem/mtest01/mtest01.c        |  14 +-
 testcases/kernel/mem/mtest06/mmap1.c          |  24 +--
 testcases/kernel/mem/mtest06/mmap3.c          |  12 +-
 testcases/kernel/mem/mtest07/mallocstress.c   |  11 +-
 testcases/kernel/mem/oom/oom01.c              |   2 +-
 testcases/kernel/mem/oom/oom02.c              |   2 +-
 testcases/kernel/mem/oom/oom03.c              |   2 +-
 testcases/kernel/mem/oom/oom04.c              |   2 +-
 testcases/kernel/mem/oom/oom05.c              |   2 +-
 testcases/kernel/mem/thp/thp04.c              |   1 +
 .../kernel/mem/tunable/min_free_kbytes.c      |   2 +-
 testcases/kernel/pty/pty03.c                  |   2 +-
 testcases/kernel/pty/pty05.c                  |   1 +
 testcases/kernel/pty/pty06.c                  |   1 +
 testcases/kernel/pty/pty07.c                  |   1 +
 testcases/kernel/sound/snd_seq01.c            |   2 +-
 testcases/kernel/sound/snd_timer01.c          |   1 +
 testcases/kernel/syscalls/bind/bind06.c       |   2 +-
 .../syscalls/clock_gettime/clock_gettime01.c  |   1 -
 .../kernel/syscalls/fanotify/fanotify22.c     |   1 -
 .../syscalls/gettimeofday/gettimeofday02.c    |  15 +-
 testcases/kernel/syscalls/inotify/inotify06.c |   7 +-
 testcases/kernel/syscalls/inotify/inotify09.c |   1 +
 testcases/kernel/syscalls/inotify/inotify11.c |   1 -
 .../kernel/syscalls/ipc/shmctl/shmctl05.c     |   2 +-
 .../kernel/syscalls/move_pages/move_pages12.c |   4 +-
 .../perf_event_open/perf_event_open03.c       |  15 +-
 testcases/kernel/syscalls/readv/readv01.c     |   1 -
 .../rt_sigqueueinfo/rt_sigqueueinfo01.c       |   1 -
 testcases/kernel/syscalls/sendmsg/sendmsg03.c |   1 +
 .../kernel/syscalls/setsockopt/setsockopt06.c |   2 +-
 .../kernel/syscalls/setsockopt/setsockopt07.c |   1 +
 .../kernel/syscalls/setsockopt/setsockopt09.c |   1 -
 testcases/kernel/syscalls/tgkill/tgkill03.c   |   1 -
 .../syscalls/timerfd/timerfd_settime02.c      |   1 +
 .../syscalls/userfaultfd/userfaultfd01.c      |   1 -
 testcases/kernel/syscalls/writev/writev03.c   |   2 +-
 63 files changed, 339 insertions(+), 296 deletions(-)
 delete mode 100644 lib/newlib_tests/test10.c
 delete mode 100644 lib/newlib_tests/test12.c
 rename lib/newlib_tests/{test18.c => test_runtime01.c} (58%)
 create mode 100644 lib/newlib_tests/test_runtime02.c
 delete mode 100644 testcases/kernel/io/ltp-aiodio/read_checkzero.c

-- 
2.35.1



More information about the ltp mailing list