[LTP] [PATCH 1/2] lib: multiply the timeout if detect slow kconfigsD

[Li Wang]

On Fri, Jan 3, 2025 at 11:48 PM Cyril Hrubis <chrubis@suse.cz> wrote:

> Hi!
> > Yes, the benefit of multiplying TIMEOUT (on a slow system) is not only
> > to avoid increasing the actual execution time of the test, but also to
> give
> >  the system more time to wait for the test to complete the final work.
> >
> > Original:
> >   |  -- timeout -- | -- max_runtime -- |
> >
> > Previous:
> >   |  -- timeout -- | -------- max_runtime * 4 -------- |
> >
> > Now:
> >   |  -------- timeout * 4 -------- | -- max_runtime --  |
>
> The problems I see here:
>
> There are tests, I think this covers mostly the IO stress tests, where
> the max runtime may be significantly larger than the timeout, so
> multiplying only the timeout may not be enough there.
>

I did a quick grep that some ltp-aiodio tests set it to 1800 sec, which
only 8/91 occupation in the LTP testcases/, I'm not sure if it's worth
adding a new field for those few stress tests.

And with the previous method, the multiple 4 max_runtime for 1800s
is 2hours per test up limit, I can't imagine how long we will get eventually
in the whole test time.

Maybe another way is to create a separate function in a header
like aio_common.h (or in high-level dir) for handling that significantly
larger runtime tests.

BTW, we have TST_UNLIMITED_RUNTIME choice or, invoke
tst_set_max_runtime() in somehow.



> I wanted to eventually move to a shorter default timeout, e.g. 10s once
> we have enough max_runtime anotation in the testcases.
>

That's fine, but most tests do not have .max_runtime. I guess what
we should consider now is making them have enough timeout time
on the slow kernel, isn't it?


$ git grep .max_runtime | wc -l
91

$ git grep .max_runtime
cve/cve-2014-0196.c:    .max_runtime = 60,
cve/cve-2015-3290.c:    .max_runtime = 180,
cve/cve-2016-7117.c:    .max_runtime = 60,
cve/cve-2017-2671.c:    .max_runtime = 40,
kernel/controllers/cgroup/cgroup_core03.c:      .max_runtime = 20,
kernel/crypto/af_alg02.c:       .max_runtime = 20,
kernel/crypto/af_alg07.c:       .max_runtime = 150,
kernel/crypto/pcrypt_aead01.c:  .max_runtime = 300,
kernel/fs/fs_fill/fs_fill.c:    .max_runtime = 300,
kernel/fs/fsplough/fsplough.c:  tst_set_max_runtime(bufsize * loop_count /
(8 * 1024 * 1024));
kernel/fs/fsx-linux/fsx-linux.c:        .max_runtime = 1800,
kernel/fs/read_all/read_all.c:          tst_res(TINFO, "Worker timeout set
to 10%% of max_runtime: %dms",
kernel/fs/read_all/read_all.c:  .max_runtime = 100,
kernel/io/ltp-aiodio/aio-stress.c:      .max_runtime = 1800,
kernel/io/ltp-aiodio/aiocp.c:   .max_runtime = 1800,
kernel/io/ltp-aiodio/aiodio_append.c:   .max_runtime = 1800,
kernel/io/ltp-aiodio/aiodio_sparse.c:   .max_runtime = 1800,
kernel/io/ltp-aiodio/dio_append.c:      .max_runtime = 1800,
kernel/io/ltp-aiodio/dio_read.c:        .max_runtime = 1800,
kernel/io/ltp-aiodio/dio_sparse.c:      .max_runtime = 1800,
kernel/io/ltp-aiodio/dio_truncate.c:    .max_runtime = 1800,
kernel/mem/ksm/ksm02.c:         tst_set_max_runtime(32 * (size /
DEFAULT_MEMSIZE));
kernel/mem/ksm/ksm02.c: .max_runtime = 32,
kernel/mem/ksm/ksm04.c:         tst_set_max_runtime(32 * (size /
DEFAULT_MEMSIZE));
kernel/mem/ksm/ksm04.c: .max_runtime = 32,
kernel/mem/mmapstress/mmapstress01.c:   .max_runtime = 12,
kernel/mem/mtest01/mtest01.c:   .max_runtime = 300,
kernel/mem/mtest06/mmap1.c:     .max_runtime = 180,
kernel/mem/mtest06/mmap3.c:     .max_runtime = 60,
kernel/mem/mtest07/mallocstress.c:      .max_runtime = 600,
kernel/mem/oom/oom01.c: .max_runtime = TST_UNLIMITED_RUNTIME,
kernel/mem/oom/oom02.c: .max_runtime = TST_UNLIMITED_RUNTIME,
kernel/mem/oom/oom03.c: .max_runtime = TST_UNLIMITED_RUNTIME,
kernel/mem/oom/oom04.c: .max_runtime = TST_UNLIMITED_RUNTIME,
kernel/mem/oom/oom05.c: .max_runtime = TST_UNLIMITED_RUNTIME,
kernel/mem/swapping/swapping01.c:       .max_runtime = 600,
kernel/mem/thp/thp04.c: .max_runtime = 150,
kernel/mem/tunable/min_free_kbytes.c:   .max_runtime =
TST_UNLIMITED_RUNTIME,
kernel/pty/pty03.c:     .max_runtime = 30,
kernel/pty/pty05.c:     .max_runtime = 150,
kernel/pty/pty06.c:     .max_runtime = 150,
kernel/pty/pty07.c:     .max_runtime = 150,
kernel/sched/cfs-scheduler/cfs_bandwidth01.c:   .max_runtime = 20,
kernel/sched/cfs-scheduler/starvation.c:
 tst_set_max_runtime(timeout);
kernel/security/dirtyc0w_shmem/dirtyc0w_shmem.c:        .max_runtime = 120,
kernel/security/kallsyms/kallsyms.c:    .max_runtime = 60,
kernel/sound/snd_seq01.c:       .max_runtime = 60,
kernel/sound/snd_timer01.c:     .max_runtime = 150,
kernel/syscalls/bind/bind06.c:  .max_runtime = 300,
kernel/syscalls/copy_file_range/copy_file_range01.c:    .max_runtime = 5
kernel/syscalls/fallocate/fallocate06.c:        .max_runtime = 120,
kernel/syscalls/fork/fork13.c:  .max_runtime = 600,
kernel/syscalls/fsync/fsync02.c:        .max_runtime = 300,
kernel/syscalls/gettimeofday/gettimeofday02.c:  .max_runtime = 10,
kernel/syscalls/inotify/inotify06.c:    .max_runtime = 600,
kernel/syscalls/inotify/inotify09.c:    .max_runtime = 150,
kernel/syscalls/inotify/inotify12.c:    .max_runtime = 10,
kernel/syscalls/ioctl/ioctl_sg01.c:     .max_runtime = 3600,
kernel/syscalls/ipc/msgstress/msgstress01.c:    .max_runtime = 180,
kernel/syscalls/ipc/semget/semget05.c:  tst_set_max_runtime(maxsems / 200);
kernel/syscalls/ipc/shmctl/shmctl05.c:  .max_runtime = 10,
kernel/syscalls/keyctl/keyctl02.c:      .max_runtime = 60,
kernel/syscalls/landlock/landlock04.c:  .max_runtime = 360,
kernel/syscalls/madvise/madvise06.c:    .max_runtime = 60,
kernel/syscalls/madvise/madvise11.c:    .max_runtime = 30,
kernel/syscalls/migrate_pages/migrate_pages03.c:        .max_runtime = 300,
kernel/syscalls/move_pages/move_pages12.c:      .max_runtime = 240,
kernel/syscalls/nice/nice05.c:  .max_runtime = 3,
kernel/syscalls/perf_event_open/perf_event_open02.c:    .max_runtime = 72
kernel/syscalls/perf_event_open/perf_event_open03.c:    .max_runtime = 300,
kernel/syscalls/preadv2/preadv203.c:    .max_runtime = 60,
kernel/syscalls/readahead/readahead02.c:         * speed, sometime test
timeout when the default max_runtime is used up.
kernel/syscalls/readahead/readahead02.c:
 tst_set_max_runtime(test.max_runtime + (usec + usec_ra) / 1000000);
kernel/syscalls/readahead/readahead02.c:
 tst_set_max_runtime(1 + testfile_size / (DEFAULT_FILESIZE/32));
kernel/syscalls/readahead/readahead02.c:        .max_runtime = 30,
kernel/syscalls/request_key/request_key03.c:    .max_runtime = 20,
kernel/syscalls/sendfile/sendfile09.c:  .max_runtime = 120,
kernel/syscalls/sendmsg/sendmsg03.c:    .max_runtime = 150,
kernel/syscalls/set_mempolicy/set_mempolicy01.c:
 tst_set_max_runtime(test.max_runtime * (1 << nodes->cnt/16));
kernel/syscalls/set_mempolicy/set_mempolicy01.c:        .max_runtime = 600,
kernel/syscalls/setsockopt/setsockopt06.c:      .max_runtime = 270,
kernel/syscalls/setsockopt/setsockopt07.c:      .max_runtime = 150,
kernel/syscalls/swapoff/swapoff01.c:    .max_runtime = 60,
kernel/syscalls/swapon/swapon01.c:      .max_runtime = 60,
kernel/syscalls/timerfd/timerfd_settime02.c:    .max_runtime = 150,
kernel/syscalls/writev/writev03.c:      .max_runtime = 75,
network/can/cve/can_bcm01.c:    .max_runtime = 30,
network/netstress/netstress.c:  .max_runtime = 300,
network/nfs/nfs_stress/nfs05_make_tree.c:       .max_runtime = 300,
network/packet/fanout01.c:      .max_runtime = 180,
network/sockets/vsock01.c:      .max_runtime = 60,


-- 
Regards,
Li Wang