[LTP] [PATCH v2] tst_test: Add min_runtime to control lower bound of scaled runtime
Li Wang
liwang@redhat.com
Tue Jun 24 03:27:09 CEST 2025
On Mon, Jun 23, 2025 at 10:42 PM Martin Doucha <mdoucha@suse.cz> wrote:
> Hi!
>
> On 23. 06. 25 16:02, Li Wang wrote:
> > When LTP_RUNTIME_MUL is set to a value < 1.0 (commonly in CI or fast test
> > modes), the effective runtime of tests may be scaled down too
> aggressively.
> > For some tests especially those relying on probabilistic triggers or
> sampling
> > (e.g., fuzzy sync, CVE stress loops, long fault injection), this can
> result in
> > too few iterations or missed conditions, leading to unreliable results.
> >
> > This patch introduces a new field: .min_runtime;
> >
> > If set, this specifies the minimum allowed runtime (in seconds) after
> applying
> > the LTP_RUNTIME_MUL scaling factor. The final runtime is calculated as:
> >
> > MAX(runtime * LTP_RUNTIME_MUL, min_runtime)
> >
> > If min_runtime is not set, a default minimum of 1 second is enforced.
> >
> > This approach replaces the need for special flags such as
> > TST_NO_FRAC_RUNTIME_MUL and provides a more systematic, flexible
> solution.
> >
> > The following tests are updated to set .min_runtime:
> > - preadv203
> > - cve-2016-7117
> > - tst_fuzzy_sync01
> > - tst_fuzzy_sync02
> >
> > Suggested-by: Martin Doucha <mdoucha@suse.cz>
> > Signed-off-by: Li Wang <liwang@redhat.com>
> > Cc: Ian Wienand <iwienand@redhat.com>
> > ---
> >
> > Notes:
> > v1 --> v2:
> > * replace the .flags by a new min_runtime filed
> > * set 1 second as the default minimal runtime regardless
> > LTP_RUNTIME_MUL value
> >
> > include/tst_test.h | 9 +++++++++
> > lib/newlib_tests/tst_fuzzy_sync01.c | 2 +-
> > lib/newlib_tests/tst_fuzzy_sync02.c | 2 +-
> > lib/tst_test.c | 11 +++++++++--
> > testcases/cve/cve-2016-7117.c | 2 +-
> > testcases/kernel/syscalls/preadv2/preadv203.c | 2 +-
> > 6 files changed, 22 insertions(+), 6 deletions(-)
> >
> > diff --git a/include/tst_test.h b/include/tst_test.h
> > index 6fd8cbae3..9c21c1728 100644
> > --- a/include/tst_test.h
> > +++ b/include/tst_test.h
> > @@ -460,6 +460,14 @@ struct tst_fs {
> > * (e.g., TIMEOUT_MUL), ensuring consistent test duration
> across
> > * different environments (e.g., debug vs. stock kernels).
> > *
> > + * @min_runtime: Optional lower bound (in seconds) applied after
> runtime is
> > + * scaled by LTP_RUNTIME_MUL. If the scaled runtime is smaller
> > + * than this value, it will be clamped up to min_runtime.
> > + * This is useful for tests that require a minimum execution
> > + * time to gather sufficient samples or trigger events (e.g.,
> > + * probabilistic or fuzzy synchronization tests).
> > + * If not set, a default minimum of 1 second is enforced.
> > + *
> > * @setup: Setup callback is called once at the start of the test in
> order to
> > * prepare the test environment.
> > *
> > @@ -584,6 +592,7 @@ struct tst_fs {
> >
> > int timeout;
> > int runtime;
> > + int min_runtime;
> >
> > void (*setup)(void);
> > void (*cleanup)(void);
> > diff --git a/lib/newlib_tests/tst_fuzzy_sync01.c
> b/lib/newlib_tests/tst_fuzzy_sync01.c
> > index b1390f460..16f30de57 100644
> > --- a/lib/newlib_tests/tst_fuzzy_sync01.c
> > +++ b/lib/newlib_tests/tst_fuzzy_sync01.c
> > @@ -246,5 +246,5 @@ static struct tst_test test = {
> > .test = run,
> > .setup = setup,
> > .cleanup = cleanup,
> > - .runtime = 150,
> > + .min_runtime = 150,
>
> Lib tests don't need .min_runtime unless you specifically want to test
> that LTP_RUNTIME_MUL gets handled correctly.
>
I set .min_runtime for all fuzzy_sync tests that contain "pair.min_samples
= XXX",
because the dedicated sampling times need more time and it is hard to
tell the real minimal time, so I just reuse the old .runtime value.
And yes we can ignore this in lib tests, but I want to make them as
examples.
>
> > };
> > diff --git a/lib/newlib_tests/tst_fuzzy_sync02.c
> b/lib/newlib_tests/tst_fuzzy_sync02.c
> > index bc079f6ff..3b0fb8a9b 100644
> > --- a/lib/newlib_tests/tst_fuzzy_sync02.c
> > +++ b/lib/newlib_tests/tst_fuzzy_sync02.c
> > @@ -223,5 +223,5 @@ static struct tst_test test = {
> > .test = run,
> > .setup = setup,
> > .cleanup = cleanup,
> > - .runtime = 150,
> > + .min_runtime = 150,
> > };
> > diff --git a/lib/tst_test.c b/lib/tst_test.c
> > index 495e022f7..75a78be9a 100644
> > --- a/lib/tst_test.c
> > +++ b/lib/tst_test.c
> > @@ -627,10 +627,14 @@ static void parse_mul(float *mul, const char
> *env_name, float min, float max)
> > static int multiply_runtime(unsigned int runtime)
> > {
> > static float runtime_mul = -1;
> > + int min_runtime = 1;
> > +
> > + if (tst_test->min_runtime)
> > + min_runtime = MAX(1, tst_test->min_runtime);
> >
> > parse_mul(&runtime_mul, "LTP_RUNTIME_MUL", 0.0099, 100);
> >
> > - return runtime * runtime_mul;
> > + return MAX(runtime * runtime_mul, min_runtime);
> > }
> >
> > static struct option {
> > @@ -662,7 +666,7 @@ static void print_help(void)
> > fprintf(stderr, "LTP_SINGLE_FS_TYPE Specifies filesystem
> instead all supported (for .all_filesystems)\n");
> > fprintf(stderr, "LTP_FORCE_SINGLE_FS_TYPE Testing only. The same
> as LTP_SINGLE_FS_TYPE but ignores test skiplist.\n");
> > fprintf(stderr, "LTP_TIMEOUT_MUL Timeout multiplier (must
> be a number >=1)\n");
> > - fprintf(stderr, "LTP_RUNTIME_MUL Runtime multiplier (must
> be a number >=1)\n");
> > + fprintf(stderr, "LTP_RUNTIME_MUL Runtime multiplier (must
> be a number >0)\n");
> > fprintf(stderr, "LTP_VIRT_OVERRIDE Overrides virtual
> machine detection (values: \"\"|kvm|microsoft|xen|zvm)\n");
> > fprintf(stderr, "TMPDIR Base directory for
> template directory (for .needs_tmpdir, default: %s)\n", TEMPDIR);
> > fprintf(stderr, "\n");
> > @@ -1996,6 +2000,9 @@ void tst_run_tcases(int argc, char *argv[], struct
> tst_test *self)
> > uname(&uval);
> > tst_res(TINFO, "Tested kernel: %s %s %s", uval.release,
> uval.version, uval.machine);
> >
> > + if (tst_test->min_runtime)
> > + tst_test->runtime =tst_test->min_runtime;
> If .runtime is set, .min_runtime should not override it. Setting
> .runtime < .min_runtime should trigger TBROK during lib setup.
>
Not necessary, in most cases, we don't need to set both simultaneously.
Even if .runtime is less than .min_runtime, MAX() will automatically
choose .min_runtime. (but I strongly do not recommend set both)
Or, maybe here we can override if .runtime not being set to avoid
lib complains no .runtime set failure.
- if (tst_test->min_runtime)
+ if (tst_test->min_runtime && !tst_test->runtime)
>
> > +
> > if (tst_test->runtime)
> > context->runtime = multiply_runtime(tst_test->runtime);
> >
> > diff --git a/testcases/cve/cve-2016-7117.c
> b/testcases/cve/cve-2016-7117.c
> > index dbec2b28b..8c488f447 100644
> > --- a/testcases/cve/cve-2016-7117.c
> > +++ b/testcases/cve/cve-2016-7117.c
> > @@ -151,7 +151,7 @@ static struct tst_test test = {
> > .test_all = run,
> > .setup = setup,
> > .cleanup = cleanup,
> > - .runtime = 60,
> > + .min_runtime = 60,
>
> The .runtime value in fuzzy_sync tests is calibrated to reliably
> reproduce the bug. You can usually set much smaller .min_runtime based
> on how long it takes to run 1024 iterations of the fuzzy loop, with some
> safety margin (x2?).
>
In theory, yes, but the difficulty is thatthe calibration time depends on
the various test platforms and system load conditions.
Here I only reset the .runtime to .min_runtime is for tests contain
'pair.min_smaples', which typically has a large value (e.g. 10000).
For those with the default 1024 iterations fuzzy_sync tests I did
nothing for them, we may need more experiments to get a proper
value.
>
> > .tags = (const struct tst_tag[]) {
> > {"linux-git", "34b88a68f26a"},
> > {"CVE", "2016-7117"},
> > diff --git a/testcases/kernel/syscalls/preadv2/preadv203.c
> b/testcases/kernel/syscalls/preadv2/preadv203.c
> > index 72a35d3ab..472543e5c 100644
> > --- a/testcases/kernel/syscalls/preadv2/preadv203.c
> > +++ b/testcases/kernel/syscalls/preadv2/preadv203.c
> > @@ -278,6 +278,6 @@ static struct tst_test test = {
> > .mntpoint = MNTPOINT,
> > .mount_device = 1,
> > .all_filesystems = 1,
> > - .runtime = 60,
> > + .min_runtime = 60,
> > .needs_root = 1,
> > };
>
>
> --
> Martin Doucha mdoucha@suse.cz
> SW Quality Engineer
> SUSE LINUX, s.r.o.
> CORSO IIa
> Krizikova 148/34
> 186 00 Prague 8
> Czech Republic
>
>
--
Regards,
Li Wang
More information about the ltp
mailing list