[LTP] [RFC PATCh] lib: redefine the overall timeout logic of test

[Li Wang]

On Wed, Jan 8, 2025 at 5:18 PM Cyril Hrubis <chrubis@suse.cz> wrote:

> Hi!
> > Key Concepts:
> >
> >  timeout: Represents the time allocated for setup, cleanup, or other
> >           non-runtime parts of the test.
> >
> >  runtime: The maximum runtime of the test's main execution loop. If
> >           exceeded, the test will exit early.
>
> This is not entirely correct. I guess that we still didn't understand
> each other. Let me try to explain once more.
>
> The runtime is supposed to be used only in tests that call
> tst_remaning_runtime() in the main loop to limit the test duration based
> on runtime. This means that the test function execution time is always
> the same (plus minus some rounding errors).
>
> And the timeout is supposed to be upper bound for everything in the test
> minus the part that is limited by the runtime.
>
> So for the tests that use tst_remaining_runtime() the timeout guards the
> setup and cleanup, but for tests that does not use
> tst_remaining_runtime() the timeout limits the whole test run.
>

Ok, I see the difference. Thanks for explaining again.


> The reasoning behind this is that if for example we have a test where
> the test function does I/O that is known to take up to 10s we want to
> multiply this in the case of the debug kernel because the test function
> may be slower.
>
> On the other hand test function that has a loop that checks the remaning
> runtime will do less iterations on debug kernel but the duration will be
> the same. And we do not want to multiply this value because it will make
> the test even slower on debug kernels.
>

Indeed.



>
> The changes mostly looks okay, but some of the tests below that had
> .max_runtime should replace that with .timeout instead. And we also
>

Btw, we have some tests that implicitly invoke tst_remaining_runtime,
for example, using the fuzzy_sync library, which also needs .runtime
but not .timeout.

need tst_set_timeout() function for the testcases that try to figure out
> the upper bound for the test duration in the test setup.
>

+1, I will add this function and comment on the details of timeout and
runtime in LTP doc/.


-- 
Regards,
Li Wang