[LTP] [RFC PATCh] lib: redefine the overall timeout logic of test

[Li Wang]

On Thu, Jan 9, 2025 at 3:46 PM Cyril Hrubis <chrubis@suse.cz> wrote:

> Hi!
> > I have a different view on the readahead02 test, because the runtime
> > resetting is based on pieces of each IO test elapsed time, then reset
> > runtime for next time. This applies to any kernels, no matter the faster
> > or slower, the elapsed time will be enough for the next tcases[].
> >
> > If we put the worst-case runtime into .timeout and reset for the next
> > tcases[], which will be multiplied again on debug kernel, actually we
> > don't need that since the dynamic runtime comes from a real test.
>
> I do not get why we need to reset anything. The test library does reset
> the timeout after each test iteration:
>
>        for (i = 0; i < tst_test->tcnt; i++) {
>                 saved_results = *results;
>                 heartbeat();
>                 ^
>                 this resets the timeout timer so each iteration runs
>                 with the whole timeout
>
>                 tst_test->test(i);
>
>                 if (tst_getpid() != main_pid)
>                         exit(0);
>
>                 tst_reap_children();
>
>                 if (results_equal(&saved_results, results))
>                         tst_brk(TBROK, "Test %i haven't reported
> results!", i);
>         }
>

Yes, that's right.


> > Maybe I missed something in the test, but we can treat readahead02
> > in a separate work. The new patch 4/4 should be modified using .runtime
> > instead of .timeout. Feel free to comment your thoughts there.
>
> Why is readahead02 so special that we need to do more than the usuall?
>
> That is:
>
> - measure how long one tests iteration takes on slower hardware
> - multiply that by two and set that as a .timeout
>

I admit that this is the recommended/standard way to use .timeout
based on the new time logic design.

P.s.

The reason for calling tst_set_runtime() in each iteration is that we can
directly utilize the real elapsed time as runtime and don't need to do
additional measurements.

That value (which comes from a real test) is more precise than (what we
get on) any other slower hardware. And the disadvantage as you pointed
it increases the timeout for each iteration.

I can give up this usage and follow your suggestion, but I still fell
that we could have one more way on .runtime.


-- 
Regards,
Li Wang