[LTP] [RFC PATCh] lib: redefine the overall timeout logic of test

[Li Wang]

On Thu, Jan 9, 2025 at 4:42 PM Li Wang <liwang@redhat.com> wrote:

>
>
> On Thu, Jan 9, 2025 at 3:46 PM Cyril Hrubis <chrubis@suse.cz> wrote:
>
>> Hi!
>> > I have a different view on the readahead02 test, because the runtime
>> > resetting is based on pieces of each IO test elapsed time, then reset
>> > runtime for next time. This applies to any kernels, no matter the faster
>> > or slower, the elapsed time will be enough for the next tcases[].
>> >
>> > If we put the worst-case runtime into .timeout and reset for the next
>> > tcases[], which will be multiplied again on debug kernel, actually we
>> > don't need that since the dynamic runtime comes from a real test.
>>
>> I do not get why we need to reset anything. The test library does reset
>> the timeout after each test iteration:
>>
>>        for (i = 0; i < tst_test->tcnt; i++) {
>>                 saved_results = *results;
>>                 heartbeat();
>>                 ^
>>                 this resets the timeout timer so each iteration runs
>>                 with the whole timeout
>>
>>                 tst_test->test(i);
>>
>>                 if (tst_getpid() != main_pid)
>>                         exit(0);
>>
>>                 tst_reap_children();
>>
>>                 if (results_equal(&saved_results, results))
>>                         tst_brk(TBROK, "Test %i haven't reported
>> results!", i);
>>         }
>>
>
> Yes, that's right.
>
>
>> > Maybe I missed something in the test, but we can treat readahead02
>> > in a separate work. The new patch 4/4 should be modified using .runtime
>> > instead of .timeout. Feel free to comment your thoughts there.
>>
>> Why is readahead02 so special that we need to do more than the usuall?
>>
>> That is:
>>
>> - measure how long one tests iteration takes on slower hardware
>> - multiply that by two and set that as a .timeout
>>
>
> I admit that this is the recommended/standard way to use .timeout
> based on the new time logic design.
>
> P.s.
>
> The reason for calling tst_set_runtime() in each iteration is that we can
> directly utilize the real elapsed time as runtime and don't need to do
> additional measurements.
>

Forgot to say, that each iteration has four sub-tcases to accomplish.
The recommended way (you gave above) is to regard them as one
big test and reset timeout by heartbeat() in tst_test->tcnt. If this value
is big enough, that's fine. But I fear that value (come from measurement)
is still not covered all situations.



>
> That value (which comes from a real test) is more precise than (what we
> get on) any other slower hardware. And the disadvantage as you pointed
> it increases the timeout for each iteration.
>
> I can give up this usage and follow your suggestion, but I still fell
> that we could have one more way on .runtime.
>
>
> --
> Regards,
> Li Wang
>


-- 
Regards,
Li Wang