[LTP] [RFC] Reduce .runtime for Long-Running Tests ?

[Li Wang]

On Fri, Jun 20, 2025 at 5:03 PM Cyril Hrubis <chrubis@suse.cz> wrote:

> Hi!
> > After a round of experiments and internal discussions (thanks to
> > Ian Wien for sharing his thoughts with me), we think that making
> > LTP_RUNTIME_MUL support floating point numbers [0, 1] may
> > a possible way to reduce the .runtime values set in tests.
> >
> > For example, setting LTP_RUNTIME_MUL to 0.1 can obviously
> > reduce the test time of 600 seconds to 60 seconds.
> >
> > One may think that reducing the .runtime value in aproduction
> > environment is potentially risky, and to some extent the answer
> > is yes.
> >
> > But looking back, LTP is triggered very frequently in CI and various
> > production flows, so to compensate for this loss, we can use floating
> > point LTP_RUNTIME_MUL only in designated quick CI, instead of
> > using it in daily tests. This will help cover more scenarios.
>
>


> Having reduced runtime for CI makes sense, as you said you are making up
> for the reduction by the number of testruns. It may not work well for
> fuzzy sync though where we depend on having good enough sampling period.
>

Against this requirement, maybe we can introduce a "switch button" to close
runtime_multiply (0.1] take effective in fuzzy sync sampling tests, even for
some other tests thatrely on tst_remaind_runtime to control the process
(e..g preadv203) as well

With this switch, we can precisely control the specified test not get
impacted
by the value of LTP_RUNTIME_MUL(whatever <1 or >1).



>
> Also limiting the smallest multiplier to be 0.1 or so does make sense. I
> assume that if we set it to 0.01 or smaller most tests with runtime
> woudn't even execute.
>
> > From our CI report, use 0.1 in runtime_mul find a few failures in the
> round
> > down problem with nice05.c (.runtime = 3), this is a defect of
> > multply_runtime().
> > Also, another preadv203 possible failure related this. But they are tiny
> > issues. And the rest .runtime tests so far no obvious problem on that.
>
> I guess that we need to make sure that the runtime stays positive. So
> maybe we need something as:
>
> runtime = MAX(2, tst_test->runtime * runtime_mul)
>
> > So I would like to start the work from this point to reduce execution
> time.
>
> Sounds good to me.
>
> --
> Cyril Hrubis
> chrubis@suse.cz
>
>

-- 
Regards,
Li Wang