[LTP] [RFC] Reduce .runtime for Long-Running Tests ?

[Li Wang]

Hi All,

After a round of experiments and internal discussions (thanks to
Ian Wien for sharing his thoughts with me), we think that making
LTP_RUNTIME_MUL support floating point numbers [0, 1] may
a possible way to reduce the .runtime values set in tests.

For example, setting LTP_RUNTIME_MUL to 0.1 can obviously
reduce the test time of 600 seconds to 60 seconds.

One may think that reducing the .runtime value in aproduction
environment is potentially risky, and to some extent the answer
is yes.

But looking back, LTP is triggered very frequently in CI and various
production flows, so to compensate for this loss, we can use floating
point LTP_RUNTIME_MUL only in designated quick CI, instead of
using it in daily tests. This will help cover more scenarios.

>From our CI report, use 0.1 in runtime_mul find a few failures in the round
down problem with nice05.c (.runtime = 3), this is a defect of
multply_runtime().
Also, another preadv203 possible failure related this. But they are tiny
issues. And the rest .runtime tests so far no obvious problem on that.

So I would like to start the work from this point to reduce execution time.

Regarding calibrating the number of loops, I think that is another good
direction to work, but may need more time to evaluate. I think that's a
long-term goal.


On Thu, May 29, 2025 at 9:53 AM Li Wang <liwang@redhat.com> wrote:

> On Wed, May 28, 2025 at 9:35 PM Cyril Hrubis <chrubis@suse.cz> wrote:
> >
> > Hi!
> > > After reviewing some LTP HTML test reports, I noticed that the ten
> tests
> > > alone take nearly 20 minutes to complete. For example:
> > >
> > > --------------------
> > > bind06           300.15s
> > > msgstress01      180.44s
> >
> > Isn't this better after:
> >
> > commit e3b85e50471b3892316a2b2c7c730e9dc8d9139e
> > Author: Ricardo B. Marlière <rbm@suse.com>
> > Date:   Wed May 21 05:31:12 2025 -0300
> >
> >     syscalls/msgstress01: Set upper bound for num_messages
>
> This commit hasn't been backported to our CI ltp, so it hasn't gotten the
> count.
> But later I will focus on that once we upgrade to the May release.
>
> >
> >
> > > fsx22            170.47s
> > > pty07            150.04s
> > > pty06            150.02s
> > > gf18             121.09s
> > > gf17             120.82s
> > > gf16             120.13s
> > > dirtyc0w_shmem   120.11s
> > > setsockopt07      76.47s
> >
> > As Peter pointed out, most of these are fuzzy sync tests and the runtime
> > is callibrated in order to get reasonable chance of reproducing the
> > results. But indeed the runtime does not scale with the speed of the
> > machine and we are not setting the pair exec_loops in the tests that run
>
> > for so long, so I suppose that callibrating the number of loops for
> > these tests that does not make the bug less reproducible would make
> > things better.
>
> Agree, this is the direction that we need to focus on improving.
>
> --
> Regards,
> Li Wang
>


-- 
Regards,
Li Wang