[LTP] [RFC] Reduce .runtime for Long-Running Tests ?

[Li Wang]

On Tue, May 27, 2025 at 7:10 PM Petr Vorel <pvorel@suse.cz> wrote:
>
> Hi Li,
>
> > Hello All,
>
> > After reviewing some LTP HTML test reports, I noticed that the ten tests
> > alone take nearly 20 minutes to complete. For example:
>
> > --------------------
> > bind06           300.15s
>
> This uses fuzzy sync, I wonder if it could be speedup. I guess better longer run
> than break the test.
>
> > msgstress01      180.44s
> > fsx22            170.47s
> > pty07            150.04s
> > pty06            150.02s
> > gf18             121.09s
> > gf17             120.82s
> > gf16             120.13s
> > dirtyc0w_shmem   120.11s
> > setsockopt07      76.47s
>
> > In total, running the full ltp-lite suite currently takes ~1h20m, which is a bit
> > long for CI or pre-merge validation pipelines.
>
> What is ltp-lite? Is it your internal CI for LTP? bind06 is in cve and syscalls
> runtest files.

Ah, that was combined by some general runtest and kicked out few
stressful tests:

Usually contains: math ipc syscalls mm sched nptl pty tracing fs ...

>
> > I'm wondering whether all these .runtime values are truly necessary to reproduce
> > the intended issues (e.g., race conditions, fuzzing, sync timing issues).
> > Or if we could:
> >  - Set a lower threshold for .runtime on general-purpose stress/fuzz tests
>
> I'd be careful for fuzzy sync.
>
> >  - Introduce a runtime _policy_ for different environments (e.g., fast
> > CI vs. full weekly runs)
>
> Or filter out the long running tests if the purpose is just to test LTP itself
> instead of the product?

Filter out some stressful test (we have done) can be acceptable, but I still
think some .runtime test is valuable to be executed in product verification.

That's why I started this email to get some inspiration.

>
> > It might be beneficial to revisit the .runtime values of long-running tests and
> > set a minimal yet effective duration that balances reproducibility with resource
> > efficiency. This could help save time and free up test resources earlier.
>
> Maybe Martin still have VM's which can trigger the problem to experiment, but
> runtime probably differs across architectures and available resources (number of
> CPU or memory).
>
> Kind regards,
> Petr
>
> > Any thoughts/comments would be appreciated.
>


-- 
Regards,
Li Wang