[LTP] LTP in valgrind :)

[Martin Cermak]

Hello guys,

On  Tue  2025-05-06  10:05 , Petr Vorel wrote:
> Hi all,
>
> [ Cc Martin and Mark from Valgrind project ]
>
> > Hi!
> > > > It also opens some interesting questions, i.e. how do we make comparing
> > > > results from two different tests easier. Currently they grep the test
> > > > results for a summary, but maybe we can do better.
>
> > > One option is to extract all TPASS/TFAIL/TWARN/TBROK/TCONF messages,
> > > discard any message contents past the file:line header and then compare
> > > whether the sanitized output is identical. That'll take care of random
> > > values in the output while ensuring that the test went through the same
> > > code paths as before. We could provide a sanitizer script for that.
>
> > Maybe we can even add an option to the test library to supress the
> > messages in output, that would be fairly simple.
>
> @Martin @Mark: feel free to comment what we can do for you :).
> Whole thread:
> https://lore.kernel.org/ltp/20250505195003.GB137650@pevik/T/#t

Nice to see you noticed!  Primarily we're trying to identify
valgrind gaps and bugs.  For our use-case it's good to have
reproducible test runs in a sense of identical test results for
repeated test runs.  We got pretty close to this by only taking
into account the testcase summaries at the very end of the test
logs.  Any steps to get closer to reproducible test runs on the
LTP side are interesting for us, and we'll try to reflect those!

Cheers,
Martin