[LTP] LTP in valgrind :)
Cyril Hrubis
chrubis@suse.cz
Wed May 7 17:29:00 CEST 2025
Hi!
> > > > > It also opens some interesting questions, i.e. how do we make comparing
> > > > > results from two different tests easier. Currently they grep the test
> > > > > results for a summary, but maybe we can do better.
> >
> > > > One option is to extract all TPASS/TFAIL/TWARN/TBROK/TCONF messages,
> > > > discard any message contents past the file:line header and then compare
> > > > whether the sanitized output is identical. That'll take care of random
> > > > values in the output while ensuring that the test went through the same
> > > > code paths as before. We could provide a sanitizer script for that.
> >
> > > Maybe we can even add an option to the test library to supress the
> > > messages in output, that would be fairly simple.
> >
> > @Martin @Mark: feel free to comment what we can do for you :).
> > Whole thread:
> > https://lore.kernel.org/ltp/20250505195003.GB137650@pevik/T/#t
>
> Nice to see you noticed! Primarily we're trying to identify
> valgrind gaps and bugs. For our use-case it's good to have
> reproducible test runs in a sense of identical test results for
> repeated test runs. We got pretty close to this by only taking
> into account the testcase summaries at the very end of the test
> logs. Any steps to get closer to reproducible test runs on the
> LTP side are interesting for us, and we'll try to reflect those!
I've proposed a simple solution that would cut the message from tests
only to contain the line numbers and results in:
https://lore.kernel.org/ltp/20250505195003.GB137650@pevik/T/#m98317262b25ede62704b005fcefc63c62198bb90
If that works for you we can get this in LTP before the next release.
--
Cyril Hrubis
chrubis@suse.cz
More information about the ltp
mailing list