[LTP] [RFC PATCH 1/1] lib: Print in summary also tests not run at all

[Li Wang]

On Sun, Jan 30, 2022 at 11:44 AM xuyang2018.jy@fujitsu.com
<xuyang2018.jy@fujitsu.com> wrote:
>
> Hi Petr
> > We count only tests which TCONF. In case tst_brk() some tests might be
> > skipped without even trying to run them), thus print them.
> >
> > Signed-off-by: Petr Vorel<pvorel@suse.cz>
> > ---
> > Hi,
> >
> > probably quite confusing to have "skipped" and "not run".
> > Maybe rename s/skipped/cancelled/ and s/not run/skipped/ ?
> > Or just to increase cound of skipped?

Both fine to me, I slightly think "not-run" might more precise to describe that.

> >
> > E.g. instead of:
> >
> > bind05.c:132: TCONF: socket(10, 2, 0) failed: EAFNOSUPPORT (97)
> >
> > Summary:
> > passed   8
> > failed   0
> > broken   0
> > skipped  1
> > warnings 0
> > not run  5
> >
> > have
> > bind05.c:132: TCONF: socket(10, 2, 0) failed: EAFNOSUPPORT (97)
> >
> > Summary:
> > passed   8
> > failed   0
> > broken   0
> > skipped  6
> > warnings 0
> >
> > Kind regards,
> > Petr
> >
> >   lib/tst_test.c | 3 +++
> >   1 file changed, 3 insertions(+)
> >
> > diff --git a/lib/tst_test.c b/lib/tst_test.c
> > index 844756fbd7..e5ea9e6165 100644
> > --- a/lib/tst_test.c
> > +++ b/lib/tst_test.c
> > @@ -811,6 +811,9 @@ static void do_exit(int ret)
> >               fprintf(stderr, "broken   %d\n", results->broken);
> >               fprintf(stderr, "skipped  %d\n", results->skipped);
> >               fprintf(stderr, "warnings %d\n", results->warnings);
> > +             fprintf(stderr, "not run  %d\n", tst_test->tcnt -
> > +                             results->passed - results->failed - results->broken -
> > +                             results->skipped - results->warnings);
> IMO, these values are not necessary related. Refer to quotactl01.c(
> tst_test->tcnt is less than results->passed)

That's because of the test looping 'tst_variant + 1' times.
Maybe we can just multiply it eliminate the distractions?

   tst_test->tcnt * (tst_variant + 1)


>
> Also, if you want to add a new tag, you should define the situation for it.

Theoritically yes, but the problem here is hard to count the
not-run numbers. Because some of the test items will never
be performed if test return by test environment unmatch.
I'm sure we have quite lot of test doing like that.


>
> Best Regards
> Yang Xu
>
> >       }
> >
> >       do_cleanup();
>
> --
> Mailing list info: https://lists.linux.it/listinfo/ltp
>


-- 
Regards,
Li Wang