[LTP] [RFC] Showing skipped tests in results?
Cyril Hrubis
chrubis@suse.cz
Fri Dec 8 14:34:59 CET 2017
Hi!
> > I think that you misunderstand how things works, the first string on
> > each line in the alltests file that is parsed by ltp-pan is just test
> > name that appears in the test result file, it does not affect anything
> > but that.
> >
> > I guess that the easiest solution would be replacing the actuall command
> > that is supposed to be executed with something that just reports
> > skipped status.
> >
> > Quick hack would be replacing these with something that just returns the
> > right exit value:
> >
> > gzip01 gzip_tests.hs -> gzip01 exit 32
> Not working.
> the second argument is expected to be a file.
Not a true, I've looked to the ltp-pan.c and it does crude heuristic
that decides if the second part of the line is supposed to be binary
name or a shell script, if it finds any shell redirection characters or
semicolon then it's executed as a script.
> gzip01 gzip_tests.hs -> gzip01 skip.sh
> should work.
> We have to create skip.sh inside ltp
> skip.sh contains single line exit 32
gzip01 foo -> gzip01 exit 32;
That one actually works.
> OTOH,
> I have noticed one more problem with sed with updated script.
>
>
> Issues:
> 1. Want to skip only "testcase" write inside skipfile and run ./ltp-skip.sh
Which script are you referring to? There is no ltp-skip.sh in LTP source
code tree.
> testcase is replaced by testcase skip.sh ( = TCONF ) which is expected.
> testcase_64 is replaced by testcase skip.sh which is incorrect testcase name
> testcase_16 is replaced by testcase skip.sh which is incorrect testcase name
> testcase_xxxx is replaced by testcase skip.sh which is incorrect testcase name
>
> diff -Narub alltests.org alltests
> --- alltests.org 2017-12-07 22:41:46.373419820 +0530
> +++ alltests 2017-12-07 23:54:53.188657734 +0530
> @@ -272,8 +272,8 @@
> fcntl33_64 fcntl33_64
> fcntl34 fcntl34
> fcntl34_64 fcntl34_64
> -fcntl35 fcntl35
> -fcntl35_64 fcntl35_64
> +fcntl35 skip.sh
> +fcntl35 skip.sh
> fcntl36 fcntl36
> fcntl36_64 fcntl36_64
That is beacuse you have broken the mathing pattern in your patch, the
original one that just deleted the line matched whitespace after the
name.
> 2. The skipfile have limitation of
> testcase_name should be a single word in SKIPFILE
>
> Feature is
> We can added comments inside SKIPFILE starting with #
> The script will ignore line starting with #
I'm okay with adding comments to skipfile.
> From c3e77bbaf03e126de15524f817960740a8c94e1b Mon Sep 17 00:00:00 2001
> From: Naresh Kamboju <naresh.kamboju@linaro.org>
> Date: Thu, 7 Dec 2017 23:14:44 +0530
> Subject: [PATCH] runltp: SKIPFILE: skip test cases should be visible as TCONF
> in results
>
> When we skip tests in LTP using "./runltp -S SKIPFILE", they get removed
> from the test list completely, leaving no trace in the results.
>
> This patch will add SKIPFILE listed test cases names to results as TCONF
> testcase_name -> testcase_name skip.sh
>
> Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
> ---
> runltp | 12 +++++++-----
> 1 file changed, 7 insertions(+), 5 deletions(-)
>
> diff --git a/runltp b/runltp
> index 8e40d67..f206fe1 100755
> --- a/runltp
> +++ b/runltp
> @@ -692,11 +692,13 @@ main()
> fi
>
> # Blacklist or skip tests if a SKIPFILE was specified with -S
> - if [ -n "$SKIPFILE" ]
> - then
> - for file in $( cat $SKIPFILE ); do
> - sed -i "/^$file[ \t]/d" ${TMP}/alltests
> - done
> + if [ -n "${SKIPFILE}" ]; then
> + while read -r test_name; do
> + case "${test_name}" in \#*) continue ;; esac
> + if [ ${test_name} ]; then
> + sed -i "/${test_name}/c\\${test_name} skip.sh" alltests
> + fi
> + done < ${SKIPFILE}
Apart from the broken sed matching regular expression and the skip.sh
that can be replaced with 'exit 32;' this looks reasonably good.
--
Cyril Hrubis
chrubis@suse.cz
More information about the ltp
mailing list