[LTP] TWARN and "ltp-pan reported PASS"

[Cyril Hrubis]

Hi!
> our shell scripts wrapping ltp-pan binary report failure when there is any TWARN.
> 
> Similar code in these scripts:
> runltp
> runltplite.sh
> testscripts/diskio.sh
> testscripts/network.sh
> testscripts/runEALtests.sh
> 
> if [ $? -eq "0" ]; then
>   echo ltp-pan reported PASS
> else
>   echo ltp-pan reported FAIL
> fi
>
> Does using TWARN always always mean failure? I mean, it's good that ltp-pan returns 4
> instead of 0, but wrapper script should return something like:
>
> if [ $? -eq 0 ]; then
>   echo ltp-pan reported PASS
> elif [ $? -eq 4 ]; then
>   echo ltp-pan reported PASS with warnings
> else
>   echo ltp-pan reported FAIL
> fi

Are you sure that ltp-pan actually passes the test return value? As far
as I can tell it actually increments the exit_stat variable on error
then does exit(exit_stat).

So I would say that the only guarantee that you have is that non-zero
return value from ltp-pan means "everything went ok" and non-zero
"something went wrong".

Frankly at this point I would rather kept the old test execution scripts
as they are and put more effort into the new execution framework.
Hopefully I will have a bit time to invest into it once we are done with
the release.

-- 
Cyril Hrubis
chrubis@suse.cz