[LTP] [PATCH v3] runltp: add -R option for randomize test order

[Li Wang]

On Sat, Sep 24, 2016 at 8:35 PM, Zorro Lang <zlang@redhat.com> wrote:
> Generally we write test entries into runtests/${testfile}, then
> runltp reads and runs lines sequentially everytime, nearly no one
> will change the test order.
>
> Recently we find bugs by change the test order，different test
> combination maybe find more bugs. So I add this new option -R to
> randomize test order.
>
> Signed-off-by: Zorro Lang <zlang@redhat.com>
> ---
>
> V2 use 'sort -R' to instead of 'randlines.awk'.
> V3 remove change log and someone useless comment
>
> Thanks,
> Zorro
>
>  runltp | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/runltp b/runltp
> index bb2e5d8..33e23ab 100755
> --- a/runltp
> +++ b/runltp
> @@ -153,6 +153,7 @@ usage()
>      -p              Human readable format logfiles.
>      -q              Print less verbose output to screen.
>      -r LTPROOT      Fully qualified path where testsuite is installed.
> +    -R              Randomize test order.
>      -s PATTERN      Only run test cases which match PATTERN.
>      -S SKIPFILE     Skip tests specified in SKIPFILE
>      -t DURATION     Execute the testsuite for given duration. Examples:
> @@ -213,12 +214,13 @@ main()
>      local EMAIL_TO=""
>      local TAG_RESTRICT_STRING=""
>      local PAN_COMMAND=""
> +    local RANDOMRUN=0
>      local DEFAULT_FILE_NAME_GENERATION_TIME=`date +"%Y_%m_%d-%Hh_%Mm_%Ss"`
>      local scenfile=
>
>      version_date=$(cat "$LTPROOT/Version")
>
> -    while getopts a:c:C:T:d:D:f:F:ehi:I:K:g:l:m:M:Nno:pqr:s:S:t:T:w:x:b:B:z:Z: arg
> +    while getopts a:c:C:T:d:D:f:F:ehi:I:K:g:l:m:M:Nno:pqr:Rs:S:t:T:w:x:b:B:z:Z: arg
>      do  case $arg in
>          a)  EMAIL_TO=$OPTARG
>              ALT_EMAIL_OUT=1;;
> @@ -410,6 +412,8 @@ main()
>
>          r)  LTPROOT=$OPTARG;;
>
> +        R)  RANDOMRUN=1;;
> +
>          s)  TAG_RESTRICT_STRING=$OPTARG;;
>
>         S)  case $OPTARG in
> @@ -740,6 +744,11 @@ main()
>           done
>      fi
>
> +    if [ "$RANDOMRUN" != "0" ]; then
> +        sort -R ${TMP}/alltests > ${TMP}/alltests.temp
> +        cat ${TMP}/alltests.temp > ${TMP}/alltests

How about combine them in one line?

  shuf ${TMP}/alltests -o ${TMP}/alltests
or,
  sort -R ${TMP}/alltests -o ${TMP}/alltests

> +    fi
> +
>      [ ! -z "$QUIET_MODE" ] && { echo "INFO: Test start time: $(date)" ; }
>      PAN_COMMAND="${LTPROOT}/bin/ltp-pan $QUIET_MODE -e -S $INSTANCES $DURATION -a $$ \
>      -n $$ $PRETTY_PRT -f ${TMP}/alltests $LOGFILE $OUTPUTFILE $FAILCMDFILE $TCONFCMDFILE"
> --
> 2.7.4
>
>
> --
> Mailing list info: https://lists.linux.it/listinfo/ltp



-- 
Regards,
Li Wang
Email: liwang@redhat.com