[LTP] [PATCH v3] runltp: add -R option for randomize test order

[Zorro Lang]

On Sun, Sep 25, 2016 at 05:38:07PM +0800, Li Wang wrote:
> On Sat, Sep 24, 2016 at 8:35 PM, Zorro Lang <zlang@redhat.com> wrote:
> > Generally we write test entries into runtests/${testfile}, then
> > runltp reads and runs lines sequentially everytime, nearly no one
> > will change the test order.
> >
> > Recently we find bugs by change the test order，different test
> > combination maybe find more bugs. So I add this new option -R to
> > randomize test order.
> >
> > Signed-off-by: Zorro Lang <zlang@redhat.com>
> > ---
> >
> > V2 use 'sort -R' to instead of 'randlines.awk'.
> > V3 remove change log and someone useless comment
> >
> > Thanks,
> > Zorro
> >
> >  runltp | 11 ++++++++++-
> >  1 file changed, 10 insertions(+), 1 deletion(-)
> >
> > diff --git a/runltp b/runltp
> > index bb2e5d8..33e23ab 100755
> > --- a/runltp
> > +++ b/runltp
> > @@ -153,6 +153,7 @@ usage()
> >      -p              Human readable format logfiles.
> >      -q              Print less verbose output to screen.
> >      -r LTPROOT      Fully qualified path where testsuite is installed.
> > +    -R              Randomize test order.
> >      -s PATTERN      Only run test cases which match PATTERN.
> >      -S SKIPFILE     Skip tests specified in SKIPFILE
> >      -t DURATION     Execute the testsuite for given duration. Examples:
> > @@ -213,12 +214,13 @@ main()
> >      local EMAIL_TO=""
> >      local TAG_RESTRICT_STRING=""
> >      local PAN_COMMAND=""
> > +    local RANDOMRUN=0
> >      local DEFAULT_FILE_NAME_GENERATION_TIME=`date +"%Y_%m_%d-%Hh_%Mm_%Ss"`
> >      local scenfile=
> >
> >      version_date=$(cat "$LTPROOT/Version")
> >
> > -    while getopts a:c:C:T:d:D:f:F:ehi:I:K:g:l:m:M:Nno:pqr:s:S:t:T:w:x:b:B:z:Z: arg
> > +    while getopts a:c:C:T:d:D:f:F:ehi:I:K:g:l:m:M:Nno:pqr:Rs:S:t:T:w:x:b:B:z:Z: arg
> >      do  case $arg in
> >          a)  EMAIL_TO=$OPTARG
> >              ALT_EMAIL_OUT=1;;
> > @@ -410,6 +412,8 @@ main()
> >
> >          r)  LTPROOT=$OPTARG;;
> >
> > +        R)  RANDOMRUN=1;;
> > +
> >          s)  TAG_RESTRICT_STRING=$OPTARG;;
> >
> >         S)  case $OPTARG in
> > @@ -740,6 +744,11 @@ main()
> >           done
> >      fi
> >
> > +    if [ "$RANDOMRUN" != "0" ]; then
> > +        sort -R ${TMP}/alltests > ${TMP}/alltests.temp
> > +        cat ${TMP}/alltests.temp > ${TMP}/alltests
> 
> How about combine them in one line?
> 
>   shuf ${TMP}/alltests -o ${TMP}/alltests
> or,
>   sort -R ${TMP}/alltests -o ${TMP}/alltests

Thanks Li:)

Hm, it works, even works for large file (>pagesize). I thought it maybe make
mistake if read and rewrite same file at same time when I sent this patch.
Now I just read the code of sort.c, it'll read the file into buffer and temp
file at first, then close the fopen(r) file and go to fopen(w) the file:

          if (buf.eof && !nfiles && !ntemps && !buf.left)
            {
              xfclose (fp, file);
              tfp = xfopen (output_file, "w");
	      ...
	      ...

So it looks safe to read and write the same file together ... But I don't know
if it worth changing this patch (the old way looks safer:). Can maintainers
choose a favorite way?

Thanks,
Zorro

> 
> > +    fi
> > +
> >      [ ! -z "$QUIET_MODE" ] && { echo "INFO: Test start time: $(date)" ; }
> >      PAN_COMMAND="${LTPROOT}/bin/ltp-pan $QUIET_MODE -e -S $INSTANCES $DURATION -a $$ \
> >      -n $$ $PRETTY_PRT -f ${TMP}/alltests $LOGFILE $OUTPUTFILE $FAILCMDFILE $TCONFCMDFILE"
> > --
> > 2.7.4
> >
> >
> > --
> > Mailing list info: https://lists.linux.it/listinfo/ltp
> 
> 
> 
> -- 
> Regards,
> Li Wang
> Email: liwang@redhat.com