[LTP] [PATCH v2 0/7] ci: run shell loader tests

[Li Wang]

Hi Petr,

Petr Vorel <pvorel@suse.cz> wrote:


> 2024-12-10T21:07:23.4757888Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:484:
> TINFO: Minimum sampling period ended
> 2024-12-10T21:07:23.4758941Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:307:
> TINFO: loop = 10000, delay_bias = 0
> 2024-12-10T21:07:23.4760498Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: start_a - start_b: { avg =   -10ns, avg_dev =    17ns, dev_ratio =
> 1.62 }
> 2024-12-10T21:07:23.4761822Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: end_a - start_a  : { avg = 15683ns, avg_dev =    76ns, dev_ratio =
> 0.00 }
> 2024-12-10T21:07:23.4763095Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: end_b - start_b  : { avg = 15947ns, avg_dev =    71ns, dev_ratio =
> 0.00 }
> 2024-12-10T21:07:23.4764323Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: end_a - end_b    : { avg =  -275ns, avg_dev =    18ns, dev_ratio =
> 0.07 }
> 2024-12-10T21:07:23.4765493Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: spins            : { avg =   341  , avg_dev =    27  , dev_ratio =
> 0.08 }
> 2024-12-10T21:07:26.9727369Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:494:
> TINFO: Reached deviation ratios < 0.10, introducing randomness
> 2024-12-10T21:07:26.9728387Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:497:
> TINFO: Delay range is [-15945, 15847]
> 2024-12-10T21:07:26.9729228Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:307:
> TINFO: loop = 218505, delay_bias = 0
> 2024-12-10T21:07:26.9730199Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: start_a - start_b: { avg =   -85ns, avg_dev =     8ns, dev_ratio =
> 0.09 }
> 2024-12-10T21:07:26.9731309Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: end_a - start_a  : { avg = 15814ns, avg_dev =     7ns, dev_ratio =
> 0.00 }
> 2024-12-10T21:07:26.9732397Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: end_b - start_b  : { avg = 15913ns, avg_dev =    10ns, dev_ratio =
> 0.00 }
> 2024-12-10T21:07:26.9733462Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: end_a - end_b    : { avg =  -183ns, avg_dev =     9ns, dev_ratio =
> 0.05 }
> 2024-12-10T21:07:26.9734493Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: spins            : { avg =   184  , avg_dev =    10  , dev_ratio =
> 0.05 }



> 2024-12-10T21:08:22.5072327Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:654:
> TINFO: Exceeded execution loops, requesting exit
> 2024-12-10T21:08:22.5073965Z
> /__w/ltp/ltp/lib/newlib_tests/tst_fuzzy_sync01.c:219: TFAIL: acs:0  act:1
> art:3  | =:24   -:2999931 +:45



> runtest TINFO: * tst_fuzzy_sync02
> ...
> 2024-12-10T21:09:01.0848787Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:484:
> TINFO: Minimum sampling period ended
> 2024-12-10T21:09:01.0849660Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:307:
> TINFO: loop = 10000, delay_bias = 9931
> 2024-12-10T21:09:01.0850640Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: start_a - start_b: { avg = -3787ns, avg_dev =   134ns, dev_ratio =
> 0.04 }
> 2024-12-10T21:09:01.0851859Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: end_a - start_a  : { avg =  6983ns, avg_dev =   645ns, dev_ratio =
> 0.09 }
> 2024-12-10T21:09:01.0852990Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: end_b - start_b  : { avg = 13154ns, avg_dev =   190ns, dev_ratio =
> 0.01 }
> 2024-12-10T21:09:01.0854078Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: end_a - end_b    : { avg = -9959ns, avg_dev =   347ns, dev_ratio =
> 0.03 }
> 2024-12-10T21:09:01.0855485Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: spins            : { avg = 16071  , avg_dev =   555  , dev_ratio =
> 0.03 }
> 2024-12-10T21:09:01.0856560Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:494:
> TINFO: Reached deviation ratios < 0.10, introducing randomness
> 2024-12-10T21:09:01.0857498Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:497:
> TINFO: Delay range is [-11296, 21198]
> 2024-12-10T21:09:01.0858300Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:307:
> TINFO: loop = 10001, delay_bias = 9931
> 2024-12-10T21:09:01.0859260Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: start_a - start_b: { avg = -3787ns, avg_dev =   134ns, dev_ratio =
> 0.04 }
> 2024-12-10T21:09:01.0860356Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: end_a - start_a  : { avg =  6983ns, avg_dev =   645ns, dev_ratio =
> 0.09 }
> 2024-12-10T21:09:01.0861433Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: end_b - start_b  : { avg = 13154ns, avg_dev =   190ns, dev_ratio =
> 0.01 }
> 2024-12-10T21:09:01.0862510Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: end_a - end_b    : { avg = -9959ns, avg_dev =   347ns, dev_ratio =
> 0.03 }
> 2024-12-10T21:09:01.0863584Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:295:
> TINFO: spins            : { avg = 16071  , avg_dev =   555  , dev_ratio =
> 0.03 }
>


> 2024-12-10T21:09:52.6826456Z /__w/ltp/ltp/include/tst_fuzzy_sync.h:654:
> TINFO: Exceeded execution loops, requesting exit
> 2024-12-10T21:09:52.6827841Z
> /__w/ltp/ltp/lib/newlib_tests/tst_fuzzy_sync02.c:164: TFAIL: 2| =:15



>From what I understand, CI environments (GitHub Actions) are shared
infrastructures with limited control over CPU allocation. CPU time slices
may
be interrupted or delayed due to resource contention with other CI jobs.

As both last TINFO print "Exceeded execution loops, requesting exit,"
they fail to meet the success criteria due to resource constraints.
So I guess we'd better stop the test once the loops terminate early.
That's the way to guarantee the precise data needed to satisfy certain
scenarios (HW, VM, CI, etc.).

I will send an improvement patch later. And your patchset itself looks good.

Reviewed-by: Li Wang <liwang@redhat.com>


-- 
Regards,
Li Wang