[LTP] [PATCH V2 1/2] ltp: Add the ability to specify the latency constraint

[Daniel Lezcano]

On 14/08/2017 16:36, Cyril Hrubis wrote:
> Hi!
>>> That explains it. Previously each of the timer testcases had it's own
>>> PASS/FAIL criteria and each of them was slightly different. We got rid
>>> of that mess recetly and so the latest git has a timer measurement
>>> library and the test only defines a sampling function now. We also did
>>> quite a lot of testing to make sure that the test are stable now.  And
>>> because of that we take more samples and apply discarded mean to get rid
>>> of random outliners. But we did most of the testing on x86 hardware so
>>> it's possible that it still needs some adjustements.
>>
>> IMO, you should not try to adjust this because there can be a so big gap
>> between some arch/platforms in term of exit_latency that can make the
>> test to miss a bug. I mean being more tolerant for one arch can make the
>> test miss a bug on another arch.
>>
>> eg.
>>
>> exynos4 : 5000us
>> at91: 10us
>> ux500: 70us
>> mediatek: 600us
>> ppc: 10us
>> x86: 86us
>> sh mobile: 2300us
>>
>> etc...
> 
> Ok.
> 
>> The simplest and cleanest way is to reduce the latency to its minimum in
>> order to reduce the energy framework impact on the tests.
>>
>> It is recent the mobile runs ltp.
> 
> Sounds reasonably then.
> 
>>> Can you, please, try with the latest git to see if these tests works for
>>> you now? And then, in a case that they stil fail, we will figure out how
>>> to fix them. Most likely we will patch the timer test library, either
>>> to loosen the crieria or to keep the cpu_dma_latecy open while we sample
>>> the timers.
>>
>> There is a misunderstanding. I ran the tests (and they fail) on the
>> latest one 4a707d417e3f95025fe6c707e2763e84b2bed29a.
> 
> Okay, and do all of the timer tests fail or just some subset?

Actually I did not run the entire test suite, I ran the tests using the
tst_timer_start() function:

  ---------------------------------------------
 | latency constraint	infinite	 0     |
  ---------------------------------------------
 | nanosleep01		failed		pass   |
  ---------------------------------------------
 | nanosleep02		pass		pass   |
  ---------------------------------------------
 | fcntl33		pass		pass   |
  ---------------------------------------------	
 | clock_nanosleep02	failed		pass   |
  ---------------------------------------------
 | epoll_wait02		failed		pass   |
  ---------------------------------------------
 | futex_wait05		failed		pass   |
  ---------------------------------------------

> And even if only subset of them fails I would still consider changing
> the timer library rather than individual testcases.

Yes, that make sense.

Do you want keep the latency option for future use?


-- 
 <http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog