[LTP] [PATCH V2 1/2] ltp: Add the ability to specify the latency constraint

[Cyril Hrubis]

Hi!
> > That explains it. Previously each of the timer testcases had it's own
> > PASS/FAIL criteria and each of them was slightly different. We got rid
> > of that mess recetly and so the latest git has a timer measurement
> > library and the test only defines a sampling function now. We also did
> > quite a lot of testing to make sure that the test are stable now.  And
> > because of that we take more samples and apply discarded mean to get rid
> > of random outliners. But we did most of the testing on x86 hardware so
> > it's possible that it still needs some adjustements.
> 
> IMO, you should not try to adjust this because there can be a so big gap
> between some arch/platforms in term of exit_latency that can make the
> test to miss a bug. I mean being more tolerant for one arch can make the
> test miss a bug on another arch.
> 
> eg.
> 
> exynos4 : 5000us
> at91: 10us
> ux500: 70us
> mediatek: 600us
> ppc: 10us
> x86: 86us
> sh mobile: 2300us
> 
> etc...

Ok.

> The simplest and cleanest way is to reduce the latency to its minimum in
> order to reduce the energy framework impact on the tests.
> 
> It is recent the mobile runs ltp.

Sounds reasonably then.

> > Can you, please, try with the latest git to see if these tests works for
> > you now? And then, in a case that they stil fail, we will figure out how
> > to fix them. Most likely we will patch the timer test library, either
> > to loosen the crieria or to keep the cpu_dma_latecy open while we sample
> > the timers.
> 
> There is a misunderstanding. I ran the tests (and they fail) on the
> latest one 4a707d417e3f95025fe6c707e2763e84b2bed29a.

Okay, and do all of the timer tests fail or just some subset?

And even if only subset of them fails I would still consider changing
the timer library rather than individual testcases.

-- 
Cyril Hrubis
chrubis@suse.cz