[LTP] [PATCH 1/2] lib: multiply the timeout if detect slow kconfigsD

[Petr Vorel]

> Hi!
> > > > I did a quick grep that some ltp-aiodio tests set it to 1800 sec, which
> > > > only 8/91 occupation in the LTP testcases/, I'm not sure if it's worth
> > > > adding a new field for those few stress tests.

> > > > And with the previous method, the multiple 4 max_runtime for 1800s
> > > > is 2hours per test up limit, I can't imagine how long we will get eventually
> > > > in the whole test time.

> > > > Maybe another way is to create a separate function in a header
> > > > like aio_common.h (or in high-level dir) for handling that significantly
> > > > larger runtime tests.

> > > > BTW, we have TST_UNLIMITED_RUNTIME choice or, invoke
> > > > tst_set_max_runtime() in somehow.

> > > I stil think that misusing max_runtime, which is supposed to be upper
> > > bound for the actual test runtime was a mistake.

> > Do you want to separate timeout for setup() and for actual test run?
> > Which one would be prolonged in case of "slow" kernels? (e.g. this patch).

> Currently the test timeout applies both for the setup, single test
> iteration and cleanup. So the short answer is both.

> > Or you want to fix -iN?

> After each test iteration the child signals the parent that the
> iteration has been finished and that restarts the timeout timer.

OK, -iN is already solved. Obviously, otherwise testing -i2000 would not work.

> The test overall timeout is computed in set_timeout() and currently is
> computed as:

> 	results->timeout = tst_multiply_timeout(timeout) + results->max_runtime;


> And the tst_multiply_tiemout() would then be changed into:

> 	return (DEFAULT_TIMEOUT + tst_test->timeout) * timeout_mul;

> > Also for docs purposes it might be useful to list long running tests.
> > Fortunately there are just few tests which calls tst_set_max_runtime()
> > for dynamically set timeout.

> That would be actually easier, because we could do:

> - add special value TST_RUNTIME_TIMEOUT
> - allow test to change timeout only if TST_RUNTIME_TIMEOUT was set in
>   tst_test
And if value does not get changed it's the default value. Also name is a bit
confusing (you suggest to have members "timeout", "runtime"), which suggest the
special value is related to both.

OK, flag which would allow us to see that time will be changed.

> Then all long running test would have either tst_test->timeout or
> tst_test->runtime set.

> Technically we would need two special timeout values
> TST_UNLIMITED_TIMEOUT and TST_RUNTIME_TIMEOUT.

OK, TST_UNLIMITED_TIMEOUT is equivalent of:

#define TST_UNLIMITED_RUNTIME (-1)

Maybe have just single definition TST_UNLIMITED, which could be used in both
tst_test->timeout and tst_test->runtime? But that's just an implementation
detail.

> > > Maybe we should have called the max_runtime a timeout and add runtime
> > > for tests that needs it. That way we would have timeout compromising of
> > > two parts, one would be the 30s that is used for all tests and second
> > > part from the tst_test structure. And then the sum of these two would be
> > > multiplied by the timeout multipliers. Then we would have a runtime,
> > > which would be used only by tests that call tst_remaining_runtime().

OK, the point of whole change is to separate some general timeout (30 sec) from
runtime (used for tst_remaining_runtime()), right?

> > > The overall test timeout would be then:

> > > (default_30s_timeout + tst_test->timeout) * TIMEOUT_MUL + tst_test->runtime * RUNTIME_MUL

(default_30s_timeout + tst_test->timeout) * TIMEOUT_MUL is meant for setup or
cleanup and library overhead, tst_test->runtime * RUNTIME_MUL for running test
function?

> > > What do you think?

> > Timeout is for setup function, right? e.g. for ioctl_sg01? If yes, timeout is
> > too generic, IMHO many people will think that it's a general test timeout.
> > I would think about general name.

> Not only, the default 30s timeout is for the whole testrun for tests
> that are quick enough. We have a lot of tests that run just for less
> than 1s even on small embedded boards.

And yes, with properly set data, 30s could be carefully lowered down a bit.

Kind regards,
Petr

> > The above formula should be written in the docs in the separate description and
> > this section should be linked in TIMEOUT_MUL and RUNTIME_MUL description [1].

> Yes.