[LTP] [PATCH 1/2] lib: multiply the timeout if detect slow kconfigsD

[Cyril Hrubis]

Hi!
> > > I did a quick grep that some ltp-aiodio tests set it to 1800 sec, which
> > > only 8/91 occupation in the LTP testcases/, I'm not sure if it's worth
> > > adding a new field for those few stress tests.
> 
> > > And with the previous method, the multiple 4 max_runtime for 1800s
> > > is 2hours per test up limit, I can't imagine how long we will get eventually
> > > in the whole test time.
> 
> > > Maybe another way is to create a separate function in a header
> > > like aio_common.h (or in high-level dir) for handling that significantly
> > > larger runtime tests.
> 
> > > BTW, we have TST_UNLIMITED_RUNTIME choice or, invoke
> > > tst_set_max_runtime() in somehow.
> 
> > I stil think that misusing max_runtime, which is supposed to be upper
> > bound for the actual test runtime was a mistake.
> 
> Do you want to separate timeout for setup() and for actual test run?
> Which one would be prolonged in case of "slow" kernels? (e.g. this patch).

Currently the test timeout applies both for the setup, single test
iteration and cleanup. So the short answer is both.

> Or you want to fix -iN?

After each test iteration the child signals the parent that the
iteration has been finished and that restarts the timeout timer.

The test overall timeout is computed in set_timeout() and currently is
computed as:

	results->timeout = tst_multiply_timeout(timeout) + results->max_runtime;


And the tst_multiply_tiemout() would then be changed into:

	return (DEFAULT_TIMEOUT + tst_test->timeout) * timeout_mul;

> Also for docs purposes it might be useful to list long running tests.
> Fortunately there are just few tests which calls tst_set_max_runtime()
> for dynamically set timeout.

That would be actually easier, because we could do:

- add special value TST_RUNTIME_TIMEOUT
- allow test to change timeout only if TST_RUNTIME_TIMEOUT was set in
  tst_test

Then all long running test would have either tst_test->timeout or
tst_test->runtime set.

Technically we would need two special timeout values
TST_UNLIMITED_TIMEOUT and TST_RUNTIME_TIMEOUT.

> > Maybe we should have called the max_runtime a timeout and add runtime
> > for tests that needs it. That way we would have timeout compromising of
> > two parts, one would be the 30s that is used for all tests and second
> > part from the tst_test structure. And then the sum of these two would be
> > multiplied by the timeout multipliers. Then we would have a runtime,
> > which would be used only by tests that call tst_remaining_runtime().
> 
> > The overall test timeout would be then:
> 
> > (default_30s_timeout + tst_test->timeout) * TIMEOUT_MUL + tst_test->runtime * RUNTIME_MUL
> 
> > What do you think?
> 
> Timeout is for setup function, right? e.g. for ioctl_sg01? If yes, timeout is
> too generic, IMHO many people will think that it's a general test timeout.
> I would think about general name.

Not only, the default 30s timeout is for the whole testrun for tests
that are quick enough. We have a lot of tests that run just for less
than 1s even on small embedded boards.

> The above formula should be written in the docs in the separate description and
> this section should be linked in TIMEOUT_MUL and RUNTIME_MUL description [1].

Yes.

-- 
Cyril Hrubis
chrubis@suse.cz