<div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-size:small"><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, May 12, 2022 at 8:36 PM Cyril Hrubis <<a href="mailto:chrubis@suse.cz" target="_blank">chrubis@suse.cz</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">This commit introduces a concept of max test runtime. In other words<br>
test runtime is capped at a certain value in order to make testruns more<br>
deterministic. Test is free to to finish before the runtime is used up,<br>
for example when maximal number of iterations was reached, but test must<br>
stop once the runtime has been used up.<br>
<br>
Testcases that run for more than a second or two must check for<br>
remaining runtime by regular calls to tst_remaining_runtime() and should<br>
exit when zero is returned.<br>
<br>
The test max runtime must be set either by the .max_iteration_runtime in<br></blockquote><div><br></div><div><div class="gmail_default" style="font-size:small">.max_runtime</div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
the tst_test structure or in the test setup by a call to<br>
tst_set_runtime().<br></blockquote><div><br></div><div><div class="gmail_default" style="font-size:small">tst_set_max_runtime()</div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
The test timeout is then computed as a sum of DEFAULT_TIMEOUT (currently<br>
set to 30 seconds) and the test runtime. The DEFAULT_TIMEOUT is nothing<br>
more than a safety margin for teardown of the test.<br>
<br>
This commit also maps the -I parameter to the test max runtime if<br>
available and introduces LTP_RUNTIME_MUL enviroment variable so that we<br>
have an easy controll over the runtime cap.<br>
<br>
Lastly but not least the function related to the timeout are turned into<br>
no-op by this commit and removed after all test are converted to the<br>
runtime API.<br>
<br>
Signed-off-by: Cyril Hrubis <<a href="mailto:chrubis@suse.cz" target="_blank">chrubis@suse.cz</a>><br>
---<br>
doc/c-test-api.txt | 42 +++++--<br>
doc/user-guide.txt | 30 ++++-<br>
include/tst_test.h | 27 ++++<br>
lib/newlib_tests/.gitignore | 5 +-<br>
lib/newlib_tests/runtest.sh | 2 +-<br>
lib/newlib_tests/test10.c | 22 ----<br>
lib/newlib_tests/test12.c | 21 ----<br>
lib/newlib_tests/test13.c | 1 -<br>
lib/newlib_tests/test18.c | 22 ----<br>
lib/newlib_tests/test_children_cleanup.c | 1 -<br>
lib/newlib_tests/test_runtime01.c | 30 +++++<br>
lib/newlib_tests/test_runtime02.c | 28 +++++<br>
lib/tst_test.c | 150 +++++++++++++++++------<br>
13 files changed, 264 insertions(+), 117 deletions(-)<br>
delete mode 100644 lib/newlib_tests/test10.c<br>
delete mode 100644 lib/newlib_tests/test12.c<br>
delete mode 100644 lib/newlib_tests/test18.c<br>
create mode 100644 lib/newlib_tests/test_runtime01.c<br>
create mode 100644 lib/newlib_tests/test_runtime02.c<br>
<br>
diff --git a/doc/c-test-api.txt b/doc/c-test-api.txt<br>
index 9f104ecd7..711b445d9 100644<br>
--- a/doc/c-test-api.txt<br>
+++ b/doc/c-test-api.txt<br>
@@ -93,14 +93,35 @@ in range of [0, '.tcnt' - 1].<br>
<br>
IMPORTANT: Only one of '.test' and '.test_all' can be set at a time.<br>
<br>
-Each test has a default timeout set to 300s. The default timeout can be<br>
-overridden by setting '.timeout' in the test structure or by calling<br>
-'tst_set_timeout()' in the test 'setup()'. There are a few testcases whose run<br>
-time may vary arbitrarily, for these timeout can be disabled by setting it to<br>
--1.<br>
+Each test has a limit on how long it can run and the limit composes of two<br>
+parts max_runtime and timeout. The max_runtime is a limit for how long can the<br>
+'.test_all' or a set of '.test' functions take and the timeout is static part<br>
+that should cover the duration of test setup and cleanup plus some safety.<br>
<br>
-Test can find out how much time (in seconds) is remaining to timeout,<br>
-by calling 'tst_timeout_remaining()'.<br>
+Any test that runs for more than a second or two has to make sure to:<br>
+<br>
+- set the runtime either by setting the '.max_runtime' in tst_test or by<br>
+ calling 'tst_set_runtime()' in the test setup<br></blockquote><div><br></div><div class="gmail_default" style="font-size:small"> ^ calling 'tst_set_max_runtime()'</div><div class="gmail_quote"><div class="gmail_default"><br></div><div class="gmail_default">Apart from the patch-22 tiny issue, the rest part looks good.</div><div class="gmail_default"><br></div><div class="gmail_default"></div></div><div class="gmail_default" style="font-size:small">Reviewed-by: Li Wang <<a href="mailto:liwang@redhat.com" target="_blank">liwang@redhat.com</a>></div><div class="gmail_default" style="font-size:small"><br></div></div><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div>Regards,<br></div><div>Li Wang<br></div></div></div></div>