[LTP] [PATCH] cve-2017-17052: tolerate ENOMEM during test

[Li Wang]

As each iteration of mmap_thread() grabs a fresh 16 MiB MAP_POPULATE
region and never releases it. As the loop runs, those regions
accumulate consuming both virtual address space and committed physical
memory right away instead of lazily.

Easily mmap() fails with ENOMEM on smaller/limit RAM resource system.

Error Log:

  cve-2017-17052.c:48: TBROK: mmap((nil),16777216,PROT_READ(1),32802,-1,0) failed: ENOMEM (12)
  tst_test.c:479: TINFO: Child process reported TBROK killing the test
  tst_test.c:1909: TINFO: Killed the leftover descendant processes

Consider there is no practical upper bound on this allocation pattern,
so setting .test.min_mem_avail may not helps. Here we just tolerate
ENOMEM during the mmap_thread() looping.

Signed-off-by: Li Wang <liwang@redhat.com>
---
 testcases/cve/cve-2017-17052.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/testcases/cve/cve-2017-17052.c b/testcases/cve/cve-2017-17052.c
index 700eb782e..61032c197 100644
--- a/testcases/cve/cve-2017-17052.c
+++ b/testcases/cve/cve-2017-17052.c
@@ -44,9 +44,16 @@ static void cleanup(void)
 
 static void *mmap_thread(void *arg)
 {
+	void *ptr;
+
 	for (;;) {
-		SAFE_MMAP(NULL, 0x1000000, PROT_READ,
+		ptr = mmap(NULL, 0x1000000, PROT_READ,
 				MAP_POPULATE|MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
+
+		if ((ptr == MAP_FAILED) && (errno == ENOMEM)) {
+			usleep(1000);
+			continue;
+		}
 	}
 
 	return arg;
-- 
2.52.0