[LTP] [PATCH] oom03: use size_t for memory length to fix 32-bit overflow

[Ben Copeland]

The alloc_mem() function is supposed to test out of memory conditions.
How it works is it calls mmap() on a giant chunk of memory.
It's either LENGTH (2GB) or "TESTMEM * 2 + TST_MB" (3GB) bytes.
This mmap() is generally supposed to succeed.  Then at the bottom of
the alloc_mem() function when we actually try to use all the memory,
the thread is supposed to die with a SIGKILL.

The problem is that length is signed so on a 32-bit system it will be
negative. That means that at the bottom of the function when we loop
through the memory, the for loop is a no-op and there is no SIGKILL.
Fix this by changing the type to size_t which is unsigned.

Signed-off-by: Ben Copeland <ben.copeland@linaro.org>
---
 testcases/kernel/mem/oom/oom.h | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/testcases/kernel/mem/oom/oom.h b/testcases/kernel/mem/oom/oom.h
index 41cc681f9..42ed181b0 100644
--- a/testcases/kernel/mem/oom/oom.h
+++ b/testcases/kernel/mem/oom/oom.h
@@ -62,13 +62,14 @@ static inline void set_global_mempolicy(int mempolicy)
 static void set_global_mempolicy(int mempolicy LTP_ATTRIBUTE_UNUSED) { }
 #endif
 
-static int alloc_mem(long int length, int testcase)
+static int alloc_mem(size_t length, int testcase)
 {
 	char *s;
-	long i, pagesz = getpagesize();
+	size_t i;
+	long pagesz = getpagesize();
 	int loop = 10;
 
-	tst_res(TINFO, "thread (%lx), allocating %ld bytes.",
+	tst_res(TINFO, "thread (%lx), allocating %zu bytes.",
 		(unsigned long) pthread_self(), length);
 
 	s = mmap(NULL, length, PROT_READ | PROT_WRITE,
@@ -111,7 +112,7 @@ static void child_alloc(int testcase, int lite, int threads)
 	pthread_t *th;
 
 	if (lite) {
-		int ret = alloc_mem(TESTMEM * 2 + TST_MB, testcase);
+		int ret = alloc_mem((size_t)TESTMEM * 2 + TST_MB, testcase);
 		exit(ret);
 	}
 
-- 
2.51.0