[LTP] [PATCH] oom03: use size_t for memory length to fix 32-bit overflow

[Li Wang]

On Thu, Oct 16, 2025 at 10:03 PM Ben Copeland <ben.copeland@linaro.org>
wrote:

> The alloc_mem() function is supposed to test out of memory conditions.
> How it works is it calls mmap() on a giant chunk of memory.
> It's either LENGTH (2GB) or "TESTMEM * 2 + TST_MB" (3GB) bytes.
> This mmap() is generally supposed to succeed.  Then at the bottom of
> the alloc_mem() function when we actually try to use all the memory,
> the thread is supposed to die with a SIGKILL.
>
> The problem is that length is signed so on a 32-bit system it will be
> negative. That means that at the bottom of the function when we loop
> through the memory, the for loop is a no-op and there is no SIGKILL.
> Fix this by changing the type to size_t which is unsigned.
>
> Signed-off-by: Ben Copeland <ben.copeland@linaro.org>
>

Reviewed-by: Li Wang <liwang@redhat.com>

---
>  testcases/kernel/mem/oom/oom.h | 9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)
>
> diff --git a/testcases/kernel/mem/oom/oom.h
> b/testcases/kernel/mem/oom/oom.h
> index 41cc681f9..42ed181b0 100644
> --- a/testcases/kernel/mem/oom/oom.h
> +++ b/testcases/kernel/mem/oom/oom.h
> @@ -62,13 +62,14 @@ static inline void set_global_mempolicy(int mempolicy)
>  static void set_global_mempolicy(int mempolicy LTP_ATTRIBUTE_UNUSED) { }
>  #endif
>
> -static int alloc_mem(long int length, int testcase)
> +static int alloc_mem(size_t length, int testcase)
>  {
>         char *s;
> -       long i, pagesz = getpagesize();
> +       size_t i;
> +       long pagesz = getpagesize();
>         int loop = 10;
>
> -       tst_res(TINFO, "thread (%lx), allocating %ld bytes.",
> +       tst_res(TINFO, "thread (%lx), allocating %zu bytes.",
>                 (unsigned long) pthread_self(), length);
>
>         s = mmap(NULL, length, PROT_READ | PROT_WRITE,
> @@ -111,7 +112,7 @@ static void child_alloc(int testcase, int lite, int
> threads)
>         pthread_t *th;
>
>         if (lite) {
> -               int ret = alloc_mem(TESTMEM * 2 + TST_MB, testcase);
> +               int ret = alloc_mem((size_t)TESTMEM * 2 + TST_MB,
> testcase);
>                 exit(ret);
>         }
>
> --
> 2.51.0
>
>
> --
> Mailing list info: https://lists.linux.it/listinfo/ltp
>
>

-- 
Regards,
Li Wang