[LTP] [PATCH] cve-2017-17052: tolerate ENOMEM during test

[Li Wang]

On Thu, Jan 29, 2026 at 6:00 PM Li Wang via ltp <ltp@lists.linux.it> wrote:
>
> As each iteration of mmap_thread() grabs a fresh 16 MiB MAP_POPULATE
> region and never releases it. As the loop runs, those regions
> accumulate consuming both virtual address space and committed physical
> memory right away instead of lazily.
>
> Easily mmap() fails with ENOMEM on smaller/limit RAM resource system.
>
> Error Log:
>
>   cve-2017-17052.c:48: TBROK: mmap((nil),16777216,PROT_READ(1),32802,-1,0) failed: ENOMEM (12)
>   tst_test.c:479: TINFO: Child process reported TBROK killing the test
>   tst_test.c:1909: TINFO: Killed the leftover descendant processes
>
> Consider there is no practical upper bound on this allocation pattern,
> so setting .test.min_mem_avail may not helps. Here we just tolerate
> ENOMEM during the mmap_thread() looping.
>
> Signed-off-by: Li Wang <liwang@redhat.com>
> ---
>  testcases/cve/cve-2017-17052.c | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/testcases/cve/cve-2017-17052.c b/testcases/cve/cve-2017-17052.c
> index 700eb782e..61032c197 100644
> --- a/testcases/cve/cve-2017-17052.c
> +++ b/testcases/cve/cve-2017-17052.c
> @@ -44,9 +44,16 @@ static void cleanup(void)
>
>  static void *mmap_thread(void *arg)
>  {
> +       void *ptr;
> +
>         for (;;) {
> -               SAFE_MMAP(NULL, 0x1000000, PROT_READ,
> +               ptr = mmap(NULL, 0x1000000, PROT_READ,
>                                 MAP_POPULATE|MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
> +
> +               if ((ptr == MAP_FAILED) && (errno == ENOMEM)) {
> +                       usleep(1000);
> +                       continue;
> +               }

Oops, I forgot to handle the other failures, so the patch should be:

--- a/testcases/cve/cve-2017-17052.c
+++ b/testcases/cve/cve-2017-17052.c
@@ -44,9 +44,19 @@ static void cleanup(void)

 static void *mmap_thread(void *arg)
 {
+       void *ptr;
+
        for (;;) {
-               SAFE_MMAP(NULL, 0x1000000, PROT_READ,
+               ptr = mmap(NULL, 0x1000000, PROT_READ,
                                MAP_POPULATE|MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
+
+               if (ptr == MAP_FAILED) {
+                       if (errno == ENOMEM) {
+                               usleep(1000);
+                               continue;
+                       }
+                       tst_brk(TBROK | TTERRNO, "Unexpected mmap() error");
+               }
        }

        return arg;



-- 
Regards,
Li Wang