[LTP] [PATCH] mmap22: Bound child memory allocation to avoid OOM

[Petr Vorel]

Hi Attila,

> Currently, stress_child() allocates memory in an infinite loop without
> bounds. Under memory pressure, once the cgroup memory limit is reached
> and the parent's droppable pages are reclaimed, the child continues
> allocating non-droppable heap memory. This eventually exhausts the cgroup
> limit and triggers the cgroup OOM killer, terminating either the child or
> the test parent process.

> Fix this by bounding the child's allocation to CHILD_ALLOC_MAX
> (ALLOC_SIZE + 32MB). The total requested memory (128MB parent + 160MB
> child = 288MB) exceeds the 256MB cgroup limit by 32MB, which reliably
> forces the kernel to reclaim MAP_DROPPABLE pages. Once CHILD_ALLOC_MAX is
> reached, the child calls pause() to keep memory resident. Since 160MB is
> well below the 256MB cgroup limit, non-droppable memory alone cannot
> trigger an OOM.

LGTM, but it'd be nice if somebody else double check.
Reviewed-by: Petr Vorel <pvorel@suse.cz>

Kind regards,
Petr

> Assisted-by: Gemini CLI
> Signed-off-by: Attila Fazekas <afazekas@redhat.com>
> ---
>  testcases/kernel/syscalls/mmap/mmap22.c | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)

> diff --git a/testcases/kernel/syscalls/mmap/mmap22.c b/testcases/kernel/syscalls/mmap/mmap22.c
> index 0b95c578f..16fcb6dd8 100644
> --- a/testcases/kernel/syscalls/mmap/mmap22.c
> +++ b/testcases/kernel/syscalls/mmap/mmap22.c
> @@ -26,6 +26,7 @@

>  #define MEM_LIMIT (256 * TST_MB)
>  #define ALLOC_SIZE (128 * TST_MB)
> +#define CHILD_ALLOC_MAX (ALLOC_SIZE + 32 * TST_MB)

>  static struct tst_cg_group *cg_child;
>  static pid_t child;
> @@ -33,13 +34,15 @@ static int page_size;

>  static void stress_child(void)
>  {
> -	for (;;) {
> +	for (size_t sz = 0; sz < CHILD_ALLOC_MAX; sz += page_size) {
>  		char *buf = malloc(page_size);

>  		if (!buf)
> -			exit(1);
> +			break;
>  		memset(buf, 'B', page_size);
>  	}
> +
> +	pause();
>  }