[LTP] [RFC] enable OOM protection for the library and test process?

[Cyril Hrubis]

Hi!
> > There are likely more processes that could become unintended targets
> > (e.g. harness process)
> > (if we haven't tried already) Could we make expected victim process
> > more appealing target by tweaking its oom_score/oom_score_adj ?
> 
> I'm afraid it won't be that easy. The main cause of OOM killer going
> postal and killing processes with tiny memory footprint is that a
> process executing the mlock() syscall cannot be targeted by OOM killer
> at all. That's a known issue in the kernel with no easy fix.

This is only single test out of at least 10 that can trigger OOM, right?

> You can protect the main test process using oom_score_adj but chances
> are that OOM killer will just kill PID 1 (kernel panic), or find no
> killable process left (also kernel panic).
> 
> Protecting the test harness is a bad idea because oom_score_adj is
> inherited by child processes and it'll affect other tests as well. Given
> the nature of OOM tests, I'd rather not assume that the protection will
> be properly removed at the end.

This should be easily doable since the test library forks right before
it executes the test, so we have a single place where the score has to
be reset.

For new library tests there is a process that does nothing but waits for
the actuall test pid to finish and kills it on timeout. It really makes
sense to protect this exact process and maybe even mlock() it into the
memory.

-- 
Cyril Hrubis
chrubis@suse.cz