[LTP] [PATCH 5/5] syscalls/fanotify10: Add test cases for evictable ignore mark
Jan Stancek
jstancek@redhat.com
Tue Jul 12 10:19:19 CEST 2022
On Sat, Jul 9, 2022 at 12:09 PM Amir Goldstein <amir73il@gmail.com> wrote:
>
> On Thu, Jul 7, 2022 at 3:49 PM Jan Stancek <jstancek@redhat.com> wrote:
> >
> > On Thu, Jun 30, 2022 at 10:21 AM Amir Goldstein <amir73il@gmail.com> wrote:
> > >
> > > On Thu, Jun 30, 2022 at 9:27 AM Jan Stancek <jstancek@redhat.com> wrote:
> > > >
> > > > On Mon, Jun 13, 2022 at 4:39 PM Amir Goldstein <amir73il@gmail.com> wrote:
> > > > >
> > > > > Test multiple groups with evictable mark with ignore mask
> > > > >
> > > > > Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> > > > > ---
> > > > > .../kernel/syscalls/fanotify/fanotify10.c | 78 +++++++++++++++++++
> > > > > 1 file changed, 78 insertions(+)
> > > > >
> > > > > diff --git a/testcases/kernel/syscalls/fanotify/fanotify10.c b/testcases/kernel/syscalls/fanotify/fanotify10.c
> > > > > index b9a50672d..52277d0b7 100644
> > > > > --- a/testcases/kernel/syscalls/fanotify/fanotify10.c
> > > > > +++ b/testcases/kernel/syscalls/fanotify/fanotify10.c
> > > > > @@ -71,6 +71,7 @@ static char event_buf[EVENT_BUF_LEN];
> > > > > static int exec_events_unsupported;
> > > > > static int fan_report_dfid_unsupported;
> > > > > static int filesystem_mark_unsupported;
> > > > > +static int evictable_mark_unsupported;
> > > > >
> > > > > #define MOUNT_PATH "fs_mnt"
> > > > > #define MNT2_PATH "mntpoint"
> > > > > @@ -90,6 +91,10 @@ static int filesystem_mark_unsupported;
> > > > > #define FILE_EXEC_PATH2 MNT2_PATH"/"TEST_APP
> > > > > #define FILE2_EXEC_PATH2 MNT2_PATH"/"TEST_APP2
> > > > >
> > > > > +#define DROP_CACHES_FILE "/proc/sys/vm/drop_caches"
> > > > > +#define CACHE_PRESSURE_FILE "/proc/sys/vm/vfs_cache_pressure"
> > > > > +
> > > > > +static int old_cache_pressure;
> > > > > static pid_t child_pid;
> > > > > static int bind_mount_created;
> > > > > static unsigned int num_classes = NUM_CLASSES;
> > > > > @@ -98,12 +103,14 @@ enum {
> > > > > FANOTIFY_INODE,
> > > > > FANOTIFY_MOUNT,
> > > > > FANOTIFY_FILESYSTEM,
> > > > > + FANOTIFY_EVICTABLE,
> > > > > };
> > > > >
> > > > > static struct fanotify_mark_type fanotify_mark_types[] = {
> > > > > INIT_FANOTIFY_MARK_TYPE(INODE),
> > > > > INIT_FANOTIFY_MARK_TYPE(MOUNT),
> > > > > INIT_FANOTIFY_MARK_TYPE(FILESYSTEM),
> > > > > + INIT_FANOTIFY_MARK_TYPE(EVICTABLE),
> > > > > };
> > > > >
> > > > > static struct tcase {
> > > > > @@ -289,14 +296,59 @@ static struct tcase {
> > > > > 0,
> > > > > FILE_PATH, FAN_OPEN, FAN_OPEN
> > > > > },
> > > > > + /* Evictable ignore mark test cases */
> > > > > + {
> > > > > + "don't ignore mount events created on file with evicted ignore mark",
> > > > > + MOUNT_PATH, FANOTIFY_MOUNT,
> > > > > + FILE_PATH, FANOTIFY_EVICTABLE,
> > > > > + 0,
> > > > > + FILE_PATH, FAN_OPEN, FAN_OPEN
> > > > > + },
> > > > > + {
> > > > > + "don't ignore fs events created on a file with evicted ignore mark",
> > > > > + MOUNT_PATH, FANOTIFY_FILESYSTEM,
> > > > > + FILE_PATH, FANOTIFY_EVICTABLE,
> > > > > + 0,
> > > > > + FILE_PATH, FAN_OPEN, FAN_OPEN
> > > > > + },
> > > > > + {
> > > > > + "don't ignore mount events created inside a parent with evicted ignore mark",
> > > > > + MOUNT_PATH, FANOTIFY_MOUNT,
> > > > > + DIR_PATH, FANOTIFY_EVICTABLE,
> > > > > + FAN_EVENT_ON_CHILD,
> > > > > + FILE_PATH, FAN_OPEN, FAN_OPEN
> > > > > + },
> > > > > + {
> > > > > + "don't ignore fs events created inside a parent with evicted ignore mark",
> > > > > + MOUNT_PATH, FANOTIFY_FILESYSTEM,
> > > > > + DIR_PATH, FANOTIFY_EVICTABLE,
> > > > > + FAN_EVENT_ON_CHILD,
> > > > > + FILE_PATH, FAN_OPEN, FAN_OPEN
> > > > > + },
> > > >
> > > > Hi,
> > > >
> > > > we are seeing some sporadic failures from this last testcase, with
> > > > recent upstream kernels (v5.19-rc4-14-g941e3e791269).
> > > > Has anyone also ran into it and knows if it's bug on test side or kernel?
>
> Hi Jan,
>
> I am traveling so I cannot work on solving the problem for the next week,
> but I can explain the problem and offer suggestions for short term solutions
> and maybe a longer term solution.
>
> > > >
> > > > fanotify10.c:496: TINFO: Test #27: don't ignore fs events created
> > > > inside a parent with evicted ignore mark
> > > > fanotify10.c:338: TPASS: No fanotify inode marks as expected
> > > > fanotify10.c:338: TPASS: No fanotify inode marks as expected
> > > > fanotify10.c:338: TPASS: No fanotify inode marks as expected
> > > > fanotify10.c:340: TFAIL: Unexpected inode mark (mflags=240,
> > > > mask=8000020 ignored_mask=20)
> > >
> > > It is a test bug.
> > > The problem is that we want to evict an inode, but there is no
> > > reliable mechanism to do that.
> > >
> > > This is the reason for this workaround in fanotify23:
> > >
> > > /* Shrinkers on other fs do not work reliably enough to
> > > guarantee mark eviction on drop_caches */
> > > .dev_fs_type = "ext2",
> > >
> > > I did not encounter the problem with fanotify10 myself, but it should
> > > be the same.
> > > fanotify10 is not filesystem dependent, so if you can apply the same workaround
> > > from fanotify23 and it works on your systems we can do that.
> >
> > Test is using default fs type, which should already be ext2.
>
> I see. You can try xfs on your system to see if it behaves better,
> because xfs has a specialized inode shriker.
>
> > Here's a more complete log from failed test:
> > https://s3.us-east-1.amazonaws.com/arr-cki-prod-datawarehouse-public/datawarehouse-public/2022/06/30/redhat:576928171/build_ppc64le_redhat:576928171_ppc64le/tests/1/results_0001/job.01/recipes/12221009/tasks/8/logs/syscalls.fail.log
> >
>
> Are the failures only in the first test iteration?
I think so. It runs in automated environment, where we see failures
about once a week. I haven't managed to reproduce it by hand yet.
>
> As a long shot, I would try to remove mount_cycle() from setup().
> I see that fanotify23 does not have it and it is not really needed, so
> it may help
> fanotify10 pass the first iteration.
>
> The problem is that the drop_caches knob is not a reliable way to evict inodes,
> so when a test like fanotify10 needs to evict an inode, random factors are mixed
> into the test run.
>
> As a quick band aid, I suggest to replace the TFAIL, "Unexpected inode mark"
> in show_fanotify_marks() with TCONF, because the test failed to setup an
> "evictable ignored mark that gets evicted", propagate a failure return
> value from
> create_fanotify_groups() => show_fanotify_marks() and skip the test case
> instead of failing it.
Thanks for suggestion.
>
> For long term, it would be nice if LTP could provide a
> drop_inode_cache() library
> function that tries harder to make inode eviction work using tricks
> like fanotify10
> and fanotify23 increases vfs_cache_pressure.
> To pressure memory shrinkers to evict more inodes, need to check the amount
> of total RAM in the system and dirty pages and add enough dirty memory pages
> that cannot be evicted to drive the inode shrinker to work harder.
> Utilizing memory cgroups with some of the new per memcg cache eviction knobs
> could also be an option, but I did not look into it and not sure on
> which kernels
> this is available.
>
> Bottom line is that there is a lot of black magic involved with memory shrinkers
> and as your report shows, hacks and trick may impact different systems in
> different ways, so it is better it the infrastructure is developed and tested in
> LTP lib and not in individual tests.
>
> Hope this helps.
>
> Thanks,
> Amir.
>
More information about the ltp
mailing list