[LTP] [PATCH v2 2/3] Add fanotify_get_supported_init_flags() helper function

[Jan Kara]

On Tue 25-10-22 10:51:01, Martin Doucha wrote:
> On 24. 10. 22 18:18, Amir Goldstein wrote:
> > Show me how this could hide a real bug.
> > Give an example.
> > It does not need to be a specific kernel
> > use an example with imaginary kernel with a backported feature if you like.
> > 
> > fanotify14 is not about making sure that flag combinations are allowed
> > it is about making sure that flag combinations are not allowed.
> > 
> > If the test case is testing illegal init flags, the outcome must be
> > fanotify_init
> > EINVAL.
> > 
> > If the test case is testing illegal mark flags, the outcome of fanotify_init
> > may be EINVAL meaning that this test case will be skipped.
> > It does not matter which specific init flag or init flag combination
> > causes this EINVAL.
> > 
> > I am ready to be proven wrong, but with examples,
> > like the one you provided with test case #8 and kernel 5.3.
> > hand waving and talking about vague "real bugs" won't convince me.
> 
> Imagine two init flags, A and B (doesn't matter which ones) that are not
> supposed to conflict in any way according to documentation. And we'll add 3
> fanotify14 test cases with the following init calls:
> - fanotify_init(A)
> - fanotify_init(B)
> - fanotify_init(A|B)
> 
> All 3 init calls are supposed to pass and then fanotify_mark() is supposed
> to fail. Now imagine that we have a buggy kernel where both flags are
> implemented but fanotify_init(A|B) hits a weird corner case and returns
> EINVAL. In your version of the code, the test will assume that it's due to a
> missing feature and report the test case as skipped. In my version of the
> code, the test will report a bug because it knows that all the required
> features are present.

Yeah, but AFAIU you are trading expanded testing for possibility of false
test failures (because the situation that for some features A and B, both A
and B are implemented but A|B got implemented only later seems equally
probable scenario).

Since I don't find this critical to test (it seems like a relatively
unlikely mistake to do), I'd prefer less testing against false test
failures. If we want to be more precise, we should be spelling out in the
testcase (ideally using some common infrastructure) that if A & B is
supported, we also expect A|B (or even some C) to work.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR