[LTP] [PATCH v2 2/3] Add fanotify_get_supported_init_flags() helper function

[Amir Goldstein]

On Tue, Oct 25, 2022 at 11:51 AM Martin Doucha <mdoucha@suse.cz> wrote:
>
> On 24. 10. 22 18:18, Amir Goldstein wrote:
> > Show me how this could hide a real bug.
> > Give an example.
> > It does not need to be a specific kernel
> > use an example with imaginary kernel with a backported feature if you like.
> >
> > fanotify14 is not about making sure that flag combinations are allowed
> > it is about making sure that flag combinations are not allowed.
> >
> > If the test case is testing illegal init flags, the outcome must be
> > fanotify_init
> > EINVAL.
> >
> > If the test case is testing illegal mark flags, the outcome of fanotify_init
> > may be EINVAL meaning that this test case will be skipped.
> > It does not matter which specific init flag or init flag combination
> > causes this EINVAL.
> >
> > I am ready to be proven wrong, but with examples,
> > like the one you provided with test case #8 and kernel 5.3.
> > hand waving and talking about vague "real bugs" won't convince me.
>
> Imagine two init flags, A and B (doesn't matter which ones) that are not
> supposed to conflict in any way according to documentation. And we'll
> add 3 fanotify14 test cases with the following init calls:
> - fanotify_init(A)
> - fanotify_init(B)
> - fanotify_init(A|B)
>
> All 3 init calls are supposed to pass and then fanotify_mark() is
> supposed to fail. Now imagine that we have a buggy kernel where both
> flags are implemented but fanotify_init(A|B) hits a weird corner case
> and returns EINVAL.
> In your version of the code, the test will assume
> that it's due to a missing feature and report the test case as skipped.
> In my version of the code, the test will report a bug because it knows
> that all the required features are present.
>

It is a valid test case to assert that the support for two flags is
independent,
but this is not the job of fanotify14.
fanotify14 checks for *illegal* flag combinations.

If you feel that there should be a test that verifies that
support of flag A is independent of support of flag B,
then please write a different test for that.

But then would you test all possible permutations of flags?
Not only flags that are used in fanotify14?
Not only flag pairs? but more concurrent flags?
I don't know if other APIs have such rigorous tests (except API fuzzers).

I agree with Jan that the value of such a test would be questionable,
but it does have a value, so I won't object to having this test, as
long as it does not blindly check for all the known fanotify init bits
are independent.

Asserting flag combination independence should be opt-in by the test
not out-out like you did with REPORT_FID and REPORT_NAME.

Thanks,
Amir.