[LTP] [kirk] Remove support for external testing suite

[Andrea Cervesato]

Hi Cyril,

Thanks for your reply. I sent a pull request to fix this:

https://github.com/linux-test-project/kirk/pull/44

Andrea

On 3/31/25 18:17, Cyril Hrubis wrote:
> Hi!
>> As you may know, Kirk supports the execution of certain kselftests and
>> liburing testing suites. It was initially created as an experiment.
>>
>> We have made attempts to implement these features, but it seems we have
>> not succeeded. They appear to be largely unused by most users and are
>> creating a bottleneck for new Kirk implementations, such as the
>> automatic installation of LTP inside the SUT. This feature would be
>> particularly beneficial for QEMU vanilla images. We had it in the past
>> already and probably it's time to get this feature back. And we should
>> probably implement it for other testing suites due to the current API:
>> something that we really don't want to do (see eBPF kselftests compile
>> madness).
>>
>> Another concern is the maintainability of supporting for additional
>> testing suites. While they are interesting, the implementation is
>> somewhat flawed. Each testing suite has its own
>> input/output/configuration/compilation methods, leading to confusion and
>> over-engineered solutions. While we understand that no solution is
>> perfect, it seems we have reached a point where Kirk = LTP. Therefore,
>> we should reconsider maintaining features that require updates due to
>> changes in external testing suites.
>>
>> For these reasons, I propose that we remove support for the additional
>> testing suites and focus exclusively on LTP for Kirk maintenance.
>>
>> What do you think?
> Sounds reasonable to me. There is quite a bit of work to be done in
> order to implement all the advanced features in kirk and it does not
> help if we get distracted by side quests.
>