[LTP] Issue with the reporting of failed tests

[Mason]

On 25/04/2016 10:04, Mason wrote:
> On 22/04/2016 15:29, Mason wrote:
>> On 21/04/2016 19:02, Mason wrote:
>>> On 21/04/2016 17:42, Mason wrote:
>>>> On 21/04/2016 17:30, Cyril Hrubis wrote:
>>>>> Hi!
>>>>>> Calling fflush does appear to solve the problem:
>>>>>>
>>>>>> # grep -c FAIL result-log.5160
>>>>>> 38
>>>>>>
>>>>>> # wc -l result-failed.5160
>>>>>> 38 result-failed.5160
>>>>>>
>>>>>> Does this mean no one ever reads FAILCMDFILE?!
>>>>>
>>>>> Just to have complete image here. I've discussed this a bit more at #ltp
>>>>> irc channel and the reason why we do not see this bug often is that the
>>>>> exec() that is done by the child after pan forks must fail (for example
>>>>> since bash is missing) then the code proceeds to exit(errno) line which
>>>>> flushes the buffers on open files...
>>>>
>>>> Note: in my latest LTP run, I disabled memory overcommit, and pan
>>>> failed to launch several tests:
>>>>
>>>> pan(5163): fork failed (tag proc01).  errno:12  Cannot allocate memory
>>>> pan(5163): fork failed (tag fs_racer).  errno:12  Cannot allocate memory
>>>> pan(5163): fork failed (tag quota_remount_test01).  errno:12  Cannot allocate memory
>>>> pan(5163): fork failed (tag isofs).  errno:12  Cannot allocate memory
>>>> pan(5163): fork failed (tag fs_perms01).  errno:12  Cannot allocate memory
>>>> [747 similar lines]
>>>>
>>>> /tmp is a ramfs. Maybe something fills it up...
>>>
>>> With overcommit enabled, all tests complete, but some problems remain:
>>>
>>> $ grep -c FAIL result-log.24583 
>>> 85
>>>
>>> $ wc -l result-failed.24583
>>> 157 result-failed.24583
>>>
>>> $ sort result-failed.24583 | uniq | wc -l
>>> 83
>>>
>>> I'm confused as to what else is causing this...
>>
>> I am now testing yet another LTP run with the following patch applied.
> 
> There still is an inconsistency between result-log and result-failed.
> 
> $ grep -c FAIL result-log.30032
> 119
> 
> $ wc -l result-failed.30032
> 319 result-failed.30032
> 
> $ sort result-failed.30032 | uniq | wc -l
> 117
> 
> Either I didn't use the modified ltp-pan binary, or the change has no effect.
> (I will print something to stderr to make sure.)

I do see an improvement now: no more duplicates. However, it seems
2 tests are missing from result-failed: aio01, aio02.

$ grep -c FAIL result-log.1125
27

$ wc -l result-failed.1125
25 result-failed.1125

Regards.