[LTP] [PATCH] syslog: fix sporadic failures due to rate-limitting

[Li Wang]

Hi Jan,

On Wed, Jul 24, 2019 at 5:07 PM Jan Stancek <jstancek@redhat.com> wrote:
>
> Some distros (Fedora30) do not have rsyslog installed by default.
> Also some test environments generate a lot of messages before they
> start syslogs tests.
>
> This creates a problem if rsyslog is using imjournal, because by default
> rate-limitting is enabled. If there is a big backlog of messages
> prior to running syslog tests, some messages from tests may be dropped.

I'm not sure if I understand this situation, from your description, if
the rsyslog is not installed,
then the /etc/rsyslog.conf will not exist, so how does the rsyslog
using imjournal to effect on the test cases? Shouldn't it choose
syslog or syslog-ng for testing?

if [ "$SYSLOG_DAEMON" = "syslog" ]; then
        CONFIG_FILE="/etc/syslog.conf"
elif [ "$SYSLOG_DAEMON" = "syslog-ng" ]; then
        CONFIG_FILE="/etc/syslog-ng/syslog-ng.conf"
elif [ "$SYSLOG_DAEMON" = "rsyslog" ]; then
        ....
fi

And from my test, if I remove(#rpm -e rsyslog) the rsyslog from
system, this test will choose to go
syslog-ng(/etc/syslog-ng/syslog-ng.conf) way to test. unfoutananly
that always failed as:

--------------------------
syslog02    0  TINFO  :  Test if messages of all levels are logged.
syslog02    0  TINFO  :  For each level, a separate configuration file is
syslog02    0  TINFO  :  created and that will be used as syslog.conf file.
syslog02    0  TINFO  :  testing whether messages are logged into log file
syslog02    0  TINFO  :  Doing level: emerg...
syslog02    0  TINFO  :  restarting syslog daemon
Broadcast message from
systemd-journald@intel-chiefriver-02.khw2.lab.eng.bos.redhat.com (Tue
2019-07-30 09:42:24 EDT):
syslogtst[26081]: syslogtst: mail emerg test.
syslog02    1  TFAIL  :  ltpapicmd.c:188: ***** Level emerg failed *****
syslog02    0  TINFO  :  Doing level: alert...
syslog02    0  TINFO  :  restarting syslog daemon
syslog02    2  TFAIL  :  ltpapicmd.c:188: ***** Level alert failed *****
syslog02    0  TINFO  :  Doing level: crit...
syslog02    0  TINFO  :  restarting syslog daemon
syslog02    3  TFAIL  :  ltpapicmd.c:188: ***** Level crit failed *****
syslog02    0  TINFO  :  Doing level: err...
syslog02    0  TINFO  :  restarting syslog daemon
syslog02    4  TFAIL  :  ltpapicmd.c:188: ***** Level err failed *****
syslog02    0  TINFO  :  Doing level: warning...
syslog02    0  TINFO  :  restarting syslog daemon
syslog02    5  TFAIL  :  ltpapicmd.c:188: ***** Level warning failed *****
syslog02    0  TINFO  :  Doing level: notice...
syslog02    0  TINFO  :  restarting syslog daemon
syslog02    6  TFAIL  :  ltpapicmd.c:188: ***** Level notice failed *****
syslog02    0  TINFO  :  Doing level: info...
syslog02    0  TINFO  :  restarting syslog daemon
syslog02    7  TFAIL  :  ltpapicmd.c:188: ***** Level info failed *****
syslog02    0  TINFO  :  Doing level: debug...
syslog02    0  TINFO  :  restarting syslog daemon
syslog02    8  TFAIL  :  ltpapicmd.c:188: ***** Level debug failed *****
syslog02    0  TINFO  :  restarting syslog daemon

> Easy way to reproduce is to stop rsyslog, delete imjournal.state,
> run kmsg01 test couple times and then run some syslog test:
>   Jul 23 15:05:13 ... begin to drop messages due to rate-limiting
>   Jul 23 15:05:21 ... 47006 messages lost due to rate-limiting
> Messages make it to systemd journal, but not to log configured in rsyslog.

I didn't reproduce this problem, if I stop rsyslog(`systemctl stop
rsyslog`) by manual, the test will enable it automatically:

---------------------
syslog02    0  TINFO  :  restarting syslog daemon
...
syslogtst[26848]: syslogtst: mail emerg test.
syslog02    1  TPASS  :  ***** Level emerg passed *****
------------------------

or maybe I missed something?

--
Regards,
Li Wang