[LTP] [PATCH v1 1/1] sched_football: harden kickoff synchronization on high-CPU systems

[Jan Polensky]

The sched_football test has been intermittently failing, most noticeably
on systems with many CPUs and/or under load, due to a startup ordering
hole around kickoff.

At game start the referee can transition into kickoff while not all
defense threads have reliably reached their blocking path yet. This
creates a window where offense threads may run and increment the_ball.

Fix the startup protocol:

- Introduce a dedicated defense_ready_barrier to rendezvous all defense
  threads with the referee before kickoff.
- Increase the pre-kickoff settle time (RT: 100 ms, non-RT: 2.5 s) to
  account for large CPU counts and loaded systems.
- Add a compiler barrier in the defense busy-loop to prevent it from
  being optimized away.
- Add an additional barrier step after the initial start phase so all
  threads are positioned deterministically before the game begins.

Signed-off-by: Jan Polensky <japo@linux.ibm.com>
---
 .../func/sched_football/sched_football.c      | 23 ++++++++++++++-----
 1 file changed, 17 insertions(+), 6 deletions(-)

diff --git a/testcases/realtime/func/sched_football/sched_football.c b/testcases/realtime/func/sched_football/sched_football.c
index 2cb85322d782..08cdc2fd8b4e 100644
--- a/testcases/realtime/func/sched_football/sched_football.c
+++ b/testcases/realtime/func/sched_football/sched_football.c
@@ -50,6 +50,7 @@ static tst_atomic_t game_over;
 static char *str_game_length;
 static char *str_players_per_team;
 static pthread_barrier_t start_barrier;
+static pthread_barrier_t defense_ready_barrier;
 
 /* These are fans running across the field. They're trying to interrupt/distract everyone */
 void *thread_fan(void *arg LTP_ATTRIBUTE_UNUSED)
@@ -81,11 +82,13 @@ void *thread_defense(void *arg LTP_ATTRIBUTE_UNUSED)
 {
 	prctl(PR_SET_NAME, "defense", 0, 0, 0);
 	pthread_barrier_wait(&start_barrier);
+	pthread_barrier_wait(&defense_ready_barrier);
 	while (!tst_atomic_load(&kickoff_flag))
 		;
 
-	/*keep the ball from being moved */
+	/* Keep the ball from being moved using a compiler barrier */
 	while (!tst_atomic_load(&game_over)) {
+		__asm__ __volatile__("" ::: "memory");
 	}
 
 	return NULL;
@@ -124,14 +127,18 @@ void referee(int game_length)
 	/* Start the game! */
 	atrace_marker_write("sched_football", "Game_started!");
 	pthread_barrier_wait(&start_barrier);
-	usleep(200000);
+
+	/* Wait for defense to be ready before starting the game */
+	pthread_barrier_wait(&defense_ready_barrier);
+
+	/* Give defense threads time to establish */
+	if (tst_check_preempt_rt())
+		usleep(100000);
+	else
+		usleep(2500000);
 
 	tst_atomic_store(0, &the_ball);
 	tst_atomic_store(1, &kickoff_flag);
-	if (tst_check_preempt_rt())
-		usleep(20000);
-	else
-		usleep(2000000);
 
 	/* Watch the game */
 	while ((now.tv_sec - start.tv_sec) < game_length) {
@@ -170,6 +177,9 @@ static void do_test(void)
 	sched_setscheduler(0, SCHED_FIFO, &param);
 	tst_atomic_store(0, &kickoff_flag);
 
+	/* Defense ready barrier: defense threads + referee */
+	pthread_barrier_init(&defense_ready_barrier, NULL, players_per_team + 1);
+
 	/*
 	 * Start the offense
 	 * They are lower priority than defense, so they must be started first.
@@ -197,6 +207,7 @@ static void do_test(void)
 	referee(game_length);
 
 	pthread_barrier_destroy(&start_barrier);
+	pthread_barrier_destroy(&defense_ready_barrier);
 }
 
 static void do_setup(void)
-- 
2.53.0