[LTP] [PATCH v1] cpuset_memory_spread: make result_check() robust to page-cache noise

[Changwei Zou]

The cpuset_memory_spread test uses empirical values that were
established in 2009. It consists of cpuset_mem_hog.c and
cpuset_memory_spread_testset.sh. cpuset_mem_hog.c reads a 100 MB file
(./DATAFILE) into memory, while cpuset_memory_spread_testset.sh compares
the global per-node FilePages counters in
/sys/devices/system/node/nodeX/meminfo.

On the expected NUMA node where the test case is running, the observed
change in FilePages should be larger than the preset upper limit
(i.e., 10000 KB). On the unexpected NUMA node where it is quiet, the
delta in FilePages should be smaller than the preset lower limit
(i.e., 2048 KB when PAGE_SIZE is 4 KB).

However, the global per-node FilePages counters also account for
unrelated page-cache activity on the system, which makes the test flaky
on large or heavily loaded NUMA machines. Some failures are listed
below.

  cpuset_memory_spread 1 TFAIL: hog the memory on the unexpected
  node(FilePages_For_Nodes(KB): _0: 106384 _1: 2248, Expect Nodes: 0).

  cpuset_memory_spread 5 TFAIL: hog the memory on the unexpected
  node(FilePages_For_Nodes(KB): _0: 7592 _1: 108328, Expect Nodes: 1).

  cpuset_memory_spread 7 TFAIL: hog the memory on the unexpected
  node(FilePages_For_Nodes(KB): _0: 2080 _1: 102788, Expect Nodes: 1).

For example, 108328 KB is much larger than the empirical upper limit of
10000 KB in cpuset_memory_spread_testset.sh, while 7592 KB is larger
than the lower limit of 2048 KB. Interestingly, 108328 KB is itself
larger than the actual file size (100 MB = 102,400 KB). This
demonstrates that the global per-node FilePages counters account for
unrelated page-cache activity elsewhere on the system.

To make the test more robust, add the following rule in result_check():

  Find the smallest get_memsinfo_val min_upper in $nodelist, and the
  largest get_memsinfo_val max_lower in $othernodelist. When their ratio
  min_upper / max_lower (in floating format) is not less than
  (upperlimit / lowerlimit), return 0 (the test passes). Otherwise
  return 1 (the test fails).

On non-NUMA machines, this test case always succeeds because it is
skipped.

Signed-off-by: Changwei Zou <changwei.zou@canonical.com>
---
 .../cpuset_memory_spread_testset.sh           | 29 +++++++++++++++++--
 1 file changed, 26 insertions(+), 3 deletions(-)

diff --git a/testcases/kernel/controllers/cpuset/cpuset_memory_spread_test/cpuset_memory_spread_testset.sh b/testcases/kernel/controllers/cpuset/cpuset_memory_spread_test/cpuset_memory_spread_testset.sh
index 4c49bb8fd..70d04ac8d 100755
--- a/testcases/kernel/controllers/cpuset/cpuset_memory_spread_test/cpuset_memory_spread_testset.sh
+++ b/testcases/kernel/controllers/cpuset/cpuset_memory_spread_test/cpuset_memory_spread_testset.sh
@@ -170,12 +170,19 @@ result_check()
 {
 	local nodelist="`echo $1 | sed -e 's/,/ /g'`"
 	local i=
+	local val=
 
+	# find the smallest value in $nodelist while checking upperlimit
+	local min_upper=0
 	for i in $nodelist
 	do
-		if [ $(get_memsinfo_val $i) -le $upperlimit ]; then
+		val=$(get_memsinfo_val $i)
+		if [ $val -le $upperlimit ]; then
 			return 1
 		fi
+		if [ $min_upper -eq 0 ] || [ $val -lt $min_upper ]; then
+			min_upper=$val
+		fi
 	done
 
 	local allnodelist="`echo $mems_all | sed -e 's/,/ /g'`"
@@ -188,12 +195,28 @@ result_check()
 		othernodelist=`echo "$othernodelist" | sed -e "s/ $i / /g"`
 	done
 
+	# find the largest value in $othernodelist
+	local max_lower=0
 	for i in $othernodelist
 	do
-		if [ $(get_memsinfo_val $i) -gt $lowerlimit ]; then
-			return 1
+		val=$(get_memsinfo_val $i)
+		if [ $val -gt $max_lower ]; then
+			max_lower=$val
 		fi
 	done
+
+	# when min_upper / max_lower (floating) is not less than
+	# upperlimit / lowerlimit, the test passes.
+	if awk -v mu="$min_upper" -v xl="$max_lower" \
+	       -v ul="$upperlimit" -v ll="$lowerlimit" \
+	       'BEGIN {
+	                if (xl == 0) { exit !(mu > 0) }
+	                exit !((mu / xl) >= (ul / ll))
+	        }'; then
+		return 0
+	fi
+
+	return 1
 }
 
 # general_memory_spread_test <cpusetpath> <is_spread> <cpu_list> <node_list> \
-- 
2.43.0