[LTP] [PATCH v3] ldt.h: Add workaround for x86_64

[Ricardo B. Marlière]

From: Ricardo B. Marlière <rbm@suse.com>

The commit be0aaca2f742 ("syscalls/modify_ldt: Add lapi/ldt.h") left behind
an important factor of modify_ldt(): the kernel intentionally casts the
return value to unsigned int. This was handled in
testcases/cve/cve-2015-3290.c but was removed. Add it back to the relevant
file.

Reported-by: Martin Doucha <mdoucha@suse.cz>
Reviewed-by: Martin Doucha <mdoucha@suse.cz>
Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
---
Changes in v3:
- EDITME: describe what is new in this series revision.
- EDITME: use bulletpoints and terse descriptions.
- Link to v2: https://lore.kernel.org/r/20250512-fixes-modify_ldt-v2-1-eaef5577e44e@suse.com

Changes in v2:
- Added TBROK for any ret != 0 in modify_ldt call in cve-2015-3290.c
- Link to v1: https://lore.kernel.org/r/20250507-fixes-modify_ldt-v1-1-70a2694cfddc@suse.com
---
 include/lapi/ldt.h | 30 +++++++++++++++++++++++++++++-
 1 file changed, 29 insertions(+), 1 deletion(-)

diff --git a/include/lapi/ldt.h b/include/lapi/ldt.h
index 6b5a2d59cb41bfc24eb5ac26c3d47d49fb8ff78f..2533e50b9c02a974947c6628e9fa20c9c626aa34 100644
--- a/include/lapi/ldt.h
+++ b/include/lapi/ldt.h
@@ -31,7 +31,27 @@ struct user_desc {
 static inline int modify_ldt(int func, const struct user_desc *ptr,
 			     unsigned long bytecount)
 {
-	return tst_syscall(__NR_modify_ldt, func, ptr, bytecount);
+	long rval;
+
+	errno = 0;
+	rval = tst_syscall(__NR_modify_ldt, func, ptr, bytecount);
+
+#ifdef __x86_64__
+	/*
+	 * The kernel intentionally casts modify_ldt() return value
+	 * to unsigned int to prevent sign extension to 64 bits. This may
+	 * result in syscall() returning the value as is instead of setting
+	 * errno and returning -1.
+	 */
+	if (rval > 0 && (int)rval < 0) {
+		tst_res(TINFO,
+			"WARNING: Libc mishandled modify_ldt() return value");
+		errno = -(int)errno;
+		rval = -1;
+	}
+#endif /* __x86_64__ */
+
+	return rval;
 }
 
 static inline int safe_modify_ldt(const char *file, const int lineno, int func,
@@ -40,7 +60,15 @@ static inline int safe_modify_ldt(const char *file, const int lineno, int func,
 {
 	int rval;
 
+	errno = 0;
 	rval = modify_ldt(func, ptr, bytecount);
+#ifdef __x86_64__
+	if (rval == -1 && errno == EINVAL) {
+		tst_brk_(file, lineno, TCONF | TTERRNO,
+			 "modify_ldt(%d, %p, %lu): 16-bit data segments are probably disabled",
+			 func, ptr, bytecount);
+	}
+#endif
 	if (rval == -1) {
 		tst_brk_(file, lineno, TBROK | TERRNO,
 			 "modify_ldt(%d, %p, %lu)", func, ptr, bytecount);

---
base-commit: b070a5692e035ec12c3d3c7a7e9e97c270fd4d7d
change-id: 20250507-fixes-modify_ldt-ebcfdd2a7d30

Best regards,
-- 
Ricardo B. Marlière <rbm@suse.com>