[LTP] [PATCH v2] scripts: Dump /proc and /sys

[Richard Palethorpe]

When a test fails during automated testing, the state of the system may be
lost due to storage constraints or hardware availability. For example, if the
system under test is virtualized, then the VM image may no longer be available
due to storage capacity limits. Or if one is testing a bare metal system then the
hardware may no longer be accessible for whatever reason.

The serial console output may be available, but lacking some information. So
in order to retain more information (about the kernel at least), we can
systematically copy the contents of /proc and /sys.

Not all sysfs or proc files are meant to be read or block indefinitely or are
infinitely large etc. So the script only tries to copy ~256K from each file
and times out after a second. Files which appear to be OK to read from are
saved to a white list which can be used during future invocations to save some
time.

Signed-off-by: Richard Palethorpe <rpalethorpe@suse.com>
---

V2:
* Fixed numerous style problems
* Instead of spawning X subshells per iteration I now only spawn $(nproc)
  subshells and pass files to them to through named pipes. This is much faster.
* Add some extra options

 scripts/proc_sys_dump.sh | 145 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 145 insertions(+)
 create mode 100755 scripts/proc_sys_dump.sh

diff --git a/scripts/proc_sys_dump.sh b/scripts/proc_sys_dump.sh
new file mode 100755
index 000000000..a6be95c2e
--- /dev/null
+++ b/scripts/proc_sys_dump.sh
@@ -0,0 +1,145 @@
+#!/bin/sh
+
+my_name=${0##*/}
+dump_dir=/tmp/dump
+dump_tar=
+white_list=/tmp/dump/white.list
+use_white_list=
+timeout=1
+proc_exclude="/proc/\(self\|thread-self\|[0-9]+\)/.*"
+page_sz=$(getconf PAGESIZE)
+nproc=$(nproc)
+
+echo "Dump /proc and /sys script, written by rpalethorpe@suse.com"
+
+while getopts d:c:w:u:t:b:h arg; do
+    case $arg in
+       d) dump_dir="$OPTARG" ;;
+       c) dump_tar="$OPTARG" ;;
+       w) white_list="$OPTARG" ;;
+       u) use_white_list="$OPTARG" ;;
+       t) timeout="$OPTARG" ;;
+       b) batch_size="$OPTARG" ;;
+       h) cat <<-EOF >&1
+
+This script partially dumps the contents of /proc and /sys to a folder. It can
+attempt to dump all the contents and then write out which files were
+successful to a white list. Or use a previously generated white list.
+
+usage:
+$my_name	[ -d dump_dir ]
+ 	 		[ -w white_list ] [ -u use_white_list ]
+ 			[ -t timeout ]
+ 			[ -b batch_size ]
+$my_name	-h
+
+-d dump_dir	  The output directory, note that this will be recreated.
+   		  Set to $dump_dir
+
+-c dump_tar	  Compress the output directory contents into a tar file
+		  specified by dump_tar.
+
+-w white_list	  The white list to output after dumping the files.
+   		  Set to $white_list
+
+-u use_white_list The list of files to dump, if this is unset then 'find' is
+   		  used to create a list.
+
+-t timeout	  How long, in seconds, we should wait for operations to finish.
+   		  Set to $timeout.
+
+-h		  Print this help message and exit.
+
+Files are added to the white list if they are small, but not empty and can be
+copied quickly. The idea is that when running the script with -w white_list
+the majority of useful information can be quickly saved during automated
+testing.
+
+It is best to run this script as root otherwise sensitive information will be
+filtered out by the kernel e.g. memory layout. Which happens to be the most
+useful information in most situations.
+
+EOF
+	  exit 0 ;;
+   esac
+done
+
+files=
+if [ ! $use_white_list ]; then
+    echo "Finding files in /proc and /sys"
+    proc_files=$(find -L /proc -maxdepth 4 -perm /u=r,g=r -not -regex "$proc_exclude" -not -type d 2> /dev/null)
+    sys_files=$(find -L /sys -maxdepth 4 -perm /u=r,g=r -not -type d 2> /dev/null)
+    files="$proc_files $sys_files"
+else
+    echo "Using file list $use_white_list"
+    files=$(cat $use_white_list)
+fi
+
+dump_dir=${dump_dir%/}
+echo "Dumping to $dump_dir"
+rm -rf $dump_dir
+mkdir $dump_dir
+
+# This is run inside background subshells with stdin redirected to a named pipe.
+spawn_dd () {
+    local f=
+
+    read -r f
+    until [ "$f" = "EXIT" ]; do
+	mkdir -p $dump_dir$(dirname $f)
+	timeout $timeout dd if=$f of=$dump_dir$f bs=$page_sz count=63 status=none
+	if [ $? -eq 124 ]; then
+	    echo "dd $f took too long"
+	    rm $dump_dir$f
+	fi
+	read -r f
+    done
+}
+
+for i in $(seq $nproc); do
+    mkfifo "$dump_dir/fifo$i"
+    ( spawn_dd ) < "$dump_dir/fifo$i" &
+done
+
+count=$nproc
+for f in $files; do
+    if [ $count -gt 0 ]; then
+	echo "$f" > "$dump_dir/fifo$count"
+	count=$((count - 1))
+    else
+	count=$nproc
+    fi
+done
+
+for i in $(seq $nproc); do
+    echo "EXIT" > "$dump_dir/fifo$i"
+done
+
+wait
+
+for i in $(seq $nproc); do
+    rm "$dump_dir/fifo$i"
+done
+
+if [ $white_list ]; then
+    echo "Finished copying files, now generating $white_list"
+    for f in $(find $dump_dir -not -type d); do
+	size=$(stat -c '%s' $f)
+
+	if [ ${size:=0} -gt 0 -a $size -lt 256000 ]; then
+	    echo "${f#$dump_dir}" >> $white_list
+	else
+	    echo "$f has invalid file size: ${size:=0}"
+	fi
+    done
+else
+    echo "Finished copying files."
+fi
+
+if [ $dump_tar ]; then
+    echo "Compressing dumped files into $dump_tar"
+    oldpwd=${pwd}
+    cd $dump_dir
+    tar -cJf $dump_tar *
+    cd $oldpwd
+fi
-- 
2.15.0