Reorganize mce-test

This new design reorganize entire structure of MCE-test. After
applying new structure, MCE-test owns new unified output format
and interface.

In principle, during this change, no functional change. Only some
minor fixes and updates are added, BTW, a few new test cases are
merged such as PFA. Other test cases will be applied after this
change is fused into current MCE-test.

Signed-off-by: Chen Gong <gong.chen@linux.intel.com>
Signed-off-by: Andi Kleen <ak@linux.intel.com>

203 files changed, 4261 insertions, 1661 deletions

diff --git a/Makefile b/Makefile
index f0cdc21..366f71f 100644
--- a/Makefile
+++ b/Makefile
@@ -1,41 +1,18 @@
-.PHONY:	test clean distclean reset test-simple test-kdump
-
 all:
 	$(MAKE) -C tools
-	$(MAKE) -C tsrc
-	$(MAKE) -C stress
+	$(MAKE) -C cases
 
 clean:
 	$(MAKE) -C tools clean
-	$(MAKE) -C tsrc clean
-	$(MAKE) -C stress clean
+	$(MAKE) -C cases clean
 	$(MAKE) reset
 
-distclean:
-	$(MAKE) -C tools distclean
-	$(MAKE) -C tsrc distclean
-	$(MAKE) -C stress distclean
-	$(MAKE) reset
-	rm -rf bin/*
+install:
+	$(MAKE) -C tools install
+	$(MAKE) -C cases install
 
 reset:
+	rm -rf bin/*
 	rm -rf work/*
 	rm -rf results/*
 
-test: test-simple test-stress
-
-test-simple:
-	$(MAKE) reset
-	./drivers/simple/driver.sh simple.conf
-	$(MAKE) -C tsrc test
-
-# requires LTP & page-types to be installed
-test-stress:
-	$(MAKE) -C stress test
-
-# requires special packages to be installed
-test-kdump:
-	$(MAKE) reset
-	./drivers/simple/driver.sh simple.conf
-	./drivers/kdump/driver.sh kdump.conf
-	$(MAKE) -C tsrc test
diff --git a/README b/README
index 4d1b1fe..524716b 100644
--- a/README
+++ b/README
@@ -2,14 +2,16 @@ MCE test suite
 ---------------
 
 The MCE test suite is a collection of tools and test scripts for
-testing the Linux kernel machine check error recovery 
-features. This is the code that deals with recovery from memory 
-bit errors and other hardware problems.
+testing the Linux RAS related features, including CPU/Memory error
+containment and recovery, ACPI/APEI support etc.
 
 For some more details on machine checks see http://www.mcelog.org
 
-The goal is to cover most Linux kernel MCE processing code paths 
-and features with automation tests.
+The goal is as follows:
+1. cover most Linux kernel MCE processing code paths and features
+2. cover functional test cases for all RAS related features
+3. provide stress test cases for some RAS features, especially for
+   memory part
 
 In the Package
 --------------
@@ -25,13 +27,9 @@ COPYING
 Makefile
 	Top level make file for MCE test suite
 
-drivers/*
-	Contains test drivers, which drive test procedure and do some
-	common works for test drivers. There is one directory for each
-	test driver, the user interface of a driver is the driver.sh
-	in corresponding directory, such as:
-	    drivers/kdump/driver.sh
-	is user interface of kdump test driver.
+bin/
+	Some tools used by test drivers or test cases will be
+	installed into this directory.
 
 cases/*
 	Contains all test cases, which may be organized in
@@ -43,14 +41,9 @@ cases/*
 	   -- cases/apei-inj/ucr/cases.sh  
 		is for test cases triggered by apei-inject.
 
-config/*
-	Contains test configuration files, which specifies the
-	parameters for test driver, which test cases are used in test,
-	the parameters for test cases, etc.
-
-tsrc/*
-	Some standalone test programs for various parts of the machine
-	check code.
+doc/*
+	Documentation for MCE test suites include howto and
+	descriptions of every test case.
 
 lib/*
 	Contains some shell scripts, in which some common shell
@@ -60,34 +53,19 @@ lib/*
 tools/*
 	Some tools used by MCE test suites.
 
-doc/*
-	Documentation for MCE test suites include howto and
-	descriptions of every test case.
-
-results/
-	When test is done, the test result will be placed in this
-	directory, test results for a specific test driver will be
-	placed in corresponding directory, such as test results of
-	kdump test driver will be placed in "results/kdump". General
-	test result is in results/$driver/result; additional results
-	of various cases may be in corresponding directory, for
-	example, files in
-	    results/kdump/soft-inj/panic/fatal/
-	is for additional result for test case soft-inj/panic/fatal.
-
 work/
-	During test, some temporary file will be put in work
-	directory, temporary files for a specific test driver will be
-	placed in corresponding directory, such as temporary files of
-	kdump test driver will be placed in "work/kdump". Test log is
-	in work/$driver/log.
+	During test, some temporary file will be put in work directory.
 
-bin/
-	Some tools used by test drivers or test cases will be
-	installed into this directory.
+groups/
+	Where all scenarios files such as coverage/function/stress are placed
+
+results/
+	When test is done, the test log will be placed in this directory.
+	Some test results such as coverage test result will be
+	placed in sub-directory under results directory.
 
-stress/
-	Linux MCE stress test suite.
+summary
+	When test is done, a summary file in *date* format is placed here.
 
 Test Instruction
 ----------------
@@ -95,10 +73,10 @@ Test Instruction
 Please refer to corresponding section in doc/howto.txt.
 
 Very quick way to test this:
-	be root
-	make sure you have a kernel with CONFIG_X86_MCE_INJECT
+	1. be root
+	2. make sure you have a kernel with CONFIG_X86_MCE_INJECT
 	and CONFIG_HWPOISON_INJECT and soft-offlining support
-	run "make test"
+	3. run "mcemenu"
 
 Futher Information
 ------------------
diff --git a/cases/Makefile b/cases/Makefile
new file mode 100644
index 0000000..883ae55
--- /dev/null
+++ b/cases/Makefile
@@ -0,0 +1,12 @@
+all:
+	$(MAKE) -C function
+	$(MAKE) -C stress
+
+install:
+	$(MAKE) -C function install
+	$(MAKE) -C stress install
+
+clean:
+	$(MAKE) -C function clean
+	$(MAKE) -C stress clean
+
diff --git a/cases/apei-inj/README b/cases/apei-inj/README
deleted file mode 100644
index 3ae91b4..0000000
--- a/cases/apei-inj/README
+++ /dev/null
@@ -1,33 +0,0 @@
-MCE APEI INJ testing cases 
-=========================
-
-Nov 6th, 2009
-
-Jiajia Zheng
-
-Start
-------------
-This test suite is based on apei-inj APEI haredware injection tool. 
-It's structure and usage are the same with soft-inj test suite.
-To be root and invoke simple test driver on test configuration file as follow
-	drivers/simple/driver.sh config/simple_apei_ucr.conf
-
-Test Dependencies
------------------
-MCE APEI INJ test suite has following dependencies on kernel and other tools:
-
-* Make sure following configuration options are enabled in linux kernel:
-	CONFIG_FTRACE=y
-	CONFIG_FUNCTION_TRACER=y
-
-* page-types:
-  A tool to query page types, which is accompanied with Linux kernel
-  source (2.6.32 or newer, $KERNEL_SRC/Documentation/vm/page-types.c).
-  For detail, please see 3.4 in doc/howto.txt.
-
-* simple-process:
-  APEI hardware injection will inject an error to a page used by this simple-process.  Intall simple-process.
-	cd mce-test/tools/simple_process
-	make
-
-
diff --git a/cases/apei-inj/ucr/cases.sh b/cases/apei-inj/ucr/cases.sh
deleted file mode 100755
index 2364bdc..0000000
--- a/cases/apei-inj/ucr/cases.sh
+++ /dev/null
@@ -1,63 +0,0 @@
-#!/bin/bash
-#
-# APEI injection based test cases : memory patrol scrub cases: test cases
-# are triggered via the apei-inject, and they will not trigger kernel panic.
-#
-# Copyright (C) 2008, Intel Corp.
-#   Author: Huang Ying <ying.huang@intel.com>
-#           Zheng Jiajia <jiajia.zheng@intel.com>
-#
-# This file is released under the GPLv2.
-#
-
-. $ROOT/lib/functions.sh
-. $ROOT/lib/dirs.sh
-. $ROOT/lib/mce.sh
-. $ROOT/lib/apei-inject.sh
-
-enumerate()
-{
-    apei_inject_enumerate
-}
-
-trigger()
-{
-    case "$bcase" in
-        mem_uncorrected)
-            start_tracing "do_machine_check"
-            ;;
-        *)
-            echo '!!! Unknown case: $this_case !!!'
-    esac
-    apei_inject_trigger
-    stop_tracing
-}
-
-get_result()
-{
-    apei_inject_get_klog
-    get_gcov drivers/acpi/apei/einj.c
-
-    case "$bcase" in
-	mem_uncorrected)
-	    get_mcelog_from_dev $mcelog_result
-	    ;;
-	*)
-	    echo '!!! Unknown case: $this_case !!!'
-    esac
-}
-
-verify()
-{
-    case "$bcase" in
-	mem_uncorrected)
-	    apei_inject_verify_mcelog
-	    verify_klog $klog
-            apei_inject_verify_trace "do_machine_check"
-	    ;;
-	*)
-	    echo "!!! Unknown case: $this_case !!!"
-    esac
-}
-
-apei_inject_main "$@"
diff --git a/cases/apei-inj/ucr/data/mem_uncorrected b/cases/apei-inj/ucr/data/mem_uncorrected
deleted file mode 100644
index a6601e9..0000000
--- a/cases/apei-inj/ucr/data/mem_uncorrected
+++ /dev/null
@@ -1 +0,0 @@
-TYPE 0x10
diff --git a/cases/apei-inj/ucr/refer/mem_uncorrected b/cases/apei-inj/ucr/refer/mem_uncorrected
deleted file mode 100644
index 43889f1..0000000
--- a/cases/apei-inj/ucr/refer/mem_uncorrected
+++ /dev/null
@@ -1,10 +0,0 @@
-CPU 0 BANK 2
-STATUS UNCORRECTED SRAO 0xcf
-MCGSTATUS RIPV MCIP
-MISC 0x8c
-ADDR 0x1234
-RIP 0x73:0x2eadbabe
-
-
-
-
diff --git a/cases/coverage/soft-inj/config/kdump.conf b/cases/coverage/soft-inj/config/kdump.conf
new file mode 100644
index 0000000..b04cb04
--- /dev/null
+++ b/cases/coverage/soft-inj/config/kdump.conf
@@ -0,0 +1,6 @@
+
+CASES="panic/cases.sh panic_npcc/cases.sh"
+COREDIR="/var/crash"
+GCOV=1
+KSRC_DIR=/lib/modules/$(uname -r)/build
+VMLINUX=$KSRC_DIR/vmlinux
diff --git a/cases/coverage/soft-inj/config/kdump_noser.conf b/cases/coverage/soft-inj/config/kdump_noser.conf
new file mode 100644
index 0000000..9b304d8
--- /dev/null
+++ b/cases/coverage/soft-inj/config/kdump_noser.conf
@@ -0,0 +1,6 @@
+
+CASES="panic/cases.sh panic_npcc/cases.sh panic_noser/cases.sh"
+COREDIR="/var/crash"
+GCOV=1
+KSRC_DIR=/lib/modules/$(uname -r)/build
+VMLINUX=$KSRC_DIR/vmlinux
diff --git a/config/kdump_panic.conf b/cases/coverage/soft-inj/config/kdump_panic.conf
index b2acc63..4d02c30 100644
--- a/config/kdump_panic.conf
+++ b/cases/coverage/soft-inj/config/kdump_panic.conf
@@ -1,5 +1,5 @@
 
-CASES="soft-inj/panic/cases.sh"
+CASES="panic/cases.sh"
 COREDIR="/var/crash"
 GCOV=1
 KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/config/kdump_panic_noser.conf b/cases/coverage/soft-inj/config/kdump_panic_noser.conf
index b4562d3..05931da 100644
--- a/config/kdump_panic_noser.conf
+++ b/cases/coverage/soft-inj/config/kdump_panic_noser.conf
@@ -1,5 +1,5 @@
 
-CASES="soft-inj/panic_noser/cases.sh"
+CASES="panic_noser/cases.sh"
 COREDIR="/var/crash"
 GCOV=1
 KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/config/kdump_panic_npcc.conf b/cases/coverage/soft-inj/config/kdump_panic_npcc.conf
index b5c5e8c..c881192 100644
--- a/config/kdump_panic_npcc.conf
+++ b/cases/coverage/soft-inj/config/kdump_panic_npcc.conf
@@ -1,5 +1,5 @@
 
-CASES="soft-inj/panic_npcc/cases.sh"
+CASES="panic_npcc/cases.sh"
 COREDIR="/var/crash"
 GCOV=1
 KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/config/kdump_panic_ucr.conf b/cases/coverage/soft-inj/config/kdump_panic_ucr.conf
index c7508b2..bcc819a 100644
--- a/config/kdump_panic_ucr.conf
+++ b/cases/coverage/soft-inj/config/kdump_panic_ucr.conf
@@ -1,5 +1,5 @@
 
-CASES="soft-inj/panic_ucr/cases.sh"
+CASES="panic_ucr/cases.sh"
 COREDIR="/var/crash"
 GCOV=1
 KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/config/kdump.conf b/cases/coverage/soft-inj/config/kdump_ser.conf
index e87ddc5..3324a79 100644
--- a/config/kdump.conf
+++ b/cases/coverage/soft-inj/config/kdump_ser.conf
@@ -1,5 +1,5 @@
 
-CASES="soft-inj/panic/cases.sh soft-inj/panic_npcc/cases.sh"
+CASES="panic/cases.sh panic_npcc/cases.sh panic_ucr/cases.sh"
 COREDIR="/var/crash"
 GCOV=1
 KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/cases/coverage/soft-inj/config/simple.conf b/cases/coverage/soft-inj/config/simple.conf
new file mode 100644
index 0000000..2ca1b8a
--- /dev/null
+++ b/cases/coverage/soft-inj/config/simple.conf
@@ -0,0 +1,3 @@
+CASES=" panic_noser/cases.sh panic_npcc/cases.sh panic/cases.sh recoverable_ucr/cases.sh poll_noser/cases.sh poll_ucr/cases.sh non-panic/cases.sh"
+GCOV=1
+KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/config/simple_non_panic.conf b/cases/coverage/soft-inj/config/simple_non_panic.conf
index 7d1a5d3..4bfa31f 100644
--- a/config/simple_non_panic.conf
+++ b/cases/coverage/soft-inj/config/simple_non_panic.conf
@@ -1,4 +1,4 @@
 
-CASES="soft-inj/non-panic/cases.sh"
+CASES="non-panic/cases.sh"
 GCOV=1
 KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/config/simple_panic_npcc.conf b/cases/coverage/soft-inj/config/simple_nopanic_noser.conf
index 04a3f23..68d2fa4 100644
--- a/config/simple_panic_npcc.conf
+++ b/cases/coverage/soft-inj/config/simple_nopanic_noser.conf
@@ -1,4 +1,4 @@
 
-CASES="soft-inj/panic_npcc/cases.sh"
+CASES="non-panic/cases.sh poll_noser/cases.sh"
 GCOV=1
 KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/cases/coverage/soft-inj/config/simple_nopanic_ser.conf b/cases/coverage/soft-inj/config/simple_nopanic_ser.conf
new file mode 100644
index 0000000..abd46bf
--- /dev/null
+++ b/cases/coverage/soft-inj/config/simple_nopanic_ser.conf
@@ -0,0 +1,4 @@
+
+CASES="non-panic/cases.sh poll_ucr/cases.sh recoverable_ucr/cases.sh"
+GCOV=1
+KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/cases/coverage/soft-inj/config/simple_noser.conf b/cases/coverage/soft-inj/config/simple_noser.conf
new file mode 100644
index 0000000..4746820
--- /dev/null
+++ b/cases/coverage/soft-inj/config/simple_noser.conf
@@ -0,0 +1,4 @@
+
+CASES="non-panic/cases.sh panic/cases.sh panic_npcc/cases.sh panic_noser/cases.sh poll_noser/cases.sh"
+GCOV=1
+KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/config/simple_panic.conf b/cases/coverage/soft-inj/config/simple_panic.conf
index 35e96c8..c024f1f 100644
--- a/config/simple_panic.conf
+++ b/cases/coverage/soft-inj/config/simple_panic.conf
@@ -1,4 +1,4 @@
 
-CASES="soft-inj/panic/cases.sh"
+CASES="panic/cases.sh"
 GCOV=1
 KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/cases/coverage/soft-inj/config/simple_panic_noser.conf b/cases/coverage/soft-inj/config/simple_panic_noser.conf
new file mode 100644
index 0000000..fe4bfdf
--- /dev/null
+++ b/cases/coverage/soft-inj/config/simple_panic_noser.conf
@@ -0,0 +1,4 @@
+
+CASES="panic_noser/cases.sh"
+GCOV=1
+KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/cases/coverage/soft-inj/config/simple_panic_npcc.conf b/cases/coverage/soft-inj/config/simple_panic_npcc.conf
new file mode 100644
index 0000000..715a18d
--- /dev/null
+++ b/cases/coverage/soft-inj/config/simple_panic_npcc.conf
@@ -0,0 +1,4 @@
+
+CASES="panic_npcc/cases.sh"
+GCOV=1
+KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/config/simple_panic_ucr.conf b/cases/coverage/soft-inj/config/simple_panic_ucr.conf
index 84d0d00..0a0ab8c 100644
--- a/config/simple_panic_ucr.conf
+++ b/cases/coverage/soft-inj/config/simple_panic_ucr.conf
@@ -1,4 +1,4 @@
 
-CASES="soft-inj/panic_ucr/cases.sh"
+CASES="panic_ucr/cases.sh"
 GCOV=1
 KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/cases/coverage/soft-inj/config/simple_poll_noser.conf b/cases/coverage/soft-inj/config/simple_poll_noser.conf
new file mode 100644
index 0000000..c74f84a
--- /dev/null
+++ b/cases/coverage/soft-inj/config/simple_poll_noser.conf
@@ -0,0 +1,4 @@
+
+CASES="poll_noser/cases.sh"
+GCOV=1
+KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/config/simple_poll_ucr.conf b/cases/coverage/soft-inj/config/simple_poll_ucr.conf
index f21038c..4214c7b 100644
--- a/config/simple_poll_ucr.conf
+++ b/cases/coverage/soft-inj/config/simple_poll_ucr.conf
@@ -1,4 +1,4 @@
 
-CASES="soft-inj/poll_ucr/cases.sh"
+CASES="poll_ucr/cases.sh"
 GCOV=1
 KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/cases/coverage/soft-inj/config/simple_recoverable_ucr.conf b/cases/coverage/soft-inj/config/simple_recoverable_ucr.conf
new file mode 100644
index 0000000..3864a24
--- /dev/null
+++ b/cases/coverage/soft-inj/config/simple_recoverable_ucr.conf
@@ -0,0 +1,4 @@
+
+CASES="recoverable_ucr/cases.sh"
+GCOV=1
+KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/cases/coverage/soft-inj/config/simple_ser.conf b/cases/coverage/soft-inj/config/simple_ser.conf
new file mode 100644
index 0000000..6d097be
--- /dev/null
+++ b/cases/coverage/soft-inj/config/simple_ser.conf
@@ -0,0 +1,4 @@
+
+CASES="non-panic/cases.sh panic/cases.sh panic_npcc/cases.sh poll_ucr/cases.sh panic_ucr/cases.sh recoverable_ucr/cases.sh"
+GCOV=1
+KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/drivers/kdump/driver.sh b/cases/coverage/soft-inj/driver_kdump.sh
index 6016582..f548ded 100755..100644
--- a/drivers/kdump/driver.sh
+++ b/cases/coverage/soft-inj/driver_kdump.sh
@@ -13,7 +13,7 @@
 #
 
 sd=$(dirname "$0")
-export ROOT=`(cd $sd/../..; pwd)`
+export ROOT=`(cd $sd/../../..; pwd)`
 
 export driver=kdump
 
diff --git a/drivers/simple/driver.sh b/cases/coverage/soft-inj/driver_simple.sh
index ebc9d2a..2f4f3ea 100755..100644
--- a/drivers/simple/driver.sh
+++ b/cases/coverage/soft-inj/driver_simple.sh
@@ -10,7 +10,7 @@
 #
 
 sd=$(dirname "$0")
-export ROOT=`(cd $sd/../..; pwd)`
+export ROOT=`(cd $sd/../../..; pwd)`
 
 export driver=simple
 
@@ -74,7 +74,7 @@ test_all()
     fi
 
     for case_sh in $CASES; do
-	for this_case in $($CDIR/$case_sh enumerate); do
+	for this_case in $($case_sh enumerate); do
 	    set_fake_panic 1
 
 	    export this_case
@@ -106,9 +106,8 @@ if [ $# -lt 1 ]; then
     die "Usage: $0 <config>"
 fi
 
-conf=$(basename "$1")
-
-. $CONF_DIR/$conf
+#$1 is config name
+. $1
 
 driver_prepare
 set_panic_on_oops 0
diff --git a/cases/soft-inj/non-panic/cases.sh b/cases/coverage/soft-inj/non-panic/cases.sh
index d277e11..d277e11 100755..100644
--- a/cases/soft-inj/non-panic/cases.sh
+++ b/cases/coverage/soft-inj/non-panic/cases.sh
diff --git a/cases/soft-inj/non-panic/data/corrected b/cases/coverage/soft-inj/non-panic/data/corrected
index 22af584..131212f 100644
--- a/cases/soft-inj/non-panic/data/corrected
+++ b/cases/coverage/soft-inj/non-panic/data/corrected
@@ -1,11 +1,11 @@
-# 
+#
 # log corrected machine checks
 CPU 0 BANK 1
 STATUS CORRECTED
 ADDR 0xabcd
 HOLD
 CPU 1 BANK 0
-#  
+#
 CPU 1 BANK 2
 STATUS CORRECTED
 MISC 0xabcd
diff --git a/cases/soft-inj/non-panic/data/corrected_hold b/cases/coverage/soft-inj/non-panic/data/corrected_hold
index 1c9da73..c36da5b 100644
--- a/cases/soft-inj/non-panic/data/corrected_hold
+++ b/cases/coverage/soft-inj/non-panic/data/corrected_hold
@@ -1,10 +1,10 @@
-# 
+#
 # log two corrected machine checks
 CPU 0 BANK 1
 STATUS CORRECTED
 ADDR 0xabcd
 HOLD
-#  
+#
 CPU 1 BANK 2
 STATUS CORRECTED
 ADDR 0x1234
diff --git a/cases/soft-inj/non-panic/data/corrected_no_en b/cases/coverage/soft-inj/non-panic/data/corrected_no_en
index 9ea50c2..9ea50c2 100644
--- a/cases/soft-inj/non-panic/data/corrected_no_en
+++ b/cases/coverage/soft-inj/non-panic/data/corrected_no_en
diff --git a/cases/soft-inj/non-panic/data/corrected_over b/cases/coverage/soft-inj/non-panic/data/corrected_over
index 21214cf..21214cf 100644
--- a/cases/soft-inj/non-panic/data/corrected_over
+++ b/cases/coverage/soft-inj/non-panic/data/corrected_over
diff --git a/cases/soft-inj/panic/cases.sh b/cases/coverage/soft-inj/panic/cases.sh
index 919f833..919f833 100755..100644
--- a/cases/soft-inj/panic/cases.sh
+++ b/cases/coverage/soft-inj/panic/cases.sh
diff --git a/cases/soft-inj/panic/data/fatal b/cases/coverage/soft-inj/panic/data/fatal
index b24942b..b24942b 100644
--- a/cases/soft-inj/panic/data/fatal
+++ b/cases/coverage/soft-inj/panic/data/fatal
diff --git a/cases/soft-inj/panic/data/fatal_eipv b/cases/coverage/soft-inj/panic/data/fatal_eipv
index 7e6a93a..7e6a93a 100644
--- a/cases/soft-inj/panic/data/fatal_eipv
+++ b/cases/coverage/soft-inj/panic/data/fatal_eipv
diff --git a/cases/soft-inj/panic/data/fatal_irq b/cases/coverage/soft-inj/panic/data/fatal_irq
index 63ef36b..63ef36b 100644
--- a/cases/soft-inj/panic/data/fatal_irq
+++ b/cases/coverage/soft-inj/panic/data/fatal_irq
diff --git a/cases/soft-inj/panic/data/fatal_no_en b/cases/coverage/soft-inj/panic/data/fatal_no_en
index 12da496..12da496 100644
--- a/cases/soft-inj/panic/data/fatal_no_en
+++ b/cases/coverage/soft-inj/panic/data/fatal_no_en
diff --git a/cases/soft-inj/panic/data/fatal_over b/cases/coverage/soft-inj/panic/data/fatal_over
index 6f44419..6f44419 100644
--- a/cases/soft-inj/panic/data/fatal_over
+++ b/cases/coverage/soft-inj/panic/data/fatal_over
diff --git a/cases/soft-inj/panic/data/fatal_ripv b/cases/coverage/soft-inj/panic/data/fatal_ripv
index 1978747..1978747 100644
--- a/cases/soft-inj/panic/data/fatal_ripv
+++ b/cases/coverage/soft-inj/panic/data/fatal_ripv
diff --git a/cases/soft-inj/panic/data/fatal_timeout b/cases/coverage/soft-inj/panic/data/fatal_timeout
index c910141..c910141 100644
--- a/cases/soft-inj/panic/data/fatal_timeout
+++ b/cases/coverage/soft-inj/panic/data/fatal_timeout
diff --git a/cases/soft-inj/panic/data/fatal_timeout_ripv b/cases/coverage/soft-inj/panic/data/fatal_timeout_ripv
index 0e34e00..0e34e00 100644
--- a/cases/soft-inj/panic/data/fatal_timeout_ripv
+++ b/cases/coverage/soft-inj/panic/data/fatal_timeout_ripv
diff --git a/cases/soft-inj/panic/data/fatal_userspace b/cases/coverage/soft-inj/panic/data/fatal_userspace
index 3b8fe7f..3b8fe7f 100644
--- a/cases/soft-inj/panic/data/fatal_userspace
+++ b/cases/coverage/soft-inj/panic/data/fatal_userspace
diff --git a/cases/soft-inj/panic/refer/fatal_no_en b/cases/coverage/soft-inj/panic/refer/fatal_no_en
index 893cc07..893cc07 100644
--- a/cases/soft-inj/panic/refer/fatal_no_en
+++ b/cases/coverage/soft-inj/panic/refer/fatal_no_en
diff --git a/cases/soft-inj/panic_noser/cases.sh b/cases/coverage/soft-inj/panic_noser/cases.sh
index 9637b9b..9637b9b 100755..100644
--- a/cases/soft-inj/panic_noser/cases.sh
+++ b/cases/coverage/soft-inj/panic_noser/cases.sh
diff --git a/cases/soft-inj/panic_noser/data/uc_over b/cases/coverage/soft-inj/panic_noser/data/uc_over
index 397c48a..397c48a 100644
--- a/cases/soft-inj/panic_noser/data/uc_over
+++ b/cases/coverage/soft-inj/panic_noser/data/uc_over
diff --git a/cases/soft-inj/panic_noser/data/uc_over_corrected b/cases/coverage/soft-inj/panic_noser/data/uc_over_corrected
index 3dac1ce..3dac1ce 100644
--- a/cases/soft-inj/panic_noser/data/uc_over_corrected
+++ b/cases/coverage/soft-inj/panic_noser/data/uc_over_corrected
diff --git a/cases/soft-inj/panic_noser/data/uc_over_timeout b/cases/coverage/soft-inj/panic_noser/data/uc_over_timeout
index 3b89831..3b89831 100644
--- a/cases/soft-inj/panic_noser/data/uc_over_timeout
+++ b/cases/coverage/soft-inj/panic_noser/data/uc_over_timeout
diff --git a/cases/soft-inj/panic_npcc/cases.sh b/cases/coverage/soft-inj/panic_npcc/cases.sh
index 35cc343..35cc343 100755..100644
--- a/cases/soft-inj/panic_npcc/cases.sh
+++ b/cases/coverage/soft-inj/panic_npcc/cases.sh
diff --git a/cases/soft-inj/panic_npcc/data/fatal_severity b/cases/coverage/soft-inj/panic_npcc/data/fatal_severity
index 4dfd8ed..4dfd8ed 100644
--- a/cases/soft-inj/panic_npcc/data/fatal_severity
+++ b/cases/coverage/soft-inj/panic_npcc/data/fatal_severity
diff --git a/cases/soft-inj/panic_npcc/data/uc_no_eripv b/cases/coverage/soft-inj/panic_npcc/data/uc_no_eripv
index 91f3b17..91f3b17 100644
--- a/cases/soft-inj/panic_npcc/data/uc_no_eripv
+++ b/cases/coverage/soft-inj/panic_npcc/data/uc_no_eripv
diff --git a/cases/soft-inj/panic_npcc/data/uc_no_eripv_timeout b/cases/coverage/soft-inj/panic_npcc/data/uc_no_eripv_timeout
index 23cd098..23cd098 100644
--- a/cases/soft-inj/panic_npcc/data/uc_no_eripv_timeout
+++ b/cases/coverage/soft-inj/panic_npcc/data/uc_no_eripv_timeout
diff --git a/cases/soft-inj/panic_npcc/data/uc_no_mcip b/cases/coverage/soft-inj/panic_npcc/data/uc_no_mcip
index c4ac81a..c4ac81a 100644
--- a/cases/soft-inj/panic_npcc/data/uc_no_mcip
+++ b/cases/coverage/soft-inj/panic_npcc/data/uc_no_mcip
diff --git a/cases/soft-inj/panic_npcc/data/uc_no_mcip_timeout b/cases/coverage/soft-inj/panic_npcc/data/uc_no_mcip_timeout
index 3f31a4f..3f31a4f 100644
--- a/cases/soft-inj/panic_npcc/data/uc_no_mcip_timeout
+++ b/cases/coverage/soft-inj/panic_npcc/data/uc_no_mcip_timeout
diff --git a/cases/soft-inj/panic_npcc/data/uncorrected b/cases/coverage/soft-inj/panic_npcc/data/uncorrected
index 3917b76..3917b76 100644
--- a/cases/soft-inj/panic_npcc/data/uncorrected
+++ b/cases/coverage/soft-inj/panic_npcc/data/uncorrected
diff --git a/cases/soft-inj/panic_npcc/data/uncorrected_timeout b/cases/coverage/soft-inj/panic_npcc/data/uncorrected_timeout
index 4a469cd..4a469cd 100644
--- a/cases/soft-inj/panic_npcc/data/uncorrected_timeout
+++ b/cases/coverage/soft-inj/panic_npcc/data/uncorrected_timeout
diff --git a/cases/soft-inj/panic_npcc/data/unknown b/cases/coverage/soft-inj/panic_npcc/data/unknown
index 25ecb5b..25ecb5b 100644
--- a/cases/soft-inj/panic_npcc/data/unknown
+++ b/cases/coverage/soft-inj/panic_npcc/data/unknown
diff --git a/cases/soft-inj/panic_ucr/cases.sh b/cases/coverage/soft-inj/panic_ucr/cases.sh
index 8aec7df..8aec7df 100755..100644
--- a/cases/soft-inj/panic_ucr/cases.sh
+++ b/cases/coverage/soft-inj/panic_ucr/cases.sh
diff --git a/cases/soft-inj/panic_ucr/data/s0_ar1 b/cases/coverage/soft-inj/panic_ucr/data/s0_ar1
index 85e931a..85e931a 100644
--- a/cases/soft-inj/panic_ucr/data/s0_ar1
+++ b/cases/coverage/soft-inj/panic_ucr/data/s0_ar1
diff --git a/cases/soft-inj/panic_ucr/data/srao_ewb_noripv b/cases/coverage/soft-inj/panic_ucr/data/srao_ewb_noripv
index 4518ff1..4518ff1 100644
--- a/cases/soft-inj/panic_ucr/data/srao_ewb_noripv
+++ b/cases/coverage/soft-inj/panic_ucr/data/srao_ewb_noripv
diff --git a/cases/soft-inj/panic_ucr/data/srao_mem_scrub_noripv b/cases/coverage/soft-inj/panic_ucr/data/srao_mem_scrub_noripv
index c9e2fd4..c9e2fd4 100644
--- a/cases/soft-inj/panic_ucr/data/srao_mem_scrub_noripv
+++ b/cases/coverage/soft-inj/panic_ucr/data/srao_mem_scrub_noripv
diff --git a/cases/soft-inj/panic_ucr/data/srar_no_en b/cases/coverage/soft-inj/panic_ucr/data/srar_no_en
index 378e58c..378e58c 100644
--- a/cases/soft-inj/panic_ucr/data/srar_no_en
+++ b/cases/coverage/soft-inj/panic_ucr/data/srar_no_en
diff --git a/cases/soft-inj/panic_ucr/data/srar_over b/cases/coverage/soft-inj/panic_ucr/data/srar_over
index 9e341d5..9e341d5 100644
--- a/cases/soft-inj/panic_ucr/data/srar_over
+++ b/cases/coverage/soft-inj/panic_ucr/data/srar_over
diff --git a/cases/soft-inj/panic_ucr/data/srar_unkown b/cases/coverage/soft-inj/panic_ucr/data/srar_unkown
index e3d42a6..e3d42a6 100644
--- a/cases/soft-inj/panic_ucr/data/srar_unkown
+++ b/cases/coverage/soft-inj/panic_ucr/data/srar_unkown
diff --git a/cases/soft-inj/panic_ucr/refer/srar_no_en b/cases/coverage/soft-inj/panic_ucr/refer/srar_no_en
index ea30561..ea30561 100644
--- a/cases/soft-inj/panic_ucr/refer/srar_no_en
+++ b/cases/coverage/soft-inj/panic_ucr/refer/srar_no_en
diff --git a/cases/soft-inj/poll_noser/cases.sh b/cases/coverage/soft-inj/poll_noser/cases.sh
index 4a3b285..4a3b285 100755..100644
--- a/cases/soft-inj/poll_noser/cases.sh
+++ b/cases/coverage/soft-inj/poll_noser/cases.sh
diff --git a/cases/soft-inj/poll_noser/data/uc_poll b/cases/coverage/soft-inj/poll_noser/data/uc_poll
index ef81543..ef81543 100644
--- a/cases/soft-inj/poll_noser/data/uc_poll
+++ b/cases/coverage/soft-inj/poll_noser/data/uc_poll
diff --git a/cases/soft-inj/poll_noser/refer/uc_poll b/cases/coverage/soft-inj/poll_noser/refer/uc_poll
index ea30561..ea30561 100644
--- a/cases/soft-inj/poll_noser/refer/uc_poll
+++ b/cases/coverage/soft-inj/poll_noser/refer/uc_poll
diff --git a/cases/soft-inj/poll_ucr/cases.sh b/cases/coverage/soft-inj/poll_ucr/cases.sh
index dc9fc6a..dc9fc6a 100755..100644
--- a/cases/soft-inj/poll_ucr/cases.sh
+++ b/cases/coverage/soft-inj/poll_ucr/cases.sh
diff --git a/cases/soft-inj/poll_ucr/data/fatal_poll b/cases/coverage/soft-inj/poll_ucr/data/fatal_poll
index 7efb316..7efb316 100644
--- a/cases/soft-inj/poll_ucr/data/fatal_poll
+++ b/cases/coverage/soft-inj/poll_ucr/data/fatal_poll
diff --git a/cases/soft-inj/poll_ucr/data/srar_poll b/cases/coverage/soft-inj/poll_ucr/data/srar_poll
index f57b5de..f57b5de 100644
--- a/cases/soft-inj/poll_ucr/data/srar_poll
+++ b/cases/coverage/soft-inj/poll_ucr/data/srar_poll
diff --git a/cases/soft-inj/poll_ucr/data/ucna b/cases/coverage/soft-inj/poll_ucr/data/ucna
index 2419b63..2419b63 100644
--- a/cases/soft-inj/poll_ucr/data/ucna
+++ b/cases/coverage/soft-inj/poll_ucr/data/ucna
diff --git a/cases/soft-inj/poll_ucr/data/ucna_over b/cases/coverage/soft-inj/poll_ucr/data/ucna_over
index c630057..c630057 100644
--- a/cases/soft-inj/poll_ucr/data/ucna_over
+++ b/cases/coverage/soft-inj/poll_ucr/data/ucna_over
diff --git a/cases/soft-inj/poll_ucr/refer/srar_poll b/cases/coverage/soft-inj/poll_ucr/refer/fatal_poll
index ea30561..ea30561 100644
--- a/cases/soft-inj/poll_ucr/refer/srar_poll
+++ b/cases/coverage/soft-inj/poll_ucr/refer/fatal_poll
diff --git a/cases/soft-inj/recoverable_ucr/refer/srao_no_en b/cases/coverage/soft-inj/poll_ucr/refer/srar_poll
index ea30561..ea30561 100644
--- a/cases/soft-inj/recoverable_ucr/refer/srao_no_en
+++ b/cases/coverage/soft-inj/poll_ucr/refer/srar_poll
diff --git a/cases/coverage/soft-inj/poll_ucr/refer/ucna b/cases/coverage/soft-inj/poll_ucr/refer/ucna
new file mode 100644
index 0000000..ea30561
--- /dev/null
+++ b/cases/coverage/soft-inj/poll_ucr/refer/ucna
@@ -0,0 +1 @@
+#empty
diff --git a/cases/coverage/soft-inj/poll_ucr/refer/ucna_over b/cases/coverage/soft-inj/poll_ucr/refer/ucna_over
new file mode 100644
index 0000000..ea30561
--- /dev/null
+++ b/cases/coverage/soft-inj/poll_ucr/refer/ucna_over
@@ -0,0 +1 @@
+#empty
diff --git a/cases/soft-inj/recoverable_ucr/cases.sh b/cases/coverage/soft-inj/recoverable_ucr/cases.sh
index ee74ccb..881827e 100755..100644
--- a/cases/soft-inj/recoverable_ucr/cases.sh
+++ b/cases/coverage/soft-inj/recoverable_ucr/cases.sh
@@ -35,16 +35,21 @@ get_result()
 
 verify()
 {
-    local removes="TSC TIME PROCESSOR"
-    case "$bcase" in
-	srao_*|srar_*)
-	    soft_inject_verify_mcelog
-	    verify_klog $klog
-	    soft_inject_verify_return_val
-	    ;;
-	*)
-	    echo "!!! Unknown case: $this_case !!!"
-    esac
+	local removes="TSC TIME PROCESSOR"
+	if [ "$bcase" = "srao_ucna" ]
+	then
+		removes="$removes RIP"
+	fi
+
+	case "$bcase" in
+		srao_*|srar_*)
+		soft_inject_verify_mcelog
+		verify_klog $klog
+		soft_inject_verify_return_val
+		;;
+		*)
+		echo "!!! Unknown case: $this_case !!!"
+	esac
 }
 
 soft_inject_main "$@"
diff --git a/cases/soft-inj/recoverable_ucr/data/srao_corrected b/cases/coverage/soft-inj/recoverable_ucr/data/srao_corrected
index ec24a0d..ec24a0d 100644
--- a/cases/soft-inj/recoverable_ucr/data/srao_corrected
+++ b/cases/coverage/soft-inj/recoverable_ucr/data/srao_corrected
diff --git a/cases/soft-inj/recoverable_ucr/data/srao_ewb b/cases/coverage/soft-inj/recoverable_ucr/data/srao_ewb
index 7dba8c0..7dba8c0 100644
--- a/cases/soft-inj/recoverable_ucr/data/srao_ewb
+++ b/cases/coverage/soft-inj/recoverable_ucr/data/srao_ewb
diff --git a/cases/soft-inj/recoverable_ucr/data/srao_mem_scrub b/cases/coverage/soft-inj/recoverable_ucr/data/srao_mem_scrub
index b4704b6..b4704b6 100644
--- a/cases/soft-inj/recoverable_ucr/data/srao_mem_scrub
+++ b/cases/coverage/soft-inj/recoverable_ucr/data/srao_mem_scrub
diff --git a/cases/soft-inj/recoverable_ucr/data/srao_no_en b/cases/coverage/soft-inj/recoverable_ucr/data/srao_no_en
index afac76b..afac76b 100644
--- a/cases/soft-inj/recoverable_ucr/data/srao_no_en
+++ b/cases/coverage/soft-inj/recoverable_ucr/data/srao_no_en
diff --git a/cases/soft-inj/recoverable_ucr/data/srao_over b/cases/coverage/soft-inj/recoverable_ucr/data/srao_over
index c5b337d..c5b337d 100644
--- a/cases/soft-inj/recoverable_ucr/data/srao_over
+++ b/cases/coverage/soft-inj/recoverable_ucr/data/srao_over
diff --git a/cases/soft-inj/recoverable_ucr/data/srao_ucna b/cases/coverage/soft-inj/recoverable_ucr/data/srao_ucna
index 80afaca..80afaca 100644
--- a/cases/soft-inj/recoverable_ucr/data/srao_ucna
+++ b/cases/coverage/soft-inj/recoverable_ucr/data/srao_ucna
diff --git a/cases/soft-inj/recoverable_ucr/data/srao_unknown b/cases/coverage/soft-inj/recoverable_ucr/data/srao_unknown
index 63e465a..63e465a 100644
--- a/cases/soft-inj/recoverable_ucr/data/srao_unknown
+++ b/cases/coverage/soft-inj/recoverable_ucr/data/srao_unknown
diff --git a/cases/soft-inj/recoverable_ucr/data/srar_usr_dcu b/cases/coverage/soft-inj/recoverable_ucr/data/srar_usr_dcu
index 335b6bc..335b6bc 100644
--- a/cases/soft-inj/recoverable_ucr/data/srar_usr_dcu
+++ b/cases/coverage/soft-inj/recoverable_ucr/data/srar_usr_dcu
diff --git a/cases/soft-inj/recoverable_ucr/data/srar_usr_dcu_irq b/cases/coverage/soft-inj/recoverable_ucr/data/srar_usr_dcu_irq
index 5572bf6..5572bf6 100644
--- a/cases/soft-inj/recoverable_ucr/data/srar_usr_dcu_irq
+++ b/cases/coverage/soft-inj/recoverable_ucr/data/srar_usr_dcu_irq
diff --git a/cases/soft-inj/recoverable_ucr/data/srar_usr_dcu_nmi b/cases/coverage/soft-inj/recoverable_ucr/data/srar_usr_dcu_nmi
index 3af1cc7..3af1cc7 100644
--- a/cases/soft-inj/recoverable_ucr/data/srar_usr_dcu_nmi
+++ b/cases/coverage/soft-inj/recoverable_ucr/data/srar_usr_dcu_nmi
diff --git a/cases/soft-inj/recoverable_ucr/refer/srao_corrected b/cases/coverage/soft-inj/recoverable_ucr/refer/srao_corrected
index 63e465a..63e465a 100644
--- a/cases/soft-inj/recoverable_ucr/refer/srao_corrected
+++ b/cases/coverage/soft-inj/recoverable_ucr/refer/srao_corrected
diff --git a/cases/coverage/soft-inj/recoverable_ucr/refer/srao_no_en b/cases/coverage/soft-inj/recoverable_ucr/refer/srao_no_en
new file mode 100644
index 0000000..ea30561
--- /dev/null
+++ b/cases/coverage/soft-inj/recoverable_ucr/refer/srao_no_en
@@ -0,0 +1 @@
+#empty
diff --git a/cases/soft-inj/recoverable_ucr/refer/srao_ucna b/cases/coverage/soft-inj/recoverable_ucr/refer/srao_ucna
index 65d78c8..a2b73d7 100644
--- a/cases/soft-inj/recoverable_ucr/refer/srao_ucna
+++ b/cases/coverage/soft-inj/recoverable_ucr/refer/srao_ucna
@@ -4,3 +4,6 @@ MCGSTATUS RIPV MCIP
 ADDR 0x1234
 RIP 0x73:0x1eadbabe
 
+CPU 1 BANK 1
+STATUS UNCORRECTED UCNA
+MCGSTATUS RIPV MCIP
diff --git a/cases/coverage/soft-inj/run_driver.sh b/cases/coverage/soft-inj/run_driver.sh
new file mode 100644
index 0000000..07fc8fb
--- /dev/null
+++ b/cases/coverage/soft-inj/run_driver.sh
@@ -0,0 +1,17 @@
+#!/bin/sh
+
+echo 0 > $TMP_DIR/error.$$
+
+pushd `dirname $0` > /dev/null
+./driver_kdump.sh config/kdump.conf
+[ $? -eq 0 ] || echo 1 > $TMP_DIR/error.$$
+popd > /dev/null
+
+grep -q "1" $TMP_DIR/error.$$
+if [ $? -eq 0 ]
+then
+	exit 1
+else
+	exit 0
+fi
+
diff --git a/cases/coverage/soft-inj/run_simple.sh b/cases/coverage/soft-inj/run_simple.sh
new file mode 100644
index 0000000..f80d308
--- /dev/null
+++ b/cases/coverage/soft-inj/run_simple.sh
@@ -0,0 +1,17 @@
+#!/bin/sh
+
+echo 0 > $TMP_DIR/error.$$
+
+pushd `dirname $0` > /dev/null
+./driver_simple.sh config/simple.conf
+[ $? -eq 0 ] || echo 1 > $TMP_DIR/error.$$
+popd > /dev/null
+
+grep -q "1" $TMP_DIR/error.$$
+if [ $? -eq 0 ]
+then
+	exit 1
+else
+	exit 0
+fi
+
diff --git a/drivers/kdump/setup.sh b/cases/coverage/soft-inj/setup.sh
index f90742c..f90742c 100755..100644
--- a/drivers/kdump/setup.sh
+++ b/cases/coverage/soft-inj/setup.sh
diff --git a/cases/function/Makefile b/cases/function/Makefile
new file mode 100644
index 0000000..6d66c0c
--- /dev/null
+++ b/cases/function/Makefile
@@ -0,0 +1,18 @@
+all:
+	$(MAKE) -C erst-inject
+	$(MAKE) -C pfa
+	$(MAKE) -C hwpoison
+#	$(MAKE) -C kvm
+
+clean:
+	$(MAKE) -C erst-inject clean
+	$(MAKE) -C pfa clean
+	$(MAKE) -C hwpoison clean
+#	$(MAKE) -C kvm clean
+
+install:
+	$(MAKE) -C erst-inject install
+	$(MAKE) -C pfa install
+	$(MAKE) -C hwpoison install
+#	$(MAKE) -C kvm install
+
diff --git a/cases/function/apei-inj/apei-inject.sh b/cases/function/apei-inj/apei-inject.sh
new file mode 100644
index 0000000..ffa5cf0
--- /dev/null
+++ b/cases/function/apei-inj/apei-inject.sh
@@ -0,0 +1,118 @@
+# Copyright (C) 2012, Intel Corp.
+# This file is released under the GPLv2.
+#
+#
+export ROOT=`(cd ../../../; pwd)`
+
+. $ROOT/lib/functions.sh
+setup_path
+. $ROOT/lib/mce.sh
+
+APEI_IF=""
+GHES_REC="Hardware error from APEI Generic Hardware Error Source"
+
+check_err_type()
+{
+	local type=`printf 0x%08x $1`
+
+	cat $APEI_IF/available_error_type 2>/dev/null | cut -f1 | grep -q $type
+	[ $? -eq 0 ] ||
+	{
+		echo "The error type \"$1\" is not supported on this platform"
+		return 1
+	}
+}
+
+# On some machines the trigger will be happend after 15 ~ 20 seconds, so
+# when no proper log is read out, just executing wait-retry loop until
+# timeout.
+check_result()
+{
+	local timeout=25
+	local sleep=5
+	local time=0
+
+	while [ $time -lt $timeout ]
+	do
+		dmesg -c | grep -q "$GHES_REC"
+		[ $? -eq 0 ] && return 0
+		time=`expr $time + $sleep`
+	done
+
+	return 1
+}
+
+main()
+{
+	#inject error type
+	local type=$1
+
+	check_debugfs
+	#APEI_IF should be defined after debugfs is mounted
+	APEI_IF=`mount | grep debugfs | cut -d ' ' -f3 | head -1`/apei/einj
+
+	#if einj is a module, it is ensured to have been loaded
+	modinfo einj > /dev/null 2>&1
+	if [ $? -eq 0 ]; then
+		[ -d $APEI_IF ] || modprobe einj param_extension=1
+		[ $? -eq 0 ] ||
+		die "module einj isn't supported or EINJ Table doesn't exist?"
+	fi
+
+	check_err_type $type
+	[ $? -ne 0 ] && return 1
+
+	mcelog &> /dev/null
+	echo $type > $APEI_IF/error_type
+	killall simple_process > /dev/null 2>&1
+	simple_process > /dev/null &
+
+	page-types -p `pidof simple_process` -LN -b ano > $TMP_DIR/pagelist.$$
+
+	ADDR=`awk '$2 != "offset" {print "0x"$2"000"}' $TMP_DIR/pagelist.$$ | sed -n -e '1p'`
+	if [ -f $APEI_IF/param1 ]
+	then
+		echo $ADDR > $APEI_IF/param1
+		echo 0xfffffffffffff000 > $APEI_IF/param2
+	fi
+
+	dmesg -c > /dev/null
+	echo "1" > $APEI_IF/error_inject 2>/dev/null
+	[ $? -ne 0 ] &&
+	{
+		cat <<-EOF
+
+		Error injection fails, it maybe happens on some
+		bogus BIOS. For example, some iomem region can't
+		be acquired when requesting some resources. Please
+		contact BIOS engineer to get further information.
+
+		EOF
+	}
+	sleep 1
+
+	check_result
+	if [ $? -eq 0 ]
+	then
+		echo "  PASSED: GHES record is ok"
+		exit 0
+	else
+		echo "  FAILED: GHES record is not expected"
+		exit 1
+	fi
+}
+
+usage()
+{
+	cat <<-EOF
+	usage: ${0##*/} [ available_error_type ]
+	example: ${0##*/} [ 0x8 | 0x10 | 0x20 | ... ]
+
+	EOF
+
+	exit 0
+}
+
+[ $# -eq 0 ] && usage
+
+main $1
diff --git a/cases/function/apei-inj/runtest.sh b/cases/function/apei-inj/runtest.sh
new file mode 100644
index 0000000..196f41b
--- /dev/null
+++ b/cases/function/apei-inj/runtest.sh
@@ -0,0 +1,36 @@
+#!/bin/sh
+# This test is the basic EINJ functional test. Only Memory Correctable Error
+# is touched because other tests are possible to cause system hang/crash.
+#
+#0x00000008      Memory Correctable
+#0x00000010      Memory Uncorrectable non-fatal
+#0x00000020      Memory Uncorrectable fatal
+
+cat <<-EOF
+
+***************************************************************************
+Pay attention:
+
+This test is basic APEI/EINJ functional test. Because other error injections
+are possible to cause system hang/crash, only Memory Correctable Error is
+injected to test the availiability of APEI/EINJ.
+***************************************************************************
+
+
+EOF
+
+echo 0 > $TMP_DIR/error.$$
+
+pushd `dirname $0` > /dev/null
+./apei-inject.sh 0x8
+[ $? -eq 0 ] || echo 1 > $TMP_DIR/error.$$
+popd > /dev/null
+
+grep -q "1" $TMP_DIR/error.$$
+if [ $? -eq 0 ]
+then
+	exit 1
+else
+	exit 0
+fi
+
diff --git a/cases/function/erst-inject/Makefile b/cases/function/erst-inject/Makefile
new file mode 100644
index 0000000..8077bd3
--- /dev/null
+++ b/cases/function/erst-inject/Makefile
@@ -0,0 +1,8 @@
+CFLAGS := -g -Wall
+
+erst-inject: erst-inject.o
+
+install: erst-inject
+
+clean:
+	rm -f *.o erst-inject
diff --git a/tsrc/erst-inj/cper.h b/cases/function/erst-inject/cper.h
index 63cd94b..63cd94b 100644
--- a/tsrc/erst-inj/cper.h
+++ b/cases/function/erst-inject/cper.h
diff --git a/tsrc/erst-inj/erst-inject.c b/cases/function/erst-inject/erst-inject.c
index 369ef22..369ef22 100644
--- a/tsrc/erst-inj/erst-inject.c
+++ b/cases/function/erst-inject/erst-inject.c
diff --git a/tsrc/erst-inject.sh b/cases/function/erst-inject/runtest.sh
index f078993..c8922b7 100755..100644
--- a/tsrc/erst-inject.sh
+++ b/cases/function/erst-inject/runtest.sh
@@ -43,66 +43,90 @@
 # Author: Chen Gong <gong.chen@intel.com>
 #
 
+cat <<-EOF
+
+***************************************************************************
+Pay attention:
+
+This test is basic APEI/ERST functional test. In this test case, it will
+test ERST functionality of READ/WRITE/ERASE. Any error in the test
+procedure will be consider as failure and reported.
+***************************************************************************
+
+
+EOF
 
 ID=0xdeadbeaf
 ERST=./erst-inject
-LOG=./erst.log
+LOG=$TMP_DIR/erst.log.$$
 MODSTATUS=0
 
 err()
 {
-       echo "$*"
-       echo "test fails"
-       exit 1
+	echo
+	echo ERROR: "$*"
+	echo ERROR: "Please check dmesg for further information"
+	echo -e "\n\nTEST FAILS"
+	exit 1
 }
 
+pushd `dirname $0` > /dev/null
+
 #prepare the test env
 ls /dev/erst_dbg >/dev/null 2>&1
-if [ ! $? -eq 0 ]; then
-       modinfo erst_dbg > /dev/null 2>&1
-       [ $? -eq 0 ] || err "please ensure module erst_dbg existing"
-       modprobe erst_dbg
-       [ $? -eq 0 ] || err "fail to load module erst_dbg"
-       MODSTATUS=1
+if [ $? -ne 0 ]; then
+	modinfo erst_dbg > /dev/null 2>&1
+	[ $? -eq 0 ] || err "Please ensure module erst_dbg existing"
+	modprobe erst_dbg
+	[ $? -eq 0 ] || err "Fail to load module erst_dbg"
+	MODSTATUS=1
 fi
 
-ls $ERST > /dev/null 2>&1
-[ $? -eq 0 ] || err "please compile the test program first"
+which $ERST &> /dev/null
+[ $? -eq 0 ] || err "Please compile the test case first"
 
-echo "write one error record into ERST..."
+echo -n "Write one error record into ERST... "
 $ERST -i $ID 1>/dev/null
 if [ ! $? -eq 0 ]; then
-       err "ERST writing operation fails"
+	err "ERST writing operation fails"
 fi
-echo "done"
+sleep 1
+echo "DONE"
 # read all error records in ERST
 $ERST -p > $LOG
-echo "check if existing the error record written before..."
+echo -n "Check if existing the error record written before... "
 grep -q $ID $LOG
 if [ ! $? -eq 0 ]; then
-       err "don't find the error record written before in ERST"
+	err "Don't find the error record written before in ERST"
 fi
-echo "done"
+sleep 1
+echo "DONE"
 
-echo "clear the error record written before..."
+echo -n "Clear the error record written before... "
 $ERST -c $ID 1>/dev/null
 if [ ! $? -eq 0 ]; then
-       err "ERST writing opertion fails"
+	err "ERST writing opertion fails"
 fi
-echo "done"
+sleep 1
+echo "DONE"
 
 #read all error records again
 $ERST -p > $LOG
 
-echo "check if the error record has been cleared..."
+echo -n "Check if the error record has been cleared... "
 grep -q $ID $LOG
 if [ $? -eq 0 ]; then
-       err "ERST clearing opertion fails"
+	err "ERST clearing opertion fails"
 fi
-echo "done"
-echo -e "\ntest passes"
+sleep 1
+echo "DONE"
+
+popd > /dev/null
 
 rm -f $LOG
 if [ $MODSTATUS -eq 1 ]; then
-       rmmod -f erst_dbg
+	rmmod -f erst_dbg
 fi
+
+echo -e "\nTEST PASSES"
+
diff --git a/tsrc/erst-inj/uuid.h b/cases/function/erst-inject/uuid.h
index 765656a..765656a 100644
--- a/tsrc/erst-inj/uuid.h
+++ b/cases/function/erst-inject/uuid.h
diff --git a/cases/function/hwpoison/Makefile b/cases/function/hwpoison/Makefile
new file mode 100644
index 0000000..1256feb
--- /dev/null
+++ b/cases/function/hwpoison/Makefile
@@ -0,0 +1,19 @@
+CFLAGS += -I -g -Wall
+
+EXE := tinjpage tsimpleinj tkillpoison tprctl tsoft tsoftinj
+EXE += thugetlb ttranshuge
+
+OBJ := $(addsuffix .o,${EXE})
+
+all: ${EXE}
+
+install: all
+	cp ttranshuge ../../../bin/
+
+tinjpage: LDFLAGS += -lpthread
+
+clean:
+	rm -f ${EXE} ${OBJ}
+	rm -f unpoison-failed offlined
+	rm -f ~test* ~poison*
+
diff --git a/cases/function/hwpoison/README b/cases/function/hwpoison/README
new file mode 100644
index 0000000..1c6de9b
--- /dev/null
+++ b/cases/function/hwpoison/README
@@ -0,0 +1,15 @@
+tinjpage
+
+tinjpage is a coverage test for the hwpoison page recovery code
+in the kernel. It tests different cases by putting pages
+into different stages, triggering poison injection on them
+and verifies the results.
+
+tinjpage requires a kernel with MADV_POISON injection support and the
+hwpoison testkit, but doesn't require a kernel tree and can be built directly with
+make tinjpage
+
+A few of the test cases are timing dependent and might require adjustment
+to run successfully.
+
+-Andi Kleen
diff --git a/tsrc/hugepage.h b/cases/function/hwpoison/hugepage.h
index ba5bb34..ba5bb34 100644
--- a/tsrc/hugepage.h
+++ b/cases/function/hwpoison/hugepage.h
diff --git a/tsrc/random_offline b/cases/function/hwpoison/random_offline
index c380a86..ecd0b3e 100755..100644
--- a/tsrc/random_offline
+++ b/cases/function/hwpoison/random_offline
@@ -13,17 +13,20 @@
 
 #mount -t debugfs none /debug
 
+ROOT=`(cd ../../../; pwd)`
+. $ROOT/lib/mce.sh
+
 THRESH=1000
 SEED=""
 RUNTIME=""
-DEBUG=/sys/kernel/debug
+DEBUG=""
 
-fail() { 
+fail() {
 	echo "ERROR: $@"
 	exit 0
 }
 
-usage() { 
+usage() {
 	echo "Usage:"
 	echo "random_offline options"
 	echo -- "-t seconds   runtime in seconds (default unlimited)"
@@ -33,7 +36,7 @@ usage() {
 }
 
 while getopts "t:m:s:" option ; do
-	case "$option" in 
+	case "$option" in
 	t) RUNTIME=$OPTARG ;;
 	m) THRESH=$OPTARG ;;
 	s) SEED=$OPTARG ;;
@@ -42,8 +45,14 @@ while getopts "t:m:s:" option ; do
 done
 
 [ "$(whoami)" != root ] && fail "Not root"
-[ ! -d $DEBUG/hwpoison ] && mount -t debugfs none $DEBUG
-[ ! -d $DEBUG/hwpoison ] && fail "No debugfs"
+check_debugfs
+DEBUG=`mount | grep debugfs | cut -d ' ' -f3 | head -1`
+#if hwpoison_inject is a module, it is ensured to have been loaded
+modinfo hwpoison_inject > /dev/null 2>&1
+if [ $? -eq 0 ]; then
+	[ -d $DEBUG/hwpoison/ ] || modprobe hwpoison_inject
+	[ $? -eq 0 ] || fail "module hwpoison_inject isn't supported ?"
+fi
 [ ! -w /sys/devices/system/memory/soft_offline_page ] && fail "No soft offlining support in kernel"
 [ ! -w $DEBUG/hwpoison/unpoison-pfn ] && fail "no unpoison support in kernel"
 
@@ -52,8 +61,8 @@ end_of_memory() {
 		case "$(< $i/type)" in
 		"System RAM") ;;
 		*) continue ;;
-		esac	
-	
+		esac
+
 		k=$(< $i/end)
 		k=${k/0x/}
 		k=$(echo $k | tr a-z A-Z)
@@ -64,15 +73,15 @@ end_of_memory() {
 
 E=$(end_of_memory)
 
-echo "soft offlining pages upto $E" 
+echo "soft offlining pages upto $E"
 
-unpoison() { 
+unpoison() {
 	if [ ! -f offlined ] ; then
 		return
 	fi
-	
+
 	echo unpoisioning
-	while read i ; do 
+	while read i ; do
 		#echo -n ,
 		#echo "u $i"
 		(( utotal++ ))
@@ -108,25 +117,25 @@ cbefore=$(grep HardwareCorrupted /proc/meminfo)
 
 (( k = 0 ))
 rm -f offlined unpoison-failed
-while true ; do 
-	T=$( 
+while true ; do
+	T=$(
 	R=$RANDOM
 	X=$(echo "obase=16; ($R%$E)*4096"  | bc)
 	echo 0x$X
-	) 
-	#echo "p $T" 
-	(( total++ )) 
+	)
+	#echo "p $T"
+	(( total++ ))
 	if echo 2>/dev/null $T >/sys/devices/system/memory/soft_offline_page ; then
 		echo $T >> offlined
-		(( success++ )) 
+		(( success++ ))
 	else
 		#echo offlining $T failed $?
 		(( failed++ ))
 		true
 	fi
-	#echo -n . 
+	#echo -n .
 
-	(( k++ )) 
+	(( k++ ))
 	if [ $k -gt $THRESH ] ; then
 		unpoison
 		(( k = 0 ))
diff --git a/tsrc/run-huge-test.sh b/cases/function/hwpoison/run-huge-test.sh
index 2b679d4..f5347f1 100755..100644
--- a/tsrc/run-huge-test.sh
+++ b/cases/function/hwpoison/run-huge-test.sh
@@ -6,7 +6,7 @@
 
 usage()
 {
-    echo "Usage: ./run-huge-test.sh hugetlbfs_directory" && exit 1
+	echo "Usage: run-huge-test.sh hugetlbfs_directory" && exit 0
 }
 
 htdir=$1
@@ -19,58 +19,58 @@ echo 1000 > /proc/sys/vm/nr_hugepages
 num=0
 
 exec_testcase() {
-    error=0
-    echo "TestCase $@"
-    hpage_size=$1
-    hpage_target=$2
-    num=$7
-
-    if [ "$3" = "head" ] ; then
-	hpage_target_offset=0
-    elif [ "$3" = "tail" ] ; then
-	hpage_target_offset=1
-    else
-	error=1
-    fi
-    hpage_target=$((hpage_target * 512 + hpage_target_offset))
-
-    if [ "$4" = "early" ] ; then
-	process_type="-e"
-    elif [ "$4" = "late_touch" ] ; then
-	process_type=""
-    elif [ "$4" = "late_avoid" ] ; then
-	process_type="-a"
-    else
-	error=1
-    fi
-
-    if [ "$5" = "anonymous" ] ; then
-	file_type="-A"
-    elif [ "$5" = "file" ] ; then
-	file_type="-f $num"
-    elif [ "$5" = "shm" ] ; then
-	file_type="-S"
-    else
-	error=1
-    fi
-
-    if [ "$6" = "fork_shared" ] ; then
-	share_type="-F"
-    elif [ "$6" = "fork_private_nocow" ] ; then
-	share_type="-Fp"
-    elif [ "$6" = "fork_private_cow" ] ; then
-	share_type="-Fpc"
-    else
-	error=1
-    fi
-
-    command="./thugetlb -x -m $hpage_size -o $hpage_target $process_type $file_type $share_type $htdir &"
-    echo $command
-    eval $command
-    wait $!
-    echo ""
-
-    return 0
+	error=0
+	echo "TestCase $@"
+	hpage_size=$1
+	hpage_target=$2
+	num=$7
+
+	if [ "$3" = "head" ] ; then
+		hpage_target_offset=0
+	elif [ "$3" = "tail" ] ; then
+		hpage_target_offset=1
+	else
+		error=1
+	fi
+	hpage_target=$((hpage_target * 512 + hpage_target_offset))
+
+	if [ "$4" = "early" ] ; then
+		process_type="-e"
+	elif [ "$4" = "late_touch" ] ; then
+		process_type=""
+	elif [ "$4" = "late_avoid" ] ; then
+		process_type="-a"
+	else
+		error=1
+	fi
+
+	if [ "$5" = "anonymous" ] ; then
+		file_type="-A"
+	elif [ "$5" = "file" ] ; then
+		file_type="-f $num"
+	elif [ "$5" = "shm" ] ; then
+		file_type="-S"
+	else
+		error=1
+	fi
+
+	if [ "$6" = "fork_shared" ] ; then
+		share_type="-F"
+	elif [ "$6" = "fork_private_nocow" ] ; then
+		share_type="-Fp"
+	elif [ "$6" = "fork_private_cow" ] ; then
+		share_type="-Fpc"
+	else
+		error=1
+	fi
+
+	command="./thugetlb -x -m $hpage_size -o $hpage_target $process_type $file_type $share_type $htdir &"
+	echo $command
+	eval $command
+	wait $!
+	echo ""
+
+	return 0
 }
 
 num=$((num+1))
@@ -165,4 +165,3 @@ exec_testcase 2 1 "tail" "late_avoid" "anonymous" "fork_private_cow" $num
 
 # free IPC semaphores used by thugetlb.c
 ipcs -s|grep $USER|cut -f2 -d' '|xargs ipcrm sem 
-
diff --git a/cases/function/hwpoison/run-transhuge-test.sh b/cases/function/hwpoison/run-transhuge-test.sh
new file mode 100644
index 0000000..5221bb0
--- /dev/null
+++ b/cases/function/hwpoison/run-transhuge-test.sh
@@ -0,0 +1,100 @@
+#
+# run-transhuge-test.sh:
+#     Script for hwpoison test of THP(Transparent Huge Page).
+#
+#!/bin/sh
+#
+
+ROOT=`(cd ../../../; pwd)`
+
+. $ROOT/lib/functions.sh
+setup_path
+
+THP_POISON_PRO="ttranshuge"
+
+THP_SYS_PATH="/sys/kernel/mm/transparent_hugepage"
+THP_SYS_ENABLED_FILE="$THP_SYS_PATH/enabled"
+
+executed_testcase=0
+failed_testcase=0
+
+error()
+{
+	echo "$1" && exit 1
+}
+
+env_check()
+{
+	which $THP_POISON_PRO > /dev/null 2>&1
+	[ $? -ne 0 ] && error "Please make sure there is file $THP_POISON_PRO."
+
+	if [ ! -d $THP_SYS_PATH ] ; then
+		error "THP(Transparent Huge Page) may be not supported by kernel."
+	fi
+
+	thp_enabled="$(cat $THP_SYS_ENABLED_FILE | awk '{print $3}')"
+	if [ "$thp_enabled" == "[never]" ] ; then
+		error "THP(Transparent Huge Page) is disabled now."
+	fi
+}
+
+result_check()
+{
+	if [ "$1" != "0" ] ; then
+		failed_testcase=`expr $failed_testcase + 1`
+	fi
+}
+
+exec_testcase()
+{
+	if [ "$1" = "head" ] ; then
+		page_position_in_thp=0
+	elif [ "$1" = "tail" ] ; then
+		page_position_in_thp=1
+	else
+		error "Which page do you want to poison?"
+	fi
+
+	if [ "$2" = "early" ] ; then
+		process_type="--early-kill"
+	elif [ "$2" = "late_touch" ] ; then
+		process_type=""
+	elif [ "$2" = "late_avoid" ] ; then
+		process_type="--avoid-touch"
+	else
+		error "No such process type."
+	fi
+
+	executed_testcase=`expr $executed_testcase + 1`
+
+	echo "------------------ Case $executed_testcase --------------------"
+
+	command="$THP_POISON_PRO $process_type --offset $page_position_in_thp"
+	echo $command
+	eval $command
+	result_check $?
+
+	echo -e "\n"
+}
+
+# Environment Check for Test.
+env_check
+
+# Execute Test Cases from Here.
+echo "============= HWPoison Test of Transparent Huge Page ================="
+
+exec_testcase "head" "early"
+
+exec_testcase "head" "late_touch"
+
+exec_testcase "head" "late_avoid"
+
+exec_testcase "tail" "early"
+
+exec_testcase "tail" "late_touch"
+
+exec_testcase "tail" "late_avoid"
+
+echo "======================================================================="
+echo -n "    Num of Executed Test Case: $executed_testcase"
+echo -e "    Num of Failed Case: $failed_testcase\n"
diff --git a/cases/function/hwpoison/run_hard.sh b/cases/function/hwpoison/run_hard.sh
new file mode 100644
index 0000000..63fef88
--- /dev/null
+++ b/cases/function/hwpoison/run_hard.sh
@@ -0,0 +1,36 @@
+#!/bin/sh
+
+cat <<-EOF
+
+***************************************************************************
+Pay attention:
+
+This test is hard mode of HWPoison functional test.
+***************************************************************************
+
+
+EOF
+
+echo 0 > $TMP_DIR/error.$$
+
+pushd `dirname $0` > /dev/null
+./tinjpage
+./tsimpleinj
+if ! ./tkillpoison
+then
+	echo "killed as expected"
+else
+	echo "didn't get killed"
+	echo 1 > $TMP_DIR/error.$$
+fi
+./tprctl
+
+popd > /dev/null
+
+grep -q "1" $TMP_DIR/error.$$
+if [ $? -eq 0 ]
+then
+	exit 1
+else
+	exit 0
+fi
diff --git a/cases/function/hwpoison/run_hugepage.sh b/cases/function/hwpoison/run_hugepage.sh
new file mode 100644
index 0000000..4cf337d
--- /dev/null
+++ b/cases/function/hwpoison/run_hugepage.sh
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+cat <<-EOF
+
+***************************************************************************
+Pay attention:
+
+This is the functional test for huge page support of HWPoison.
+***************************************************************************
+
+
+EOF
+
+pushd `dirname $0` > /dev/null
+
+HT=$TMP_DIR/hugepage
+mkdir -p $HT
+mount -t hugetlbfs none $HT
+./run-huge-test.sh $HT
+umount $HT
+popd > /dev/null
diff --git a/cases/function/hwpoison/run_soft.sh b/cases/function/hwpoison/run_soft.sh
new file mode 100644
index 0000000..71edc01
--- /dev/null
+++ b/cases/function/hwpoison/run_soft.sh
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+cat <<-EOF
+
+***************************************************************************
+Pay attention:
+
+This test is soft mode of HWPoison functional test.
+***************************************************************************
+
+
+EOF
+
+pushd `dirname $0` > /dev/null
+
+./tsoft
+./tsoftinj
+echo "Running soft offline for 60 seconds"
+./random_offline -t 60
+
+popd > /dev/null
diff --git a/cases/function/hwpoison/run_thp.sh b/cases/function/hwpoison/run_thp.sh
new file mode 100644
index 0000000..5060fce
--- /dev/null
+++ b/cases/function/hwpoison/run_thp.sh
@@ -0,0 +1,27 @@
+#!/bin/sh
+
+cat <<-EOF
+
+***************************************************************************
+Pay attention:
+
+This is the functional test for transparent huge page support of HWPoison.
+***************************************************************************
+
+
+EOF
+
+echo 0 > $TMP_DIR/error.$$
+
+pushd `dirname $0` > /dev/null
+./run-transhuge-test.sh
+[ $? -eq 0 ] || echo 1 > $TMP_DIR/error.$$
+popd > /dev/null
+
+grep -q "1" $TMP_DIR/error.$$
+if [ $? -eq 0 ]
+then
+	exit 1
+else
+	exit 0
+fi
diff --git a/tsrc/thugetlb.c b/cases/function/hwpoison/thugetlb.c
index 0f07a6a..92dc7d2 100644
--- a/tsrc/thugetlb.c
+++ b/cases/function/hwpoison/thugetlb.c
@@ -5,7 +5,7 @@
 #define _GNU_SOURCE 1
 #include <stdlib.h>
 #include <stdio.h>
-#include <string.h> 
+#include <string.h>
 #include <fcntl.h>
 #include <signal.h>
 #include <unistd.h>
diff --git a/tsrc/tinjpage.c b/cases/function/hwpoison/tinjpage.c
index bee38af..2ea74a3 100644
--- a/tsrc/tinjpage.c
+++ b/cases/function/hwpoison/tinjpage.c
@@ -2,7 +2,7 @@
  * Test program for Linux poison memory error recovery.
  * This injects poison into various mapping cases and triggers the poison
  * handling.  Requires special injection support in the kernel.
- * 
+ *
  * Copyright 2009, 2010 Intel Corporation
  *
  * tinjpage is free software; you can redistribute it and/or
@@ -16,8 +16,8 @@
  * General Public License for more details.
  *
  * You should find a copy of v2 of the GNU General Public License somewhere
- * on your Linux system; if not, write to the Free Software Foundation, 
- * Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA 
+ * on your Linux system; if not, write to the Free Software Foundation,
+ * Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
  *
  * Authors: Andi Kleen, Fengguang Wu
  */
@@ -178,25 +178,25 @@ u64 page_to_pfn(char *page)
 	u64 pfn;
 
 	if (pagemap_fd < 0)  {
-		pagemap_fd = open("/proc/self/pagemap", O_RDONLY); 
+		pagemap_fd = open("/proc/self/pagemap", O_RDONLY);
 		if (pagemap_fd < 0)
 			err("/proc/self/pagemap not supported");
 	}
 
-	if (pread(pagemap_fd, &pfn, sizeof(u64), 
+	if (pread(pagemap_fd, &pfn, sizeof(u64),
 		((u64)page / PS)*sizeof(u64)) != sizeof(u64))
 		err("Cannot read from pagemap");
 
-	pfn &= (1ULL<<56)-1; 
+	pfn &= (1ULL<<56)-1;
 	return pfn;
 }
 
-/* 
- * Inject Action Optional #MC 
+/*
+ * Inject Action Optional #MC
  * with mce-inject using the software injector.
- * 
+ *
  * This tests the low level machine check handler too.
- * 
+ *
  * Slightly racy with page migration because we don't mlock the page.
  */
 void inject_mce_inject(char *page)
@@ -211,17 +211,17 @@ void inject_mce_inject(char *page)
 		exit(1);
 	}
 
-	fprintf(mce_inject, 
+	fprintf(mce_inject,
 		"CPU 0 BANK 3 STATUS UNCORRECTED SRAO 0xc0\n"
 		"MCGSTATUS RIPV MCIP\n"
 		"ADDR %#llx\n"
 		"MISC 0x8c\n"
 		"RIP 0x73:0x1eadbabe\n", pfn);
 
-	if (ferror(mce_inject) || fclose(mce_inject) < 0) { 
+	if (ferror(mce_inject) || fclose(mce_inject) < 0) {
 		fprintf(stderr, "mce-inject failed: %s\n", strerror(errno));
 		exit(1);
-	} 
+	}
 }
 
 void (*inject)(char *page) = inject_madvise;
@@ -233,7 +233,7 @@ void poison(char *msg, char *page, enum rmode mode)
 
 	if (sigsetjmp(early_recover_ctx, 1) == 0) {
 		inject(page);
-		
+
 		if (early_kill && (mode == MWRITE || mode == MREAD)) {
 			printf("XXX: %s: process is not early killed\n", msg);
 			failure++;
@@ -304,7 +304,7 @@ void expecterr(char *msg, int err)
 	}
 }
 
-/* 
+/*
  * Any optional error is really a deficiency in the kernel VFS error reporting
  * and should be eventually fixed and turned into a expecterr
  */
@@ -383,13 +383,13 @@ static void do_file_clean(int flags, char *name)
 	if (fd < 0)
 		err("open temp file");
 	write(fd, fn, 4);
-	page = checked_mmap(NULL, PS, PROT_READ|PROT_WRITE, MAP_SHARED|flags, 
+	page = checked_mmap(NULL, PS, PROT_READ|PROT_WRITE, MAP_SHARED|flags,
 		fd, 0);
 	fsync(fd);
 	close(fd);
 	testmem(name, page, MREAD_OK);
 	 /* reread page from disk */
-	printf("\t reading %x\n", *(unsigned char *)page);	
+	printf("\t reading %x\n", *(unsigned char *)page);
 	testmem(name, page, MWRITE_OK);
 }
 
@@ -417,7 +417,7 @@ static void do_file_dirty(int flags, char *name)
 	fn[0] = 0;
 	int fd = playfile(fn);
 
-	page = checked_mmap(NULL, PS, PROT_READ, 
+	page = checked_mmap(NULL, PS, PROT_READ,
 			MAP_SHARED|MAP_POPULATE|flags, fd, 0);
 	testmem(ndesc(nbuf, name, "initial"), page, MREAD);
 	expecterr("msync expect error", msync(page, PS, MS_SYNC) < 0);
@@ -426,7 +426,7 @@ static void do_file_dirty(int flags, char *name)
 
 	fd = open(fn, O_RDONLY);
 	if (fd < 0) err("reopening temp file");
-	page = checked_mmap(NULL, PS, PROT_READ, MAP_SHARED|MAP_POPULATE|flags, 
+	page = checked_mmap(NULL, PS, PROT_READ, MAP_SHARED|MAP_POPULATE|flags,
 				fd, 0);
 	recover(ndesc(nbuf, name, "populated"), page, MREAD_OK);
 	close(fd);
@@ -506,7 +506,7 @@ static void nonlinear(void)
 	close(fd);
 }
 
-/* 
+/*
  * These tests are currently too racy to be enabled.
  */
 
@@ -766,7 +766,7 @@ cleanup:
 		else
 			munmap_reserve(shared_page, PS);
 	}
-	if (shm_id >= 0 && shmctl(shm_id, IPC_RMID, NULL) < 0) 
+	if (shm_id >= 0 && shmctl(shm_id, IPC_RMID, NULL) < 0)
 		err("shmctl IPC_RMID");
 	if (sem_id >= 0 && semctl(sem_id, 0, IPC_RMID) < 0)
 		err("semctl IPC_RMID");
@@ -862,7 +862,7 @@ struct testcase {
 };
 
 struct testcase snipercases[] = {
-	{ under_io_dirty, "under io dirty" }, 
+	{ under_io_dirty, "under io dirty" },
 	{ under_io_clean, "under io clean" },
 };
 
@@ -877,17 +877,17 @@ void usage(void)
 
 void handle_opts(char **av)
 {
-	while (*++av) { 
-		if (!strcmp(*av, "--sniper")) { 
+	while (*++av) {
+		if (!strcmp(*av, "--sniper")) {
 			struct testcase *t;
 			for (t = cases; t->f; t++)
 				;
 			*t++ = snipercases[0];
 			*t++ = snipercases[1];
 		}
-		else if (!strcmp(*av, "--mce-inject")) { 
-			inject = inject_mce_inject;			
-		} else 
+		else if (!strcmp(*av, "--mce-inject")) {
+			inject = inject_mce_inject;
+		} else
 			usage();
 	}
 }
@@ -913,7 +913,7 @@ int main(int ac, char **av)
 	struct testcase *t;
 	/* catch signals */
 	sigaction(SIGBUS, &sa, NULL);
-	for (t = cases; t->f; t++) { 
+	for (t = cases; t->f; t++) {
 		printf("---- testing %s\n", t->name);
 		t->f();
 	}
diff --git a/tsrc/tkillpoison.c b/cases/function/hwpoison/tkillpoison.c
index ce99429..ce99429 100644
--- a/tsrc/tkillpoison.c
+++ b/cases/function/hwpoison/tkillpoison.c
diff --git a/tsrc/tprctl.c b/cases/function/hwpoison/tprctl.c
index bcc1e49..bcc1e49 100644
--- a/tsrc/tprctl.c
+++ b/cases/function/hwpoison/tprctl.c
diff --git a/tsrc/tsimpleinj.c b/cases/function/hwpoison/tsimpleinj.c
index a7029cb..a7029cb 100644
--- a/tsrc/tsimpleinj.c
+++ b/cases/function/hwpoison/tsimpleinj.c
diff --git a/tsrc/tsoft.c b/cases/function/hwpoison/tsoft.c
index 4fbaa15..4fbaa15 100644
--- a/tsrc/tsoft.c
+++ b/cases/function/hwpoison/tsoft.c
diff --git a/tsrc/tsoftinj.c b/cases/function/hwpoison/tsoftinj.c
index c2dfc6c..c2dfc6c 100644
--- a/tsrc/tsoftinj.c
+++ b/cases/function/hwpoison/tsoftinj.c
diff --git a/tsrc/ttranshuge.c b/cases/function/hwpoison/ttranshuge.c
index 6313591..6313591 100644
--- a/tsrc/ttranshuge.c
+++ b/cases/function/hwpoison/ttranshuge.c
diff --git a/tsrc/utils.h b/cases/function/hwpoison/utils.h
index 8c8ab26..8c8ab26 100644
--- a/tsrc/utils.h
+++ b/cases/function/hwpoison/utils.h
diff --git a/kvm/README b/cases/function/kvm/README
index d79fb52..d79fb52 100644
--- a/kvm/README
+++ b/cases/function/kvm/README
diff --git a/kvm/guest/guest_run.sh b/cases/function/kvm/guest/guest_run.sh
index 0f5886b..0f5886b 100644
--- a/kvm/guest/guest_run.sh
+++ b/cases/function/kvm/guest/guest_run.sh
diff --git a/kvm/host/SRAO b/cases/function/kvm/host/SRAO
index cecffa1..cecffa1 100644
--- a/kvm/host/SRAO
+++ b/cases/function/kvm/host/SRAO
diff --git a/kvm/host/guest_init b/cases/function/kvm/host/guest_init
index df34ad6..df34ad6 100644
--- a/kvm/host/guest_init
+++ b/cases/function/kvm/host/guest_init
diff --git a/kvm/host/host_run.sh b/cases/function/kvm/host/host_run.sh
index 500e275..500e275 100755..100644
--- a/kvm/host/host_run.sh
+++ b/cases/function/kvm/host/host_run.sh
diff --git a/kvm/patches/fake_ser_p.patch b/cases/function/kvm/patches/fake_ser_p.patch
index 4b22544..4b22544 100644
--- a/kvm/patches/fake_ser_p.patch
+++ b/cases/function/kvm/patches/fake_ser_p.patch
diff --git a/kvm/patches/p2v.patch b/cases/function/kvm/patches/p2v.patch
index 328212b..328212b 100644
--- a/kvm/patches/p2v.patch
+++ b/cases/function/kvm/patches/p2v.patch
diff --git a/cases/function/pfa/Makefile b/cases/function/pfa/Makefile
new file mode 100644
index 0000000..26dc574
--- /dev/null
+++ b/cases/function/pfa/Makefile
@@ -0,0 +1,10 @@
+CFLAGS := -g -Wall
+
+all: pfa busy
+pfa: pfa.o
+busy: busy.o
+
+install: all
+
+clean:
+	rm -f *.o pfa busy
diff --git a/cases/function/pfa/busy.c b/cases/function/pfa/busy.c
new file mode 100644
index 0000000..b8df1b5
--- /dev/null
+++ b/cases/function/pfa/busy.c
@@ -0,0 +1,7 @@
+#include <unistd.h>
+
+int main()
+{
+	while (1)
+		usleep(1000);
+}
diff --git a/cases/function/pfa/load.sh b/cases/function/pfa/load.sh
new file mode 100644
index 0000000..e7e9633
--- /dev/null
+++ b/cases/function/pfa/load.sh
@@ -0,0 +1,25 @@
+#!/bin/sh
+
+usage()
+{
+	cat <<-EOF
+	This script is used to add CPU load in the test procedure.
+	Please kill tese loads in the background after the tests.
+
+	usage: ${0##*/} [program to load]
+	example: ${0##*/} ./load.sh ./busy
+
+	EOF
+
+	exit 0
+}
+
+
+[ X"$1" = X ] && usage
+
+cpu=`cat /proc/cpuinfo |grep -c processor`
+cpu=`expr $cpu - 1`
+for i in `seq 0 $cpu`
+do
+	taskset -c $i $1 &
+done
diff --git a/cases/function/pfa/pfa.c b/cases/function/pfa/pfa.c
new file mode 100644
index 0000000..97dfe3b
--- /dev/null
+++ b/cases/function/pfa/pfa.c
@@ -0,0 +1,101 @@
+/*
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the Free
+ * Software Foundation; version 2.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should find a copy of v2 of the GNU General Public License somewhere on
+ * your Linux system; if not, write to the Free Software Foundation, Inc., 59
+ * Temple Place, Suite 330, Boston, MA 02111-1307 USA.
+ *
+ * Copyright (C) 2012 Intel corporation
+ */
+
+#include <stdio.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include <sys/mman.h>
+
+/*
+ * Definition of /proc/pid/pagemap
+ * Bits 0-54  page frame number (PFN) if present
+ * Bits 0-4   swap type if swapped
+ * Bits 5-54  swap offset if swapped
+ * Bits 55-60 page shift (page size = 1<<page shift)
+ * Bit  61    reserved for future use
+ * Bit  62    page swapped
+ * Bit  63    page present
+ */
+
+struct pagemaps {
+	unsigned long long	pfn:55;
+	unsigned long long	pgshift:6;
+	unsigned long long	rsvd:1;
+	unsigned long long	swapped:1;
+	unsigned long long	present:1;
+};
+
+
+/*
+ * get information about address from /proc/{pid}/pagemap
+ */
+unsigned long long vtop(unsigned long long addr)
+{
+	static int pagesize;
+	struct pagemaps pinfo;
+	unsigned int pinfo_size = sizeof pinfo;
+	long offset;
+	int fd, pgmask;
+	char pagemapname[64];
+
+	if (!pagesize)
+		pagesize = getpagesize();
+	offset = addr / pagesize * pinfo_size;
+	sprintf(pagemapname, "/proc/%d/pagemap", getpid());
+	fd = open(pagemapname, O_RDONLY);
+	if (fd == -1) {
+		perror(pagemapname);
+		return 0;
+	}
+	if (pread(fd, (void*)&pinfo, pinfo_size, offset) != pinfo_size) {
+		perror(pagemapname);
+		close(fd);
+		return 0;
+	}
+	close(fd);
+	if (!pinfo.present)
+		return ~0ull;
+	pgmask = (1 << pinfo.pgshift) - 1;
+	return (pinfo.pfn << pinfo.pgshift) | (addr & pgmask);
+}
+
+int main()
+{
+	char *p;
+	unsigned long long phys, newphys;
+
+	p = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
+	if (p == MAP_FAILED) {
+		perror("mmap");
+		return 1;
+	}
+	*p = '*'; /* make kernel allocate page */
+	phys = vtop((unsigned long long)p);
+
+	printf("allocated page: virtual = %p physical = 0x%llx\n", p, phys);
+	fflush(stdout);
+
+	for (;;) {
+		newphys = vtop((unsigned long long)p);
+		if (phys != newphys) {
+			printf("Page was replaced. New physical address = 0x%llx\n", newphys);
+			fflush(stdout);
+			phys = newphys;
+		}
+		sleep(1);
+	}
+}
diff --git a/cases/function/pfa/run_pfa.sh b/cases/function/pfa/run_pfa.sh
new file mode 100644
index 0000000..116931e
--- /dev/null
+++ b/cases/function/pfa/run_pfa.sh
@@ -0,0 +1,91 @@
+#!/bin/sh
+
+INJ_TYPE=0x00000008
+APEI_IF=""
+PFA_BIN=""
+
+invalid()
+{
+	echo $*
+	exit 1
+}
+
+check_debugfs()
+{
+	mount|grep -q debugfs
+	[ $? -eq 0 ] && return
+	mount -t debugfs none /sys/kernel/debug
+	mount|grep -q /sys/kernel/debug
+	[ $? -ne 0 ] && invalid "Kernel without debugfs support ?"
+}
+
+apei_inj()
+{
+	echo $INJ_TYPE > $APEI_IF/error_type
+	echo $1 > $APEI_IF/param1
+	echo 0xfffffffffffff000 > $APEI_IF/param2
+	echo 1 > $APEI_IF/error_inject
+}
+
+usage()
+{
+	cat <<-EOF
+	usage: ${0##*/} [PFA program] [trigger interval time]
+	example: ${0##*/} ./pfa 10
+
+	EOF
+}
+
+main()
+{
+	if [ X"$1" = X -o X"$2" = X ]
+	then
+		usage
+		exit 0
+	fi
+
+	PFA_BIN=$1
+	check_debugfs
+
+	APEI_IF=`mount | grep debugfs | cut -d ' ' -f3 | head -1`/apei/einj
+
+	#if einj is not builtin, just insmod it
+	if [ ! -d $APEI_IF ]; then
+		#if einj is a module, it is ensured to have been loaded
+		modprobe einj param_extension=1 > /dev/null 2>&1
+		[ $? -eq 0 ] || invalid "module einj isn't supported?"
+	fi
+	[ -f $APEI_IF/param1 ] ||
+	invalid "no BIOS extension support for APEI on this platform"
+
+	#check if the platform supports Correctable Memory Error injection
+	cat $APEI_IF/available_error_type | grep -q $INJ_TYPE
+	[ $? -ne 0 ] &&
+	invalid "Necessary Error Injection for PFA is not supported on this platform"
+
+	killall $PFA_BIN > /dev/null 2>&1
+	$PFA_BIN | tee log &
+	#wait to flush stdout into log
+	sleep 1
+	addr=`cat log |cut -d' '  -f8|tail -1`
+	last_addr=$addr
+	while :
+	do
+		echo inject address = $addr
+		apei_inj $addr
+		sleep $2
+		addr=`cat log |cut -d' '  -f8|tail -1`
+		if [ X"$last_addr" != X"$addr" ]
+		then
+			break
+		fi
+	done
+}
+
+cleanup()
+{
+	rm -f trigger log
+}
+
+trap "cleanup" 0
+main "$@"
diff --git a/cases/function/pfa/runtest.sh b/cases/function/pfa/runtest.sh
new file mode 100644
index 0000000..3a6155a
--- /dev/null
+++ b/cases/function/pfa/runtest.sh
@@ -0,0 +1,37 @@
+#!/bin/sh
+
+cat <<-EOF
+
+***************************************************************************
+Pay attention:
+
+This test is for memory PFA support test. PFA test will conflict with EDAC.
+Before the test EDAC related drivers must be removed from the kernel (Not
+built-in or rmmod). Moreover, PFA support need correct BIOS setting and
+mcelog setting. If you are not familiar with it, please skip this test.
+
+NOTE: CPU sleep may decrease the test efficiency. To avoid this situation,
+one can run *load.sh" by hand before the formal test!
+***************************************************************************
+
+
+EOF
+
+echo 0 > $TMP_DIR/error.$$
+
+pushd `dirname $0` > /dev/null
+./run_pfa.sh ./pfa 8
+[ $? -eq 0 ] || echo 1 > $TMP_DIR/error.$$
+killall ./pfa
+popd > /dev/null
+
+grep -q "1" $TMP_DIR/error.$$
+if [ $? -eq 0 ]
+then
+	echo "PFA test FAILS"
+	exit 1
+else
+	echo "PFA test PASSES"
+	exit 0
+fi
+
diff --git a/cases/stress/Makefile b/cases/stress/Makefile
new file mode 100644
index 0000000..3337643
--- /dev/null
+++ b/cases/stress/Makefile
@@ -0,0 +1,9 @@
+all:
+	$(MAKE) -C hwpoison
+
+install:
+	$(MAKE) -C hwpoison
+
+clean:
+	$(MAKE) -C hwpoison clean
+
diff --git a/cases/stress/hwpoison/Makefile b/cases/stress/hwpoison/Makefile
new file mode 100644
index 0000000..da1cbd8
--- /dev/null
+++ b/cases/stress/hwpoison/Makefile
@@ -0,0 +1,13 @@
+CFLAGS := -g -Wall
+LDFLAGS := -g
+
+export CFLAGS
+export LDFLAGS
+
+all:
+	mkdir -p bin
+	$(MAKE) -C tools install
+
+clean:
+	$(MAKE) -C tools clean
+	rm -rf bin log result runtest hwpoison
diff --git a/stress/README b/cases/stress/hwpoison/README
index 7523905..7523905 100644
--- a/stress/README
+++ b/cases/stress/hwpoison/README
diff --git a/stress/hwpoison.sh b/cases/stress/hwpoison/hwpoison.sh
index 106ee9b..6cd3b82 100755..100644
--- a/stress/hwpoison.sh
+++ b/cases/stress/hwpoison/hwpoison.sh
@@ -21,12 +21,18 @@
 #
 
 #set -x
-sd=$(dirname "$0")
-export ROOT=`(cd $sd/..; pwd)`
+export ROOT=`(cd ../../../; pwd)`
 
+. $ROOT/lib/functions.sh
+setup_path
 . $ROOT/lib/mce.sh
 
 DEBUG=0
+YELLOW_COLOR="\\033[0;33m"
+GREEN_COLOR="\\033[0;32m"
+RED_COLOR="\\033[0;31m"
+BLUE_COLOR="\\033[0;34m"
+RESET_COLOR="\\033[0;39m"
 
 silent_exec()
 {
@@ -54,55 +60,46 @@ silent_exec_background()
 
 _print()
 {
-	echo $* > $g_tty
+	echo -en $* > $g_tty
 }
 
 dbp()
 {
 	[ $DEBUG -ne 1 ] && return
-	_print -en "\\033[0;33m" # set font color as yellow
-	_print "[debug] $*" > $g_tty
-	echo "[debug] $*" >> $g_logfile
-	_print -en "\\033[0;39m"    # restore font color to normal
+	_print $YELLOW_COLOR
+	echo "[debug] $*" | tee -a $g_logfile
+	_print $RESET_COLOR
 }
 
 log()
 {
-	_print -en "\\033[0;33m" # set font color as yellow
-	_print "[info] $*" > $g_tty
-	echo "[info] $*" >> $g_logfile
-	_print -en "\\033[0;39m"    # restore font color to normal
+	_print $YELLOW_COLOR
+	echo "[info] $*" |tee -a $g_logfile
+	_print $RESET_COLOR
 }
 
 begin()
 {
-	_print -n "$*" > $g_tty
-	_print -en "\\033[0;32m" # set font color as green
-	_print -e "\t [start]" > $g_tty
-	echo -e "$* \t [start]" >> $g_logfile
-	_print -en "\\033[0;39m"    # restore font color to normal
+	echo -n "$*" | tee -a $g_logfile
+	_print $GREEN_COLOR
+	echo -e "\t [start]" | tee -a $g_logfile
+	_print $RESET_COLOR
 }
 
 end()
 {
-	_print -n "$*" > $g_tty
-	_print -en "\\033[0;32m" # set font color as green
-	_print -e "\t [done]" > $g_tty
-	echo -e "$* \t [done]" >> $g_logfile
-	_print -en "\\033[0;39m"    # restore font color to normal
+	echo -n "$*" | tee -a $g_logfile
+	_print $GREEN_COLOR
+	echo -e "\t [done]" | tee -a $g_logfile
+	_print $RESET_COLOR
 }
 
 err()
 {
-	_print -en "\\033[0;31m" # set font color as red
-	echo > $g_tty
-	echo "Test aborted by unexpected error!" > $g_tty
-	_print "[error] !!! $* !!!" > $g_tty
-	echo > $g_tty
-	echo "Test aborted by unexpected error!" >> $g_result 
-	echo "[error] !!! $* !!!" >> $g_result 
-	echo "[error] !!! $* !!!" >> $g_logfile 
-	_print -en "\\033[0;39m"    # restore font color to normal
+	_print $RED_COLOR
+	echo -e "\nTest aborted by unexpected error!" | tee -a $g_result
+	echo "[error] !!! $* !!!" | tee -a $g_result $g_logfile
+	_print $RESET_COLOR
 	exit 1
 }
 
@@ -113,27 +110,19 @@ die()
 
 invalid()
 {
-	_print -en "\\033[0;31m" # set font color as red
-	echo > $g_tty
-	echo "Test aborted by unexpected error!" > $g_tty
-	_print "[error] !!! $* !!!" > $g_tty
-	echo > $g_tty
-	echo "Try \`./hwposion -h\` for more information." > $g_tty
-	echo > $g_tty
-	echo "Test aborted by unexpected error!" >> $g_result 
-	echo "[error] !!! $* !!!" >> $g_result 
-	echo "[error] !!! $* !!!" >> $g_logfile 
-	_print -en "\\033[0;39m"    # restore font color to normal
+	_print $RED_COLOR
+	echo -e "\nTest aborted by unexpected error!" | tee -a $g_result
+	echo "[error] !!! $* !!!" | tee -a $g_result $g_logfile
+	echo -e "\nTry \"./hwposion -h\" for more information."
+	_print $RESET_COLOR
 	exit 1
 }
 
 result()
 {
-	_print -en "\\033[0;34m" # set font color as blue
-	_print -e "$*" > $g_tty
-	echo -e "$*" >> $g_result 
-	echo -e "$*" >> $g_logfile
-	_print -en "\\033[0;39m"    # restore font color to normal
+	_print $BLUE_COLOR
+	echo -e "$*" | tee -a $g_result $g_logfile
+	_print $RESET_COLOR
 }
 
 setup_meminfo()
@@ -233,7 +222,7 @@ setup_fs()
 check_env()
 {
 	check_debugfs
-	g_debugfs=`mount | grep debugfs | cut -d ' ' -f3`
+	g_debugfs=`mount | grep debugfs | cut -d ' ' -f3 | head -1`
 	[ -z "$g_tty" ] && invalid "$g_tty does not exist"
 	if [ $g_test -eq 0 ]; then
 		if [ $g_fstype = "nfs" -o $g_fstype = "cifs" ]; then
@@ -277,7 +266,9 @@ check_env()
 			[ $? -eq 0 ] || invalid "module apei_inj isn't supported ?"
 		fi
 	fi
-	[ -d $g_ltproot -a -f $g_ltppan ] || invalid "no ltp-pan on the machine: $g_ltppan"
+	silent_exec which $g_ltppan || invalid "no $g_ltppan tool on the system"
+	g_ltppan=`which $g_ltppan`
+	dbp "Found the tool: $g_ltppan"
 	if [ $g_runltp -eq 1 ]; then
 		[ -d $g_ltproot -a -f $g_ltproot/runltp ] || invalid "no runltp on the machine"
 	fi
@@ -289,10 +280,8 @@ setup_log()
 	mkdir -p $g_resultdir
 	rm -rf $g_logdir
 	mkdir -p $g_logdir
-	echo "# hwpoison.sh $g_parameter" > $g_logfile
-	echo "# hwpoison.sh $g_parameter" > $g_result
-	[ $g_test -eq 0 ] && clear > $g_tty
-	echo "# hwpoison.sh $g_parameter" > $g_tty
+	[ $g_test -eq 0 ] && clear
+	echo "# hwpoison.sh $g_parameter" | tee $g_logfile $g_result
 }
 
 setup_env() 
@@ -629,7 +618,7 @@ _apei_inj()
 	local type=$2
 
 	echo $type > $g_debugfs/apei/einj/error_type
-	echo "0x${pfn}000" > $g_debugfs/apei/err_inj/error_address
+	echo "0x${pfn}000" > $g_debugfs/apei/einj/param1
 	echo "1" > $g_debugfs/apei/einj/error_inject
 }
 
@@ -857,9 +846,10 @@ usage()
 	echo -e "\t-h \t\t: print this page"
 	echo
 	echo -e "device:" 
-	echo -e "\tthis is a mandatory argument. typically, it's a disk partition." 
-	echo -e "\tall temporary files will be created on this device." 
-	echo -e "\terror injector will just inject errors to the pages associated" 
+	echo -e "\tThis is a mandatory argument when -T is not used."
+	echo -e "\tTypically, it's a disk partition." 
+	echo -e "\tAll temporary files will be created on this device." 
+	echo -e "\tError injector will just inject errors to the pages associated" 
 	echo -e "\twith this device (except for the testing thru madvise syscall)." 
 	echo
 	echo -e "pagetype:"
@@ -973,7 +963,7 @@ let "g_duration=120"
 g_interval=5
 g_runltp=0
 g_ltproot="/ltp"
-g_ltppan="$g_ltproot/pan/ltp-pan"
+g_ltppan="ltp-pan"
 g_pagetool="page-types"
 g_madvise=0
 g_apei=0
@@ -1028,8 +1018,7 @@ do
 		t) g_duration=$OPTARG;;
 		i) g_interval=$OPTARG;;
 		n) g_netdev=$OPTARG;;
-		o) g_ltproot=$OPTARG
-		   g_ltppan="$g_ltproot/pan/ltp-pan";;
+		o) g_ltproot=$OPTARG;;
 		p) g_pgtype=$OPTARG;;
 		s) g_pgsize=$OPTARG;;
 		r) g_result=$OPTARG;;
diff --git a/cases/stress/hwpoison/run_hard.sh b/cases/stress/hwpoison/run_hard.sh
new file mode 100644
index 0000000..163f217
--- /dev/null
+++ b/cases/stress/hwpoison/run_hard.sh
@@ -0,0 +1,52 @@
+#! /bin/bash
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public
+# License as published by the Free Software Foundation; version
+# 2.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# General Public License for more details.
+#
+# You should find a copy of v2 of the GNU General Public License somewhere
+# on your Linux system; if not, write to the Free Software Foundation, 
+# Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA 
+#
+# Copyright (C) 2012, Intel Corp.
+#
+
+cat <<-EOF
+
+***************************************************************************
+Pay attention:
+
+This test is for HWPoison stress test. In this test case, it will
+try to inject errors via madvice instead of soft-offline. Usually
+this test needs to touch many pages and some operations will fail
+on some pages. Commonly it is OK when a few failures happen.
+The test is always considered as PASS, even if failure happens in test
+procedure. When meeting this situation, please contact experts to confirm
+whether or not it is a real error.
+***************************************************************************
+
+
+EOF
+
+echo 0 > $TMP_DIR/error.$$
+
+pushd `dirname $0` > /dev/null
+echo "run hard stress tester for 60 seconds"
+./hwpoison.sh -T -C 20 -t 60 -M
+[ $? -eq 0 ] || echo 1 > $TMP_DIR/error.$$
+popd > /dev/null
+
+grep -q "1" $TMP_DIR/error.$$
+if [ $? -eq 0 ]
+then
+	exit 1
+else
+	exit 0
+fi
+
diff --git a/cases/stress/hwpoison/run_soft.sh b/cases/stress/hwpoison/run_soft.sh
new file mode 100644
index 0000000..8e86be1
--- /dev/null
+++ b/cases/stress/hwpoison/run_soft.sh
@@ -0,0 +1,51 @@
+#! /bin/bash
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public
+# License as published by the Free Software Foundation; version
+# 2.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# General Public License for more details.
+#
+# You should find a copy of v2 of the GNU General Public License somewhere
+# on your Linux system; if not, write to the Free Software Foundation, 
+# Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA 
+#
+# Copyright (C) 2012, Intel Corp.
+#
+
+cat <<-EOF
+
+***************************************************************************
+Pay attention:
+
+This test is for HWPoison stress test. In this test case, it will
+try to inject errors via soft-offline instead of madvice. Usually
+this test needs to touch many pages and no failure happens on these pages.
+The test is always considered as PASS, even if failure happens in test
+procedure. When meeting this situation, please contact experts to confirm
+whether or not it is a real error.
+***************************************************************************
+
+
+EOF
+
+echo 0 > $TMP_DIR/error.$$
+
+pushd `dirname $0` > /dev/null
+echo "run soft stress tester for 60 seconds"
+./hwpoison.sh -T -C 1 -t 60 -S
+[ $? -eq 0 ] || echo 1 > $TMP_DIR/error.$$
+popd > /dev/null
+
+grep -q "1" $TMP_DIR/error.$$
+if [ $? -eq 0 ]
+then
+	exit 1
+else
+	exit 0
+fi
+
diff --git a/stress/tools/Makefile b/cases/stress/hwpoison/tools/Makefile
index ef78a90..ef78a90 100644
--- a/stress/tools/Makefile
+++ b/cases/stress/hwpoison/tools/Makefile
diff --git a/stress/tools/fs-metadata/Makefile b/cases/stress/hwpoison/tools/fs-metadata/Makefile
index 3a88dfb..3a88dfb 100644
--- a/stress/tools/fs-metadata/Makefile
+++ b/cases/stress/hwpoison/tools/fs-metadata/Makefile
diff --git a/stress/tools/fs-metadata/README b/cases/stress/hwpoison/tools/fs-metadata/README
index 9d20a9d..9d20a9d 100644
--- a/stress/tools/fs-metadata/README
+++ b/cases/stress/hwpoison/tools/fs-metadata/README
diff --git a/stress/tools/fs-metadata/fs-metadata.sh b/cases/stress/hwpoison/tools/fs-metadata/fs-metadata.sh
index aac5adb..aac5adb 100755..100644
--- a/stress/tools/fs-metadata/fs-metadata.sh
+++ b/cases/stress/hwpoison/tools/fs-metadata/fs-metadata.sh
diff --git a/stress/tools/fs-metadata/k-thread.sh b/cases/stress/hwpoison/tools/fs-metadata/k-thread.sh
index 75a29e4..75a29e4 100755..100644
--- a/stress/tools/fs-metadata/k-thread.sh
+++ b/cases/stress/hwpoison/tools/fs-metadata/k-thread.sh
diff --git a/stress/tools/fs-metadata/k-tree-gen.c b/cases/stress/hwpoison/tools/fs-metadata/k-tree-gen.c
index 4f0e83a..4f0e83a 100644
--- a/stress/tools/fs-metadata/k-tree-gen.c
+++ b/cases/stress/hwpoison/tools/fs-metadata/k-tree-gen.c
diff --git a/stress/tools/fs-metadata/k-tree-trav.c b/cases/stress/hwpoison/tools/fs-metadata/k-tree-trav.c
index 85b12ab..85b12ab 100644
--- a/stress/tools/fs-metadata/k-tree-trav.c
+++ b/cases/stress/hwpoison/tools/fs-metadata/k-tree-trav.c
diff --git a/stress/tools/page-poisoning/Makefile b/cases/stress/hwpoison/tools/page-poisoning/Makefile
index ef93ade..ef93ade 100644
--- a/stress/tools/page-poisoning/Makefile
+++ b/cases/stress/hwpoison/tools/page-poisoning/Makefile
diff --git a/stress/tools/page-poisoning/README b/cases/stress/hwpoison/tools/page-poisoning/README
index 58c11d7..58c11d7 100644
--- a/stress/tools/page-poisoning/README
+++ b/cases/stress/hwpoison/tools/page-poisoning/README
diff --git a/stress/tools/page-poisoning/page-poisoning.c b/cases/stress/hwpoison/tools/page-poisoning/page-poisoning.c
index 0b392ac..0b392ac 100644
--- a/stress/tools/page-poisoning/page-poisoning.c
+++ b/cases/stress/hwpoison/tools/page-poisoning/page-poisoning.c
diff --git a/config/kdump_noser.conf b/config/kdump_noser.conf
deleted file mode 100644
index e44a135..0000000
--- a/config/kdump_noser.conf
+++ /dev/null
@@ -1,6 +0,0 @@
-
-CASES="soft-inj/panic/cases.sh soft-inj/panic_npcc/cases.sh soft-inj/panic_noser/cases.sh"
-COREDIR="/var/crash"
-GCOV=1
-KSRC_DIR=/lib/modules/$(uname -r)/build
-VMLINUX=$KSRC_DIR/vmlinux
diff --git a/config/kdump_ser.conf b/config/kdump_ser.conf
deleted file mode 100644
index 71182a6..0000000
--- a/config/kdump_ser.conf
+++ /dev/null
@@ -1,6 +0,0 @@
-
-CASES="soft-inj/panic/cases.sh soft-inj/panic_npcc/cases.sh soft-inj/panic_ucr/cases.sh"
-COREDIR="/var/crash"
-GCOV=1
-KSRC_DIR=/lib/modules/$(uname -r)/build
-VMLINUX=$KSRC_DIR/vmlinux
diff --git a/config/simple.conf b/config/simple.conf
deleted file mode 100644
index be6d2f1..0000000
--- a/config/simple.conf
+++ /dev/null
@@ -1,4 +0,0 @@
-
-CASES="soft-inj/non-panic/cases.sh soft-inj/panic/cases.sh soft-inj/panic_npcc/cases.sh"
-GCOV=1
-KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/config/simple_nopanic_noser.conf b/config/simple_nopanic_noser.conf
deleted file mode 100644
index 835d9e6..0000000
--- a/config/simple_nopanic_noser.conf
+++ /dev/null
@@ -1,4 +0,0 @@
-
-CASES="soft-inj/non-panic/cases.sh soft-inj/poll_noser/cases.sh"
-GCOV=1
-KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/config/simple_nopanic_ser.conf b/config/simple_nopanic_ser.conf
deleted file mode 100644
index a82cdb7..0000000
--- a/config/simple_nopanic_ser.conf
+++ /dev/null
@@ -1,4 +0,0 @@
-
-CASES="soft-inj/non-panic/cases.sh soft-inj/poll_ucr/cases.sh soft-inj/recoverable_ucr/cases.sh"
-GCOV=1
-KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/config/simple_noser.conf b/config/simple_noser.conf
deleted file mode 100644
index b562070..0000000
--- a/config/simple_noser.conf
+++ /dev/null
@@ -1,4 +0,0 @@
-
-CASES="soft-inj/non-panic/cases.sh soft-inj/panic/cases.sh soft-inj/panic_npcc/cases.sh soft-inj/panic_noser/cases.sh soft-inj/poll_noser/cases.sh"
-GCOV=1
-KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/config/simple_panic_noser.conf b/config/simple_panic_noser.conf
deleted file mode 100644
index 6694b31..0000000
--- a/config/simple_panic_noser.conf
+++ /dev/null
@@ -1,4 +0,0 @@
-
-CASES="soft-inj/panic_noser/cases.sh"
-GCOV=1
-KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/config/simple_poll_noser.conf b/config/simple_poll_noser.conf
deleted file mode 100644
index 6b7eb79..0000000
--- a/config/simple_poll_noser.conf
+++ /dev/null
@@ -1,4 +0,0 @@
-
-CASES="soft-inj/poll_noser/cases.sh"
-GCOV=1
-KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/config/simple_recoverable_ucr.conf b/config/simple_recoverable_ucr.conf
deleted file mode 100644
index d140845..0000000
--- a/config/simple_recoverable_ucr.conf
+++ /dev/null
@@ -1,4 +0,0 @@
-
-CASES="soft-inj/recoverable_ucr/cases.sh"
-GCOV=1
-KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/config/simple_ser.conf b/config/simple_ser.conf
deleted file mode 100644
index b6a3bf2..0000000
--- a/config/simple_ser.conf
+++ /dev/null
@@ -1,4 +0,0 @@
-
-CASES="soft-inj/non-panic/cases.sh soft-inj/panic/cases.sh soft-inj/panic_npcc/cases.sh soft-inj/poll_ucr/cases.sh soft-inj/panic_ucr/cases.sh soft-inj/recoverable_ucr/cases.sh"
-GCOV=1
-KSRC_DIR=/lib/modules/$(uname -r)/build
diff --git a/doc/kvm b/doc/kvm
new file mode 100644
index 0000000..4e66b48
--- /dev/null
+++ b/doc/kvm
@@ -0,0 +1,2 @@
+SRAO test for KVM can' automatically run, please follow the instructions
+in cases/function/kvm/README to do SRAO test for KVM.
diff --git a/groups/coverage b/groups/coverage
new file mode 100644
index 0000000..ef51a95
--- /dev/null
+++ b/groups/coverage
@@ -0,0 +1,5 @@
+#Coverage test is out-of-date, it is not guaranteed
+#working well with newer kernel
+
+SIMPLE cases/coverage/soft-inj/run_simple.sh off
+#KDUMP cases/coverage/soft-inj/run_kdump.sh off
diff --git a/groups/function b/groups/function
new file mode 100644
index 0000000..1361121
--- /dev/null
+++ b/groups/function
@@ -0,0 +1,10 @@
+APEI-INJ cases/function/apei-inj/runtest.sh on
+ERST-INJ cases/function/erst-inject/runtest.sh on
+#PFA test depends on correct BIOS/mcelog setting
+PFA cases/function/pfa/runtest.sh on
+#HWpoison
+HWPOISON-SOFT cases/function/hwpoison/run_soft.sh on
+HWPOISON-HARD cases/function/hwpoison/run_hard.sh on
+HWPOISON-HUGEPAGE cases/function/hwpoison/run_hugepage.sh on
+HWPOISON-THP cases/function/hwpoison/run_thp.sh on
+#KVM
diff --git a/groups/stress b/groups/stress
new file mode 100644
index 0000000..de85f14
--- /dev/null
+++ b/groups/stress
@@ -0,0 +1,2 @@
+STRESS-HWPOISON-SOFT cases/stress/hwpoison/run_soft.sh on
+STRESS-HWPOISON-HARD cases/stress/hwpoison/run_hard.sh on
diff --git a/hwpoison b/hwpoison
deleted file mode 120000
index 1a95255..0000000
--- a/hwpoison
+++ /dev/null
@@ -1 +0,0 @@
-tsrc/
\ No newline at end of file
diff --git a/lib/apei-inject.sh b/lib/apei-inject.sh
deleted file mode 100644
index 73fae28..0000000
--- a/lib/apei-inject.sh
+++ /dev/null
@@ -1,180 +0,0 @@
-#
-# APEI library: APEI specific functions
-#
-# Copyright (C) 2008, Intel Corp.
-#   Author: Huang Ying <ying.huang@intel.com>
-#           Zheng Jiajia <jiajia.zheng@intel.com>
-# This file is released under the GPLv2.
-#
-
-. $ROOT/lib/soft-inject.sh
-
-if [ -n "$this_case" ]; then
-    bcase=$(basename $this_case)
-fi
-mcelog_result=$RDIR/$this_case/mcelog
-klog=$RDIR/$this_case/klog
-
-apei_mce_reformat()
-{
-    local inf="$1"
-    local outf="$2"
-    local tmpf=$WDIR/mce_reformat_for_cmp
-    sed "s/0x//g" $inf | grep -v 'STATUS 0x0' | \
-        grep -v 'STATUS 0x800000000000000' | sort > "$tmpf"
-    awk '/^STATUS/ {print $2}' $tmpf | cut -b 1-3 > $outf
-    awk '/MCGSTATUS/ {if ($4=="") print $2; else print $4;}' $tmpf >> $outf
-}
-
-apei_mce_cmp()
-{
-    [ $# -eq 2 ] || die "missing parameter for mce_cmp"
-    local m1="$1"
-    local m2="$2"
-    local tmpf1=$WDIR/mce_cmp_1
-    local tmpf2=$WDIR/mce_cmp_2
-
-    apei_mce_reformat "$m1" $tmpf1 
-    apei_mce_reformat "$m2" $tmpf2 
-    diff $tmpf1 $tmpf2 > /dev/null
-}
-
-apei_inject_verify_mcelog()
-{
-    if [ -f $RDIR/$this_case/mcelog ]; then
-        mcelog_refer=$SDIR/refer/$bcase
-        mce-inject --dump $mcelog_refer > $RDIR/$this_case/mcelog_refer
-        if apei_mce_cmp $RDIR/$this_case/mcelog $RDIR/$this_case/mcelog_refer; then 
-            echo "  Passed: MCE log is ok"
-        else
-            echo "  Failed: MCE log is different from input"
-        fi
-    else
-        echo "  Failed: no MCE log result"
-    fi
-}
-
-
-apei_inject_get_klog()
-{
-    soft_inject_get_klog
-}
-
-apei_inject_get_mcelog()
-{
-    soft_inject_get_mcelog
-}
-
-# verify return value
-apei_inject_verify_return_val()
-{
-    soft_inject_verify_return_val
-}
-
-apei_inject_verify_kill()
-{
-    soft_inject_verify_kill
-}
-
-apei_inject_enumerate()
-{
-    soft_inject_enumerate
-}
-
-apei_inject_trigger()
-{
-    check_debugfs
-    #APEI_IF should be defined after debugfs is mounted
-    APEI_IF=`mount | grep debugfs | cut -d ' ' -f3`/apei/einj
-
-    #if einj is a module, it is ensured to have been loaded
-    modinfo einj > /dev/null 2>&1
-    if [ $? -eq 0 ]; then
-	[ -d $APEI_IF ] || modprobe einj param_extension=1
-        [ $? -eq 0 ] || die "module einj isn't supported ?"
-    fi
-
-    mcelog &> /dev/null
-    TYPE=`awk '/^TYPE/{print $2}' $SDIR/data/$bcase`
-    echo $TYPE > $APEI_IF/error_type
-    killall simple_process
-    $TDIR/simple_process/simple_process > /dev/null &
-
-    page-types -p `pidof simple_process` -LN -b ano > $RDIR/$this_case/page
-
-    ADDR=`awk '$2 != "offset" {print "0x"$2"000"}' $RDIR/$this_case/page | sed -n -e '1p'`
-    echo $ADDR > $APEI_IF/param1
-
-    echo "1" > $APEI_IF/error_inject
-
-    ret=$?
-    echo $ret > $RDIR/$this_case/return
-    sleep 1
-}
-
-start_tracing()
-{
-    [ $# -eq 1 ] || die "missing parameter for get_panic_from_mcelog: please set filter for ftrace"
-    [ -d /sys/kernel/debug/tracing ] || die "no tracing"
-    echo "function_graph" > /sys/kernel/debug/tracing/current_tracer
-    echo $1 > /sys/kernel/debug/tracing/set_ftrace_filter
-    echo "1" > /sys/kernel/debug/tracing/tracing_enabled
-}
-
-stop_tracing()
-{
-    [ -d /sys/kernel/debug/tracing ] || die "no tracing"
-    echo "0" > /sys/kernel/debug/tracing/tracing_enabled
-    cp /sys/kernel/debug/tracing/trace $RDIR/$this_case/
-    echo "nop" > /sys/kernel/debug/tracing/current_tracer
-}
-
-apei_inject_verify_trace()
-{
-    [ $# -eq 1 ] || die "missing parameter for apei_inject_verify_trace"
-    if grep "$1" $RDIR/$this_case/trace; then
-       echo "Passed: trace is correct"
-    else
-       echo "Failed: Nothing is traced"
-    fi
-}
-
-apei_inject_verify_panic()
-{
-    local mce_panic="$1"
-    verify_panic_via_klog $klog "$mce_panic"
-}
-
-apei_inject_verify_exp()
-{
-    verify_exp_via_klog $klog "$@"
-}
-
-apei_inject_verify_fail()
-{
-    verify_fail_via_klog $klog "$@"
-}
-
-apei_inject_main()
-{
-    op="$1"
-    shift
-
-    case "$op" in
-	enumerate)
-	    enumerate
-	    ;;
-	trigger)
-	    trigger "$@"
-	    ;;
-	get_result)
-	    get_result
-	    ;;
-	verify)
-	    verify
-	    ;;
-	*)
-	    die "Usage: $0 enumerate|trigger|get_result|verify"
-    esac
-    exit 0
-}
diff --git a/lib/dirs.sh b/lib/dirs.sh
index 9ca42b2..140f28f 100644
--- a/lib/dirs.sh
+++ b/lib/dirs.sh
@@ -1,22 +1,16 @@
 #
 # Setup environment variable for various directories.
 #
-# Copyright (C) 2008, Intel Corp.
+# Copyright (C) 2008-2012 Intel Corp.
 #   Author: Huang Ying <ying.huang@intel.com>
 #
 # This file is released under the GPLv2.
 #
 
-RCDIR=cases
-RCONF_DIR=config
-RRDIR=results
-RWDIR=work
-RLDIR=lib
-
 SDIR=$(script_dir)
-CDIR=$ROOT/cases
-CONF_DIR=$ROOT/config
-RDIR=$ROOT/results/$driver
-WDIR=$ROOT/work/$driver
+CDIR=$ROOT/cases/coverage/soft-inj
+CONF_DIR=$SDIR/config
+RDIR=$LOG_DIR/$driver
+WDIR=$TMP_DIR/$driver
 LDIR=$ROOT/lib
-TDIR=$ROOT/tools
+TDIR=$TOOL_DIR
diff --git a/lib/functions.sh b/lib/functions.sh
index 1abccfc..2bd1db4 100644
--- a/lib/functions.sh
+++ b/lib/functions.sh
@@ -9,82 +9,82 @@
 
 setup_path()
 {
-    export PATH=$ROOT/bin:$PATH
+	export PATH=$ROOT/bin:$PATH
 }
 
 script_dir()
 {
-    local rd=$(dirname "$0")
-    (cd $rd; pwd)
+	local rd=$(dirname "$0")
+	(cd $rd; pwd)
 }
 
 relative_path()
 {
-    local len1=${#1}
-    local len2=${#2}
-    if [ $len1 -eq 0 -o $len1 -ge $len2 -o "${2:0:$len1}" != "$1" ]; then
-	die "$2 is not the sub-path of $1!"
-    fi
-    len1=$((len1 + 1))
-    echo "${2:$len1}"
+	local len1=${#1}
+	local len2=${#2}
+	if [ $len1 -eq 0 -o $len1 -gt $len2 -o "${2:0:$len1}" != "$1" ]; then
+		die "$2 is not the sub-path of $1!"
+	fi
+	len1=$((len1 + 1))
+	echo "${2:$len1}"
 }
 
 die()
 {
-    echo "DIE: $@"
-    echo "DIE: $@" 1>&2
-    exit -1
+	echo "DIE: $@"
+	echo "DIE: $@" 1>&2
+	exit -1
 }
 
 driver_prepare()
 {
-    mkdir -p $WDIR/stamps
+	mkdir -p $WDIR/stamps
 }
 
 check_kern_warning_bug()
 {
-    local f="$1"
-    [ -n "$f" ] || die "missing parameter for check_kern_warning"
-    grep -e '----\[ cut here \]---' $f > /dev/null || \
+	local f="$1"
+	[ -n "$f" ] || die "missing parameter for check_kern_warning"
+	grep -e '----\[ cut here \]---' $f > /dev/null || \
 	grep -e 'BUG:' $f > /dev/null
 }
 
 random_sleep()
 {
-    local s=$((RANDOM / 13107 + 5))
-    sleep $s
+	local s=$((RANDOM / 13107 + 5))
+	sleep $s
 }
 
 start_background()
 {
-    if [ -n "$BACKGROUND" ]; then
-	pid_background=$(bash -i -c "$BACKGROUND &>$WDIR/background_log & echo \$!")
-	if ! ps -p $pid_background > /dev/null; then
-	    die "Failed to start background testing: $BACKGROUND"
+	if [ -n "$BACKGROUND" ]; then
+		pid_background=$(bash -i -c "$BACKGROUND &>$WDIR/background_log & echo \$!")
+		if ! ps -p $pid_background > /dev/null; then
+			die "Failed to start background testing: $BACKGROUND"
+		fi
 	fi
-    fi
 }
 
 stop_background()
 {
-    if [ -n "$pid_background" ]; then
-	if ! kill -TERM -$pid_background &> /dev/null; then
-	    kill $pid_background || true
+	if [ -n "$pid_background" ]; then
+		if ! kill -TERM -$pid_background &> /dev/null; then
+			kill $pid_background || true
+		fi
 	fi
-    fi
 }
 
 filter_fake_panic()
 {
-    local orig_klog=$1
-    local new_klog=$2
-    [ $# -eq 2 ] || die "missing parameter for filter_fake_panic"
+	local orig_klog=$1
+	local new_klog=$2
+	[ $# -eq 2 ] || die "missing parameter for filter_fake_panic"
 
-    local pn
-    pn=$(grep -n "Fake kernel panic" $orig_klog | cut -d ':' -f 1 | head -1)
-    if [ -z "$pn" ]; then
-	cp $orig_klog $new_klog
-    else
-	sed -n "1,${pn}p" < $orig_klog > $new_klog
-    fi
+	local pn
+	pn=$(grep -n "Fake kernel panic" $orig_klog | cut -d ':' -f 1 | head -1)
+	if [ -z "$pn" ]; then
+		cp $orig_klog $new_klog
+	else
+		sed -n "1,${pn}p" < $orig_klog > $new_klog
+	fi
 }
diff --git a/lib/mce.sh b/lib/mce.sh
index 39eb3a8..464f369 100644
--- a/lib/mce.sh
+++ b/lib/mce.sh
@@ -9,320 +9,321 @@
 
 extract_mce_from_log()
 {
-    [ $# -eq 2 ] || die "missing parameter for extract_mce_from_log"
-    local log="$1"
-    local outf="$2"
+	[ $# -eq 2 ] || die "missing parameter for extract_mce_from_log"
+	local log="$1"
+	local outf="$2"
 
-    sed '1,/HARDWARE ERROR/d' "$log" | \
+	sed '1,/HARDWARE ERROR/d' "$log" | \
 	mcelog --no-dmi --dump-raw-ascii --ascii > "$outf"
 }
 
 mce_reformat()
 {
-    [ $# -eq 2 ] || die "missing parameter for mce_reformat"
-    local org="$1"
-    local outf="$2"
+	[ $# -eq 2 ] || die "missing parameter for mce_reformat"
+	local org="$1"
+	local outf="$2"
 
-    mce-inject --dump "$org" > "$outf"
+	mce-inject --dump "$org" > "$outf"
 }
 
 mce_reformat_for_cmp()
 {
-    local inf="$1"
-    local outf="$2"
-    local removes="$3"
+	local inf="$1"
+	local outf="$2"
+	local removes="$3"
 
-    local tmpf=$WDIR/mce_reformat_for_cmp
+	local tmpf=$WDIR/mce_reformat_for_cmp
 
-    mce-inject --dump "$inf" > $tmpf
+	mce-inject --dump "$inf" > $tmpf
 
-    if [ -n "$removes" ]; then
-	for remove in $removes; do
-	    sed "/$remove/d" -i $tmpf
-	done
-    fi
+	if [ -n "$removes" ]; then
+		for remove in $removes; do
+			sed "/$remove/d" -i $tmpf
+		done
+	fi
 
-    cat $tmpf | tr '\n' '#' | sed '1,$s/##/\n/g' | \
+	cat $tmpf | tr '\n' '#' | sed '1,$s/##/\n/g' | \
 	grep -v '#STATUS 0x0#' | \
 	grep -v '#STATUS 0x800000000000000#' | sort > "$outf"
 }
 
 mce_cmp()
 {
-    [ $# -eq 3 ] || die "missing parameter for mce_cmp"
-    local m1="$1"
-    local m2="$2"
-    local removes="$3"
+	[ $# -eq 3 ] || die "missing parameter for mce_cmp"
+	local m1="$1"
+	local m2="$2"
+	local removes="$3"
 
-    local tmpf1=$WDIR/mce_cmp_1
-    local tmpf2=$WDIR/mce_cmp_2
+	local tmpf1=$WDIR/mce_cmp_1
+	local tmpf2=$WDIR/mce_cmp_2
 
-    mce_reformat_for_cmp "$m1" $tmpf1 "$removes"
-    mce_reformat_for_cmp "$m2" $tmpf2 "$removes"
-    diff $tmpf1 $tmpf2 > /dev/null
+	mce_reformat_for_cmp "$m1" $tmpf1 "$removes"
+	mce_reformat_for_cmp "$m2" $tmpf2 "$removes"
+	diff $tmpf1 $tmpf2 > /dev/null
 }
 
 get_mcelog_from_dev()
 {
-    [ $# -eq 1 ] || die "missing parameter for get_mcelog_from_dev"
-    local mcelog_result="$1"
-    if mcelog --dump-raw-ascii > "$mcelog_result"; then
-	true
-    else
-	echo "  Failed: can not get mce log from /dev/mcelog"
-    fi
+	[ $# -eq 1 ] || die "missing parameter for get_mcelog_from_dev"
+	local mcelog_result="$1"
+	if mcelog --dump-raw-ascii > "$mcelog_result"; then
+		true
+	else
+		echo "  Failed: can not get mce log from /dev/mcelog"
+	fi
 }
 
 # extract mcelog from kernel log
 get_mcelog_from_klog()
 {
-    [ $# -eq 2 ] || die "missing parameter for get_mcelog_from_klog"
-    local klog="$1"
-    local mcelog_result="$2"
-    if [ -f "$klog" ] && extract_mce_from_log "$klog" "$mcelog_result"; then
-	true
-    else
-	echo "  Failed: Can not extract mcelog from console log"
-    fi
+	[ $# -eq 2 ] || die "missing parameter for get_mcelog_from_klog"
+	local klog="$1"
+	local mcelog_result="$2"
+	if [ -f "$klog" ] && extract_mce_from_log "$klog" "$mcelog_result"; then
+		true
+	else
+		echo "  Failed: Can not extract mcelog from console log"
+	fi
 }
 
 mcelog_filter()
 {
-    [ $# -eq 2 ] || die "missing parameter for mcelog_filter"
-    local inf="$1"
-    local pat="$2"
+	[ $# -eq 2 ] || die "missing parameter for mcelog_filter"
+	local inf="$1"
+	local pat="$2"
 
-    mce-inject --dump "$inf" | tr '\n' '#' | sed '1,$s/##/\n/g' | \
+	mce-inject --dump "$inf" | tr '\n' '#' | sed '1,$s/##/\n/g' | \
 	grep -e "$pat"
 }
 
 chk_gcov()
 {
-    if [ -z "$GCOV" ]; then
-	return 1
-    fi
-
-    if [ -f /sys/kernel/debug/gcov/reset ] && which gcov > /dev/null; then
-	return 0
-    else
-	return 1
-    fi
+	if [ -z "$GCOV" ]; then
+		return 1
+	fi
+
+	if [ -f /sys/kernel/debug/gcov/reset ] && which gcov > /dev/null; then
+		return 0
+	else
+		return 1
+	fi
 }
 
 reset_gcov()
 {
-    if [ -z "$GCOV" ]; then
-	return
-    fi
-    case $GCOV in
-	copy)
-	    echo 1 > /sys/kernel/debug/gcov/reset
-	    ;;
-	dump)
-	    true;
-	    ;;
-	*)
-	    echo "  Failed: can not reset gcov, invalid GCOV=$GCOV"
-	    return
-	    ;;
-    esac
+	if [ -z "$GCOV" ]; then
+		return
+	fi
+	case $GCOV in
+		copy)
+		echo 1 > /sys/kernel/debug/gcov/reset
+		;;
+		dump)
+		true;
+		;;
+		*)
+		echo "  Failed: can not reset gcov, invalid GCOV=$GCOV"
+		return
+		;;
+	esac
 }
 
 get_gcov()
 {
-    [ $# -eq 1 ] || die "missing parameter for get_gcov"
-    local src_path=$1
-    local src_fn=$(basename $src_path)
-    local src_dir=$(dirname $src_path)
-    if [ -z "$GCOV" ]; then
-	return
-    fi
-    local abs_dir=$(cd -P $KSRC_DIR/$src_dir; pwd)
-    case $GCOV in
-	copy)
-	    for f in /sys/kernel/debug/gcov/$abs_dir/*.gc*; do
-		bf=$(basename $f)
-		cat $f > $abs_dir/$bf
-	    done
-	    ;;
-	dump)
-	    true
-	    ;;
-	*)
-	    echo "  Failed: can not get gcov path, invalid GCOV=$GCOV"
-	    return
-	    ;;
-    esac
-    if ! (cd $KSRC_DIR; gcov -o $src_dir $src_fn &> /dev/null) || \
-	! [ -s $KSRC_DIR/$src_fn.gcov ]; then
-	echo "  Failed: can not get gcov graph"
-	return
-    fi
-    cp $KSRC_DIR/$src_fn.gcov $RDIR/$this_case
+	[ $# -eq 1 ] || die "missing parameter for get_gcov"
+	local src_path=$1
+	local src_fn=$(basename $src_path)
+	local src_dir=$(dirname $src_path)
+	if [ -z "$GCOV" ]; then
+		return
+	fi
+	local abs_dir=$(cd -P $KSRC_DIR/$src_dir; pwd)
+	case $GCOV in
+		copy)
+		for f in /sys/kernel/debug/gcov/$abs_dir/*.gc*; do
+			bf=$(basename $f)
+			cat $f > $abs_dir/$bf
+		done
+		;;
+		dump)
+		true
+		;;
+		*)
+		echo "  Failed: can not get gcov path, invalid GCOV=$GCOV"
+		return
+		;;
+	esac
+	if ! (cd $KSRC_DIR; gcov -o $src_dir $src_fn &> /dev/null) || \
+		! [ -s $KSRC_DIR/$src_fn.gcov ]; then
+		echo "  Failed: can not get gcov graph"
+		return
+	fi
+	cp $KSRC_DIR/$src_fn.gcov $RDIR/$this_case
 }
 
 reset_severity_cov()
 {
-    echo 1 > /sys/kernel/debug/mce/severities-coverage
+	echo 1 > /sys/kernel/debug/mce/severities-coverage
 }
 
 get_severity_cov()
 {
-    local sev_cor=/sys/kernel/debug/mce/severities-coverage
-    if [ ! -f $sev_cor ]; then
-	echo "  Failed: can not get severities_coverage"
-	return
-    fi
-    cp $sev_cor $RDIR/$this_case
+	local sev_cor=/sys/kernel/debug/mce/severities-coverage
+	if [ ! -f $sev_cor ]; then
+		echo "  Failed: can not get severities_coverage"
+		return
+	fi
+	cp $sev_cor $RDIR/$this_case
 }
 
 verify_klog()
 {
-    [ $# -eq 1 ] || die "missing parameter for verify_klog"
-    local klog="$1"
-    if [ -f "$klog" ]; then
-	if check_kern_warning_bug "$klog"; then
-	    echo "  Failed: kernel warning or bug during MCE"
+	[ $# -eq 1 ] || die "missing parameter for verify_klog"
+	local klog="$1"
+	if [ -f "$klog" ]; then
+		if check_kern_warning_bug "$klog"; then
+			echo "  Failed: kernel warning or bug during MCE"
+		else
+			echo "  Passed: No kernel warning or bug"
+		fi
 	else
-	    echo "  Passed: No kernel warning or bug"
+		echo "  Failed: no kernel log"
 	fi
-    else
-	echo "  Failed: no kernel log"
-    fi
 }
 
 verify_panic_via_klog()
 {
-    [ $# -eq 2 ] || die "missing parameter for verify_panic"
-    local klog="$1"
-    local mce_panic="$2"
-    if [ ! -f "$klog" ]; then
-	echo "  Failed: no kernel log for checking panic"
-	return -1
-    fi
-
-    if grep "panic" "$klog" | grep "$mce_panic" > /dev/null; then
-	echo "  Passed: correct panic"
-    else
-	echo "  Failed: uncorrect panic, expected: $mce_panic"
-    fi
+	[ $# -eq 2 ] || die "missing parameter for verify_panic"
+	local klog="$1"
+	local mce_panic="$2"
+	if [ ! -f "$klog" ]; then
+		echo "  Failed: no kernel log for checking panic"
+		return -1
+	fi
+
+	if grep "panic" "$klog" | grep "$mce_panic" > /dev/null; then
+		echo "  Passed: correct panic"
+	else
+		echo "  Failed: uncorrect panic, expected: $mce_panic"
+	fi
 }
 
 verify_timeout_via_klog()
 {
-    [ $# -eq 1 ] || die "missing parameter for verify_timeout"
-    local klog="$1"
-    if [ ! -f "$klog" ]; then
-	echo "  Failed: No kernel log for checking timeout"
-	return -1
-    fi
-
-    if grep "Some CPUs didn't answer in synchronization" "$klog" \
-	> /dev/null; then
-	echo "  Passed: timeout detected"
-    else
-	echo "  Failed: no timeout detected"
-    fi
+	[ $# -eq 1 ] || die "missing parameter for verify_timeout"
+	local klog="$1"
+	if [ ! -f "$klog" ]; then
+		echo "  Failed: No kernel log for checking timeout"
+		return -1
+	fi
+
+	if grep "Some CPUs didn't answer in synchronization" "$klog" \
+		> /dev/null; then
+		echo "  Passed: timeout detected"
+	else
+		echo "  Failed: no timeout detected"
+	fi
 }
 
 verify_exp_via_klog()
 {
-    [ $# -ge 2 ] || die "missing parameter for verrify_exp_via_klog"
-    local klog="$1"
-    shift
-    if [ ! -f "$klog" ]; then
-	echo "  Failed: No kernel log for checking MCE exp"
-	return -1
-    fi
-
-    for exp in "$@"; do
-	if grep "Machine check: " "$klog" | grep "$exp" > /dev/null; then
-	    echo "  Passed: correct MCE exp"
-	    return
+	[ $# -ge 2 ] || die "missing parameter for verrify_exp_via_klog"
+	local klog="$1"
+	shift
+	if [ ! -f "$klog" ]; then
+		echo "  Failed: No kernel log for checking MCE exp"
+		return -1
 	fi
-    done
-    echo "  Failed:  uncorrected MCE exp, expected: $exp"
+
+	for exp in "$@"; do
+		if grep "Machine check: " "$klog" | grep "$exp" > /dev/null; then
+			echo "  Passed: correct MCE exp"
+			return
+		fi
+	done
+	echo "  Failed:  uncorrected MCE exp, expected: $exp"
 }
 
 get_panic_from_mcelog()
 {
-    [ $# -eq 1 ] || die "missing parameter for get_panic_from_mcelog"
-    local mcelog="$1"
-    local tmpf=$WDIR/get_panic_from_mcelog
-    local addr
-    if mcelog_filter $mcelog "#BANK 219#" | head -1 > $tmpf; then
-	local F="$(sed '1,$s/#/\n/g' $tmpf | awk '/MISC / { print $2 }')"
-	case "$F" in
-	    0x1) echo "Fatal machine check" ;;
-	    0x2) echo "Machine check from unknown source" ;;
-	    0x3) echo "Uncorrected data corruption machine check" ;;
-	    0x4) echo "Fatal machine check" ;;
-	    *) echo unknown panic $F ;;
-	esac
-    fi
+	[ $# -eq 1 ] || die "missing parameter for get_panic_from_mcelog"
+	local mcelog="$1"
+	local tmpf=$WDIR/get_panic_from_mcelog
+	local addr
+	if mcelog_filter $mcelog "#BANK 219#" | head -1 > $tmpf; then
+		local F="$(sed '1,$s/#/\n/g' $tmpf | awk '/MISC / { print $2 }')"
+		case "$F" in
+			0x1) echo "Fatal machine check" ;;
+			0x2) echo "Machine check from unknown source" ;;
+			0x3) echo "Uncorrected data corruption machine check" ;;
+			0x4) echo "Fatal machine check" ;;
+			*) echo unknown panic $F ;;
+		esac
+	fi
 }
 
 verify_panic_msg()
 {
-    [ $# -eq 2 ] || die "missing parameter for verify_panic_msg"
-    local panic_msg="$1"
-    local mce_panic="$2"
-
-    if echo ": $panic_msg" | grep -e "$mce_panic" &> /dev/null; then
-	echo "  Passed: correct panic"
-    else
-	echo "  Failed: uncorrect panic, expected: $mce_panic"
-    fi
+	[ $# -eq 2 ] || die "missing parameter for verify_panic_msg"
+	local panic_msg="$1"
+	local mce_panic="$2"
+
+	if echo ": $panic_msg" | grep -e "$mce_panic" &> /dev/null; then
+		echo "  Passed: correct panic"
+	else
+		echo "  Failed: uncorrect panic, expected: $mce_panic"
+	fi
 }
 
 verify_timeout_via_mcelog()
 {
-    [ $# -eq 1 ] || die "missing parameter for verify_timeout"
-    local mcelog="$1"
-
-    if mcelog_filter $mcelog "#BANK 218#" &> /dev/null; then
-	echo "  Passed: timeout detected"
-    else
-	echo "  Failed: no timeout detected"
-    fi
+	[ $# -eq 1 ] || die "missing parameter for verify_timeout"
+	local mcelog="$1"
+
+	if mcelog_filter $mcelog "#BANK 218#" &> /dev/null; then
+		echo "  Passed: timeout detected"
+	else
+		echo "  Failed: no timeout detected"
+	fi
 }
 
 set_tolerant()
 {
-    [ $# -eq 1 ] || die "missing parameter for set_tolerant"
-    echo -n $1 > /sys/devices/system/machinecheck/machinecheck0/tolerant
+	[ $# -eq 1 ] || die "missing parameter for set_tolerant"
+	echo -n $1 > /sys/devices/system/machinecheck/machinecheck0/tolerant
 }
 
 get_tolerant()
 {
-    cat /sys/devices/system/machinecheck/machinecheck0/tolerant
+	cat /sys/devices/system/machinecheck/machinecheck0/tolerant
 }
 
 check_debugfs()
 {
-	mount|grep /sys/kernel/debug > /dev/null 2>&1
-	[ ! $? -eq 0 ] && mount -t debugfs none /sys/kernel/debug
-	mount|grep /sys/kernel/debug > /dev/null 2>&1
-	[ ! $? -eq 0 ] && die "Kernel without debugfs support ?"
+	mount | grep -q /sys/kernel/debug
+	[ $? -eq 0 ] && return
+	mount -t debugfs none /sys/kernel/debug
+	mount | grep -q /sys/kernel/debug
+	[ $? -ne 0 ] && die "Kernel without debugfs support ?"
 }
 
 # should be called after check_debugfs
 check_mce()
 {
-    DEBUGFS=`mount | grep debugfs | cut -d ' ' -f3 | head -1`
-    [ ! -d ${DEBUGFS}/mce ] && die "Kernel without CONFIG_X86_MCE_INJECT ?"
+	DEBUGFS=`mount | grep debugfs | cut -d ' ' -f3 | head -1`
+	[ ! -d ${DEBUGFS}/mce ] && die "Kernel without CONFIG_X86_MCE_INJECT ?"
 }
 
 set_fake_panic()
 {
-    check_debugfs
-    check_mce
-    [ $# -eq 1 ] || die "missing parameter for set_fake_panic"
-    echo -n $1 > /sys/kernel/debug/mce/fake_panic
+	check_debugfs
+	check_mce
+	[ $# -eq 1 ] || die "missing parameter for set_fake_panic"
+	echo -n $1 > /sys/kernel/debug/mce/fake_panic
 }
 
 set_panic_on_oops()
 {
-    [ $# -eq 1 ] || die "missing parameter for set_panic_on_oops"
-    echo -n $1 > /proc/sys/kernel/panic_on_oops
+	[ $# -eq 1 ] || die "missing parameter for set_panic_on_oops"
+	echo -n $1 > /proc/sys/kernel/panic_on_oops
 }
diff --git a/mcemenu b/mcemenu
new file mode 100644
index 0000000..33b311c
--- /dev/null
+++ b/mcemenu
@@ -0,0 +1,405 @@
+#!/bin/sh
+################################################################################
+##                                                                            ##
+## This program is free software;  you can redistribute it and#or modify      ##
+## it under the terms of the GNU General Public License as published by       ##
+## the Free Software Foundation; either version 2 of the License, or          ##
+## (at your option) any later version.                                        ##
+##                                                                            ##
+## This program is distributed in the hope that it will be useful, but        ##
+## WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY ##
+## or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License   ##
+## for more details.                                                          ##
+##                                                                            ##
+## You should have received a copy of the GNU General Public License          ##
+## along with this program;  if not, write to the Free Software               ##
+## Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA    ##
+##                                                                            ##
+################################################################################
+
+#
+# File:        runmcetest
+#
+# Description: This program is a Graphical User Interface (GUI) 
+#              Control Centre for MCE Test Suite. The Control Centre provides 
+#              functionality to Compile, Execute and View Results of
+#              MCE Test Cases.
+#              
+# Author:      Gong Chen <gong.chen@linux.intel.com>
+# 
+# These codes are heavily reused from ltpmenu.
+# Thanks very much for Manoj Iyer <manjo@mail.utexas.edu>
+#
+
+# Global variables.
+TOOL_DIR=./bin
+TMP_DIR=./work
+LOG_DIR=./results
+SUM_DIR=./summary
+SCENARIO_DIR=./groups
+
+SUM_FILE=""
+LOG_FILE=""
+TOTAL_COUNT=0
+PASS_COUNT=0
+FAIL_COUNT=0
+RUNALL_TIMES=1
+
+#Dialog Resource definition
+BACKTITLE="Linux MCE Test Suite Control Center"
+
+#main menu window resources
+TITLE_MAIN="Main Menu"
+MENU_MAIN="Move using[UP] [DOWN], Select using [ENTER]"
+ABOUT_MAIN="About MCE Test Suite"
+COMPILE_MAIN="Compile Whole Test Suite"
+DETAIL_MAIN="Details of Scenario Files"
+EXECUTE_MAIN="Execute Selected Test Cases"
+RESULTS_MAIN="Display a Summary of Test Results"
+GOODBYE_TITLE="Good Bye!"
+GOODBYE_MSG="Thank you for using Linux MCE Test Suite"
+
+#about window resources
+ABOUT_TITLE="About MCE Test Suite"
+ABOUT_MSG="The MCE Test Suite can be used to to compile, \
+install and execute all kinds of Linux MCE Tests. "
+
+#compile window resources
+COMPILE_TITLE="Compiling LTP testsuite"
+COMPILE_YESNO="This will compile all the test cases in \
+MCE Test Suite and place the executables \
+in corresponding directory. Do \
+you wish to continue ??" 
+COMPILE_TILE="Compiling MCE Test Suite"
+COMPILE_MSG="ERROR: command \$cmd not found, \$cmd is \
+required to compile MCE Test Suite. Please \
+install \$cmd or export PATH correctly before \
+running this program"
+COMPLIE_ERROR_1="ERROR in \'make clean\' - exiting."
+COMPLIE_ERROR_2="ERROR in \'make all\' - exiting."
+COMPLIE_ERROR_3="ERROR in \'make install\' - exiting."
+
+#result window resources
+RESULT_TITLE="MCE Test Results"
+RESULT_MENU="Move using[UP] [DOWN], Select using [ENTER]\n\n
+To get more detail information during the test procedure \
+please check directory $LOG_DIR.\n"
+NOFILE_MSG="ERROR: No files to view in $SUM_DIR directory."
+
+#scenario/group window resources
+SCE_TITILE="MCE Test Scenario Files"
+CHECKLIST_MAIN="Move using[UP] [DOWN], Select using [SPACE]"
+
+#execute window resources
+EXE_TITILE="Executing MCE testcases"
+EXE_MSG1="The testcases must to be compiled in order \
+to execute them. Returning to main menu. \
+Please select the Compile option." 
+EXE_MSG2="At least one test case should be choosed. \
+Please return to *Scenario* menu to select \
+at least one test case." 
+
+#flag prompt window resources
+PROMPT_TITLE="Test Duration - Times Specification"
+PROMPT_MSG1="Would you like to specify test duration? \
+Default is the length of one loop."
+PROMPT_MSG2="Please enter the duration numbers to run. \
+0 menas infinite loop."
+
+# Function:    cleanup
+#
+# Description: Remove all temporary files created by this program. Cleanup 
+#              always called on program exit.
+cleanup()
+{
+    rm -rf $TMP_DIR/*
+}
+
+
+# Function:    display_info_msg 
+#
+# Description: Displays informational messages window. This window may
+#              may be used to display information like errors, instructions
+#              etc to the user. The window is dismissed when the user hits
+#              the [ENTER] key.
+display_info_msg()
+{
+	dialog --backtitle "$BACKTITLE" \
+		--title " $1 " \
+		--msgbox " $2 " 10 70 
+	return $?
+}
+
+
+# Function:    compile_rastest
+#
+# Description: Checks for commands that are pre-reqs for compiling and 
+#              installing MCE Test Suite. It displays a confirmation window
+#              in order to confirm the choice made by the user.
+compile_rastest()
+{
+	dialog --backtitle "$BACKTITLE" \
+	--title "$COMPILE_TITLE" \
+	--yesno "$COMPILE_YESNO" 7 70 || RC=$?
+	case $RC in
+		0) for cmd in cc make; do
+			which $cmd &> /dev/null
+			if [ $? -ne 0 ]; then
+				msg=`eval echo $COMPILE_MSG`
+				display_info_msg "$COMPILE_TILE" "$msg"
+				return
+			fi
+		done
+		make clean
+		if [ $? -ne 0 ]; then
+			echo $COMPLIE_ERROR_1
+			exit
+		fi 
+		make
+		if [ $? -ne 0 ]; then
+			echo $COMPLIE_ERROR_2
+			exit
+		fi 
+		make install
+		if [ $? -ne 0 ]; then
+			echo $COMPLIE_ERROR_3
+			exit
+		fi 
+		return ;;
+
+		1|255)  return ;;
+	esac
+}
+
+
+# Function:    disp_result
+#
+# Description: The summary generated after the execution located under
+#              $SUM_DIR/ directory in a text (ASCII) file called 
+#              todaysdate.time.log. This function displays this file in a
+#              window. If the results file does not exist it displays an 
+#              info message window notifing the user that MCE test cases
+#              need to be executed in order to view results.
+disp_result()
+{
+	RC=0
+	local RESULTS_LIST=$(for i in `ls -1 -A -I ".git*" -B $SUM_DIR`;\
+			do echo -n "$i [more...] "; done)
+	if ! [ -z $RESULTS_LIST ] ;then
+		while [ $RC -ne "1" ] 
+		do
+			dialog --clear
+			dialog --backtitle "$BACKTITLE" \
+			--title "$RESULT_TITLE" \
+			--menu "$RESULT_MENU" 15 70 8 \
+			$RESULTS_LIST \
+			2>$TMP_DIR/rastest.results.$$ || RC=$?
+			results_item=$(cat $TMP_DIR/rastest.results.$$)
+			if ! [ -z $results_item ];then
+				dialog --clear
+				dialog --backtitle "$BACKTITLE" \
+				--title "$RESULT_TITLE" \
+				--textbox $SUM_DIR/$results_item 17 70
+			fi
+		done
+	else
+		dialog --clear
+		dialog --backtitle "$BACKTITLE" \
+		--title "$RESULT_TITLE" \
+		--msgbox "$NOFILE_MSG" 5 53 
+	fi
+}
+
+
+# Function:    flags_prompt
+#
+# Description: Prompt for and record user options for run duration
+flags_prompt()
+{
+	local RC=0
+
+	RUNALL_TIMES=1
+
+	dialog --clear
+	dialog --backtitle "$BACKTITLE" \
+	--title "$PROMPT_TITLE" --clear\
+	--yesno "$PROMPT_MSG1" 7 80
+	RC=$?
+	if [ $RC -eq "0" ]
+	then
+		dialog --backtitle "$BACKTITLE" \
+		--title "$PROMPT_TITLE" --clear\
+		--inputbox "$PROMPT_MSG2" 7 80 \
+		2>$TMP_DIR/rastest.length.$$ ;
+		RUNALL_TIMES=$(cat $TMP_DIR/rastest.length.$$ | awk '{print $1}')
+	fi
+}
+
+# Function:    execute_test
+#
+# Description: This function provides a menu of testcases that can be
+#              selected for execution.
+#              The function checks to see if the file $TOOL_DIR/page-types
+#              was created, this file is created when the whole test suite
+#              is compiled and installed, if it is not found
+#              an info message window will notify the user that it needs to
+#              be compiled before tests can be executed.
+#              This function creates the testcase list based on the users
+#              choice of scenario groups and execute these tests.
+execute_test()
+{
+	local RC=0
+	local DATE_START=""
+	local DATE_END=""
+	local size=0
+	local TST_CHOICE=$TMP_DIR/rastest.choice.$$
+	local CASELIST=$TMP_DIR/rastest.caselist.$$
+
+	if ! [ -f $TOOL_DIR/page-types ]
+	then
+		display_info_msg "$EXE_TITILE" \
+		"$EXE_MSG1"
+		return
+	fi
+
+	touch $TST_CHOICE
+	cat $SCENARIO_DIR/* > $TST_CHOICE
+	awk '{if ($3 == "on") {print $1" "$2}}' $TST_CHOICE > $CASELIST
+
+	size=`wc -m $CASELIST | awk '{print $1}'`
+	if [ $size -eq 0 ];then
+		RC=1
+	fi
+
+	case $RC in 
+		0)
+		flags_prompt
+		dialog --clear
+
+		./runmcetest -t $TMP_DIR -s $SUM_DIR -o $LOG_DIR -b $TOOL_DIR\
+			-l $CASELIST -r $RUNALL_TIMES
+
+		return ;;
+		1|255)
+		display_info_msg "$EXE_TITILE" \
+				"$EXE_MSG2"
+		return ;;
+	esac
+}
+
+
+# Function:    about_rastest
+#
+# Description: This function displays a window containing a brief message
+#              describing this programs functionality.
+about_rastest()
+{
+    display_info_msg "$ABOUT_TITLE" "$ABOUT_MSG"
+}
+
+
+# Function:    ras_scenarios
+#
+# Description: This function displays a list of scenario files located 
+#              in scenario directory.  Users can list the contents of
+#              each file and do some adjustment
+#
+ras_scenarios()
+{
+	RC=0
+	local RETVAL=0
+	local TMPMODE=""
+	local TMPSEL=$TMP_DIR/rastest.groupcases.$$
+	local SCENARIOS=$(for i in `ls -1 -A -I ".git*" -B $SCENARIO_DIR`;
+				do echo -n "$i [more...] "; done; )
+
+	while [ $RC -ne "1" ] 
+	do
+		dialog --clear
+		dialog --cancel-label QUIT --backtitle "$BACKTITLE" \
+		--title "$SCE_TITILE" \
+		--menu "$MENU_MAIN" 15 70 8 \
+		$SCENARIOS \
+		2>$TMP_DIR/rastest.scenario.$$ || RC=$?
+		scenario=$(cat $TMP_DIR/rastest.scenario.$$)
+		if ! [ -z $scenario ];then
+			#ignore comment line in group file
+			scenario_item=`grep -v "^#" $SCENARIO_DIR/$scenario`
+			dialog --clear
+			dialog --separate-output --backtitle "$BACKTITLE" \
+			--title "$SCE_TITILE" \
+			--checklist "$CHECKLIST_MAIN" 20 60 14 \
+				$scenario_item 2> $TMPSEL
+
+			RETVAL=$?
+			if [ $RETVAL -eq 0 ]; then
+				# convert original mode to another style
+				# e.g.
+				# item1
+				# item2
+				# item3
+				# =========== convert ===========
+				# item1\|item2\|item3
+				TMPMODE=$(cat $TMPSEL | while read line
+					do
+						echo -n \\\|$line
+					done)
+				#remove the first prefix "\|"
+				TMPMODE=${TMPMODE:2}
+				sed -i -e "s/\(.*\)on/\1off/g" $SCENARIO_DIR/$scenario
+				if [ X${TMPMODE} != X ]; then
+				sed -i -e "s/\(\(${TMPMODE}\).*\)off/\1on/g" \
+					$SCENARIO_DIR/$scenario
+				fi
+			fi
+		fi
+
+	done
+}
+
+
+                 
+# Function:    main
+#
+# Description: Displays the main menu to the MCE Test Suite. The menu
+#              provides options to Compile, Execute, and View test execution
+#              results.
+
+# test for dialog program exist
+if [ ! -x /usr/bin/dialog ]; then
+       echo "Sorry, MCE Test Suite GUI not available, can't find dialog. Exiting...";
+       exit 1;
+fi
+
+# call cleanup function on program exit.
+trap "cleanup" 0
+
+# wait in a loop until user hits [Cancel] button on the main menu.
+while :
+do
+	RC=0
+	dialog --clear
+	dialog --cancel-label QUIT --backtitle "$BACKTITLE" \
+	--title "$TITLE_MAIN" \
+	--menu "$MENU_MAIN" 15 70 5 \
+		About		"$ABOUT_MAIN" \
+		Compile		"$COMPILE_MAIN" \
+		Scenario	"$DETAIL_MAIN" \
+		Execute		"$EXECUTE_MAIN" \
+		Results		"$RESULTS_MAIN" \
+	2>$TMP_DIR/rastest.mainmenu.$$ || RC=$?
+
+	case $RC in 
+		0) mmenu_item=`cat $TMP_DIR/rastest.mainmenu.$$`
+			case $mmenu_item in
+				About)		about_rastest;;
+				Compile)	compile_rastest;;
+				Scenario)	ras_scenarios;;
+				Execute)	execute_test;;
+				Results)	disp_result;;
+			esac ;;
+
+		1|255) display_info_msg "$GOODBYE_TITLE" "$GOODBYE_MSG"
+		exit ;;
+	esac
+done
diff --git a/runmcetest b/runmcetest
new file mode 100644
index 0000000..bf87d74
--- /dev/null
+++ b/runmcetest
@@ -0,0 +1,267 @@
+#!/bin/sh
+################################################################################
+##                                                                            ##
+## This program is free software;  you can redistribute it and#or modify      ##
+## it under the terms of the GNU General Public License as published by       ##
+## the Free Software Foundation; either version 2 of the License, or          ##
+## (at your option) any later version.                                        ##
+##                                                                            ##
+## This program is distributed in the hope that it will be useful, but        ##
+## WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY ##
+## or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License   ##
+## for more details.                                                          ##
+##                                                                            ##
+## You should have received a copy of the GNU General Public License          ##
+## along with this program;  if not, write to the Free Software               ##
+## Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA    ##
+##                                                                            ##
+################################################################################
+
+#
+# File:        runmcetest
+#
+# Description: This program is the core executer for MCE Test Suite.
+#              
+# Author:      Chen Gong <gong.chen@linux.intel.com>
+# 
+
+#set -x
+
+TMP_DIR=""
+SUM_DIR=""
+LOG_DIR=""
+TOOL_DIR=""
+CASELIST=""
+SUM_FILE=""
+LOG_FILE=""
+TOTAL_COUNT=0
+PASS_SIGN="PASS"
+FAIL_SIGN="FAIL"
+RUNALL_TIMES=""
+
+EXE_MSG="The testcases must to be compiled and installed in order \
+to execute them."
+NOFILE_MSG="Some file or directory doesn't exist or assigned, \
+please check and run again."
+NOCASE_MSG="The case list file is not valid, please check \
+and run again."
+ROUND_MSG="The running time is not a valid value, please make
+use of a nonnegtive integer and run again."
+
+# Function:    cleanup
+#
+# Description: Remove all temporary files created by this program. Cleanup 
+#              always called on program exit.
+cleanup()
+{
+	if ! [ "X${TMP_DIR}" == "X" ]
+	then
+		rm -rf $TMP_DIR/*
+	fi
+}
+
+# Function:    run_test
+# 
+# Description: Execute actual tests and print messages by test cases.
+run_test()
+{
+	local IFSBACK=$IFS
+
+	# some test cases maybe meet issues when using WHILE-LOOP
+	# because of input redirection, instead of using FOR-LOOP
+	IFS=$'\n'
+	for caseid in `cat $1`
+	do
+		if [ ${caseid:0:1} != "#" ]
+		then
+			case_name=`echo $caseid | cut -d' ' -f1`
+			case_script=`echo $caseid | cut -d' ' -f2`
+			case_output=$TMP_DIR/rastest.caseoutput.$$
+
+			cat <<-EOF | tee -a $LOG_FILE
+			<<<<<<<<<<<<<<<<<<<    TEST BEGIN    >>>>>>>>>>>>>>>>>>>
+			Case ID: $case_name"
+			--------------------------------------------------------
+			EOF
+
+			# because pipe is used here, it is impossible to count
+			# PASS/FAIL internally
+			if ./$case_script
+			then
+				printf "%-36s%-10s\n" $case_name $PASS_SIGN >> $SUM_FILE
+			else
+				printf "%-36s%-10s\n" $case_name $FAIL_SIGN >> $SUM_FILE
+			fi | tee -a $LOG_FILE
+			TOTAL_COUNT=`expr $TOTAL_COUNT + 1`
+
+			cat <<-EOF | tee -a $LOG_FILE
+			--------------------------------------------------------
+			<<<<<<<<<<<<<<<<<<<<    TEST END    >>>>>>>>>>>>>>>>>>>>
+
+			EOF
+		fi
+	done
+	IFS=$IFSBACK
+
+	sleep 2
+}
+
+
+# Function:    execute_test
+#
+# Description: This function provides a menu of testcases that can be
+#              selected for execution.
+#              The function checks to see if the file $TOOL_DIR/page-types
+#              was created, this file is created when the whole test suite
+#              is compiled and installed, if it is not found
+#              an info message window will notify the user that it needs to
+#              be compiled before tests can be executed.
+#              This function creates the testcase list based on the users
+#              choice of scenario groups and execute these tests.
+execute_test()
+{
+	local DATE_START=""
+	local DATE_END=""
+
+	if ! [ -f $TOOL_DIR/page-types ] 
+	then 
+		echo $EXE_MSG
+		return
+	fi 
+
+	DATE_START=`date +%Y-%m-%d.%H.%M.%S`
+	SUM_FILE=$SUM_DIR/$DATE_START.log
+	LOG_FILE=$LOG_DIR/$DATE_START.output
+	TOTAL_COUNT=0
+
+	cat > $SUM_FILE <<-EOF
+	Test Start Time: $DATE_START
+	----------------------------------------------
+	testcase                            result
+	-------------------                 ----------
+	EOF
+
+	if [ $RUNALL_TIMES -eq 0 ]; then
+		while :
+		do
+			run_test $CASELIST
+		done
+	else
+		inc=0
+		while [ $inc -lt $RUNALL_TIMES ]
+		do
+			inc=`expr $inc + 1`
+			run_test $CASELIST
+		done
+	fi
+
+	DATE_END=`date +%Y-%m-%d.%H.%M.%S`
+	cat >> $SUM_FILE <<-EOF
+	----------------------------------------------
+	Test End Time: $DATE_END
+	Total Tests: $TOTAL_COUNT
+	Total Passes: `grep -c $PASS_SIGN $SUM_FILE`
+	Total Failures: `grep -c $FAIL_SIGN $SUM_FILE`
+	Kernel Version: `uname -r`
+	Machine Architecture: `uname -m`
+	EOF
+}
+
+usage()
+{
+	cat <<-EOF
+
+	usage: ${0##*/} [ -t TMPDIR ] [ -s SUMFILE ] [ -o OUTDIR ]
+	[ -b TOODIR ] [ -l CASEFILE ] [ -r RUNTIMES ]
+
+	-t TMPDIR       Directory where temporary files will be created.
+	-s SUMDIR       Directory where summary files will be created.
+	-o OUTDIR       Directory where raw test outputs will be created.
+	-b TOODIR	Directory where tool utilities will be placed.
+	-l CASEFILE     File where test case list will be used in the test.
+	-r RUNTIMES	Assign the whole test rounds. 0 means infinite loop.
+	-h              Help. Prints all available options.
+
+	example: ${0##*/} -t ./work/ -s ./summary -o ./results -b ./bin -l ./work/caselist -r 1
+
+
+	EOF
+
+	exit 0
+}
+
+main()
+{
+	if [ "$UID" -ne 0 ]; then
+		echo "Sorry, Must be root to run MCE Test Suite. Exiting..."
+		exit 1;
+	fi
+
+	[ $# -eq 0 ] && usage
+
+	for cmd in mcelog mce-inject; do
+		which $cmd &> /dev/null
+		[ $? -ne 0 ] &&
+		{
+			echo "Sorry, Please install $cmd first. Exiting..."
+			exit 1
+		}
+	done
+
+	while getopts t:s:o:b:l:r:h arg; do
+		case $arg in
+			t) TMP_DIR=$OPTARG ;;
+			s) SUM_DIR=$OPTARG ;;
+			o) LOG_DIR=$OPTARG ;;
+			b) TOOL_DIR=$OPTARG ;;
+			l) CASELIST=$OPTARG ;;
+			r) RUNALL_TIMES=$OPTARG ;;
+			h|\?) usage;;
+		esac
+	done
+
+	if ! [ -d "$TMP_DIR" -a -d "$SUM_DIR" -a -d "$LOG_DIR" \
+		-a -d "$TOOL_DIR" -a -e "$CASELIST" ]
+	then
+		echo $NOFILE_MSG
+		exit 2
+	fi
+
+	size=`wc -m $CASELIST | awk '{print $1}'`
+	if [ $size -eq 0 ]
+	then
+		echo $NOCASE_MSG
+		exit 3
+	fi
+
+	#when RUNALL_TIMES is NULL/string/negtive integer, exit...
+	test=`echo $RUNALL_TIMES | sed -n "/^[0-9]\+$/p"`
+	[ X"$test" = X ] &&
+	{
+		echo $ROUND_MSG
+		exit 4
+	}
+
+	#get absoluate path for env setting, some test cases requirement
+	pushd ./ > /dev/null
+	cd $TMP_DIR
+	TMP_DIR=`pwd`
+	cd - > /dev/null
+	cd $LOG_DIR
+	LOG_DIR=`pwd`
+	cd -  > /dev/null
+	cd $TOOL_DIR > /dev/null
+	TOOL_DIR=`pwd`
+	popd > /dev/null
+
+	export TMP_DIR
+	export LOG_DIR
+	export TOOL_DIR
+
+	execute_test
+}
+
+# call cleanup function on program exit.
+trap "cleanup" 0
+main "$@"
+
diff --git a/stress/Makefile b/stress/Makefile
deleted file mode 100644
index 2469b30..0000000
--- a/stress/Makefile
+++ /dev/null
@@ -1,25 +0,0 @@
-CFLAGS := -g -Wall
-LDFLAGS := -g
-
-export CFLAGS
-export LDFLAGS
-
-all:
-	mkdir -p bin
-	$(MAKE) -C tools install
-
-clean:
-	$(MAKE) -C tools clean
-	rm -rf bin log result runtest
-
-distclean: clean
-
-test: all stress-soft stress-hard
-
-stress-soft:
-	@echo "run soft stress tester for 60 seconds"
-	./hwpoison.sh -T -C 1 -t 60 -S
-
-stress-hard:
-	@echo "run hard stress tester for 60 seconds"
-	./hwpoison.sh -T -C 20 -t 60 -M
diff --git a/summary/.gitignore b/summary/.gitignore
new file mode 100644
index 0000000..f604d92
--- /dev/null
+++ b/summary/.gitignore
@@ -0,0 +1,3 @@
+# Ignore everything in this directory 
+* 
+# Except this file !.gitignore 
diff --git a/tools/Makefile b/tools/Makefile
index 42afa1c..e0a7e1e 100644
--- a/tools/Makefile
+++ b/tools/Makefile
@@ -1,7 +1,14 @@
-
 all:
 	$(MAKE) -C simple_process
+	$(MAKE) -C ltp-pan
+	$(MAKE) -C page-types
+
+install:
 	$(MAKE) -C simple_process install
+	$(MAKE) -C ltp-pan install
+	$(MAKE) -C page-types install
+
 clean:
 	$(MAKE) -C simple_process clean
-distclean: clean
+	$(MAKE) -C ltp-pan clean
+	$(MAKE) -C page-types clean
diff --git a/tools/ltp-pan/Makefile b/tools/ltp-pan/Makefile
new file mode 100644
index 0000000..cf4f432
--- /dev/null
+++ b/tools/ltp-pan/Makefile
@@ -0,0 +1,8 @@
+CFLAGS := -g -Wall
+
+ltp-pan: ltp-pan.o zoolib.o splitstr.o
+
+install: ltp-pan
+	cp -f ltp-pan ../../bin
+clean:
+	rm -f *.o ltp-pan
diff --git a/tools/ltp-pan/ltp-pan.c b/tools/ltp-pan/ltp-pan.c
new file mode 100644
index 0000000..86cd154
--- /dev/null
+++ b/tools/ltp-pan/ltp-pan.c
@@ -0,0 +1,1317 @@
+/*
+ * Copyright (c) 2000 Silicon Graphics, Inc.  All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+ *
+ * Further, this software is distributed without any warranty that it is
+ * free of the rightful claim of any third person regarding infringement
+ * or the like.  Any license provided herein, whether implied or
+ * otherwise, applies only to this software file.  Patent licenses, if
+ * any, provided herein do not apply to combinations of this program with
+ * other software, or any other product whatsoever.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write the Free Software Foundation, Inc., 59
+ * Temple Place - Suite 330, Boston MA 02111-1307, USA.
+ *
+ * Contact information: Silicon Graphics, Inc., 1600 Amphitheatre Pkwy,
+ * Mountain View, CA  94043, or:
+ *
+ * http://www.sgi.com
+ *
+ * For further information regarding this notice, see:
+ *
+ * http://oss.sgi.com/projects/GenInfo/NoticeExplan/
+ *
+ * Changelog:
+ * 
+ *	Added timer options: William Jay Huie, IBM
+ *	01/27/03 - Added: Manoj Iyer, manjo@mail.utexas.edu
+ *			   - option '-p' (pretty printing)i to enabled formatted printing 
+ *			     of results.
+ *
+ *	01/27/03 - Added: Manoj Iyer, manjo@mail.utexas.edu
+ *			   - added code to print system information
+ *
+ *	01/28/03 - Added: Manoj Iyer, manjo@mail.utexas.edu
+ *			   - added code to print test exit value.
+ *
+ *	01/29/03 - Added: Manoj Iyer, manjo@mail.utexas.edu
+ *			   - added code supresses test start and test end tags.
+ *
+ * 	07/22/07 - Added: Ricardo Salveti de Araujo, rsalveti@linux.vnet.ibm.com
+ *			   - added option to create a command file with all failed tests.
+ * 	
+ */
+/* $Id: ltp-pan.c,v 1.4 2009/10/15 18:45:55 yaberauneya Exp $ */
+
+#include <errno.h>
+#include <string.h>
+#include <sys/param.h>
+#include <sys/types.h>
+#include <sys/times.h>
+#include <sys/wait.h>
+#include <sys/stat.h>
+#include <time.h>
+#include <stdlib.h>
+#include <limits.h>
+#include <sys/utsname.h>
+
+#include "splitstr.h"
+#include "zoolib.h"
+
+/* One entry in the command line collection.  */
+struct coll_entry
+{
+    char *name;		/* tag name */
+    char *cmdline;	/* command line */
+    char *pcnt_f;	/* location of %f in the command line args, flag */
+    struct coll_entry *next;
+};
+
+struct collection
+{
+    int cnt;
+    struct coll_entry **ary;
+};
+
+struct tag_pgrp
+{
+    int pgrp;
+    int stopping;
+    time_t mystime;
+    struct coll_entry *cmd;
+    char output[PATH_MAX];
+};
+
+struct orphan_pgrp
+{
+    int pgrp;
+    struct orphan_pgrp *next;
+};
+
+static pid_t run_child(struct coll_entry *colle, struct tag_pgrp *active,
+				int quiet_mode);
+static char *slurp(char *file);
+static struct collection *get_collection(char *file, int optind, int argc,
+					 char **argv);
+static void pids_running(struct tag_pgrp *running, int keep_active);
+static int check_pids(struct tag_pgrp *running, int *num_active,
+		      int keep_active, FILE * logfile, FILE * failcmdfile, 
+		      struct orphan_pgrp *orphans, int fmt_print,
+		      int *failcnt, int quiet_mode);
+static void propagate_signal(struct tag_pgrp *running, int keep_active,
+			     struct orphan_pgrp *orphans);
+static void dump_coll(struct collection *coll);
+static char *subst_pcnt_f(struct coll_entry *colle);
+static void mark_orphan(struct orphan_pgrp *orphans, pid_t cpid);
+static void orphans_running(struct orphan_pgrp *orphans);
+static void check_orphans(struct orphan_pgrp *orphans, int sig);
+
+static void copy_buffered_output(struct tag_pgrp *running);
+static void write_test_start(struct tag_pgrp *running);
+static void write_test_end(struct tag_pgrp *running, const char *init_status,
+			   time_t exit_time, char *term_type, int stat_loc,
+			   int term_id, struct tms *tms1, struct tms *tms2);
+
+//wjh
+static char PAN_STOP_FILE[] = "PAN_STOP_FILE";
+
+static char *panname = NULL;
+static char *test_out_dir = NULL;	/* dir to buffer output to */
+zoo_t zoofile;
+static char *reporttype = NULL;
+
+/* zoolib */
+int rec_signal; 	/* received signal */
+int send_signal;	/* signal to send */
+
+/* Debug Bits */
+int Debug = 0;
+#define Dbuffile	0x000400	/* buffer file use */
+#define	Dsetup		0x000200	/* one-time set-up */
+#define	Dshutdown	0x000100	/* killed by signal */
+#define	Dexit		0x000020	/* exit status */
+#define	Drunning	0x000010	/* current pids running */
+#define	Dstartup	0x000004	/* started command */
+#define	Dstart		0x000002	/* started command */
+#define Dwait		0x000001	/* wait interrupted */
+
+int
+main(int argc, char **argv)
+{
+    extern char *optarg;
+    extern int optind;
+    char *zooname = NULL;	/* name of the zoo file to use */
+    char *filename = "/dev/null";	/* filename to read test tags from */
+    char *logfilename = NULL;
+    char *failcmdfilename = NULL;
+    char *outputfilename = NULL;
+    struct collection *coll = NULL;
+    struct tag_pgrp *running;
+    struct orphan_pgrp *orphans, *orph;
+	struct utsname unamebuf;
+    FILE *logfile = NULL;
+    FILE *failcmdfile = NULL;
+    int keep_active = 1;
+    int num_active = 0;
+	int failcnt = 0;           /* count of total testcases that failed. */
+    int err, i;
+    int starts = -1;
+    int timed = 0;
+    int run_time = -1; char modifier = 'm'; int ret = 0;
+    int stop;
+    int go_idle;
+    int has_brakes = 0;		/* stop everything if a test case fails */
+    int sequential = 0;		/* run tests sequentially */
+    int fork_in_road = 0;
+    int exit_stat;
+    int track_exit_stats = 0;	/* exit non-zero if any test exits non-zero */
+	int fmt_print = 0;          /* enables formatted printing of logfiles. */
+	int quiet_mode = 0;			/* supresses test start and test end tags. */
+    int c;
+    pid_t cpid;
+    struct sigaction sa;
+
+    while ((c = getopt(argc, argv, "AO:Sa:C:d:ef:hl:n:o:pqr:s:t:x:y")) != -1) {
+	switch (c) {
+	case 'A':	/* all-stop flag */
+	    has_brakes = 1;
+	    track_exit_stats = 1;
+	    break;
+	case 'O':	/* output buffering directory */
+	    test_out_dir = strdup(optarg);
+	    break;
+	case 'S':	/* run tests sequentially */
+	    sequential = 1;
+	    break;
+	case 'a':	/* name of the zoo file to use */
+	    zooname = strdup(optarg);
+	    break;
+	case 'C':	/* name of the file where all failed commands will be */
+	    failcmdfilename = strdup(optarg);
+	    break;
+	case 'd':	/* debug options */
+	    sscanf(optarg, "%i", &Debug);
+	    break;
+	case 'e':	/* exit non-zero if any test exists non-zero */
+	    track_exit_stats = 1;
+	    break;
+	case 'f':	/* filename to read test tags from */
+	    filename = strdup(optarg);
+	    break;
+	case 'h':	/* help */
+	    fprintf(stdout, "Usage: pan -n name [ -SyAehpq ] [ -s starts ]"
+				 " [-t time[s|m|h|d] [ -x nactive ] [ -l logfile ]\n\t"
+				 "[ -a active-file ] [ -f command-file ] "
+				 "[ -C fail-command-file ] "
+				 "[ -d debug-level ]\n\t[-o output-file] "
+				 "[-O output-buffer-directory] [cmd]\n");
+	    exit(0);
+	case 'l':	/* log file */
+	    logfilename = strdup(optarg);
+	    break;
+	case 'n':	/* tag given to pan */
+	    panname = strdup(optarg);
+	    break;
+	case 'o':	/* send test output here */
+	    outputfilename = strdup(optarg);
+	    break;
+	case 'p':	/* formatted printing. */
+		fmt_print = 1;
+		break;
+	case 'q':	/* supress test start and test end messages */
+		quiet_mode = 1;
+		break;
+	case 'r':	/* reporting type: none, rts */
+	    reporttype = strdup(optarg);
+	    break;
+	case 's':	/* number of tags to run */
+	    starts = atoi(optarg);
+	    break;
+	case 't':	/* run_time to run */
+	    ret = sscanf(optarg, "%d%c", &run_time, &modifier);
+            if (ret == 0) { fprintf(stderr, "Need proper time input: ####x where"
+                                    "x is one of s,m,h,d\n"); break; }
+            else if (ret == 1) { fprintf(stderr, "Only got a time value of %d "
+                                 "modifiers need to come immediately after #"
+                                 " assuming %c\n", run_time, modifier); }
+            else
+            {
+               switch (modifier)
+               {
+                  case 's': run_time = run_time; break; 
+                  case 'm': run_time = run_time * 60; break; 
+                  case 'h': run_time = run_time * 60 * 60; break; 
+                  case 'd': run_time = run_time * 60 * 60 * 24; break;
+                  default: 
+                     fprintf(stderr, "Invalid time modifier, try: s|h|m|d\n"); exit(-1);
+               }
+	       if (!quiet_mode)
+                  printf("PAN will run for %d seconds\n", run_time);
+            }
+            timed = 1; //-t implies run as many starts as possible, by default
+	    break;
+	case 'x':	/* number of tags to keep running */
+	    keep_active = atoi(optarg);
+	    break;
+	case 'y':	/* restart on failure or signal */
+	    fork_in_road = 1;
+	    break;
+	}
+    }
+
+    if (panname == NULL) {
+	fprintf(stderr, "pan: Must supply -n\n");
+	exit(1);
+    }
+    if (zooname == NULL) {
+	zooname = zoo_getname();
+	if (zooname == NULL) {
+	    fprintf(stderr,
+		    "pan(%s): Must supply -a or set ZOO env variable\n",
+		    panname);
+	    exit(1);
+	}
+    }
+    if (reporttype) {
+	/* make sure we understand the report type */
+	if (strcasecmp(reporttype, "rts")
+			&& strcasecmp(reporttype, "none")
+			/* && strcasecmp(reporttype, "xml")*/)
+	    reporttype = "rts";
+    } else {
+	/* set the default */
+	reporttype = "rts";
+    }
+
+    if (logfilename != NULL) {
+	time_t startup;
+	char *s;
+
+	if (!strcmp(logfilename, "-")) {
+	    logfile = stdout;
+	} else {
+	    if ((logfile = fopen(logfilename, "a+")) == NULL) {
+		fprintf(stderr,
+			"pan(%s): Error %s (%d) opening log file '%s'\n",
+			panname, strerror(errno), errno, logfilename);
+		exit(1);
+	    }
+	}
+
+	time(&startup);
+	s = ctime(&startup);
+	*(s + strlen(s) - 1) = '\0';
+	if (!fmt_print)
+		fprintf(logfile, "startup='%s'\n", s);
+	else
+	{
+		fprintf(logfile, "Test Start Time: %s\n", s);
+		fprintf(logfile, "-----------------------------------------\n");
+		fprintf(logfile, "%-30.20s %-10.10s %-10.10s\n", 
+				"Testcase", "Result", "Exit Value");
+		fprintf(logfile, "%-30.20s %-10.10s %-10.10s\n",
+			   	"--------", "------", "------------");
+	}
+    }
+
+    coll = get_collection(filename, optind, argc, argv);
+    if(!coll)
+        exit(1);
+    if (coll->cnt == 0) {
+	fprintf(stderr,
+		"pan(%s): Must supply a file collection or a command\n",
+		panname);
+	exit(1);
+    }
+
+    if (Debug & Dsetup)
+	dump_coll(coll);
+
+    /* a place to store the pgrps we're watching */
+    running = (struct tag_pgrp *)malloc((keep_active + 1) * sizeof(struct tag_pgrp));
+    if (running == NULL) {
+        fprintf(stderr, "pan(%s): Failed to allocate memory: %s\n", panname,
+                strerror(errno));
+	exit(2);
+    }
+    memset(running, 0, keep_active * sizeof(struct tag_pgrp));
+    running[keep_active].pgrp = -1;	/* end sentinel */
+
+    /* a head to the orphaned pgrp list */
+    orphans = (struct orphan_pgrp *) malloc(sizeof(struct orphan_pgrp));
+    memset(orphans, 0, sizeof(struct orphan_pgrp));
+
+    srand48(time(NULL) ^ (getpid() + (getpid() << 15)));
+
+    /* Supply a default for starts.  If we are in sequential mode, use
+     * the number of commands available; otherwise 1.
+     */
+    if (timed == 1 && starts == -1) {	/* timed, infinite by default */
+	starts = -1;
+    } else if (starts == -1) {
+	if (sequential) {
+	    starts = coll->cnt;
+	} else {
+	    starts = 1;
+	}
+    } else if (starts == 0) {	/* if the user specified infinite, set it */
+	starts = -1;
+    } else {			/* else, make sure we are starting at least keep_active processes */
+	if (starts < keep_active)
+	    starts = keep_active;
+    }
+
+    /* if we're buffering output, but we're only running on process at a time,
+     * then essentially "turn off buffering"
+     */
+    if (test_out_dir && (keep_active == 1)) {
+	free(test_out_dir);
+	test_out_dir = NULL;
+    }
+
+    if (test_out_dir) {
+	struct stat sbuf;
+
+	if (stat(test_out_dir, &sbuf) < 0) {
+	    fprintf(stderr,
+		    "pan(%s): stat of -O arg '%s' failed.  errno: %d  %s\n",
+		    panname, test_out_dir, errno, strerror(errno));
+	    exit(1);
+	}
+	if (!S_ISDIR(sbuf.st_mode)) {
+	    fprintf(stderr, "pan(%s): -O arg '%s' must be a directory.\n",
+		    panname, test_out_dir);
+	    exit(1);
+	}
+	if (access(test_out_dir, W_OK | R_OK | X_OK) < 0) {
+	    fprintf(stderr,
+		    "pan(%s): permission denied on -O arg '%s'.  errno: %d  %s\n",
+		    panname, test_out_dir, errno, strerror(errno));
+	    exit(1);
+	}
+    }
+
+    if (outputfilename) {
+	if (!freopen(outputfilename, "a+", stdout)) {
+	    fprintf(stderr,
+		    "pan(%s): Error %s (%d) opening output file '%s'\n",
+		    panname, strerror(errno), errno, outputfilename);
+	    exit(1);
+	}
+    }
+
+    if (failcmdfilename) {
+    	if (!(failcmdfile = fopen(failcmdfilename, "a+"))) {
+	    fprintf(stderr,
+		    "pan(%s): Error %s (%d) opening fail cmd file '%s'\n",
+		    panname, strerror(errno), errno, failcmdfilename);
+	    exit(1);
+    	}
+    }
+
+    if ((zoofile = zoo_open(zooname)) == NULL) {
+	fprintf(stderr, "pan(%s): %s\n", panname, zoo_error);
+	exit(1);
+    }
+    if (zoo_mark_args(zoofile, getpid(), panname, argc, argv)) {
+	fprintf(stderr, "pan(%s): %s\n", panname, zoo_error);
+	exit(1);
+    }
+
+    /* Allocate N spaces for max-arg commands.
+     * this is an "active file cleanliness" thing
+     */
+    {
+	char *av[2], bigarg[82];
+
+	memset(bigarg, '.', 81);
+	bigarg[81] = '\0';
+	av[0] = bigarg;
+	av[1] = NULL;
+
+	for (c = 0; c < keep_active; c++) {
+	    if (zoo_mark_cmdline(zoofile, c, panname, "")) {
+		fprintf(stderr, "pan(%s): %s\n", panname, zoo_error);
+		exit(1);
+	    }
+	}
+	for (c = 0; c < keep_active; c++) {
+	    if (zoo_clear(zoofile, c)) {
+		fprintf(stderr, "pan(%s): %s\n", panname, zoo_error);
+		exit(1);
+	    }
+	}
+    }
+
+    rec_signal = send_signal = 0;
+    if (run_time != -1) { alarm(run_time); }
+
+    sigemptyset(&sa.sa_mask);
+    sa.sa_flags = 0;
+    sa.sa_handler = wait_handler;
+
+    sigaction(SIGALRM, &sa, NULL);
+    sigaction(SIGINT, &sa, NULL);
+    sigaction(SIGTERM, &sa, NULL);
+    sigaction(SIGHUP, &sa, NULL);
+    sigaction(SIGUSR1, &sa, NULL);	/* ignore fork_in_road */
+    sigaction(SIGUSR2, &sa, NULL);	/* stop the scheduler */
+
+    c = 0;			/* in this loop, c is the command index */
+    stop = 0;
+    exit_stat = 0;
+    go_idle = 0;
+    while (1) {
+
+	while ((num_active < keep_active) && (starts != 0)) {
+	    if (stop || rec_signal || go_idle)
+		break;
+
+	    if (!sequential)
+		c = lrand48() % coll->cnt;
+
+	    /* find a slot for the child */
+	    for (i = 0; i < keep_active; ++i) {
+		if (running[i].pgrp == 0)
+		    break;
+	    }
+	    if (i == keep_active) {
+		fprintf(stderr, "pan(%s): Aborting: i == keep_active = %d\n",
+			panname, i);
+		wait_handler(SIGINT);
+		exit_stat++;
+		break;
+	    }
+
+	    cpid = run_child(coll->ary[c], running + i, quiet_mode);
+	    if (cpid != -1)
+		++num_active;
+	    if ((cpid != -1 || sequential) && starts > 0)
+		--starts;
+
+	    if (sequential)
+		if (++c >= coll->cnt)
+		    c = 0;
+
+	} /* while( (num_active < keep_active) && (starts != 0) ) */
+
+	if (starts == 0)
+	{ 
+		if (!quiet_mode)
+			printf("incrementing stop\n"); 
+		++stop; 
+	}
+	else if (starts == -1) //wjh
+	{
+	   FILE *f = (FILE*)-1;
+	   if ((f = fopen(PAN_STOP_FILE, "r")) != 0)
+	   {  printf("Got %s Stopping!\n", PAN_STOP_FILE);
+		  fclose(f); unlink(PAN_STOP_FILE); stop++; 
+	   }
+	}
+
+	if (rec_signal) {
+	    /* propagate everything except sigusr2 */
+
+	    if (rec_signal == SIGUSR2) {
+		if (fork_in_road)
+		    ++go_idle;
+		else
+		    ++stop;
+		rec_signal = send_signal = 0;
+	    } else {
+		if (rec_signal == SIGUSR1)
+		    fork_in_road = 0;
+		propagate_signal(running, keep_active, orphans);
+		if (fork_in_road)
+		    ++go_idle;
+		else
+		    ++stop;
+	    }
+	}
+
+	err = check_pids(running, &num_active, keep_active, logfile,
+			 failcmdfile, orphans, fmt_print, &failcnt, quiet_mode);
+	if (Debug & Drunning) {
+	    pids_running(running, keep_active);
+	    orphans_running(orphans);
+	}
+	if (err) {
+	    if (fork_in_road)
+		++go_idle;
+	    if (track_exit_stats)
+		exit_stat++;
+	    if (has_brakes) {
+		fprintf(stderr, "pan(%s): All stop!%s\n", panname,
+			go_idle ? " (idling)" : "");
+		wait_handler(SIGINT);
+	    }
+	}
+
+	if (stop && (num_active == 0))
+	    break;
+
+	if (go_idle && (num_active == 0)) {
+	    go_idle = 0;	/* It is idle, now resume scheduling. */
+	    wait_handler(0);	/* Reset the signal ratchet. */
+	}
+    }
+
+    /* Wait for orphaned pgrps */
+    while (1) {
+	for (orph = orphans; orph != NULL; orph = orph->next) {
+	    if (orph->pgrp == 0)
+		continue;
+	    /* Yes, we have orphaned pgrps */
+	    sleep(5);
+	    if (!rec_signal) {
+		/* force an artificial signal, move us
+		 * through the signal ratchet.
+		 */
+		wait_handler(SIGINT);
+	    }
+	    propagate_signal(running, keep_active, orphans);
+	    if (Debug & Drunning)
+		orphans_running(orphans);
+	    break;
+	}
+	if (orph == NULL)
+	    break;
+    }
+
+    if (zoo_clear(zoofile, getpid())) {
+	fprintf(stderr, "pan(%s): %s\n", panname, zoo_error);
+	++exit_stat;
+    }
+    fclose(zoofile);
+	if (logfile && fmt_print)
+	{
+		if (uname(&unamebuf) == -1)
+			fprintf(stderr, "ERROR: uname(): %s\n", strerror(errno));
+		fprintf(logfile, "\n-----------------------------------------------\n");
+		fprintf(logfile, "Total Tests: %d\n", coll->cnt);
+		fprintf(logfile, "Total Failures: %d\n", failcnt);
+		fprintf(logfile, "Kernel Version: %s\n", unamebuf.release);
+		fprintf(logfile, "Machine Architecture: %s\n", unamebuf.machine);
+		fprintf(logfile, "Hostname: %s\n\n", unamebuf.nodename);
+	}
+    if (logfile && (logfile != stdout))
+	fclose(logfile);
+
+    exit(exit_stat);
+}
+
+
+
+static void
+propagate_signal(struct tag_pgrp *running, int keep_active,
+		 struct orphan_pgrp *orphans)
+{
+    int i;
+
+    if (Debug & Dshutdown)
+	fprintf(stderr, "pan was signaled with sig %d...\n", rec_signal);
+
+    if (rec_signal == SIGALRM)
+    {
+       printf("PAN stop Alarm was received\n");
+       rec_signal = SIGTERM;
+    }
+
+    for (i = 0; i < keep_active; ++i) {
+	if (running[i].pgrp == 0)
+	    continue;
+
+	if (Debug & Dshutdown)
+	    fprintf(stderr, "  propagating sig %d to %d\n",
+		    send_signal, -running[i].pgrp);
+	if (kill(-running[i].pgrp, send_signal) != 0) {
+	    fprintf(stderr,
+		    "pan(%s): kill(%d,%d) failed on tag (%s).  errno:%d  %s\n",
+		    panname, -running[i].pgrp, send_signal,
+		    running[i].cmd->name, errno, strerror(errno));
+	}
+	running[i].stopping = 1;
+    }
+
+    check_orphans(orphans, send_signal);
+
+    rec_signal = send_signal = 0;
+}
+
+
+static int
+check_pids(struct tag_pgrp *running, int *num_active, int keep_active,
+	   FILE * logfile, FILE * failcmdfile, struct orphan_pgrp *orphans,
+	   int fmt_print, int *failcnt, int quiet_mode)
+{
+    int w;
+    pid_t cpid;
+    int stat_loc;
+    int ret = 0;
+    int i;
+    time_t t;
+    char *status;
+    int signaled = 0;
+    struct tms tms1, tms2;
+    clock_t tck;
+
+    check_orphans(orphans, 0);
+
+    tck = times(&tms1);
+    if (tck == -1) {
+	fprintf(stderr, "pan(%s): times(&tms1) failed.  errno:%d  %s\n",
+		panname, errno, strerror(errno));
+    }
+    cpid = wait(&stat_loc);
+    tck = times(&tms2);
+    if (tck == -1) {
+	fprintf(stderr, "pan(%s): times(&tms2) failed.  errno:%d  %s\n",
+		panname, errno, strerror(errno));
+    }
+
+    if (cpid < 0) {
+	if (errno == EINTR) {
+	    if (Debug)
+		fprintf(stderr, "pan(%s): wait() interrupted\n", panname);
+	} else if (errno != ECHILD) {
+	    fprintf(stderr, "pan(%s): wait() failed.  errno:%d  %s\n",
+		    panname, errno, strerror(errno));
+	}
+    } else if (cpid > 0) {
+
+	if (WIFSIGNALED(stat_loc)) {
+	    w = WTERMSIG(stat_loc);
+	    status = "signaled";
+	    if (Debug & Dexit)
+		fprintf(stderr, "child %d terminated with signal %d\n", cpid,
+			w);
+	    --*num_active;
+	    signaled = 1;
+	} else if (WIFEXITED(stat_loc)) {
+	    w = WEXITSTATUS(stat_loc);
+	    status = "exited";
+	    if (Debug & Dexit)
+		fprintf(stderr, "child %d exited with status %d\n", cpid, w);
+	    --*num_active;
+	    if (w != 0)
+		ret++;
+	} else if (WIFSTOPPED(stat_loc)) {	/* should never happen */
+	    w = WSTOPSIG(stat_loc);
+	    status = "stopped";
+	    ret++;
+	} else {		/* should never happen */
+	    w = 0;
+	    status = "unknown";
+	    ret++;
+	}
+
+	for (i = 0; i < keep_active; ++i) {
+	    if (running[i].pgrp == cpid) {
+		if ((w == 130) && running[i].stopping &&
+		    (strcmp(status, "exited") == 0)) {
+		    /* The child received sigint, but
+		     * did not trap for it?  Compensate
+		     * for it here.
+		     */
+		    w = 0;
+		    ret--;	/* undo */
+		    if (Debug & Drunning)
+			fprintf(stderr,
+				"pan(%s): tag=%s exited 130, known to be signaled; will give it an exit 0.\n",
+				panname, running[i].cmd->name);
+		}
+		time(&t);
+		if (logfile != NULL) {
+			if (!fmt_print)
+				fprintf(logfile,
+				 "tag=%s stime=%d dur=%d exit=%s stat=%d core=%s cu=%d cs=%d\n",
+					running[i].cmd->name, (int) (running[i].mystime),
+					(int) (t - running[i].mystime), status, w,
+					(stat_loc & 0200) ? "yes" : "no",
+					(int) (tms2.tms_cutime - tms1.tms_cutime),
+					(int) (tms2.tms_cstime - tms1.tms_cstime));
+			else
+			{
+					if (w != 0) 
+						++*failcnt;
+					fprintf(logfile, "%-30.30s %-10.10s %-5d\n", 
+							running[i].cmd->name, ((w != 0) ? "FAIL" : "PASS"),
+							w);
+			}
+
+			fflush(logfile);
+		}
+
+		if ((failcmdfile != NULL) && (w !=0)) {
+			fprintf(failcmdfile, "%s %s\n", running[i].cmd->name, running[i].cmd->cmdline);
+		}
+
+		if (running[i].stopping)
+		    status = "driver_interrupt";
+
+		if (test_out_dir) {
+			if (!quiet_mode)
+				write_test_start(running+i);
+			copy_buffered_output(running + i);
+			unlink(running[i].output);
+		}
+		if (!quiet_mode)
+			write_test_end(running+i, "ok", t, status,
+			   stat_loc, w, &tms1, &tms2);
+
+		/* If signaled and we weren't expecting
+		 * this to be stopped then the proc
+		 * had a problem.
+		 */
+		if (signaled && !running[i].stopping)
+		    ret++;
+
+		running[i].pgrp = 0;
+		if (zoo_clear(zoofile, cpid)) {
+		    fprintf(stderr, "pan(%s): %s\n", panname, zoo_error);
+		    exit(1);
+		}
+
+		/* Check for orphaned pgrps */
+		if ((kill(-cpid, 0) == 0) || (errno == EPERM)) {
+		    if (zoo_mark_cmdline(zoofile, cpid, "panorphan",
+					  running[i].cmd->cmdline)) {
+			fprintf(stderr, "pan(%s): %s\n", panname, zoo_error);
+			exit(1);
+		    }
+		    mark_orphan(orphans, cpid);
+		    /* status of kill doesn't matter */
+		    kill(-cpid, SIGTERM);
+		}
+
+		break;
+	    }
+	}
+    }
+    return ret;
+}
+
+
+static pid_t
+run_child(struct coll_entry *colle, struct tag_pgrp *active, int quiet_mode)
+{
+    int cpid;
+    int c_stdout = -1;		/* child's stdout, stderr */
+    int capturing = 0;		/* output is going to a file instead of stdout */
+    char *c_cmdline;
+    static long cmdno = 0;
+    int errpipe[2];		/* way to communicate to parent that the tag  */
+    char errbuf[1024];		/* didn't actually start */
+    int errlen;
+
+    /* Try to open the file that will be stdout for the test */
+    if (test_out_dir) {
+	capturing = 1;
+	do {
+	    sprintf(active->output, "%s/%s.%ld",
+		    test_out_dir, colle->name, cmdno++);
+	    c_stdout = open(active->output, O_CREAT | O_RDWR | O_EXCL | O_SYNC, 0666);
+	} while (c_stdout < 0 && errno == EEXIST);
+	if (c_stdout < 0) {
+	    fprintf(stderr,
+		    "pan(%s): open of stdout file failed (tag %s).  errno: %d  %s\n  file: %s\n",
+		    panname, colle->name, errno, strerror(errno),
+		    active->output);
+	    return -1;
+	}
+    } 
+
+    /* get the tag's command line arguments ready.  subst_pcnt_f() uses a
+     * static counter, that's why we do it here instead of after we fork.
+     */
+    if (colle->pcnt_f) {
+	c_cmdline = subst_pcnt_f(colle);
+    } else {
+	c_cmdline = colle->cmdline;
+    }
+    
+    if (pipe(errpipe) < 0) {
+	fprintf(stderr, "pan(%s): pipe() failed. errno:%d %s\n",
+			panname, errno, strerror(errno));
+	if (capturing) {
+	    close(c_stdout);
+            unlink(active->output);
+        }
+	return -1;
+    }
+
+    time(&active->mystime);
+    active->cmd = colle;
+
+    if (!test_out_dir) 
+	if (!quiet_mode)
+		write_test_start(active);
+
+    if ((cpid = fork()) < 0) {
+	fprintf(stderr, "pan(%s): fork failed (tag %s).  errno:%d  %s\n",
+		panname, colle->name, errno, strerror(errno));
+	if (capturing) {
+            unlink(active->output);
+	    close(c_stdout);
+        }
+	close(errpipe[0]);
+	close(errpipe[1]);
+	return -1;
+    } else if (cpid == 0) {
+	/* child */
+
+	fclose(zoofile);
+	close(errpipe[0]);
+	fcntl(errpipe[1], F_SETFD, 1);  /* close the pipe if we succeed */
+	setpgrp();
+
+	umask(0);
+
+	/* if we're putting output into a buffer file, we need to do the
+	 * redirection now.  If we fail
+	 */
+	if (capturing) {
+	    if (dup2(c_stdout, fileno(stdout)) == -1) {
+		errlen = sprintf(errbuf, "pan(%s): couldn't redirect stdout for tag %s.  errno:%d  %s",
+				panname, colle->name, errno, strerror(errno));
+		write(errpipe[1], &errlen, sizeof(errlen));
+		write(errpipe[1], errbuf, errlen);
+		exit(2);
+	    }
+	    if (dup2(c_stdout, fileno(stderr)) == -1) {
+		errlen = sprintf(errbuf, "pan(%s): couldn't redirect stderr for tag %s.  errno:%d  %s",
+				panname, colle->name, errno, strerror(errno));
+		write(errpipe[1], &errlen, sizeof(errlen));
+		write(errpipe[1], errbuf, errlen);
+		exit(2);
+	    }
+	} else { /* stderr still needs to be redirected */
+	    if (dup2(fileno(stdout), fileno(stderr)) == -1) {
+		errlen = sprintf(errbuf, "pan(%s): couldn't redirect stderr for tag %s.  errno:%d  %s",
+				panname, colle->name, errno, strerror(errno));
+		write(errpipe[1], &errlen, sizeof(errlen));
+		write(errpipe[1], errbuf, errlen);
+		exit(2);
+	    }
+	}
+	/* If there are any shell-type characters in the cmdline
+	 * such as '>', '<', '$', '|', etc, then we exec a shell and
+	 * run the cmd under a shell.
+	 *
+	 * Otherwise, break the cmdline at white space and exec the
+	 * cmd directly.
+	 */
+	if (strpbrk(c_cmdline, "\"';|<>$\\")) {
+	    execlp("sh", "sh", "-c", c_cmdline, (char*)0);
+	    errlen = sprintf(errbuf, 
+		    "pan(%s): execlp of '%s' (tag %s) failed.  errno:%d %s",
+		    panname, c_cmdline, colle->name, errno, strerror(errno));
+	} else {
+	    char **arg_v;
+
+	    arg_v = (char **)splitstr(c_cmdline, NULL, NULL);
+	    
+	    execvp(arg_v[0], arg_v);
+    	    errlen = sprintf(errbuf,
+		    "pan(%s): execvp of '%s' (tag %s) failed.  errno:%d  %s",
+		    panname, arg_v[0], colle->name, errno, strerror(errno));
+	}
+	write(errpipe[1], &errlen, sizeof(errlen));
+	write(errpipe[1], errbuf, errlen);
+	exit(errno);
+    }
+
+    /* parent */
+
+    /* subst_pcnt_f() allocates the command line dynamically
+     * free the malloc to prevent a memory leak
+     */
+    if (colle->pcnt_f) free(c_cmdline); 
+
+    close(errpipe[1]);
+
+    /* if the child couldn't go through with the exec, 
+     * clean up the mess, note it, and move on
+     */
+    if(read(errpipe[0], &errlen, sizeof(errlen))) {
+	int status;
+	time_t end_time;
+	int termid;
+	char *termtype;
+	struct tms notime = {0, 0, 0, 0};
+
+	read(errpipe[0], errbuf, errlen);
+	close(errpipe[0]);
+	errbuf[errlen] = '\0';
+	/* fprintf(stderr, "%s", errbuf); */
+	waitpid(cpid, &status, 0);
+	if (WIFSIGNALED(status)) {
+	    termid = WTERMSIG(status);
+	    termtype = "signaled";
+	} else if (WIFEXITED(status)) {
+	    termid = WEXITSTATUS(status);
+	    termtype = "exited";
+	} else if (WIFSTOPPED(status)) {
+	    termid = WSTOPSIG(status);
+	    termtype = "stopped";
+	} else {
+	    termid = 0;
+	    termtype = "unknown";
+	}
+	time(&end_time);
+	if (!quiet_mode) 
+	{
+		//write_test_start(active, errbuf);
+		write_test_end(active, errbuf, end_time, termtype, status, 
+			termid, &notime, &notime);
+	}
+        if (capturing) {
+            close(c_stdout);
+            unlink(active->output);
+        }
+	return -1;
+    }
+
+    close(errpipe[0]);
+    if (capturing) close(c_stdout);
+
+    active->pgrp = cpid;
+    active->stopping = 0;
+
+    if (zoo_mark_cmdline(zoofile, cpid, colle->name, colle->cmdline)) {
+	fprintf(stderr, "pan(%s): %s\n", panname, zoo_error);
+	exit(1);
+    }
+
+    if (Debug & Dstartup)
+	fprintf(stderr, "started %s cpid=%d at %s",
+		colle->name, cpid, ctime(&active->mystime));
+
+    if (Debug & Dstart) {
+	fprintf(stderr, "Executing test = %s as %s", colle->name, colle->cmdline);
+	if (capturing)
+	    fprintf(stderr, "with output file = %s\n", active->output);
+	else
+    	    fprintf(stderr, "\n");
+    }
+
+    return cpid;
+}
+
+
+static char *
+subst_pcnt_f(struct coll_entry *colle)
+{
+    static int counter = 1;
+    char pid_and_counter[20];
+    char new_cmdline[1024];
+
+    /* if we get called falsely, do the right thing anyway */
+    if (!colle->pcnt_f) 
+	return colle->cmdline;
+
+    snprintf(pid_and_counter, 20, "%d_%d", getpid(), counter++);
+    snprintf(new_cmdline, 1024, colle->cmdline, pid_and_counter);
+    return strdup(new_cmdline);
+}
+
+static struct collection *
+get_collection(char *file, int optind, int argc, char **argv)
+{
+    char *buf, *a, *b;
+    struct coll_entry *head, *p, *n;
+    struct collection *coll;
+    int i;
+
+    buf = slurp(file);
+    if(!buf)
+        return NULL;
+
+    coll = (struct collection *) malloc(sizeof(struct collection));
+    coll->cnt = 0;
+
+    head = p = n = NULL;
+    a = b = buf;
+    while (a) {
+	/* set b to the start of the next line and add a NULL character
+	 * to separate the two lines */
+	if ((b = strchr(a, '\n')) != NULL)
+            *b++ = '\0';
+
+	/* If this is line isn't a comment */
+	if ((*a != '#') && (*a != '\0') && (*a != ' ')) {
+	    n = (struct coll_entry *) malloc(sizeof(struct coll_entry));
+	    if ((n->pcnt_f = strstr(a, "%f"))) {
+		n->pcnt_f[1] = 's';
+	    }
+	    n->name = strdup(strsep(&a, " \t"));
+	    n->cmdline = strdup(a);
+	    n->next = NULL;
+	    
+	    if (p) {
+		p->next = n;
+	    }
+	    if (head == NULL) {
+		head = n;
+	    }
+	    p = n;
+	    coll->cnt++;
+	}
+        a = b;
+    }
+    free(buf);
+
+    /* is there something on the commandline to be counted? */
+    if (optind < argc) {
+	char workstr[1024] = "";
+	int workstr_left = 1023;
+	
+	/* fill arg list */
+	for (i = 0; optind < argc; ++optind, ++i) {
+	    strncat(workstr, argv[optind], workstr_left);
+	    workstr_left = workstr_left - strlen(argv[optind]);
+	    strncat(workstr, " ", workstr_left);
+	    workstr_left--;
+	}
+
+    	n = (struct coll_entry *) malloc(sizeof(struct coll_entry));
+    	if ((n->pcnt_f = strstr(workstr, "%f"))) {
+	    n->pcnt_f[1] = 's';
+	}
+	n->cmdline = strdup(workstr);
+	n->name = "cmdln";
+	n->next = NULL;
+	if (p) {
+	    p->next = n;
+	}
+	if (head == NULL) {
+	    head = n;
+	}
+	coll->cnt++;
+    }
+
+    /* get an array */
+    coll->ary = (struct coll_entry **) malloc(coll->cnt *
+					      sizeof(struct coll_entry *));
+
+    /* fill the array */
+    i = 0;
+    n = head;
+    while (n != NULL) {
+	coll->ary[i] = n;
+	n = n->next;
+	++i;
+    }
+    if (i != coll->cnt)
+	fprintf(stderr, "pan(%s): i doesn't match cnt\n", panname);
+
+    return coll;
+}
+
+
+static char *
+slurp(char *file)
+{
+    char *buf;
+    int fd;
+    struct stat sbuf;
+
+    if ((fd = open(file, O_RDONLY)) < 0) {
+	fprintf(stderr, "pan(%s): open(%s,O_RDONLY) failed.  errno:%d  %s\n",
+		panname, file, errno, strerror(errno));
+	return NULL;
+    }
+
+    if (fstat(fd, &sbuf) < 0) {
+	fprintf(stderr, "pan(%s): fstat(%s) failed.  errno:%d  %s\n",
+		panname, file, errno, strerror(errno));
+	return NULL;
+    }
+
+    buf = (char *) malloc(sbuf.st_size + 1);
+    if (read(fd, buf, sbuf.st_size) != sbuf.st_size) {
+	fprintf(stderr, "pan(%s): slurp failed.  errno:%d  %s\n",
+		panname, errno, strerror(errno));
+	return NULL;
+    }
+    buf[sbuf.st_size] = '\0';
+
+    close(fd);
+    return buf;
+}
+
+static void
+check_orphans(struct orphan_pgrp *orphans, int sig)
+{
+    struct orphan_pgrp *orph;
+
+    for (orph = orphans; orph != NULL; orph = orph->next) {
+	if (orph->pgrp == 0)
+	    continue;
+
+	if (Debug & Dshutdown)
+	    fprintf(stderr, "  propagating sig %d to orphaned pgrp %d\n",
+		    sig, -(orph->pgrp));
+	if (kill(-(orph->pgrp), sig) != 0) {
+	    if (errno == ESRCH) {
+		/* This pgrp is now empty */
+		if (zoo_clear(zoofile, orph->pgrp)) {
+		    fprintf(stderr, "pan(%s): %s\n", panname, zoo_error);
+		}
+		orph->pgrp = 0;
+	    } else {
+		fprintf(stderr,
+			"pan(%s): kill(%d,%d) on orphaned pgrp failed.  errno:%d  %s\n",
+			panname, -(orph->pgrp), sig, errno, strerror(errno));
+	    }
+	}
+    }
+}
+
+
+static void
+mark_orphan(struct orphan_pgrp *orphans, pid_t cpid)
+{
+    struct orphan_pgrp *orph;
+
+    for (orph = orphans; orph != NULL; orph = orph->next) {
+	if (orph->pgrp == 0)
+	    break;
+    }
+    if (orph == NULL) {
+	/* make a new struct */
+	orph = (struct orphan_pgrp *) malloc(sizeof(struct orphan_pgrp));
+
+	/* plug in the new struct just after the head */
+	orph->next = orphans->next;
+	orphans->next = orph;
+    }
+    orph->pgrp = cpid;
+}
+
+
+
+static void
+copy_buffered_output(struct tag_pgrp *running)
+{
+    char *tag_output;
+
+    tag_output = slurp(running->output);
+    if (tag_output) {
+	printf("%s", tag_output);
+	/* make sure the output ends with a newline */
+	if (tag_output[strlen(tag_output) - 1] != '\n')
+	    printf("\n");
+	fflush(stdout);
+	free(tag_output);
+    }
+}
+
+
+static void
+write_test_start(struct tag_pgrp *running)
+{
+    if (!strcmp(reporttype, "rts")) {
+
+	printf("%s\ntag=%s stime=%ld\ncmdline=\"%s\"\ncontacts=\"%s\"\nanalysis=%s\n%s\n",
+			"<<<test_start>>>",
+			running->cmd->name, running->mystime, running->cmd->cmdline, "",
+			"exit",
+			"<<<test_output>>>");
+    }
+    fflush(stdout);
+}
+
+
+static void
+write_test_end(struct tag_pgrp *running, const char *init_status,
+		time_t exit_time, char *term_type, int stat_loc,
+		int term_id, struct tms *tms1, struct tms *tms2)
+{
+    if (!strcmp(reporttype, "rts")) {
+	printf("%s\ninitiation_status=\"%s\"\nduration=%ld termination_type=%s "
+		"termination_id=%d corefile=%s\ncutime=%d cstime=%d\n%s\n",
+		  	"<<<execution_status>>>", init_status,
+			(long) (exit_time - running->mystime),
+			term_type, term_id, (stat_loc & 0200) ? "yes" : "no",
+			(int) (tms2->tms_cutime - tms1->tms_cutime),
+			(int) (tms2->tms_cstime - tms1->tms_cstime),
+			"<<<test_end>>>");
+    }
+    fflush(stdout);
+}
+
+/* The functions below are all debugging related */
+
+static void
+pids_running(struct tag_pgrp *running, int keep_active)
+{
+    int i;
+
+    fprintf(stderr, "pids still running: ");
+    for (i = 0; i < keep_active; ++i) {
+	if (running[i].pgrp != 0)
+	    fprintf(stderr, "%d ", running[i].pgrp);
+    }
+    fprintf(stderr, "\n");
+}
+
+static void
+orphans_running(struct orphan_pgrp *orphans)
+{
+    struct orphan_pgrp *orph;
+
+    fprintf(stderr, "orphans still running: ");
+    for (orph = orphans; orph != NULL; orph = orph->next) {
+	if (orph->pgrp != 0)
+	    fprintf(stderr, "%d ", -(orph->pgrp));
+    }
+    fprintf(stderr, "\n");
+}
+
+static void
+dump_coll(struct collection *coll)
+{
+    int i;
+
+    for (i = 0; i < coll->cnt; ++i) {
+	fprintf(stderr, "coll %d\n", i);
+	fprintf(stderr, "  name=%s cmdline=%s\n", coll->ary[i]->name,
+		coll->ary[i]->cmdline);
+    }
+}
+
+void
+wait_handler( int sig )
+{
+    static int lastsent = 0;
+
+    if( sig == 0 ){
+	lastsent = 0;
+    } else {
+	rec_signal = sig;
+	if( sig == SIGUSR2 )
+	    return;
+	if( lastsent == 0 )
+	    send_signal = sig;
+	else if( lastsent == SIGUSR1 )
+	    send_signal = SIGINT;
+	else if( lastsent == sig )
+	    send_signal = SIGTERM;
+	else if( lastsent == SIGTERM )
+	    send_signal = SIGHUP;
+	else if( lastsent == SIGHUP )
+	    send_signal = SIGKILL;
+	lastsent = send_signal;
+    }
+}
+
diff --git a/tools/ltp-pan/splitstr.c b/tools/ltp-pan/splitstr.c
new file mode 100644
index 0000000..6c4cea8
--- /dev/null
+++ b/tools/ltp-pan/splitstr.c
@@ -0,0 +1,197 @@
+/*
+ * Copyright (c) 2000 Silicon Graphics, Inc.  All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+ *
+ * Further, this software is distributed without any warranty that it is
+ * free of the rightful claim of any third person regarding infringement
+ * or the like.  Any license provided herein, whether implied or
+ * otherwise, applies only to this software file.  Patent licenses, if
+ * any, provided herein do not apply to combinations of this program with
+ * other software, or any other product whatsoever.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write the Free Software Foundation, Inc., 59
+ * Temple Place - Suite 330, Boston MA 02111-1307, USA.
+ *
+ * Contact information: Silicon Graphics, Inc., 1600 Amphitheatre Pkwy,
+ * Mountain View, CA  94043, or:
+ *
+ * http://www.sgi.com
+ *
+ * For further information regarding this notice, see:
+ *
+ * http://oss.sgi.com/projects/GenInfo/NoticeExplan/
+ *
+ */
+/* $Id: splitstr.c,v 1.2 2000/09/21 20:42:31 nstraz Exp $ */
+/*
+ * Synopsis
+ *
+ * const char **splitstr(const char *str, const char *separator, int *argcount)
+ *
+ * Description
+ * This function splits a string (str) into components that are separated by
+ * one or more of the characters in the (separator) string.  An array of
+ * strings is returned, along with argcount being set to the number of strings
+ * found.  Argcount can be NULL.  There will always be a NULL element in the
+ * array after the last valid element.  If an error occurs, NULL will be
+ * returned and argcount will be set to zero.
+ *
+ * To rid yourself of the memory allocated for splitstr(), pass the return
+ * value from splitstr() unmodified to splitstr_free():
+ *
+ * void splitstr_free( const char ** return_from_splitstr );
+ *
+ */
+#include <stdio.h>
+#include <malloc.h>
+#include <string.h>        /* for string functions */
+#ifdef UNIT_TEST
+#include <assert.h>
+#endif /* UNIT_TEST */
+#include "splitstr.h"
+
+const char **
+splitstr(const char *str, const char *separator, int *argcount)
+{
+    char *arg_string =NULL,
+         **arg_array =NULL,
+         *cur_tok    =NULL;
+
+    int  num_toks    =0,
+         max_toks    =20,
+         i;
+
+    /*
+     * In most recoverable errors, if argcount is not NULL,
+     * set argcount to 0. Then return NULL.
+     */
+    if ( str == NULL )
+    {
+      if ( argcount != NULL )
+        *argcount = 0;
+      return(NULL);
+    }
+
+    /*
+     * set aside temporary space to work on the string.
+     */
+    arg_string = strdup( str );
+
+    if ( arg_string == NULL )
+    {
+      if ( argcount != NULL )
+        *argcount = 0;
+      return(NULL);
+    }
+
+    /*
+     * set aside an initial char ** array for string array.
+     */
+    arg_array = (char **)malloc( sizeof(char *) * max_toks );
+
+    if ( arg_array == NULL )
+    {
+      if ( argcount != NULL )
+        *argcount = 0;
+      return(NULL);
+    }
+
+    if(separator==NULL)
+      separator = " \t";
+
+    /*
+     * Use strtok() to parse 'arg_string', placing pointers to the
+     * individual tokens into the elements of 'arg_array'.  Expand
+     * 'arg_array' if necessary.
+     */
+    cur_tok = strtok(arg_string, separator);
+    while ( cur_tok != NULL )
+    {
+      arg_array[num_toks++] = cur_tok;
+      cur_tok = strtok(NULL, separator);
+      if ( num_toks == max_toks )
+      {
+        max_toks += 20;
+        arg_array = (char **)realloc((void *)arg_array, sizeof(char *)*max_toks );
+      }
+    }
+    arg_array[num_toks] = NULL;
+
+    /*
+     * If there are any spaces left in our array, make them NULL
+     */
+    for(i=num_toks+1;i<max_toks;i++)
+      arg_array[i] = NULL;
+
+    /* This seems nice, but since memory is allocated on a page basis, this
+     * isn't really helpful:
+     * arg_array = (char **)realloc((void *)arg_array, sizeof(char *)*num_toks+1 );*/
+
+    if ( argcount != NULL )
+      *argcount = num_toks;
+
+    /*
+     * Return the argument array.
+     */
+    return((const char **)arg_array);
+}
+
+/*
+ * splitster_free( const char ** )
+ *
+ * This takes the return value from splitster() and free()s memory
+ * allocated by splitster.  Assuming: ret=splitster(...), this
+ * requires that ret and *ret returned from splitster() have not
+ * been modified.
+ */
+void splitstr_free( const char **p_return )
+{
+  if ( *p_return != NULL )
+    free( (char *)*p_return );
+  if ( p_return != NULL )
+    free( (char **)p_return );
+}
+
+#ifdef UNIT_TEST
+
+int main()
+{
+  int i,y,test_size=1000,size_ret;
+  char test_str[32768];
+  char buf[16];
+  char *test_str_array[test_size];
+  const char **ret;
+
+  for(i=0;i<test_size;i++)
+  {
+    snprintf(buf,16,"arg%d",i);
+    test_str_array[i] = strdup(buf);
+  }
+
+  for(i=0;i<test_size;i++)
+  {
+    test_str[0]='\0';
+    for(y=0;y<i;y++)
+    {
+      snprintf(buf,16,"arg%d ",y);
+      strncat(test_str,buf,16);
+    }
+    ret = splitstr(test_str,NULL,&size_ret);
+    assert(size_ret == i);
+    for(y=0;y<i;y++)
+      assert( strcmp(ret[y],test_str_array[y])==0 );
+
+    splitstr_free(ret);
+  }
+  return 0;
+}
+
+#endif
diff --git a/tools/ltp-pan/splitstr.h b/tools/ltp-pan/splitstr.h
new file mode 100644
index 0000000..2ffa24f
--- /dev/null
+++ b/tools/ltp-pan/splitstr.h
@@ -0,0 +1,36 @@
+#ifndef _SPLITSTR_H_
+#define _SPLITSTR_H_
+/*
+ * Synopsis
+ *
+ * const char **splitstr(const char *str, const char *separator, int *argcount)
+ *
+ * Description
+ * This function splits a string (str) into components that are separated by
+ * one or more of the characters in the (separator) string.  An array of
+ * strings is returned, along with argcount being set to the number of strings
+ * found.  Argcount can be NULL.  There will always be a NULL element in the
+ * array after the last valid element.  If an error occurs, NULL will be
+ * returned and argcount will be set to zero.
+ *
+ * To rid yourself of the memory allocated for splitstr(), pass the return
+ * value from splitstr() unmodified to splitstr_free():
+ *
+ * void splitstr_free( const char ** return_from_splitstr );
+ *
+ */
+const char **
+splitstr(const char *, const char *, int *);
+
+/*
+ * splitster_free( const char ** )
+ *
+ * This takes the return value from splitster() and free()s memory
+ * allocated by splitster.  Assuming: ret=splitster(...), this
+ * requires that ret and *ret returned from splitster() have not
+ * been modified.
+ */
+void
+splitstr_free( const char ** );
+
+#endif
diff --git a/tools/ltp-pan/zoolib.c b/tools/ltp-pan/zoolib.c
new file mode 100644
index 0000000..8f7bf6f
--- /dev/null
+++ b/tools/ltp-pan/zoolib.c
@@ -0,0 +1,474 @@
+/*
+ * Copyright (c) 2000 Silicon Graphics, Inc.  All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+ *
+ * Further, this software is distributed without any warranty that it is
+ * free of the rightful claim of any third person regarding infringement
+ * or the like.  Any license provided herein, whether implied or
+ * otherwise, applies only to this software file.  Patent licenses, if
+ * any, provided herein do not apply to combinations of this program with
+ * other software, or any other product whatsoever.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write the Free Software Foundation, Inc., 59
+ * Temple Place - Suite 330, Boston MA 02111-1307, USA.
+ *
+ * Contact information: Silicon Graphics, Inc., 1600 Amphitheatre Pkwy,
+ * Mountain View, CA  94043, or:
+ *
+ * http://www.sgi.com
+ *
+ * For further information regarding this notice, see:
+ *
+ * http://oss.sgi.com/projects/GenInfo/NoticeExplan/
+ *
+ */
+/* $Id: zoolib.c,v 1.8 2009/06/09 17:59:46 subrata_modak Exp $ */
+/* 
+ * ZooLib
+ *
+ * A Zoo is a file used to record what test tags are running at the moment.
+ * If the system crashes, we should be able to look at the zoo file to find out
+ * what was currently running.  This is especially helpful when running multiple
+ * tests at the same time.  
+ *
+ * The zoo file is meant to be a text file that fits on a standard console.
+ * You should be able to watch it with `cat zoofile`
+ *
+ * zoo file format:
+ * 	80 characters per line, ending with a \n
+ * 	available lines start with '#'
+ * 	expected line fromat: pid_t,tag,cmdline
+ *
+ */
+
+#include <stdlib.h> /* for getenv */
+#include <string.h>
+#include "zoolib.h"
+
+char zoo_error[ZELEN];
+
+#ifdef __linux__
+/* glibc2.2 definition needs -D_XOPEN_SOURCE, which breaks other things. */
+extern int sighold (int __sig);
+extern int sigrelse (int __sig);
+#endif
+
+/* zoo_mark(): private function to make an entry to the zoo 
+ * 	returns 0 on success, -1 on error */
+static int zoo_mark(zoo_t z, char *entry);
+static int zoo_lock(zoo_t z);
+static int zoo_unlock(zoo_t z);
+/* cat_args(): helper function to make cmdline from argc, argv */
+char *cat_args(int argc, char **argv);
+
+
+/* zoo_getname(): create a filename to use for the zoo */
+char *
+zoo_getname()
+{
+    char buf[1024];
+    char *zoo;
+    
+    zoo = getenv( "ZOO" );
+    if (zoo) {
+	snprintf(buf, 1024, "%s/%s", zoo, "active");
+	return strdup(buf);
+    } else {
+	/* if there is no environment variable, we don't know where to put it */
+	return NULL;
+    }
+}
+
+
+/* zoo_open(): open a zoo for use */
+zoo_t
+zoo_open(char *zooname)
+{
+    zoo_t new_zoo;
+
+    new_zoo = (zoo_t)fopen(zooname, "r+");
+    if (!new_zoo) {
+	if (errno == ENOENT) {
+	    /* file doesn't exist, try fopen(xxx, "a+") */
+	    new_zoo = (zoo_t)fopen(zooname, "a+");
+	    if (!new_zoo) {
+		/* total failure */
+		snprintf(zoo_error, ZELEN, 
+				"Could not open zoo as \"%s\", errno:%d %s", 
+				zooname, errno, strerror(errno));
+		return 0;
+	    }
+	    fclose(new_zoo);
+	    new_zoo = fopen(zooname, "r+");
+	} else {
+	    snprintf(zoo_error, ZELEN,
+			    "Could not open zoo as \"%s\", errno:%d %s",
+			    zooname, errno, strerror(errno));
+	}
+    }
+    return new_zoo;
+}
+
+int
+zoo_close(zoo_t z)
+{
+    int ret;
+
+    ret = fclose(z);
+    if (ret) {
+	snprintf(zoo_error, ZELEN,
+			"closing zoo caused error, errno:%d %s",
+			errno, strerror(errno));
+    }
+    return ret;
+}
+
+
+static int
+zoo_mark(zoo_t z, char *entry)
+{
+    FILE *fp = (FILE *)z;
+    int found = 0;
+    long pos;
+    char buf[BUFLEN];
+
+    if (fp == NULL)
+	return -1;
+    
+    if (zoo_lock(z))
+	return -1;
+    
+    /* first fit */
+    rewind(fp);
+
+    do {
+	pos = ftell(fp);
+
+	if (fgets(buf, BUFLEN, fp) == NULL) 
+	    break;
+
+	if (buf[0] == '#') {
+	    rewind(fp);
+	    if (fseek(fp, pos, SEEK_SET)) {
+		/* error */
+		snprintf(zoo_error, ZELEN,
+			"seek error while writing to zoo file, errno:%d %s",
+			errno, strerror(errno));
+		return -1;
+	    }
+	    /* write the entry, left justified, and padded/truncated to the 
+	     * same size as the previous entry */
+	    fprintf(fp, "%-*.*s\n", (int)strlen(buf)-1, (int)strlen(buf)-1, entry);
+	    found = 1;
+	    break;
+	}
+    } while (1);
+
+    if (!found) {
+	if (fseek(fp, 0, SEEK_END)) {
+	    snprintf(zoo_error, ZELEN,
+			    "error seeking to end of zoo file, errno:%d %s",
+			    errno, strerror(errno));
+	    return -1;
+	}
+	fprintf(fp, "%-*.*s\n", 79, 79, entry);
+    }
+    fflush(fp);
+
+    if (zoo_unlock(z))
+	return -1;
+    return 0;
+}
+
+int
+zoo_mark_cmdline(zoo_t z, pid_t p, char *tag, char *cmdline)
+{
+    char new_entry[BUFLEN];
+    
+    snprintf(new_entry, 80, "%d,%s,%s", p, tag, cmdline);
+    return zoo_mark(z, new_entry);
+}
+
+int
+zoo_mark_args(zoo_t z, pid_t p, char *tag, int ac, char **av)
+{
+    char *cmdline;
+    int ret;
+
+    cmdline = cat_args(ac, av);
+    ret = zoo_mark_cmdline(z, p, tag, cmdline);
+    
+    free(cmdline);
+    return ret;
+}
+
+int
+zoo_clear(zoo_t z, pid_t p)
+{
+    FILE *fp = (FILE *)z;
+    long pos;
+    char buf[BUFLEN];
+    pid_t that_pid;
+    int found = 0;
+
+
+    if (fp == NULL)
+	return -1;
+
+    if (zoo_lock(z))
+	return -1;
+    rewind(fp);
+
+    do {
+	pos = ftell(fp);
+
+	if (fgets(buf, BUFLEN, fp) == NULL) 
+	    break;
+
+	if (buf[0] == '#')
+	    continue;
+
+	that_pid = atoi(buf);
+	if (that_pid == p) {
+	    if (fseek(fp, pos, SEEK_SET)) {
+		/* error */
+		snprintf(zoo_error, ZELEN,
+			"seek error while writing to zoo file, errno:%d %s",
+			errno, strerror(errno));
+		return -1;
+	    }
+	    if (ftell(fp) != pos) {
+		printf("fseek failed\n");
+	    }
+	    fputs("#", fp);
+	    found = 1;
+	    break;
+	}
+    } while (1);
+
+    fflush( fp );
+
+    /* FIXME: unlock zoo file */
+    if (zoo_unlock(z))
+	return -1;
+
+    if(!found) {
+	snprintf(zoo_error, ZELEN, 
+			"zoo_clear() did not find pid(%d)", 
+			p);
+	return 1;
+    }
+    return 0;
+
+}
+
+pid_t 
+zoo_getpid(zoo_t z, char *tag)
+{
+    FILE *fp = (FILE *)z;
+    char buf[BUFLEN], *s;
+    pid_t this_pid = -1;
+
+
+    if (fp == NULL)
+	return -1;
+
+    if (zoo_lock(z))
+	return -1;
+
+    rewind(fp);
+    do {
+	if (fgets(buf, BUFLEN, fp) == NULL) 
+	    break;
+
+	if (buf[0] == '#')
+	    continue; /* recycled line */
+
+	if ((s = strchr(buf, ',')) == NULL) 
+	    continue; /* line was not expected format */
+
+	if (strncmp(s+1, tag, strlen(tag)))
+	    continue; /* tag does not match */
+
+	this_pid = atoi(buf);
+	break;
+    } while (1);
+
+    if (zoo_unlock(z))
+	return -1;
+    return this_pid;
+}
+
+int
+zoo_lock(zoo_t z)
+{
+    FILE *fp = (FILE *)z;
+    struct flock zlock;
+    sigset_t block_these;
+    int ret;
+    
+    if (fp == NULL)
+	return -1;
+
+    zlock.l_whence = zlock.l_start = zlock.l_len = 0;
+    zlock.l_type = F_WRLCK;
+
+    sigemptyset(&block_these);
+    sigaddset(&block_these, SIGINT);
+    sigaddset(&block_these, SIGTERM);
+    sigaddset(&block_these, SIGHUP);
+    sigaddset(&block_these, SIGUSR1);
+    sigaddset(&block_these, SIGUSR2);
+    sigprocmask(SIG_BLOCK, &block_these, NULL);
+
+    do {
+	ret = fcntl(fileno(fp), F_SETLKW, &zlock);
+    } while (ret == -1 && errno == EINTR);
+
+    sigprocmask(SIG_UNBLOCK, &block_these, NULL);
+    if (ret == -1) {
+	snprintf(zoo_error, ZELEN,
+			"failed to unlock zoo file, errno:%d %s",
+			errno, strerror(errno));
+	return -1;
+    } 
+    return 0;
+
+}
+
+int
+zoo_unlock(zoo_t z)
+{
+    FILE *fp = (FILE *)z;
+    struct flock zlock;
+    sigset_t block_these;
+    int ret;
+    
+    if (fp == NULL)
+	return -1;
+
+    zlock.l_whence = zlock.l_start = zlock.l_len = 0;
+    zlock.l_type = F_UNLCK;
+
+    sigemptyset(&block_these);
+    sigaddset(&block_these, SIGINT);
+    sigaddset(&block_these, SIGTERM);
+    sigaddset(&block_these, SIGHUP);
+    sigaddset(&block_these, SIGUSR1);
+    sigaddset(&block_these, SIGUSR2);
+    sigprocmask(SIG_BLOCK, &block_these, NULL);
+
+    do {
+	ret = fcntl(fileno(fp), F_SETLKW, &zlock);
+    } while (ret == -1 && errno == EINTR);
+
+    sigprocmask(SIG_UNBLOCK, &block_these, NULL);
+
+    if (ret == -1) {
+	snprintf(zoo_error, ZELEN,
+			"failed to lock zoo file, errno:%d %s",
+			errno, strerror(errno));
+	return -1;
+    } 
+    return 0;
+}
+
+char *
+cat_args(int argc, char **argv)
+{
+    int a, size;
+    char *cmd;
+
+    for( size = a = 0; a < argc; a++) {
+	size += strlen(argv[a]);
+	size++;
+    }
+
+    if( (cmd = (char *)malloc(size)) == NULL ) {
+	snprintf(zoo_error, ZELEN, 
+			"Malloc Error, %s/%d", 
+			__FILE__, __LINE__);
+	return NULL;
+    }
+
+    *cmd='\0';
+    for(a = 0; a < argc ; a++) {
+	if(a != 0)
+	    strcat(cmd, " ");
+	strcat(cmd, argv[a]);
+    }
+
+    return cmd;
+}
+
+#if defined(UNIT_TEST)
+
+
+void 
+zt_add(zoo_t z, int n)
+{
+    char cmdline[200];
+    char tag[10];
+
+    snprintf(tag, 10, "%s%d", "test", n);
+    snprintf(cmdline, 200, "%s%d %s %s %s", "runtest", n, "one", "two", "three");
+    
+    zoo_mark_cmdline(z, n, tag, cmdline);
+}
+    
+int
+main(int argc, char *argv[])
+{
+
+    char *zooname;
+    zoo_t test_zoo;
+    char *test_tag = "unittest";
+    int i,j;
+
+
+    zooname = zoo_getname();
+
+    if (!zooname) {
+	zooname = strdup("test_zoo");
+    }
+    printf("Test zoo filename is %s\n", zooname);
+
+    if ((test_zoo = zoo_open(zooname)) == NULL) {
+	printf("Error opennning zoo\n");
+	exit(-1);
+    }
+    
+
+    zoo_mark_args(test_zoo, getpid(), test_tag, argc, argv);
+    
+
+    for(j = 0; j < 5; j++) {
+	for(i = 0; i < 20; i++) {
+	    zt_add(test_zoo, i);
+	}
+	
+	for(; i >=0; i--) {
+	    zoo_clear(test_zoo, i);
+	}
+    }
+    
+    zoo_clear(test_zoo, getpid());
+    
+
+    return 0;
+}
+
+
+
+
+
+
+#endif
diff --git a/tools/ltp-pan/zoolib.h b/tools/ltp-pan/zoolib.h
new file mode 100644
index 0000000..8b4049e
--- /dev/null
+++ b/tools/ltp-pan/zoolib.h
@@ -0,0 +1,83 @@
+/*
+ * Copyright (c) 2000 Silicon Graphics, Inc.  All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+ *
+ * Further, this software is distributed without any warranty that it is
+ * free of the rightful claim of any third person regarding infringement
+ * or the like.  Any license provided herein, whether implied or
+ * otherwise, applies only to this software file.  Patent licenses, if
+ * any, provided herein do not apply to combinations of this program with
+ * other software, or any other product whatsoever.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write the Free Software Foundation, Inc., 59
+ * Temple Place - Suite 330, Boston MA 02111-1307, USA.
+ *
+ * Contact information: Silicon Graphics, Inc., 1600 Amphitheatre Pkwy,
+ * Mountain View, CA  94043, or:
+ *
+ * http://www.sgi.com
+ *
+ * For further information regarding this notice, see:
+ *
+ * http://oss.sgi.com/projects/GenInfo/NoticeExplan/
+ *
+ */
+/* $Id: zoolib.h,v 1.5 2006/06/27 09:37:34 vapier Exp $ */
+#ifndef ZOOLIB_H
+#define ZOOLIB_H
+
+#include <stdio.h>
+#include <sys/types.h>
+#include <unistd.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <sys/signal.h>
+
+typedef FILE *zoo_t;
+#define ZELEN 512
+extern char zoo_error[ZELEN];
+#define BUFLEN 81
+
+int lock_file( FILE *fp, short ltype, char **errmsg );
+/* FILE *open_file( char *file, char *mode, char **errmsg ); */
+
+void wait_handler();
+
+/*  char *zoo_active( void ); */
+/* zoo_getname(): create a filename to use for the zoo
+ * 	returns NULL on error */
+char *zoo_getname(void);
+
+/* zoo_open(): open a zoo file for use 
+ * 	returns NULL on error */
+zoo_t zoo_open(char *zooname);
+
+/* zoo_close(): close an open zoo file */
+int zoo_close(zoo_t z);
+
+/* zoo_mark_cmdline(): make an entry to the zoo 
+ *	returns 0 on success, -1 on error */
+int zoo_mark_cmdline(zoo_t z, pid_t p, char *tag, char *cmdline);
+
+/* zoo_mark_args(): make an entry to the zoo using argc argv
+ *	returns 0 on success, -1 on error */
+int zoo_mark_args(zoo_t z, pid_t p, char *tag, int ac, char **av);
+
+/* zoo_clear(): mark a pid as completed
+ *	returns 0 on success, -1 on error, 1 as warning */
+int zoo_clear(zoo_t z, pid_t p);
+
+/* zoo_getpid(): get the pid for a specified tag
+ * 	returns pid_t on success and 0 on error */
+pid_t zoo_getpid(zoo_t z, char *tag);
+
+
+#endif /* ZOOLIB_H */
diff --git a/tools/page-types/Makefile b/tools/page-types/Makefile
new file mode 100644
index 0000000..feba1af
--- /dev/null
+++ b/tools/page-types/Makefile
@@ -0,0 +1,9 @@
+CFLAGS := -g -Wall
+
+page-types: page-types.o
+
+install: page-types
+	cp -f page-types ../../bin
+
+clean:
+	rm -f page-types *.o
diff --git a/tools/page-types.c b/tools/page-types/page-types.c
index cc96ee2..cc96ee2 100644
--- a/tools/page-types.c
+++ b/tools/page-types/page-types.c
diff --git a/tools/gcov_merge.py b/tools/scripts/gcov_merge.py
index 0ac9bed..0ac9bed 100755..100644
--- a/tools/gcov_merge.py
+++ b/tools/scripts/gcov_merge.py
diff --git a/tools/grep_result.sh b/tools/scripts/grep_result.sh
index 4ef74ad..4ef74ad 100755..100644
--- a/tools/grep_result.sh
+++ b/tools/scripts/grep_result.sh
diff --git a/tools/loop-mce-test.sh b/tools/scripts/loop-mce-test.sh
index 3eb4e3e..3eb4e3e 100755..100644
--- a/tools/loop-mce-test.sh
+++ b/tools/scripts/loop-mce-test.sh
diff --git a/tools/mce_shell.sh b/tools/scripts/mce_shell.sh
index 09800cf..09800cf 100755..100644
--- a/tools/mce_shell.sh
+++ b/tools/scripts/mce_shell.sh
diff --git a/tools/scov_merge.py b/tools/scripts/scov_merge.py
index f83b922..f83b922 100755..100644
--- a/tools/scov_merge.py
+++ b/tools/scripts/scov_merge.py
diff --git a/tools/simple_process/Makefile b/tools/simple_process/Makefile
index d7c4e47..8a0ee28 100644
--- a/tools/simple_process/Makefile
+++ b/tools/simple_process/Makefile
@@ -1,13 +1,9 @@
 CFLAGS := -g -Wall
 
-all: simple_process
-
-install:
-	cp simple_process ../../bin
-
 simple_process: simple_process.o
 
-simple_process.o: simple_process.c
+install: simple_process
+	cp -f simple_process ../../bin
 
 clean:
 	rm -f simple_process *.o
diff --git a/tsrc/Makefile b/tsrc/Makefile
deleted file mode 100644
index 498cb10..0000000
--- a/tsrc/Makefile
+++ /dev/null
@@ -1,75 +0,0 @@
-LSRC := ../linux
-BROWSER := firefox
-
-CFLAGS += -I ${LSRC}/arch/x86/kernel/cpu/mcheck/ -g -Wall
-
-KFLAGS := -I ./kinclude
-
-EXE := tinjpage tsimpleinj tkillpoison tprctl tsoft tsoftinj thugetlb erst-inject
-EXE += ttranshuge
-EXEKERNEL := tring ttable
-
-OBJ := $(addsuffix .o,${EXE})
-OBJKERNEL := $(addsuffix .o,${EXEKERNEL})
-
-.PHONY: clean distclean see test hard soft standalone
-
-all: standalone
-
-standalone: ${EXE}
-
-kernel: ${EXEKERNEL}
-
-tcases: tcases.c ${LSRC}/arch/x86/kernel/cpu/mcheck/mce-severity.c ${LSRC}/arch/x86/kernel/cpu/mcheck/mce-internal.h
-	${CC} ${CFLAGS} ${KFLAGS} -o tcases tcases.c
-
-ttable: ttable.c ${LSRC}/arch/x86/kernel/cpu/mcheck/mce-severity.c ${LSRC}/arch/x86/kernel/cpu/mcheck/mce-internal.h
-	${CC} ${CFLAGS} ${KFLAGS} -o ttable ttable.c
-
-tprctl: tprctl.o
-
-tring: tring.o 
-
-tring : LDFLAGS += -lpthread
-
-x.html: ttable
-	./ttable ${TFLAGS} > x.html
-
-include erst-inj/erst-inj.mk
-
-.PHONY: see
-
-see: x.html
-	${BROWSER} x.html
-
-tinjpage: LDFLAGS += -lpthread
-
-.PHONY: clean distclean
-
-clean:
-	rm -f ${EXE} ${OBJ}
-	rm -f ${EXEKERNEL} ${OBJKERNEL}
-	rm -f x.html unpoison-failed offlined
-	rm -f ~test* ~poison*
-
-distclean: clean
-
-test: soft hard
-
-hard: standalone
-	./tinjpage
-	./tsimpleinj
-	if ! ./tkillpoison ; then echo "killed as expected" ; exit 0 ; else echo "didn't get killed" ; exit 1 ; fi
-	./tprctl
-
-soft: standalone
-	./tsoft
-	./tsoftinj
-	echo "Running soft offliner for 60 seconds"
-	./random_offline -t 60
-
-test-kernel: tcases
-	./tcases
-
-test-erst: erst-inject
-	./erst-inject.sh
diff --git a/tsrc/README b/tsrc/README
deleted file mode 100644
index 015f9e7..0000000
--- a/tsrc/README
+++ /dev/null
@@ -1,59 +0,0 @@
-These are some standalone test programs for various parts of the
-machine check code:
-
-They can be all tested together by running "make test"
-This requires root rights and a kernel with soft offlining and hard
-offlining support.
-
-To use ttable and tcases you need to specify a linux source tree
-with the mce improvements patchkit added with make LSRC=/path/to/linux
-or symlink the linux tree to ../linux
-
-The programs requiring kernel sources are not in the all make target.
-If you want to build them use make requireskernel after you
-set up the kernel sources or specified LSRC.
-
-tcases
-
-Simple tester of a few test cases by running the machine check grader code
-in user space. This is an alternative to the more complete coverage
-in the main mce-test test cases.
-
-Doesn't aim to be complete, but more a quick sanity check, that 
-can be done without booting a kernel.
-
-ttable
-
-Generate a table of the output of the machine check grader. This gives
-an overview how all the status bits in the machine check architecture 
-are processed. Note that a few bits are tested outside the grader
-and this doesn't apply to corrected machine check interrupts.
-
-Use make see to render the output  using links -g (note this doesn't work
-with elinks as it is used on many distributions, in this case specify
-a different browser with make BROWSER=browserbinary see)
-
-tinjpage
-
-tinjpage is a coverage test for the hwpoison page recovery code
-in the kernel. It tests different cases by putting pages
-into different stages, triggering poison injection on them 
-and verifies the results.
-
-tinjpage requires a kernel with MADV_POISON injection support and the
-hwpoison testkit, but doesn't require a kernel tree and can be built directly with
-make tinjpage
-
-A few of the test cases are timing dependent and might require adjustment
-to run successfully.
-
-tinjpage-working
-
-Old version of tinjpage that tests much less cases, but might be more reliable.
-
-tring
-
-Old unit test program for the ring buffer used in mca recovery.
-Not in default Makefile target.
-
--Andi Kleen
diff --git a/tsrc/erst-inj/erst-inj.mk b/tsrc/erst-inj/erst-inj.mk
deleted file mode 100644
index 32f253c..0000000
--- a/tsrc/erst-inj/erst-inj.mk
+++ /dev/null
@@ -1,4 +0,0 @@
-CFLAGS := -g -Wall
-
-erst-inject: erst-inj/erst-inject.c
-	${CC} ${CFLAGS} -o erst-inject erst-inj/erst-inject.c
diff --git a/tsrc/kinclude/README b/tsrc/kinclude/README
deleted file mode 100644
index 4521cd1..0000000
--- a/tsrc/kinclude/README
+++ /dev/null
@@ -1,3 +0,0 @@
-
-Fake kernel includes to build some kernel code in user context.
-
diff --git a/tsrc/kinclude/linux/debugfs.h b/tsrc/kinclude/linux/debugfs.h
deleted file mode 100644
index a366984..0000000
--- a/tsrc/kinclude/linux/debugfs.h
+++ /dev/null
@@ -1,15 +0,0 @@
-
-static inline struct dentry *debugfs_create_file(const char *name, mode_t mode,
-				   struct dentry *parent, void *data,
-				   const struct file_operations *fops)
-{
-	return NULL;
-}
-
-static inline struct dentry *debugfs_create_dir(const char *name, struct dentry *parent)
-{
-	return NULL;
-}
-
-
-static inline void debugfs_remove(struct dentry *dentry) { } 
diff --git a/tsrc/kinclude/linux/fs.h b/tsrc/kinclude/linux/fs.h
deleted file mode 100644
index 964b343..0000000
--- a/tsrc/kinclude/linux/fs.h
+++ /dev/null
@@ -1,10 +0,0 @@
-struct inode;
-struct file;
-struct dentry;
-
-struct file_operations {
-	ssize_t (*read) (struct file *, char  *, size_t, loff_t *);
-	int (*open) (struct inode *, struct file *);
-	int (*release) (struct inode *, struct file *);
-	ssize_t (*write) (struct file *, const char *, size_t, loff_t *);
-};
diff --git a/tsrc/kinclude/linux/init.h b/tsrc/kinclude/linux/init.h
deleted file mode 100644
index 77cd3ac..0000000
--- a/tsrc/kinclude/linux/init.h
+++ /dev/null
@@ -1,5 +0,0 @@
-
-#define __init
-#define __user
-
-#define late_initcall(x) typeof(x) x __attribute__((used))
diff --git a/tsrc/kinclude/linux/percpu.h b/tsrc/kinclude/linux/percpu.h
deleted file mode 100644
index 55b111a..0000000
--- a/tsrc/kinclude/linux/percpu.h
+++ /dev/null
@@ -1,9 +0,0 @@
-
-#define DECLARE_PER_CPU(x,y)
-#define BITS_PER_LONG (sizeof(long)*8)
-#define DECLARE_BITMAP(x,y) unsigned long x[((y) + BITS_PER_LONG - 1) / BITS_PER_LONG];
-#define MAX_NR_BANKS 32
-#define ARRAY_SIZE(x) (sizeof(x) / sizeof(*(x)))
-
-struct cpuinfo_x86;
-
diff --git a/tsrc/kinclude/linux/seq_file.h b/tsrc/kinclude/linux/seq_file.h
deleted file mode 100644
index 9b0c9f3..0000000
--- a/tsrc/kinclude/linux/seq_file.h
+++ /dev/null
@@ -1,22 +0,0 @@
-
-#include <linux/fs.h>
-
-struct file;
-
-struct seq_file {
-};
-
-struct seq_operations {
-	void * (*start) (struct seq_file *m, loff_t *pos);
-	void (*stop) (struct seq_file *m, void *v);
-	void * (*next) (struct seq_file *m, void *v, loff_t *pos);
-	int (*show) (struct seq_file *m, void *v);
-};
-
-#define seq_printf(a, b, c...) printf(b , ## c)
-static inline int seq_open(struct file *f, const struct seq_operations *o) { return -1; }
-
-static inline ssize_t seq_read(struct file *a, char  *b, size_t c, loff_t *d) { return 0; }
-static inline ssize_t seq_write(struct file *a, const char  *b, size_t c, loff_t *d) { return 0; }
-static inline int seq_release(struct inode *a, struct file *b) { return 0; }
-
diff --git a/tsrc/kinclude/linux/sysdev.h b/tsrc/kinclude/linux/sysdev.h
deleted file mode 100644
index a6d1cd8..0000000
--- a/tsrc/kinclude/linux/sysdev.h
+++ /dev/null
@@ -1,2 +0,0 @@
-
-struct sysdev_attribute {};
diff --git a/tsrc/run-transhuge-test.sh b/tsrc/run-transhuge-test.sh
deleted file mode 100755
index 4cf9fab..0000000
--- a/tsrc/run-transhuge-test.sh
+++ /dev/null
@@ -1,97 +0,0 @@
-#
-# run-transhuge-test.sh:
-#     Script for hwpoison test of THP(Transparent Huge Page).
-#
-#!/bin/sh
-#
-
-THP_POISON_PRO_FILE_NAME="ttranshuge"
-THP_POISON_PRO="./$THP_POISON_PRO_FILE_NAME"
-
-THP_SYS_PATH="/sys/kernel/mm/transparent_hugepage"
-THP_SYS_ENABLED_FILE="$THP_SYS_PATH/enabled"
-
-executed_testcase=0
-failed_testcase=0
-
-error()
-{
-	echo "$1" && exit 1
-}
-
-env_check()
-{
-    if [ ! -f $THP_POISON_PRO_FILE_NAME ] ; then
-	error "Please make sure there is file $THP_POISON_PRO_FILE_NAME."
-    fi
-
-    if [ ! -d $THP_SYS_PATH ] ; then
-	error "THP(Transparent Huge Page) may be not supported by kernel."
-    fi
-
-    thp_enabled="$(cat $THP_SYS_ENABLED_FILE | awk '{print $3}')"
-    if [ "$thp_enabled" == "[never]" ] ; then
-	error "THP(Transparent Huge Page) is disabled now."
-    fi
-}
-
-result_check()
-{
-    if [ "$1" != "0" ] ; then
-	failed_testcase=`expr $failed_testcase + 1`
-    fi
-}
-
-exec_testcase()
-{
-    if [ "$1" = "head" ] ; then
-	page_position_in_thp=0
-    elif [ "$1" = "tail" ] ; then
-	page_position_in_thp=1
-    else
-	error "Which page do you want to poison?"
-    fi
-
-    if [ "$2" = "early" ] ; then
-	process_type="--early-kill"
-    elif [ "$2" = "late_touch" ] ; then
-	process_type=""
-    elif [ "$2" = "late_avoid" ] ; then
-	process_type="--avoid-touch"
-    else
-	error "No such process type."
-    fi
-
-    executed_testcase=`expr $executed_testcase + 1`
-
-    echo "------------------ Case $executed_testcase --------------------"
-
-    command="$THP_POISON_PRO $process_type --offset $page_position_in_thp"
-    echo $command
-    eval $command
-    result_check $?
-
-    echo -e "\n"
-}
-
-# Environment Check for Test.
-env_check
-
-# Execute Test Cases from Here.
-echo "============= HWPoison Test of Transparent Huge Page ================="
-
-exec_testcase "head" "early"
-
-exec_testcase "head" "late_touch"
-
-exec_testcase "head" "late_avoid"
-
-exec_testcase "tail" "early"
-
-exec_testcase "tail" "late_touch"
-
-exec_testcase "tail" "late_avoid"
-
-echo "======================================================================="
-echo -n "    Num of Executed Test Case: $executed_testcase"
-echo -e "    Num of Failed Case: $failed_testcase\n"
diff --git a/tsrc/tcases.c b/tsrc/tcases.c
deleted file mode 100644
index 9c9797e..0000000
--- a/tsrc/tcases.c
+++ /dev/null
@@ -1,101 +0,0 @@
-/* 
- * Verify MCA grading engine against some examples.
- */
-#include <sys/types.h>
-#include <stdio.h>
-#define __KERNEL__ 1
-#include <asm/types.h>
-#include <asm/mce.h>
-#include <errno.h>
-
-#define ARRAY_SIZE(x) (sizeof(x)/sizeof(*(x)))
-
-typedef unsigned long long u64;
-
-#define MCI_STATUS_S	 (1ULL<<56)  /* Signaled machine check */
-#define MCI_STATUS_AR	 (1ULL<<55)  /* Action required */
-
-int mce_ser = 1;
-int tolerant = 1;
-int panic_on_oops = 0;
-
-#include "mce-severity.c"
-
-char *resname[] = {
-#define R(x) [MCE_ ## x ## _SEVERITY] = #x
-	R(NO),
-	R(KEEP),
-	R(SOME),
-	R(AO),
-	R(AR),
-	R(PANIC),
-};
-#define VAL MCI_STATUS_VAL
-#define EN MCI_STATUS_EN
-#define PCC MCI_STATUS_PCC
-#define S MCI_STATUS_S
-#define AR MCI_STATUS_AR
-#define UC MCI_STATUS_UC
-
-int ring = 3;
-int fail;
-
-void test2(u64 flag, char *flagname, u64 mcg, char *mcgname, int result)
-{
-	struct mce m = {
-		.ip = 1,
-		.cs = ring,
-		.status = flag,
-		.mcgstatus = mcg,
-	};
-	int r;
-	char *msg;
-
-	if ((r = mce_severity(&m, tolerant, &msg)) != result) { 
-		printf("%s %s expected %s got %s msg %s\n",
-		       flagname, mcgname, resname[result], resname[r], msg);	
-		fail++;
-	}
-}
-
-
-#define TEST(flag, result) \
-	test2(flag, #flag, MCG_STATUS_MCIP|MCG_STATUS_RIPV, "mcip,ripv", \
-		MCE_ ## result ## _SEVERITY)
-
-void test(void)
-{
-	// corrected
-	TEST(VAL|EN, KEEP);
-
-	// uncorrected fatal
-	TEST(VAL|UC|PCC|EN|S|AR, PANIC);
-	TEST(VAL|UC|PCC|EN|S, PANIC);
-	TEST(VAL|UC|PCC|EN, PANIC);
-
-	// SW recoverable action required
-	// unknown mcacod -> panic
-	TEST(VAL|UC|EN|S|AR, PANIC);
-
-	// SW recoverable action optional
-	TEST(VAL|UC|EN|S|0xc0, AO);
-	// unknown mcacod
-	TEST(VAL|UC|EN|S|1, SOME);
-
-	// UCNA
-	TEST(VAL|UC|EN, KEEP);
-	TEST(VAL|UC, NO);	// linux clears. correct?	
-}
-
-int main(void)
-{
-	ring = 3;
-	test();
-	ring = 0;
-	test();
-	if (fail == 0)
-		printf("SUCCESS\n");
-	else
-		printf("%d FAILURES\n", fail);
-	return fail;
-}
diff --git a/tsrc/tring.c b/tsrc/tring.c
deleted file mode 100644
index 0ed14f5..0000000
--- a/tsrc/tring.c
+++ /dev/null
@@ -1,87 +0,0 @@
-/* Unit tester for ring buffer code in mce.c */
-#define DEFINE_PER_CPU(a,b) a b
-#define __get_cpu_var(x) x
-#define barrier() asm volatile("" ::: "memory")
-#define rmb() barrier()
-#define wmb() barrier()
-
-/*
- * Simple lockless ring to communicate PFNs from the exception handler with the
- * process context work function. This is vastly simplified because there's
- * only a single reader and a single writer.
- */
-#define MCE_RING_SIZE 16	/* we use one entry less */
-
-struct mce_ring {
-	unsigned short start;
-	unsigned short end;
-	unsigned long ring[MCE_RING_SIZE];
-};
-static DEFINE_PER_CPU(struct mce_ring, mce_ring);
-
-static int mce_ring_empty(void)
-{
-	struct mce_ring *r = &__get_cpu_var(mce_ring);
-
-	return r->start == r->end;
-}
-
-static int mce_ring_get(unsigned long *pfn)
-{
-	struct mce_ring *r = &__get_cpu_var(mce_ring);
-
-	if (r->start == r->end)
-		return 0;
-	*pfn = r->ring[r->start];
-	r->start = (r->start + 1) % MCE_RING_SIZE;
-	return 1;
-}
-
-static int mce_ring_add(unsigned long pfn)
-{
-	struct mce_ring *r = &__get_cpu_var(mce_ring);
-	unsigned next;
-
-	next = (r->end + 1) % MCE_RING_SIZE;
-	if (next == r->start)
-		return -1;
-	r->ring[r->end] = pfn;
-	wmb();
-	r->end = next;
-	return 0;
-}
-
-#include <stdio.h>
-#include <assert.h>
-#include <pthread.h>
-
-void *thread(void *arg)
-{
-	long i = 0;
-	for (;;) { 
-		if (mce_ring_add(i) >= 0)
-			i++;
-	}
-}
-
-int main(void)
-{
-	long k;
-	
-	pthread_t thr;
-	pthread_create(&thr, NULL, thread, NULL);
-
-	k = 0;
-	for (;;) { 	
-		while (!mce_ring_empty()) { 
-			unsigned long pfn;
-			int r = mce_ring_get(&pfn);
-			assert(r != 0);
-			if (pfn != k) 
-				printf("got %lu expected %lu delta %ld\n", pfn, k, k-pfn);
-			k++;
-		}
-	}
-
-	return 0;
-}
diff --git a/tsrc/ttable.c b/tsrc/ttable.c
deleted file mode 100644
index c3837bf..0000000
--- a/tsrc/ttable.c
+++ /dev/null
@@ -1,260 +0,0 @@
-/* 
- * Print table of MCA status bit combinations with results in HTML.
- * Author: Andi Kleen
- */
-#define _GNU_SOURCE 1
-#include <stdio.h>
-#include <stdlib.h>
-#include <stddef.h>
-#include <string.h>
-#include <assert.h>
-#include <unistd.h>
-#include <errno.h>
-#define __KERNEL__ 1
-#include <asm/types.h>
-#include <asm/mce.h>
-
-#define ARRAY_SIZE(x) (sizeof(x)/sizeof(*(x)))
-
-typedef unsigned long long u64;
-
-
-#define MCI_STATUS_S	 (1ULL<<56)  /* Signaled machine check */
-#define MCI_STATUS_AR	 (1ULL<<55)  /* Action required */
-
-int tolerant = 1;
-int panic_on_oops = 0;
-int mce_ser = 1;
-
-#include "mce-severity.c"
-
-int disable_opt = 0;
-
-struct rname { 
-	char *name;
-	unsigned color;
-	char *desc;
-} rnames[] = {
-#define R(x,col,d) [MCE_ ## x ## _SEVERITY] = { #x, col, d }
-	R(NO, 0xc0c0c0, "Ignored"),
-	R(KEEP, 0x800080, "Ignore. Keep for CMC"),
-	R(SOME, 0x808080, "Log & Clear"),
-	R(AO, 0xffff00, "Kill address owner"),
-	R(UC, 0x700000, "Kill or panic"),
-	R(AR, 0x00ff00, "Kill current context"),
-	R(PANIC, 0xff0000, "Shutdown"),
-#undef R
-};
-
-struct bit { 
-	char *name;
-	unsigned offset;
-	u64 bit;
-} bits[] = { 
-#define O(x) offsetof(struct mce, x)
-#define S(x) { #x, O(status), MCI_STATUS_ ## x }
-	{ "RIPV", O(mcgstatus), MCG_STATUS_RIPV },
-	{ "EIPV", O(mcgstatus), MCG_STATUS_EIPV },
-	{ "MCIP", O(mcgstatus), MCG_STATUS_MCIP },
-	S(EN),
-	S(VAL),
-	S(UC),
-	S(S),
-	S(AR),
-	S(PCC),
-	S(OVER),
-	{ "SCRB-ERR", O(status), 0xc0 },
-#undef S
-#undef O
-};
-
-struct mce basem;
-
-#define bit_for_each(i,v) for (i = 0; i < 64; i++) if ((v) & (1ULL << i)) 
-
-struct result { 
-	int res;
-	unsigned dontcare;
-	char *msg;
-};
-
-void genstate(struct mce *m, unsigned num)
-{
-	int i;
-	*m = basem;
-	
-	bit_for_each (i, num)
-		*(u64 *)((char *)m + bits[i].offset) |= bits[i].bit;
-}
-
-// find don't care bits
-// brute force version because andi is not clever enough to make the clever 
-// version work. luckily the tables are small
-
-#define for_rr(start, i) for (i = start; i < num; i++) if (rr[i].res >= 0) 
-#define mask_of(x) ((1U << (x))-1)
-
-static void disable(struct result *rr, int i, int src)
-{
-	//fprintf(stderr, "disabling %d from %d\n", i, src);
-	rr[i].res = -1;
-}
-
-// handle case: one bit set always the same outcome
-static void one_bit_all(struct result *rr, int num, int mask)
-{
-	int first, k;
-	if (mask >= num)
-		return;
-	first = mask;
-	for_rr (first, k) { 
-		if (!(k & mask))
-			continue;
-		if (rr[k].res != rr[first].res)
-			return;
-	}
-	rr[first].dontcare = mask_of(ARRAY_SIZE(bits)) & ~mask;
-	for_rr (first + 1, k) { 
-		if (k & mask)
-			disable(rr, k, k);
-	}
-}
-
-// check if toggling one bit gives the same outcome
-static void neighbour_same(struct result *rr, int num, int mask)
-{
-	int k, other;
-	for_rr (mask, k) { 
-		if (!(k & mask) || (rr[k].dontcare & mask))
-			continue;
-		other = k ^ mask;
-		if (other >= num)
-			continue;
-		if (rr[other].res == rr[k].res && rr[other].msg == rr[k].msg) { 
-			disable(rr, other, k);
-			rr[k].dontcare |= mask;
-		}
-	}
-}
-
-void optimizer(struct result *rr, int num)
-{
-	int i;
-
-	for (i = 1; i <= 1 << ARRAY_SIZE(bits); i <<= 1)
-		one_bit_all(rr, num, i);
-	for (i = 1; i <= 1 << ARRAY_SIZE(bits); i <<= 1)
-		neighbour_same(rr, num, i);
-}
-
-int bitcount(u64 v)
-{
-	int num = 0;
-	while (v) {
-		if (v & 1)
-			num++;
-		v >>= 1;
-	}
-	return num;
-}
-
-void table(char *title)
-{
-	struct mce m;
-	int i, w, num; 
-	
-	struct result *rr = calloc(sizeof(struct result), 1U << ARRAY_SIZE(bits));
-
-	num = 0;
-	for (i = 0; i < 1U << ARRAY_SIZE(bits); i++) {
-		genstate(&m, i);
-		rr[num].res = mce_severity(&m, tolerant, &rr[num].msg);
-		num++;
-	}
-
-	if (!disable_opt)
-		optimizer(rr, num);
-	
-	printf("<p><table border=1>\n");
-	printf("<chaption>%s</chaption>\n", title);
-
-	printf("<tr>\n");	
-	for (i = 0; i < ARRAY_SIZE(bits); i++) { 
-		printf("<th>%s</th>", bits[i].name);
-	}
-	printf("<th>Result</th><th>Rule</th><th>Action</th>\n");
-	printf("</tr>\n");
-
-	for_rr (0, i) { 
-		printf("<tr>");
-		for (w = 0; w < ARRAY_SIZE(bits); w++) { 
-			char *p = "0"; 
-			char *col = "";
-			unsigned mask = 1U << w;
-
-			if (mask & rr[i].dontcare) {
-				p = "x";
-				col = " bgcolor=\"888888\"";
-			} else if (mask & i) { 
-				if (bitcount(bits[w].bit) > 1) 
-					asprintf(&p, "%llx", bits[w].bit);
-				else
-					p = "1";
- 				col = " bgcolor=\"ffffff\"";
-			}
-			printf("<td%s>%s</td>", col, p);
-		}
-		struct rname *rname = &rnames[rr[i].res];
-		if ((unsigned)rr[i].res >= ARRAY_SIZE(rnames))
-			rname = &((struct rname) { .name = "out of bounds", .color = 0xff00ff });
-		assert(rname->name != NULL);
-		printf("<td bgcolor=\"%06x\">%s</td>", rname->color, rname->name);
-		assert(rr[i].msg != NULL);
-		printf("<td>%s</td>", rr[i].msg);
-		printf("<td>%s</td>", rname->desc);
-		printf("</tr>\n");	
-	}
-	printf("</table>\n");
-}
-
-void usage(void)
-{
-	fprintf(stderr, "ttable [-a]\n"
-			"-a don't print don't care bits, but all states\n");
-	exit(1);
-}
-
-int main(int ac, char **av)
-{
-	int opt;
-	while ((opt = getopt(ac, av, "a")) != -1) { 
-		switch (opt) { 
-		case 'a': 
-			disable_opt = 1;
-			break;
-		default:
-			usage();
-		}
-	}
-
-	printf("<html><body>\n");
-	printf("<!-- Auto generated. Changes will be overwritten -->\n");
-	basem.ip = 1;
-	printf("<h1>Linux kernel machine check grading</h1>\n");
-	printf("Caveats: Only scrubber error AO MCACOD. Only applies to exceptions.\n");
-	mce_ser = 1;
-	basem.cs = 0;
-	table("With MCA recovery ring 0");
-	tolerant = 0;
-	table("With MCA recovery ring 0 tolerant = 0");
-	tolerant = 1;
-	basem.cs = 3;
-	table("With MCA recovery ring 3");
-	basem.cs = 0;
-	mce_ser = 0;
-	table("Without MCA recovery ring 0");
-	basem.cs = 3;
-	table("Without MCA recovery ring 3");
-	printf("</body></html>\n");
-	return 0;
-}