Scientific Benchmarking with Temporal Logic Decision Procedures

Hustadt, U. and Schmidt, R. A. (2002)

In Fensel, D. and Giunchiglia, F. and McGuinness, D. and Williams, M.-A. (eds), Principles of Knowledge Representation and Reasoning: Proceedings of the Eighth International Conference (KR'2002). Morgan Kaufmann, 533-544. BiBTeX, PDF.

In this paper we propose a hypothesis-driven design of the empirical analysis of different decision procedures which we refer to as scientific benchmarking. The approach is to start by choosing the benchmark problems for which, on the basis of analytical considerations, we expect a particular decision procedure to exhibit a behaviour different from another decision procedure. Then empirical tests are performed in order to verify the particular hypothesis concerning the decision procedures under consideration. As a case study, we apply this methodology to compare different decision procedures for propositional temporal logic. We define two classes of randomly generated temporal logic formulae which we use to investigate the behaviour of two tableaux-based temporal logic approaches using the Logics Workbench, a third tableaux-based approach using the STeP system, and temporal resolution using a new prover called TRP.



Renate A. Schmidt
Home | Publications | Tools | FM Group | School | Man Univ

Last modified: 02 Oct 05
Copyright © 2002 Renate A. Schmidt, School of Computer Science, Man Univ, schmidt@cs.man.ac.uk