23 September 2011 2011 Alasdair J. G. Gray Jason Sadler Oles Kit Kostis Kyzirakos Manos Karpathiotakis Jean-Paul Calbimonte Kevin Page Raul Garcia-Castro Alex Frazer Ixent Galpin Alvaro A. A. Fernandes Norman W. Paton Oscar Corcho Manolis Koubarakis David De Roure Kirk Marinez Asuncion Gomez-Perez http://www.mdpi.com/1424-8220/11/9/8855/ Sensors 11(9) 8855-8887 sensors2011 Sensing devices are increasingly being deployed to monitor the physical world around us. One class of application for which sensor data is pertinent is environmental decision support systems, e.g., flood emergency response. For these applications, the sensor readings need to be put in context by integrating them with other sources of data about the surrounding environment. Traditional systems for predicting and detecting floods rely on methods that need significant human resources. In this paper we describe a semantic sensor web architecture for integrating multiple heterogeneous datasets, including live and historic sensor data, databases, and map layers. The architecture provides mechanisms for discovering datasets, defining integrated views over them, continuously receiving data in real-time, and visualising on screen and interacting with the data. Our approach makes extensive use of web service standards for querying and accessing data, and semantic technologies to discover and integrate datasets. We demonstrate the use of our semantic sensor web architecture in the context of a flood response planning web application that uses data from sensor networks monitoring the sea-state around the coast of England. Ixent Galpin Robert Taylor Alasdair J. G. Gray Christian C. Y. Brenninkmeijer Alvaro A. A. Fernandes Norman W. Paton July 2011 Manchester, UK proceedings 28th British National Conference on Databases (BNCOD 2011) George Valkanas Alexis Kotsifakos Dimitrios Gunopulos Ixent Galpin Alasdair J. G. Gray Alvaro A. A. Fernandes Norman W. Paton June 2011 Lulea, Sweeden proceedings 12th International Conference on Mobile Data Management (MDM 2011) Alasdair J. G. Gray Raul Garcia-Castro Kostis Kyzirakos Manos Karpathiotakis Jean-Paul Calbimonte Kevin Page Jason Sadler Alex Frazer Ixent Galpin Alvaro A. A. Fernandes Norman W. Paton Oscar Corcho Manolis Koubarakis David De Roure Kirk Marinez Asuncion Gomez-Perez June 2011 Heraklion (Crete, Greece) http://dx.doi.org/10.1007/978-3-642-21064-8_21 300-314 proceedings 8th Extended Semantic Web Conference (ESWC 2011) Publications/eswc2011Presentation.pptx eswc2011 Sensing devices are increasingly being deployed to monitor the physical world around us. One class of application for which sensor data is pertinent is environmental decision support systems, e.g. flood emergency response. However, in order to interpret the readings from the sensors, the data needs to be put in context through correlation with other sensor readings, sensor data histories, and stored data, as well as juxtaposing with maps and forecast models. In this paper we use a flood emergency response planning application to identify requirements for a semantic sensor web. We propose a generic service architecture to satisfy the requirements that uses semantic annotations to support well-informed interactions between the services. We present the SemSorGrid4Env realisation of the architecture and illustrate its capabilities in the context of the example application. Oscar Corcho Alasdair J. G. Gray Kostis Kyzirakos Jean-Paul Calbimonte Kevin Page June 2011 Heraklion (Crete, Greece) http://semsorgrid4env.eu/index.php/events-meetings/49-building-semantic-sensor-webs-and-applications-eswc-2011 proceedings 8th Extended Semantic Web Conference (ESWC 2011) eswctutorial2011 The term Semantic Sensor Web has recently been used to refer to the combination of sensor network, Web and semantic technologies with the view of addressing the opportunity that we have to unify the real and the virtual world. Sensor networks promise to bridge the gap that, for too long, has separated computing applications from the physical world that they model and in which they are ultimately embedded. Many scientific and technological challenges need to be tackled before sensor networks can be exploited to their full capacity. Additionally, as more and more sensor networks are independently developed and deployed, it becomes increasingly important to support their reuse in applications that were not foreseen or that transcend their original purpose. This will facilitate the use of sensor web technology to support decision-making that requires on-the-fly integration of data of differing modalities, e.g. sensed data with data stored in databases, as well as the ad hoc generation of mashups over data stemming from computations that combine real-time and legacy historical data. This, in turn, will enable the enacting of decisions based on such real-time sensed data. This tutorial aims at presenting the components that comprise a semantic sensor web and the mechanisms to exploit these in creating an application. The tutorial targets researchers and practitioners interested in working with semantic sensor web technologies. April 2011 Publications/iclSeminar2011.pptx Seminar talk given to the Large-Scale Distributed Systems Research Group at Imperial College London iclSeminar2011 The availability of streaming data sources is progressively increasing thanks to the development of ubiquitous data capturing technologies such as sensor networks, which are increasingly being deployed to monitor the physical world around us. One class of application for which sensor data is pertinent is environmental decision support systems, e.g. flood emergency response. For these applications, the sensor readings need to be put in context by integrating them with other sources of data about the surrounding environment. In this talk we describe a semantic sensor web architecture that enables the integration of multiple heterogeneous datasets, including live and historic sensor data, databases, and map layers. The architecture provides mechanisms for discovering datasets, defining integrated views over them, and visualising on screen and interacting with the data. The heterogeneity of the data sources introduces the requirement of providing data access in a unified and coherent manner, whilst allowing the user to express their needs at an ontological level. We shall present the mechanisms by which we support ontology-based streaming data integration. Finally, we will demonstrate the use of our semantic sensor web architecture and data integration approach in the context of a flood response planning web application that uses data from sensor networks monitoring the sea-state around the coast of England. 2011 Ixent Galpin Christian Y A Brenninkmeijer Alasdair J G Gray Farhana Jabeen Alvaro A A Fernandes Norman W Paton http://www.springerlink.com/content/2314991240243387/ Distributed and Parallel Databases 29(1-2) 31-85 dpdb2011 A wireless sensor network (WSN) can be construed as an intelligent, large-scale device for observing and measuring properties of the physical world. In recent years, the database research community has championed the view that if we construe a WSN as a database (i.e., if a significant aspect of its intelligent behavior is that it can execute declaratively-expressed queries), then one can achieve a significant reduction in the cost of engineering the software that implements a data collection program for the WSN while still achieving, through query optimization, very favorable cost:benefit ratios. This paper describes a query processing framework for WSNs that meets many desiderata associated with the view of WSN as databases. The framework is presented in the form of compiler/optimizer, called SNEE, for a continuous declarative query language over sensed data streams, called SNEEql. SNEEql can be shown to meet the expressiveness requirements of a large class of applications. SNEE can be shown to generate effective and efficient query evaluation plans. More specifically, the paper describes the following contributions: (1) a user-level syntax and physical algebra for SNEEql, an expressive continuous query language over WSNs; (2) example concrete algorithms for physical algebraic operators defined in such a way that the task of deriving memory, time and energy analytical cost-estimation models (CEMs) for them becomes straightforward by reduction to a structural traversal of the pseudocode; (3) CEMs for the concrete algorithms alluded to; (4) an architecture for the optimization of SNEEql queries, called SNEE, building on well-established distributed query processing components where possible, but making enhancements or refinements where necessary to accommodate the WSN context; (5) algorithms that instantiate the components in the SNEE architecture, thereby supporting integrated query planning that includes routing, placement and timing; and (6) an empirical performance evaluation of the resulting framework. Jean-Paul Calbimonte Oscar Corcho Alasdair J. G. Gray November 2010 Shanghai, China iswc2010.pdf 96-111 proceedings 9th International Semantic Web Conference (ISWC 2010) Publications/iswc2010Presentation.pptx iswc2010 The availability of streaming data sources is progressively increasing thanks to the development of ubiquitous data capturing technologies such as sensor networks. The heterogeneity of these sources introduces the requirement of providing data access in a unified and coherent manner, whilst allowing the user to express their needs at an ontological level. In this paper we describe an ontology-based streaming data access service. Sources link their data content to ontologies through S2O mappings. Users can query the ontology using SPARQLStream, an extension of SPARQL for streaming data. A preliminary implementation of the approach is also presented. With this proposal we expect to set the basis for future efforts in ontology-based streaming data integration. Alasdair J. G. Gray July 2010 Dundee, UK bncod2010.pdf proceedings 27th British National Conference on Databases (BNCOD 2010) Publications/bncod2010Presentation.pptx bncod2010 The British National Conference on Databases (BNCOD), now in its 27th edition, has covered a broad range of database research topics: from the purely theoretical, to more application-oriented subjects. It has proved to be a forum for intellectual debate, and has fostered a sense of community amongst British and overseas database researchers. Databases have been incredibly successful with most businesses relying on them. However, there are still plenty of challenges that remain to be solved. This paper reflects back on the successes of the BNCOD series, and identifies the challenges that remain, thus showing the continuing importance of the BNCOD conference series. 2010 Alasdair J G Gray Norman Gray Christopher W Hall Iadh Ounis http://dx.doi.org/10.1016/j.ipm.2009.09.004 Information Processing and Management 46(4) 470-478 ipm2010 Astronomy, like many domains, already has several sets of terminology in general use, referred to as controlled vocabularies. For example, the keywords for tagging journal articles, or the taxonomy of terms used to label image files. These existing vocabularies can be encoded into skos, a W3C proposed recommendation for representing vocabularies on the Semantic Web, so that computer systems can help users to search for and discover resources tagged with vocabulary concepts. However, this requires a search mechanism to go from a user-supplied string to a vocabulary concept. In this paper, we present our experiences in implementing the Vocabulary Explorer, a vocabulary search service based on the Terrier Information Retrieval Platform. We investigate the capabilities of existing document weighting models for identifying the correct vocabulary concept for a query. Due to the highly structured nature of a skos encoded vocabulary, we investigate the effects of term weighting (boosting the score of concepts that match on particular fields of a vocabulary concept), and query expansion. We found that the existing document weighting models provided very high quality results, but these could be improved further with the use of term weighting that makes use of the semantic evidence April 2010 Publications/cardiffSeminar2010.pptx Seminar talk given to the Knowledge and Information Systems research group at the University of Cardiff cardiffSeminar2010 The availability of streaming data sources is progressively increasing thanks to the development of ubiquitous data capturing technologies such as sensor networks. Users of streaming data often need to combine multiple sources with widely different capabilities: streams of data from sensors with extremely limited resources, streams of data from robust network sources, historic archives of stream data, and stored data from databases. The availability of resources in the last three types of sources is seemingly unlimited when compared with the wireless sensor networks. We propose distributed query processing as a solution to the problem of co-ordinating access to the distributed data sources in an efficient and timely manner whilst considering the capabilities of the sources and the characteristics of the data. Several challenges that must be overcome, in particular different query processing capabilities, different data characteristics, and the availability of resources for processing and storing data. Another key challenge is that there is no standardised query language for incorporating data from pull-stream, push-stream, and stored sources. In this talk we present SNEEql as a declarative query language, which has a well defined evaluation semantics, for expressing data needs in a unified way over such data sources. We then show how the execution of a SNEEql query is divided into parts that are evaluated over the relevant data sources. August 2009 Publications/imgSeminar2009-explicator.pptx Seminar talk given to the Information Management Group at the University of Manchester imgSeminar2009 In this seminar I will report on the work conducted in the initial 18 months of the Explicator project which aims to use semantic web tools to integrate astronomy data. We addressed two of the key challenges when integrating data from multiple heterogeneous sources: (i) locating those data sources that contain data of interest, and (ii) understanding the data models in order to pose a query to extract the required data. For the first of these challenges, we investigated how to search for relevant sources using SKOS encoded vocabularies and the information retrieval techniques implemented in the Terrier Platform. For the second we investigated the use of RDB2RDF tools for exposing existing relational data sources. June 2009 Publications/bozen-seminar-2009.pptx Seminar talk given to the Knowledge Representation meets Databases group at the Free University of Bozen-Bolzano bozenSeminar2009 There are two key challenges when integrating data from multiple heterogeneous sources: (i) locating those data sources that contain data of interest, and (ii) understanding the data models in order to pose a query to extract the required data. We will illustrate these problems with examples drawn from astronomy and environmental monitoring; and examine how Semantic Web technologies may be employed to solve them. In particular, we will consider how an astronomer can locate and access data using terminology and a data model with which they are familiar by making use of SKOS encoded vocabularies, RDB2RDF tools, and semantic mappings. We will also consider how streams of data from sensor networks can be combined with tradiational data sources and accessed using ontologies. Alasdair J. G. Gray Norman Gray Iadh Ounis June 2009 Heraklion (Crete, Greece) eswc2009.pdf 491-505 proceedings 6th European Semantic Web Conference (ESWC 2009) Publications/eswc2009Presentation.pptx eswc2009 Many science archive centres publish very large volumes of image, simulation, and experiment data. In order to integrate and analyse the available data, scientists need to be able to (i) identify and locate all the data relevant to their work; (ii) understand the multiple heterogeneous data models in which the data is published; and (iii) interpret and process the data they retrieve. RDF has been shown to be a generally successful framework within which to perform such data integration work. It can be equally successful in the context of scientific data, if it is demonstrably practical to expose that data as RDF. In this paper we investigate the capabilities of RDF to enable the integration of scientific data sources. Specifically, we discuss the suitability of SPARQL for expressing scientific queries, and the performance of several triple stores and RDB2RDF tools for executing queries over a moderately sized sample of a large astronomical data set. We found that more research and improvements are required into SPARQL and RDB2RDF tools to efficiently expose existing science archives for data integration. A.J.G. Gray N. Gray I. Ounis February 2009 Exploiting Semantic Annotations in Information Retrieval (ESAIR 2009) Barcelona (Spain) 1-5 Publications/esair2009.pdf esair2009 Within most domains of discourse, there exists different terminology used by distinct sub-groups. Often the terms used can be, or have already been, organised into controlled vocabularies which can be encoded into SKOS, a W3C standard for representing vocabularies. This terminology can then be used to help users to search for and discover resources. This requires a search mechanism to go from a user-supplied string to a vocabulary concept. In this paper, we discuss the issues encountered in developing a web service for searching and exploring the concepts in SKOS encoded astronomical vocabularies. Our prototype service takes in a query and responds with the concepts which are the best match. It then supports the user in exploring the concepts' formal definition and alternative forms, as well as their relationship to other concepts. When we add mappings between the concepts in different vocabularies (where available), these further enrich the explorations of vocabulary concepts. A.J.G. Gray N. Gray I. Ounis November 2008 Québec City (Canada) proceedings of 18th Astronomical Data Analysis Software and Systems conference (ADASS XVIII) Publications/adass2008.pdf Publications/adass2008Presentation.pptx adass2008 There are multiple controlled vocabularies, or thesauri, within astronomy, of which the best known are the 1993 IAU thesaurus and the keyword list maintained by A&A, ApJ and MNRAS. Over the last year, there has been an effort within the IVOA to formalise the presentation of these vocabularies into a form that will allow greater automated interaction with them, in particular on the Web. The IVOA has adopted the W3C SKOS standard, which makes links with the Semantic Web and looks forward to richer applications using the technologies of that domain. We will summarise this work, and highlight some of the immediate and medium-term possibilities of vocabularies and their more sophisticated cousins, ontologies. Vocabulary aware applications can benefit from improvements in both precision and recall when searching for bibliographic or science data, and lightweight intelligent filtering for services such as VOEvent streams. An important part of this is mapping between precisely or roughly comparable terms in two vocabularies or, in the longer term, two analogous but distinct data models. In particular, we will describe and demonstrate our VocabularyExplorer application, and its companion Mapping Editor. These combine Semantic Web and Information Retrieval technologies to (i) illustrate the way in which formal vocabularies might be used in a practical application, (ii) provide an online service which will allow astronomers to explore and relate existing vocabularies, and (iii) provide an embeddable service to help applications translate free text user queries into formal vocabulary terms. A.J.G. Gray N. Gray I. Ounis September 2008 Edinburgh (Scotland) proceedings of the UK All Hands Meeting (AHM 2008) Publications/ahmAbstract2008.pdf Publications/ahm2008Presentation.pptx A.J.G. Gray N. Gray I. Ounis proceedings of the British National Conference on Databases (BNCOD 25) July 2008 Cardiff (Wales) 189-192 Publications/bncod25.pdf bncod25 One problem faced by astronomers using the virtual observatory is finding which of the multitude of data resources is relevant for them. The current tool, VOExplorer, relies on matching searches against tags provided by the resources. This paper shows how skos encoded vocabularies can be used to improve the search results. The techniques are general and applicable to any loose collaborations sharing their resources. November 2008 http://www.ivoa.net/Documents/latest/Vocabularies.html A.J.G. Gray N. Gray F.V. Hessmann A. Preite Martinez (Eds) IVOA Proposed Recommendation vocab-standard As the astronomical information processed within the Virtual Observatory becomes more complex, there is an increasing need for a more formal means of identifying quantities, concepts, and processes not confined to things easily placed in a FITS image, or expressed in a catalogue or a table. This document specifies a standard format for vocabularies based on the W3C's Resource Description Framework (RDF) and Simple Knowledge Organization System (SKOS). By adopting a standard and simple format, the IVOA will permit different groups to create and maintain their own specialised vocabularies while letting the rest of the astronomical community access, use, and combine them. The use of current, open standards ensures that VO applications will be able to tap into resources of the growing semantic web. Several examples of useful astronomical vocabularies are provided, including work on a common IVOA thesaurus intended to provide a semantic common base for VO applications. December 2007 Publications/vocabulariesInAstronomy.pdf Alasdair J. G. Gray Norman Gray Paul Millar Iadh Ounis Technical report, University of Glasgow tech-tep-glasgow Astronomy as a discipline incorporates a broad range of topics, from x-rays to radio, and a wide range of expertise, from professional researchers via amateurs to popular interest. As a result, multiple independent controlled vocabularies have evolved to meet the various needs of these different user groups. Unfortunately, several of these vocabularies and thesauri are only semi-formal, and often lack significant cross-application support. As part of the international Virtual Observatory interoperability effort, we are developing techniques to enable application support for astronomers to query resources described in any vocabulary using one with which they are familiar. In this paper, we describe our experience of formalising and linking astronomy vocabularies via peer-to-peer mappings using SKOS, a language for modelling knowledge organisation systems. We also present our findings on the types of mappings required to relate the concepts of the vocabularies and describe the relationship with the Virtual Observatory's ontology developments. October 2007 Publications/thesis-final_web-copy.pdf Alasdair J. G. Gray PhD Thesis thesis There is an increasing amount of information being made available as data streams, e.g. stock tickers, data from sensor networks, smart homes, monitoring data, etc. In many cases, this data is generated by distributed sources under the control of many different organisations. Users would like to seamlessly query such data without prior knowledge of where it is located or how it is published. This is similar to the problem of integrating data residing in multiple heterogeneous stored data sources. However, the techniques developed for stored data are not applicable due to the continuous and long-lived nature of queries over data streams. This thesis proposes an architecture for a stream integration system. A key feature of the architecture is a republisher component that collects together distributed streams and makes the merged stream available for querying. A formal model for the system has been developed and is used to generate plans for executing continuous queries which exploit the redundancy introduced by the republishers. Additionally, due to the long-lived nature of continuous queries, mechanisms for maintaining the plans whenever there is a change in the set of data sources have been developed. A prototype of the system has been implemented and performance measures made. The work of this thesis has been motivated by the problem of retrieving monitoring information about Grid resources. However, the techniques developed are general and can be applied wherever there is a need to publish and query distributed data involving data streams. 2007 Alasdair J G Gray Werner Nutt Howard Williams Publications/ijwis2007.pdf International Journal on Web Information Systems 3(1/2) 41-60 ijwis2007 Purpose of this paper Distributed data streams are an important topic of current research. In such a setting, data values will be missed, e.g. due to network errors. This paper allows this incompleteness to be detected and overcome with either the user not being affected or the effects of the incompleteness being reported to the user. Design/Methodology/Approach A model for representing the incomplete information has been developed that captures the information that is known about the missing data. Techniques for query answering involving certain and possible answer sets have been extended so that queries over incomplete data stream histories can be answered. Findings It is possible to detect when a distributed data stream is missing one or more values. When such data values are missing there will be some information that is known about the data and this is stored in an appropriate format. Even when the available data is incomplete, it is possible in some circumstances to answer a query completely. When this is not possible, additional meta-data can be returned to inform the user of the effects of the incompleteness. Research limitations/implications The techniques and models proposed in this paper have only been partially implemented. Practical Implications The proposed system is general and can be applied wherever there is a need to query the history of distributed data streams. The work in this paper enables the system to answer queries when there are missing values in the data. What is original/value of paper This paper presents a general model of how to detect, represent, and answer historical queries over incomplete distributed data streams. Alasdair J G Gray M Howard Williams Werner Nutt Publications/iiwas2006.pdf December 2006 Publications/iiwas2006talk.ppt Information Integration and Web-based Application and Services (iiWAS2006) 259-268 Yogyakarta (Indonesia) iiWAS2006 R-GMA is a system that integrates distributed data streams to provide a global view of all the streams. A user can pose a query to R-GMA and locate data of interest without needing to know where it is located. R-GMA has been developed as a grid information and monitoring system although the techniques developed can be applied wherever there is a need to publish and query distributed streams. Knowing the values that have occurred on a stream in the past can often lead to a better understanding of how it might behave in the future. For example, in the grid monitoring context this information can be used to spot anomalous behaviour. In order to support this, the history of the streams must be archived. Data streams published by distributed sources are prone to missing data values, e.g. due to a network failure. Since the stream has missed some values, the stored history of the stream contains gaps. This paper considers how to generate the most complete answer possible to a positive conjunctive query over the available stream history. A model for representing the incompleteness in the stream history is provided along with an algorithm that distinguishes when and how the missing data affects the answer to a query. Alasdair J G Gray Werner Nutt M Howard Williams http://www.springerlink.com/content/r7724n1732352g30/?p=78e05af5f2ce4cfeb1c56f212ae3b9cd July 2006 Publications/bncod23talk.pdf British National Conference on Databases (BNCOD23) 94-101 Belfast (UK) bncod23 There is a wide variety of data, both static and streaming, being published on and about computing Grids. However one aspect of Grid data that has received relatively little attention thus far is that of incompleteness. With potentially many data sources available to choose from, each with its own limitations in terms of coverage of data sets and of overall reliability, there are many opportunities for inaccuracy due to incompleteness in the data. In this short paper different types of incompleteness are identified in the context of R-GMA as part of an ongoing research project aimed at finding solutions to some of these. Alasdair J G Gray Werner Nutt November 2005 Publications/coopis05.pdf International Conference on Cooperative Information Systems (CoopIS 2005) Publications/coopis05Talk.ppt 420-438 Agia Napa (Cyprus) coopis05 In data stream applications, streams typically arise from a geographically distributed collection of producers and may be queried by consumers, which may be distributed as well. In such a setting, a query can be seen as a subscription asking to be informed of all tuples that satisfy a specific condition. We propose to support the publishing and querying of distributed data streams by a publish/subscribe architecture. To enable such a system to scale to a large number of producers and consumers requires the introduction of republishers which collect together data streams and make the merged stream available. If republishers consume from other republishers, a hierarchy of republishers results. We present a formalism that allows distributed data streams, published by independent stream producers, to be integrated as views on a mediated schema. We use the formalism to develop methods to adapt query plans to changes in the set of available data streams and allow consumers to dynamically change which streams they subscribe to. April 2005 Publications/HW-MACS-TR-0031.pdf Alasdair J G Gray Werner Nutt This technical report gives the formal arguments of the ideas presented in our BNCOD22 paper (see conferences above) techRep31 In data stream applications, streams typically arise from a geographically distributed collection of producers and may be queried by consumers, which may be distributed as well. In such a setting, a query can be seen as a subscription asking to be informed of all tuples that satisfy a specific condition. We propose to support the publishing and querying of distributed data streams by a publish/subscribe architecture. To enable such a system to scale to a large number of producers and consumers requires the introduction of republishers which collect together data streams and make the merged stream available. If republishers consume from other republishers, a hierarchy of republishers results. We present a formalism that allows distributed data streams, published by independent stream producers, to be integrated as views on a mediated schema. We then use this formalism to develop query planning techniques for the republishers, and methods to adapt plans when the set of available data streams changes. Alasdair J G Gray Werner Nutt July 2005 Publications/bncod22.pdf British National Conference on Databases (BNCOD22) Publications/bncod22talk.ppt 179-184 Sunderland (UK) bncod22 We present a publish/subscribe framework for integrating data streams published by distributed producers. We introduce the idea of republishers which merge a set of data streams, either from producers or other republishers. The resulting hierarchy of producers and republishers can then be used to answer consumer queries over the streams. We discuss how to compute query plans to create such a hierarchy and the maintenance of these plans when the set of streams changes. Rob Byrom Brian A. Coghlan Andrew W. Cooke Roney Cordenonsi Linda Cornwall Martin Craig Abdeslem Djaoui Alastair Duncan Steve Fisher Alasdair J. G. Gray Steve Hicks Stuart Kenny Jason Leake Oliver Lyttleton James Magowan Robin Middleton Werner Nutt David O'Callaghan Norbert Podhorszki Paul Taylor John Walk Antony J. Wilson http://www.springerlink.com/content/57t3lrhcqu4x39m1/ February 2005 European Grid Conference (ECG 2005) 751-760 Amsterdam (Netherlands) ecg2005 R-GMA (Relational Grid Monitoring Architecture) [1] is a grid monitoring and information system that provides a global view of data distributed across a grid system. R-GMA creates the impression of a single centralised repository of information, but in reality the information can be stored at many different locations on the grid. The Registry and Schema are key components of R-GMA. The Registry matches queries for information to data sources that provide the appropriate information. The Schema defines the tables that can be queried. Without the combined availability of these components, R-GMA ceases to operate as a useful service. This paper presents an overview of R-GMA and describes the Registry replication design and implementation. A replication algorithm for the Schema has also been designed. A W Cooke A J G Gray W Nutt J Magowan M Oevers P Taylor R Cordenonsi R Byrom L Cornwall A Djaoui L Field S M Fisher S Hicks J Leake R Middleton A Wilson X Zhu N Podhorszki B Coghlan S Kenny D O'Callaghan J Ryan December 2004 Publications/rgmaMediating.pdf Journal of Grid Computing 2(4) 323-339 jogc We have developed and implemented the Relational Grid Monitoring Architecture (R-GMA) as part of the DataGrid project, to provide a flexible information and monitoring service for use by other middleware components and applications. R-GMA presents users with a virtual database and mediates queries posed at this database: users pose queries against a global schema and R-GMA takes responsibility for locating relevant sources and returning an answer. R-GMA s architecture and mechanisms are general and can be used wherever there is a need for publishing and querying information in a distributed environment. We discuss the requirements, design and implementation of R-GMA as deployed on the DataGrid testbed. We also describe some of the ways in which R-GMA is being used. 2005 Andy Cooke Alasdair J G Gray Werner Nutt Publications/streamIntegrationTechniques-JoDS.pdf jods Grids are distributed systems that provide access to computational resources in a transparent fashion. Providing information about the status of the Grid itself is called Grid monitoring. As an approach to this problem, we present the Relational Grid Monitoring Architecture (R-GMA), which tackles Grid monitoring as an information integration problem. A novel feature of R-GMA is its support for integrating stream data via a simple "local as view" approach. We describe the infrastructure that R-GMA provides for publishing and querying monitoring data. In this context, we discuss the semantics of continuous queries, provide characterisations of query plans, and present an algorithm for computing such plans. The concepts and mechanisms offered by R-GMA are general and can be applied in other areas where there is a need for publishing and querying information in a distributed fashion. Journal on Data Semantics 2 136-175 July 2004 Publications/bathSeminar.pdf Seminar talk given at Bath University bathSeminar Grids are distributed systems that provide access to computational resources in a transparent fashion. Collecting and providing information about the status of the Grid itself is called Grid monitoring. We have been involved in the development of the Relational Grid Monitoring Architecture (R-GMA), in which we approach Grid monitoring as an information integration problem where information is published as data streams. We have formalised this integration problem for the simple setting of continuous selection queries. One feature are republishers which are similar to a view in a database: they pose a query and make the answer stream available for other queries. This allows more efficient query plans to be constructed. The query planning problem in this setting is nontrivial for two reasons: (1) In such a distributed environment, it is not clear what the answer to a continuous query should be, and (2) Republishers give rise to redundancy in the available data streams. This gives rise to a choice of sources for answering a query. In this context, we discuss the semantics of continuous queries and give algorithms for computing query plans. We also present methods for adapting the plans in situations where new stream sources come and old ones go during the life time of the query. Our formalisation allows us to show the soundness and correctness of the algorithms. We intend to build upon this work to create and maintain plans for more complex queries. Alasdair J G Gray Andy Cooke Werner Nutt April 2004 dagstuhlWorkshop The publication of distributed data streams can be found in the setting of Grid monitoring. In this setting, components of the Grid must be able to discover information about other Grid resources. In the Relation Grid Monitoring Architecture (R-GMA), we approach this as a "local as view" information integration problem. We have developed a formalisation for the simple setting of continuous selection queries. In this context, we discuss the semantics of continuous queries, provide characterisations of query plans and present an algorithm for computing such plans. The formalisation also allows for straightforward plan maintenance. The main purpose of this work was to get the concepts correct in a simple setting. This allows for the correctness of the planning and maintenance algorithms to be shown. We intend to use this as a foundation for more complex queries. Publications/dagstuhlPresentation.pdf Workshop on Logic Based Information Agents Dagstuhl (Germany) Alasdair J G Gray Werner Nutt April 2004 Publications/prep2004Abstract.pdf PREP2004 (Poster) prep2004 In a computational Grid resources need to make details of their current status available to allow scheduling to take place. With our system monitoring information can be published and queried with the latest values always being available. Hatfield (UK) Publications/prep2004Poster.ppt February 2004 Publications/dept2ndYearSeminar.pdf Talk given at Heriot-Watt University during my 2nd year describing the status of my work Andy Cooke Alasdair J G Gray Lisha Ma Werner Nutt James Magowan Manfred Oevers Paul Taylor Rob Byrom Laurence Field Steve Hicks Jason Leake Manish Soni Antony Wilson Roney Cordenonsi Linda Cornwall Abdeslem Djaoui Steve Fisher Norbert Podhorszki Brian Coghlan Stuart Kenny David O'Callaghan November 2003 Publications/coopis03.pdf Cooperative Information Systems (CoopIS 2003) coopis03 Computational Grids are distributed systems that provide access to computational resources in a transparent fashion. Collecting and providing information about the status of the Grid itself is called Grid monitoring. We describe R-GMA (Relational Grid Monitoring Architecture) as a solution to the Grid monitoring problem. It uses a local as view approach to information integration and will be a component of the European Union's DataGrid. The R-GMA architecture and mechanisms are general and could be used in other areas where there is a need for publishing and querying information in a distributed fashion. 462-481 Catania (Italy)